Creating a Student-Centered Learning Environment Online

Abstract

Shortly after we created a face-to-face, student-centered, general education classroom at the University of Kentucky, we were asked to create a similar environment online. In this article, we describe what we did and what we learned in the process. This is not a comparative, controlled study, but an anthology of examples and reflection. Our aim is to share experiences that other instructors might find useful when constructing their online classes. © 2018, © 2018 The Authors. Published with license by Taylor & Francis.

Full text

Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=ujse20
Journal of Statistics Education
ISSN: (Print) 1069-1898 (Online) Journal homepage: http://www.tandfonline.com/loi/ujse20
Creating a Student-Centered Learning
Environment Online
William Rayens & Amanda Ellis
To cite this article: William Rayens & Amanda Ellis (2018) Creating a Student-Centered
Learning Environment Online, Journal of Statistics Education, 26:2, 92-102, DOI:
10.1080/10691898.2018.1475205
To link to this article: https://doi.org/10.1080/10691898.2018.1475205
© 2018 The Authors. Published with license
by Taylor & Francis
Published online: 21 Aug 2018.
Submit your article to this journal
View Crossmark data

Creating a Student-Centered Learning Environment Online
William Rayens
a
and Amanda Ellis
b
a
Department of Statistics, University of Kentucky, Lexington, KY;
b
Department of Mathematics and Statistics, Eastern Kentucky University, Richmond, KY
ABSTRACT
Shortly after we created a face-to-face, student-centered, general education classroom at the University of
Kentucky, we were asked to create a similar environment online. In this article, we describe what we did
and what we learned in the process. This is not a comparative, controlled study, but an anthology of
examples and reflection. Our aim is to share experiences that other instructors might find useful when
constructing their online classes.
KEYWORDS
Active learning; Peer grading;
Peer instruction
1. The Redesigned Course
A statistical reasoning course has been taught at our university
for over thirty years. In fact, the University of Kentucky was
one of the first in the nation to do so, thanks in large part to
the vision of then-faculty member Dr. Dennis Haack, who
authored what is arguably the first ever textbook for teaching
statistical reasoning. Our introductory level course covers most
of the same topics as any introductory statistics course, but
emphasizes the reasoning necessary for intelligent consumption
of statistical arguments over computations. It has long been the
primary class that students use to satisfy an inferential reason-
ing general education requirement. For approximately 25 years,
each class consisted of six sections of approximately 24 students
per section. Students would meet together with the primary
instructor for two class hours per week and break out into a
recitation with just their section for the third hour of the course
each week. Until 2010 the time spent with the primary instruc-
tor was in a lecture format (typically, a PowerPoint presenta-
tion) and the time spent in recitation with a teaching assistant
was for discovery and discussion. In 2010 we piloted a new stu-
dent-centered format for this course and in 2011 all sections—
encompassing approximately 3000 students per year—transi-
tioned to a similar format.
People mean different things by “student-centered.”Borrow-
ing the words of STEM (science, technology, engineering, and
mathematics) educator Robert Talbert, our goal for a student-
centered classroom is to change the mindset of the student from
a“renter”to an “owner.”In a classroom of renters, fees are paid
and the management is expected to deliver. In a classroom of
owners, students are forced to realize that it is their responsibil-
ity to engage, absorb, and retain. The instructor’s job is to pre-
pare an environment in which that can happen.
One of our primary motivations for adopting the new for-
mat in 2010 was the structure and staffing of the recitation. As
a rank-one research institution we, like many of our colleagues
around the nation used graduate students to staff our recita-
tions. This was trickier in our concepts course, though, than it
was in our traditional first-year methods course. In the former,
recitations were designed as discussion sections, not problem
sessions, and it was up to the teaching assistant to reinforce the
core concepts with activities and discussion. More often than
not, new graduate students come to our program with out-
standing mathematical resumes, but they are sometimes as
challenged by the conceptual material as the undergraduates
are. It would be unfair to say this is the fault of the graduate stu-
dents. They are unlikely to have ever actually discussed, for
example, what a p-value really is and what added intellectual
value it brings to the badging of a statistical result, as opposed
to how to compute it in various contexts. In the end, even
though we worked hard to script and supervise the recitations,
we weren’t doing a stellar job delivering many of the core con-
cepts within that structure.
It was, in a way, irrational that we were leaving the most
important, or at least most difficult part of the class pedagogy
to the teaching assistant. To change this practice, we had to
reimagine what we would do in the classroom and how we
would do it. We ended up having to shift much of the rote parts
of the traditional lecture outside the class time, first to videos,
and then later to a choice of videos or short, focused readings
that the students have to prepare before coming to class. In
turn, the recitations were structured to both capture the talents
of the teaching assistants and to meet the needs of the student.
We started using them for computations (e.g., working with
proportions) that would emerge as critical for understanding
harder ideas (e.g., Simpson’s Paradox, or sensitivity and speci-
ficity), and for practice with other tools (e.g., Excel and CamS-
canner) that we need in class. The professor retained the
responsibility for delivering the deep ideas, either through in-
class activities or some other means, but she could focus on the
ideas, since some attention has already been devoted to
CONTACT William Rayens rayens@uky.edu Department of Statistics, University of Kentucky, Lexington, KY 40506.
© 2018 The Authors. Published with license by Taylor & Francis
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use,
distribution, and reproduction in any medium, provided the original work is properly cited. The moral rights of the named author(s) have been asserted.
JOURNAL OF STATISTICS EDUCATION
2018, VOL. 26, NO. 2, 92–102
https://doi.org/10.1080/10691898.2018.1475205

vocabulary and simple computations that often limit the stu-
dents’abilities to successful grasp those ideas.
Still yet, the primary motivation for our redesign was to
increase student engagement. We wanted to shift the main
responsibility for learning to the student, and away from the pro-
fessor and her PowerPoints. We had become concerned about
how dependent students were on us to serve up content that they
would then passively write down and take away. And we were
sobered by multiple research studies suggesting that students
remember very little of what we teach them beyond the first few
weeks after the semester ends. The use of activities to engage stu-
dents and improve recollection of course material is supported
by Cherney (2008) and it continues to be in the news, evidenced
by recent events at the University of Vermont (Cornish, 2017).
As mentioned above, to make room for our pedagogical
changes, we had to move some of our content out of the class-
room. Initially, a series of video lectures were created using
Camtasia Studio and PowerPoint with audio narration devel-
oped from transcripts. Both the transcripts and ADA-compli-
ant versions of the videos are still made available to students.
The use of content videos is discussed and supported by Choi
and Johnson (2005, p. 215). They found that
There was a significant difference in learners’motivation in terms
of attention between the video-based instruction and traditional
text-based instruction. In addition, the learners reported that the
video-based instruction was more memorable than the traditional
text-based instruction. This study implies that context-based videos
in online courses have the potential to enhance learners’retention
and motivation.”
To better facilitate engagement in our new classroom envi-
ronment, Rayens wrote and published an activities book (Ray-
ens, 2017) for use in the course. The book is structured around
more than 100 daily exercises, a handful of slightly larger proj-
ects, and two extensive capstone projects. Many of the work-
book prompts require the student to access additional reading
materials from the web in order to complete the activities. Typ-
ically, instructors have some of these completed as homework,
and have the remainder completed in class, either in groups or
as part of some orchestrated classroom dynamic. In almost all
cases, time is set aside for students to report out on what they
have done. This happens many different ways, depending on
the instructor and on the physical classroom space. Until
recently, the content was only available in the content videos,
but the book was recently adapted to contain mini-chapters
based on the video transcripts. This addition came from listen-
ing to students, many of whom liked the video instruction, but
some of whom missed a more traditional tactile source for the
content. To encourage students to watch the videos or do the
short readings, instructors routinely ask students to complete a
worksheet or take a short quiz associated with the content.
In short, the fundamental changes we made to our face-to-
face class can be summarized as follows:
The harder deliverables were moved back into the hands
of the professors.
Space was made in the traditional lecture time by off-
loading routine content to videos or short readings.
Students were forced to take responsibility for their own
learning by preparing the content outside of class and partic-
ipating in a student-centered environment inside of class.
This is not a profoundly new way of thinking about statistics
education. These kinds ofpedagogical changes are supported by
Woodard and McGowan (2012) who describe the redesigning
of an introductory course to incorporate the Guidelines for
Assessment and Instruction in Statistics Education (GAISE).
That paper discusses the GAISE recommendations for intro-
ductory college courses. They are:
1. Emphasize statistical literacy and develop statistical
thinking.
2. Use real data.
3. Stress conceptual understanding rather than mere
knowledge of procedures.
4. Foster active learning in the classroom.
5. Use technology for developing conceptual understanding
and analyzing data.
6. Use assessments to improve and evaluate student learning.
Our face-to-face course was developed with these kinds of
recommendations in mind.
2. Creating the Course in an Online Environment
We were not quite through our second semester of piloting the
above-mentioned changes to our face-to-face environment
when our college administration asked us to offer the same
class online. They requested that we include as much of our
new student-centered focus as we could. As far as we could
determine, this had never been done before, and the challenge
to take this course online was initially daunting. We did not
want a course that was essentially a distance learning course
that revolved around static PowerPoints, and allowed students
to be isolated from any kind of actual classroom dynamic. But
it was not clear how we could capture what we valued most
about the face-to-face course and deliver nearly the same
online. We knew something about the learning styles, expecta-
tions and general needs of online students from Mupinga,
Nora, and Yaw (2006, p. 186). In their paper, they write: “…
the top three expectations of the online students were commu-
nication with the professor, instructor feedback, and challeng-
ing online courses.”While communication and instructor
feedback alone do not necessarily equate to a student-centered
course, they do describe an online environment that is not
static. This was part of what we wanted. We added personaliza-
tion and daily involvement to communication and timely feed-
back, and took these as the design principles used to develop
our course. Ultimately, we developed an online structure with
the following characteristics:
A“moving window”which controls the pace for class-
room activities, exercises, prompts and content and
required on-going student engagement.
A current “daily message”from the instructor.
Periodic, required video responses from students for
selected prompts.
Real-time facilitation of discussion, response, and
interaction.
Peer grading, both to facilitate timely feedback and to
keep students in the learning loop on all homework
prompts.
Previous work in online education both supports the choices
we made and the need for them. Tudor (2006) discussed teaching
JOURNAL OF STATISTICS EDUCATION 93

an introductory online statistics course and stresses the impor-
tance of interaction between instructor and students. The peda-
gogy and methods she used are similar to those in our paper,
with notable differences including the pairing of the workbook
and the videos, moving window of availability, daily videos and
the discussion of student assessment. Mills and Raju (2011)pro-
vided a detailed summary of online courses up to 2009, and it is
clear from their work that many of these course stop well short
of the kind of course we attempted to construct. In what follows,
we will briefly elaborate on each of the bulleted course character-
istics mentioned above. We will end with a reflectiononwhatwe
feel worked best and what did not work so well.
2.1. Moving Windows for Assignments
When developing our course we tried to anticipate a student
preconception that "online" is synonymous with correspon-
dence. Many students view a correspondence course as one
that presents them with all of the assignments at the beginning,
and allows them the freedom to turn assignments in anytime
during the duration of the course. It turns out that anticipating
this preconception was a worthy concern. The first time the
course was taught was during a six-week summer session we
had many students with those expectations. Other students
requested to be unavailable for large portions of the course. We
had requests for as many as 3 weeks off from the 6-week term,
with promises that they would make up the work once they
came back. The reasons varied, with family vacations and mili-
tary training being the two most common reasons. However,
we felt that if students were going to be able to have anything
close to the immersion experience we were trying to create in
our face-to-face classroom, then we would need to gently
coerce a sustained exposure to material. We tried to build this
in by creating and using a moving window for assignments.
Here is an example of how many of our instructors have
implemented the moving window. The topic for the day, along
with instructions and requirements for any associated activities,
readings, or homework prompts, appears on the course man-
agement website at 12:01 a.m. and disappears at 6 p.m. the fol-
lowing day. All work associated with that topic, including
homework or outside readings, have to be completed and sub-
mitted during this 42-hr window. This allows students plenty
of flexibility to access and work on material during a time of
day that best fits their schedules, but at the same time requires
them to keep a pace similar to the in-class version of the course.
Although the details vary among instructors, it is common to
have an electronic folder appear on the course management
system that contains all of the material for the next 42-hr time
period, and then have it disappear once the deadline passes.
The choice of a 42-hr window was arbitrary. Any window that
provides adequate time to allow for different work schedules,
while being short enough to mimic the sustained exposure in
our face-to-face class would work similarly. In retrospect this
idea seems very simple, but it was in some ways fundamental to
the tone of the course and the forced immersion experience.
Figure 1 shows an example of what students see when they
access work for our course in our course management system.
The figures in this article show excerpts from when we used
BlackboardTM as our course management system. We now use
CanvasTM, but the basic management strategies remain the
same. The 6-week online course is naturally partitioned into
three subject modules, with each module lasting approximately
2 weeks. When students log into the course management sys-
tem, they have access to “Daily Assignments”(or similarly
named) folder. Within this folder, students then can access
folders for each of the three modules. Once the student opens a
module folder, they are shown assignments that are currently
accessible. Notice that within the Daily Assignments, the avail-
ability of each assignment is displayed. We have found that it is
quite useful to have these dates and times in multiple places
since assignments will expire as the moving window closes.
Managing the availability of the folders can be simplified by
setting folder expiration dates and times beforehand, provided
your course management system allows that option. Below is
the instructor’s view of the available work folder, which shows
that information on the subfolders. By placing expiration dates
to control the visibility and accessibility of a folder, we do not
have to place an expiration date on each item in the folder.
This provides an easy way to control student access to the
assignments and greatly reduces the work for the instructor,
since each folder is likely to contain several assignments.
We have experimented with placing actual timers in the
folders once they are opened. That is, while the expiration date
and time are automatically set in the course management sys-
tem, an explicit timer can also be positioned in the folder itself
and synchronized with the time that the folder will close. This
timer is the first item the student sees when the folder is
opened. These (literal) timers are relatively easy to embed using
html code. Immediately below the timer, students see a brief
description of the work for that day, a description of each
Figure 1. Student view of daily assignments.
94 W. RAYENS AND A. ELLIS

assignment, and the instructions for each assignment. Once the
timer at the top of the page stops, the folder automatically
closes and students can no longer access the assignments.
Figure 2 shows an example of what the students see.
While having assignment folders disappear at fixed times helps
keep the students on pace, the fact remains that they may legiti-
mately want access to the contents at some later time. To accom-
modate this need, some of our instructors just create a separate
link on the course management system that acts as an archive for
any material the student may want to reference later. That archive
would contain all videos, notes, and assignments that the student
would have previously encountered. Of course with most course
management systems, students are able to access all previously
submitted materials from within the grade center.
Some instructors construct informal video guides that
explain this folder structure and the moving window idea. We
have linked three of those below from an early iteration of our
online course. Again, these kinds of videos are often provided
on the first day of course
http://screencast.com/t/hOfWY2lw06v;
http://screencast.com/t/NwYaoRePSFd;
http://screencast.com/t/qjUEXKd2.
The above videos were created using JING, which we will
say more about later. Additional information about JING can
be found at https://www.techsmith.com/jing-features.html.
2.2. Daily Messages from Instructor
We also very much wanted our online class to be personal. We
wanted the students to feel like they knew the instructor and, in
some small way, have a sense of community similar to what we
try to construct in the face-to-face version of the class. So we
built into our design a “daily message”from the instructor. This
message briefly introduces the topic which will be covered that
day, explains how that material connects with what was just fin-
ished the day before, and what kinds of activities and exercises
to expect in the next activity window. For most of us, these are
videos we make and have appeared at midnight, which is the
beginning of the next moving window. Some instructors mix
videos with typed messages, but the point is that they are freshly
created, a way of having the instructor greet the class at the
beginning of each moving window. Some instructors embed the
recorded video on the course management site, while others just
post a YouTube link on that site. No matter, the purpose is to
create a daily communication with the instructor that also serves
to explain what the students were expected to learn and achieve
for the day. Rose (2009,p.1),foundthat
…student perceptions of this instructional strategy (Instructor-
made videos) were overwhelmingly positive, with all of the students
surveyed expressing satisfaction with this method of instructor-
learner interaction. Open-ended answers revealed that students felt
more connected to the instructor when able to view instructor-
made videos.
It is cumbersome to give the reader a feel for the daily message
in a print medium such as this. However, the following link to
one on YouTube that will give you the basic idea of what we
did: https://www.youtube.com/watch?vDlO4OBx3w2zk.Remem-
ber, there may not be anything particularly profound said in
any videos. Rather, their importance is that they are present
each morning when the 42-hr window opens, creating a con-
nection between the student and the instructor.
The videos are not elaborate productions in any sense, but
rather short videos of the instructor casually talking into a camera
on their computer or tablet. The message is a personal, new-and-
not-recycled greeting from the instructor, explaining what is about
to happen and how it relates to what has just happened in the last
window. In the video presented above, the instructor gives a brief
introduction to the work for that day, and emphasizes the impor-
tance of that work in the larger context of the course. This partic-
ular video introduces the vocabulary words parameter,
population, statistic and sample. The instructor gives an example
and also explains that the tasks at hand will focus on these words.
Without the video, students may not understand the important
role those words will have for the rest of the course.
As mentioned above, not all the daily messages were on
video, though that is the medium many of us feel is best for
them. Some were just notes from the instructor. For example:
Daily Message 7/31
Since we are heading into our final week of class I want to thank
everyone for all of your hard work. You have all been great, I have
been more than impressed with everyone. I know taking an online
class has its own set of challenges, but you have met them.
I know this last module has been challenging and is the hardest of
the three. Make sure to take the time to look over all of the answer
keys.
In order to help you prepare for the Exam and Beyond the Class 3, I
am going to start a thread on the discussion board titled "Module 3,
Review material" in this thread leave me anything you are having
problems with and would like me to clarify. This evening I will pre-
pare a video to address any topics you would like help with. When
leaving messages try to be a specific as possible as to what you
would like me to discuss, also if there are any problems that you do
not understand (after looking at the key) I can work through those
and explain the answers. Just like the other Exams you will have a
review sheet the day before the exam.
We are approaching the home stretch so keep up the hard work!
Once again, the point is that the contact is daily and per-
sonal. Our goal is to create a paced, real-time, connection
between the instructor and the students. We want to reinforce
to the students that the instructor is actively engaged in the
Figure 2. Student view of daily work folder.
JOURNAL OF STATISTICS EDUCATION 95

material and partnering with them in the daily progress of the
course. We do feel that it is important that the daily messages
are not made too far in advance. The fresher the better. We
have found that a near real-time production can address issues
the students had with the material the day before, and allows
for the inclusion of pop culture references as they happen. For
example, during the presidential debates in the fall of 2016, the
use of statistical concepts were repeatedly misused, and one of
our instructors surfaced these mistakes in her daily videos,
which gave the online course a dynamic element.
2.3. Video Responses to Prompts
When we first began to envision the online version of this
course, we knew we wanted to be able to judge the depth and
integrity of student responses to prompts. In a face-to-face
environment, we can do that in a variety of ways, some as sim-
ple as giving quizzes or tests under tightly controlled circum-
stances. This has long been a problem for online classes and
continues to be. Lockdown browsers and ethical agreements
meet some of the need. More recently online video and audio
proctoring services have become widely available and aside
from the extra expense for students have real potential. We
took a simpler approach, and try to address this concern by
requiring certain prompts be answered by way of video submis-
sion. That is, students are asked to explain a certain concept or
computation and film themselves doing that. We typically place
restrictions on the length of the response (e.g., no more than 3
min) and control within reason the video type. We also require
that the video be practiced more than once so that the final
product is somewhat refined and organized. Typically, we con-
struct a rubric in advance, and use that to grade the video sub-
mission. Two examples are given below. We are not able to link
to actual student responses because of privacy issues.
Example Prompts
1. This prompt addresses parameters and statistics. Your
job is to create a 3-min video in which you do the follow-
ing. Make sure you post that video on our course man-
agement site by the due date.
Give an example that illustrates a parameter, sample,
statistic, and population.
Explain the difference between a statistic and a param-
eter. Which one do we typically know?
Explain the role that a statistic and a parameter have
in the use of a confidence interval (CI).
2. This prompt addresses correlation and causation. Your
job is to create a 3-min video in which you do the follow-
ing. Make sure you post that video on our course man-
agement site by the due date.
What is correlation? How does it differ from
causation?
What is the correlation coefficient? Use three pictures
to help with your answer. You need to show the pic-
tures in the video.
Give an example when the correlation and causation
are likely the same.
Give an example when the two are likely not the same.
Explain why they are likely not the same.
What started out as a way of imposing a simple quality control
check on the students’work became much more. It has created
a kind of personal buy-in for students that we did not antici-
pate. Indeed, by and large the students really like responding
this way, once concerns over the mechanics of how to make the
videos fade. Students now have a way to deliver a personalized
message to the instructor, just as the instructor is delivering a
personalized daily message to the class. So far, we have gotten
videos from farms in rural Georgia, vacation homes in the Car-
olinas, and dining rooms from several states! We have students
showing us PowerPoints, writing on paper, on dry erase boards,
and just talking. One student even used sidewalk chalk and did
her presentation outside on a stone surface, while her beagle
bayed his disapproval in the background.
Students typically use either a webcam or their phones to
record their videos, and most of them choose to then upload
the videos to YouTube and submit the corresponding url for
their assignment. It should also be noted perhaps that these
videos are no harder to grade than short papers. In some ways
they are notably easier since when it is clear that a student
knows what she is talking about and has spent time with the
prompt, it is easy to slide quickly through the video looking for
key parts identified by the rubric. Rarely, do we have to listen
to the entire 3-min video to judge the quality of the response.
To be honest, they are often a lot of fun to grade.
2.4 Discussion Boards and Online Instructor Access
Students in the in-class version of our course have the opportu-
nity to work in small groups as well as having real-time access
to the instructor. We want students in the online class to have
similar opportunities that would still offer flexibility of schedule
and location. We attempt to address this with the use of discus-
sion boards and virtual office hours.
Discussion boards are far from novel, but we ended up with
a few interesting and perhaps unique twists in our implementa-
tion. Our primary goal is to have a way for students to ask ques-
tions so that other students can see both the question and the
answer, and to avoid duplication of questions. Many of our
instructors insist that all questions about the course material be
posted in the discussion board. Questions about grades or other
personal matters need to be emailed directly to the instructor.
The following is specific to one instructor, but typical of what
others do. When a student initially enters the discussion board,
they have five forums available to them as follows.
Introductions—This forum is for introducing students to
each other in the class. Students were asked to write a
couple sentences about themselves, including hometown,
hobbies, year in school, favorite movie, etc.
Questions on material from Module 1—This forum is for
asking, answering, and archiving questions about Module 1.
Questions on material from Module 2—This forum is for
asking, answering, and archiving questions about Module 2.
Questions on material from Module 3—This forum is for
asking, answering, and archiving questions about Module 3.
Issues with Quizzes—This forum is for asking questions
about quiz.
Miscellaneous—This forum is for anything that does not
fit in the five above.
96 W. RAYENS AND A. ELLIS

The introductions forum is used the first day of the course,
on which students are required to post an introductory message
to the discussion board. As noted above, students are asked to
include some personal, though not invasive information. We
feel this accomplishes two things. First, when students intro-
duce themselves to each other and to the instructor, this adds
to the personalized feel of the course. Second, requiring this ini-
tial post forces a first-day familiarization with how to access the
discussion board and create a post.
The forums for questions over material from the three mod-
ules are where the majority of content questions are posted.
Once a student enters one of the modules, she see a post created
for each day of the course. See Figure 3 for a partial view of
what this page looks like. Threads are fixed and, to minimize
confusions, students are unable to create new threads. Students
click on the link corresponding to the day’s material that they
have questions about.
Once selected, they are able to ask a question or respond to
other students’questions. Students can access any part of the
discussion board during the course, even if a module is already
over. The instructor monitors these forums and follows up as
necessary when peer answers are either in short supply or
incomplete. Many of our instructors will respond to an emerg-
ing common misconception by posting a short, customized
tutorial video.
The “Issues with Quizzes”forum is used to address any
issues with the quizzes. Students are asked to leave the date of
the quiz with a brief description of what they were confused
about, what might have been misstated, or was otherwise
unclear. They are forbidden to post any answers to the quizzes.
This is useful to us as instructors because if there is a mistake
on a quiz, students can see that it was already brought to the
instructor’s attention via this forum. This saves the instructor
from having to answer multiple emails about the same issue.
Students are directed to look at the forum first if these kinds of
questions arise.
As noted, using discussion boards is hardly new. One well-
known challenge to using them is the potential difficulty for the
instructor to keep up. There is no perfect answer to this. The
more interactive an online class is, the more work, potentially,
it will be for the instructor. In our case, we do the following
and have found that the workload is quite manageable.
Limit need for trolling—Once a student posts to any of
the threads on any of the forums, the instructor receives
an email alerting them that there was a new post. This
way the instructor doesn’t have to constantly troll the
course management site. Some of us route these emails
to special folders so that we can check them when we
have the time. We have tried requiring postings and
even having the course management system automati-
cally grade participation. The upside is that all students
have to get involved. The downside is that the instructor
gets a lot of email alerts and many of the postings are
not urgent. After all, if the course management system is
grading postings automatically it is only able to give
credit for a student accessing and posting something. It
has no way of assessing whether the post is in any way
substantive. So our inclination has been to have the dis-
cussion boards be voluntary, a way of creating a useful
though imperfect version of interaction students may be
used to in a face-to-face class.
Have preset times for responding—While students are told
not to expect immediate responses to postings, we often
set particular times of the day when students know post-
ings will be read and responded to. The pace varies
according to instructor. We have some who like to
respond within an hour and others who agree to respond
only once or twice a day. Keep in mind that even in the
face-to-face version of the course students unfortunately
do not ask that many questions. Our online experience
has not been so different. Rather, our goal was to make
this online experience different than the stereotypical,
wherein the instructor may not be available at all, or
rarely.
Let peers take some of the load—Questions are not always
or even most often about some subtle content issue. The
Figure 3. Organization of module threads.
JOURNAL OF STATISTICS EDUCATION 97

same is true in our face to face course. Many questions
concern generic issues such as where to find a certain defi-
nition in the readings or videos, what the second sentence
in the third prompt means, or how to use Excel TM to cre-
ate a scatterplot. Peers are very good at answering these
types of questions. Again, at the risk of over-making this
point, instructor personalities vary a lot here, even in our
small group of online instructors at Kentucky. Some are
more comfortable than others allowing peers to handle
questions without too much oversight, others not so much.
Keep responsibility on the questioner when appropriate—
For many questions, the answers are simple: look in the
syllabus, review the previous reading, Google on “scatter-
plots in ExcelTM,”etc. This not only keeps the workload
more reasonable for the instructor, it is in our view the
right thing to do for the student. As much as is possible
we try to keep the pressure where it belongs.
As noted above, having students post to the discussion
board reduces the need to answer the same question over and
over. To help students become comfortable with the discussion
board, we include information about the discussion board in
both the syllabus and information given on the first day of the
course. If students email us with questions that are not some-
how private, we often just ask them to post the question on the
discussion board and look for an answer there. If we happen to
see a student trying to address a harder content item (e.g., “why
is a p-value not the same thing as an alpha level”), we intervene,
either with an encouraging message affirming that the response
was correct or offering advice if not. In any case, it is best just
to make sure the students realize the discussion board is about
as reliable as peer discussions in class. They need to realize that
while the interactions are useful, peer responses do not come
with guarantees and the ultimate responsibility for knowing a
response is correct rests with the student.
An interesting note is that students by and large do not fish
for answers to the homework. In our experiences so far stu-
dents use the discussion board as intended and ask genuine
questions, though most are about execution and procedure and
not about subtle content. Just below, we have included an
example interaction on one of the discussion boards. We have
repeated the student syntax exactly. The “3.14”refers to a par-
ticular exercise in the required class workbook.
Student 1
3.14
On Problem 2 are we supposed to try and solve it for the fpr? How
do we know the phat and Po are?
Student 2
RE: 3.14
I think you’re just supposed to say what the Ho and Ha are and
then write what the FPR would be since the results are not statisti-
cally significant like we did the other day.
Instructor
RE: 3.14
[Student 2] is correct. Good Work.
This is a pretty typical example of the discussion board
interaction, and a good example of a peer helping out.
Wealsouseonlineoffice hours so students can have face-to-face
contact with the instructor. Different instructors use different
methods, and we do not micromanage what instructors use as long
as they provide some version of this personalized contact. That
having been said, most instructors here use Adobe Connect or
Skype. Adobe Connect is a web-conferencing software that allows
an instructor to set up meeting times with individuals or groups of
students. With Adobe Connect, instructors are able to present stu-
dents with materials, as well as have typed, audio, or video discus-
sions. Instructors have extensive control over the interaction, with
an ability, for example, to have a private conversation with a stu-
dent if the student wants, or the ability to silence a student who
might be attending the session just to complain. Admittedly, we
have never had to do the latter! Our university has a license to use
this software so any student who signs up for our courses can access
it for free. As soon as the instructor initiates an Adobe Connect ses-
sion, students are notified and can join if they want. Skype also
offers an easy, familiar tool for students and instructors to connect
face-to-face. Indeed, students tend to be more familiar with Skype
and some of us find it easier to use than Adobe Connect despite the
fact that Skype has less features and the instructor has less control
over the communication and interaction.
Initially, we all offered preset office hours where we would
hold an open session in Adobe Connect and students could
join during that time. Students would typically not come at
these fixed times and instead often requested to meet at another
time. Of course this is not surprising since similar things occur
with office hours in all of our face-to-face classes! The upshot is
that some of our instructors just stopped holding fixed hours
and hold virtual sessions on demand instead. The pattern we
see is that most routine questions get answered by the discus-
sion board. Students needing help beyond the discussion
boards typically just contact us for a one-on-one virtual session.
This may seem daunting at first, since, in theory, one could
end up with an impossible number of requests. We have not
had that happen. In fact, the demand is very similar to what we
see with our regular classes. Rather we only see one or two vir-
tual sessions here and there requested. Let us not lose sight of
the bigger point, which is that we wanted to make sure the con-
tact was offered and available in an online setting, much the
way it is in a face-to-face class. If for some reason dozens of stu-
dents started requesting individual sessions we would have to
change our approach and go back to the fixed hours. Consistent
with encouraging more student responsibility, that may be the
best thing to do anyway. Again, our own instructors vary on
how they choose to handle the online contact.
It should also be noted that other technological methods
may be better (e.g., Google Hangouts), but we have not yet tried
them. We encourage you to go with what you are most com-
fortable with, provided you allow some kind of personal contact
this way. We have had lots of good feedback from students
about how this helped connect them to the instructor and the
course. One of authors was even stopped on campus the year
after teaching this course online and greeted by an online stu-
dent who recognized him from the previous summer. That is
quite unusual for an online class. We are sufficiently enamored
with the idea here at the University of Kentucky that we have
98 W. RAYENS AND A. ELLIS

even built it into a new online applied master degree we
recently started. These sessions are billed as “Meet the Expert”
sessions and they are proving to be very popular with the grad-
uate student population as well.
2.5. Peer Grading
The workbook required for this course emphasizes short
answer student responses. While these short answer responses
are beneficial to student learning, they also tend to create a
challenging grading burden for instructors. Many of our
instructors for this course use peer grading, which can be espe-
cially easy to adapt to an online class. We have used two auto-
mated methods of peer grading. One is called “Calibrated Peer
ReviewTM,”often just referred to as CPR, which is a utility out
of UCLA. For each homework assignment, students are elec-
tronically calibrated, assigned peer documents to grade, and
then required to grade their own assignments as well. Each step
of this process is scored automatically as a part of the overall
assignment grade. Of course, we had to build all the calibration
materials and rubrics for our course into the UCLA sponsor
site. There is a learning curve for instructors and students to
familiarize themselves with CPR, and some students struggle
initially with the complexity, but the payback on that time
investment can be excellent. The peer review tool on Canvas is
also an excellent choice. While this tool has far fewer bells and
whistles than CPR, it is much easier for students and instruc-
tors to use. That can be a good trade-of and either can be used
as seamlessly online as in a face-to-face setting. We currently
use both here at the University of Kentucky.
Why do we like peer grading? Even if we could manage to
grade huge numbers of submissions ourselves, it became clear
to us long back that students rarely look at the results of that
effort. In that sense we felt like the learning loop was not being
closed. We want students to know what was valued about the
assignment and to see suggestions for what reasonable answers
might have been. By using rubric-driven peer grading, we are
able to gently force an encounter between the student and a
rubric and, hence, more effectively communicate what was val-
ued about the assignment. This is part of the assignment in
some of our classes and full credit for the assignment is not
possible if the assigned peer review is not completed. Indeed,
we consider the closing of that learning loop as a critical part of
the homework assignment.
We would encourage you to not think of peer grading as
only a means of coping with large numbers of students, though
that is a good outcome in itself. Gunersel et al. (2008) suggested
that using CPR actually helped students develop writing and
reviewing skills in biology. Pittard (2014) discusses CPR in the
context of the course in question, and more information can be
found on the CPR home page: http://cpr.molsci.ucla.edu/Home.
aspx.
3. Student Assessment
Hodgson and Pang (2012) make the case for providing students
multiple venues of assessment, and we agree. For our course,
students are assessed multiple ways. They have daily graded
assignments and tests over larger quantities of material. They
are also peer-assessed on homework, and sometimes self-
assessed, depending on the instructor and the peer-assessment
tool used. Most of us divide their course grades into three sour-
ces: Daily Work [the Beyond the Numbers (BN) assignments],
Beyond the Classroom work (BCs), and Exams. The weight for
each category, and what is actually included in each category,
depends on the individual instructor. Daily Work usually
includes the daily activities students perform, including quizzes
and the BN worksheets, some of which may be peer graded. BC
assignments are mini-projects and these are discussed in more
detail below. Typical BN worksheets and BC Worksheets can
be found at http://www.statconcepts.com/look-inside.Exams are
given two–three times during the term, and usually consist of a
multiple choice portion and an open response portion. The
open response portion typically requires students to create
either a text or video response.
We purposely structured our course so that students complete
work daily, and receive some type of feedback every day. Our goal
is not to make the course more work than a face-to-face class, but
rather to make it more like a face-to-face class than a correspon-
dence course. We were also mindful of the importance of keeping
the scope within the practical limits of what we as instructors were
able to do. Most of us simply use online quizzes for the daily feed-
back. These are easily constructed in advance and deployed on our
course management system. While it depends on instructor, most
just have these quizzes be multiple choice quizzes that are graded
electronically. These quizzes can take a variety of forms. Sometimes
we adapt questions from a homework assignment into a quiz for-
mat, and sometimes we use these auto-graded quizzes as checks on
whether videos were watched or readings were completed. In all
cases, we believe this helps keep the student involved in the course
on a daily, or nearly daily pace, and keeps the instructor engaged
with the students on a level that is manageable with respect to
workload.
At our university we typically have a two-person team teach
this course online. For the online course, the instructor consists
of a primary instructor and a grader. We should emphasize
that the way we have set up the course, graders are very useful
but not necessary. Having a grading assistant simply adds
another pair of eyes to the discussion board, provides another
person to answer emails, and perhaps another person to help
with the organization of the learning management site. What
makes the online course different is not really dependent on us
having a grader. If we have to work without a grader, or with
an inexperienced TA, then we adapt just the way we all do in a
face-to-face environment. As mentioned, electronically graded
quizzes are very helpful in this regard, as is peer grading, and
grading with rubrics in general. Some of our instructors will
grade for effort only, and some may select only certain items to
grade. These are the same compromises we might have to con-
sider in a face-to-face class, and may be necessary in an online
class if the enrollments are not capped to be similar to the face-
to-face environment. While we do not endorse any particular
compromise approach, we do want to point out that we feel
strongly about the use of peer grading for increasing the stu-
dent-centered experience, and we hope the student learning,
while substantially reducing the grading load for the instructor.
With the use of peer grading, auto-graded course man-
agement assessments, and video submissions, we are able to
JOURNAL OF STATISTICS EDUCATION 99

both provide timely assessment and increase the students’
involvement in their own course. These assessment strate-
gies also keep the grading load at an acceptable level for us.
This is very consistent with our mantra that one of the pri-
mary duties for an instructor in a student-centered environ-
ment is to prepare that environment in such a way that the
student is gently forced to take responsibility for her own
learning. While this creates more work up front for the
instructor, that work tends be more manageable (and less
time sensitive) and tends to be more focused on creating
assessment plans for the entire class, as opposed to scoring
individual papers for class members.
4. Reflection
We are generally very pleased with the course we have created.
It has been used as a model for other online courses in our col-
lege, and—as mentioned above—for our new online master in
applied statistics. On the one hand, much of what we have
described above points simply to design challenges that we
would all face when creating any online class in virtually any
subject. The larger point is how we have tried to address those
challenges while at the same time maintaining some of the
innovative qualities of a good face-to-face environment. In
addition, we have described strategies we are using in an
attempt to capture online some of what we feel is special about
our student-centered offline environment.
We have taught this class online now for approximately six
summer terms, five winter terms, and two fall terms. Therefore,
we have had ample time to reflect and calibrate based on what
we hear from the students and experience as instructors. An
anecdotal reflection on what we think is working and not at
this point would include the following.
Students in our face-to-face version of this class are not
always happy about the extra work that comes with a stu-
dent-centered environment. Our online students largely
feel the same way about the workload. In truth, this is not
so much about student workload—that is very manage-
able—as it is about consistent, ongoing student engage-
ment. Our moving window design forces that
engagement much like the natural structure of a three-
day-a-week face-to-face class would.
Our instructors are generally very positive about the effect
of the daily messages and the moving window. They feel
that this gives them a personal connection with many of
the students and forces the students to stay involved in
the class on a regular basis. One instructor noted that
“the moving windows for assignments forces the students
to treat the class as they would a face to face summer
class. So instead of trying to take two or three massive
bites of information they are forced to process smaller
bites at a time.”
There is still concern about cheating. But this is not spe-
cific to our class, rather is a concern about all online clas-
ses. One instructor lamented that if a student were
determined to cheat then all he would need to do is pro-
vide his access ID to another student who could take the
entire class for him. Another instructor uses an online
proctoring system because of similar worries.
The discussion boards are generally seen as better to have
than not have, though many of us feel that once you are
in the online environment it is simply very difficult to cre-
ate the kind of personal connection that you can have
with as student face to face.
Studentswhoareunhappywiththecoursearemostly
unhappy with the moving window, which instructors are
the most positive about. Some students are not willing to
accept the dynamic nature of the course and are upset that
if they stop participating for two weeks that they are then
not allowed to go back and make up material they missed.
At some point, experienced instructors simply have to gravi-
tate toward what they feel is best. Felder and Brent (2017),
responding to similar student pushback in any student-cen-
tered environment, implore the instructors to stay the course:
This is no time to give up. To counter students’resistance, try to
persuade them that you are neither playing a game nor performing
an experiment, but teaching in a way known to help students learn
more and understand better.
This is just the approach we are taking. We try to explain to
the students why we have the course structured the way it is, why
this is a better product for the money they are spending, and how
it is in their best interest to have a student-centered experience
even online. To be fair, however, most of our students seem to
appreciate how the course is structured and are cognizant of the
attempt to make it the best course for them that we can.
We do not have any possibility to compare the way we offer
this class online to some other way of offering it. Indeed, when
the course was brought online in 2011, it was the first time it
was ever taught online in any fashion. It is interesting, however,
to compare the grade distributions and the DEW and pass rates
for this class and its in-class counterpart. The DEW rate is a
measure of success used by the University of Kentucky and it is
the percentage of students who get either a D, an E (failing
grade), or withdraw from the course. Figures 4–6are for the
same instructor, over several semesters in-class and several
teaching the online class.
Figures 5 and 6offer similar comparisons for DEW and pass
rates. The error bars exhibited are simple 95% error bars. Spe-
cifically, the CIs are as follows.
CIs for DEW
1. Online 0.1453–0.2316
2. In-class 0.0723–0.1365
Figure 4. Comparison of grades in in-class vs. online.
100 W. RAYENS AND A. ELLIS

CIs for PASS
1. Online 0.8631–0.9302
2. In-class 0.9113–0.9623
Likewise, a two-tailed ztest comparing DEW rates produces
azof ¡3.1445 and a p-value of 0.00168. This is consistent with
the CI. A two-tailed ztest comparing pass rates produces a zof
1.9213 and a p-value of 0.05486, which in this case is also con-
sistent with the CI perspective.
Clearly, the online class is not performing quite a well as the
face-to-face version of the same class. However, research shows
that the differences between two such offerings are usually
much greater than we are seeing here. There are several sources
suggesting that a higher dropout rate for online courses is
expected (Jaggars and Bailey 2010; Lee and Choi 2011), with
some suggesting that the online dropout rate can be two–three
times higher than in-class (Blackner 2000; Zavarella 2008).
This suggests that, although our withdraw rate is higher in the
online class, we are still doing better than previous studies
would have predicted.
We are continuing to think about how to make the
online version of this course better. At the present we are
trying to determine the best, most cost-effective proctoring
software to build into the course offering. Students already
pay a course fee that is designed to support infrastructure,
but since these online students are no physically sitting in
our classrooms using our computer equipment we want to
try and divert those fees to partially underwrite state-of-the
art proctoring support. We are currently looking at several
different brands, some of which monitor eye movement
using the student’s web camera, some record the student’s
full movements, others have an actual person doing the
monitoring. We have experienced these being used success-
fully at other places on our campus (e.g., engineering) and
at other universities (e.g., Miami of Ohio).
Finally, we are also working with a vendor to move the book
to a new and innovative online format. In this new format, stu-
dents would have to purchase a key code to have access to the
book, but all answers to activities could then be created online
and exported to system-personalized pdf files for peer grading
or general submission to Canvas. This would help us as instruc-
tors know that all students have the book, something which is
impossible to know with online students. And having system-
authenticated pdf files created by the online book management
software could help with integrity issues.
5. Contact Information
The authors are excited about what they have learned from this
online experiment and would be very happy to engage in fur-
ther discussion with any reader interested in knowing more
about our experiences. Our contact information is given on the
opening page of the article.
References
Blackner, D. M. (2000), “Prediction of Community College Students’Suc-
cess in Developmental Math With Traditional Classroom, Computer-
Based On-Campus and Computer-Based at a Distance Instruction
Using Locus of Control, Math Anxiety and Learning Style,”unpub-
lished Ph.D. dissertation, Department of Applied Technology, Training
and Development, University of North Texas.
Cherney, I. D. (2008), “The Effects of Active Learning on Students’Memo-
ries for Course Content,”Active Learning in Higher Education, 9, 152–
171.
Choi, H. J., and Johnson, S. D. (2005), “The Effect of Context-Based Video
Instruction on Learning and Motivation in Online Courses,”The
American Journal of Distance Education, 19, 215–227.
Cornish, A. (2017), “Vermont Medical School Says Goodbye to Lectures,”
National Public Radio [online], available at http://www.npr.org/sec
tions/health-shots/2017/08/03/541411275/vermont-medical-school-says-
goodbye-to-lectures
Felder, R., and Brent, R. (2017), “Navigating the Bumpy Road to Student-
Centered Instruction,”College Teaching, 44, 43–47.
Gunersel, A. B., Simpson, N. J., Aufderheide, K. J., and Wang, L. (2008),
“Effectiveness of Calibrated Peer Review [TM] for Improving Writing
and Critical Thinking Skills in Biology Undergraduate Students,”Jour-
nal of the Scholarship of Teaching and Learning,8,25–37.
Hodgson, P., and Pang, M. Y. (2012), “Effective Formative E-Assessment
of Student Learning: A Study on a Statistics Course,”Assessment and
Evaluation in Higher Education, 37, 215–225.
Jaggars, S., and Bailey, T. R. (2010), “Effectiveness of Fully Online Courses
for College Students: Response to a Department of Education Meta-
Analysis,”Community College Research Center,1–18 [online]. Avail-
able at https://eric.ed.gov/?idDED512274
Lee, Y., and Choi, J. (2011), “A Review of Online Course Dropout
Research: Implications for Practice and Future Research,”Educational
Technology Research and Development, 59, 593–618.
Mills, J. D., and Raju, D. (2011), “Teaching Statistics Online: A Decade’s
Review of the Literature About What Works,”Journal of Statistics Edu-
cation, 19, 1–28.
Mupinga, D. M., Nora, R. T., and Yaw, D. C. (2006), “The Learning Styles,
Expectations, and Needs of Online Students,”College Teaching, 54,
185–189.
Pittard, M, and Rayens, W. (2014). “Using Calibrated Peer ReviewTM in
Introductory Statistics Courses.”in Proceedings of the International
Conference on Teaching Statistics, pp. 1–4.
Figure 5. Comparison of DEW rates in-class vs. online.
Figure 6. Comparison of pass rates in-class vs. online.
JOURNAL OF STATISTICS EDUCATION 101

Rayens,W.(2017), Beyond the Numbers, Student-Centered Activities for
Learning Statistical Reasoning (Vol. 7). Cincinnati, OH: Van-Griner, LLC.
Rose, K. K. (2009), “Student Perceptions of the Use of Instructor-Made
Videos in Online and Face-To-Face Classes,”MERLOT Journal of
Online Learning and Teaching, 5, 487–495.
Tudor, G. (2006), “Teaching Introductory Statistics Online–Satisfying the
Students,”Journal of Statistics Education, 14, 1–18.
Woodard, R., and McGowan, H. (2012), “Redesigning a Large Introduc-
tory Course to Incorporate the GAISE Guidelines,”Journal of Statistics
Education, 20, 1–24.
Zavarella,C.A.(2008), “Computer-Based Instruction and Remedial
Mathematics: A Study of Student Retention at a Florida Commu-
nity College,”Ph.D. Dissertation, University of South Florida.
pp.1–159.
102 W. RAYENS AND A. ELLIS