Conference PaperPDF Available

MOOCs Completion Rates and Possible Methods to Improve Retention - A Literature Review

Authors:
  • university of technology and applied science

Abstract

Many MOOCs initiatives continue to report high attrition rates among distance education students. This study investigates why students dropped out or failed their MOOCs. It also provides strategies that can be implemented to increase the retention rate as well as increasing overall student satisfaction. Through studying literature, accurate data analysis and personal observations, the most significant factors that cause high attrition rate of MOOCs are identified. The reasons found are lack of time, lack of learners’ motivation, feelings of isolation and the lack of interactivity in MOOCs, insufficient background and skills, and finally hidden costs. As a result, some strategies are identified to increase the online retention rate, and will allow more online students to graduate.
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
MOOCs Completion Rates and Possible Methods to Improve Retention
- A Literature Review
Hanan Khalil
Instructional Technology department, Faculty of education
Mansoura University
Mansoura, Egypt
Hanan81@mans.edu.eg
Martin Ebner
Information Technology Services / Department of Social Learning
Graz University of Technology
Graz, Austria
martin.ebner@tugraz.at
Abstract: Many MOOCs initiatives continue to report high attrition rates among distance
education students. This study investigates why students dropped out or failed their
MOOCs. It also provides strategies that can be implemented to increase the retention rate
as well as increasing overall student satisfaction. Through studying literature, accurate
data analysis and personal observations, the most significant factors that cause high
attrition rate of MOOCs are identified. The reasons found are lack of time, lack of
learners’ motivation, feelings of isolation and the lack of interactivity in MOOCs,
insufficient background and skills, and finally hidden costs. As a result, some strategies
are identified to increase the online retention rate, and will allow more online students to
graduate.
Introduction
Over the past year, Massive Open Online Courses (MOOCs) have received a great deal of attention
from the academic community as well as the press (Gaebel, 2013). MOOC is a model for delivering
learning content online to virtually any person with no limit on attendance who wants to take the course.
Participants can be students enrolled at the institution hosting the MOOC or anyone worldwide with
Internet access. A MOOC throws open the doors of a course and invites anyone to enter, resulting in a new
learning dynamic, one that offers remarkable collaborative and conversational opportunities for students to
gather and discuss the course content. The “open” students, who pay nothing to participate, can join in
some or all of the course activities, which might include watching videos, posting on discussion boards and
blogs, and commenting via social media platforms (Thompson, 2011). In a MOOC, learners construct their
own knowledge and develop their personal learning network from nodes and connections in the digital
environment. Assisted by computing technologies such as RSS and an aggregator that regularly
summarizes the learning events of notes that have occurred on the massive network of materials, links,
blogs, and discussion forums, each learner remixes the content and interactions in ways that are personally
meaningful to him or her, repurposes it to suit his/her needs and objectives, and feeds it forward and shares
the remixed, repurposed content with others (Siemens, Downes, Cormier, & Kop, 2010 ).
Massive Open Online Courses (MOOCs) can rapidly change the role of higher education, executive
education and employee development in general. They are attempts to create free or even open access
online courses that provide no constraints on class size (Sharples et al, 2012). MOOCs have the potential to
enable free (university-level) education on an enormous scale. However, a major concern often raised about
MOOCs is that although thousands enroll for courses, a very small proportion finally completes such
courses. The release of information about enrollment and completion rates from MOOCs points out a very
low completion rates (Balsh, 2013). Generally speaking, for most of the courses, completion rate is defined
as people who earned a certificate or passed the course (Jordan, 2013). Of course, one can’t compare
MOOC completion rates with those of traditional online or on-campus courses. Since MOOC students
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
neither pay tuition nor earn university credits, the motivation for completing a course is largely intrinsic.
The statements of accomplishment earned might have value for some students, but they are not equivalent
to course credit.
According to a visualization of MOOC completion rates assembled by Katy Jordan (2013), the 50
investigated MOOCs have generated 50.000 enrollments on average, with the typical completion rate
hovering below 10%. Put it somewhere around 7.5%, or 3.700 completions per 50.000 enrollments. Meyer
(2012) reported that the dropout rates of MOOCs offered by Stanford, MIT and UC Berkley were 80-95%.
For example, only 7% of the 50.000 students completed who took the Coursera-UC-Berkeley course in
Software Engineering. There is a similar reported dropout rate in Coursera’s Social Network Analysis class
where only 2% of participants earned a basic certificate and 0.17% earned the higher level programming
with distinction certificate. For this low completion rate, it is widely useful to improve the retention rates of
MOOCs by finding out why students drop out of courses. Therefore our research work aims to investigate
the reasons that may cause students drop-out or withdraw of their MOOCs, and suggest strategies that may
improve retention. As such, this study asks the following research questions:
1. Why do students drop out of MOOCs?
2. What are the techniques that increase retention rate for MOOCs?
Retention Rates for MOOCs
In the age of lifelong learning, MOOCs are a means of providing learning and development to
virtually everyone, anytime, anywhere in the world with internet access (Ryan, 2013). The promise of
MOOCs is that they will provide free access, cutting edge courses that could drive down the cost of
university-level education and potentially disrupt the existing models of higher education. Furthermore
anyone is allowed to choose courses from interest on his/her favorite university. However, Markoff (2013)
has shown that although thousands enroll for MOOCs, only a very small number actually complete the
MOOCs .
Rivard (2013) pointed out that hundreds of thousands of people across the world are signing up for
MOOCs in the first glance. Courses offered by MOOC providers are in general free and don’t earn students
any college credit. There are also no enforced prerequisites as there are for sometimes for normal college or
university courses. In spite of this only few students complete the course and get a certificate. Of 104.000
students who enrolled in the 2011 online machine-learning class which was an earlier version of the later
Coursera course, 46.000 submitted at least one assignment, 20.000 completed a substantial portion of the
course and 13.000, or 12.5%, passed (Rosanna Tamburri ,2012). In a study of Bioelectricity MOOC by
Duke University (Belanger & Hornton, 2013) approximately 12.000 students enrolled and about 3200 of
these students attempted a quiz within the first week. Only 700 students of those 3200 earned perfect scores
finally, a dropout rateof 94 percent. Another study given by Bruff (2013) mentioned a MOOC, launched
on March 4, 2013, on Pattern-Oriented Software Architectures for Concurrent and Networked Software
(POSA) by Doug Schmidt. The course ran for ten weeks with about 31.000 enrolled students. These are
students who did something beyond enrollment watch a video, take a quiz, visit the discussion forum.
There were 23.313 active students, 20.933 of them (90%) watched at least one lecture video, 5.702 (24%)
took at least one quiz, 2.072 (9%) submitted at least one assignment for peer grading, and 942 (4%) posted
at least once in the discussion forums. Of the 23.313 active students, 1.051 (4.5%) earned a standard
statement of accomplishment and 592 (2.5%) earned a statement of accomplishment “with distinction”, so a
total of 1.643 (7%) students earned some form of statement. In the same way Jordan (2013) a PhD student
at the Open University of the UK described their effort to synthesize MOOC completion rate data from
xMOOCs in particular and mostly from Coursera. The average completion rate of her examined xMOOCs
is 7.6%, with a minimum of 0.67% and a maximum of 19.2%. The 19.2% appears to be an outlier
Functional Programming Principles in Scala from Switzerland's École Polytechnique Fédérale de
Lausanne, offered on the MOOC platform Coursera. While the lowest rate of completion was A History of
the World since 1300 from Princeton University, offered also on the MOOC platform Coursera.
Research Methodology
This study aims to investigate the reasons that may cause students drop-out or withdraw of their
MOOCs, and suggest strategies that may improve retention. The framework of research used descriptive
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
research methodologies to provide an accurate description of reasons that may cause drop out. The data
source of this research was from 42 MOOCs analyses of the course completion rates, content, documents,
and class discussions. These 42 offered through most popular platforms (Coursera, Edx, Udacity, MITx,
and Moodle). The completion rate data of these MOOCs according to Jordan's (2013) research is pointed
out in Table 1.
Table 1: Completion rate date according to Jordan's (2013) research
MOOC
Students
enrolled
Students
completed
Percentage
of
Completion
Platform
Equine Nutrition
23322
8416
36.1%
Coursera
Astrobiology
39556
7707
19.5%
Coursera
Functional Programming Principles in Scala
50000
9593
19.2%
Coursera
Computing for Data Analysis
50899
6420
12.6%
Coursera
introduction to Machine Learning
104000
13000
12.5%
Coursera
Databases
60000
6500
10.8%
Coursera
Image and video processing - From Mars to
Hollywood with a stop at the hospital
40000
4069
10.2%
Coursera
Internet History, Technology and Security
46000
4595
10.1%
Coursera
Gamification
81600
8280
10.1%
Coursera
Introduction to Philosophy
98128
9445
9.6%
Coursera
Critical Thinking
75884
6909
9.1%
Coursera
Mathematical Biostatistics Bootcamp
21916
2087
9.5%
Coursera
Sports and Society
19281
1626
8.4%
Coursera
Introduction to Mathematical Thinking
27930
1950
7%
Coursera
Software Engineering for SaaS
50000
3500
7%
Coursera
Introduction to International Criminal Law
21000
1432
6.8%
Coursera
Drugs and the Brain
66800
4400
6.6%
Coursera
Listening to World Music
36295
2191
6%
Coursera
Data Analysis
102000
5500
5.4%
Coursera
Pattern-Oriented Software Architectures for
Concurrent and Networked Software
30979
1643
5.3%
Coursera
Introduction to Genetics and Evolution
33000
1705
5.2%
Coursera
Computational Investing, Part 1
53205
2554
4.8%
Coursera
An Introduction to Operations Management
87000
4000
4.6%
Coursera
Greek and Roman Mythology
55000
2500
4.5%
Coursera
E-learning and Digital Cultures
42844
1719
4%
Coursera
Introduction to Astronomy
60000
2141
3.6%
Coursera
Introduction to Sociology
40000
1283
3.2%
Coursera
Social Network Analysis
61285
1410
2.3%
Coursera
Human-Computer Interaction (studio track)
29105
791
2.7%
Coursera
Bioelectricity - a quantitative approach
12000
313
2.6%
Coursera
Human-Computer Interaction (studio track)
29105
791
2.7%
Coursera
A Beginner's Guide to Irrational Behaviour
142839
3892
2.7%
Coursera
Think Again: How to Reason and Argue
226652
5322
2.3%
Coursera
Social Network Analysis
61285
1410
2.3%
Coursera
Medical Neuroscience
44980
756
1.7%
Coursera
A History of the World since 1300
83000
605
0.7%
Coursera
Stat2.1x Introduction to Statistics -
Descriptive Statistics
52661
8181
15.5%
EdX
3.091x Introduction to solid state chemistry
28512
2082
7.3%
EdX
6.002x Circuits and Electronics
46000
3008
6.5%
EdX
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
CS50x - Introduction to Computer Science I
150349
1388
0.9%
EdX
Introduction to Inforgraphics and Data
Visualization
2000
140
7%
Moodle
Introduction to Artificial Intelligence
160000
20000
12.5%
Udacity
Moreover, the researchers used available research publications related to such area of work provided by
experienced online learning colleagues. Some carefully chosen documents turned up with specifically
related to completion rate of MOOCs.
Finally the authors examined interactivity in different MOOCs to study about how this happens in
large courses (Khalil & Ebner, 2013a) and how participants as well as lecturers think about (Khalil &
Ebner, 2013b). However, the single results are putted together and discussed afterwards.
Results and discussions
Why do Students Dropout of MOOCs?
The high drop out rate in MOOCs has been attributed to many factors like:
Lack of time: Belanger & Thornton (2013) reported that completing MOOCs takes too much time.
Time is a significant factor, which may prevent students from completing the course requirements. Many
students pointed out that watching online lectures and completing homework assignments and quizzes was
simply too much to incorporate into their schedules. In a class discussion, one of students posted the
following thread in "Introduction to Mathematical Thinking" course offered on Coursera platform: “I really
love this course and many other courses on Coursera. But it does not look like I will be finishing it on
schedule because I do not have enough time to do the homework in addition to viewing the videos and
sifting through forums within the enforced deadline. Realistically, at this point, I can already see what will
be happening: I will end up downloading the videos, watching it on the sideline and reenrolling again in
the future to be able to do the homework.
In addition, Bruff (2013) suggested that some students want to move through the course week by
week, others want to have freedom to explore the content during the entire run of the course, and others
want to get all the lecture videos and assignments right up from the very first beginning.
Learners’ motivation: Motivation to participate in MOOCs is a significant area of interest. One of
the most important factors that may prevent students from completing MOOC is learners’ motivation.
According to (Yuan & Bowel, 2013) there are many factors that influence students’ motivation to learn;
these include future economic benefit, development of personal and professional identity, challenge and
achievement, enjoyment and fun. What motivates the MOOC learner? Surveys conducted by researchers at
Duke University show that student motivations typically fell into one of four categories (Belanger and
Thornton, 2013):
To support lifelong learning or gain an understanding of the subject matter, with no particular
expectations for completion or achievement.
For fun, entertainment, social experience and intellectual stimulation.
Convenience, often in conjunction with barriers to traditional education options.
To experience or explore online education.
On the pre-course survey, fun and enjoyment were selected as important reasons for enrolling by a large
majority of students (95%) and on the post-course survey, most reported that they have a general interest in
the topic (87%). Students used the online course to help them decide if they want to take college/university
classes (15%) while a significant minority of students claimed that they could not afford to pursue a formal
education (10%).
Feelings of isolation and the lack of interactivity in MOOCS: Palloff and Pratt (2003) believes that
feelings of isolation are the inherent result poor course design. Physical isolation can be overcome by
focusing more on social interactions. Many researchers pointed out the importance of interaction in high
quality MOOCs (Mcauley, Stewart, Siemens and Cormier, 2010; Waard, 2011; Levy and Schrire; 2012;
Fisher, 2012; Khalil & Ebner, 2013a). They confirmed the role of interaction and communication in
MOOCs as learners construct their own knowledge and develop their personal learning network from the
nodes and connections in the digital environment. Mak, Williams, and Mackness (2010) indicated that
interaction in MOOCs helps students to develop their own ideas, express themselves, establish a presence,
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
and make thoughtful long-term relationships. Without regular communication and interaction they lose
their focus. In addition, miscommunication and the lack of prompt, clear feedback from the instructor can
contribute to the student’s feelings of frustrations. Problems that could be solved in just a few minutes in
the classroom, or on the phone, can take hours or even days to solve via email. Murray finds that
communication is even more important than course content. Martz, et al. (2004) also mentioned that
prompt and personalized communications with the faculty have a significant impact on students’
satisfaction. But interaction can be time-consuming and difficult for faculty to sustain, especially with
larger class sizes. In our own study (Khalil & Ebner, 2013b) 35% of students stated their level of
satisfaction in MOOCs as less satisfied or not satisfied. They reported their dissatisfaction due to the lack of
instructor interaction. They had also complaints about online discussion forums with MOOC fellow
students that weren’t nearly as helpful as traditional in-class exchanges. When students have problems in a
normal classroom, they can turn to the other students, the teacher, or administration for almost immediate
support and feedback. The asynchronous nature of many MOOCs, combined with unusual study patterns,
and global time zones, means that students may not receive the support they need in a timely fashion,
reinforcing their feelings of isolation. InComputational Investing, Part I” course a student posted this
discussion thread I was excited about this course before it started but it has been down hill from the very
first lecture. The instructor seems to have no interaction with his students, he doesn't answer our questions
or reply our discussions and I hope coursera will ensure that this is never repeated. Based on this course, I
would never bother to look into any other course offered by this instructor.
Insufficient background knowledge and skills: Insufficient background knowledge and skills are an
important cause of low completion rate (Belanger and Thornton, 2013). Many students are not able to
complete the course because they haven't enough background and skills. Students' complaints often about
an assumed “knowledge base” that was often essential to understanding the course material. Murray (2001)
also recognizes the possibility that students may lack the required skills to be successful in online courses.
Students not only require the technical skills to get online, but they also need to possess strong reading,
writing and typing skills, since so much of the interaction in online courses is typically text-based. Students
with poor typing skills may find themselves frustrated, and unable to participate, especially if the course
requires frequent use of synchronous “chat” programs as part of the learning process. In a thread posted in
discussion forums in Model thinking” course offered by Coursera, a student find difficulty to complete the
course because the course need a certain background or skills. He wroteI am just trying to find out which
background I should go after to understand because I couldn't get past the second quiz of week 1. I realize
that this quiz requires a certain culture or background which I don't seem to have”.
Hidden costs: Hidden costs may cause low completion rate of MOOCs. Students were surprised to
see that, despite MOOCs’ reputation as a free online educational resource, they were sometimes required to
purchase pricey textbooks recommended by professors. In a thread discussion in "Introduction to
Computational Finance and Financial Econometrics " course available on coursera , a student posted the
following threadThe first book is not yet published and the other two are copyrighted books. You can't
expect legal (free text of them in public domain). But you can check out the previews of these books in
books.google.com. Few pages are intentionally removed from such previews but you have to live with them
if you wantto use free legal text.
In addition, O’Reilly (2013) indicated that some students of MOOC have to pay for their certificates.
What are the techniques that increase retention in MOOCs?
While there are many suggestions and recommendations in literature that are offered to increase
retention (Fisher & Han, 2008), the most favored techniques that may increase the retention in MOOCs are
the following:
Accommodating students on different time tables: Bruff (2013) suggested that instructors should
accommodate different student pacing as much as they could, however there were some limitations for this
technique. For instance, peer-graded assignments require students to participate with some synchronicity.
All student work must be submitted before any student work can be distributed for peer grading, and all
peer assessments must be finished before the results of those assessments can be shared back with students.
One could move through the material at any pace and still receive a standard statement, but achieving the
“with distinction” statement required more of a week-by-week pacing.
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
Promoting student completion: Instructors should remember that there’s in general no credit being
offered for students who study MOOCs, just a statement of accomplishment “signed” by the instructor.
People take MOOCs largely because they want to learn something of interest or value to them. As a result,
they should be motivated to complete the course otherwise they leave it. Here are some techniques that may
promote students to complete their MOOCs (Belanger and Thornton, 2013):
Formal recognition of accomplishment - Although the market or educational credential value of
this certificate is not yet clear, students cited this formal recognition of accomplishment as a factor
in motivating them to enroll initially as well as to persist in completing the course requirements.
Professional development - many students expected that the knowledge or skills would enhance
their professional work, improve job performance or promote their advancement in the workplace.
Participation in the forums and other student interaction - In addition to responses by the
instructor, students frequently responded to one another, encouraged one another and shared
supplementary resources. It also gives students a sense of belonging to a virtual learning
community that they can turn to when they need help. In addition, (Gleason, 2004) indicated that
this sense of community makes it easier if the students need to work in groups on various class
projects.
Enhance student to students and student to instructor interaction: Interaction is the key to
MOOC success among participants. Frankola (2012) pointed out that the best kind of interactivity not only
creates a sense of community for participants; it also stimulates learning through discussing ideas and
practicing skills. Student satisfaction in online courses can be significantly impacted by perceived
interaction in the online environment (Wuensch et al., 2008). According to Moore (1989) “student to
student” interaction refers to the exchange of information and ideas amongst students with or without the
real-time presence of an instructor. To enhancestudent to studentinteraction, MOOC lecturers should
organizeface to facestudy groups in various physical locations or separate online forums for participants
to promote learning and understanding through the sharing of ideas, perspectives and experiences with
other participants. In addition “student to instructor” interactions refer to the interaction between students
and experts, which establish an environment that encourages students to understand the content better.
Wuensch et al. added, “Instructor interaction had the largest influence on student satisfaction with the
distance course. Student-instructor interactions enhance student retention, self-motivation, and pass rates”
(p. 525). Most dissatisfaction among students was the result of perceived poor quality of interaction with
the instructors and among other students (Wuensch et al.). Khalil & Ebner (2013b) suggested the following
techniques to enhance student to instructorinteraction:
Trained teaching assistants (TAs): As long as, it is impossible for instructors to interact with this
huge number of students, TAs can assist the instructor to interact with students. TAs help students
who can't complete tasks. They can answer students questions, provide their advices if students
have technical problems, post some discussion topics, monitor the discussion forum on a regular
basis, and can filter out questions that need an instructor response.
Peer-based assessment: In addition, instructors can use also peer-based rather than computer-based
assessment to make MOOCs more interactive. It has been shown that students are willing to step
in and help others. Peer assessment is a key challenge in the delivery of MOOCs. Coursera also
acknowledges “In many courses, the most meaningful assignments do not lend themselves easily
to automated grading by a computer. Peer assessments in Coursera leverage a “grading rubric” to
help students to assess others reliably and provide useful feedback. Cronenweth (2012) pointed
out that peer assessment process is a useful form of learning for students. In addition, Wong
(2013) stated that peer assessment process does a good job of exposing students to someone else’s
work. “That is where the learning is at.” Using the previous techniques will enhance interaction
and make students more satisfied of interaction in MOOCs.
Supplemental tutoring: An option that could include assistance with specific course assignments or
more general training in prerequisite skills (Castles, 2004; Lentell & O'Rourke, 2004).
Better Development of Online Learning Skills: The typical online course may require a set of
skills in addition to those required in traditional face-to-face classrooms, such as technological
skills, self-directed learning and time management. These may represent a strong challenge to
many students, particularly those without or less academic background. Therefore, colleges may
need to provide more active support to students to help them understand the types of skills
required for successful online learning and to explicitly help them develop those skills.
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
Conclusion
Massive Open Online Courses (MOOCs) have the potential to enable free university-level
education on an enormous scale. However, many MOOCs initiatives continue to report high attrition rates
among distance education students. As a result, it is widely agreed that it would be useful to improve the
retention rates of MOOCs by finding out why students drop out of courses and to suggest strategies that
that can be implemented to increase the retention rate. According to literature, accurate data analysis and
own observations on running MOOCs we collected crucial factors for the high drop out rate in MOOCs:
lack of time, lack of learners’ motivation, feelings of isolation and the lack of interactivity in MOOCs,
insufficient background and skills and hidden costs. Consequently, some techniques should be used to
increase the online retention rate, and allow more online students to graduate. For example accommodating
students to different timetables, promoting student completion or enhancing "student to students " and
"student to instructor" interaction as well as increasing online learning skills. Finally it must be pointed out
that this research work is a first contribution to improve the retention rate and that in future these
suggestions must be implemented and evaluated.
References
Balsh, T. (2013). Why the “Low MOOC Completion Rate” Statistic is a Bogus Argument, Retrieved 27th
August, 2013, available at http://augmentedtrader.wordpress.com/2013/07/24/why-the-low-mooc-
completion-rate-statistic-is-a-bogus-argument/
Belanger, Y. & Thornton, J. (2013). Bioelectricity: A Quantitative Approach, Duke University’s First
MOOC, Duke Center for Instructional Technology.
Bruff, D. (2013). Lessons Learned from Vanderbilt's First MOOCs - Center for Teaching, Retrieved 2nd
August 2013 , from http://www.irrodl.org/content/v5.1/lentell_orourke.html
Castles, J. (2004). Persistence and the adult learner. Active Learning in Higher Education, 5 (2), 166-179.
Cronenweth, S. (2012). Peer Assessment in MOOCs and Online Courses, Retrieved 16th February 2013,
available at: http://blog.socrato.com/peer-assessment-in-moocs-and-online-courses/
Fisher, A. & Han, A. (2008). Designing Online Courses To Promote Student Retention, J. Educational
Technology Systems, Vol. 36(1) 105-112, 2007-2008
Fisher, D. H. (2012). Regional Sections of Massively Open Online Courses, Presentation to the 18th
Annual Conference of the Coalition of Urban and Metropolitan Universities, Retrieved 12th January, 2012,
available at: http://www.vuse.vanderbilt.edu/~dfisher/CUMU-2012-MOOC-Abstract.pdf
Frankola, K. (2012). Why Online Learners Drop Out High dropout rates are e-learning’s embarrassing
secret. Here’s what you can do about it. , Retrieved 12th August 2013, available at: http://www.c3l.uni-
oldenburg.de/cde/support/readings/frankola.htm.
Gaebel, M .(2013). MOOCs Massive Open Online Courses, European University Association , Retrived
2nd December, 2013,available at:
http://www.eua.be/Libraries/Publication/EUA_Occasional_papers_MOOCs.sflb.ashx
Jordan, K. (2013). MOOC Completion Rates: The Data, Retrieved 27th July 2013, available at:
http://www.katyjordan.com/MOOCproject.html
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
Khalil, H. & Ebner, M. (2013a) Interaction Possibilities in MOOCs How Do They Actually Happen?,
International Conference on Higher Education Development, p. 1-24, Mansoura University, Egypt
Khalil, H & Ebner, M. (2013b). " How satisfied are you with your MOOC?" – A research Study on
Interaction in Huge Online Courses. In proceedings of the World Conference on Educational Multimedia,
Hypermedia and Telecommunications 2013, 830- 839.
Lentell, H., & O'Rourke, J. (2004). Tutoring large numbers: an unmet challenge. The International Review
of Research in Open and Distance Learning, 5 (1), retrieved August 27, 2013, from
http://www.irrodl.org/content/v5.1/lentell_orourke.html
Levy, D & Schrire, S. (2012) The Case of a Massive Open Online Course at a College of Education,
retrieved 30th August 2013, available at : http://conference.nmc.org/files/smkbMOOC.pdf
Mak, S., Williams, R., & Mackness, J. (2010). Blogs and Forums as Communication and Learning Tools in
a MOOC, Proceedings of the 7th International Conference on Networked Learning, ISBN 978-1-86220-
225-2, p.275-284.
Martz, W. B., Reddy, V. K., & Sangermano, K. (2004). Looking for Indicators of Success for Distance
Education. In C. Howard, K. Schenk & R. Discenza (Eds.), Distance learning and university effectiveness:
changing education paradigms for online learning. Hershey, PA: Information Science Pub.
McAuley, A., Stewart, B., George Siemens, G. & Dave Cormier, D. (2010). Massive Open Online Courses
Digital ways of knowing and learning, The MOOC model For Digital Practice, Retrieved 12th July, 2013,
available at http://www.elearnspace.org/Articles/MOOC_Final.pdf
Meyer, R. (2012). What it’s like to teach a MOOC (and what the heck’s a MOOC?), Retrieved 20th July
2013, available at: http://tinyurl.com/cdfvvqy
Moore, M. G. (1989). Editorial: Three types of interaction. The American Journal of Distance Education,
3(2), p. 1-6.
O’Reilly (2013). The Big Problem for MOOCs Visualized, open culture, retrieved 20th July 2013, available
at: http://www.openculture.com/2013/04/the_big_problem_for_moocs_visualized.html.
Rivard, R. (2013) Measuring the MOOC drop-out rate Inside Higher Education, March 8 2013, retrieved
12th August, 2013, available at:http://www.insidehighered.com/news/2013/03/08/researchers-explore-who-
taking-moocs-and-why-so-many-drop-out
Sharples, M. (2012). Innovating Pedagogy 2012, Exploring new forms of teaching, learning and
assessment, to guide educators and policy makers, Open University Innovation Report 1.
Siemens, Downes, Cormier, & Kop. (2010). Teaching in Social and Technological Networks. Connectivism
blog. Retrieved 20th November, 2013 , available at: Nov 30, 2011, from http://www.connectivism.ca/?cat=3
Tamburri, R.(2012) : All about MOOCs, Retrived 2nd December 2013 , available at:
http://www.universityafairs.ca/all-about-moocs.aspx.
Kelvin Thompson, K . (2011). 7 Things You Should Know About MOOCs - EDUCAUSE.edu, Retrived 2nd
December, 2012 , available at: http://www.educause.edu/library/resources/7-things-you-should-know-
about-moocs.
Originally published in: Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve
Retention - A Literature Review. In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications 2014 (pp. 1236-1244). Chesapeake, VA: AACE.
Waard, I. (2011). Explore a New Learning FrontierMOOCs (Jul 11), retrieved 2nd September 2013,
available at: http://www.cedma-
europe.org/newsletter%20articles/eLearning%20Guild/Explore%20a%20New%20Learning%20Frontier%2
0-%20MOOCs%20(Jul%2011).pdf.
Wong, M. (2013). Online Peer Assessment in MOOCs: Students Learning from Students, Center or
Teaching, Learning and Technology, The University of British Columbia, retrieved 1st March 2013,
available at: http://ctlt.ubc.ca/2013/03/28/online-peer-assessment-in-moocs-students-learning-from-
students/.
Wuensch, K. L., Aziz, S., Ozan, E., Kinshore, M., & Tabrizi, M. H. N. (2008). Pedagogical characteristics
of online and face-to-face classes. International Journal on E-Learning, 7(3), 523-532.
Yuan, L. & Bowel, S. (2013) MOOCs and Open Education: Implications for Higher Education, Retrieved
25th July 2013, available at: http://publications.cetis.ac.uk/2013/667
... Within this expansion, several issues have evolved into serious dilemmas that affect the different stakeholders in these learning environments. Such issues are the dropout and incompletion rate (Khalil & Ebner, 2014), repetition of learning scenarios, lack of interaction with the 1 http://www.openeducationeuropa.eu/de/european_scoreboard_moocs (last access August 2015) Draft Version-Originally published in: Spector, M., Lockee, B., Childress, M. (Ed.), Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy, Springer International Publishing, pp. 1-30. ...
... A research study performed by (Jordan, 2013) found that 7.6% is the average completion rate in MOOCs. Furthermore, MOOCs are familiar with high attrition rates and a low motivation environment for learners (Khalil and Ebner, 2014). (Rivard, 2013) stated that a Coursera MOOC called "Bioelectricity" lost 80% of its students before the course actually began, the course finished up with 350 certified students out of 12,700 registrants. ...
Preprint
Massive Open Online Courses (MOOCs) are the road that led to a revolution and a new era of learning environments. Educational institutions have come under pressure to adopt new models that assure openness in their education distribution. Nonetheless, there is still altercation about the pedagogical approach and the absolute information delivery to the students. On the other side with the use of Learning Analytics, powerful tools become available which mainly aim to enhance learning and improve learners performance. In this chapter, the development phases of a Learning Analytics prototype and the experiment of integrating it into a MOOC platform, called iMooX will be presented. This chapter explores how MOOC Stakeholders may benefit from Learning Analytics as well as it reports an exploratory analysis of some of the offered courses and demonstrate use cases as a typical evaluation of this prototype in order to discover hidden patterns, overture future proper decisions and to optimize learning with applicable and convenient interventions.
... To deliver technical workforce development at scale, the virtual method was the best option. Traditionally, many online courses have extremely high attrition levels (e.g., massive online courses (MOOCs) have an <10% completion rate) [Khalil and Ebner 2014]; However, the L'SPACE Program has been able to achieve an online program design for technical training that has allowed them to achieve a 90% completion rate -even through the disruptive time of COVID-19. This completion rate is quite a significant achievement since, in the L'SPACE model, the students receive no academic credit or stipend for their participation. ...
Article
Full-text available
NASA’s Lucy Student Pipeline and Competency Enabler (L’SPACE) is a novel student collaboration program designed and implemented for the NASA Lucy Mission. This technical STEM workforce development program has been designed to provide an authentic, team-based learning environment to support STEM majors as they prepare to enter the space exploration ecosystem. L’SPACE recruits and serves from colleges and universities all over the United States of America and is designed at every level to foster a STEM identity in the students, working to create a more diverse workforce in the future. From the program’s inception in Fall of 2018 through the Fall of 2023, over 8967 unique students have been accepted and participated in the program, with 8070 having completed the L’SPACE academies. Through active mentoring and support, the completion rate for the L’SPACE Program is 90%. L’SPACE participants come from all 50 states, the District of Columbia, and Puerto Rico and represent 915 US colleges and universities, including 328 community colleges. The level of effort of the participants across all the elements of L’SPACE represents over 750,000 hours of workforce development training delivered and completed. The L’SPACE Team’s staff, internal and external evaluators actively evaluate all elements of the program to make sure that the program is meeting the participants’ and industry’s needs. L’SPACE uses a fast track and iterative approach, being informed by the data to make needed adjustments to the program as often as needed to reflect participant-centered and/or industry changes in the rapidly evolving workforce development environment.
... Stewart 2010;Breslow et al. 2013). Over time, as the number of MOOCs has increased, participation size and completion rate have become popular metrics to show the relative success of individual MOOCs, or to measure learner satisfaction (Adamopoulos 2013;Jordan 2013;Khalil and Ebner 2014;Jordan 2015;Onah, Sinclair, and Boyatt 2014;Stephens-Martinez, Hearst, and Fox 2014). These metrics are relatively easy to calculate and mirror common metrics already used by universities when evaluating formal education courses. ...
Article
Full-text available
It can be difficult to assess the design of, and learning, within Massive Open Online Courses (MOOCs). It is especially hard when trying to analyse this at the level of the individual learner. This study has developed a tool, inspired by Sankey diagrams, to visualise learners’ behaviour and paths through MOOC content. This tool can be used to investigate if learners are interacting with the content as planned when the course was designed. It has been designed iteratively through four stages of rapid prototyping. This paper presents the narrative of the development of the tool with an emphasis on validation via feedback from three user groups at each prototype stage.
... This, among other factors, causes them to have become increasingly popular over the last few years and attract hundreds or thousands of users in some cases. However, the number of students that effectively complete the courses is significantly smaller, as the dropout rate of many MOOCs is above 90% [1]. Having significant amounts of data collected on the users within a course lends itself well for quantitative statistical analysis with respect to the factors indicative of completion or dropout. ...
Preprint
Massive Open Online Courses (MOOCs) are attracting the attention of people all over the world. Regardless the platform, numbers of registrants for online courses are impressive but in the same time, completion rates are disappointing. Understanding the mechanisms of dropping out based on the learner profile arises as a crucial task in MOOCs, since it will allow intervening at the right moment in order to assist the learner in completing the course. In this paper, the dropout behaviour of learners in a MOOC is thoroughly studied by first extracting features that describe the behavior of learners within the course and then by comparing three classifiers (Logistic Regression, Random Forest and AdaBoost) in two tasks: predicting which users will have dropped out by a certain week and predicting which users will drop out on a specific week. The former has showed to be considerably easier, with all three classifiers performing equally well. However, the accuracy for the second task is lower, and Logistic Regression tends to perform slightly better than the other two algorithms. We found that features that reflect an active attitude of the user towards the MOOC, such as submitting their assignment, posting on the Forum and filling their Profile, are strong indicators of persistence.
... Reich (2014) finds that intention to complete is a stronger predictor than any of several demographic traits measured across a sample of 9 MOOCs. In a review of works surveying potential causes of MOOC dropout rates, Khalil and Ebner (2014) also find that lack of time, lack of motivation, lack of interaction, and "hidden costs" (such as paid textbooks needed for reference, or paid certificates of which learners were unaware) contribute to MOOC dropout. ...
Preprint
Predictive models of student success in Massive Open Online Courses (MOOCs) are a critical component of effective content personalization and adaptive interventions. In this article we review the state of the art in predictive models of student success in MOOCs and present a categorization of MOOC research according to the predictors (features), prediction (outcomes), and underlying theoretical model. We critically survey work across each category, providing data on the raw data source, feature engineering, statistical model, evaluation method, prediction architecture, and other aspects of these experiments. Such a review is particularly useful given the rapid expansion of predictive modeling research in MOOCs since the emergence of major MOOC platforms in 2012. This survey reveals several key methodological gaps, which include extensive filtering of experimental subpopulations, ineffective student model evaluation, and the use of experimental data which would be unavailable for real-world student success prediction and intervention, which is the ultimate goal of such models. Finally, we highlight opportunities for future research, which include temporal modeling, research bridging predictive and explanatory student models, work which contributes to learning theory, and evaluating long-term learner success in MOOCs.
... Challenges in course content were revealed in several studies exploring attrition rates in MOOCs (Khalil & Ebner, 2014;Kizilcec & Halawa, 2015;Shapiro et al., 2017). If enrollment is primarily based upon curiosity and interest, then comprehension of the course material could be difficult for those enrolled in MOOCs. ...
Article
Full-text available
The study identified university students’ levels of motivation and engagement, perceived benefits, and challenges in Massive Open Online Course (MOOC). It used the mixed-method research design and involved third- and fourth-year English language students (n = 61) who completed the Content-based Instruction (CBI) – MOOC offered by the Regional English Language Office, Manila, Philippines. The validated MOOC Learner Engagement and Motivation Scale (MEM Scale) was adopted to determine the learners' levels of motivation and engagement in MOOCs. A focus group discussion was also done to determine the students’ perceived benefits of and challenges in MOOCs. Results of the MEM Scale show that the students had upper-medium level of motivation (x̄ = 3.54) and a high level of engagement (x̄ = 4.22) in the CBI – MOOC. Also, the data revealed five perceived benefits of MOOCs: 1) appreciation of the self-paced learning; 2) complementation to formal learning; 3) international certification; 4) provision of professional development opportunities; and 5) attainment of intercultural competencies. Generally, the students experienced five challenges while enrolled in the course: 1) language barriers; 2) lack of social interaction; 3) digital divide; 4) time-related constraints; and 5) challenging course content. Given the benefits of MOOCs, higher education institutions (HEIs) are encouraged to incorporate open online courses in the learning experiences of their students. Further, it is recommended that HEIs extend available resources to participants to lessen the perceived challenges of MOOCs which could promote higher levels of motivation and engagement of the students in the courses.
... These include, but are not limited to, assessment mapping, visual storyboarding or utilising a backward design approach (Kerr et al. 2021;Conole 2014). There is also a large body of evidence developed over the last decade that focuses directly on MOOC retention rates and learner motivations for undertaking MOOCs (Badali et al., 2022;Liyanagunawardena, Williams & Adams 2013;Khalil & Ebner 2014;Ho et al. 2015). Therefore, this research draws inspiration from this but does not focus on traditional motivational enrolment points. ...
Article
This paper proposes practical methods implemented in the educational process for assessing the quality of an online course to optimize the work of the course development team and reduce labor costs for supporting the online course. The first method is the collection and analysis of statistical indicators, which allows you to assess the objective behavior of students in the online course, interact with all course elements, and identify the most and least popular elements of the online course. The second method relies on students’ subjective perception of the course. Correctly compiled feedback questionnaires will allow you to identify the systematic nature of complaints about specific problem patterns and speed up work with them when building a course. Both methods for assessing course elements distribute them to one of three subblocks: theoretical, practical, and additional. A default weight was assigned to each of these subblocks, and weights from the course plan were used for elements related to the course user assessment. At the same time, objective and subjective methods of assessing the quality of the course allow a comprehensive examination of the course and find the most problematic areas of the course, which allows for saving the labor resources of developers for direct work on the course. Together, both methods fully cover the course as a whole, making it easier for the course team to optimize their work on its improvement. The study considered the parameters and questionnaires for evaluating online courses for the Simon Kuznets Kharkiv National University of Economics website of Personal Learning Systems. Keywords: online course quality, educational process, higher education institution, education seekers, personal learning systems.
Technical Report
Full-text available
This report sets out to help decision makers in higher education institutions gain a better understanding of the phenomenon of Massive Online Open Courses (MOOCs) and trends towards greater openness in higher education and to think about the implications for their institutions. The phenomena of MOOCs are described, placing them in the wider context of open education, online learning and the changes that are currently taking place in higher education at a time of globalisation of education and constrained budgets. The report is written from a UK higher education perspective, but is largely informed by the developments in MOOCs from the USA and Canada. A literature review was undertaken focussing on the extensive reporting of MOOCs through scholarly blogs, press releases as well as openly available reports and research papers. This identified current debates about new course provision, the impact of changes in funding and the implications for greater openness in higher education. The theory of disruptive innovation is used to help form the questions of policy and strategy that higher education institutions need to address.
Article
Full-text available
A MOOC is a Massive Open Online Course. It is a gathering of participants, of people willing to jointly exchange information and collaboratively enhance their knowledge. Over 10,000 learners worldwide have participated in MOOCs on a variety of topics. MOOCs appeal to knowledge workers, trainers, and teachers. Why? Because MOOCs enable a high-end knowledge exchange to occur. It is a learning model that fits expert training, interdisciplinary learning, ad hoc education — any type of learning that allows information to flow within a network of peers. MOOCs fit the contemporary shift towards networked learning. George Siemens, one of the pioneers in this area, wrote, "Learning is now happening through communities of practice, personal networks, and through completion of work-related tasks," in an environment in which, "know-how and know-what is being supplemented with know-where (the understanding of where to find knowledge needed)" (2005, p. 4). Use of educational technology is becoming more mainstream through social media and mobile devices. Thus there is rising interest in finding methodologies that build upon these new technologies to enhance the learning and teaching process. MOOC is one of these emerging formats. A MOOC can boost your institutional, corporate, or NGO knowledge, if you are open to its innovative approach.
Article
Full-text available
Although the issue of student retention is a campus-wide one, it is of special interest in online distance learning courses, where retention rates are reported to be lower than in face-to-face classes. Among the explanations and theories of retention rates in online courses, one that struck us as most useful is a structural one, namely, course design. The question we raise is, can online course designs promote student retention? In this article, we report on how we used Quality Matters to design and revise online courses in psychology and statistics. Quality Matters, a research-based initiative, advocates the use of eight general review standards to review online courses. In our psychology and statistics courses, our retention rate across multiple offerings of both courses is approximately 95%.
Article
Full-text available
Conference Paper
Full-text available
This paper presents the findings of research carried out into the use of blogs and forums as communication and learning tools in Connectivism and Connective Knowledge (CCK08), a Massive Open Online Course (MOOC) that was run between September and December 2008. The course was unique in many ways: it included a small credit-bearing course within a network; it was completely open and very large, and it included a unique aggregated network of blogs, which was one of the reasons why an unusually large number of online interactions took place in blogs rather than just in forums. The research used a survey and email interviews to explore a number of themes identified in the postings in the course which included: the importance of conceptual connections, personal connections, personal autonomy, media affordances and approaches to learning. The research identified three distinct clusters of respondents, with the following preferred modes of interaction: (1) blogging, (2) using the forums, (3) using both blogging and forums. A number of other modes of interaction were also used by the respondents. Many respondents used both blogs and forums for at least some of the time in the course. To a large extent, blogging and forum use correlated with specific individual learning styles and media affordances: the use of blogs was associated with the ability to create personal space for personal learning, quiet reflection and developing personal relationships with bloggers and others. The use of forums was associated with fast paced challenging interaction, relationships based on sharing of ideas, more open discussion and more links to the discussed themes and bigger picture. However, the research also identified three dimensions (home> <bazaar, long-loop> < short-loop, and engagement> < reflection) within which the new media provided different and new opportunities for participating in learning networks. Respondents used these to explore and develop their own affordances for learning in varied and even surprising and novel ways, sometimes using blogs or forums for particular purposes, but sometimes using them almost interchangeably. The research points to a maturing of media users in online learning. These users are developing their own personal learning networks, and using the new media in innovative and nuanced ways.
Article
Currently, many students have had experience with both face-to-face and online classes. We asked such students at 46 different universities in the United States to evaluate the pedagogical characteristics of their most recently completed face-to-face class and their most recently completed online class. The results show that students rate online classes as greatly superior to face-to-face classes in terms of convenience and allowing self-pacing, but they also rate online classes as inferior on a number of other characteristics. Online and face-to-face instructional formats each have their own strengths and weaknesses. Detailing those strengths and weaknesses should help us modify both methods of teaching to reduce the weaknesses and maintain the strengths.
Article
This research is part of a larger study of the factors affecting part-time adult learners. It covers a literature review dealing with three areas of factors that might affect adult learners: (i) social and environmental, (ii) traumatic and (iii) intrinsic factors. This produced 36 factors and was followed by a qualitative study to validate and refine the factors. This part of the research was undertaken with students from the Open University of the UK. The research resulted in the construction of a model of persistence containing 12 factors. The model was subjected to quantitative testing by questionnaire, and preliminary analysis showed that most of the factors were valid indicators of persistence. The model needs to be further refined in order to be predictive, but still indicates that institutions can help to overcome at least some of the problems preventing persistence.
Lessons Learned from Vanderbilt's First MOOCs -Center for Teaching Persistence and the adult learner
  • D J Bruff
Bruff, D. (2013). Lessons Learned from Vanderbilt's First MOOCs -Center for Teaching, Retrieved 2nd August 2013, from http://www.irrodl.org/content/v5.1/lentell_orourke.html Castles, J. (2004). Persistence and the adult learner. Active Learning in Higher Education, 5 (2), 166-179.