Conference PaperPDF Available

How can Gamification Improve MOOC Student Engagement?

Authors:

Abstract and Figures

Massive Open Online Courses (MOOCs) require students' motivation either intrinsically or extrinsically to complete any of its courses. Even though MOOCs enjoy great popularity and bring many benefits to the educational community, some concerns arise with MOOC advancement. In fact, MOOCs are affected by low completion rate and face issues with respect to interactivity and student engagement along MOOC duration, which may convert student excitement to boredom and then drop out at any stage. A key result of research in the past couple of years has proved that students' engagement in MOOCs is strongly related to their activities online. These activities are related to the interaction between student and logging in the MOOC, reading and writing in the MOOC discussion forum, watching videos and doing quizzes. In this research paper, we present our research in deploying a gamification mechanic in MOOCs to increase student engagement. The gamification approach relies on weekly feedback to drive student intrinsic and extrinsic motivation. Following learning analytics on students' data from a MOOC offered in 2014, 2015, and 2016, the outcome of this approach showed an obvious increase in students' activity and engagement in discussion forums, login frequency and quiz trials. The active students' cohort allotment has increased in comparison with previous versions of the same MOOC as well as the completion rate has incremented up to 26% of the total number of participants.
Content may be subject to copyright.
How can Gamification Improve MOOC Student Engagement?
Mohammad Khalil1, Martin Ebner2 and Wilfried Admiraal3
1Delft University of Technology, The Netherlands
2Graz University of Technology, Austria
3Leiden University, The Netherlands
m.f.d.khalil@tudelft.nl
martin.ebner@tugraz.at
w.f.admiraal@iclon.leidenuniv.nl
Abstract: Massive Open Online Courses (MOOCs) require students’ motivation either intrinsically or extrinsically to complete
any of its courses. Even though MOOCs enjoy great popularity and bring many benefits to the educational community, some
concerns arise with MOOC advancement. In fact, MOOCs are affected by low completion rate and face issues with respect
to interactivity and student engagement along MOOC duration, which may convert student excitement to boredom and then
drop out at any stage. A key result of research in the past couple of years has proved that students’ engagement in MOOCs
is strongly related to their activities online. These activities are related to the interaction between student and logging in the
MOOC, reading and writing in the MOOC discussion forum, watching videos and doing quizzes. In this research paper, we
present our research in deploying a gamification mechanic in MOOCs to increase student engagement. The gamification
approach relies on weekly feedback to drive student intrinsic and extrinsic motivation. Following learning analytics on
students’ data from a MOOC offered in 2014, 2015, and 2016, the outcome of this approach showed an obvious increase in
students’ activity and engagement in discussion forums, login frequency and quiz trials. The active students’ cohort allotment
has increased in comparison with previous versions of the same MOOC as well as the completion rate has incremented up
to 26% of the total number of participants.
Keywords: gamification, massive open online courses (MOOCs), learning analytics, motivation, retention, dropout
1. Introduction
Massive Open Online Courses (MOOCs) have gained a lot of attention and impetus in the last five years. They
are distinguished among other learning environments by being open for everyone, easy to enrol, and having a
heterogeneous community. MOOCs were first coined by George Siemens and David Cormier in 2008 when they
described a four-month course on connectivism theory (Cormier & Siemens, 2010). Years later, Sebastian Thrun,
a Stanford University professor, offered an online course called “Introduction to Artificial Intelligence” that
received a wide publicity of over 160,000 student registrations from all over the world (Yuan & Powell, 2013).
Since then, MOOC becomes an environment that was bet on bringing revolution to higher education as well as
to elementary education (Khalil & Ebner, 2015) based on factors of their popularity and massiveness of
enrolments (Martin, 2012).
Given that MOOCs are online systems with minimal direct interaction between students and tutors, learners are
required to self-regulate their learning. In Self-Regulated Learning (SRL) in classrooms, learners are asked to
adjust their attitude based on the educational context where learning happens (Zimmerman, 2002). While in
MOOCs, learners have to identify what type and how much of activities they need to engage with (Littlejohn et
al., 2016).
In both traditional and online learning, student engagement is a crucial aspect for learning (Lam et al., 2012).
Carini, Kuh, and Klein (2006) found that student engagement were linked positively with learning outcomes like
quiz performance and critical thinking. On the other hand, Archambault et al (2009) identified that student
engagement can be used as a forecast element for dropout in schools. Handelsman (2005) used engagement as
a short-term indictor to distinguish between students in higher education settings to either performance-goal
oriented or learning-goal oriented students. Hew (2016) mentioned that student engagement might be well
suited for social cognitive behaviour and academic achievement prediction.
In respect to MOOCs, Milligan, Margaryan, and Littlejohn (2013) stated that motivation is an important
benefactor to student engagement as well as an indicator of student engagement.
1 The author’s research is supported by the Leiden-Delft-Erasmus Centre for Education and Learning
819
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
Despite the fact that MOOCs are rapidly developing, they are affected by low completion rate and weak student-
system and student-teacher interaction (Balakrishnan & Coetzee, 2013; Khalil & Ebner, 2016; Littlejohn et al.,
2016). In one of the recent studies, this has been linked to the poor engaging design of MOOCs (Chang & Wei,
2016). Researchers have proposed different approaches to improve student engagement through concentrating
at motivation from both sides of one coin, intrinsic and extrinsic motivations. For instance, one study linked the
intrinsic motivation of MOOC learners to the inner intention to learn, compete and satisfy their curiosity (Wang
& Baker, 2015). Other research studies tried to increase student engagement through drawing focus on the
extrinsic side of motivation by introducing open badges and certificates to attract online learners (Wüster &
Ebner, 2016). Although classical reward certificates were promising when they first were introduced, learners
think that completing a MOOC to have completion certificates is not sufficient enough to finish a course (Hew,
2016). As a result, various new strategies that targeted to ignite the extrinsic motivation of learners were
proposed; gamification as one of these strategies. The main aspect of using gamification in educational settings
were mentioned in different studies as a tool, strategy, and methodology to entertain students (Gené, Núñez &
Blanco , 2014; Chang & Wei, 2016; Wüster & Ebner, 2016).
Motivated by the increased impetus of gamification in education (Chang & Wei, 2016) and the youth generation
of online learners (Hansen & Reich, 2015), in this publication we present our experiment on using an early-
developed gamification framework (Khalil & Ebner, 2017) in a massive open online course. The goal of this
research is to study the use of a gamified strategy that looks at increasing students’ incentive to study MOOCs
yet sparking their curiosity taking into account both intrinsic and extrinsic motivation factors. The intrinsic
motivation will be driven by curiosity-guided strategy and the extrinsic motivation by a rewarding strategy
influenced by the theory study of Ryan and Deci (2000). The study takes place in one of the offered MOOCs from
the Austrian MOOC provider, iMooX (http://www.imoox.at). The main research question of this study is to
examine whether gamification can increase student engagement and activities and henceforth improve the
retention rate of MOOC learners. To carry out this research, we used a local developed learning analytics tool to
support us in collecting and analysing the data from the MOOC.
The remainder of the paper is organized as follows: Section 2 is the related work of gamification use in MOOCs.
Section 3 gives a short background of the tested MOOC and the iMooX platform. Section 4 lists the followed
gamification approach and the deployment phase. Section 5 presents the results and discussion of this study.
Finally, section 6 draws the conclusions.
2. Related work
This section enriches the paper by reviewing some of the related topics of gamification and engagement in the
context of MOOCs. Through our examination in different academic libraries, the return results showed some
relevant studies that surveyed gamification in MOOCs and others that aimed at improving students’ engagement
through the use of gamification mechanics. Gamification is defined as “the use of game design elements in non-
game contexts” (Deterding et al., 2011). Given the issues of dropout and motivation in MOOCs, gamification
mechanics were considered promising to alleviate such dilemmas. Practitioners linked gamification strategies in
non-game settings to add fun and increase engagement and enjoyment of a service or a product (Hsu, Chang, &
Lee, 2013). Gamification incorporates entertainment on its surface, but may function as playground for
competition and collaboration. In respect to MOOCs, the popular MOOC providers like Coursera and edX became
aware of the gamification impact on learners and introduced the use of rewards and badges as strategies to
attract the students. The language-learning platform Duolingo has gone further with gamifying the platform.
Duolingo uses skill points, badges, certificates, and a levelling system as incentive to encourage students to study
and increase the competition level among them. Another popular MOOC platform, Khan Academy, employed
gamification in various ways. Khan Academy has used Knowledge maps, badges and progress indicators in order
to enhance students’ motivation (Morrison & DiSalvo, 2014).
Through our search in the literature and to our knowledge, the biggest survey study on gamification in MOOCs,
was done by Chang and Wei (2016). Their study scrutinized over 40 gamification techniques and their impact on
student activities and motivation in MOOCs. Chang and Wei (2016) stated that not every gamification technique
could enthral online learners. They found out that virtual goods, redeemable points, team leader boards,
where’s Wally game, and trophies are the top five most engaging gamification mechanics in MOOCs. The authors
concluded the critical role of gamification in MOOCs as factors that push learners to spend a higher average
time, enhance self and content interaction.
820
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
An empirical study by Vaibhav and Gupta (2014) examined the use of gamification in a MOOC through an A/B
testing planned task. The researchers found out that the gamified quiz attracted a larger number of students
than those without gamification in regards to the number of quizzes student submitted. Additionally, Vaibhav
and Gupta (2014) realized that the success rate of the quizzes were higher for the cohort who were supported
with gamification and therefore recorded a slight increased retention in comparison to the control group.
Gené, Núñez, and Blanco (2014) suggested applying gamification in MOOCs by replicating an experiment from
Moodle learning system. The researchers looked at promoting cooperation and motivation in MOOCs and
lowering dropout rate by using different gamification strategies such as ranking rating, voluntary activities,
course progress and certificates. Two years later, the authors (Gené, Núñez, & Blanco, 2016) published a
qualitative interview and survey study stating that gamification tools have deepen student learning and
increased student motivation and engagement within the MOOC content.
Another practical experience by Morales et al (2016) used three gamification strategies (leader boards, badges,
and rewards) to increase student participation in their MOOC. The gamification approaches were deployed in
discussion forum, assignments section and completed activities. Results of this study showed that the rewarding
system was the most successful one based on a qualitative post survey while leader boards in the discussion
forum were evaluated the lowest engaging strategy. Nevertheless, the authors reported that students faced
challenges in regards to the dynamics of understanding each gamification strategy.
Although the previous studies shed light on gamification in MOOCs, the current literature involves little empirical
validation. Our study seeks to bridge, to some extent, this gap with additional evidences on the significant use
of gamification in MOOCs and its capabilities to improve student engagement.
3. The MOOC platform and the studied MOOC
3.1 The MOOC platform
As previously mentioned, this study is carried out cooperating with the Austrian MOOC platform (iMooX). iMooX
is an online stage and the first Austrian MOOC platform founded in 2013 by Graz University of Technology and
University of Graz. Since the first year launch of iMooX, over 5,000 registrations have been recorded in the
database. Most of the courses are offered in German language and target not only academic holders but also
elderly as well as school children. The pedagogical approaches of iMooX stand on the cognitive-behaviourist and
social-constructive pedagogies through providing a rich interactive discussion forum, a convenient structure of
information exchange, and a stimulus demonstration of active online learning videos. All the quizzes in iMooX
follow the multiple-choice questions system by which every quiz is setup to give students the ability to try each
weekly quiz more than once. The questions of each quiz trial are randomized for every attempt. The main reason
behind this is to less stress students so that each student behaves in a more comfortable manner by picking the
highest grade out of student tries, and to support the self-assessment learning guidance in MOOCs. When the
students successfully finish the quizzes and fill out an final evaluation form at the end of a MOOC, they are
rewarded with certificates completely for free.
3.2 MOOC studied
We implemented our research study on one iMooX MOOC called Free Online Learning course or as it is named
originally in German language Gratis Online Lernen (Ebner, Schön & Käfmüller, 2015) that was offered in 2016.
To ease denoting the course name in the context of the paper, we abbreviate the MOOC name to GOL in
corresponds to the German name. The course was offered in 2014, 2015, and 2016 respectively and was
instructed by Graz University of Technology. The structure of the MOOC was organized to be presented over a
time-period of eight weeks with a set load of 2 hours/week. The GOL MOOC topic focused on the general topic
of learning with the Internet (online learning) as well as on informing the public on the significant rising
momentum of Open Educational Resources (OER), as well as the right to access, share, and adapt them.
Following similar MOOCs, GOL course was supported with a set of affluent short videos, multiple-choice quizzes,
a discussion forum, and recommended articles that are available to download. The MOOC was open to everyone
without a need for any prerequisite knowledge. Furthermore the MOOC is following the didactical concept of
Inverse Blended Learning, first introduced by Schön & Ebner in 20142. The Inverse Blended Learning concept
2 https://elearningblog.tugraz.at/archives/7412
821
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
focuses on meeting up the online learners on a round table, handing them additional printed materials as well
as sharing thoughts and results in the online MOOC discussion forum. In comparison to other iMooX MOOCs,
GOL attracted that largest number of participants in its first version with around 1,000 students.
4. The gamification approach
The research at hand followed our early-published concept Activity-Motivation Framework (see Figure 1; Khalil
& Ebner, 2017). The main idea behind the conceptual framework was to increase retention rate through
enhancing student engagement in MOOCs. To do so, the framework relied on a weekly-gamified feedback to
drive student motivation. The scheme of the Activity-Motivation approach corresponds to the iMooX platform
potential of offering various MOOC variable data: 1) quiz attempts, 2) watching learning videos, 3) reading in
discussion forum, 4) posting in discussion forum, and 5) logging in MOOCs. In fact, the framework was set after
testing the hypothesis that says students who complete MOOCs are more likely to perform extra activities (like
watching videos, more involvement in forums..etc.) than those who do not (Khalil & Ebner, 2017), or in other
way around, the more students participate and engage the more likely they finish a course. Based on that, the
Activity-Motivation framework was developed in reliance on MOOC activities and the concept of motivating
students to increase the general engagement hoping for an increased completion rate.
In short, figure 1 displays a battery gamification element that is weekly charged based on four-dimensional
MOOC variables (the number of logins, the number of videos watched and rewatched, the number of quizzed
completed and the level of participation in the discussion forum). Our reason for choosing the battery icon
returns to our thought of what happens to a battery is somehow similar to what a student does in MOOC
platforms. We aimed at charging students with motivation and incentive by sparking their curiosity (Ryan & Deci,
2000). To attain the curiosity strategy, the algorithm was concealed on purpose and students never know how
the battery symbol is charged/filled. The intention was to stimulate student intrinsic motivation to do more
activities and actions in the MOOC by driving them to guess how the gamification approach works.
Each of the four dimensions contributes with a portion to fill the battery cumulatively. If a student logs in the
course, watches a video, makes activities in the discussion forum and does a quiz, the battery will be fully
charged. By the end of each week, the students are rewarded with the battery icon that only illustrates their
prior activities and a motivational statement. Any shortage of one or a combination of these MOOC variables,
the battery will be less charged based on student MOOC actions.
Figure 1: The MOOC activity-motivation framework employs MOOC variables in a gamification approach (Khalil
& Ebner, 2017)
4.1 Deployment of the gamification approach
In this part, we show how the gamification approach was deployed in the GOL MOOC. The process of the
gamification approach implementation was done manually since we were looking for evaluation results at the
first stage. A second stage of automatic implementation can be systemized on upcoming MOOCs if the results
are promising to the iMooX higher management. Our first step was to design the battery gamification element
that should be attached to each profile. We chose an open source software called Inkscape
(http://www.inkscape.org, last accessed: April 2017). Through our design, we aimed at having symbols that
822
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
supports the information-oriented aspect in which they reflect an easy understanding of visual elements. The
design of the symbols followed the recommendation list for an effective visual communication of graphical user
interface by Suzanne (1995). A key principle of Suzanne’s (1995) list is to have a clear and strong visual identity
of designed symbols. As a result, our design was simple and clear of a 2-D layout with a fine mix of light colours.
Figure 2 shows five categories of the battery gamification symbol by which each of them represents a single
status.
Figure 2: Five categories of the battery gamification element in iMooX MOOC platform
Each symbol in figure 2 was displayed on the top left side of each user homepage of the GOL-2016 MOOC and a
brief news feed was posted on the MOOC’s homepage advertising about the gamification symbols.
Every battery symbol represents the recorded MOOC activity of the user based on the learner’s previous week’s
interaction. For example, by the end of the first MOOC week, we show the student activity progress of that week
on the first day of the second week and so forth except for the period of the first week where the system was
automated to show 0% battery status since all students started with no activity.
The MOOC activities that were planned to be logged in our implementation were: a) logging into the MOOC
homepage, b) doing a quiz, c) posting/commenting at least once or reading two threads in the MOOC’s
discussion forum, and d) watching a video. Nevertheless, we faced a major technical problem with logging user
video activities. The problem was detected just a couple of days before the MOOC launch date
(10th.October.2016) when we discovered that the MOOC videos were embedded using IFRAME instead of
OBJECT on the iMooX platform. Therefore, our simple and quick redress was that by assuming a user logs in, we
show the “50%” battery symbol. Given this issue, we excluded the video activity and the quarterly-charged
battery symbol.
The final trailed approach for showing the battery symbols, thus, was established by the following guidelines
based on every week activities:
Login activity: When a student logs into the MOOC, the activity will reflect relatively on the gamification
element (battery). The first segment of the battery will be 50% charged. Several logins will not increase the
charged portion.
Quiz activity: The battery symbol will be filled with one extra portion (25%) when a student takes a quiz. As
described before, the iMooX MOOC-platform allows each student to try the weekly quiz up to five times.
However, just one trial would be enough to indicate that the student is active not just as a lurker. Several
attempts will not increase the battery’s charged portion.
Discussion forum activity: If a student is engaged in the forums either by writing at least one post or reading
threads twice, then the battery-charging portion will add another 25%.
To support a multiple form of representation for gamification symbols, we took into consideration showing a
percentage numeral that denotes progress and arranged a tooltip for each symbol so that students can recognize
what these symbols imply (see table 1).
Table 1: The gamification battery symbols with their tooltips
Battery Symbol
Tooltip
No activity last week –
we are looking forward
to seeing you again this week!
823
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
Battery Symbol
Tooltip
Your activity last week was 50%. Good!
Increase your activities to score better!
Your activity last week
was
75%. Great! Keep it
up!
Your activity in the previous week
was 100%.
Congratulations! Your commitment is
excellent. Keep it up!
5. Results and discussion
The further step was to evaluate the efficacy of the gamification approach and to tackle our research question.
The total number of enrolees of the GOL-2016 MOOC was not as much as the previous versions GOL-2014 and
GOL-2015 (Table 2). A brief explanation for the lower involvement might belong to the iMooX huge
advertisement back in 2014 when iMooX was first launched. That is, Gratis Online Lernen 2014 MOOC was one
of the few offered courses at that time.
Table 2: Total number of enrolments of the Gratis Online Lernen MOOC
Course
GOL-2014
GOL-2015
GOL-2016
Next, we collected through our manual implementation the total number of students and their engagement
level based on the gamification statuses as shown in figure 3.
Figure 3: Summary of the battery results (left to right 50%, 75%, 100%) of active students in GOL-2016 MOOC.
non-active students or the 0% battery are excluded
The figure shows an elevation of the 75% battery status in the number of students across all the MOOC weeks.
By investigating student behaviour in the discussion forums and the submitted quizzes, students who are
824
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
committed to complete the MOOC have intended to log in at least once a week and do the weekly quiz. That is,
students were more involved in doing quizzes than being involved in the discussion forums which is an expected
behaviour in order to have a certificate.
Also, the figure shows that there is a slight decrement of the 50% battery status from week1 to week8 wavering
from (N=19) in the first week to (N=7) in the eighth week. This was interesting since active students tried to push
more efforts to score higher than the 50% or the 0% battery status. For instance, the number of students who
were active has increased by 15.5% than the status of week1. Week4 showed the minimum score of 50% status.
On the other hand, the full activity status 100% was at its highest in the second week with around (N=38)
students. Our explanation of this behaviour can be interpreted by the fact that students were pushing more
efforts to improve their battery status influenced by the motivational triggers. Likewise, the stability of
participation in quizzes is clear across all the weeks. It is worth pointing out that some students might do quizzes
in different weeks. This can be complicated to track. Nevertheless, our tracking records were based on every
week’s quizzes, logins and forum activities.
To check the validity of how active the students were within the MOOC variables, we inspected the quiz activities
across every week of GOL-2014, GOL-2015 and GOL-2016 using the iMooX learning analytics tool. Figure 4
depicts bar plots for the number of students who did at least a one quiz in every MOOC week. The x-axis
represents the MOOC weeks; the y-axis represents the number of students (identical students not repetitive).
We preferred to show a plot for each MOOC since there is a substantial difference of enrolments among every
MOOC.
Figure 4: The number of students who did one quiz of each described week (a) Top-left: Gratis Online Lernen
2014 MOOC (b) Top-right: Gratis Online Lernen 2015 MOOC (c) Bottom: Gratis Online Lernen 2016
MOOC
In figure 4a, we can see that the number of students who did the GOL-2014 quizzes dropped across the weeks
except for a slight increase in the last week. Student engagement usually reveals a high attrition scale in activities
in the first two weeks (Balakrishnan & Coetzee, 2013). Likewise in figure 4b, the plot shows nearly the same
direction of GOL-2014 behaviour by which students were doing quizzes of GOL-2015 actively in the first two
825
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
weeks and then dropped till the last week of the MOOC. On the contrary, figure 4c shows a very interesting
student engagement behaviour in the GOL-2016 MOOC where the gamification approach was applied. The
number of students who did the second week quiz nearly doubled when compared with the first week. We
believe the reason behind is because every student was given a 0% battery status when the MOOC started. Given
that the gamification element mechanism was hidden as a type of curiosity-driven behaviour, i.e. intrinsic
motivation, students tried to figure out how to progress better the week after. An alternative explanation can
be related to the extrinsic triggers as well in which students pushed efforts in order to gain the highest status of
the battery gamification element.
The figure further shows that the number of students in week3 till week8 represents a stable participation rate
in weekly quizzes in comparison to GOL-2014 and GOL-2015 participation.
Before moving to the completion ratio of GOL MOOCs, we wanted to examine the first part of the research
question that hypothesise if our gamification approach has increased student engagement. With this in mind,
we categorized students to a) registrants and those who register in the MOOC; b) active students and those are
the students who make at least one quiz or post in the discussion forums; and c) certified who successfully
complete a MOOC and are granted a certificate.
With the available filtered data from the learning analytics tool, we categorized the students of GOL-2014 and
GOL-2016. The cohort distribution for these two MOOCs is shown in table 3. The offered MOOC in 2014 recorded
(N=1,003) registrations, GOL-2015 (N=476), while GOL-2016 had a total number of (N=284) registered
participants. Given the active student definition, the grouping of participants resulted in having (N=475,
P=47.3%) in GOL-2014, (N=188, P=39.49%) in GOL-2105, and (N=209, P=73.5%) in GOL-2016 of active students.
Surprisingly, the ratio of active student in GOL-2016 has increased (P=55.3%), by which denotes a significant
difference in comparison to GOL-2016 MOOC. This demonstrates that students were more digitally engaged and
did additional activities to have a higher score with the gamification symbol.
On the other side, the certification ratio results were also promising for the gamification deployed MOOC. GOL-
2016 had a (N=74, P=26.05%) of certified students which is relatively higher than the previous offered MOOCs.
GOL-2014 and GOL-2015 MOOCs have lower certification ratio equalled to (P=17.54% and P=19.74%)
respectively. Although the certification rate of the GOL-2016 MOOC was not of that big difference with the other
MOOCs, the students in GOL-2016 were more digitally engaged in comparison. Besides the quiz activity in the
GOL-2016 MOOC, student other actions were actively present in discussion forums and login frequency.
Table 3: Overview of Gratis Online Lernen MOOCs cohort distribution
Course
Registrants
Active students
Certified
GOL-2014
1,003
475 (47.35%)
176 (17.54%)
GOL-2015
476
188 (39.49%)
94 (19.74%)
GOL-2016
284
209 (73.59%)
74 (26.05%)
6. Conclusion
The use of gamified mechanics in non-game contexts has become popular recently (Deterding et al., 2011; Hsu
et al., 2013). Gamification is looked at with potential to leverage student engagement and motivation in
educational contexts. Thereupon the gamification capabilities and the given MOOC dilemmas, this paper
presented our experiment and the results of gamification deployment in a massive open online course that
focused on two main issues of MOOCs: participation and engagement.
We used a simple gamified approach that supported a weekly feedback. The deployment results show that the
MOOC by which gamification was applied to, has gained an increased level of students attention and
engagement. The gamification approach of this experiment was instrumental in increasing the student
interactions with MOOC variables on one side and increasing the student motivation to complete quizzes on the
other side, which leads to students wanting to complete the rest of the quizzes. In addition, the analysis
identified a stable participation rate in weekly quizzes in comparison to previous tested MOOCs. The outcome
of this research study also confirmed a slight increase in the certification ratio in MOOC when gamification was
deployed.
826
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
We believe our approach was distinct than previous research studies through targeting intrinsic motivation and
extrinsic motivation factors together. The intrinsic motivation was driven by obtaining student curiosity while
the extrinsic motivation was driven by the battery gamification symbols progress. However, the explanation of
this research study and how students react to both motivation triggers are only based on numeric and learning
analytics. A future direction by doing a post survey can further explain how student perceived our gamification
approach.
Finally, gamification may carry tremendous potential behind. That is, it ties strongly with student motivation and
therefore increases the general completion rate. The big MOOC players like edX, Khan Academy, and Coursera
have become aware of the importance of gamification designs and the future will carry new techniques that will
verify their impact on the success of MOOCs.
References
Archambault, I., Janosz, M., Fallu, J.S. and Pagani, L.S., 2009. Student engagement and its relationship with early high
school dropout. Journal of adolescence, 32(3), pp.651-670.
Balakrishnan, G. and Coetzee, D., 2013. Predicting student retention in massive open online courses using hidden markov
models. Electrical Engineering and Computer Sciences University of California at Berkeley.
Carini, R.M., Kuh, G.D. and Klein, S.P., 2006. Student engagement and student learning: Testing the linkages. Research in
higher education, 47(1), pp.1-32.
Chang, J.W. and Wei, H.Y., 2016. Exploring Engaging Gamification Mechanics in Massive Online Open Courses. Educational
Technology & Society, 19(2), pp.177-203.
Cormier, D. and Siemens, G., 2010. The open course: Through the open door--open courses as research, learning, and
engagement. Educause Review, 45(4), p.30.
Deterding, S., Dixon, D., Khaled, R. and Nacke, L., 2011. From game design elements to gamefulness: defining gamification.
In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, (pp.
9-15). ACM.
Ebner, M., Schön, S., Käfmüller, K., 2015. Inverse Blended Learning bei „Gratis Online Lernen“ – über den Versuch, einen
Online-Kurs für viele in die Lebenswelt von EinsteigerInnen zu integrieren. In Digitale Medien und Interdisziplinarität.
Nistor, N. & Schirlitz, S. (Hrsg). Waxmann, Medien in der Wissenschaft Bd 68. pp. 197-206.
Gené, O.B., Núñez, M.M. and Blanco, Á.F., 2014. Gamification in MOOC: challenges, opportunities and proposals for
advancing MOOC model. In Proceedings of the Second International Conference on Technological Ecosystems for
Enhancing Multiculturality (pp. 215-220). ACM.
Gené, O.B., Núñez, M.M. and Blanco, Á.F., 2016. New Challenges for the motivation and learning in engineering education
using gamification in MOOC. International Journal of Engineering Education, 32(1), pp.501-512.
Handelsman, M.M., Briggs, W.L., Sullivan, N. and Towler, A., 2005. A measure of college student course engagement. The
Journal of Educational Research, 98(3), pp.184-192.
Hansen, J.D. and Reich, J., 2015. Socioeconomic status and MOOC enrollment: enriching demographic information with
external datasets. In Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (pp. 59-
63). ACM.
Hew, K.F., 2016. Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS.
British Journal of Educational Technology, 47(2), pp.320-341.
Hsu, S.H., Chang, J.W. and Lee, C.C., 2013. Designing attractive gamification features for collaborative storytelling websites.
Cyberpsychology, Behavior, and Social Networking, 16(6), pp.428-435.
Khalil, M. and Ebner, M., 2015. A STEM MOOC for school children—What does learning analytics tell us?. In the
International Conference on Interactive Collaborative Learning 2015 (ICL) (pp. 1217-1221). IEEE.
Khalil, M. and Ebner, M., 2017. Driving Student Motivation in MOOCs through a Conceptual Activity-Motivation
Framework. Zeitschrift für Hochschulentwicklung, 12(1), pp.101-122.
Littlejohn, A., Hood, N., Milligan, C. and Mustain, P., 2016. Learning in MOOCs: Motivations and self-regulated learning in
MOOCs. The Internet and Higher Education, 29, pp.40-48.
Martin, F.G., 2012. Will massive open online courses change how we teach?. Communications of the ACM, 55(8), pp.26-28.
Milligan, C., Margaryan, A. and Littlejohn, A., 2013. Patterns of engagement in massive open online courses. Journal of
Online Learning with Technology, 9(2), pp.149-159.
Morales M., Amado-Salvatierra H.R., Hernández R., Pirker J., Gütl C. 2016. A Practical Experience on the Use of
Gamification in MOOC Courses as a Strategy to Increase Motivation. In: Uden L., Liberona D., Feldmann B. (eds)
Learning Technology for Education in Cloud – The Changing Face of Education. LTEC 2016. Communications in
Computer and Information Science, vol 620. Springer, Cham.
Morrison, B.B. and DiSalvo, B., 2014. Khan academy gamifies computer science. In Proceedings of the 45th ACM technical
symposium on Computer science education (pp. 39-44). ACM.
Ryan, R.M. and Deci, E.L., 2000. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary
educational psychology, 25(1), pp.54-67.
827
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
Suzanne, M., 1996. Effective Visual Communication for Graphical User Interfaces. Home: WPI EDU. [online],
https://web.cs.wpi.edu/~matt/courses/cs563/talks/smartin/int_design.html
Vaibhav, A. and Gupta, P., 2014. Gamification of MOOCs for increasing user engagement. In 2014 IEEE International
Conference on MOOC, Innovation and Technology in Education (MITE), (pp. 290-295). IEEE.
Wang, Y. and Baker, R., 2015. Content or platform: Why do students complete MOOCs?. Journal of Online Learning and
Teaching, 11(1), p.17.
Wüster, M. and Ebner, M., 2016. How to integrate and automatically issue Open Badges in MOOC platforms. Proceedings
of the European Stakeholder Summit on Experiences and Best Practices in and Around MOOCs (EMOOCS 2016), Graz,
Austria, pp.279-286.
Yuan, L. and Powell, S., 2013. MOOCs and open education: Implications for higher education. CETIS White Papers. [online],
http://publications.cetis.org.uk/2013/667
Zimmerman, B.J., 2002. Becoming a self-regulated learner: An overview. Theory into practice, 41(2), pp.64-70.
828
... It also evaluated gamification effects on several factors like engagement, motivation, participation rate, homework performance, and students' interaction patterns and satisfaction. [5], [6], [7], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23]. Understanding student bevaviour [4], [11], [21], [24], [25]. ...
... Some studies developed tools or adopted open source plugins for LA. For example, [10] developed an algorithm for badges calculation as indicators, and [18], [22] offered a complete logging system for all MOOC variables in iMooX through (iLAP) tool to categorize students according to their activities. ...
Conference Paper
The growing adoption of learning analytics (LA) approaches and data mining (DM) techniques using educational gamification data sets is reflected in increased publications on this topic. However, with different gamified contexts and a variety of LA methods available, no comprehensive review summarized the obtained findings. Therefore, this research aims to identify studies' characteristics, objectives, and methods used in gamification learning analytics (GaLA) research. To identify these, this study comprehensively reviewed the literature of 24 studies selected from an initial pool of 221 search results. The findings show that GaLA methods can be categorized into: visualization, data mining, social network analysis (SNA), statistics, and correlations. In conclusion, GaLA is defined as a data-driven approach using various methods of data analysis and mining techniques in gamified contexts for collecting, analyzing, and reporting data to assess or enhance the gameful experience, understand student behaviour, and improve learning outcomes.
... Awarding digital badges and other gamification elements to stimulate motivation and offering course completion certificates x Bonafini et al. [13] x Labarthe et al. [63] x Borrás-Gené et al. [14] x Sharif and Guilland [93] x Sun and Bin [95] x Cook et al. [26] x Cassidy et al. [18] x Crosslin et al. [29] x Khalil et al. [58] x Petronzi and Hadi [79] x Zheng et al. [120] x Ferguson and Clow [37] x Chang and Wei [21] x Rodriguez et al. [85] x Walji et al. [105] x Lu et al. [68] x Vaibhav and Gupta [104] x Sunar et al. [97] x Perez-Alvarez et al. [78] x Dubbaka and Gopalan [36] x Anutariya and Thongsuntia [4] x Ramesh et al. [82] x Kaveri et al. [55] x Floratos et al. [40] x Romero-Rodriguez et al. [86] x Jung and Lee [54] x Li and Baker [66] x Williams et al. [112] x Phan et al. [80] x Barak et al. [11] x Gregori et al. [44] x Gallego-Romero et al. [41] x Lan and Hew [64] x Alharbi et al. [2] x Balasooriya et al. [9] x Rizzardini and Amado-Salvatierra [83] x Khalil and Ebner [57] x Goldberg et al. [42] x Rebecca Ferguson et al. [39] x Antonaci et al. [3] x Shi and Cristea [94] x (continued on next page) A.A. Ogunyemi et al. the various actions taking place during the activity, broadly classifying them into instructional and platform design. Instructional design process indicators are course-based. ...
... x Bonafini et al. [13] x x Labarthe et al. [63] x Borrás-Gené et al. [14] x Sharif and Guilland [93] x Sun and Bin [95] Cook et al. [26] x Cassidy et al. [18] x Crosslin et al. [29] x x Khalil et al. [58] x x Petronzi and Hadi [79] x Zheng et al. [120] x Ferguson and Clow [37] x Chang and Wei [21] x Rodriguez et al. [85] x x Walji et al. [105] x Lu et al. [68] x x Vaibhav and Gupta [104] x Sunar et al. [97] x Perez-Alvarez et al. [78] x x x x Dubbaka and Gopalan [36] x Anutariya and Thongsuntia [4] x x x ...
Article
Full-text available
Massive open online courses (MOOCs) have paved a new learning path for the 21st-century world. The potential to reach a massive geographically dispersed audience is one of the major advantages of MOOCs. Moreover, they can be offered on a self-paced and self-regulated basis and have become an integral part of lifelong learning, especially in workplaces. However, one persistent problem is the lack of learners’ engagement. A harmonisation of studies providing a holistic view into aggregating indicators for enhancing learners’ engagement in MOOCs is lacking. The coronavirus pandemic has accelerated MOOC adoption, and learners’ engagement in MOOCs has become even more essential for the success of this educational innovation. We examine the existing literature to derive indicators important for enhancing learners’ engagement in MOOC learning environments. Using a systematic approach, 83 empirical studies were examined, and 10 indicators were identified as important considerations for enhancing learners’ engagement while designing MOOCs—from initiatives for individual learners to platform and instructional design perspectives. We also present a table describing these indicators and offer a structured discussion on each one. We believe the results provide guidelines for MOOC designers and instructors, educational policymakers, higher education institutions, and MOOC engagement researchers.
... Recently, academic engagement has attracted the attention of researchers and educators due to its comprehensiveness in describing students' motivation and learning, and also as a strong predictor of student's performance, progress, and success (Lam et al, 2016). In both traditional and online learning, student engagement is a crucial aspect of learning (Khalil et al., 2017). Archambault et al (2009) identified that student engagement can be used as a forecasting element for dropout in schools. ...
... "Collaborative learning" and "Social networks" as participants learn and participate in cooperative environments that promote learning communities within external hypermedia environments, such as social networks (Cruz-Benito et al., 2017). "Learning analytics" to collect and analyze MOOC data for improvement (Khalil et al., 2017). And "Game theory," among others. ...
Article
Currently, educational systems have assumed a relevant role in developing knowledge and strengthening skills in individuals, an aspect that has become a determining factor for the advancement of society. However, these systems present constant challenges, especially influenced by the advance in information and communication technologies, access to the Internet, and mobile devices, which implies transformations in the new paradigms of teaching and learning methodologies. In this scenario, gamification has been one of the strategies used within virtual learning environments such as MOOCs to increase student motivation in the development of courses. In the last decade, interest in this topic has been evidenced. Therefore, this study aims to identify the main research trends in studies on gamification in MOOCs in the last ten years. For this purpose, a bibliometric analysis was carried out using the Scopus database, from which 265 publications were obtained. The main actors (authors and journals) most cited and thematic trends were identified based on the recurrence of keywords. Among the findings, it was identified that researchers are interested in e-Learning, motivation, online learning, serious games, student engagement, badges and rewards, and the use of the Internet as a tool for learning.
... For the last decade, gamification has been evolving in education, as has its influence on student learning [2]. Gamification, which is defined as using "game design elements in non-game contexts" ( [3], p. 9), has been studied for its potential to induce student engagement and motivation in learning and teaching [4]. It is also believed to improve learner participation and interaction and to stimulate learners to increase their knowledge [2,5]. ...
Article
Full-text available
This study explored students' perceptions about using gamified e-quizzes and conventional online quizzes for their class engagement. The participants were 130 female university students in Seoul with various majors. As a quasi-experimental study, this study compared the attitudes of a gamified e-quiz group (n=92) and a conventional online quiz group (n=38) after experiencing their respective quiz in-tervention of either nine gamified e-quizzes or nine online quizzes over a 15-week semester. Each group responded to a survey at the end of the semester. The quan-titative analyses of the surveys indicated that the perception of the two groups did not display statistically significant differences, each displaying positive views to-ward their quiz interventions for emotional, behavioral, and cognitive engage-ment. In addition, the two groups demonstrated neutral to positive attitudes for each quiz intervention for agentic engagement. Among many reported benefits, the students in both groups expressed that the quiz experiences facilitated them to understand the content knowledge and enjoy the assessment activities. These two advantages of the two quiz modes were seen to be related to close student-teacher interaction.
... The technology of online training grounds creates an effective combination with the trend towards the gamification of education [24,31,43,44]. It is quite simple to create competition from the passage of such training grounds and reward the best students. ...
Article
The contemporary smart educational environment uses different information technologies like social networks, virtual laboratories, augmented reality, artificial intelligence, big data, and so on. Each of these technologies has its security and privacy threats profile, but their integration in one system can lead to completely new challenges. The article analyses the technological development of smart educational environments from the point of view of their security and privacy issues. Any technological or legislative security control could be broken as the result of one mistake caused by human factors. People with different levels of competence are interacting every day with each other in educational environments. The risks of personal data leaking or hacking of educational services should be minimized during this interaction. Therefore, not only the key technologies that form the architecture of the educational environment but also the main points of interaction between the users and the education environment should be taken into account in the analysis. The article provides a basic analysis of security and privacy risks for smart education environments. As the result, the analysis identifies key information security technologies development of which is necessary for the sustainable development of a smart educational environment as part of a smart city.
... But of course, the phenomena of drop-outs or other course non-completers does not mean that there is nothing to do to prevent drop-out and foster motivation to stay active. We therefore suggest to design short courses (four-week MOOCs), embed granular certificates and suspense peak narratives [16], gamification [17], foster discussion in the forum [18], or implement "inverse blended learning" (see next chapter; [19,20]). ...
Chapter
Full-text available
This paper discusses the general thesis that massive open online courses (in short MOOC), open educational resources (in short OER) and learning analytics are an impactful trio for future education, especially if combined. The contribution bases upon our practical experience as service providers and researchers in the department “Educational Technology” at Graz University of Technology (TU Graz) in Austria. The team members provide support to lecturers, teachers and researchers in these addressed fields for several years now, for example as host of the MOOC platform iMooX.at, providing only OER since 2015. Within this contribution, we will show, against some doubtful or conflicting opinions and positions, that (a) MOOCs are opening-up education; (b) learning analytics give insights and support learning, not only online learning, if implemented in MOOCs; and (c) that OER has the potential for sustainable resources, innovations and even more impact, especially if implemented in MOOCs.
Article
Full-text available
Massive open online courses (MOOCs) aim at unlimited participation and open access via the web. There are concerns about the actual value of such courses. This is predominantly due to higher dropout rates. According to studies, only 7-13% go on to complete these courses. The high dropout rate in MOOCs is a challenge for education providers. This paper aims to explore reasons for high dropout rates within MOOCs and how they can be minimized. With this in mind, two research questions have been set for this study: 1) Why do MOOC participants not complete their courses? 2) How can the course completion rate be increased? Implementation of the strategies investigated in this paper can increase completion rates in MOOCs. In conclusion, after analyzing the collected data, the final results have shown that gamification increased the completion rate of MOOCs.
Article
Full-text available
This article presents the description of the course in Polish Sign Language conducted as an MOOC (Massive Open Online Course), and, in particular, the possibility of the implementation of inter-cultural aspects into the curriculum of Polish Sign Language as a foreign language with the application of this very formula. The technical structure of the course itself, and also methodological assumptions behind conducting it, are presented in this article as well. The description of the emplotment of the contents contained in it as a method of counteracting the phenomenon of abandoning the course and including intercultural contents into the curriculum of the course. What is discussed as well is the advantages resulting from including intercultural elements into instruction into Polish Sign Language.
Article
MOOC platforms have seen significant membership growth in recent years. MOOCs are leading the education world that has been digitized, remote, and highly competitive, and the competition is intense in the MOOC world. Based on the observation for top-rated MOOCs, this study proposes a research question, “What makes a great MOOC? What makes a hit?” To explore the answers, this study applies a crowdsourcing approach and interprets the semantics of reviews for the top-rated courses on Coursera.org. The paper has multiple steps and findings relevant to MOOC programs at universities worldwide. First, through exploratory analysis of learner reviews and expert judgment, this study identifies two distinct course categories focusing on learners' outcome intent, namely knowledge-seeking MOOCs and skill-seeking MOOCs. Further, this study uses a topical ontology of keywords and sentiment techniques to derive the intent of learners based on their comments. Through sentiment analysis and correlation analysis, it shows that knowledge-seeking MOOCs are driven by the quality of course design and materials. Skill-seeking MOOCs are driven by the instructor and their ability to present lectures and integrate course materials and assignments. This crowdsourcing method obtains the insights from large samples of learners’ reviews without the priming or self-selection biases of open surveys or interviews. The findings demonstrate the effectiveness of leveraging online learner reviews and offer practical implications for what truly “makes a hit” for top-rated MOOCs.
Article
Full-text available
Massive Open Online Courses (MOOCs) require students' commitment and engagement to earn the completion, certified or passing status. This study presents a conceptual Learning Analytics Activity-Motivation framework that looks into increasing students' activity in MOOCs. The proposed framework followed an empirical data analysis from MOOC variables using different case studies. The results of this analysis show that students who are more active within the offered environment are more likely to complete MOOCs. The framework strongly relies on a direct gamified feedback that seeks driving students' inner motivation of competency.
Article
Full-text available
URL: http://repositorio.grial.eu/bitstream/grial/560/1/19_ijee3155ns.pdf The phenomenon of MOOC (Massive Online Open Courses) is increasingly experienced and is giving rise to new scenarios and challenges with several features that are different from previous approaches to online education. In the field of engineering education, Information and Communication Technologies are making continuous innovation in methods of teaching and learning for students. Engineering Education institutions, like the Technical University of Madrid (Spain), are expanding their online offerings and making a more effective use of technologies for learning. This research presents a gamification cooperative MOOC model (gcMOOC) that can be applied in the design of this type of course. Using an explanatory sequential mixed methods design, which integrates the quantitative and qualitative methods, the study investigates the factors that influence motivation, collaboration and learning in gcMOOC. This work also suggests a set of practical recommendations and tools to improve the motivation, learning level and completion rate of participants in MOOC course in Engineering Educational when the gcMOOC model is implemented. The results of this study state that the incorporation of virtual communities and gamification methodologies increase participant learning motivation in engineering MOOC courses. Additionally, these gamification tools aid students to deepen their learning and involve them in the course increasing their motivation and the completion rates in MOOCs.
Conference Paper
Full-text available
Massive Open Online Courses (MOOCs) have been tremendously spreading among Science, Technology, Engineering and Mathematics (STEM) academic disciplines. These MOOCs have served an agglomeration of various learner groups across the world. The leading MOOCs platform in Austria, the iMooX, offers such courses. This paper highlights authors’ experience of applying Learning Analytics to examine the participation of secondary school pupils in one of its courses called “Mechanics in everyday life”. We sighted different patterns and observations and on the contrary of the expected jubilant results of any educational MOOC, we will show, that pupils seemingly decided to consider it not as a real motivating learning route, but rather as an optional homework.
Chapter
The rapid and constant pace of change in technology and the increasing involvement of educational institutions in the massive online open courses (MOOC) movement elicit a large myriad of opportunities and challenges. One of the main issues is the reported high dropout rate. In this sense, gamification strategies have been proposed as a complement to existing learning approaches providing a powerful and motivational learning experience to students. Examples of gamification strategies for MOOC environments include rewards for learning activities, applying levels and leader-boards to encourage progress and competition, and badges for participation in forums. The aim of this study is to contribute to the analysis of motivational factors to provide improved learning experiences for cloud-based learning services. This paper presents lessons learned from the MOOC course “Authoring tools for e-learning courses”. 1678 participants experienced a mix of gamification strategies: Badges – Leaderboard forums; Students Classifier League and Reward strategy. Findings revealed the reward strategy as the most effective one, and indicated increased motivation to complete the assigned learning activities.
Conference Paper
To minimize barriers to entry, massive open online course (MOOC) providers collect minimal demographic information about users. In isolation, these data are insufficient to address important questions about socioeconomic status (SES) and MOOC enrollment and performance. We demonstrate the use of third-party datasets to enrich demographic portraits of MOOC students and answer fundamental questions about SES and MOOC enrollment. We derive demographic information from registrants' geographic location by matching self-reported mailing addresses with data available from Esri at the census block group level and the American Community Survey at the zip code level. We then use these data to compare neighborhood income and parental education for US registrants in HarvardX courses to the US population as a whole. Overall, HarvardX registrants tend to reside in more affluent neighborhoods. Registrants on average live in neighborhoods with median incomes approximately. 45 standard deviations higher than the US population. Higher levels of parental education are also associated with a higher likelihood of registration.
Article
Massive open online courses (MOOCs) require individual learners to be able to self-regulate their learning, determining when and how they engage. However, MOOCs attract a diverse range of learners, each with different motivations and prior experience. This study investigates the self-regulated learning (SRL) learners apply in a MOOC, in particular focusing on how learners' motivations for taking a MOOC influence their behaviour and employment of SRL strategies. Following a quantitative investigation of the learning behaviours of 788 MOOC participants, follow-up interviews were conducted with 32 learners. The study compares the narrative descriptions of behaviour between learners with self-reported high and low SRL scores. Substantial differences were detected between the self-described learning behaviours of these two groups in five of the sub-processes examined. Learners' motivations and goals were found to shape how they conceptualised the purpose of the MOOC, which in turn affected their perception of the learning process.
Article
Massive open online courses (MOOCs) have developed rapidly and become tremendously popular because of their plentiful gamification designs, such as reputation points, rewards, and goal setting. Although previous studies have mentioned a broad range of gamification designs that might influence MOOC learner engagement, most gamified MOOCs fail to meet learning objectives because of a lack of research regarding suitable game design, as well as poor rationale for or design of gamification mechanics. This study aims to explore and identify engaging gamification mechanics for MOOC learners. We conducted a focus group interview with 25 MOOC frequent users to identify 40 gamification mechanics. This study then determined the relative engagingness of these gamification mechanics by administering an online survey to 5,020 MOOC learners. The results indicated that the 10 most engaging gamification mechanics accounted for more than 50% of the engagingness. The mechanics of the Where's Wally game is extremely engaging for MOOC learners; however, they it is not been demonstrated in previous relevant studies. Finally, we discuss the top five engaging gamification mechanics and their implications.