Conference PaperPDF Available

How can Gamification Improve MOOC Student Engagement?

Authors:

Abstract and Figures

Massive Open Online Courses (MOOCs) require students' motivation either intrinsically or extrinsically to complete any of its courses. Even though MOOCs enjoy great popularity and bring many benefits to the educational community, some concerns arise with MOOC advancement. In fact, MOOCs are affected by low completion rate and face issues with respect to interactivity and student engagement along MOOC duration, which may convert student excitement to boredom and then drop out at any stage. A key result of research in the past couple of years has proved that students' engagement in MOOCs is strongly related to their activities online. These activities are related to the interaction between student and logging in the MOOC, reading and writing in the MOOC discussion forum, watching videos and doing quizzes. In this research paper, we present our research in deploying a gamification mechanic in MOOCs to increase student engagement. The gamification approach relies on weekly feedback to drive student intrinsic and extrinsic motivation. Following learning analytics on students' data from a MOOC offered in 2014, 2015, and 2016, the outcome of this approach showed an obvious increase in students' activity and engagement in discussion forums, login frequency and quiz trials. The active students' cohort allotment has increased in comparison with previous versions of the same MOOC as well as the completion rate has incremented up to 26% of the total number of participants.
Content may be subject to copyright.
How can Gamification Improve MOOC Student Engagement?
Mohammad Khalil1, Martin Ebner2 and Wilfried Admiraal3
1Delft University of Technology, The Netherlands
2Graz University of Technology, Austria
3Leiden University, The Netherlands
m.f.d.khalil@tudelft.nl
martin.ebner@tugraz.at
w.f.admiraal@iclon.leidenuniv.nl
Abstract: Massive Open Online Courses (MOOCs) require students’ motivation either intrinsically or extrinsically to complete
any of its courses. Even though MOOCs enjoy great popularity and bring many benefits to the educational community, some
concerns arise with MOOC advancement. In fact, MOOCs are affected by low completion rate and face issues with respect
to interactivity and student engagement along MOOC duration, which may convert student excitement to boredom and then
drop out at any stage. A key result of research in the past couple of years has proved that students’ engagement in MOOCs
is strongly related to their activities online. These activities are related to the interaction between student and logging in the
MOOC, reading and writing in the MOOC discussion forum, watching videos and doing quizzes. In this research paper, we
present our research in deploying a gamification mechanic in MOOCs to increase student engagement. The gamification
approach relies on weekly feedback to drive student intrinsic and extrinsic motivation. Following learning analytics on
students’ data from a MOOC offered in 2014, 2015, and 2016, the outcome of this approach showed an obvious increase in
students’ activity and engagement in discussion forums, login frequency and quiz trials. The active students’ cohort allotment
has increased in comparison with previous versions of the same MOOC as well as the completion rate has incremented up
to 26% of the total number of participants.
Keywords: gamification, massive open online courses (MOOCs), learning analytics, motivation, retention, dropout
1. Introduction
Massive Open Online Courses (MOOCs) have gained a lot of attention and impetus in the last five years. They
are distinguished among other learning environments by being open for everyone, easy to enrol, and having a
heterogeneous community. MOOCs were first coined by George Siemens and David Cormier in 2008 when they
described a four-month course on connectivism theory (Cormier & Siemens, 2010). Years later, Sebastian Thrun,
a Stanford University professor, offered an online course called “Introduction to Artificial Intelligence” that
received a wide publicity of over 160,000 student registrations from all over the world (Yuan & Powell, 2013).
Since then, MOOC becomes an environment that was bet on bringing revolution to higher education as well as
to elementary education (Khalil & Ebner, 2015) based on factors of their popularity and massiveness of
enrolments (Martin, 2012).
Given that MOOCs are online systems with minimal direct interaction between students and tutors, learners are
required to self-regulate their learning. In Self-Regulated Learning (SRL) in classrooms, learners are asked to
adjust their attitude based on the educational context where learning happens (Zimmerman, 2002). While in
MOOCs, learners have to identify what type and how much of activities they need to engage with (Littlejohn et
al., 2016).
In both traditional and online learning, student engagement is a crucial aspect for learning (Lam et al., 2012).
Carini, Kuh, and Klein (2006) found that student engagement were linked positively with learning outcomes like
quiz performance and critical thinking. On the other hand, Archambault et al (2009) identified that student
engagement can be used as a forecast element for dropout in schools. Handelsman (2005) used engagement as
a short-term indictor to distinguish between students in higher education settings to either performance-goal
oriented or learning-goal oriented students. Hew (2016) mentioned that student engagement might be well
suited for social cognitive behaviour and academic achievement prediction.
In respect to MOOCs, Milligan, Margaryan, and Littlejohn (2013) stated that motivation is an important
benefactor to student engagement as well as an indicator of student engagement.
1 The author’s research is supported by the Leiden-Delft-Erasmus Centre for Education and Learning
819
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
Despite the fact that MOOCs are rapidly developing, they are affected by low completion rate and weak student-
system and student-teacher interaction (Balakrishnan & Coetzee, 2013; Khalil & Ebner, 2016; Littlejohn et al.,
2016). In one of the recent studies, this has been linked to the poor engaging design of MOOCs (Chang & Wei,
2016). Researchers have proposed different approaches to improve student engagement through concentrating
at motivation from both sides of one coin, intrinsic and extrinsic motivations. For instance, one study linked the
intrinsic motivation of MOOC learners to the inner intention to learn, compete and satisfy their curiosity (Wang
& Baker, 2015). Other research studies tried to increase student engagement through drawing focus on the
extrinsic side of motivation by introducing open badges and certificates to attract online learners (Wüster &
Ebner, 2016). Although classical reward certificates were promising when they first were introduced, learners
think that completing a MOOC to have completion certificates is not sufficient enough to finish a course (Hew,
2016). As a result, various new strategies that targeted to ignite the extrinsic motivation of learners were
proposed; gamification as one of these strategies. The main aspect of using gamification in educational settings
were mentioned in different studies as a tool, strategy, and methodology to entertain students (Gené, Núñez &
Blanco , 2014; Chang & Wei, 2016; Wüster & Ebner, 2016).
Motivated by the increased impetus of gamification in education (Chang & Wei, 2016) and the youth generation
of online learners (Hansen & Reich, 2015), in this publication we present our experiment on using an early-
developed gamification framework (Khalil & Ebner, 2017) in a massive open online course. The goal of this
research is to study the use of a gamified strategy that looks at increasing students’ incentive to study MOOCs
yet sparking their curiosity taking into account both intrinsic and extrinsic motivation factors. The intrinsic
motivation will be driven by curiosity-guided strategy and the extrinsic motivation by a rewarding strategy
influenced by the theory study of Ryan and Deci (2000). The study takes place in one of the offered MOOCs from
the Austrian MOOC provider, iMooX (http://www.imoox.at). The main research question of this study is to
examine whether gamification can increase student engagement and activities and henceforth improve the
retention rate of MOOC learners. To carry out this research, we used a local developed learning analytics tool to
support us in collecting and analysing the data from the MOOC.
The remainder of the paper is organized as follows: Section 2 is the related work of gamification use in MOOCs.
Section 3 gives a short background of the tested MOOC and the iMooX platform. Section 4 lists the followed
gamification approach and the deployment phase. Section 5 presents the results and discussion of this study.
Finally, section 6 draws the conclusions.
2. Related work
This section enriches the paper by reviewing some of the related topics of gamification and engagement in the
context of MOOCs. Through our examination in different academic libraries, the return results showed some
relevant studies that surveyed gamification in MOOCs and others that aimed at improving students’ engagement
through the use of gamification mechanics. Gamification is defined as “the use of game design elements in non-
game contexts” (Deterding et al., 2011). Given the issues of dropout and motivation in MOOCs, gamification
mechanics were considered promising to alleviate such dilemmas. Practitioners linked gamification strategies in
non-game settings to add fun and increase engagement and enjoyment of a service or a product (Hsu, Chang, &
Lee, 2013). Gamification incorporates entertainment on its surface, but may function as playground for
competition and collaboration. In respect to MOOCs, the popular MOOC providers like Coursera and edX became
aware of the gamification impact on learners and introduced the use of rewards and badges as strategies to
attract the students. The language-learning platform Duolingo has gone further with gamifying the platform.
Duolingo uses skill points, badges, certificates, and a levelling system as incentive to encourage students to study
and increase the competition level among them. Another popular MOOC platform, Khan Academy, employed
gamification in various ways. Khan Academy has used Knowledge maps, badges and progress indicators in order
to enhance students’ motivation (Morrison & DiSalvo, 2014).
Through our search in the literature and to our knowledge, the biggest survey study on gamification in MOOCs,
was done by Chang and Wei (2016). Their study scrutinized over 40 gamification techniques and their impact on
student activities and motivation in MOOCs. Chang and Wei (2016) stated that not every gamification technique
could enthral online learners. They found out that virtual goods, redeemable points, team leader boards,
where’s Wally game, and trophies are the top five most engaging gamification mechanics in MOOCs. The authors
concluded the critical role of gamification in MOOCs as factors that push learners to spend a higher average
time, enhance self and content interaction.
820
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
An empirical study by Vaibhav and Gupta (2014) examined the use of gamification in a MOOC through an A/B
testing planned task. The researchers found out that the gamified quiz attracted a larger number of students
than those without gamification in regards to the number of quizzes student submitted. Additionally, Vaibhav
and Gupta (2014) realized that the success rate of the quizzes were higher for the cohort who were supported
with gamification and therefore recorded a slight increased retention in comparison to the control group.
Gené, Núñez, and Blanco (2014) suggested applying gamification in MOOCs by replicating an experiment from
Moodle learning system. The researchers looked at promoting cooperation and motivation in MOOCs and
lowering dropout rate by using different gamification strategies such as ranking rating, voluntary activities,
course progress and certificates. Two years later, the authors (Gené, Núñez, & Blanco, 2016) published a
qualitative interview and survey study stating that gamification tools have deepen student learning and
increased student motivation and engagement within the MOOC content.
Another practical experience by Morales et al (2016) used three gamification strategies (leader boards, badges,
and rewards) to increase student participation in their MOOC. The gamification approaches were deployed in
discussion forum, assignments section and completed activities. Results of this study showed that the rewarding
system was the most successful one based on a qualitative post survey while leader boards in the discussion
forum were evaluated the lowest engaging strategy. Nevertheless, the authors reported that students faced
challenges in regards to the dynamics of understanding each gamification strategy.
Although the previous studies shed light on gamification in MOOCs, the current literature involves little empirical
validation. Our study seeks to bridge, to some extent, this gap with additional evidences on the significant use
of gamification in MOOCs and its capabilities to improve student engagement.
3. The MOOC platform and the studied MOOC
3.1 The MOOC platform
As previously mentioned, this study is carried out cooperating with the Austrian MOOC platform (iMooX). iMooX
is an online stage and the first Austrian MOOC platform founded in 2013 by Graz University of Technology and
University of Graz. Since the first year launch of iMooX, over 5,000 registrations have been recorded in the
database. Most of the courses are offered in German language and target not only academic holders but also
elderly as well as school children. The pedagogical approaches of iMooX stand on the cognitive-behaviourist and
social-constructive pedagogies through providing a rich interactive discussion forum, a convenient structure of
information exchange, and a stimulus demonstration of active online learning videos. All the quizzes in iMooX
follow the multiple-choice questions system by which every quiz is setup to give students the ability to try each
weekly quiz more than once. The questions of each quiz trial are randomized for every attempt. The main reason
behind this is to less stress students so that each student behaves in a more comfortable manner by picking the
highest grade out of student tries, and to support the self-assessment learning guidance in MOOCs. When the
students successfully finish the quizzes and fill out an final evaluation form at the end of a MOOC, they are
rewarded with certificates completely for free.
3.2 MOOC studied
We implemented our research study on one iMooX MOOC called Free Online Learning course or as it is named
originally in German language Gratis Online Lernen (Ebner, Schön & Käfmüller, 2015) that was offered in 2016.
To ease denoting the course name in the context of the paper, we abbreviate the MOOC name to GOL in
corresponds to the German name. The course was offered in 2014, 2015, and 2016 respectively and was
instructed by Graz University of Technology. The structure of the MOOC was organized to be presented over a
time-period of eight weeks with a set load of 2 hours/week. The GOL MOOC topic focused on the general topic
of learning with the Internet (online learning) as well as on informing the public on the significant rising
momentum of Open Educational Resources (OER), as well as the right to access, share, and adapt them.
Following similar MOOCs, GOL course was supported with a set of affluent short videos, multiple-choice quizzes,
a discussion forum, and recommended articles that are available to download. The MOOC was open to everyone
without a need for any prerequisite knowledge. Furthermore the MOOC is following the didactical concept of
Inverse Blended Learning, first introduced by Schön & Ebner in 20142. The Inverse Blended Learning concept
2 https://elearningblog.tugraz.at/archives/7412
821
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
focuses on meeting up the online learners on a round table, handing them additional printed materials as well
as sharing thoughts and results in the online MOOC discussion forum. In comparison to other iMooX MOOCs,
GOL attracted that largest number of participants in its first version with around 1,000 students.
4. The gamification approach
The research at hand followed our early-published concept Activity-Motivation Framework (see Figure 1; Khalil
& Ebner, 2017). The main idea behind the conceptual framework was to increase retention rate through
enhancing student engagement in MOOCs. To do so, the framework relied on a weekly-gamified feedback to
drive student motivation. The scheme of the Activity-Motivation approach corresponds to the iMooX platform
potential of offering various MOOC variable data: 1) quiz attempts, 2) watching learning videos, 3) reading in
discussion forum, 4) posting in discussion forum, and 5) logging in MOOCs. In fact, the framework was set after
testing the hypothesis that says students who complete MOOCs are more likely to perform extra activities (like
watching videos, more involvement in forums..etc.) than those who do not (Khalil & Ebner, 2017), or in other
way around, the more students participate and engage the more likely they finish a course. Based on that, the
Activity-Motivation framework was developed in reliance on MOOC activities and the concept of motivating
students to increase the general engagement hoping for an increased completion rate.
In short, figure 1 displays a battery gamification element that is weekly charged based on four-dimensional
MOOC variables (the number of logins, the number of videos watched and rewatched, the number of quizzed
completed and the level of participation in the discussion forum). Our reason for choosing the battery icon
returns to our thought of what happens to a battery is somehow similar to what a student does in MOOC
platforms. We aimed at charging students with motivation and incentive by sparking their curiosity (Ryan & Deci,
2000). To attain the curiosity strategy, the algorithm was concealed on purpose and students never know how
the battery symbol is charged/filled. The intention was to stimulate student intrinsic motivation to do more
activities and actions in the MOOC by driving them to guess how the gamification approach works.
Each of the four dimensions contributes with a portion to fill the battery cumulatively. If a student logs in the
course, watches a video, makes activities in the discussion forum and does a quiz, the battery will be fully
charged. By the end of each week, the students are rewarded with the battery icon that only illustrates their
prior activities and a motivational statement. Any shortage of one or a combination of these MOOC variables,
the battery will be less charged based on student MOOC actions.
Figure 1: The MOOC activity-motivation framework employs MOOC variables in a gamification approach (Khalil
& Ebner, 2017)
4.1 Deployment of the gamification approach
In this part, we show how the gamification approach was deployed in the GOL MOOC. The process of the
gamification approach implementation was done manually since we were looking for evaluation results at the
first stage. A second stage of automatic implementation can be systemized on upcoming MOOCs if the results
are promising to the iMooX higher management. Our first step was to design the battery gamification element
that should be attached to each profile. We chose an open source software called Inkscape
(http://www.inkscape.org, last accessed: April 2017). Through our design, we aimed at having symbols that
822
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
supports the information-oriented aspect in which they reflect an easy understanding of visual elements. The
design of the symbols followed the recommendation list for an effective visual communication of graphical user
interface by Suzanne (1995). A key principle of Suzanne’s (1995) list is to have a clear and strong visual identity
of designed symbols. As a result, our design was simple and clear of a 2-D layout with a fine mix of light colours.
Figure 2 shows five categories of the battery gamification symbol by which each of them represents a single
status.
Figure 2: Five categories of the battery gamification element in iMooX MOOC platform
Each symbol in figure 2 was displayed on the top left side of each user homepage of the GOL-2016 MOOC and a
brief news feed was posted on the MOOC’s homepage advertising about the gamification symbols.
Every battery symbol represents the recorded MOOC activity of the user based on the learner’s previous week’s
interaction. For example, by the end of the first MOOC week, we show the student activity progress of that week
on the first day of the second week and so forth except for the period of the first week where the system was
automated to show 0% battery status since all students started with no activity.
The MOOC activities that were planned to be logged in our implementation were: a) logging into the MOOC
homepage, b) doing a quiz, c) posting/commenting at least once or reading two threads in the MOOC’s
discussion forum, and d) watching a video. Nevertheless, we faced a major technical problem with logging user
video activities. The problem was detected just a couple of days before the MOOC launch date
(10th.October.2016) when we discovered that the MOOC videos were embedded using IFRAME instead of
OBJECT on the iMooX platform. Therefore, our simple and quick redress was that by assuming a user logs in, we
show the “50%” battery symbol. Given this issue, we excluded the video activity and the quarterly-charged
battery symbol.
The final trailed approach for showing the battery symbols, thus, was established by the following guidelines
based on every week activities:
Login activity: When a student logs into the MOOC, the activity will reflect relatively on the gamification
element (battery). The first segment of the battery will be 50% charged. Several logins will not increase the
charged portion.
Quiz activity: The battery symbol will be filled with one extra portion (25%) when a student takes a quiz. As
described before, the iMooX MOOC-platform allows each student to try the weekly quiz up to five times.
However, just one trial would be enough to indicate that the student is active not just as a lurker. Several
attempts will not increase the battery’s charged portion.
Discussion forum activity: If a student is engaged in the forums either by writing at least one post or reading
threads twice, then the battery-charging portion will add another 25%.
To support a multiple form of representation for gamification symbols, we took into consideration showing a
percentage numeral that denotes progress and arranged a tooltip for each symbol so that students can recognize
what these symbols imply (see table 1).
Table 1: The gamification battery symbols with their tooltips
Battery Symbol
Tooltip
No activity last week –
we are looking forward
to seeing you again this week!
823
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
Battery Symbol
Tooltip
Your activity last week was 50%. Good!
Increase your activities to score better!
Your activity last week
was
75%. Great! Keep it
up!
Your activity in the previous week
was 100%.
Congratulations! Your commitment is
excellent. Keep it up!
5. Results and discussion
The further step was to evaluate the efficacy of the gamification approach and to tackle our research question.
The total number of enrolees of the GOL-2016 MOOC was not as much as the previous versions GOL-2014 and
GOL-2015 (Table 2). A brief explanation for the lower involvement might belong to the iMooX huge
advertisement back in 2014 when iMooX was first launched. That is, Gratis Online Lernen 2014 MOOC was one
of the few offered courses at that time.
Table 2: Total number of enrolments of the Gratis Online Lernen MOOC
Course
GOL-2014
GOL-2015
GOL-2016
Next, we collected through our manual implementation the total number of students and their engagement
level based on the gamification statuses as shown in figure 3.
Figure 3: Summary of the battery results (left to right 50%, 75%, 100%) of active students in GOL-2016 MOOC.
non-active students or the 0% battery are excluded
The figure shows an elevation of the 75% battery status in the number of students across all the MOOC weeks.
By investigating student behaviour in the discussion forums and the submitted quizzes, students who are
824
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
committed to complete the MOOC have intended to log in at least once a week and do the weekly quiz. That is,
students were more involved in doing quizzes than being involved in the discussion forums which is an expected
behaviour in order to have a certificate.
Also, the figure shows that there is a slight decrement of the 50% battery status from week1 to week8 wavering
from (N=19) in the first week to (N=7) in the eighth week. This was interesting since active students tried to push
more efforts to score higher than the 50% or the 0% battery status. For instance, the number of students who
were active has increased by 15.5% than the status of week1. Week4 showed the minimum score of 50% status.
On the other hand, the full activity status 100% was at its highest in the second week with around (N=38)
students. Our explanation of this behaviour can be interpreted by the fact that students were pushing more
efforts to improve their battery status influenced by the motivational triggers. Likewise, the stability of
participation in quizzes is clear across all the weeks. It is worth pointing out that some students might do quizzes
in different weeks. This can be complicated to track. Nevertheless, our tracking records were based on every
week’s quizzes, logins and forum activities.
To check the validity of how active the students were within the MOOC variables, we inspected the quiz activities
across every week of GOL-2014, GOL-2015 and GOL-2016 using the iMooX learning analytics tool. Figure 4
depicts bar plots for the number of students who did at least a one quiz in every MOOC week. The x-axis
represents the MOOC weeks; the y-axis represents the number of students (identical students not repetitive).
We preferred to show a plot for each MOOC since there is a substantial difference of enrolments among every
MOOC.
Figure 4: The number of students who did one quiz of each described week (a) Top-left: Gratis Online Lernen
2014 MOOC (b) Top-right: Gratis Online Lernen 2015 MOOC (c) Bottom: Gratis Online Lernen 2016
MOOC
In figure 4a, we can see that the number of students who did the GOL-2014 quizzes dropped across the weeks
except for a slight increase in the last week. Student engagement usually reveals a high attrition scale in activities
in the first two weeks (Balakrishnan & Coetzee, 2013). Likewise in figure 4b, the plot shows nearly the same
direction of GOL-2014 behaviour by which students were doing quizzes of GOL-2015 actively in the first two
825
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
weeks and then dropped till the last week of the MOOC. On the contrary, figure 4c shows a very interesting
student engagement behaviour in the GOL-2016 MOOC where the gamification approach was applied. The
number of students who did the second week quiz nearly doubled when compared with the first week. We
believe the reason behind is because every student was given a 0% battery status when the MOOC started. Given
that the gamification element mechanism was hidden as a type of curiosity-driven behaviour, i.e. intrinsic
motivation, students tried to figure out how to progress better the week after. An alternative explanation can
be related to the extrinsic triggers as well in which students pushed efforts in order to gain the highest status of
the battery gamification element.
The figure further shows that the number of students in week3 till week8 represents a stable participation rate
in weekly quizzes in comparison to GOL-2014 and GOL-2015 participation.
Before moving to the completion ratio of GOL MOOCs, we wanted to examine the first part of the research
question that hypothesise if our gamification approach has increased student engagement. With this in mind,
we categorized students to a) registrants and those who register in the MOOC; b) active students and those are
the students who make at least one quiz or post in the discussion forums; and c) certified who successfully
complete a MOOC and are granted a certificate.
With the available filtered data from the learning analytics tool, we categorized the students of GOL-2014 and
GOL-2016. The cohort distribution for these two MOOCs is shown in table 3. The offered MOOC in 2014 recorded
(N=1,003) registrations, GOL-2015 (N=476), while GOL-2016 had a total number of (N=284) registered
participants. Given the active student definition, the grouping of participants resulted in having (N=475,
P=47.3%) in GOL-2014, (N=188, P=39.49%) in GOL-2105, and (N=209, P=73.5%) in GOL-2016 of active students.
Surprisingly, the ratio of active student in GOL-2016 has increased (P=55.3%), by which denotes a significant
difference in comparison to GOL-2016 MOOC. This demonstrates that students were more digitally engaged and
did additional activities to have a higher score with the gamification symbol.
On the other side, the certification ratio results were also promising for the gamification deployed MOOC. GOL-
2016 had a (N=74, P=26.05%) of certified students which is relatively higher than the previous offered MOOCs.
GOL-2014 and GOL-2015 MOOCs have lower certification ratio equalled to (P=17.54% and P=19.74%)
respectively. Although the certification rate of the GOL-2016 MOOC was not of that big difference with the other
MOOCs, the students in GOL-2016 were more digitally engaged in comparison. Besides the quiz activity in the
GOL-2016 MOOC, student other actions were actively present in discussion forums and login frequency.
Table 3: Overview of Gratis Online Lernen MOOCs cohort distribution
Course
Registrants
Active students
Certified
GOL-2014
1,003
475 (47.35%)
176 (17.54%)
GOL-2015
476
188 (39.49%)
94 (19.74%)
GOL-2016
284
209 (73.59%)
74 (26.05%)
6. Conclusion
The use of gamified mechanics in non-game contexts has become popular recently (Deterding et al., 2011; Hsu
et al., 2013). Gamification is looked at with potential to leverage student engagement and motivation in
educational contexts. Thereupon the gamification capabilities and the given MOOC dilemmas, this paper
presented our experiment and the results of gamification deployment in a massive open online course that
focused on two main issues of MOOCs: participation and engagement.
We used a simple gamified approach that supported a weekly feedback. The deployment results show that the
MOOC by which gamification was applied to, has gained an increased level of students attention and
engagement. The gamification approach of this experiment was instrumental in increasing the student
interactions with MOOC variables on one side and increasing the student motivation to complete quizzes on the
other side, which leads to students wanting to complete the rest of the quizzes. In addition, the analysis
identified a stable participation rate in weekly quizzes in comparison to previous tested MOOCs. The outcome
of this research study also confirmed a slight increase in the certification ratio in MOOC when gamification was
deployed.
826
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
We believe our approach was distinct than previous research studies through targeting intrinsic motivation and
extrinsic motivation factors together. The intrinsic motivation was driven by obtaining student curiosity while
the extrinsic motivation was driven by the battery gamification symbols progress. However, the explanation of
this research study and how students react to both motivation triggers are only based on numeric and learning
analytics. A future direction by doing a post survey can further explain how student perceived our gamification
approach.
Finally, gamification may carry tremendous potential behind. That is, it ties strongly with student motivation and
therefore increases the general completion rate. The big MOOC players like edX, Khan Academy, and Coursera
have become aware of the importance of gamification designs and the future will carry new techniques that will
verify their impact on the success of MOOCs.
References
Archambault, I., Janosz, M., Fallu, J.S. and Pagani, L.S., 2009. Student engagement and its relationship with early high
school dropout. Journal of adolescence, 32(3), pp.651-670.
Balakrishnan, G. and Coetzee, D., 2013. Predicting student retention in massive open online courses using hidden markov
models. Electrical Engineering and Computer Sciences University of California at Berkeley.
Carini, R.M., Kuh, G.D. and Klein, S.P., 2006. Student engagement and student learning: Testing the linkages. Research in
higher education, 47(1), pp.1-32.
Chang, J.W. and Wei, H.Y., 2016. Exploring Engaging Gamification Mechanics in Massive Online Open Courses. Educational
Technology & Society, 19(2), pp.177-203.
Cormier, D. and Siemens, G., 2010. The open course: Through the open door--open courses as research, learning, and
engagement. Educause Review, 45(4), p.30.
Deterding, S., Dixon, D., Khaled, R. and Nacke, L., 2011. From game design elements to gamefulness: defining gamification.
In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, (pp.
9-15). ACM.
Ebner, M., Schön, S., Käfmüller, K., 2015. Inverse Blended Learning bei „Gratis Online Lernen“ – über den Versuch, einen
Online-Kurs für viele in die Lebenswelt von EinsteigerInnen zu integrieren. In Digitale Medien und Interdisziplinarität.
Nistor, N. & Schirlitz, S. (Hrsg). Waxmann, Medien in der Wissenschaft Bd 68. pp. 197-206.
Gené, O.B., Núñez, M.M. and Blanco, Á.F., 2014. Gamification in MOOC: challenges, opportunities and proposals for
advancing MOOC model. In Proceedings of the Second International Conference on Technological Ecosystems for
Enhancing Multiculturality (pp. 215-220). ACM.
Gené, O.B., Núñez, M.M. and Blanco, Á.F., 2016. New Challenges for the motivation and learning in engineering education
using gamification in MOOC. International Journal of Engineering Education, 32(1), pp.501-512.
Handelsman, M.M., Briggs, W.L., Sullivan, N. and Towler, A., 2005. A measure of college student course engagement. The
Journal of Educational Research, 98(3), pp.184-192.
Hansen, J.D. and Reich, J., 2015. Socioeconomic status and MOOC enrollment: enriching demographic information with
external datasets. In Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (pp. 59-
63). ACM.
Hew, K.F., 2016. Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS.
British Journal of Educational Technology, 47(2), pp.320-341.
Hsu, S.H., Chang, J.W. and Lee, C.C., 2013. Designing attractive gamification features for collaborative storytelling websites.
Cyberpsychology, Behavior, and Social Networking, 16(6), pp.428-435.
Khalil, M. and Ebner, M., 2015. A STEM MOOC for school children—What does learning analytics tell us?. In the
International Conference on Interactive Collaborative Learning 2015 (ICL) (pp. 1217-1221). IEEE.
Khalil, M. and Ebner, M., 2017. Driving Student Motivation in MOOCs through a Conceptual Activity-Motivation
Framework. Zeitschrift für Hochschulentwicklung, 12(1), pp.101-122.
Littlejohn, A., Hood, N., Milligan, C. and Mustain, P., 2016. Learning in MOOCs: Motivations and self-regulated learning in
MOOCs. The Internet and Higher Education, 29, pp.40-48.
Martin, F.G., 2012. Will massive open online courses change how we teach?. Communications of the ACM, 55(8), pp.26-28.
Milligan, C., Margaryan, A. and Littlejohn, A., 2013. Patterns of engagement in massive open online courses. Journal of
Online Learning with Technology, 9(2), pp.149-159.
Morales M., Amado-Salvatierra H.R., Hernández R., Pirker J., Gütl C. 2016. A Practical Experience on the Use of
Gamification in MOOC Courses as a Strategy to Increase Motivation. In: Uden L., Liberona D., Feldmann B. (eds)
Learning Technology for Education in Cloud – The Changing Face of Education. LTEC 2016. Communications in
Computer and Information Science, vol 620. Springer, Cham.
Morrison, B.B. and DiSalvo, B., 2014. Khan academy gamifies computer science. In Proceedings of the 45th ACM technical
symposium on Computer science education (pp. 39-44). ACM.
Ryan, R.M. and Deci, E.L., 2000. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary
educational psychology, 25(1), pp.54-67.
827
Mohammad Khalil, Martin Ebner and Wilfried Admiraal
Suzanne, M., 1996. Effective Visual Communication for Graphical User Interfaces. Home: WPI EDU. [online],
https://web.cs.wpi.edu/~matt/courses/cs563/talks/smartin/int_design.html
Vaibhav, A. and Gupta, P., 2014. Gamification of MOOCs for increasing user engagement. In 2014 IEEE International
Conference on MOOC, Innovation and Technology in Education (MITE), (pp. 290-295). IEEE.
Wang, Y. and Baker, R., 2015. Content or platform: Why do students complete MOOCs?. Journal of Online Learning and
Teaching, 11(1), p.17.
Wüster, M. and Ebner, M., 2016. How to integrate and automatically issue Open Badges in MOOC platforms. Proceedings
of the European Stakeholder Summit on Experiences and Best Practices in and Around MOOCs (EMOOCS 2016), Graz,
Austria, pp.279-286.
Yuan, L. and Powell, S., 2013. MOOCs and open education: Implications for higher education. CETIS White Papers. [online],
http://publications.cetis.org.uk/2013/667
Zimmerman, B.J., 2002. Becoming a self-regulated learner: An overview. Theory into practice, 41(2), pp.64-70.
828
... Awareness of using a theory in gamification research started to appear in 2017. Khalil et al. (2017), developed their gamification learning environment based on Activity-Motivation Framework that is based on intrinsic and extrinsic motivations by Ryan and Deci (2000). Although Hung (2017) did not mention the theory applied in his educational gamification, his work adopted a framework called user-centered design framework for meaningful gamification introduced by Nicholson (2012) from the information system domain where four theories: organismic integration theory (OIT), situational relevance and situated motivational affordance, universal design for learning, player-generated content were combined and adapted. ...
... Out of eight studies, only two studies mentioned the theory for their framework (Khalil et al., 2017;Hammerschall, 2019). Khalil et al., (2017)'s ...
... Out of eight studies, only two studies mentioned the theory for their framework (Khalil et al., 2017;Hammerschall, 2019). Khalil et al., (2017)'s ...
Chapter
Full-text available
Final year project or FYP is a form of project-based learning commonly implemented in undergraduate programmes for the students to manifest the knowledge, skills, and learning experiences accumulated through their academic studies. This chapter elaborates how gamification was implemented through a guided constructive alignment in the undergraduate FYPs of the Bachelor of Design (Digital Games) with the honours programme.
... The fulfillment levels of the three psychological needs will affect people's motivation status, help transfer extrinsic motivation to intrinsic motivation, and increase engagement. Instructional designers and educators often apply these theories as the designing principles to explain and support their gamification design thinking in online contexts (Gené et al., 2016;Khalil et al., 2017;Ortega-Arranz et al., 2017Romero-Rodriguez et al., 2019;Tsay et al., 2018). ...
Article
Full-text available
As online learning and teaching are becoming an educational trend, online students’ engagement will directly impact the learning and teaching effects and outcomes. A scientific application of gamification in online learning, teaching, and online course design will improve online learners’ learning experience and help build a better virtual learning context for online learners worldwide. This study focuses on how gamification can engage online students from skills, emotional, participation, and performance perspectives. The gamification components investigated in this study are Points (also called Anar Seeds in the context of this study) and Badges, the most widely used components in gamification design in education, and online instructors have primarily used them to motivate students’ learning and acknowledge their achievements by timely rewarding specific learning behavior or performance. A mixed method has been applied to explore further the relationships between gamification components and online students’ engagement and how online students perceive the impacts of gamification on their online learning experience. The findings show a significant correlation between Points and online students’ skills, emotional, participation, and performance engagement, while it exists only between Badges and online students’ participation engagement. The findings also reveal mixed perceptions of online students towards gamification. Study limitations regarding lack of age criterion, limited measurement indicators, and oversimplification of survey responses have also been discussed. It is suggested that future research can be conducted from either the instructors’ or the gamification designers’ perspectives to determine any other factors that might contribute to the implementation of gamification in the online learning context.
... Moreover, the incorporation of gamification elements in educational contexts can increase the attractiveness of learning activities (Fredricks 2014;Bower et al. 2017). In this perspective, gamification and serious games have been used to increase the involvement of students (Khalil et al. 2017), improve the attitude towards a lecture, or make a learning activity more enjoyable (Hew et al. 2016). ...
Article
Full-text available
Advances in digital factory technologies are offering great potential to innovate higher education, by enabling innovative learning approaches based on virtual laboratories that increase the involvement of students while delivering realistic experiences. This article introduces a framework for the development of virtual learning applications by addressing multidisciplinary requirements. The implementation of the framework can be eased by the use of the proposed virtual learning factory application (VLFA), an open-source solution that takes advantage of virtual reality to support innovative higher-education learning activities in industrial engineering. A complete design and development workflow is described, starting from the identification of the requirements, to the design of software modules and underlying technologies, up to the final implementation. The framework and the VLFA have been tested to implement a serious game related to the design and analysis of manufacturing systems, also collecting the feedback of students and teachers.
... Recently, academic engagement has attracted the attention of researchers and educators due to its comprehensiveness in describing students' motivation and learning, and also as a strong predictor of student's performance, progress, and success (Lam et al, 2016). In both traditional and online learning, student engagement is a crucial aspect of learning (Khalil et al., 2017). Archambault et al (2009) identified that student engagement can be used as a forecasting element for dropout in schools. ...
... "Collaborative learning" and "Social networks" as participants learn and participate in cooperative environments that promote learning communities within external hypermedia environments, such as social networks (Cruz-Benito et al., 2017). "Learning analytics" to collect and analyze MOOC data for improvement (Khalil et al., 2017). And "Game theory," among others. ...
Article
Full-text available
Currently, educational systems have assumed a relevant role in developing knowledge and strengthening skills in individuals, an aspect that has become a determining factor for the advancement of society. However, these systems present constant challenges, especially influenced by the advance in information and communication technologies, access to the Internet, and mobile devices, which implies transformations in the new paradigms of teaching and learning methodologies. In this scenario, gamification has been one of the strategies used within virtual learning environments such as MOOCs to increase student motivation in the development of courses. In the last decade, interest in this topic has been evidenced. Therefore, this study aims to identify the main research trends in studies on gamification in MOOCs in the last ten years. For this purpose, a bibliometric analysis was carried out using the Scopus database, from which 265 publications were obtained. The main actors (authors and journals) most cited and thematic trends were identified based on the recurrence of keywords. Among the findings, it was identified that researchers are interested in e-Learning, motivation, online learning, serious games, student engagement, badges and rewards, and the use of the Internet as a tool for learning.
Article
This systematic review investigates the effectiveness of motivational strategies on learner engagement and retention rates in Massive Open Online Courses (MOOCs). Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we analyzed 140 studies published between 2014 and 2023 from key academic databases. The objective was to identify and evaluate motivational strategies that significantly reduce MOOC dropout rates. Our findings reveal that personalized learning, interactive content, and peer collaboration are strongly correlated with increased learner engagement and persistence. These strategies align well with learners' intrinsic goals, enhancing their educational experience and adherence to courses. The review also identifies gaps, such as the need for longitudinal studies and culturally tailored motivational strategies, offering a refined agenda for future research in MOOC education. This study contributes to the field by systematically synthesizing existing research, providing new insights into effective educational strategies, and highlighting areas for improvement in MOOC design and implementation. Doi: 10.28991/ESJ-2024-SIED1-08 Full Text: PDF
Article
The unprecedented global movement of school education to find technological and intelligent solutions to keep the learning ecosystem working was not enough to recover the impacts of Covid-19, not only due to learning-related challenges but also due to the rise of negative emotions, such as frustration, anxiety, boredom, risk of burnout and the so-called “Covid fatigue”. Although this is not a new problem, it was deepened during the pandemic, and we need to face old and new challenges in different ways. Despite focusing only on the learning system’s inefficiencies and on the hegemony of solutions to tackle the learning gap, we also need to shed light on the strengths and the positive aspects of the learning process to promote wellbeing. As highlighted by John Self, an intelligent tutoring system would behave as if it genuinely cared about the student’s success. This note from John Self sheds light on the importance and reflection of what success means and for whom. This study presents a roadmap for positive psychology and artificial intelligence in education. It is intended to identify and understand how the intersection of Positive Psychology and Artificial Intelligence in Education can support the promotion of learning and wellbeing of students, teachers, and other educational stakeholders. As such, a bibliometric analysis of positive psychology and artificial intelligence in education was made as the so-called Positive Artificial Intelligence in Education (P-AIED). The search string was performed in 2021, and the total number of studies gathered was 10,777. After all the PRISMA steps, 256 studies were approved according to the inclusion criteria. The main conclusions were the high number of institutions and researchers with related publications indicate a new trend for the community of AIED; the high number of collaboration from different countries indicates a possible global movement toward P-AIED; Positive Emotion and Engagement were the main Positive Psychology constructs identified in the studies; the lack of well-grounded theories of Positive Psychology indicates an excellent research opportunity; Positive Learning Analytics (P-LA), Positive Educational Data Mining (P-EDM) and Positive Intelligent Tutoring Systems (P-ITS) are three hot topics for P-AIED.
Article
Full-text available
Massive Open Online Courses (MOOCs) require students’ commitment and engagement to earn the completion, certified or passing status. This study presents a conceptual Learning Analytics Activity-Motivation framework that looks into increasing students’ activity in MOOCs. The proposed framework followed an empirical data analysis from MOOC variables using different case studies. The results of this analysis show that students who are more active within the offered environment are more likely to complete MOOCs. The framework strongly relies on a direct gamified feedback that seeks driving students’ inner motivation of competency. 29.03.2017 | Mohammad Khalil & Martin Ebner
Article
Full-text available
URL: http://repositorio.grial.eu/bitstream/grial/560/1/19_ijee3155ns.pdf The phenomenon of MOOC (Massive Online Open Courses) is increasingly experienced and is giving rise to new scenarios and challenges with several features that are different from previous approaches to online education. In the field of engineering education, Information and Communication Technologies are making continuous innovation in methods of teaching and learning for students. Engineering Education institutions, like the Technical University of Madrid (Spain), are expanding their online offerings and making a more effective use of technologies for learning. This research presents a gamification cooperative MOOC model (gcMOOC) that can be applied in the design of this type of course. Using an explanatory sequential mixed methods design, which integrates the quantitative and qualitative methods, the study investigates the factors that influence motivation, collaboration and learning in gcMOOC. This work also suggests a set of practical recommendations and tools to improve the motivation, learning level and completion rate of participants in MOOC course in Engineering Educational when the gcMOOC model is implemented. The results of this study state that the incorporation of virtual communities and gamification methodologies increase participant learning motivation in engineering MOOC courses. Additionally, these gamification tools aid students to deepen their learning and involve them in the course increasing their motivation and the completion rates in MOOCs.
Conference Paper
Full-text available
Massive Open Online Courses (MOOCs) have been tremendously spreading among Science, Technology, Engineering and Mathematics (STEM) academic disciplines. These MOOCs have served an agglomeration of various learner groups across the world. The leading MOOCs platform in Austria, the iMooX, offers such courses. This paper highlights authors’ experience of applying Learning Analytics to examine the participation of secondary school pupils in one of its courses called “Mechanics in everyday life”. We sighted different patterns and observations and on the contrary of the expected jubilant results of any educational MOOC, we will show, that pupils seemingly decided to consider it not as a real motivating learning route, but rather as an optional homework.
Chapter
The rapid and constant pace of change in technology and the increasing involvement of educational institutions in the massive online open courses (MOOC) movement elicit a large myriad of opportunities and challenges. One of the main issues is the reported high dropout rate. In this sense, gamification strategies have been proposed as a complement to existing learning approaches providing a powerful and motivational learning experience to students. Examples of gamification strategies for MOOC environments include rewards for learning activities, applying levels and leader-boards to encourage progress and competition, and badges for participation in forums. The aim of this study is to contribute to the analysis of motivational factors to provide improved learning experiences for cloud-based learning services. This paper presents lessons learned from the MOOC course “Authoring tools for e-learning courses”. 1678 participants experienced a mix of gamification strategies: Badges – Leaderboard forums; Students Classifier League and Reward strategy. Findings revealed the reward strategy as the most effective one, and indicated increased motivation to complete the assigned learning activities.
Conference Paper
To minimize barriers to entry, massive open online course (MOOC) providers collect minimal demographic information about users. In isolation, these data are insufficient to address important questions about socioeconomic status (SES) and MOOC enrollment and performance. We demonstrate the use of third-party datasets to enrich demographic portraits of MOOC students and answer fundamental questions about SES and MOOC enrollment. We derive demographic information from registrants' geographic location by matching self-reported mailing addresses with data available from Esri at the census block group level and the American Community Survey at the zip code level. We then use these data to compare neighborhood income and parental education for US registrants in HarvardX courses to the US population as a whole. Overall, HarvardX registrants tend to reside in more affluent neighborhoods. Registrants on average live in neighborhoods with median incomes approximately. 45 standard deviations higher than the US population. Higher levels of parental education are also associated with a higher likelihood of registration.
Article
Massive open online courses (MOOCs) require individual learners to be able to self-regulate their learning, determining when and how they engage. However, MOOCs attract a diverse range of learners, each with different motivations and prior experience. This study investigates the self-regulated learning (SRL) learners apply in a MOOC, in particular focusing on how learners' motivations for taking a MOOC influence their behaviour and employment of SRL strategies. Following a quantitative investigation of the learning behaviours of 788 MOOC participants, follow-up interviews were conducted with 32 learners. The study compares the narrative descriptions of behaviour between learners with self-reported high and low SRL scores. Substantial differences were detected between the self-described learning behaviours of these two groups in five of the sub-processes examined. Learners' motivations and goals were found to shape how they conceptualised the purpose of the MOOC, which in turn affected their perception of the learning process.
Article
Massive open online courses (MOOCs) have developed rapidly and become tremendously popular because of their plentiful gamification designs, such as reputation points, rewards, and goal setting. Although previous studies have mentioned a broad range of gamification designs that might influence MOOC learner engagement, most gamified MOOCs fail to meet learning objectives because of a lack of research regarding suitable game design, as well as poor rationale for or design of gamification mechanics. This study aims to explore and identify engaging gamification mechanics for MOOC learners. We conducted a focus group interview with 25 MOOC frequent users to identify 40 gamification mechanics. This study then determined the relative engagingness of these gamification mechanics by administering an online survey to 5,020 MOOC learners. The results indicated that the 10 most engaging gamification mechanics accounted for more than 50% of the engagingness. The mechanics of the Where's Wally game is extremely engaging for MOOC learners; however, they it is not been demonstrated in previous relevant studies. Finally, we discuss the top five engaging gamification mechanics and their implications.