ArticlePDF Available

Abstract and Figures

In this paper, we reflect on the implementation of a gamified application for helping students learn important facts about their study program. We focus on two design features, of which different configurations were tested in a field experiment among Dutch university students (N = 101). The first feature is feedback, which is expected to increase engagement, with personalized (“tailored”) feedback being more effective than generic feedback. The second feature is a session limit that was designed to prevent users from “binging” the game, because this could prevent deep learning. Results showed that generic feedback was more effective than tailored feedback, contrasting our expectations. The session limit, however, did prevent binging without reducing the overall number of sessions played. Our findings suggest that careful consideration of game properties may impact sustaining and encouraging play via a gamified application.
Content may be subject to copyright.
Gamification as a tool for
engaging student learning:
A field experiment with a
gamified app
Kasper Welbers, Elly A Konijn,
Christian Burgers and Anna Bij de Vaate
Department of Communication Science, Vrije Universiteit Amsterdam,
the Netherlands
Allison Eden
Department of Communication, Michigan State University, MI, USA
Britta C Brugman
Department of Communication Science, Vrije Universiteit Amsterdam,
the Netherlands
In this paper, we reflect on the implementation of a gamified application for helping students learn
important facts about their study program. We focus on two design features, of which different
configurations were tested in a field experiment among Dutch university students (N¼101). The
first feature is feedback, which is expected to increase engagement, with personalized (“tailored”)
feedback being more effective than generic feedback. The second feature is a session limit that
was designed to prevent users from “binging” the game, because this could prevent deep learning.
Results showed that generic feedback was more effective than tailored feedback, contrasting our
expectations. The session limit, however, did prevent binging without reducing the overall
number of sessions played. Our findings suggest that careful consideration of game properties
may impact sustaining and encouraging play via a gamified application.
Gamification, education, feedback, distributed practice, adaptive learning
Corresponding author:
Kasper Welbers, De Boelelaan 1105, 1081 HV Amsterdam, the Netherlands.
E-Learning and Digital Media
2019, Vol. 16(2) 92–109
!The Author(s) 2018
Article reuse guidelines:
DOI: 10.1177/2042753018818342
New technologies offer exciting opportunities to engage student learning in new ways. One
of the new-technology potentials for motivating students to learn is gamification, which can
be defined as “the use of game-design elements in non-game contexts” (Deterding et al.,
2011: 9). In the past decade, the popularity of gamification increased rapidly, and various
cases are known in which businesses, web designers, and education workers used gamifica-
tion in order to engage and motivate a target group with successful outcomes (Chou, 2017;
Mollick and Rothbard, 2014). However, more systematic research is needed to know when
and how gamification can be used to the greatest benefit in an education setting. Many
different gamification options exist and usage varies widely, which we briefly review in the
next sections to contextualize our study.
The vast interest in gamification instigated a wide array of studies across many different
topics, audiences, and disciplines. For instance, a 2012 literature review found >125 empir-
ical studies examining effects of gamification in a variety of contexts (Connolly et al., 2012).
By contrast, a more recent systematic review that only searched for gamification in the
context of education found only 15 studies on some aspect of gamification in an education
setting (Nah et al., 2014). This latter review, focusing specifically on the education context,
indicates that feedback seems to be a potentially useful mechanism for increasing engage-
ment among students for a specific course or learning outcome. However, research on
feedback in other contexts shows that the effectiveness of feedback is not only dependent
on the verbal feedback itself, but also on situational factors (e.g. Burgers et al., 2015; Hatala
et al., 2014; Kluger and DeNisi, 1996). One of the aspects of feedback that is often associ-
ated with effectiveness is the use of personalized (or “tailored”) feedback (e.g. Krebs et al.,
2010; Lustria et al., 2013). Thus, we studied whether personalized feedback also outperforms
non-personalized (“generic”) feedback in stimulating play and learning in the context of
gamified learning tools for education.
Second, gamification for learning can only be successful when students play for an
extended amount of time, such that they process enough (new) information for learning
to take place. In addition, for learning a new skill or knowledge, literature shows that
distributed practice enhances learning (for a review, see Dunlosky et al., 2013). This
means that it is more effective to spend the time learning spread over short time intervals
over several days rather than in one long session (see also Heidt et al., 2016; Rohrer, 2015).
However, in some contexts, players choose to “binge” an entire game over one longer period
of time rather than spacing out play over several short sessions. Thus, we tested whether
“enforcing” distributed learning through imposing a daily session limit positively impacts
play and learning or whether it backfires.
To answer these questions, the current study investigated the potential use of gamifica-
tion for engaging university students with an online learning platform at a Dutch university.
A gamified application, in the form of a multiple-choice quiz app, was launched among
undergraduate students in a Faculty of Social Sciences. The purpose of the app was to teach
students relevant information about various aspects of student rules, regulations, and socials
in university life, ranging from exam regulations to social events at campus. Fitz-Walter
et al. (2011: 123) found that “new students often feel lost, having trouble meeting new
friends and finding what services and events are available on campus” and argue that
games can help alleviate this. Accordingly, our game was tailored toward new and existing
students who may not yet be aware of specific campus services or may be disinclined to
browse or search through the university website. The gamified application was a modified
version of a popular online quiz game which rewards users for playing and correctly
Welbers et al. 93
answering questions about specific topics. The game aspects include points, rewards, and
feedback from the governing system. Our content was identical to content available,
at various places, on the university website, but brought together in this game and thus
presented within this novel learning environment. To introduce the app, we sent out emails,
presented the app in classes, and put up posters. Students could participate by following a
link to a short survey where they could enter basic personal information and register for
using the app.
We hypothesized that learning engagement, in the form of persistent play, would be
highest for players who (a) received tailored feedback and (b) had a session limit to enforce
distributed learning. In order to test our hypotheses, we created experimental conditions in
which we tailored feedback within the app for some participants but not others, limited play
lengths for some but not others, in addition to tracking user data for all participants,
including surveying participation and engagement. The results of tracking and survey
data (e.g. the interest of students in the app) can provide lessons for future designs and
launch campaigns in university settings. The following sections elaborate on the background
of gamification and the distributed practice in the field of education to further specify the
hypotheses that guided our research.
Despite its current popularity, the term “gamification” is still young. Its first documented
use is often credited to Brett Terill (2008), who talked about “gamification” in a blog post to
define the act of “taking game mechanics and applying them to other web properties to
increase engagement.” In its current usage, the concept of gamification is not restricted to
web properties, but more generally refers to “the use of game design elements in non-game
contexts” (Deterding et al., 2011: 9) or “using game-based mechanics, aesthetics and game
thinking to engage people, motivate action, promote learning, and solve problems” (Kapp,
2012: 10). In the current paper, we focused on the properties of gamification that can be used
to stimulate learning.
Gamification is related, but not identical, to the concept of game-based learning. Where
gamification is about the use of game design elements in a non-game context, game-based
learning refers to the use of actual games to acquire skills or knowledge. In game-based
learning, the skills that are put to the test in the game correspond to the learning task (Gee,
2013), as is for instance the case in a game where medical students or personnel perform
surgical procedures in a simulated environment (Kapp, 2012).
For certain cases, such as the multiple-choice quiz app used in the current study, the
distinction between a gamified experience and game-based learning can be blurred. Cheong
et al. (2013: 207) argue that gamification “can be viewed as a continuum ranging from
serious games at one end of the spectrum to normal activities to which game elements
have been added at the other end of the spectrum.” The gamified multiple-choice quiz
app falls in the middle of this spectrum. The simple learning task of memorizing a list of
facts is made into a game through the presentation of facts in a quiz format. However, in
itself, this quiz format still lacks important game elements, such as progression, rewards, and
competition. To add these elements, tested gamification mechanics have been implemented
in the app, such as avatars, experience points, and badges. Accordingly, the application can
more accurately be classified as a gamified multiple-choice quiz, similar to the application
used by Cheong et al. (2013), rather than as a dedicated game for learning.
94 E-Learning and Digital Media 16(2)
Although gamification for learning and game-based learning are two different concepts,
they share common ground on the idea that game elements can make learning experiences
more engaging. Accordingly, research into whether and why certain game elements are
effective in games for learning, and in games in general, is relevant for understanding the
efficacy of gamification. Academic interest in effectively using game design elements dates
back at least 30 years to Malone (1982), who studied the appealing features of computer
games for the purpose of using these features to make user interfaces more interesting and
enjoyable. Sweetser and Wyeth (2005) contributed greatly to our understanding of the
features that make games enjoyable by developing a scale to measure game enjoyment,
although they did not elaborate on the use of these features in a non-game context.
Building on their work, Fu et al. (2009) developed a scale to measure the enjoyment of
people playing educational games. Cozar-Gutierrez and Saez-Lopez (2016) recently reported
that teachers’ interest in and perceived innovativeness of using games in the classroom is
strong, showing a desire to understand the best practices in incorporating games and gami-
fied education in the classroom.
Despite academic interest in understanding and using the appealing features of games,
academia has been slow to react to the surge of gamification projects in businesses and on
the Internet (Huotari and Hamari, 2012). Initial support for the efficacy of gamification
mainly came from businesses, where the idea that tasks can be made more efficient and
engaging by wrapping them in game design elements rapidly gained popularity. Yu-Kai
Chou, an influential gamification expert, collected and published a list of 95 documented
gamification cases, based on the criterion that the documentation reports return on invest-
ment indicators (Chou, 2017). Overall, these cases show that gamification can indeed have a
strong, positive impact on engagement and performance in various activities. Although it is
not reported how these cases were selected, and there could very well be a bias toward
successful cases, this adds weight to the claim that gamification can work, given the right
context and implementation.
A recent literature study of academic gamification research found that most studies on
the subject verified that gamification can work, even though effects differ across contexts
(Hamari et al., 2014). By context, the authors refer to the type of activity being gamified,
such as exercise, sustainable consumption, monitoring, or education. In the context of
education—which is the focus of the current study—the outcomes of gamification were
mostly found to be positive, as seen in the forms of increased motivation, engagement,
and enjoyment (e.g. Cheong et al., 2013; Denny, 2013; Dong et al., 2012; Li et al., 2012).
Another literature study that focused broadly on the effects of games but also discussed
games for learning in particular found that “players seem to like the game-based approach
to learning and find it motivating and enjoyable” (Connolly et al., 2012: 671). Yet, they also
argue that the motivational features of learning oriented games should be examined in more
detail and note that evidence for more effective learning was not strong.
In all, these studies concluded that games have the potential to be useful tools for learn-
ing, but stress that their efficacy depends heavily on the use of different game features and
how they are implemented (Connolly et al., 2012; Hamari et al., 2014). Features such as
feedback options and the way in which the level of difficulty adapts to a player’s skills can be
critical to a game’s success and need to be investigated in more detail. For example, Barata
et al. (2015) report success in clustering student types based on learning performance in a
gamified engineering course. Over two years, by targeting student groups who responded
differently to the learning environment, Barata et al. (2015) were able to reduce
Welbers et al. 95
underperforming students from 40% of the class to 25% of the class. Therefore, by targeting
learning and interventions to particular types of players, the outcomes of gamified applica-
tions in higher education can be improved.
The efficacy of gamifying learning through feedback
Lee and Hammer (2011) argue that feedback is central to the potential of gamification.
First, to make a person feel that they are successfully improving and heading toward a goal,
games can provide explicit feedback to show this progress. Studies indicate that even simple,
virtual reward systems such as experience points and badges can increase the engagement of
players (Denny, 2013; Fitz-Walter et al., 2011). For instance, Hatala et al. (2014) conducted
a meta-analysis to investigate if feedback positively impacted learning of procedural skills in
medical education. Their results demonstrate that providing feedback moderately enhances
learning. In addition, they found that terminal feedback (i.e. feedback given at the end of the
learning activity) was more effective than concurrent feedback (i.e. feedback given during
the learning activity). These analyses point toward the effectiveness of using feedback as a
mechanism to enhance learning. Thus, for the current study, we expect a similar pattern
leading to:
H1. Students receiving feedback play more sessions of the gamified app compared to students
receiving no feedback.
However, not all feedback is equally effective in achieving its goal (e.g. Burgers et al.,
2015; Kluger and DeNisi, 1996). For instance, one study shows that the effectiveness of
negative feedback (i.e. feedback emphasizing the elements that could be improved upon)
and positive feedback (i.e. feedback emphasizing the elements that went well during an
activity) depends on the task at hand (Burgers et al., 2015). Negative feedback was more
effective than positive feedback when the problem could immediately be repaired (e.g. in the
case of a game which enables a new session to be started immediately). By contrast, positive
feedback was more effective than negative feedback when repair was delayed (e.g. in the case
of a game which only enables one session per specified time period). Thus, when using
feedback, it is important to match the specific type of feedback to the specific task at hand.
One type of feedback which has been associated with enhancing effectiveness is the use of
tailored feedback over generic feedback (e.g. De Vries et al., 2008; Krebs et al., 2010). In
tailored feedback, the specific content is personalized (“tailored”) to the individual, through
mechanisms like personalization (i.e. addressing the receiver by name) or by adapting the
feedback to their individual performance (e.g. by including descriptive statistics that refer to
the receiver’s personal performance). By contrast, generic feedback is similar for all addres-
sees receiving the feedback. Tailored messaging may take the form of, for example, frequent
prompt or reminder emails (Neff and Fry, 2009), often edited (or tailored) to include infor-
mation specific to particular participants (Schneider et al., 2013). While simple interventions
such as emails can increase participants’ logging into online systems, tailored information
can further increase desired behaviors in specific cases (Krebs et al., 2010; Neff and
Fry, 2009).
Nevertheless, other studies show different results (e.g. Kroeze et al., 2006; Noble et al.,
2015). For instance, a systematic review by Kroeze et al. (2006) demonstrates that the
effectiveness of tailoring depends on the specific kind of behavior targeted. For instance,
96 E-Learning and Digital Media 16(2)
for 11 out of 14 interventions targeting fat reduction, the authors found positive effects of
tailoring over a generic intervention. By contrast, for only 3 out of 11 interventions targeting
physical education, did the authors find such positive effects of tailoring. Thus, the question
whether or not tailoring improves effectiveness may also be dependent on contextual factors
like the targeted behavior. In the current study, we aim to motivate students to continue
using a gamified app to increase learning. For this specific context, we do not yet have
information on whether tailoring is an effective strategy or not. Yet, given that, across
behaviors, tailoring typically boosts performance compared to generic information (e.g.
Krebs et al., 2010), we expect that:
H2. Students receiving tailored feedback play more sessions of the gamified app compared to
students receiving generic feedback.
Distributed practice in education
In pilot studies of the application used in this study, it was observed that some players
tended to binge play. Within a matter of days, they would play so many sessions that they
quickly learned the answers to most questions. Although this can be considered as a success
in terms of engagement, it can actually be harmful for long-term recall. One of the chal-
lenges for gamification in a learning task—and more so for games for learning—is thus not
to create as much engagement as possible, but to create the right type and amount of
engagement to best achieve the learning goal.
One of the goals of any education intervention is to stimulate deep learning, which means
that students retain the most important information, even when the education intervention
has been completed. In that light, many studies highlight the positive aspects of distributed
practice. This means that students spread (“distribute”) their learning and practice activities
over a number of relatively short time intervals, as compared to cramming all learning in
one long session. For instance, a recent study by Heidt and colleagues (2016) contrasted two
versions of a training program meant to teach students how to conduct a specific type of
police interview. One of the versions contained a single session of two hours containing all
information, while the other version spaced the program into two one-hour sessions (i.e.
distributed learning). Results demonstrated that, on average, participants in the distributed-
learning condition performed better in that they asked more open questions and were able to
elicit more detailed information through these open questions. This suggests that distributed
learning could positively enhance training outcomes.
This study by Heidt and colleagues (2016) is not the only one that shows these advantages
of distributed learning. A review by Dunslosky and colleagues (2013) argues that distributed
practice is one of the best researched topics in the field of education studies. Typically,
distributed practice focuses on two elements, referring to the spacing of activities (i.e. the
number of learning activities planned to cover all materials) and the time lag between
activities. The review by Dunlosky et al. (2013) demonstrated that distributed practice
enhances learning across a variety of learning contexts. Thus, the literature suggests that
it is better to spread out learning over several sessions instead of concentrating it in one
large binge.
Many studies on distributed practice focus on specific training programs with set dates
for education activities (such as the interview-training sessions described by Heidt and
Welbers et al. 97
colleagues, 2016). Gamification interventions offer the technological possibilities of also
stimulating distributed learning on an individual basis through computer design. One way
to do this is by imposing a daily limit on players. Such a daily limit prevents users from
gorging on all content in one large binge and may instead stimulate players to return to the
content on later dates, thus encouraging distributed practice. However, an important con-
dition for this feature is that it should not reduce the overall amount of sessions played. We
test whether this effect can be achieved through gamification, formulating the follow-
ing hypothesis:
H3. Students with a daily limit play an equal amount of sessions on more different days compared
to students without a daily limit in playing the gamified app.
Increasing the challenge
To motivate a person to achieve their best performance, games can be designed to interac-
tively increase the difficulty of an activity to match the player’s growth in skill (Barata et al.,
2015; Garris et al., 2002; Malone and Lepper, 1987). Ensuring the right level of challenge
can, however, be difficult depending on the goal of an educational or serious game. In the
current study, the goal of the app is knowledge acquisition—letting students learn a list of
facts. For this type of learning task, it is not always possible to implement progressive
difficulty levels. In our case, the facts are mostly orthogonal, in the sense that there is no
overarching skill that can be learned, as would for instance be the case for math or language
acquisition tasks. Players thus either know the answer to a question or they do not, and
when posed a novel question, players have no previous skill to rely on. This complicates the
goal of keeping players perform at their best.
Still, it is possible to influence how well players perform based on what questions are
asked. If players perform very well, to the point where they might experience the activity to
be too easy, one might next present more difficult questions such as those that the player has
not yet answered before in previous rounds. Conversely, if a player is performing poorly,
one might present easy questions, for example, questions that the player had already
answered correctly in previous rounds. This interactive selection of questions is a distinct
advantage compared to textbook learning. Furthermore, this is where an important poten-
tial lies for client-server infrastructures. Many contemporary applications connect to serv-
ers, making it possible for the host of a game to monitor and aggregate playing results. This
global-level information can be used to optimize the question selection algorithm, for
instance by determining how difficult questions are based on average performance.
At present, there does not appear to be any empirical support that player performance on
a quiz affects prolonged play. In the current study, we test this assumption by analyzing to
what extent prolonged play can be predicted by a player’s recent performance. Based on the
theory that the level of challenge should be neither too low nor too high in view of an
individual’s mastery of the task (Csikszentmihalyi, 1990; Garris et al., 2002; Giora et al.,
2004), we expect that players get demotivated if the task is too easy or too difficult. This
implies that the effect of recent performance on prolonged play should form an inverted
u-curve, with most playing at the level of “optimal innovation” (Giora et al., 2004) when the
task is neither too easy nor too difficult. Thus, we hypothesize that:
98 E-Learning and Digital Media 16(2)
H4. The relation between performance and prolonged play follows an inverted u-curve, with pro-
longed play decreasing if players perform (a) very poorly, and (b) very well.
The study was conducted within two periods in the Netherlands at a Faculty of Social
Sciences of a public university between March and April 2016 (period 1; including all
students of the School of Social Sciences) and October to December 2016 (period 2; only
including freshmen students of the same Faculty of Social Sciences). We informed students
of the app by sending out emails, putting up posters, and presenting the app in lectures. To
participate, students had to fill out a one-minute online introductory survey that could be
completed on a mobile phone. The link to the survey was distributed per email using a
shortened link and made accessible as a QR-code. After completion, students were auto-
matically assigned to conditions, and within 24 hours received an invitation to use the app.
The app itself could be downloaded for free from official app stores such as Google Play, for
which students were given directions in the campaign material and at the end of the brief
introductory survey.
In total, 2444 students were contacted in period 1 and 706 students were contacted in
period 2 (grand total: 3150 students). Of all students contacted, 307 (9.8%) completed the
survey, of which 223 (9.1%) in the first period and 84 (11.9%) in the second period. One
hundred and one students (3.2%) finished all steps of the registration, of which 70 (2.9%) in
the first period and 31 (4.4%) in the second period.
Table 1 contains the descriptive statistics of the participants. Overall, there were slightly
more female participants (65.3%) than male, and most participants were in their early
twenties (M¼22.53; SD ¼3.51). As expected, first-year students in the bachelor and pre-
master were generally more interested in the app as the acquired information is more new
Table 1. Descriptive statistics.
Variable Levels n%
Gender Female 66 65.3
Male 35 34.7
Age 18–21 43 42.6
22–25 43 41.6
26–31 16 15.8
First year Yes 67 66.3
No 34 33.7
How well do you know the school? Very badly 0 0.0
Badly 8 7.9
Neutral 32 31.7
Good 52 51.5
Very good 9 8.9
All 101 100.0
Welbers et al. 99
and relevant to them (e.g. details of exam regulations; see above). The over-representation of
the Communication Science and Culture, Organization, and Management students is pro-
portionate to the relatively high number of students in these programs. Furthermore, stu-
dents were asked to judge their prior knowledge about the university on a five-point scale,
which showed that on average, students considered their knowledge to be between neutral
and good (M¼3.61; SD ¼0.77). Notably, the 206 students that did fill in the survey but did
not use the app did not score any different (M¼3.61; SD ¼0.76), indicating that prior
knowledge was not a relevant factor in their decision not to participate.
The experiment had a 2 (daily limit: present vs. absent) 3 (no feedback, generic feedback,
personalized feedback) between-subjects experimental design. Participants in the condition
with a daily limit were limited in their play to four sessions per day. Participants in the
condition without a daily limit could play as many sessions per day as they wanted.
Participants were randomly assigned to the six conditions.
Participants in the feedback conditions received a weekly email on Monday for three
consecutive weeks that encouraged them to play (if they did not play that week) or to play
more. The difference between the generic and personalized feedback conditions was limited
to the information provided in the email. Participants in the generic-feedback condition
were not addressed by name, and the email only reported whether or not they played in the
previous week. Participants in the personalized-feedback condition were addressed by their
first name, and the email reported the exact number of sessions they played in the previous
week. The encouragement message also changed depending on how many sessions were
played. The personalized feedback condition deliberately did not include additional support,
such as offering tips or replying to specific questions that the participant answered incor-
rectly. The condition thereby focuses purely on whether the participant was addressed as a
generic and anonymous user versus as an individual that is personally monitored. The effect
investigated in this study is thus only a communication effect and not an effect of offering a
different learning experience.
The number of participants per condition is reported in Table 2. The distribution of
participants is not perfectly balanced, because not all students who completed the survey
(upon which they were assigned to a condition) actually participated.
The application
The application used for this study, Knowingo (, was developed by a
partner company. It is normally licensed to businesses that use it as a tool to disseminate
Table 2. Participants per condition.
Message condition
None Generic Tailored Total
No limit 14 13 19 46
Limit 17 20 18 55
Total 31 33 37 101
100 E-Learning and Digital Media 16(2)
factual knowledge throughout their organizations. Traditionally, learning this type of
knowledge would require employees to study non-interactive documents. The purpose of
Knowingo is to make this type of knowledge acquisition more engaging and efficient by
presenting the learning task as a multiple-choice quiz.
By itself, a simple multiple-choice quiz still lacks many important game elements. To
make the application more engaging, Knowingo therefore incorporates various tested gami-
fication features. The first time players log in, they need to choose an avatar that is visible to
themselves and other players whom they can challenge. By playing, and by giving the right
answers, players receive experience points to grow in levels and unlock virtual rewards. Each
day players also receive new quests, such as playing for streaks of correct answers, that give
additional experience and rewards.
Furthermore, the game algorithm has been designed to make users play short consecutive
sessions. Sessions consist of seven questions, and each question has a time limit. To prevent
users from getting bored or frustrated, the selection of questions takes the session history of
users into account. If players are performing poorly, they can be given more easy questions
or questions that they already answered before to boost their score, and if players have
perfect scores, they are more likely to receive new and difficult questions. One of the devel-
opment goals of the application is to optimize this question selection algorithm by using the
client-server infrastructure to collect information from all users, in order to learn the diffi-
culty of different questions and use this to provide an adaptive learning experience. The
version used in this study does not yet implement this adaptive learning experience.
For the current study, the Knowingo app was used to help students learn relevant infor-
mation about their university, ranging from exam regulations to social events. We developed
200 unique multiple choice questions, each with four possible answers, to ensure that there
would be enough content to prevent users from getting the same questions too often. For
example, some of the questions were: “what does ECTS stand for?,” “what must you always
bring to an exam?,” “who can help you with course registration issues?,” and “when is the
pub quiz in the [campus cafe]?”
Predictors of player participation
To test the hypotheses about the effects of feedback on player participation, we used regres-
sion analysis to explain the variance in the number of sessions played by each participant.
Figure 1 shows the distribution of this dependent variable. Note that this is a heavily over-
dispersed count variable: most participants only played one or a few sessions, but there are
also several players that continued for more than 200 sessions. Accordingly, we used a
negative binomial generalized linear model. Results are presented in Table 3. Contrary to
our hypothesis, we did not find a significant effect of feedback messages in general on the
number of sessions played (b ¼0.449, p=0.310). Therefore, we reject H1.
Our second hypothesis was that the tailored feedback messages have a stronger effect on
the number of sessions played than the generic feedback messages—or, given that we did not
find a general effect of feedback, that there would be an effect for tailored feedback.
However, our results showed that tailored feedback messages did not have an effect
(b ¼–0.118, p=0.341), but rather the other way around. The number of sessions played
is significantly higher for participants in the generic feedback condition (b ¼0.821, p<0.05).
Welbers et al. 101
Additionally, male participants tend to play more sessions compared to female participants
(b ¼0.924, p<0.01), and older participants tend to play less sessions (b ¼–0.102, p<0.05).
Effect of daily limit on prolonged play
Our third hypothesis concerns the impact of a daily limit, where we hypothesized that
students with a daily limit play a similar amount of sessions on more different days com-
pared to students without a daily limit. To test this, we ran a negative binomial regression
analysis with the unique number of days played as the dependent variable. For this analysis,
we only included participants that at least once played four or more sessions on one day
(n¼35), since the other participants would not have experienced an effect of the daily limit.
To account for the small sample size, we only included two independent variables: the daily
limit condition (dichotomous) controlled for the number of sessions played. The results,
Figure 1. Histogram of number of sessions (logarithmic scale) per player (N¼101).
Table 3. Negative binomial regression predicting the number of sessions played per user.
Sessions played
M1 M2a M2b
Freshman 0.359 (0.326) 0.249 (0.325) 0.231 (0.318)
Gender (M ¼1) 1.008** (0.310) 0.959** (0.310) 0.924** (0.303)
Age –0.103* (0.042) –0.096* (0.042) –0.102* (0.055)
Prior knowledge 0.329 (0.203) 0.276 (0.203) 0.295 (0.199)
Daily limit –0.030 (0.287) 0.038 (0.282)
Feedback message 0.449 (0.310)
Generic message 0.821* (0.350)
Tailored message –0.118 (0.341)
Constant 3.415*** (1.253) 3.289*** (1.236)
N(users) 101 101 101
Log Likelihood –376.621 –375.726 –371.925
14.74** 1.79 9.39*
Note: *p <0.05; **p <0.01; ***p <0.001.
102 E-Learning and Digital Media 16(2)
as presented in Table 4, show that participants in the daily-limit condition indeed played on
more unique days (b ¼0.620, p<0.01). Based on these results, we can accept H3a.
Playing on more different days, however, would not be beneficial to learning if the overall
number of sessions played suffers from the daily limit condition. Our results showed that
participants in the daily limit condition—who could only play four sessions each day—did
not play less sessions overall (M¼66.44, Mdn ¼17.5) compared to participants without a
daily limit (M¼41.29, Mdn ¼22). The nonparametric Mann–Whitney–Wilcoxon test shows
that the difference is not significant (W¼156.5, p¼0.921). This supports H3b. This indi-
cates that participants in the daily limit condition were not less motivated than those with-
out a daily limit, but rather spread out their sessions over more different days—which is the
intended effect of this feature.
Effect of performance on prolonged play
To investigate whether a player’s experience of difficulty affects their motivation to play
(H4), we analyzed whether their performance in each individual session affects the proba-
bility to continue playing. For this analysis, we used a multilevel logistic regression analysis
with random intercepts, where cases are individual sessions that are nested in participants.
The dependent variable, prolonged play, indicated whether participants played a new ses-
sion within 15 minutes after finishing the current session. Our independent variable of
interest is session performance, which indicates how many of the seven questions in the
current session were answered correctly. We control for the participants’ streak, which is the
number of times participants already continued playing (with 15-minute intervals). We also
control for the daily limit condition and its interaction with one’s streak, since participants
in this condition cannot continue playing after four sessions.
Results are presented in Table 5, which shows a negative effect of performance on
prolonged play (b ¼–0.134, p<0.05). This indicates that participants might indeed lose
motivation or become bored the more they perform above their average. This is in line
with H4 that there is an inverted u-curve relation between performance and prolonged play.
However, we did not find any indications that participants also lose motivation if they are
performing below their average.
This is likely related to a ceiling effect in performance: on
average, 5.6 out of 7 questions were answered correctly (SD ¼1.30). This suggests that for
many students, the questions in the current application might not have been challenging
enough. Based on these results, we conclude that H4a is not supported, but H4b
is supported.
Table 4. Negative binomial regression predicting the number of unique
days played by users.
Days played
Sessions 0.008*** (0.001)
Daily limit 0.620** (0.191)
Constant 1.221*** (0.153)
N(users) 35
Log Likelihood –94.733
Note: *p<0.05; **p <0.01; ***p <0.001.
Welbers et al. 103
Conclusion and discussion
This study investigated the use of a quiz app designed to support university students to
acquire essential information about how their university functions and analyzed how manip-
ulating game features and feedback can enhance engagement and learning. The first part of
our study examined the effect of feedback on player participation. No differences were
found between the feedback conditions and the no feedback condition. However, a closer
inspection of the generic versus personalized (“tailored”) feedback conditions revealed that
generic feedback does have a positive effect on player participation, whereas this could not
be established for the personalized feedback. Personalized feedback, which included per-
sonally identifiable information (e.g. the number of sessions one had played thus far), might
possibly induce a boomerang effect (Wattal et al., 2012). This is an important topic for
future studies, which could focus on whether the amount and type of personalization makes
a difference. In particular, a potential explanation that requires inquiry is that it could
matter whether personalization has a clear benefit to the user. In our study, the personalized
feedback condition deliberately did not receive additional help or benefits. More personal-
ization might not induce a boomerang effect if it is clear to benefit the user, such as feedback
on specific answers or links with more information.
While not a prominent focus of this study, it is interesting to note that player participa-
tion was also affected by player demographics. Male and older students tended to play more
sessions compared to female and younger students, respectively. Understanding the effects
of player demographics is important for effective use of gamification, because it can inform
the development of applications with specific audiences in mind. However, we recommend
caution in generalizing our results in this regard. Prior research shows mixed findings, for
instance, in gender effects for both engagement and learning outcomes (Khan et al., 2017;
Su and Cheng, 2015). Current research into gamification is rather diverse, with different
types of applications built for different learning goals, which makes it difficult to draw
conclusions on effects of social and cultural factors. A meta-analysis of demographic factors
in gamification would be a welcome contribution to the field.
The second part of the study investigated whether introducing a daily limit on the number
of sessions a participant can play per day can promote distributed learning. A daily limit
Table 5. Logistic regression analysis, with random intercepts for users, predicting prolonged play.
Prolonged play
M1 M2 M3
Streak –0.170*** (0.031) –0.038 (0.030) –0.040 (0.030)
Daily limit 1.371** (0.482) 1.395** (0.485)
Streak daily limit –1.052*** (0.098) –0.134* (0.058)
Performance –0.134* (0.058)
Constant 1.427*** (0.247) 1.491*** (0.329) 2.173*** (0.448)
N(sessions) 1380 1380 1380
Log Likelihood –816.566 –729.674 –727.027
31.31*** 173.79*** 5.29*
Note: *p <0.05; **p <0.01; ***p <0.001.
104 E-Learning and Digital Media 16(2)
feature can prevent people from binge playing a game, but a concern is that users will simply
play less sessions, rather than maintain their interest over a longer period of time. Findings
of this study show that participants in the daily limit condition indeed played on more
different days compared to participants that could “binge” as many sessions as they
wanted, while playing a similar amount of sessions. So, it seems the daily limit did not
demotivate them and spread out the learning experience over more sessions. This suggests
that including a daily limit in the gamified app in an education setting can be a useful tool to
prevent binge playing and enhance distributed learning.
Finally, we investigated whether a participant’s performance in the game affects pro-
longed play. Results show that if participants perform very well (in the current study often
having a perfect score), they become less likely to continue playing, which confirms the
importance of ensuring that participants are sufficiently challenged. For a multiple-choice
quiz about mostly independent facts, it can be difficult to manipulate the difficulty of the
learning task. However, it is possible to estimate the likelihood that a participant answers a
question correctly and to use this to manipulate performance. By using a client-server
architecture, where all user data is collected, information from all users can be used to
improve this estimate. More large-scale research with this type of application can help us
better understand how performance affects prolonged play.
This study has several limitations that need to be taken into account. The number of
unique participants in our sample was small due to a relatively small sample population. In
addition, we found that three factors made it difficult to get students to play. First, volun-
tary participation might be perceived by students as additional and unnecessary work. Since
we are interested in the extent to which the app alone manages to engage students, we did
not offer any form of compensation for participating. Second, the app is mainly directed at
students that know little about the university, such as starting year students, so our pool of
all students in the School of Social Sciences is (purposefully) too broad in the first period of
our data collection. Third, the current project, which served as a pilot, was launched in the
Spring, which is close to the end of the academic year. To compensate for the low partic-
ipation rate, we included a second wave of data collection only pertaining to freshmen
students and starting the data collection earlier, that is in the Autumn. The relative response
rates in the second period are higher compared to the first period, but the number of
interested students remains to be a small minority.
Aside from limiting our sample, this also tells us something important about the chal-
lenges of gamification projects in natural occurrences of an educational field setting, where it
would be inappropriate to compensate students for participation. This calls for more field
research that investigates whether and how we can get students to actually participate. The
results of this study do show hope: once students started playing, a non-trivial number of
students did become engaged, with some students playing for many hours. More than
improving the game itself, the challenge could be to have students take that first step. In
addition, experimental research into the efficacy of educational games may complement the
valuable insights from field studies.
In conclusion, while our findings are preliminary, we are cautiously optimistic about
continuing this line of research in the future. Our findings suggest that careful manipulation
of game mechanisms can have an impact on sustaining and encouraging play via a gamified
application. Despite the limitations of our study, we are encouraged by these findings and
hope to continue this line of work in gamifying educational settings.
Welbers et al. 105
We thank Loren Roosendaal and IC3D Media for allowing us to use the Knowingo application for the
purpose of this research and their support in accessing the API for extracting the log data of our
participants. Furthermore, we thank the education office of the Faculty of Social Sciences at the Vrije
Universiteit Amsterdam, and in particular Karin Bijker, for enabling us to perform this study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or
publication of this article.
The author(s) received no financial support for the research, authorship, and/or publication of
this article.
1. The common approach to test for an inverted u-curve would be to fit a quadratic term. However,
for our current data, showing a ceiling effect for performance, this would not be a good way to test
our hypothesis. The reason is that it would test whether prolonged play decreases if a user performs
below his or her average, but even then a user does not perform poorly on average. For reference,
we also fitted a quadratic effect, but this did not improve the model (v
¼0.346, df ¼1, p ¼ns).
Barata G, Gama S, Jorge J, et al. (2015) Gamification for smarter learning: tales from the trenches.
Smart Learning Environments 2(10): 1–23.
Burgers C, Eden A, van Engelenburg M, et al. (2015) How feedback boosts motivation and play in a
brain-training game. Computers in Human Behavior 48: 94–103.
Cheong C, Cheong F and Filippou J (2013) Quick quiz: a gamified approach for enhancing learning.
In: Proceedings of the Pacific Asia conference on information systems (PACIS), Jeju Island, Korea.
Chou Y (2017) A comprehensive list of 90þgamification studies with ROI stats [Blog post]. Available
at: (accessed 20
October 2017).
Connolly TM, Boyle EA, MacArthur E, et al. (2012) A systematic literature review of empirical
evidence on computer games and serious games. Computers and Education 59(2): 661–686.
´rrez R and Sa
opez JM (2016) Game-based learning and gamification in initial teacher
training in the social sciences: an experiment with MinecraftEdu. International Journal of
Educational Technology in Higher Education 13(2): 1–11.
Csikszentmihalyi M (1990) Flow: The Psychology of Optimal Performance. New York: Cambridge
University Press.
de Vries H, Kremers SPJ, Smeets T, et al. (2008) The effectiveness of tailored feedback and action
plans in an intervention addressing multiple health behaviors. American Journal of Health
Promotion 22(6): 417–424.
Denny P (2013) The effect of virtual achievements on student engagement. In: Proceedings of the
SIGCHI conference on human factors in computing systems, Paris, France, 27 April–2 May 2013,
pp.763–772. New York: ACM.
Deterding S, Dixon D, Khaled R, et al. (2011) From game design elements to gamefulness: defining
gamification. In: Proceedings of the 15th international academic MindTrek conference: envisioning
future media environments, Tampere, Finland, 28–30 September 2011, pp.9–15. New York: ACM.
106 E-Learning and Digital Media 16(2)
Dong T, Dontcheva M, Joseph D, et al. (2012) Discovery-based games for learning software. In:
Proceedings of the SIGCHI conference on human factors in computing systems, Austin Texas,
USA, 5–10 May, 2012, pp.2083–2086. New York: ACM.
Dunlosky J, Rawson KA, Marsh EJ, et al. (2013) Improving students’ learning with effective learning
techniques: promising directions from cognitive and educational psychology. Psychological Science
in the Public Interest 14(1): 4–58.
Fitz-Walter Z, Tjondronegoro D and Wyeth P (2011) Orientation passport: using gamification to
engage university students. In: Proceedings of the 23rd Australian computer-human interaction con-
ference, Canberra, Australia, 28 November–2 December 2011, pp.122–125. New York: ACM.
Fu FL, Su RC and Yu SC (2009) Egameflow: a scale to measure learners enjoyment of e-learning
games. Computers and Education 52(1): 101–112.
Garris R, Ahlers R and Driskell JE (2002) Games, motivation, and learning: a research and practice
model. Simulation and Gaming 33(4): 441–467.
Gee JP (2013) Games for learning. Educational Horizons 91(4): 16–20.
Giora R, Fein O, Kronrod A, et al. (2004) Weapons of mass distraction: optimal innovation and
pleasure ratings. Metaphor and Symbol 19(2): 115–141.
Hamari J, Koivisto J and Sarsa H (2014) Does gamification work? A literature review of empirical
studies on gamification. In: 2014 47th Hawaii international conference on system sciences, Waikoloa
Hawaii, USA, 6–9 January 2014, pp.3025–3034. New York: IEEE.
Hatala R, Cook DA, Zendejas B, et al. (2014) Feedback for simulation-based procedural skills train-
ing: a meta-analysis and critical narrative synthesis. Advances in Health Sciences Education
19(2): 251–272.
Heidt CT, Arbuthnott KD and Price HL (2016) The effects of distributed learning on enhanced
cognitive interview training. Psychiatry, Psychology and Law 23(1): 47–61.
Huotari K and Hamari J (2012) Defining gamification: a service marketing perspective. In: Proceeding
of the 16th international academic MindTrek conference, Tampere, Finland, 3–5 October 2012,
pp.17–22. New York: ACM.
Kapp KM (2012) The Gamification of Learning and Instruction: Game-based Methods and Strategies for
Training and Education. San Francisco, CA: Pfeiffer.
Khan A, Ahmad FH and Malik MM (2017) Use of digital game based learning and gamification in
secondary school science: the effect on student engagement, learning and gender difference.
Education and Information Technologies 22(6): 2767–2804.
Kluger AN and DeNisi A (1996) The effects of feedback interventions on performance: a historical
review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin
119(2): 254–284.
Krebs P, Prochaska JO and Rossi JS (2010) A meta-analysis of computer-tailored interventions for
health behavior change. Preventive Medicine 51(3): 214–221.
Kroeze W, Werkman A and Brug J (2006) A systematic review of randomized trials on the effective-
ness of computer-tailored education on physical activity and dietary behaviors. Annals of
Behavioral Medicine 31(3): 205–223.
Lee JJ and Hammer J (2011) Gamification in education: what, how, why bother? Academic Exchange
Quarterly 15(2): 146.
Li W, Grossman T and Fitzmaurice G (2012) Gamicad: a gamified tutorial system for first time
autocad users. In: Proceedings of the 25th annual ACM symposium on user interface software and
technology, Cambridge Massachusetts, USA, 7–10 October 2012, pp.103–112. New York: ACM.
Lustria MLA, Noar SM, Cortese J, et al. (2013) A meta-analysis of web-delivered tailored health
behavior change interventions. Journal of Health Communication 18(9): 1039–1069.
Malone TW (1982) Heuristics for designing enjoyable user interfaces: lessons from computer games.
In: Proceedings of the 1982 conference on Human factors in computing systems, Gaithersburg
Maryland, USA, 15-17 March 1982, pp.63–68. New York: ACM.
Welbers et al. 107
Malone TW and Lepper MR (1987) Making learning fun: a taxonomy of intrinsic motivations for
learning. In: Snow RE and Farr MJ (eds) Aptitude, Learning, and Instruction. vol. 3, London,
England: Routledge, pp.223–253.
Mollick ER and Rothbard N (2014) Mandatory fun: consent, gamification and the impact of games at
work. The Wharton School Research Paper Series. Epub ahead of print 4 December 2017. DOI:
Nah FFH, Zeng Q, Telaprolu VR, et al. (2014) Gamification of education: a review of literature. In:
International conference on HCI in business, Heraklion Crete, Greece, 22-27 June 2014, pp.401–409.
Switzerland: Springer International Publishing.
Neff R and Fry J (2009) Periodic prompts and reminders in health promotion and health behavior
interventions: systematic review. Journal of Medical Internet Research 11(2): e16.
Noble N, Paul C, Carey M, et al. (2015) A randomised trial assessing the acceptability and effective-
ness of providing generic versus tailored feedback about health risks for a high need primary care
sample. BMC Family Practice 16: 95–103.
Rohrer D (2015) Student instruction should be distributed over long time periods. Educational
Psychology Review 27(4): 635–643.
Schneider F, de Vries H, Candel M, et al. (2013) Periodic email prompts to re-use an internet-delivered
computer-tailored lifestyle program: influence of prompt content and timing. Journal of Medical
Internet Research 15(1): e23.
Su CH and Cheng CH (2015) A mobile gamification learning system for improving the learning
motivation and achievements. Journal of Computer Assisted Learning 31(3): 268–286.
Sweetser P and Wyeth P (2005) Gameflow: a model for evaluating player enjoyment in games.
Computers in Entertainment 3(3): 3.
Terill B (2008) My coverage of lobby of the social gaming summit [Blog post]. Available at: www. (accessed October 12 2017).
Wattal S, Telang R, Mukhopadhyay T, et al. (2012) What’s in a “name”? Impact of use of customer
information in e-mail advertisements. Information Systems Research 23(3): 679–697.
Author Biographies
Kasper Welbers is a postdoctoral researcher at the Department of Communication Science
at Vrije Universiteit Amsterdam. His research focuses on how news distribution processes
have changed due to the proliferation of new media technologies.
Elly A Konijn is a full professor in Media Psychology at the Department of Communication
Science at Vrije Universiteit Amsterdam and chair of the program Media Psychology
Amsterdam. Her research program moves along three main lines: 1) Relating to media
figures, virtual humans, and social robots. 2) Emotions and media-based reality perceptions.
3) Media use among adolescents (e.g., cyberbullying, violent video gaming, thin-body ideal,
self-presentation). See:
Christian Burgers is an associate professor in the Departement of Communication Science at
Vrije Universiteit Amsterdam. His research focuses on strategic communication, with an
emphasis on framing and figurative language across domains of discourse.
Anna Bij de Vaate is a PhD-candidate and lecturer at the Department of Communication
Science at the Vrije Universiteit Amsterdam. She studies how enhanced visual online self-
presentation influences well-being either positively or negatively. She is a member of the
Media Psychology Program Amsterdam.
108 E-Learning and Digital Media 16(2)
Allison Eden is an assistant professor in the Department of Communication at the Michigan
State University. Her work focuses on understanding media enjoyment from a psychological
perspective. She focuses on the role enjoyment plays in attention to and selection of media
content, and more broadly the effects of entertainment on behavior and well-being.
Britta C Brugman is a PhD candidate at the Department of Communication Science at Vrije
Universiteit Amsterdam. She studies the nature and effects of figurative language in political
Welbers et al. 109
... There is a considerable volume of publication on these three domains; as such, scientometrics and collaboration network analysis and plotting can provide a comprehensive view of the status of knowledge in these areas. However, in gaming, most of the previous studies do not consider gamification, GBL, and SG together, and solely focus on one or two types of games (21,(25)(26)(27)(28). The current study can practically represent the relationships at the level of authors, journals, time intervals, keywords, etc. in the research literature of these domains. ...
... The entry of z-generation learners (born from the mid-1990s to the early 2000s) whose birth coincided with the expansion of the World Wide Web (24) to universities may have affected the use of educational games in educational processes, thereby extending research and publications. The results indicated a rising trend of articles on gamification since 2016; similarly, recent studies confirm that the application of gamification to engage and motivate target learners has had a rising trend with successful outcomes in recent years (22,(25)(26)(27)(28). A study examining gamification publications in Google Scholar, Scopus, and Web of Science reported that the number of publications is increasing, and >60% of them are Scopus-indexed . ...
... A study examining gamification publications in Google Scholar, Scopus, and Web of Science reported that the number of publications is increasing, and >60% of them are Scopus-indexed . Evidence suggests that educational gamification in medical education promotes players' cognitive, psychomotor, and emotional competencies (28). According to experts, gamification is a suitable tool for the professional promotion of those working in medical professions and provides an opportunity for interdisciplinary education (29). ...
Full-text available
Introduction: Game in education aims to enhance human motivation and performance in a given activity. Gamification experts and health researchers are still unsure about the status of progress of game for health. So, to fill in this gap, the present study aimed to analyze scientific productions to identify production trends, subject areas, countries, institutes, and authors in these three areas on gamification, game-based learning, and serious games in medical education, as well as to determine co-authorship patterns. Methods: The present descriptive quantitative research was conducted through scientometric analysis by using co-authorship networks in gamification, game-based learning, and serious games. First, an advanced search was performed from 1990 to 2020 and the studies were retrieved from Web of sciences, on Aug 17, 2021 The plain text format of data was inputted to Microsoft Excel, CiteSpace and Gephi to analyze scientometric maps for the three domains. Subsequently, the required indicators to review co-authorship networks were obtained: Degree centrality, Betweenness centrality, Closeness centrality, Density, Clustering coefficient, collaboration index and collaboration coefficient. Results: There were 466 documents in gamification, 155 documents in game-based learning, and 295 documents in serious games. The results indicated the rising trend of scientific publications on the three domains. US was a prolific country in all three domains. Author collaboration has remarkably increased, although the number of single-author articles is still high. Conclusion: Due to the increasing growth of publications on these three domains, research can be continued by forming specialized groups and supporting joint publications. Also, research policy-makers should promote author collaborations on the national and international scale.
... At the same time, gamification has seen a surge in popularity in the educational sector. It has been used as a tool to engage students and encourage learning in a range of contexts [18]. This presents an opportunity to investigate the potential of gamification to scaffold coding learning and increase the sense of belonging among Chinese international students in introductory programming courses. ...
Full-text available
Programming skills are rapidly becoming essential for many educational paths and career opportunities. Yet, for many international students, the traditional approach to teaching introductory programming courses can be a significant challenge due to the complexities of the language, the lack of prior programming knowledge, and the language and cultural barriers. This study explores how large language models and gamification can scaffold coding learning and increase Chinese students' sense of belonging in introductory programming courses. In this project, a gamification intelligent tutoring system was developed to adapt to Chinese international students' learning needs and provides scaffolding to support their success in introductory computer programming courses. My research includes three studies: a formative study, a user study of an initial prototype, and a computer simulation study with a user study in progress. Both qualitative and quantitative data were collected through surveys, observations, focus group discussions and computer simulation. The preliminary findings suggest that GPT-3-enhanced gamification has great potential in scaffolding introductory programming learning by providing adaptive and personalised feedback, increasing students' sense of belonging, and reducing their anxiety about learning programming.
... Another possibility is that leaderboards start with intrinsic motivation and then turn to extrinsic motivation with prolonged play. If this is the case, it would be strongly linked to the inverted U-shape relationship between extended use and performance, a pattern often observed in gamification systems (Amo et al., 2020;Welbers et al., 2019;Yang and Li, 2021). ...
Purpose Gamification is a booming motivational approach in information systems. Leaderboards play a key role in gamification; however, there are mixed findings regarding the heterogeneous motivational impacts of leaderboard positions. This study aims to clarify the motivational effects of high and low leaderboard positions by assembling diverse behavioral measures and self-reports. The measures used in this study shed a light on the quantitative and qualitative dynamics of motivation facilitated by leaderboard positions. The authors inspect motivation in relation to satisfaction and frustration of competence need. Design/methodology/approach The authors conducted an online experiment set in a crowdsourcing context, asking the participants to compete in an image tagging game. Participants' leaderboard positions were manipulated to be either high or low for five consecutive rounds. The number of clicks, tags, duration of tagging and persistence on the task were measured as indicators of motivation. Findings High ranks on leaderboards induced complacent behaviors choosing easy ways to maintain their positions, while low ranks led the participants to stick to the right process of the task with intensified motivation round after round. However, neither of the motivations seemed to be of intrinsic nature. Originality/value The present study provides conclusive evidence on the varying motivational impact of leaderboard positions. The authors also demonstrate how the “needs-as-motive” model (Sheldon and Gunz, 2009) applies to gamification. Its implications in self-determination theory and gamification literature are discussed.
... Gamification itself is still in relation but not identical with the model of game-based learning, it's referring to the usage of game design components in a non-game framework or situation. This game-based learning referring to the usage of real and actual game to acquire the skills, knowledge and competencies [6]. ...
Industrial 4.0 has created changes in so many aspects of human life which require people not only to master their field of expertise but also to keep themselves at par with the latest trend and technology. Changes that happen rapidly, creating a volatile and complex condition with high level of uncertainty and ambiguity, which known as VUCA era. This state forcing workforces to be able to adapt in fast pace. E-learning which combines the traditional teaching process with the use of technology such as internet, Learning Management System (LMS) considered as the appropriate method to be implemented in order to meet all the challenges and requirements of industry 4.0. In retail industry, PT XYZ. and its subsidiaries have been utilizing Learning Management System (LMS), MOOC (Massive Open Online Courses), and virtual classes as part of the development program given to the employees. The main goal is to ensure that the employees as their most important asset will be able to keep themselves updated to the latest trend, topic and communication from the company. While the intention of this article is to see how the impact of E-Learning for people development is in this Industry 4.0 era, particularly in Indonesia’s largest lifestyle retail company.
... Digital books that are used as learning media can use games or games or the term gamification. Gamification in recent years has developed very quickly and has begun to penetrate various business fields such as web designers and even workers who have something to do with education such as teachers have started using gamification as a medium specifically developed to achieve learning outcomes from learning (Welbers et al., 2019). This gamification can be used on mobile phones and can be made online or offline or both at once. ...
... Moreover, inside of a game, other aspects and be added to make students think or know about very different topics. This is the case of reference [33], where the app proposed by authors aims to introduce students in information about their university, i.e., about regulations, socials life at the campus, evaluation regulations, etc. ...
In recent years, several active methodologies and approaches are being used at the university level. Some courses are easily adapted to different teaching and learning methods. In the case of an optional subject about Cryptography, the use of game-based learning is certainly widely accepted by students and faculty. The use of games for learning engages students and makes them pay attention to the contents and competencies while playing. Biometric systems are included in an initial course about cryptography and an escape game was proposed to students. During this game students learn about biometric systems and cryptography while acquire soft skills.
Purpose This rapid review is based on 14 articles published in the year 2019 and 2020 that report on the flipped classroom. This review study aims to examine underpinning theories to design flipped classroom and the models of the flipped classroom utilized in the different disciplinary, curricular, and technological contexts. Design/Approach/Methods The ProQuest database was used to search the articles for the study with the keyword “flipped classroom” and “Flipped learning.” Cochrane rapid review method was used as a methodology to investigate theoretical underpinnings, models, and the disciplinary and technological context of the studies. Findings This study found that flipped classroom approach was practiced in various disciplinary contexts and was mostly based on the theory of constructivism. Technological use in the flipped classroom models ranges from just using videos to fully online using different applications and learning management systems. The findings of the studies showed that there is a positive impact of the flipped classroom on learning, motivation, and anxiety. Originality/Value This study utilized the Cochrane rapid review method in the educational setting.
The aim of this study is to investigate the effect of mathematics activities enrıchment with game elements chosen according to the students’ learner profiles on mathematics achievement, attitudes, and motivation toward mathematics. The importance of this study is to define learners students’ learner profiles and design mathematics activities enrichment with game elements. A user profiles test tool was developed for this purpose. In the study, the level determination was applied to the students who were studying in the 6th grade of a private school, and the experimental and control groups were determined. There were 12 students in experimental and 24 students in the control group, so the number of participants was 24. After the determination of the groups, academic achievement, attitude, and motivation tests were applied as a pre-test in experimental and control groups. Additionally, in the experimental group, learning profiles and player types tests were used. As a result of these data, game elements were defined. Mathematics activities enrichment with those game elements was designed and the course had been taught. As a result of the application, academic success, attitude, and motivation tests were applied as a post-test in the experimental and control groups again. The data obtained was analyzed and reported with qualitative methods. As a result of the analysis, the effectiveness of mathematical elements enrichment with game elements was examined. According to the data obtained from the study, the results were in favor of the experimental group according to motivation and attitudes toward mathematics lessons.
Gamification is the process of implementing a game strategy in the learning process. Gamification in the lecture environment increases creativity and interactivity and provides students with a sensation of accomplishment (Sense of accomplishment). The development of communication technology increased rapidly, which impacted the development of the games industry, which attracted Indonesian people in this case. Therefore, gamification can be an alternative to represent innovative and exciting learning for students in Indonesia. In this Research, the authors tried to analyze the determinants of the effectiveness of learning using gamification Keywords: Gamification; Learning; Critical Success Factors; University Students; Learning Process eISSN: 2398-4287 © 2022. The Authors. Published for AMER ABRA cE-Bs by E-International Publishing House, Ltd., UK. This is an open-access article under the CC BY-NC-ND license ( Peer-review under the responsibility of AMER (Association of Malaysian Environment-Behavior Researchers), ABRA (Association of Behavioral Researchers on Asians), and cE-Bs (Centre for Environment-Behavior Studies), Faculty of Architecture, Planning & Surveying, Universiti Teknologi MARA, Malaysia.
Full-text available
This study aimed to identify the impact of a game based learning (GBL) application using computer technologies on student engagement in secondary school science classrooms. The literature reveals that conventional Science teaching techniques (teacher-centered lecture and teaching), which foster rote learning among students, are one of the major concerns in Pakistan Education system. This leads to student disengagement in science lessons eventually resulting in student absenteeism and dropouts from the schools. This study consisted of five stages: (1) examining the impact of Digital Game-Based Learning (DGBL) and gamification on engagement, learning and gender difference, and literature related to using DGBL models for instructional design; (2) planning learning activities and developing a GBL application based on a specific content in Science; (3) conducting an intervention with a sample of 72 participants of 8th grade (aged 12–15) in a low cost private school of Pakistan following quasi-experimental research framework; (4) observing behaviour and emotions of the participants during science lessons; (5) conducting pre and post tests to assess the learning outcomes of participants followed by focus groups discussion. Analysis from Friedman test, Mann-Whitney U test, and Wilcoxon Signed Rank test show that the GBL application has a positive influence on student engagement. However, GBL application was not equally effective for all students since girls outperformed boys in terms of engagement and learning outcomes. This study gives insights into the development of better educational games to promote student learning.
Full-text available
This study analyzes the application of game-based learning and gamification using MinecraftEdu, which allows for an exploration of the possibilities regarding immersive learning environments. We analyze the contributions of second-year university students who are pursuing a degree in Primary Education and are enrolled in a subject entitled Social Sciences II: History and Teaching at Castilla-La Mancha University. On four scales, we detail descriptive data and statistical inference through a quasi-experimental design using a Wilcoxon test and a sign test. The instruments provide content and construct validation based on data triangulation as a strategy. Despite the fact that participants consider video games as non-essential tools in an educational context, they value the fact that game-based learning through immersive environments allows for learning that involves a higher level of activity and engagement of the students. Interest level, educational innovation and motivation are valued positively and show statistically significant improvements.
Full-text available
In many academic courses, students encounter a particular fact or concept many times over a period of a few weeks and then do not see it again during the remainder of the course. Are these brief instructional periods sufficient, or should the same amount of instruction be distributed over longer periods of time? This question was the focus of several recent studies in which a fixed amount of instruction was distributed over time periods of varying duration and followed by a delayed posttest. With few exceptions, the results showed that longer instructional periods produced greater posttest scores if the posttest was delayed by at least a month or so. Notably, the search criteria for this review excluded several oft-cited studies favoring short foreign language courses over longer ones, but a closer look at these studies reveals limitations (e.g., no delayed posttest or confounding variables). In brief, the best reading of the data is that long-term learning is best achieved when the exposures to a concept are distributed over time periods that are longer rather than shorter.
Full-text available
Interview training for police officers is generally limited and, when it occurs, rarely translates into optimal interviews. Training ineffectiveness may be partly due to the structure of the training programme. In the present study, 60 participants received two hours of training on the Enhanced Cognitive Interview (ECI), in either a massed (one two-hour session) or spaced (two one-hour sessions) format. Following training, participants conducted an ECI. Advantages for spaced training were found in open-ended prompt use, perpetrator-specific details elicited from open prompts, and the utilization of two critical ECI components. These results suggest that a simple alteration in training protocols could improve forensic interviewing skills. © 2015 The Australian and New Zealand Association of Psychiatry, Psychology and Law
Full-text available
Tailored feedback has been shown to be effective for modifying health risk behaviours and may aid the provision of preventive care by general practitioners (GPs). However, provision of tailored patient feedback for vulnerable or socially disadvantaged groups is not well explored. The aims of this study were to examine the acceptability and effectiveness of providing generic compared to tailored feedback on self-reported health risk behaviours among a high need sample of people attending an Aboriginal Community Controlled Health Service (ACCHS). Participants attending two ACCHSs in regional New South Wales completed a touch screen health risk survey and received either generic or tailored health risk feedback. Participants were asked to complete an exit survey after their appointment. The exit survey asked about feedback acceptability and effectiveness. Self-reported ease of understanding, relevance and whether the generic versus tailored feedback helped patients talk to their GP was compared using Chi-square analysis; The mean number of survey health risks talked about or for which additional actions were undertaken (such as provision of lifestyle advice or referral) was compared using t-tests. Eighty seven participants (36 % consent rate) completed the exit survey. Tailored feedback was rated as more relevant and was more likely to be shown to the participant's GP than generic feedback. There was no difference in the mean number of health risk topics discussed or number of additional actions taken by the GP by type of feedback. Tailored and generic feedback showed no difference in effectiveness, and little difference in acceptability, among this socially disadvantaged population. Completing a health risk survey and receiving any type of feedback may have overwhelmed more subtle differences in outcomes between the generic and the tailored feedback. Future work to rigorously evaluate the longer-term effectiveness of the provision of tailored health risk feedback for Aboriginal Australians, as well as other high need groups, is still needed. Australian New Zealand Clinical Trials Registry ANZCTRN12614001205628. Registered 11 November 2014.
Full-text available
Gamification has been explored recently as a way to promote content delivery in education, yielding promising results. However, little is known regarding how it helps different students experience learning and acquire knowledge. In this paper we study and analyze data from a gamified engineering course, to search for distinct behavior patterns. We examined data collected from two gamified years, between which game changes took place. By clustering students according to their performance, we identified three distinct student types, common to both years: Achievers, Disheartened, and Underachievers. Interestingly, in the second year a new type of student emerged: the Late Awakeners. In this paper we carefully describe each student type, and explain how gamification can provide for smarter learning by catering to students with different profiles. Furthermore, we discuss how our findings, both in gamification and cluster analysis can be used to develop adaptive and smart learning environments.
Full-text available
Games are important vehicles for learning and behavior change as long as players are motivated to continue playing. We study the impact of verbal feedback in stimulating player motivation and future play in a brain-training game. We conducted a 2 (feedback valence: positive vs. negative) × 3 (feedback type: descriptive, comparative, evaluative) between-subjects experiment (N = 157, 69.4% female, Mage = 32.07). After playing a brain-training game and receiving feedback, we tapped players’ need satisfaction, motivation and intention to play the game again. Results demonstrate that evaluative feedback increases, while comparative feedback decreases future game play. Furthermore, negative feedback decreases players’ feeling of competence, but also increases immediate game play. Positive feedback, in contrast, satisfies competence and autonomy needs, thereby boosting intrinsic motivation. Negative feedback thus motivates players to repair poor short-term performances, while positive feedback is more powerful in fostering long-term motivation and play.
Conference Paper
We synthesized the literature on gamification of education by conducting a review of the literature on gamification in the educational and learning context. Based on our review, we identified several game design elements that are used in education. These game design elements include points, levels/stages, badges, leaderboards, prizes, progress bars, storyline, and feedback. We provided examples from the literature to illustrate the application of gamification in the educational context.
This paper aims to investigate how a gamified learning approach influences science learning, achievement and motivation, through a context-aware mobile learning environment, and explains the effects on motivation and student learning. A series of gamified learning activities, based on MGLS (Mobile Gamification Learning System), was developed and implemented in an elementary school science curriculum to improve student motivation and to help students engage more actively in their learning activities. The responses to our questionnaire indicate that students valued the outdoor learning activities made possible by the use of a smartphone and its functions. Pre- and post-test results demonstrated that incorporating mobile and gamification technologies into a botanical learning process could achieve a better learning performance and a higher degree of motivation than either non-gamified mobile learning or traditional instruction. Further, they revealed a positive relationship between learning achievement and motivation. The correlation coefficient for ARCS dimensions and post-test shows that the ARCS-A (attention) is greater than ARCS-R, ARCS-C and ARCS-S. This means that the attention (ARCS-A) of this system is an important dimension in this research. The results could provide parents, teachers and educational organizations with the necessary data to make more relevant educational decision.