Abstract and Figures

Several studies have been conducted in recent years into the effects of gamification on learner motivation. However, little is known about how learner profiles affect the impact of specific game elements. This research analyzes the effect of a gamified mathematic learning environment on the motivation and the motivated behaviors of 258 learners in secondary schools in France. Overall, results indicate that randomly assigned game elements generally demotivate learners. A more thorough analysis revealed that gamification has a positive impact on the most amotivated learners to do mathematic, although different effects were observed on learners. In particular, we noticed significant influences of their initial level of motivation and their player type on the variation in motivation during the study. We show that these influences vary according to the game element they used. These findings suggest that to increase efficiency, gamification should be tailored not only to the player profile but also to their level of initial motivation for the learning task.
Content may be subject to copyright.
1
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
AbstractSeveral studies have been conducted in recent years
into the effects of gamification on learner motivation. However,
little is known about how learner profiles affect the impact of
specific game elements. This research analyzes the effect of a
gamified mathematic learning environment on the motivation and
the motivated behaviors of 258 learners in secondary schools in
France. Overall, results indicate that randomly assigned game
elements generally demotivate learners. A more thorough analysis
revealed that gamification has a positive impact on the most
amotivated learners to do mathematic, although different effects
were observed on learners. In particular, we noticed significant
influences of their initial level of motivation and their player type
on the variation in motivation during the study. We show that
these influences vary according to the game element they used.
These findings suggest that to increase efficiency, gamification
should be tailored not only to the player profile but also to their
level of initial motivation for the learning task.
Index TermsInteractive learning environment, Gamification,
Learner motivation, Player types.
I. INTRODUCTION
n the last decade, gamification, which is commonly defined
as the use of game design elements within nongame
contexts [1], has promised to enhance human motivation and
engagement in different contexts such as education [2]–[4],
health [5], [6], and the workplace [7]–[9]. In education, Kapp
[10]–[11] argues that gamification serves several purposes such
as making learning easier from a cognitive and emotional point
of view, enabling automatic feedback, personalizing and
individualizing learning, and changing behaviors, but above all,
Manuscript received September 16, 2020; revised February 26, 2021;
accepted . Date of publication; date of current version. This work was supported
in part by the E-FRAN project, in part by the Caisse des Dépôts.
(Corresponding authors: Reyssier, S.; Hallifax, S.; Serna, A.; Marty, J.-C.;
Simonian, S.; Lavoué, É).
S. Reyssier is with the Laboratory Éducation, Cultures, Politiques,
University of Lyon, 69007 LYON (e-mail: stephanie.reyssier@univ-lyon2.fr).
S. Hallifax was with the Laboratoire d’InfoRmatique en Images et Systèmes
d’information, University of Lyon, 69007 LYON. He is now with the
Stratford School of Interaction Design and Business and the School of
Computer Science, University of Waterloo, ON N2L 3G1, CANADA (e-mail:
stuart.hallifax@uwaterloo.ca).
A. Serna is with Laboratoire d’InfoRmatique en Images et Systèmes
d’information, INSA Lyon, 69100 VILLEURBANNE (e-mail:
Audrey.serna@insa-lyon.fr).
encouraging learner engagement in the task, thus making
learners more active in their learning.
According to Nacke and Deterding [12], the first studies in
gamified education were essentially focused on the effect of a
set of game elements on users, which did not enable
identification of the impact of each game element taken
separately. These studies did not consider the individual
characteristics of learners, which can account for the different
and sometimes contradictory impacts of gamification observed
on learner motivation and engagement [13]–[14].
Several studies focus on the relationships between user
player type and game elements or game mechanics [15]–[18].
Generally, we adapt by assigning users to specific categories
and by providing different game elements for each category.
“Player type” represent a user’s preference for video games. For
instance, the Hexad [17] typology distinguishes six player types
(Players, Socializers, Free Spirits, Achievers, Philanthropists,
and Disruptors), and users are categorized into whichever type
they score highest. We can also consider all of the users’ scores
for each player type, thus creating their player profile.” Some
studies also consider that motivation can greatly affect the
effects of gamification [4]. However, no study has yet
considered these two aspects when evaluating the impact of
different game elements on learner motivation.
In this paper, we propose to study the impact of gamification
according to both learner initial motivation and player profile.
For this, we ran a largescale field study in four secondary
schools in France. 258 learners used a gamified mathematic
learning environment in their habitual classroom activities,
J.C. Marty is with Laboratoire d’InfoRmatique en Images et Systèmes
d’information, University of Savoie-Mont Blanc, 73000 CHAMBERY (e-
mail : Jean-charles.marty@univ-smb.fr).
S. Simonian is with the Laboratory Education, Cultures, Politiques,
Université of Lyon, 69007 LYON (e-mail: stephane.simonian@univ-lyon2.fr).
É. Lavoué is with the Laboratoire d’InfoRmatique en Images et Systèmes
d’information, Université of Lyon, 69007 LYON (e-mail : elise.lavoue@univ-
lyon3.fr).
The Impact of Game Elements on Learner
Motivation: Influence of Initial Motivation and
Player Profile
S. Reyssier, ECP, University of Lyon, S. Hallifax, LIRIS, University of Lyon, A. Serna, LIRIS, INSA
Lyon, J.-C. Marty, LIRIS, University of Lyon, S. Simonian, ECP, University of Lyon, and É. Lavoué,
LIRIS, University of Lyon
I
2
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
representing ten lessons and 45 exercises in literal calculation.
We then analyzed their usage data in order to determine the
impact of six game elements (score, badges, avatar, ranking,
progress, and timer). Each learner received randomly one of
these game elements, allowing the impact of game elements to
be studied according to learner initial motivation and player
profile.
The results of this study show that the gamification approach
generally demotivates learners. A more thorough analysis
revealed that gamification, although it might have positive
impacts on the motivation of learners who were initially less
motivated, might also have a negative impact on the most
motivated ones. Also, the impact of each game element on
learner motivation varies according to their initial level of
intrinsic motivation, extrinsic motivation, and amotivation.
This impact may also depend on player types. In general, we
recommend Avatar for learners with a high initial amotivation
or with a high Player score. We recommend Badges for
Disruptors and for learners who are initially intrinsically
motivated for mathematic. We recommend Progress for
learners with a high initial amotivation. We recommend
Ranking for high Free Spirit learners. We recommend Score for
learners with a high initial intrinsic motivation but who are also
strong Socializers. We recommend Timer for learners with high
initial amotivation or high Achiever or Free Spirit scores.
These results highlight the necessity to tailor gamification,
considering both player types and learner initial motivation
something that has been recently investigated in the field of
proenvironmental behavior by Vanhoudt et al. [19]. Finally,
we provide design recommendations on the game elements to
use in education according to these two factors.
II. RELATED WORK
A. Effects of Gamification on Learners
Theoretical foundations of research into the effects of
gamification on user motivation are mainly based on the self-
determination Theory (SDT) [20], which argues that human
beings are intrinsically motivated to engage in activities that
satisfy three basic psychological needs: competence (sense of
efficacy), autonomy (volition and personal agency), and
relatedness (social connectedness). SDT also argues that
humans strive to fulfil these needs in order to enhance well-
being. This theory is concerned with how individuals regulate
themselves to a greater or lesser extent depending on external
constraints. It explains why subjects are more or less self-
determined, and whether their motivations to act are intrinsic
and linked to the notion of pleasure or extrinsic and linked to
external contingencies (cognitive evaluation theory). In
particular, it has been established that rewards degrade intrinsic
motivation, and particularly those that act as controlling factors.
Meaningful gamification should spontaneously provoke the
satisfaction of these three user needs [21], especially the sense
of competence [1], and so enhance learner motivation.
However, as we will show in this section, the effects of
gamification on learner motivation are somewhat varied, and
often contradictory.
Hamari [22] showed that badges motivated users to increase
their activity in a trading/sharing app. Landers et al. [13]
demonstrated the effectiveness of leaderboards for simple tasks,
where they served as a goal setting tool for users. However,
their effectiveness dropped off as task difficulty increased.
Sailer et al. [23] tested two gamified situations in their order
picking simulation to motivate and engage participants with the
task: one using badges, leaderboards, and performance graphs,
and another using avatars, meaningful stories, and teammates.
They found that the first condition positively affected the
satisfaction of their competence needs and increased feelings of
task meaningfulness in participants. The second condition
increased feelings of social relatedness and relevance.
Regarding more particularly the educational domain,
Filsecker and Hickey [24] tested the effects of external rewards
on motivation and engagement in fifth graders. They expected
that the inclusion of external rewards would decrease intrinsic
motivation in their learners. They found that, by including these
rewards in a gameful-like manner, they could avoid the
expected decrease in intrinsic motivation and even increase
learner conceptual understanding of the studied topic. Kyewski
and Kramer [25] obtained more nuanced results when testing
badges in three different conditions (visible only by the learner,
visible to everyone, and no badges). They found that their
badges had less impact on learner motivation and performance
than they had initially assumed. Those badges, only visible by
the learner, were better evaluated than those that were visible
by everyone. In addition, in a study on how gamification affects
online learning discussion, Ding et al. [26] showed that learners
were more interested in the game elements directly linked to
their grades. Learners showed greater controlled motivation
(motivated by grades and instructor opinion) than autonomous
motivation (intrinsically motivated for learning). Also, Denny
et al. [3] tested the effect of badges and scores on learner
behavior. They found that only badges had an effect on how
participants behaved in their experiment, increasing the number
of self-assessments made. They also found that this directly
resulted in better examination performance for those
participants.
Several studies compare the impact of gamified and non-
gamified learning environments. For instance, Zainuddin et al.
[27] tested two versions of a flipped class setting: one with
gamification (points, badges, and leaderboards) and one
without. They found that learners provided with the gamified
environment had increased levels of perceived competence,
autonomy, and relatedness, better performance, and were able
to achieve better results during the tests. On the contrary,
Monterrat et al. [28] showed that learners who were free to use
a non-gamified learning environment had a higher level of
intrinsic motivation after the experimentation, compared to
learners using a gamified environment. Finally, Jagust et al.
[29] tested two adaptive situations. In the first situation, the
time learners had to answer questions changed depending on
how quickly they answered the previous question. In the second
situation, a target score changed depending on group
performance. In both situations, learners completed more tasks
3
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
than in a non-gamified situation, where the first situation had a
greater effect.
These results tend to show that there is no consensus on the
effect of gamification on learner motivation and that this effect
may vary according to the type of game elements used. This is
echoed, for example, in the study presented by Van Roy and
Zaman [30], who tested how a gamified system designed to
support students’ needs affected their motivation. They showed
that “the effects are highly personal and can differ widely
between different learners. A recent literature review of
gamification research by Koivisto and Hamari [31] also points
out that “while the results in general lean towards positive
findings ... the amount of mixed results is remarkable”.
B. Learner Characteristics Influencing the Effect of
Gamification
Many studies investigate the effects of tailored gamification
on user motivation depending on different user characteristics.
For instance, Orji et al. [32] showed that the different
motivational strategies implemented in game elements affect
different categories of users based on their BrainHex player
type [15] (archetypal reasons for which users play and are
motivated to play games). Recently, Hallifax et al. [41] ran a
crowdsourced study to explore factors that should be
considered for tailored gamification. They tested 12 different
game elements in a contextless setting and compared their
results to those in various studies from the related literature (in
educational, health, and sport settings). They showed that the
choice of player profile and the user activity or domain have a
major impact on how gamification affects user motivation, as
many of their findings from this contextless setting are different
from those found in the specific contexts. They also showed that
the Hexad [17] player profile is the most appropriate for tailored
gamification.
Several studies were conducted in the educational domain.
For instance, Roosta et al. [33] presented learners with different
game elements based on their “motivation type.They used a
questionnaire called the Elliot Achievement Goal
Questionnaire, which provides an assessment of users
achievement motivations. Learners used an online tool for one
month. The authors found that learners, who had game elements
suited to their motivation type, showed significant differences
in motivation, engagement, and quiz results compared to
learners who had randomly assigned game elements. They used
learner participation rates in the online activities as a metric to
gauge motivation and engagement. Kickmeier-Rust et al. [34]
ran a study where learners used an adaptive gamified system
over two sessions to learn divisions. The system was adapted to
learner behavior (the amount and number of mistakes made by
learners) by tailoring the game element feedback. According to
the authors, the personalized system reduced the amount of
errors that learners made. Learners using the adaptive situation
showed a greater decrease in errors made in the second session
than learners who used the nonadaptive situation.
Other studies based their adaptation on the learner player
type. Mora et al. [35] reported a general positive impact from
their adaptation based on the Hexad profile, with an increase in
behavioral and emotional motivation in learners who used a
personalized gamified collaborative problemsolving tool.
Lavoué et al. [4] also showed that, amongst the most engaged
learners, those with adapted game elements depending on their
player profile (BrainHex typology) spent more time on the
online learning environment.
All these studies highlight the need to consider learner
characteristics, such as player type and initial motivation, when
providing learners with game elements. In line with these
studies, the aim of this paper is to understand how learner player
type and initial motivation influence the impact of specific
game elements on their motivation.
III. RESEARCH QUESTIONS
In our study, we propose to answer the following questions:
--RQ1 How does gamification affect learner motivation? We
studied the variation in learner motivation from the beginning
to the end of the course. We split learners into subgroups based
on which game element they used in order to evaluate how each
game element affected their motivation, as well as the number
of motivated behaviors generated.
--RQ2 How do individual learner characteristics influence
the impact of each game element on their motivation? We more
particularly studied the influence of player profile types and the
initial level of motivation scores on the variation in motivation,
as well as the motivated behaviors they generated.
IV. LEARNING ENVIRONMENT
The participants used a gamified version of the Moodle
learning management system called “LudiMoodle” (see Fig. 1),
that was developed for the project. In total, it proposes six
different game elements, designed in collaboration with the
teachers involved in the project and improved thanks to learner
feedback. The six game elements used are described in Section
IV. B: Avatar, Badges, Progress, Ranking, Score, and Timer.
Fig. 1. The LudiMoodle platform: example of a gamified quiz. The upper part
shows a timer, while the lower part contains a quiz question
4
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
A. Learning Content
We built the learning content using a co-design method with
the participating teachers in order to remain as close as possible
to their usual teaching practices. In total, ten lessons were
designed to cover the topic of basic algebra (calcul litteral in
French). Each lesson is composed of 4 to 10 quizzes.
B. Game Elements
Each of the six game elements designed should appeal to at least
one of the different Hexad player types. It is important to note
that neither the Disruptor nor the Philanthropist types were
specifically targeted by one of our game elements. We made
this choice to avoid increasing the number of game elements
(restricting ourselves to six commonly used in the literature,
and in frameworks such as [1]), as both these types generally
form a minority [40].
1) Avatar
The Avatar game element showed a goblin-like character that
explored different universes (a different universe for each
lesson). As the learner progressed in a lesson, they would
unlock a different piece of clothing or an item that the character
was holding. There was one object to unlock per quiz (that was
unlocked after the learner correctly answered at least 70% of
the questions in the quiz). Game elements such as this are
generally recommended for Free Spirits, as these Avatars
provide them with a personalized representation of themselves
[17].
2) Badges
The badges game element proposed three levels of badges
per quiz. When the learners correctly resolved three different
levels of questions in the quiz (generally 7085100 % of each
quiz), they would unlock a new level of badge (bronzesilver
gold). An icon on the left-hand side showed how many badges
the student unlocked for the current lesson. Badges are
generally shown to be motivating for all users [41], but are
normally particularly effective for Players and Achievers, as
they represent clear-cut goals for them to achieve with attractive
rewards [17].
3) Progress
This game element showed different colored spaceships that
traveled from the earth to the moon. Each lesson launched a new
spaceship, and if the learner completed at least 70% of the
lesson, the spaceship would land on the moon. This game
element should prove particularly interesting for the Achiever
player type as, just like Badges, we have a clear goal [17].
4) Ranking
The learners assigned to this game element could compare
themselves to a fictional class of learners. The Ranking game
element showed a “race” where, as the learners answered
questions correctly, they would progress in the race at the same
pace as the other fictional learners. If they failed to answer a
question correctly, they would fall back in the ranking. We
calibrated the ranking system to ensure that a learner who
completed at least 70% of a lesson would finish in the top 50%
of the ranking to ensure they were not demotivated. As this
game element allows learners to compare themselves to others,
(even if fictional), it should be motivating for Socializers [17].
5) Score
Learners are awarded 1000 points for each correct answer
they give. Each lesson had its own score counter, with a detailed
view showing the number of points they scored for each quiz,
so that learners could pinpoint the missing points. As this game
element gives learners a clear representation of how well they
are doing in the course and rewards them for performing better,
it should prove attractive to Players [17].
6) Timer
This game element showed a timer for each quiz. Learners
were asked to try and beat a reference time” for each question.
The reference times were calculated based on the times for their
previous questions in the same quiz. Each time a learner beat
their reference time, an animation changed, showing a character
that ran faster and faster. Here, learners are challenged to beat
themselves in a race, thus making the Timer attractive to
Achievers [17].
TABLE I
PARTICIPANT INFORMATION
Class
Number in
class
Male/
female
Badges
Progress
Ranking
Score
Timer
A
24
15/9
4
5
3
2
5
B
22
12/10
2
4
3
5
4
C
22
9/13
3
5
3
3
4
D
22
12/10
3
5
3
4
3
E
19
11/8
3
3
4
3
3
F
22
7/15
2
4
5
5
2
G
20
7/13
3
3
4
2
4
H
22
8/14
4
4
3
4
4
I
23
11/12
4
4
3
4
3
J
18
9/9
3
4
2
3
3
K
23
13/10
5
4
3
3
4
L
21
9/12
3
3
4
4
3
All participants were native french speakers aged between 14 to 15 years old.
5
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
V. STUDY DESIGN
A. Participants
A total of five teachers and 258 students (14 to 15 years old)
in 12 classes (an average of 25 students per class), from four
different secondary schools, participated in the study (see Table
I). Teachers were involved in the co-design of the game
elements and in the construction of the course content. Game
elements were randomly distributed, while respecting parity
between genders, classes, and colleges. We ensured that there
was no class or gender effect at the outset. Learners were free
to discuss the game elements they received.
B. Material and Data
1) Motivated Behaviors
Learner interactions with the learning environment were
tracked using the Moodle data logging system. In order to
determine learner-motivated behaviors, we distinguished
learner-motivated behaviors that were different from normal
and expected behaviors induced by the pedagogical scenario:
--Restarted Quiz Count: we identified the number of
quizzes they retried after having completed them. Learners
were required to correctly answer at least 70% of each quiz to
access the next one. If a learner successfully completed a quiz
and then retried to achieve more than 70%, it showed that they
were particularly engaged to achieve a higher result.
--Question Ratio: we looked at the question ratio of correct
and incorrect answers given by the learners as a measure of their
cognitive involvement in the task.
2) Profile Questionnaires
We used the motivation scale proposed by Vallerand et al.
[36], inspired by SDT [20]. This scale, called the Academic
Motivation Scale (AMS), is composed of 28 items subdivided
into seven sub-scales, assessing, with a 5-point Likert-type
scale, seven dimensions of motivation (three for intrinsic
motivation (IM), three for extrinsic motivation (EM), and one
for amotivation (AMOT), following a continuum of self-
determination:
--Intrinsic Motivation for Knowledge, that is, performing
an activity for the pleasure and satisfaction of doing something
new: “I like learning new things.”
--Intrinsic Motivation for Accomplishment, that is,
performing an activity for the pleasure of overcoming a
challenge: “I like to see that I am able to solve problems.”
--Intrinsic Motivation for Stimulation, that is, performing
an activity for fun or excitement: “I really like math.”
--External Regulation, that is, performing an activity to gain
some kind of external rewards: “I want to get a good grade.”
--Introjected Regulation, that is, performing an activity to
avoid shame or increase self-esteem: “I want to prove that I can
do well in math.”
--Identified Regulation, that is, performing an activity in
order to achieve precise objectives: “I will be able to choose my
future studies thanks to math.”
--Amotivation, that is, the absence of intention to perform
an activity: “I don’t know why I went to math class; I feel like
I’m wasting my time.”
We identified the learner player profile using a translated
version of the original Hexad questionnaire [17], which defines
six player types, distributed along a continuum of “willing to
play”:
--Player, motivated by their personal success: “I like
competitions where a prize can be won.”
--Socializer, motivated by social contact: “Interacting with
others is important to me.”
--Free Spirit, motivated by creation and exploration: “It is
important to me to follow my own path.”
--Achiever, motivated by challenges: “I like overcoming
obstacles.”
--Philanthropist, whose goal is to help others: “It makes me
happy if I am able to help others.”
--Disruptor, motivated by change: “I like to provoke.
Some player profile types are correlated with some of the
basic psychological needs described in the SDT: Player and
Achiever profiles are positively correlated with the need for
competence; Philanthropist and Socializer profiles are
positively correlated with the need for relatedness; Free Spirit
and Disruptor profiles are linked to the need for autonomy.
3) Procedure
The experiment was conducted over three consecutive
TABLE II
MOTIVATIONAL VARIATIONS IN TOTAL AND PER GAME ELEMENT (W VALUES)
Game Element
All
Avatar
Badges
Progress
Ranking
Score
Timer
∆ Intrinsic motivations (∆ IM)
∆ Knowledge
-9.769
-4.627
-4.22
-3.747
-4.629
-3.829
-2.969
∆ Accomplishment
-1.235
-0.121
-2.217
-0.415
-0.703
-0.621
-0.197
∆ Stimulation
-1.261
-0.414
-1.278
-0.019
-1.882
-0.763
-0.33
∆ Extrinsic motivations (∆ EM)
∆ Identified reg.
-0.128
-0.082
-2.259
-0.197
-0.685
-1.211
-1.322
∆ Introjected reg.
-0.659
-0.54
-1.917
-0.534
-0.354
-0.209
-0.809
∆ External reg.
-6.209
-2.976
-3.363
-4.007
-1.448
-0.83
-2.536
∆ Amotivation (∆ AMOT)
∆ Amotivation
10.78
4.125
5.225
3.683
5.397
4.523
3.561
Values in gray are not significant (p > .05), values highlighted in light gray are significant (p < .05), values highlighted in dark gray are highly significant
(p < .01), and values highlighted in black are very significant (p < .001).
6
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
weeks. Each lesson was conducted in the same way: 10 to 15
minutes of written notes (these notes were handed out to
learners by the teachers to ensure that they had access to the
same content), followed by 25 to 30 minutes for answering
quizzes related to the lesson topic. The learners used an
individual tablet to access the quizzes. Teachers answered
technical questions asked by learners individually. Learners had
to answer both the AMS and Hexad questionnaires at the
beginning of the experiment (Pretest). At the end of the
experiment (Posttest), they only had to answer the AMS
questionnaire to be able to measure the variation in motivation.
4) Statistical Tests
To answer our first research question (RQ1), we compared
the score of each motivation subscale between the pretest and
the posttest, using a nonparametric Wilcoxon signed-rank test,
as our data were not normally distributed. We choose to indicate
whether the normalized W test is significant at .05, .01, and
.001, instead of the familywise error rate normally used for the
parametric test (see Table II). Motivational scores figure in
Table III.
To answer our second research question (RQ2), we used the
partial least squares path modeling (PLS PM) method [37] to
calculate the influence between the learner profile, defined by
the initial motivation scores and the player type scores, and both
the variations in motivations and the number of motivated
behaviors. PLS PM is a method of structural equation modeling
used to estimate complex causeeffect relationship models with
latent variables. It has already been used in research studying
the effects of gamification on user motivation [18], [38]–[41].
This method provides us with an estimation of the extent of the
influence of an input value on an output value. As this is a
statistical evaluation, we use the calculated pvalue to
determine the validity of the given influences. Our model is
illustrated in Fig. 2.
The inner model is represented by the independent variables
relative to the scores of initial motivations and player profiles,
which, when grouped together (only for motivations), allow
measurement of latent variables (IM, EM, AMOT). For
example, the three intrinsic motivation scores: Knowledge,
Accomplishment, and Stimulation were linked to create a
general initial Intrinsic Motivation construct (IM). The outer
model is represented by the dependent variables relative to the
variations in scores of motivations between pre and posttest
(which, when grouped together, allow measurement of latent
variables, ∆IM, ∆EM, ∆AMOT), and the “motivated behavior”
latent variable (which groups the “restart” and “questions ratio”
variables). We verified the reliability, the internal consistency,
the convergent validity of all our latent variables in Table V,
and the coefficient of determination,
𝑅!
, for the latent variables
of the outer model in Table IV.
VI. RESULTS
A. RQ1. How does Gamification Affect Learner Motivation?
Our first analysis, using a nonparametric Wilcoxon signed-
rank test, was applied considering all learners. It shows a
significant decrease in intrinsic motivation to knowledge and in
external motivation to regulation, as well as a significant
increase in amotivation, at the end of the experimentation (see
Table II).
TABLE III
MOTIVATIONAL SCORES IN PRETEST AND POSTTEST
Pretest
Posttest
Average
SD
Average
SD
MICO
13.7907
3.55430
11.80
2.668
MIAC
15.4031
3.51875
15.19
3.686
MIST
11.9186
4.30604
11.75
4.062
MEID
14.8450
4.31980
14.80
4.028
MEIN
13.8992
3.58845
13.71
3.896
MERE
15.7248
3.41857
14.52
3.715
AMOT
6.92
3.314
9.76
2.856
TABLE IV
R SQUARED VALUES
Dependent Variables
∆ IM
.343
∆ EM
.12
∆ AMOT
.451
Motivated Behaviors
.026
Calculated for each dependent variable and showing the influence of both
the Hexad profile and the initial motivation on the final values of each of these
variables.
Fig. 2. Partial least squares path modeling analysis diagram. In rectangle
borders the outer model, and in circle borders the inner model. On the left: the
initial profile values of participants (i.e., Hexad profile, initial motivation for
mathematics). On the right the observed outcomes (i.e., motivational
variations and motivated behaviors)
7
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
We then investigated the variations in motivation according
to the game elements used, performing a nonparametric
Wilcoxon test. These analyses confirmed the results obtained
considering all game elements: there is a decrease in motivation
for all game elements, nonsignificant for the Ranking and Score
game elements. For those elements, we do not notice any
significant decrease in external regulation. Finally, we observed
a decrease in intrinsic motivation for accomplishment and
identified regulation with the badges game element (see Table
II).
We also noted significant differences in motivated behaviors,
depending on the game elements received. Regarding the
question ratio, the results highlight a significant difference (p =
.04 < .05) between learners who used a Timer and those who
received Badges, with a higher ratio of correct answers for
Badges (see Fig. 3(a)).
Concerning the restarted quiz count, we noted that learners
who used a Timer restarted significantly less often than learners
who received the Progress, Ranking or Score game elements
(see Fig. 3(b)).
B. RQ2. What Learner Characteristics Influence how Each
Game Element Impacts Their Motivation?
1) PLS Model
We performed a PLS PM in order to examine the influence
of the “initial motivation” and “player profile” factors on the
motivational variations and motivated behaviors during the
experiment. The Path model used is described in Section IV.
Based on the PLS Path analysis, we noted the importance of
taking into account the initial motivations and the learner player
profile, as 34.3% of the variation in intrinsic motivation, 12%
of the variation in extrinsic motivation, and 45.1% of the
variation in amotivation, could be accounted for by the level of
initial motivations and the learner player profile (see Table IV).
We also noted that the input variables only account for 2.6% of
the motivated behavior of the learners. We wanted to know the
weight of each of these input variables in these final variations.
We generated T-statistics to test the significance of both the
inner and the outer model (see Table V), using a bootstrapping
method [42].
TABLE V
RESULTS SUMMARY FOR OUR REFLEXIVE INNER AND OUTER MODELS
Latent variables
Indicators
Loadings
Composite reliability
AVE
Rho A
Init IM
Init Knowledge
.922
.919
.791 > .5
.868 > .7
Init Accomplishment
.851
Init Stimulation
.893
∆ IM
Knowledge
.874
.803
.580 > .5
.760 > .7
Accomplishment
.639
Stimulation
.752
Init EM
Init Identified reg.
.770
.818
.603 > .5
.705 > .7
Init Introjected reg.
.878
Init External reg.
.688
∆ EM
Identified reg.
.679
.795
.566 > .5
.635 < .7
Introjected reg.
.842
External reg.
.725
Indicator reliability > .70; internal consistency reliability > .70 or .60 in an exploratory research; convergent validity > .50.
(a) (b)
Fig. 3. Distributions for the motivated behavior metrics per game
element. (a) Average question ratio. (b) Average restarted quiz
count.
8
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
2) Effect of Initial Motivation on The Variation in
Motivation
Results (see Table VI) show a negative influence of the level
of amotivation on the variation in amotivation. Moreover, the
initial level of amotivation has a positive influence on the
variation in intrinsic motivation. These two influences mean
that the more amotivated a learner is initially, the less
amotivated and the more motivated intrinsically they are at the
end. We also notice that the level of initial intrinsic motivation
negatively influenced the variation in intrinsic motivation, and
that the level of extrinsic motivation influenced negatively the
variation in extrinsic motivation. This means that the more a
learner is intrinsically or extrinsically motivated initially, the
less motivated they are at the end for this motivation type.
3) Effect of Player Profile on The Variation in Motivation
Results (see Table VII) show contrasting effects depending
on the player profile considered. We noted a significant
increase in both intrinsic and extrinsic motivation for the
Achiever, with a significant decrease in amotivation and a
positive influence on motivated behaviors. The Player score
seems to increase both intrinsic motivation and extrinsic
motivation. The Free Spirit score also increases extrinsic
motivation. Finally, the Socializer, Disruptor, and
Philanthropist scores show no significant influence.
4) Different Effects Depending on The Game Element
For each game element used, we ran a PLS path analysis to
determine the influence of the initial motivation scores and the
learner Hexad player profile on both the variations in each
motivation type and the motivated behavior markers (see Fig.
2). These analyses were performed using groups of learners that
had the same game element when using the learning
environment. This enabled us to acquire a more precise insight
into how each of these game elements impacted the variations
in motivation types and which player profile types contributed
to these variations, even if these results deserve to be
consolidated with larger samples.
--Avatar: We found four statistically significant influences
(see Table VII) for learners with the Avatar game element.
Learner initial amotivation score positively influenced the
variation in their intrinsic motivation, and negatively the
variation in their amotivation. The Player score also positively
influenced the variation in intrinsic motivation, while the
Socializer score only negatively influenced this same variation
in motivation.
--Badges: We only found two statistically significant
influences for learners who used badges (see Table VIII).
TABLE VI
RESULTS OF THE PLS PATH ANALYSIS USING THE ENTIRE LEARNER BASE
Init IM
Init EM
Init AMOT
Achiever
Player
Socializer
Free spirit
Disruptor
Philanthropist
∆ IM
-.698
.098
.156
.247
.193
-.048
-.006
-.022
-.064
∆ EM
.041
-.528
.004
.230
.132
-.025
.174
-.119
-.074
AMOT
.113
-.040
-.656
-.179
.095
.047
.087
.086
-.042
Motivated Behavior
.107
-.044
-.014
.193
-.015
-.129
-.003
.048
.104
Values in gray are not significant (p > .05), values highlighted in light gray are significant (p < .05), values highlighted in dark gray are highly significant (p <
.01), and values highlighted in black are very significant (p < .001)
TABLE VII
PLS PATH COEFFICIENTS OBSERVED FOR LEARNERS WHO RECEIVED THE AVATAR GAME ELEMENT
Init IM
Init EM
Init AMOT
Achiever
Player
Socializer
Free spirit
Disruptor
Philanthropist
∆ IM
-.407
-.029
.366
.145
.478
-.372
-.049
.006
.183
∆ EM
.431
-.390
.100
-.056
.296
-.363
-.018
-.028
-.091
∆ AMOT
.029
.168
-.426
-.111
.064
.041
.305
-.154
-.059
Motivated Behavior
.345
.055
.179
-.041
-.186
-.089
-.008
.139
.316
Values in gray are not significant (p > .05), values highlighted in light gray are significant (p < .05), values highlighted in dark gray are highly significant (p
< .01), and values highlighted in black are very significant (p < .001)
TABLE VIII
PLS PATH COEFFICIENTS OBSERVED FOR LEARNERS WHO RECEIVED THE BADGES GAME ELEMENT
Init IM
Init EM
Init AMOT
Achiever
Player
Socializer
Free spirit
Disruptor
Philanthropist
∆ IM
-.337
-.099
.085
-.007
.308
-.056
-.308
-.016
.091
∆ EM
.292
-.408
-.095
-.160
.095
-.206
-.198
-.181
.149
∆ AMOT
.497
-.111
-.361
-.419
-.117
.036
.199
-.013
-.036
Motivated Behavior
-.544
-.039
-.302
.097
-.053
.014
.136
.505
.322
Values in gray are not significant (p > .05), values highlighted in light gray are significant (p < .05), values highlighted in dark gray are highly significant (p
< .01), and values highlighted in black are very significant (p < .001)
9
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
Learner initial intrinsic motivation negatively influenced the
motivated behaviors, whereas their Disruptor score positively
influenced these behaviors.
--Progress: We observe three significant influences for
learners who used the Progress game element (see Table IX).
Each of the initial motivations negatively influenced the
variation in the same motivation type.
--Ranking: Results show many significant influences for the
Ranking element (see Table X). Initial intrinsic motivation
negatively influenced the variation in intrinsic motivation, and
positively the variation in amotivation. Initial extrinsic
motivation negatively influenced the variation in extrinsic
motivation. For the Achiever profile, Score negatively
influenced the variation in amotivation. The Free Spirit score
positively influenced the variation in extrinsic motivation,
whereas the Disruptor score negatively influenced it. The latter
score also positively influenced the variation in amotivation.
--Score: Score is the game element that showed the most
statistically significant influences (see Table XI). Initial
intrinsic motivation negatively influenced the variation in
intrinsic motivation, and positively the motivated behaviors
observed. Initial extrinsic motivation negatively influenced
both the variation in extrinsic motivation and the motivated
behaviors. Initial amotivation negatively influenced the
variations in intrinsic motivation and amotivation, as well as the
motivated behaviors observed. For the player profile, the
Socializer score positively influenced the variation in extrinsic
motivation and the Disruptor score had a similar effect on the
variation in amotivation. The Philanthropist score negatively
influenced the motivated behaviors observed.
--Timer: We found eight significant influences for this game
element (see Table XII). Initial intrinsic motivation negatively
influenced the variation in intrinsic motivation, while initial
amotivation negatively influenced the variation in amotivation.
For the player profile, the Achiever score positively influenced
the variation in intrinsic and extrinsic motivation, as well as the
motivated behaviors generated. The Free Spirit score positively
influenced the variation in extrinsic motivation. Finally, the
Philanthropist score negatively influenced the variations in both
intrinsic and extrinsic motivations.
TABLE X
PLS PATH COEFFICIENTS OBSERVED FOR LEARNERS WHO RECEIVED THE RANKING GAME ELEMENT
Init IM
Init EM
Init AMOT
Achiever
Player
Socializer
Free spirit
Disruptor
Philanthropist
∆ IM
-.466
-.143
.248
.150
.018
-.001
.055
-.039
-.140
∆ EM
.319
-.609
.108
.122
.116
.120
.323
-.396
-.172
∆ AMOT
.477
-.052
-.328
-.447
.163
.088
.018
.326
-.091
Motivated Behavior
-.016
-.125
.006
.225
-.326
-.199
.114
.263
-.036
Values in gray are not significant (p > .05), values highlighted in light gray are significant (p < .05), values highlighted in dark gray are highly significant (p <
.01), and values highlighted in black are very significant (p < .001).
TABLE IX
PLS PATH COEFFICIENTS OBSERVED FOR LEARNERS WHO RECEIVED THE PROGRESS GAME ELEMENT
Init IM
Init EM
Init AMOT
Achiever
Player
Socializer
Free spirit
Disruptor
Philanthropist
∆ IM
-.680
.229
-.108
-.048
.178
-.015
-.115
.374
-.028
∆ EM
.092
-.574
-.108
.234
.295
.068
.112
-.014
-.207
∆ AMOT
.122
-.092
-.853
-.231
.128
-.001
.043
-.071
-.026
Motivated Behavior
.018
-.165
-.076
.425
-.018
-.124
.022
-.063
.132
Values in gray are not significant (p > .05), values highlighted in light gray are significant (p < .05), values highlighted in dark gray are highly significant (p <
.01), and values highlighted in black are very significant (p < .001).
TABLE XI
PLS PATH COEFFICIENTS OBSERVED FOR LEARNERS WHO RECEIVED THE SCORE GAME ELEMENT
Init IM
Init EM
Init AMOT
Achiever
Player
Socializer
Free spirit
Disruptor
Philanthropist
∆ IM
-.909
-.098
-.615
.261
-.202
.208
-.032
.120
-.183
∆ EM
.304
-1.032
-.302
.250
.040
.465
-.030
-.289
-.202
∆ AMOT
-.342
.156
-.752
.041
.208
-.005
-.058
.490
.015
Motivated Behavior
.786
-.721
-.423
.071
-.127
.131
.168
.204
-.631
Values in gray are not significant (p > .05), values highlighted in light gray are significant (p < .05), values highlighted in dark gray are highly significant (p <
.01), and values highlighted in black are very significant (p <. 001)
10
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
VII. DISCUSSION
A. Gamification Approach that Generally Demotivates
This research allows us to draw meaningful conclusions
regarding the impact of gamification on learner motivation. We
first showed that randomly assigned game elements generally
result in a decrease in motivation. We found that the external
regulation of learners was lower after the experiment. One
possible explanation is that learners motivated by their
mathematic grades were frustrated that they did not receive any
grades for the completed quizzes completed during the
experiment (a choice made by the teachers for the experiment).
We also noticed a general decrease in intrinsic motivation for
knowledge, which raises questions about the perceived value of
the learning activity. It seems that learners perceived the
exercises more as a game than as a serious learning activity,
which echoes the findings by Barata et al. [16]. This may also
be due to the duration of the study, as teachers testified that
some learners were a little bored after ten quiz sessions.
We then showed that learner amotivation generally increased
for all learners regardless of the game element they used,
meaning that they found fewer reasons to do mathematic. This
result is similar to that found in a previous study we conducted
with a gamified learning environment dedicated to learning
French grammar [4]. Learners provided with game elements
that were not adapted to their player profile showed higher
levels of amotivation. This result could reflect one of the main
effects of the learning activity itself, merely moderated by
gamification.
Regardless of the game element used, we noticed a decrease
in intrinsic motivation for knowledge and external regulation,
except for learners who used the Ranking and the Score game
elements. This may be due to the fact that these game elements
closely emulated the feeling of receiving a grade for their work
(i.e., the Score gave a numerical rating of their performance,
while the Ranking showed them if they were performing better
than others). With Badges, we observed that more types of
motivation were negatively impacted compared to other game
elements (intrinsic motivation for accomplishment and
identified regulation). This corroborates the results presented
by Hanus et al. [43], which suggest that Badges and other
rewards are considered as controlling rewards, since they
encourage action but constrain it to the objectives proposed by
Badges. This perception could degrade learner intrinsic
motivation.
B. More Contrasting Effects Depending on Learner Profile
We show in Sections VI-B-2 and VI-B-3 that these general
effects on learner motivation vary depending on their initial
motivation and player profile.
1) The Influence of Initial Motivation
The negative influence between each type of motivation on
the variation in this motivation (e.g., initial intrinsic motivation
negatively influences the variation in intrinsic motivation)
highlighted the fact that gamification motivated learners who
were less motivated initially. Learners who were initially
intrinsically and extrinsically demotivated, were more
motivated after the experimentation, while those who were
more intrinsically and extrinsically motivated, were those most
demotivated at the end. This result has great implications for a
gamification approach not adapted to learners. Such an
approach should be used with extreme caution depending on
learner initial motivation for the discipline.
The analysis performed per game element (see Section
VI.B.4) allowed us to further investigate these results and to
show that game elements affect learners differently.
Among the positive influences, we noted that the Avatar
game element increased intrinsic motivation and decreased the
amotivation of the more amotivated learners. This result could
be accounted for by an increase in the satisfaction of their need
for social relatedness [20], as shown by Sailer et al. [23]. The
Progress game element also decreases the amotivation of the
most amotivated learners. This could be accounted for by an
increase in the feeling of competence from this game element
[20], [44]. The Score game element has a positive influence on
the variation in motivated behaviors of intrinsically motivated
learners, which could also be accounted for by a desire to do
better and to feel more competent. The fact that the Score game
element is, in this study, a non-controlling reward (learners
have the choice of restarting the exercise or not), contributes to
this increase in their intrinsic motivation [45]. Lastly, the more
amotivated learners, who received the Timer game element,
saw their amotivation decrease, suggesting once again that this
performance incentive was perceived more as an affirmation of
their need for competence [23], [45].
However, we found that certain game elements degraded the
motivation of some learners, a fact also observed when
considering the learners as an entire group. Learners with a high
level of intrinsic motivation who used the Badges game
element, experienced a decrease in their motivated behaviors.
This could be accounted for by the controlling nature of this
TABLE XII
PLS PATH COEFFICIENTS OBSERVED FOR LEARNERS WHO RECEIVED THE TIMER GAME ELEMENT
Init IM
Init EM
Init AMOT
Achiever
Player
Socializer
Free spirit
Disruptor
Philanthropist
∆ IM
-.571
-.101
.325
.639
.104
.180
-.170
-.228
-.366
∆ EM
-.421
-.111
.288
.689
-.015
.065
.318
-.013
-.407
∆ AMOT
-.325
-.140
-1.112
.097
.056
.124
.073
.011
-.226
Motivated Behavior
.120
.130
.287
.749
.313
.011
-.332
-.222
-.435
Values in gray are not significant (p > .05), values highlighted in light gray are significant (p < .05), values highlighted in dark gray are highly significant (p
< .01), and values highlighted in black are very significant (p < .001).
11
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
game element [43]. The Progress and Ranking game elements
degraded the levels of intrinsic and extrinsic motivation of
learners who had high initial levels of these motivations. These
results suggest that game elements that foster social comparison
[48]–[49] could be detrimental to learner motivation. The Score
game element also decreased the motivated behaviors of the
most amotivated learners, as well as their intrinsic motivation.
Learners may have perceived this game element more as
negative feedback [48], confronting them with their own
difficulties in mathematic. Finally, learners initially
intrinsically motivated to do mathematic, who received the
Timer, experienced a decrease in their intrinsic motivation. This
game element may have generated stress among the most
motivated learners, a fact that was also reported by teachers
following the experiment. This result is common with many
gamification studies that show Timers as stressful for learners
[40], [50].
To conclude, all types of initial motivation have an influence
on the variation in motivation regardless of the game element
used. These influences are mostly negative, meaning that the
more motivated learners are, the less they will be motivated.
However, the opposite is also true: the less motivated learners
are, the more motivated they will become. Only the variation in
amotivation and motivated behaviors are positively impacted
when using Avatar, Progress, Timer or Score. For the Avatar,
we assume that learners regarded the various items that they
could collect for their Avatar as a set of fixed goals, which upon
completion, satisfied both their needs for autonomy and for
competence (which explains the increase in their motivation
[21]). Progression, as an informational feedback, decreases the
amotivation of the most amotivated, which can be justified by
an increase in their feeling of competence [20]–[21]. Score
would have acted as a non-controlling reward since it reduced
the amotivation of the most amotivated. Learners could repeat
exercises to improve their score if they wished. Therefore, they
were not penalized by a score that might interfere with the
continuation of the exercise [20]. Finally, the Timer decreased
the amotivation of the most amotivated and increased their
other motivations (intrinsic and extrinsic). It acted as an
incentive for performance, which allowed the most amotivated
learners to satisfy their need for competence, while giving them
more pleasure [20], [44].
Based on these findings, we can conclude that game elements
do not have the same potential to affect learner motivation
according to their level of initial motivation, and thus that these
initial motivations must be considered if we do not want
gamification to be detrimental to learners. It is therefore
necessary to design gamification approaches that take into
account not only the learner player profile but also their interest
in the subject (i.e., initial motivation). Future research should
be conducted to investigate adaptation models that can combine
these two types of profile to offer effective adaptation [51].
2) The Influence of Initial Player Types
When looking at learner player profile, the most impactful
game elements vary considerably. The Timer had the greatest
impact, involving an increase in both the intrinsic and extrinsic
motivations for Achiever and Free Spirit learners. However, for
Philanthropists, this game element had the opposite effect,
generally demotivating them. These findings nuance the results
obtained in the study conducted by Hallifax et al. [41]
independently from a specific domain and context where they
conclude that Timers are generally less preferred and should be
avoided. The Ranking game element came next and showed
four influences. Learners with high Free Spirit scores gained in
extrinsic motivation, while Achievers became less amotivated.
As Achievers are motivated by competence [17], it is not
surprising that the virtual challenge of the ranking system
motivated them. Free Spirit learners possibly looked for a way
to “stand out from the crowd” [17] and therefore tried to come
first. However, learners with high Disruptor scores lost
extrinsic motivation and gained in amotivation. This game
element could have made them feel demotivated since such
learners are looking to go against the rules and will not be
challenged by the ranking system, which mainly highlights
learners who follow the rules. For Score, we observe positive
effects only on socializers with an increase in extrinsic
motivation. Both Disruptors and Philanthropists had,
respectively, an increase in their amotivation and a decrease in
their motivated behaviors. This is not surprising, as scoring
systems are generally not recommended for motivated learners
[41], [50]. This finding for Socializers is coherent with the
results obtained in [26] that noted that learners like to compare
their scores with others. With the Avatar game element, we
noted an increase in motivation for those with a high Player
score, as well as a decrease for those with a high Socializer
score. Being able to develop their Avatar based on their correct
answers was probably perceived by learners as a way to satisfy
their personal success [23]. As there were no possibilities to
show their Avatars to others, it is not surprising that a negative
effect is observed for Socializers. We also found that Badges
game elements encouraged motivated behaviors only for
learners with a high Disruptor score. This is surprising as
Badges are one of the most widely used game elements for
gamification [25], [31], [41] and are generally accepted as
motivating. Finally, no influences were observed for the
Progress game element from the player profile scores. This
result is contradictory with other studies, such as that conducted
in [41] that shows influences depending on the Socializer and
Disruptor player types. These differences may be due to the
design of the game element itself. These findings show that five
player types have an influence on the impact of game elements
on learner motivation, and that all types of motivation and
motivated behaviors are impacted, but in very different ways
depending on the game elements involved. The Achiever and
Disruptor player types have the most impact. These results
allow us to provide some recommendations, described in the
following section.
3) Game Element Recommendations
Based on all our findings, we can get a better understanding
of how each game element influences the variations in learner
12
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
motivation. This will allow us to make design recommendations
based on how learner motivation should vary with a game
element using initial motivation and Hexad type:
--We recommend Avatar for learners with a high initial
amotivation or with a high player score. This is similar to the
recommendation made by Tondello et al. [40], who suggest that
the player type may also prefer collections. However, Avatar
should be avoided for learners with high Socializer scores.
Thisdiffers from the recommendations made by Orji et al. [38],
who showed that the Socializer score has a positive influence
on all game elements tested in a gameful health system. This
could suggest that recommendations may vary depending on
the context or the gamified task as shown in [41].
--Badges should be used for Disruptors and should be
avoided for learners who are initially intrinsically motivated for
mathematic. While Hallifax et al. [41] proposed that Badges be
recommended for all users, they did not evaluate the influence
of user initial motivation. This reinforces the need to consider
this dimension when proposing such game elements.
--Progress is relevant for learners with a high initial
amotivation, as it increases both their intrinsic and extrinsic
motivations. Having clear goals or objectives is something that
is also recommended in other studies [26]. Similar game
elements are frequently recommended for various player types
(never all of them though) in [38], [40], [41].
--Ranking works for high Free Spirit learners, but should be
avoided for others, as learners with high intrinsic or extrinsic
motivation, amotivation, or with high Achiever or Disruptor
scores, will most likely feel demotivated by this game element.
Leaderboards have already been shown to be demotivating and
detrimental to learning in several studies [13], [41].
--Score can be recommended for learners with a high initial
intrinsic motivation but who are also strong Socializers.
However, as with ranking, this game element should be
generally avoided. Learners with high extrinsic motivation,
amotivation, or Disruptor or Philanthropist scores will find it
demotivating. This is in line with other studies such as [41], [50]
that show score game elements to be problematic. The positive
influence of the Socializer type should be nuanced as Orji et al.
[38] found positive influences for all game elements with
Socializers.
--Last but not least, Timer will generally be motivating for
learners with high initial amotivation, or high Achiever or Free
Spirit scores. However, learners with strong initial intrinsic
motivation, or Philanthropist scores might find this game
element demotivating. These results are quite different from a
previous study that shows that this game element motivates
learners with high intrinsic motivation [49]. However, it is in
line with other studies, such as that presented by Hallifax et al.
[41], which concludes that Timers are generally to be avoided.
Tondello et al. [40] showed that challenges work quite well for
Achievers, and it is possible that Timers in our context provided
enough challenge for Achievers.
VIII. LIMITATIONS
We identified a few limitations in our study that may affect
the generalization of our results. First in line are the domain and
the targeted users. While we focus specifically on secondary
school level Algebra, results may vary with a different subject,
with learners of another age and in another domain than
education. Next, as pointed out by Lessel et al. [52], the effect
of gamification widely varies for willing participants (i.e.,
participants performed better when they had a choice in using
the game elements). As the learners in our study did not choose
their gamification, this could have affected their motivation or
behavior. Finally, as pointed out by Ooge et al. [53], the Hexad
questionnaire may function sub optimally in a teenage
population in languages other than English. This could
potentially account for part of our results.
IX. CONCLUSION
In this paper we presented the results from a large-scale study
on how gamification affects learner motivation and motivated
behaviors. Our study ran for close to six weeks in four different
secondary schools in France. We analyzed the data used by 258
students from 12 different classes, from over ten specifically
designed mathematic lessons, gamified using six different game
elements. Our results show that, in general, gamification is
more adapted to less motivated or amotivated learners, who do
not perceive mathematic as interesting. Our results also
highlight the necessity to adapt gamification not only based on
a learner player profile as commonly acknowledged in the
literature, but also based on their initial motivation. Both these
factors are important for determining how a game element will
affect learner motivation or behavior. Furthermore, the results
obtained considering each game element separately highlight
that they affect learner motivation differently. Gamification
cannot be reduced to a behaviorist approach. Since the right
game element is adapted to the right learner profile, it promotes
self-determination. Care must be taken when proposing game
elements to learners, as these may have contradictory effects
depending on their profile.
These results open up new perspectives, such as an
adaptation to the learner profile. Some authors propose a static
adaptation, classifying individuals according to their profile
type, prior to the activity, using questionnaires such as
BrainHex [15], [41], [54], or Hexad [35], [40]. Others consider
that play preferences can change during activity because, as
Lazzaro [55] points out, the motivations to play are not fixed
and can change in the course of the day. Our future works will
be directed toward the analysis of learner engagement based on
their interaction traces with the learning environment in order
to recommend dynamically adapted game elements in situation
[29].
REFERENCES
[1] S. Deterding, “The Lens of Intrinsic Skill Atoms: A Method for Gameful
Design,” Human–Computer Interaction, vol. 30, no. 34, pp. 294335,
May. 2015, doi: 10.1080/07370024.2014.993471.
[2] R. N. Landers and A. K. Landers, “An Empirical Test of the Theory of
Gamified Learning: The Effect of Leaderboards on Time-on-Task and
13
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
Academic Performance,” Simulation & Gaming, vol. 45, no. 6, pp. 769
785, Dec. 2014, doi: 10.1177/1046878114563662.
[3] P. Denny, F. McDonald, R. Empson, P. Kelly, and A. Petersen, “Empirical
Support for a Causal Relationship Between Gamification and Learning
Outcomes,” in Proc. 2018 CHI Conference on Human Factors in
Computing Systems, (CHI’18), Montreal, QC, Canada, April 21-26, 2018,
2018, paper no. 311. doi: 10.1145/3173574.3173885.
[4] É. Lavoué, B. Monterrat, M. Desmarais and S. George, "Adaptive
Gamification for Learning Environments," in IEEE Transactions on
Learning Technologies, vol. 12, no. 1, pp. 16-28, 1 Jan.-March 2019, doi:
10.1109/TLT.2018.2823710.
[5] T. Lehto and H. Oinas-Kukkonen, “Persuasive features in web-based
alcohol and smoking interventions: A systematic review of the literature,”
J. Med. Internet Res., vol. 13, e46, Jul., 2011, doi: 10.2196/jmir.1559.
[6] Y. Jia, B. Xu, Y. Karanam, and S. Voida, “Personality-targeted
gamification: A survey study on personality traits and motivational
affordances,in Proc. 2016 CHI Conference on Human Factors in
Computing Systems, (CHI’16), San Jose, USA, 2016, pp. 20012013. doi:
10.1145/2858036.2858515
[7] D. R. Flatla, C. Gutwin, L. E. Nacke, S. Bateman, and R. L. Mandryk,
“Calibration games: Making calibration tasks enjoyable by adding
motivating game elements,” in Proc. 24th Annu. ACM Symp. User
Interface Software and Technology (UIST’11), Santa Barbara, CA, USA,
Oct. 1619, 2011, pp. 403412, doi: 10.1145/2047196.2047248.
[8] C. Eickhoff, C. G. Harris, A. P. de Vries, and P. Srinivasan, “Quality
through flow and immersion: Gamifying crowdsourced relevance
assessments,” in Proc. 35th Int. ACM SIGIR Conf. Research and
Development in Information Retrieval (SIGIR’12), Portland, OR, USA,
Aug.1216, 2012, pp. 871880. doi: 10.1145/2348283.2348400.
[9] F. Oprescu, C. Jones, and M. Katsikitis, I PLAY AT WORK—Ten
principles for transforming work processes through gamification,”
Frontiers psychol., vol. 5, 2014, Art. no 14, doi:
10.3389/fpsyg.2014.00014.
[10] K. M. Kapp, The gamification of learning and instruction: Game-based
methods and strategies for training and education, Hoboken, NJ, USA:
Wiley, 2012.
[11] K. M. Kapp, The gamification of learning and instruction fieldbook: Ideas
into practice, Hoboken, NJ, USA: Wiley, 2013.
[12] L. Nacke and S. Deterding, Editorial: The maturing of gamification
research,” Comput. Human Behav., vol. 71, pp. 450454, Jun. 2017, doi:
10.1016/j.chb.2016.11.062.
[13] R. Landers, K. Bauer, and R. Callan, “Gamification of task performance
with leaderboards: A goal setting experiment”, Comput. Human Behav.,
vol. 71, pp. 508515, Jun. 2017, doi: 10.1016/j.chb.2015.08.008.
[14] M. D. Hanus and J. Fox, Source customization reduces psychological
reactance to a persuasive message via user control and identity
perceptions,” J. Interact. Advertising, vol. 17, pp. 1–12, Feb. 2017, doi:
10.1080/15252019.2017.1287023.
[15] L. E. Nacke, C. Bateman, and R. L. Mandryk, “BrainHex: A
neurobiological gamer typology survey”, Entertainment Comput., vol. 5,
issue 1, pp. 5562, Jan. 2014, doi: 10.1016/j.entcom.2013.06.002.
[16] G. Barata, S. Gama, J. Jorge, and D. Gonalves, Studying student
differentiation in gamified education: A long-term study,” Comput.
Human Behav., vol. 71, pp. 550-585, Jun. 2017, doi:
10.1016/j.chb.2016.08.049.
[17] A. C. Marczewski, Even Ninja Monkeys like to play, London, United
Kingdom: Blurb Inc, 2015.
[18] R. Orji, L. E. Nacke, and C. DiMarco, “Towards personality-driven
persuasive health games and gamified systems,” in Proc. 2017 CHI
Conference on Human Factors in Computing Systems, (CHI’17), May
2017, pp. 1015-1027, doi: 10.1145/3025453.3025577.
[19] L. Van Houdt, M. Millecamp, K. Verbert, and V. Vanden Abeele,
“Disambiguating preferences for gamification strategies to motivate pro-
environmental behaviour,” in Proc. 2020 CHI Conf. on Comput.Human
Interact (CHI’ 2020), New York, NY, USA, November, 2020, pp. 241
253, doi: 10.1145/3410404.3414244.
[20] E. L. Deci and R. M. Ryan, “The ‘What’ and ‘Why’ of goal pursuits:
Human needs and the Self-Determination of behavior,Psychological
Inquiry, vol. 11, no. 4, pp. 227268, Oct. 2000, doi:
10.1207/S15327965PLI1104_01.
[21] S. Rigby and R. M. Ryan, Glued to games: How video games draw us, in
and hold us Spellbound, ABC-CLIO, Santa Barbara, USA, 2011.
[22] J. Hamari, Do badges increase user activity? A field experiment on the
effects of gamification,” Comput. Human Behav., vol. 71, pp. 469478,
Jun. 2017, doi: 10.1016/j.chb.2015.03.036.
[23] M. Sailer, J. U. Hense, S. K. Mayr, and H. Mandl, How gamification
motivates: An experimental study of the effects of specific game design
elements on psychological need satisfaction,” Comput. Human Behav., vol.
69, pp. 371380, Apr. 2017, doi: 10.1016/j.chb.2016.12.033.
[24] M. Filsecker and D. T. Hickey, A multilevel analysis of the effects of
external rewards on elementary students’ motivation, engagement and
learning in an educational game,Comput. & Educ., vol. 75, pp. 136148,
Jun. 2014, doi: 10.1016/j.compedu.2014.02.008.
[25] E. Kyewski and N. C. Krmer, To gamify or not to gamify? An
experimental field study of the influence of badges on motivation, activity,
and performance in an online learning course,” Comput. & Educ., vol. 118,
pp. 25-37, Mar. 2018, doi: 10.1016/j.compedu.2017.11.006.
[26] L. Ding, C. Kim, and M. Orey, Studies of student engagement in gamified
online discussions,” Comput. & Educ., vol. 115, pp. 126142 Dec. 2017,
doi: 10.1016/j.compedu.2017.06.016.
[27] Z. Zainuddin, Students’ learning performance and perceived motivation
in gamified flipped-class instruction,” Comput. & Educ., vol. 126, pp. 75-
88, Nov. 2018, doi: 10.1016/j.compedu.2018.07.003.
[28] B. Monterrat, M. Desmarais, E. Lavoué, and S. George, “A player model
for adaptive gamification in learning environments,” Artif. Intell. in Educ.,
vol. 9112, pp. 297306, Jun. 2015, doi: 10.1007/978-3-319-19773-9_30.
[29] T. Jagut, I. Botiki, and H.-J. So, Examining competitive, collaborative
and adaptive gamification in young learners’ math learning,” Comput. &
Educ., vol. 125, pp. 444457, Oct. 2018, doi:
10.1016/j.compedu.2018.06.022.
[30] R. van Roy and B. Zaman, Need-supporting gamification in education:
An assessment of motivational effects over time,” Comput. & Educ., vol.
127, pp. 283297, Dec. 2018, doi: 10.1016/j.compedu.2018.08.018.
[31] J. Koivisto and J. Hamari, The rise of motivational information systems:
A review of gamification research,” Int. J. of Inf. Management, vol. 45, pp.
191210, Apr. 2019, doi: 10.1016/j.ijinfomgt.2018.10.013.
[32] R. Orji, R. L. Mandryk, J. Vassileva, and K. M. Gerling, “Tailoring
persuasive health games to gamer type,” in: Proc. SIGCHI Conf. on Human
Factors in Computing Syst., ACM, 2013, pp. 24672476, doi:
10.1145/2470654.2481341
[33] F. Roosta, F. Taghiyareh, and M. Mosharraf, “Personalization of
gamification-elements in an e-learning environment based on learners’
motivation,” in 2016 8th International Symp. on Telecomm. (IST), pp. 637
642, doi: 10.1109/ISTEL.2016.7881899
[34] M. D. Kickmeier-Rust, E.-C. Hillemann, and D. Albert, “Gamification and
smart feedback: Experiences with a primary school level math app,” Int. J.
Game-Based Learn., vol. 4, pp. 3546, Feb. 2014, doi:
10.4018/ijgbl.2014070104.
[35] A. Mora, G. F. Tondello, L. E. Nacke and J. Arnedo-Moreno, "Effect of
personalized gameful design on student engagement," 2018 IEEE Global
Engineering Education Conf. (EDUCON), 2018, pp. 1925-1933, doi:
10.1109/EDUCON.2018.8363471.
[36] R. J. Vallerand, L. G. Pelletier, M. R. Blais, N. M. Briere, C. Senecal, and
E. F. Vallieres, “The academic motivation scale: A measure of intrinsic,
extrinsic, and amotivation in education, educational and psychological
measurement,” Educ. and Psychol. Meas., vol. 52, no. 4, pp. 10031017,
Dec. 1992, doi: 10.1177/0013164492052004025.
[37] M. Sarstedt, C. M. Ringle, and J. F. Hair, Jr., “Partial least squares
structural equation modeling,” in Handbook of Market Research, C.
Homburg, M. Klarmann, and A. E. Vomberg, Eds., Cham, Switzerland:
Springer, doi: 10.1007/978-3-319-05542-8_15-1.
[38] R. Orji, G. F. Tondello, and L. E. Nacke, “Personalizing persuasive
strategies in gameful systems to gamification user types,” in Proc. 2018
CHI Conference on Human Factors in Computing Systems, April 2018,
Paper no. 435, doi: 10.1145/3173574.3174009.
[39] R. Orji, J. Vassileva, and R. L. Mandryk, “Modeling the efficacy of
persuasive strategies for different gamer types in serious games for health,”
User Model. and User-Adapted Interact., vol. 24, pp. 453498, Jul. 2014,
doi: 10.1007/s11257-014-9149-8.
[40] G. F. Tondello, R. R. Wehbe, L. Diamond, M. Busch, A. Marczewski, and
L. E. Nacke, “The gamification user types hexad scale,” in Proc. 2016
Annu. Symp. on Computer-Human Interaction in Play, Oct. 2016, pp. 229
243. doi: 10.1145/2967934.2968082.
[41] S. Hallifax, A. Serna, J.-C. Marty, G. Lavoué, and E. Lavoué, “Factors to
consider for tailored gamification,” in Proc. Annu. Symp. on Computer-
Human Interaction in Play, Sept. 2019, doi: 10.1145/3311350.3347167.
14
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
[42] B. Efron and R. Tibshirani, An introduction to the bootstrap, Chapman &
Hall/CRC Monographs on Statist. & Applied Probability, 1993.
[43] M. D. Hanus and J. Fox, Assessing the effects of gamification in the
classroom: A longitudinal study on intrinsic motivation, social comparison,
satisfaction, effort, and academic performance,” Comput. & Educ., vol. 80,
pp. 152161, Jan. 2015, doi: 10.1016/j.compedu.2014.08.019.
[44] W. Peng, J.-H. Lin, K. A. Pfeiffer, and B. Winn, Need satisfaction
supportive game features as motivational determinants: An experimental
study of a Self-Determination Theory guided exergame,” Media Psychol.,
vol. 15, pp. 175196, May. 2012, doi: 10.1080/15213269.2012.673850.
[45] E. L. Deci, R. Koestner, and R. M. Ryan, Extrinsic rewards and intrinsic
motivation in education: Reconsidered once again,” Review Educ.
Research, vol. 71, no. 1, pp. 1–27, Mar. 2001, doi:
10.3102/00346543071001001.
[46] L. Festinger, “A theory of social comparison processes,” Human
Relations, vol. 7, pp. 117140, Jul. 1954, doi:
10.1177/001872675400700202.
[47] G. Orosz, D. Farkas, and C. P. Roland-Levy, Are competition and
extrinsic motivation reliable predictors of academic cheating?,” Frontiers
in Psychol., vol. 4, article 87, Feb. 2013, doi: 10.3389/fpsyg.2013.00087.
[48] R. J. Vallerand and G. Reid, On the causal effects of perceived
competence on intrinsic motivation: A test of cognitive evaluation theory,”
J. Sport and Exercise Psychol., vol. 6, pp. 94102, 1984, doi:
10.1123/JSP.6.1.94.
[49] J. Jang, J. J. Y. Park, and M. Y. Yi, “Gamification of online learning, in:
Artificial Intelligence in Education,” in Comput. Science, Springer Int.
Publishing, pp. 646649, Jun. 2015, doi: 10.1007/978-3-319-19773-
9_82.
[50] Y. Attali and M. Arieli-Attali, Gamification in assessment: Do points
affect test performance?,” Comput. & Educ., vol. 83, pp. 57-63, Apr. 2015,
doi: 10.1016/j.compedu.2014.12.012.
[51] S. Hallifax, E. Lavoué, and A. Serna, “To tailor or not to tailor
gamification? An analysis of the impact of tailored game elements on
learners’ behaviours and motivation,” in Artificial intellig. educ., 2020, pp.
216227, doi: 10.1007/978-3-030-52237-7_18.
[52] P. Lessel, M. Altmeyer, L. V. Schmeer, and A. Krger, “‘Enable or disable
gamification?’: Analyzing the impact of choice in a gamified image
tagging task,in Proc. 2019 CHI Conf. Human Factors in Computing Syst.
- CHI ’19, May 2019, Paper no: 150, doi: 10.1145/3290605.3300380.
[53] B. Monterrat, E. Lavoué, and S. George, Toward an adaptive
gamification system for learning environments,” Comput. Supported
Educ., Springer, vol. 510, pp. 115129, Dec. 2015, doi: 10.1007/978-3-
319-25768-6_8.
[54] J. Ooge, R. De Croon, K. Verbert, and V. Vanden Abeele, “Tailoring
gamification for adolescents: A validation study of big five and Hexad in
dutch,in Proc. 2020 Symp. on Comput.-Human Interact. in Play. Assoc.
for Comput. Machinery, Nov. 2020, doi: 10.1145/3410404.3414267.
[55] N. Lazzaro, “Why we play games: Four keys to more emotion.In Proc.
Game Developers Conference 2004.
15
> REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <
S. Reyssier was a Ph.D. student in
educational sciences at the Education,
Culture, and Policy (ECP) Lab, Lumière
University Lyon 2, Lyon, France. She
received the Ph.D. degree in Educational
Sciences from the University of Lyon, in
2021, and is now a postdoctoral at the
Institute for Science and Practice in
Education and Training (ISPEF),
University of Lyon 2.
Her fields of specialization are gamification, self-
determination, and the affordances of numeric environments in
education. Her Ph.D. research examines teachers’ perceptions
of a numeric gamified environment for mathematics education
that was co-designed with the teachers, as well as the impact of
using the environment on learners’ motivation.
S. Hallifax was a Ph.D. in the Laboratory
for Image Informatics and Information
Systems (LIRIS) computer science
laboratory, University of Lyon, Lyon,
France. He is now a postdoctoral fellow in
David R. Cheriton School of Computer
Science, Faculty of Mathematics,
University of Waterloo, Waterloo,
CANADA.
His main interests are gamification, user-behavior modeling,
and user profiling. His Ph.D. research focuses on adaptive and
gamified computer-based pedagogical resources.
A. Serna is an associate professor with
the National Institute of Applied Sciences
of Lyon (INSA Lyon), Lyon, France.
Her interests lie at the confluence of
computer science (humancomputer
interaction and usability/ergonomics) and
cognitive science (cognitive modeling/user
modeling). More specifically, she focuses
on the design of novel interactive systems capable of adapting
to the contexts of use, which become increasingly variable at
runtime and unforeseeable at design time.
Her research is concerned mainly with two complementary
aspects: user modeling and interaction with users.
J.C. Marty is an associate professor with
the LIRIS laboratory, University of Lyon,
Lyon, France.
His research interests center around the
observation of collaborative activities
through the traces of those activities. The
results of his research have applications in
technology-enhanced learning, and in
particular, game-based learning,
environments. Dr. Marty is an active contributor to several
projects in this field (Learning Adventure, Learning Games
Factory, Serious Lab for Innovation, Pegase, Jen.lab, Janus). He
organized an International School on Game-Based Learning in
France in June 2011, and is a member of several conference
committees and journal editorial boards of journals in the field.
S. Simonian is a professor in educational
sciences at the ISPEF, Lumière University
Lyon 2, Lyon, France.
Until the 1st September 2020, he was the
director of the ISPEF, and he is now the
director of the ECP Lab at the same
university.
As a Specialist in educational
technologies, his research focuses on learning scenarios and use
cases, based on the concept on sociocultural affordance.
É. Lavoué is an associate professor of
computer Science with the iaelyon School
of Management, University of Lyon, Lyon,
France and head of the Situated Interaction,
Collaboration, Adaptation, and Learning
(SICAL) research group within the LIRIS
lab at the same university. She was a
visiting professor at the ATLAS Lab,
McGill University, Montréal, Canada,
from January to August 2016.
Her research involves the design of learning environments to
support learner self-regulation and engagement, and spans the
fields of technology-enhanced learning, computer-supported
collaborative learning, and humancomputer interaction.
She has authored or co-authored over 90 publications,
including journal articles, book chapters, and conference
papers, in these areas. She served as general co-chair
(CSCL’19), organizing chair (EC-TEL’16), program
committee co-chair (EC-TEL’17), and program committee
member for the EC-TEL, CSCL, LAK, and ITS international
conferences, and is also member of the EC-TEL Steering
Committee.
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
Gamification researchers deem adolescents a particularly interesting audience for tailored gamification. However, empirical validation of popular player typologies and personality trait models thus far has been limited to adults. As adolescents exhibit complex behaviours that differ from older adults, these models may need adaptation. To that end, we collected a unique data set of Big Five Inventory and Hexad questionnaire answers in Dutch from 402 adolescents. Confirmatory factor analysis showed that the Dutch forms of the BFI-10, BFI-44 and Hexad scales performed substandard when used with adolescents. Through exploratory factor analysis, we investigated underlying problems, and provide preliminary suggestions on how to improve measurements. In particular, we propose to simplify the Hexad model, and to reformulate specific items. With this study, we hope to contribute to the debate on how to improve the tailoring of interactive systems for adolescents.
Chapter
Full-text available
Gamification, defined as the use of game elements in non game situations, is a widely used method to foster learner engagement and motivation. It is generally accepted that in order to be effective, gamification should be tailored to users. Currently, most systems adapt by assigning different game elements based on a single learner profile (e.g. dominant player type, personality or gender). However, there is no study yet that analyse the effect of combining several profiles. In this paper, we study the usage data from 258 students who used a gamified learning environment as a part of their mathematics class. By simulating different adaptation techniques, we show that the learner model chosen to tailor gamification has significant effects on learners’ motivation and engaged behaviours depending on the profile(s) used in this context. We also show that tailoring to initial motivation to learn mathematics can improve intrinsic motivation. Finally, we show that tailoring to both player type and motivation profiles can improve intrinsic motivation, and decrease amotivation, compared to a single adaptation only based on learner motivation. We discuss the implications of our findings regarding the choice of a learner model for tailoring gamification in educational environments.
Conference Paper
Full-text available
Gamification is widely used to foster user motivation. Recent studies show that users can be more or less receptive to different game elements, based on their personality or player profile. Consequently, recent work on tailored gamification tries to identify links between user types and motivating game elements. However findings are very heterogeneous due to different contexts, different typologies to characterize users, and different implementations of game elements. Our work seeks to obtain more generalizable findings in order to identify the main factors that will support design choices when tailoring gamification to users' profiles and provide designers with concrete recommendations for designing tailored gamification systems. For this purpose, we ran a crowdsourced study with 300 participants to identify the motivational impact of game elements. Our study differs from previous work in three ways: first, it is independent from a specific user activity and domain; second, it considers three user typologies; and third, it clearly distinguishes motivational strategies and their implementation using multiple different game elements. Our results reveal that (1) different implementations of a same motivational strategy have different impacts on motivation, (2) dominant user type is not sufficient to differentiate users according to their preferences for game elements, (3) Hexad is the most appropriate user typology for tailored gamification and (4) the motivational impact of certain game elements varies with the user activity or the domain of gamified systems.
Article
Full-text available
Today, our reality and lives are increasingly game-like, not only because games have become a pervasive part of our lives, but also because activities, systems and services are increasingly gamified. Gamification refers to designing information systems to afford similar experiences and motivations as games do, and consequently, attempting to affect user behavior. In recent years, popularity of gamification has skyrocketed and manifested in growing numbers of gamified applications, as well as a rapidly increasing amount of research. However, this vein of research has mainly advanced without an agenda, theoretical guidance or a clear picture of the field. To make the picture more coherent, we provide a comprehensive review of the gamification research (N = 819 studies) and analyze the research models and results in empirical studies on gamification. While the results in general lean towards positive findings about the effectiveness of gamification, the amount of mixed results is remarkable. Furthermore, education, health and crowdsourcing as well as points, badges and leaderboards persist as the most common contexts and ways of implementing gamification. Concurrently, gamification research still lacks coherence in research models, and a consistency in the variables and theoretical foundations. As a final contribution of the review, we provide a comprehensive discussion, consisting of 15 future research trajectories, on future agenda for the growing vein of literature on gamification and gameful systems within the information system science field.
Conference Paper
This paper investigates a simple form of customization: giving users the choice to enable or disable gamification. We present a study (N=77) in the context of image tagging, in which a gamification approach was shown to be effective in previous work. In our case, some participants could enable or disable gamification after they had experienced the task with and without it. Other participants had no choice and did the task with or without game elements. The results indicate that those who are not attracted by the elements can be motivated to tag more through this choice. In contrast, those that like the elements are not affected by it. This suggests that systems should provide the option to disable gamification in the absence of more sophisticated tailoring.
Article
Although many studies have focused on the potential of implementing gamification in education, the existing literature remains inconclusive about its effectiveness. In order to make sense of the contradictory findings regarding the effectiveness of implementing game design elements in an online learning environment, this paper complements the available body of research by addressing three holes. We have (1) analysed gamification's underlying motivational processes from a Self-Determination Theory-perspective, thereby accounting for the motivational effects of various game design implementations; (2) empirically assessed subtle motivational changes over time, and (3) accounted for the potential individual differences in motivational effects of gamification. Over a period of 15 weeks, we administered four surveys to measure the possible evolution in students’ (N = 40) motivational levels in response to interacting with need-supporting game elements that were implemented in Google + Communities used in a university course. Participants’ autonomous and controlled motivation was curve linear, showing an initial downward trend that surprisingly shifted to an upward tendency towards the end of the semester. Their controlled motivation stayed stable throughout. The results illustrated the significance of the individual nature of motivational processes, the importance of sensitive longitudinal motivation measurements, and the relevance of the implemented game elements’ design characteristics. We end this article by opening the debate on using theoretical lenses when designing gamification, and by providing avenues for future research.
Article
This paper presents the results of an empirical study conducted on three different types of gamified learning activities—namely competitive, collaborative, and adaptive—in lower primary mathematics classes. The participants were students from two second-grade and one third-grade classes who used tablet computers and digital learning lessons for learning mathematics. The study included a non-gamified and competitive, adaptive, and collaborative gamified conditions, which were integrated into lesson plans. The collected log data were used to calculate the changes in performance levels through the dimensions of task completion and time under each condition, and the data were further analyzed and compared across conditions. The quantitative analysis results were triangulated with interview data from the students. Overall, the results show that gamified activities contributed to increased student performance levels in math learning. Significantly higher performance levels appeared in a gamified condition combining competition, a narrative, and adaptivity with individual performance game elements. Although the highest performance levels appeared in conjunction with the most incorrect attempts by the students, the total number of correct attempts was unaffected. Our findings suggest that whether gamification works or not is not the result of individual game elements but rather the consequence of their balanced combination.
Conference Paper
Preparing for exams is an important yet stressful time for many students. Self-testing is known to be an effective preparation strategy, yet some students lack motivation to engage or persist in self-testing activities. Adding game elements to a platform supporting self-testing may increase engagement and, by extension, exam performance. We conduct a randomized controlled experiment (n=701) comparing the effect of two game elements -- a points system and a badge system -- used individually and in combination. We find that the badge system elicits significantly higher levels of voluntary self-testing activity and this effect is particularly pronounced amongst a relatively small cohort. Importantly, this increased activity translates to a significant improvement in exam scores. Our data supports a causal relationship between gamification and learning outcomes, mediated by self-testing behavior. This provides empirical support for Landers' theory of gamified learning when the gamified activity is conducted prior to measuring learning outcomes.