Conference PaperPDF Available

Using Gamification in Education: A Systematic Literature Review



Gamification systems have the potential to foster students' engagement and enhance their learning performance. Although the literature to date has provided relevant contributions in explaining how gamification design can be effective, several studies seek to address the constructs used to measure the effects of gamification. To analyze how gamification outcomes should be properly measured, this paper systematically reviews the literature and provides a map of the most frequent scales used to measure gamification outcomes in the educational context. As research findings, we identify motivation, engagement, self-efficacy, and flow/cognitive absorption as the primary constructs addressed to experiential outcomes. Additionally, there are research opportunities to develop a better understanding of the effects of extrinsic motivation rewards on experiential outcomes and problem-solving transfer posited as instrumental outcome.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 1
Using Gamification in Education: A
Systematic Literature Review
Completed Research Paper
Fabricio de C. Inocencio
474, Itapeva St - 9th floor
SP-Brazil 01332-000
Gamification systems have the potential to foster students’ engagement and enhance their
learning performance. Although the literature to date has provided relevant
contributions in explaining how gamification design can be effective, several studies seek
to address the constructs used to measure the effects of gamification. To analyze how
gamification outcomes should be properly measured, this paper systematically reviews
the literature and provides a map of the most frequent scales used to measure
gamification outcomes in the educational context. As research findings, we identify
motivation, engagement, self-efficacy, and flow/cognitive absorption as the primary
constructs addressed to experiential outcomes. Additionally, there are research
opportunities to develop a better understanding of the effects of extrinsic motivation
rewards on experiential outcomes and problem-solving transfer posited as instrumental
Keywords: Gamification, systematic literature review, education, learning outcomes
Gamification is defined as the use of game elements in nongame contexts (Deterding et al. 2011).
Gamification’s application in Information Systems (IS) has been growing in different industries, such as
commerce, health, work ideation and education (Hamari et al. 2014b). The belief about gamification
effectiveness is taken from the association with the games experience, given its characteristics of being fun
and intrinsically motivating. The goal of a gamified system is to foster user engagement and improve a
target outcome, such as user participation, learning, purchase, social interaction, and, ultimately,
productivity (Hamari and Koivisto 2013).
Although the literature on gamification to date has provided relevant contributions in explaining how
gamification design can be effective, there is a certain degree of controversy in regard to gamification
effectiveness, with studies showing that the gamification effect on motivation is lower than the expectations
created by the hype (Broer 2014). These unsatisfactory results may be partially explained by a problem of
measurement. Indeed, the literature does not provide a consensus on what constructs should be addressed
to create and measure gamification outcomes (O'Brien and Toms 2010; Tomaselli et al. 2015), which might
lead to some controversy about the effectiveness of gamification in past research.
A recent paper provided a new classification of the nature of dependent variables of systems gamification
by arranging them into instrumental and experiential outcomes (Liu et al. 2017). While experiential
outcomes evoke users’ psychological states, instrumental outcomes are the ultimate utilitarian results of
gamification. The authors call this arrangement meaningful engagement by defining that gamified systems
should address experiential outcomes in advance as a requirement to engender a subsequent instrumental
outcome (Liu et al. 2017). Similar arrangements, addressed previously in the literature, apply divergent
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 2
psychological states as gamification outcomes (Burke and Hiltbrand 2011; Hamari et al. 2014b; Kankanhalli
et al. 2012; Nel et al. 1999; Webster and Ahuja 2006). However, Liu et al. (2017) simplify these multiple
definitions, categorize these psychological states as experiential outcomes, and define the instrumental
outcomes as the desirable outcomes of gamification.
In the educational context, the use of gamification elements has largely increased in recent years and has
attracted attention from researchers. As an informational activity on its own, there is a considerable
potential to enhance learning performance of students by gamifying learning systems (Dichev and Dicheva
2017; Landers and Landers 2014; Ortiz-Rojas et al. 2017). The goal is to motivate the students in new ways,
by reducing feelings of tediousness in some activities and fostering engagement on learning activities that
would affect positively the learning outcomes (Hanus and Fox 2015). While gamification elements such as
badges, points, and leaderboards can be applied to the educational systems to motivate or engage students
in order to increase participation and learning, there is again no clear segregation in IS literature to
characterize what are experiential and instrumental outcomes of gamified educational systems. Hence,
while the literature still applies distinct constructs interchangeably (Dichev and Dicheva 2017), additional
research is needed to move this subject forward.
To shed some light on how these outcomes should be properly measured, this paper aims to systematically
review the literature and provide a map of current gamified systems outcomes, especially to classify them
into the experiential-instrumental typology. Additionally, this study focuses on identifying the most
frequent scales for measuring the experiential outcomes in gamified systems, related to the educational
The remainder of the paper is organized as follows. First, we present a theoretical background section that
covers the related literature concerning gamification outcomes, how to differentiate between instrumental
and experiential outcomes, and a review of gamification outcomes applied to the educational context.
Second, we develop a methods section, presenting the systematic literature review procedures, followed by
the results section, which provides an overview of papers, outcomes classifications and key constructs. Next,
the discussion and research insights section presents the identified literature gaps and new opportunities
for research development. Finally, the conclusion section presents key takeaways. For additional details on
the reviewed scales, we also present a final Appendix.
This study provides some findings and research insights as contributions. Related to experiential outcomes,
we found that motivation, engagement, self-efficacy, and flow/cognitive absorption have emerged as the
most relevant constructs with reliable scales and consistent theories. Although satisfaction and attitude are
frequently used, these constructs do not provide similar qualities. In addition, we suggest that the effect of
extrinsic rewards on experiential outcomes can also be explored in future research. Related to instrumental
outcomes, learning performance can be investigated as a downstream effect of studying behavior. Finally,
we recommend the research on transfer and other high order learning outcomes in a gamification context,
that remains underexplored.
Theoretical Background
Gamification Concept
Liu et al. (2017) define gamification as the incorporation of game design elements into a target system while
retaining the target system’s instrumental functions. This definition starts from the principle that a
gamification is a game layer in a nongame system (Santhanam et al. 2016). The gamification design would
add features, be focused on stimulating user participation, and keep all original instrumental functionality
of a target system, while a game would sacrifice some instrumental functionality of a target system in order
to sustain its entertainment value (Liu et al. 2017). Deterding et al. (2011b) state that a gamified system may
or may not be in a serious context, but it definitely does not require a full-fledged system, as games do. In
this study, we are guided by the definition of gamification from Liu et al. (2017) as a way of setting the limits
of the scope of analysis, filtering out any full-fledged game.
Gamification Outcomes
Since gamification design borrows elements from video games, it may stimulate similar hedonic
experiences, evoking game-like player behavior (Kankanhalli et al. 2012). At the same time, to be effective,
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 3
the gamification systems should use this experience to change a person’s behavior toward a desirable
outcome. The literature maintains that this double effect of gamification is required to be successful,
nevertheless using different concepts (Burke and Hiltbrand 2011; Hamari et al. 2014b; Kankanhalli et al.
2012; Nel et al. 1999; Webster and Ahuja 2006).
To provide a taxonomy about gamification terminologies in IS, Liu et al. (2017) classify the effect of
gamification elements as experiential outcomes and instrumental outcomes. The experiential outcome is
generally associated with user perceptions, such as a feeling, thought or emotion, while the instrumental
outcomes are associated with the utilitarian result of gamification. Liu et al. (2017) also establish a set of
principles that describe how gamification can provide meaningful engagement. The authors state that
experiential outcomes should fit task context with gamified elements and desired instrumental outcomes.
Given the broader definition of gamification outcomes, we use the dual outcome framework (Liu et al. 2017)
to classify the main categories of gamification outcomes presented in Figure 1.
In general, the identification of instrumental outcomes is straightforward given the direct association to the
task context, while the identification of experiential outcomes is a more complex task (Liu et al. 2017).
Dichev and Dicheva (2017) found that the empirical studies provide a diverse list of constructs, which are
more difficult to group into logical categories. This inaccuracy can be partially explained by the failure in
defining the conceptual domain of the constructs that may lead to some issues, such as the
misunderstanding about what the constructs truly refer to, the overlapping of constructs that already exist
in the field, and invalid conclusions about the relationship among constructs (MacKenzie et al. 2011).
Gamification Outcomes in the Educational Context
The literature dedicated to gamification in the educational contexts inherits the characteristics of the
gamification literature in general. For example, the dual-outcome principle (experiential-instrumental) can
also be identified in gamified systems dedicated to enhancing learning (Dichev and Dicheva 2017; Fitz-
Walter et al. 2017; Landers 2014; Looyestyn et al. 2017). In that case, the experiential outcomes are mostly
associated with hedonic or affective outcomes while the instrumental outcomes are associated with
observable schooling results, such as learning performance, retention and student participation (Dichev
and Dicheva 2017; Hamari et al. 2014b). However, the same incongruence of outcomes classification
presented in the gamification literature is also present in studies dedicated to the educational context. For
example, in a literature review, Dichev and Dicheva (2017) listed the frequent categories of outcomes
applied in educational gamified systems (i.e., knowledge acquisition, perceptions, behavioral, engagement,
motivational and social), but no additional information was provided on whether these outcomes are
experiential or instrumental.
Selection of Papers
The relevant literature was identified along the guidelines provided by Webster and Watson (2002). To
cover the full multidisciplinary nature of gamification, different fields were explored (i.e., IS, Computer
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 4
Science, Education and Psychology) with no specific predefinitions of journals. The search was conducted
in the following databases: EBSCO host (including Academic Search Complete, Business Source Complete,
Education Research Complete, ERIC, Professional Development Collection, Psychology and Behavioral
Sciences Collection); Science Direct and AISeL.
Considering the difference in experiences that educational games and gamification may cause on the user,
we chose to restrict the focus of this study to gamification papers only, using the definition from Liu et al.
(2017) to set the scope of analysis. As a result, studies related to educational games were not included. Given
that the terms “game-based”, “gameful” and “gamify” are ambiguous, their inclusion in the systematic
review final list could require abstract and full text individual inspection. Therefore, the search strategy was
focused on finding papers by using Boolean search with gamif* OR gamef* terms combined with educat*
OR train* OR learn* terms. These terms were searched in the title, keywords and abstract fields. To sustain
the level of quality of the sample, we limited the search to peer reviewed journals. For proceedings
conferences, we applied a go forward revision strategy to include the most cited papers (more than 10) by
using the Web of Science system. The flow of papers selection and inclusion/exclusion criteria is presented
in Figure 2.
For the final group of 95 papers, the most frequent constructs were then selected (more than 1 record) for
analysis in order to identify the scales with adequate reliability and validity characteristics and to identify
the background theories.
Classification of Constructs
To provide a clearer understanding of the conceptual domains of a construct, MacKenzie et al. (2011) specify
their basic properties in terms of whether the construct refers to a thought, feeling, perception, action,
outcome, or intrinsic characteristic. However, depending on the gamification goals, even a perceptual or
behavioral construct can be referred to as an instrumental outcome in the literature. Therefore, it is relevant
to identify the constructs properties but also to analyze the relation among them in a gamification context,
so the classification is more accurate.
In addition, we also take into consideration the nomological sequence of constructs; hence, for experiential
outcomes, the constructs should be referred as a direct effect of a gamified design and positioned as
antecedents of any final outcome. For instrumental outcomes, the constructs should refer to an indirect
effect of gamification and be positioned as final outcomes. Another issue is that some studies intend to
investigate a certain construct but operated with another. For example, Baxter et al. (2016) study enjoyment
using a satisfaction scale, and Santhanam et al. (2016) study engagement using a cognitive absorption scale.
To avoid miscounting of constructs, this study targets and counts the constructs that were applied as
operating variables in the research.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 5
Papers Overview
The selected list of papers represents growing interest in the subject over the period of 10 years, with 2, 9,
9, 26, and 44 papers published respectively in the years between 2013 and 2017. No papers before 2013
were found. In the first quarter of 2018, 5 additional papers meeting the selection criteria were published.
The journals Computers & Education and Computers in Human Behavior lead the gamification publication
frequency with 14 and 12 papers, respectively. However, dispersion is high, as a total of 13 journals
published between 2-4 papers and a total of 36 journals published just one paper. In this group, just one
paper belongs to an IS senior basket journal (Santhanam et al. 2016). In fact, Liu et al. (2017) argue that
even though IS researchers have an appropriate and diverse background to research and develop successful
principles for gamification systems, the number of gamification papers in the IS field still is very limited.
Related to the methods, the greater part of them (49%) have applied pairs or multiple sample comparison
techniques, such as ANOVA, ANCOVA, MANOVA and two-group T-tests. These techniques were used in
62% of the total of papers, including those cases where another technique was combined (13%). This can be
explained by the similarity of research goals, which are focused on measuring the gamification effect in real
applications scenarios, most of them comparing a gamified design to a nongamified design in two distinct
groups. All papers but one (Galbis-Córdova et al. 2017) involve experimental or quasi-experimental
research with the manipulation of gamification artifacts that were submitted to respondents.
Instrumental Outcome Constructs
Table 1 presents the most common instrumental outcome constructs found in the papers.
Table 1. Instrumental Outcome Constructs
Construct and Frequency
based on user
Interaction (1); Response Accuracy (1); Student Participation (1);
Time Spent (1); Response Time (1); Formation of Good Habits (1);
Time-on-Task (1); Interactivity (1); Student Attendance (1);
Attendance (1); Task Completion (1); Usefulness (1); Procrastination
(1); Carefulness (1); Contribution (1); Productivity (2); Behavioral
Engagement (5); User Activity (6); Participation (10).
based on self-
Meaningful Learning (1); Perceived Achievement (1); Declarative
Knowledge (1); Perceived Stakes (1); Perceived Learning (6);
Perceived Efficiency (1).
based on
Retention of Knowledge (1); Procedural Knowledge (1); Declarative
Knowledge (1); Knowledge Acquisition (5); Skills Improvement (1);
Learning Outcomes (1); Driving Behavior (1); Student Achievement
(1); Academic Success (1); Training Performance (1); Learning Gain
(2); Academic Performance (2); Student Performance (2); Learning
(3); Learning Achievement (5); Learning Performance (33).
based on user
Attrition (1); Retention (1);
Persistence (1); Players Retention (1).
Table 1. Instrumental Outcome Constructs
Considering that many papers provide models with more than one construct of interest, and since this
analysis is topic-centric (Webster and Watson 2002), the sum represents all constructs approached by all
the papers reviewed.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 6
Four categories of instrumental outcome constructs have emerged: Participation based on user activity
(36%), Performance based on self-perception (9%), Performance based on student grades (52%) and
Retention based on user activity (3%). The overwhelming majority of the papers (88%) studies constructs
associated with performance and participation based on user grades and user activity, respectively. Few
papers (9%) applied psychometric scales to measure instrumental outcomes. Indeed, product data are
recommended more for measuring cognition domain of learning and engagement (e.g., pretests posttests
and transfer tests), while self-reported measures (e.g., perceived learning, perceived achievement) are more
suitable for student’s beliefs about themselves (Azevedo 2015).
Even though the literature does not seem to properly cover how gamification affects problem-solving
transfer. Few papers have acknowledged the challenge of measuring transfer, suggesting that it be
addressed in future research or included in research limitations (Armstrong and Landers 2017; Henning et
al. 2017; Landers and Armstrong 2017).
Experiential Outcome Constructs
The experiential outcomes have a more dispersed list of constructs compared to instrumental outcomes, as
presented in Table 2. The constructs are categorized in Perceptions (experience in general), Motivation,
Attitude, Satisfaction, Engagement, Self-Efficacy, Flow/Cognitive Absorption and Others (aggregated
constructs that were identified just one time). Flow and Cognitive Absorption were merged due to
similarities presented in the review papers. All constructs with more than 1 record were analyzed. To
compose the detailed scales analysis (see Appendix A), we sought constructs and scales with a clear
definition in their conceptual domain, theory support, and reliability and validity information about the
scales. We also provided a backward review to inspect the scales’ original study, as well as to double check
and supplement data on the scales.
Table 2. Experiential Outcome Constructs
Perceptions (experience in general)
Attitude (toward gamification, lessons, badges, in general)
Engagement (emotional and cognitive)
Flow/Cognitive Absorption
Others: Playfulness; Anxiety; Relevance; Cognitive Load; Pedagogical
Affect; Confidence; Psychological Effects; Distraction; Cognitive
Effects; Student Perception (toward badges); Negative Effects;
Enjoyment; Attention; Self-Regulation; Positive Effects; Reactions-
to-Training; Task Meaningfulness; Valence; Acceptance; Social
Comparison; Fun; Usability; Interest; Mood State.
(1 each)
Table 2. Experiential Outcome Constructs
Perceptions (experience in general)
A total of 18 studies (20%) applied perceptual scales in order to gather overall opinions of students about
the gamification experience. These studies focus on proposing a gamification framework or measuring the
instrumental outcome as a primary objective. In those cases, the experiential outcome seems to be a
secondary objective with a lack validity and reliability information about the scales. It is not clear what
constructs were addressed; therefore, we do not provide any further analysis for that group.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 7
Motivation can be demonstrated by an individual’s choice to engage and persist in an activity (Dichev and
Dicheva 2017). According to the Self-Determination Theory - SDT (Ryan and Deci 2000), motivation can
be distinguished into two categories: extrinsic and intrinsic. The former stems from external incentives and
rewards while the latter relies on the sense of fulfillment associated with the activity itself, satisfying basic
psychological needs. The greater part of the studies presents a focus on capturing the intrinsic motivation
associated with the interest of students in accomplishing the learning activities in a gamified system. This
is the case with EQ, IMI, IMN, and ARCS (see Appendix A). Given the association of gamification with video
games’ hedonic experience, this intrinsic motivation approach seems to be a reasonable strategy to bring
about a desirable instrumental outcome. Nevertheless, some researchers (Hanus and Fox 2015; Lee and
Doh 2012) argue that the gamification elements based on rewards systems are also associated with extrinsic
motivation, which can affect learning performance as well (Dichev and Dicheva 2017). In a similar vein,
some studies applied scales that seek to capture a holistic view of motivation. These include the cases of
LSR and AM scales.
In social psychology, attitude refers to an individual’s disposition toward or against a target object, which
can be a phenomenon, person or thing (Dawson 1992). The reviewed papers present individual attitudes
toward gamification in different ways. In a broader sense, some studies assess attitude as a user’s
preferences toward the gamification experience (Armstrong and Landers 2017; Landers and Armstrong
2017) and as overall perception or satisfaction (de-Marcos et al. 2014; de-Marcos et al. 2016; Galbis-
Córdova et al. 2017; Ngan et al. 2017). In a narrow sense, some studies focused on specific but different
aspects of gamification, such as attitude toward lessons (Yildirim 2017), learning subjects (Baxter et al.
2016; Smith 2017) and badges (Kyewski and Krämer 2018).
In an IS context, satisfaction can be defined as a positive cognitive and emotional evaluation that represents
an individual’s expectations toward a system’s experience and performance (Bhattacherjee and Premkumar
2004). Satisfaction is associated with positive attitudes that lead to intentions of system continuance,
including in the learning and hedonic contexts of IS (Lowry et al. 2015). All papers presented scales similar
to the perceptions (experience in general) group, appearing to be an overall and secondary measure, or not
providing reliability/validity information about scales or background theory.
Engagement can be defined as the effort students dedicate in order to achieve learning outcomes (Kuh
2009). Azevedo (2015) explains that engagement is widely misused and overgeneralized by the educational
community, including by researchers. Fredricks et al. (2004) explain that the literature strives to
conceptualize the label engagement as a single construct. The authors state that the construct has a
multifaceted nature and propose that engagement should be defined in three ways: behavioral engagement,
emotional engagement, and cognitive engagement. Some years later, Reeve and Tseng (2011) brought
agentic engagement, which occurs when a student proactively contributes with the learning process during
the instruction. Furthermore, Sinatra et al. (2015) explain that the theoretical perspective of engagement
may change by the level at which the construct is conceptualized, varying from a microlevel to a macrolevel.
Given the complexity of the construct, we narrow our analysis on microlevel engagement (i.e., individual),
acknowledging the multidimensionality proposed by Fredricks et al. (2004) and Reeve and Tseng (2011).
All final scales present a multifaceted nature of engagement with different compositions, as presented in
Appendix A. In some cases, theories and scales of motivation and engagement are applied interchangeably.
For example, Filsecker and Hickey (2014) measure the effect of motivation based on a cognitive engagement
theory (Fredricks et al. 2004). De Sousa Monteiro et al. (2016) retrieved the motivational variables from an
engagement scale (Greene et al. 2004) and Ding and Orey (2018) and Ding et al. (2017) compose the
engagement scale based on a mix of engagement and motivation theories.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 8
In a general sense, there is no consensus in the reviewed papers about how the types of engagement should
be measured. For example, some studies (Ding et al. 2017; Ding and Orey 2018) address enjoyment and
perceived relatedness, retrieved from intrinsic motivation theories, to compose emotional engagement.
Behavioral engagement is frequently measured using data retrieved from gamification systems as an
outcome for participation, attendance, interaction, retention or use (Ding et al. 2017; Fitz-Walter et al. 2017;
Hew et al. 2016; Pechenkina et al. 2017; Rose et al. 2016; Tsay et al. 2018). Hew et al. (2016) measure
cognitive engagement with students’ test scores, difficulty level and quality of tasks completed by students.
In the same manner, Pilotti et al. (2017) measure cognitive engagement using the depth and lexical density
of students’ posts in the online discussion. Indeed, measuring cognitive engagement has unique challenges
compared to the others type of engagement. The field acknowledges that self-reported and product data
measures have their own limitations and gains, and a multimethod approach is always welcome (Greene
Self-efficacy (SE) refers to an individual’s beliefs or expectations about her or his ability to successfully
perform a task or behavior (Bandura 1977). SE has been positively associated with learning performance
and career interest in mathematics, science and technology (Huang and Mayer 2018; Luzzo et al. 1999;
Zeldin et al. 2008) and intrinsic motivation (Chentanez et al. 2005; Schunk et al. 1987). Three papers apply
SE in gamification in a learning context. Rachels and Rockinson-Szapkiw (2018) investigated the effect of
a mobile gamification app on students’ achievement, controlled by SE (Midgley et al. 2000). Banfield and
Wilkerson (2014) studied how gamification affects intrinsic motivation and SE (Zimmerman and Cleary
2006) of students. Santhanam et al. (2016) studied the effect of vicarious experiences on SE, learning
performance and engagement in gamified training. SE is focused on computer learning as a trait, control
variable (Santhanam et al. 2008) and an experiential outcome (Zweig and Webster 2004). For a scales
review, see Appendix A.
Flow/Cognitive Absorption
Flow state occurs when a person is fully immersed in a feeling of energized focus and enjoyment while
performing an activity (Csikszentmihalyi 1990). According to the flow theory, the individual should be
highly skilled and perceive the activity as challenging enough to achieve the flow state. In other conditions,
the individual feels anxiety (in a high challenge vs low skills scenario) or boredom (in a low challenge vs
high skills scenario).
Given that perceived challenge and skills are the basic conditions for flow, Hamari et al. (2016) investigate
these constructs as antecedents of engagement and immersion and perceived learning, but no information
about the scales is provided. Su and Hsaio (2015) develop a scale specific for gamified learning systems that
embeds all flow experience, antecedents and effects into a single construct in eight dimensions: sense of
control, concentration, clear goal, challenge and skills, feedback, immersion, knowledge improvement and
Based on flow theory, Santhanam et al. (2016) applied Cognitive Absorption (CA) construct (Agarwal and
Karahanna 2000) to measure engagement on a gamified learning system. CA describes the deep
involvement that an individual has with technology. Since its publication in MISQ, the construct has been
largely used in IS research in different contexts (Chandra et al. 2012; Goel et al. 2011; Lowry et al. 2013;
Reychav and Wu 2015; Santhanam et al. 2016; Weniger and Loebbecke 2011). For a scales review see
Appendix A.
Discussion and Research Insights
Motivation, engagement, self-efficacy and flow/cognitive absorption are the most frequent constructs found
in this study. To provide additional knowledge about how the experiential outcomes are operated in a
gamified learning system, we address insights related to those constructs and comment on other
opportunities for research.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 9
Intrinsic vs Extrinsic Motivation
The first research opportunity concerns motivation. Motivation has been placed in the researchers’ focus as
a key experiential outcome. Given the background theories related to learning and motivation (Black and
Deci 2000; Skinner et al. 2009) and games and motivation (Liu et al. 2013; Yee 2006), one can logically
address similar associations for gamification in the learning context. This link presents a strong inclination
for intrinsic motivation, reflecting the autonomous choice of users to reproduce the hedonic experiences
while they are playing video games.
However, unlike educational games, the gamification is not intended to embed an entirely hedonic
experience into the task; inversely, an additional layer is added to the context. The original task remains in
the system, sometimes unaltered. The gamification elements, working as motivational affordances, are
intended to evoke a gaming behavior and stimulate the learner. While in educational games the hedonic
and utilitarian elements are mixed, in gamification, the learner can easily identify the gamification elements
and the task activities he or she is supposed to complete (Lee and Doh 2012).
Despite the fact that the reviewed papers have addressed recurrently intrinsic motivation scales to
gamification (see Appendix A), some researchers argue that gamification also has an extrinsic motivation
side, mainly if it is based on extrinsic rewards systems (Hanus and Fox 2015; Lee and Doh 2012). This
extrinsic side is almost absent in the reviewed paper in this study. Therefore, there are research
opportunities for unveiling how motivational affordances carrying extrinsic motivation rewards affect
extrinsic motivation constructs (e.g., external motivation, introjected motivation, and identified
Engagement Concept Elucidation
A second research opportunity concerns the engagement construct. The literature has been misusing
engagement as a generic term in the educational context (Azevedo 2015; Fredricks et al. 2004; Looyestyn
et al. 2017). Therefore, there is an unclear zone whether the construct should be addressed as a second order
construct or which dimensions (i.e., emotion, cognitive, behavioral and agentic) should be addressed
separately in a gamified context. In addition, it is not clear whether behavioral engagement should be placed
on the instrumental side of the gamification effect, provided that its properties are more associated with an
action or activity (e.g., studying behavior), rather than a feeling, emotion or thought (MacKenzie et al. 2011).
Finally, given the divergence of measures found in the studies, there are research opportunities to test and
develop scales related to all types of engagement, mainly agentic, which remains unexplored in the
gamification literature.
Extrinsic Rewards and Self-Efficacy
Related to Self-Efficacy (SE), one first research opportunities involves assessing how extrinsic rewards
would affect (negatively) SE given its close relation to intrinsic motivation. Additionally, given that SE can
be a trait (Santhanam et al. 2008) or an experiential outcome (Zweig and Webster 2004), it is important to
understand the model arrangement that can lead to better instrumental outcome results in a gamified
Flow and Cognitive Absorption Dimensionality
Related to Flow and Cognitive absorption (CA), it is timely to explore which dimensions have a better fit or
not in gamification context. CA scale exhibits five dimensions (i.e., temporal dissociation, focused
immersion, heightened enjoyment, control and curiosity). Agarwal and Karahanna (2000) explain that CA
incorporates control, curiosity and focused attention from a flow scale (Trevino and Webster 1992), the
heighted enjoyment represents a synthesis of intrinsic interest of flow (Webster et al. 1993) and perceived
enjoyment (Davis et al. 1992), and the authors added temporal dissociation.
Some flow/CA dimensions can be controversial with respect to learning activity in gamified systems, mainly
if extrinsic rewards are present. For example, Webster and Ho (1997) excluded control, arguing that it is
not necessary because ‘passive engagement (watching TV)’ exists while ‘passive flow’ is impossible (Weniger
and Loebbecke 2011). The same analogy could be applicable for some learning activity via an IS (e.g.,
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 10
watching a lecture online vs doing homework online with due dates). Another example is temporal
dissociation. If a time constraint is applied as a gamification element, this will probably diminish the effect
of a user being unable to register the passage of time while in a CA state (Agarwal and Karahanna 2000) .
Satisfaction and Attitude Nomological Sequence
There are also research opportunities for satisfaction and attitudes. In the case of satisfaction, is unclear
whether it should be posited as an experiential outcome (immediately after the gamification effect and
before any instrumental outcome), or after the whole process of a gamification experience (after any
instrumental outcome), as a continuance intention factor. In the case of attitude, it is opportune to
investigate what types of attitude might work as a moderating effect or antecedent of experiential outcomes.
Considering that both constructs did not present enough information about the scales, there is opportunity
for new scales development as well.
Transfer and Perceived Usefulness as Instrumental Outcomes
The reviewed papers tend to address learning performance with low cognitive outcomes, avoiding
measuring high cognitive outcomes. For example, the literature on gamification barely refers to transfer,
but it should address the topic properly in order to move the field forward. The problem-solving transfer is
a key aspect for education, which the learner will actually develop applicative knowledge as an ultimate
instrumental outcome rather than replicative knowledge (Broudy 2017). By inserting outcomes related to
transfer in a gamified system, it might be appropriate to understand how the entire gamified learning
system would affect the perceived usefulness of a user toward the applicative knowledge gain.
Studying Behavior and Learning Outcomes
Finally, given the challenges faced by researchers in providing evidence of real learning gain in gamified
systems, it may be appropriate to investigate how experiential outcomes would positively affect a studying
behavior first, as an upstream outcome, and positing any learning outcome later, as a downstream result in
the instrumental outcomes group.
Figure 3 presents the map of the constructs analyzed in this study. It shows the most frequent constructs
found in the reviewed papers and adds constructs we identify as opportunities for future research.
This study presents a systematic literature review focused on gamification scales in the educational context,
making several important contributions. The literature seeks to conceptualize the constructs addressed in
the research models while still presenting studies with methodological issues (Hamari et al. 2014a; Hamari
et al. 2014b), which may lead to questionable conclusions since random or wrong choices may affect the
results (Liu et al. 2017). A list of 95 empirical papers was selected in order to identify the top constructs that
have emerged in the literature. A total of 17 papers presented reliability, validity and theory information
about scales they applied. Motivation, engagement, self-efficacy, and flow/cognitive absorption have
emerged as the most frequent constructs that can be used as experiential outcomes in research. Their scales
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 11
were mapped and analyzed, and the results are presented in Appendix A as a suggestion for future
applications. Additionally, this study addresses some research opportunities with respect to specific
Although gamification has been explored in other fields, it can be better addressed by IS researchers, since
their background in multidisciplinary fields provide the necessary understanding of this complex topic (Liu
et al. 2017). Therefore, we expect that the insights proposed in this study can contribute to the development
of future research related to measurement of gamification outcomes and support the gamification design
itself, as a consequence.
This study has several limitations. First, the logical interpretation of classifying the constructs into
experiential or instrumental outcomes is challenging. Some empirical studies apply similar constructs
interchangeably, with unclear definitions or without enough data. Therefore, some allocation decisions may
require further debate. Second, given our scope focused on the most frequent constructs addressed to
experiential outcomes, we analyze constructs with more than one record on the reviewed papers. This
represents 73% of the total, and further study can encompass more constructs (or all of them), tracking
reliable scales that can also be applied for gamification. Finally, the scales were analyzed at the dimension
level. A deeper analysis can go further in the measurement models by analyzing the items of the scales along
with their research results, as well.
Agarwal, R., and Karahanna, E. 2000. "Time Flies When You're Having Fun: Cognitive Absorption and
Beliefs About Information Technology Usage," MIS Quarterly (24:4), pp. 665-694.
Armstrong, M. B., and Landers, R. N. 2017. "An Evaluation of Gamified Training: Using Narrative to
Improve Reactions and Learning," Simulation & Gaming (48:4), pp. 513-538.
Azevedo, R. 2015. "Defining and Measuring Engagement and Learning in Science: Conceptual, Theoretical,
Methodological, and Analytical Issues," Educational Psychologist (50:1), pp. 84-94.
Baard, P. P., Deci, E. L., and Ryan, R. M. 2004. "Intrinsic Need Satisfaction: A Motivational Basis of
Performance and WeilBeing in Two Work Settings," Journal of applied social psychology (34:10), pp.
Bandura, A. 1977. "Self-Efficacy: Toward a Unifying Theory of Behavioral Change," Psychological review
(84:2), p. 191.
Bandura, A. 2006. "Guide for Constructing Self-Efficacy Scales," Self-efficacy beliefs of adolescents (5:1),
pp. 307-337.
Banfield, J., and Wilkerson, B. 2014. "Increasing Student Intrinsic Motivation and Self-Efficacy through
Gamification Pedagogy," Contemporary Issues in Education Research (7:4), pp. 291-298.
Barata, G., Gama, S., Jorge, J., and Gonçalves, D. 2014. "Identifying Student Types in a Gamified Learning
Experience," International Journal of Game-Based Learning (4:4), pp. 19-36.
Baxter, R. J., Holderness Jr, D. K., and Wood, D. A. 2016. "Applying Basic Gamification Techniques to It
Compliance Training: Evidence from the Lab and Field," Journal of Information Systems (30:3), pp.
Bhattacherjee, A., and Premkumar, G. 2004. "Understanding Changes in Belief and Attitude toward
Information Technology Usage: A Theoretical Model and Longitudinal Test," MIS quarterly), pp. 229-
Black, A. E., and Deci, E. L. 2000. "The Effects of Instructors' Autonomy Support and Students'
Autonomous Motivation on Learning Organic Chemistry: A SelfDetermination Theory Perspective,"
Science education (84:6), pp. 740-756.
Broer, J. 2014. "Gamification and the Trough of Disillusionment," Mensch & Computer 2014-
Broudy, H. S. 2017. "Types of Knowledge and Purposes of Education," in Schooling and the Acquisition of
Knowledge. Routledge, pp. 1-17.
Buckley, P., Doyle, E., and Doyle, S. 2017. "Game On! Students' Perceptions of Gamified Learning," Journal
of Educational Technology & Society (20:3), pp. 1-10.
Burke, M., and Hiltbrand, T. 2011. "How Gamification Will Change Business Intelligence," Business
Intelligence Journal (16:2), pp. 8-16.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 12
Chandra, S., Srivastava, S. C., and Yin-Leng, T. 2012. "Cognitive Absorption and Trust for Workplace
Collaboration in Virtual Worlds: An Information Processing Decision Making Perspective," Journal of
the Association for Information Systems (13:10), pp. 797-835.
Chentanez, N., Barto, A. G., and Singh, S. P. 2005. "Intrinsically Motivated Reinforcement Learning,"
Advances in neural information processing systems, pp. 1281-1288.
Csikszentmihalyi, M. 1990. "Flow: The Psychology of Optimal Performance." New York: Harper and Row.
Davis, F. D., Bagozzi, R. P., and Warshaw, P. R. 1992. "Extrinsic and Intrinsic Motivation to Use Computers
in the Workplace 1," Journal of applied social psychology (22:14), pp. 1111-1132.
Dawson, K. P. 1992. "Attitude and Assessment in Nurse Education," Journal of Advanced Nursing (17:4),
pp. 473-479.
de Sousa Monteiro, B., Gomes, A. S., and Mendes Neto, F. M. 2016. "Youubi: Open Software for Ubiquitous
Learning," Computers in Human Behavior (55), pp. 1145-1164.
de-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., and Pagés, C. 2014. "An Empirical Study Comparing
Gamification and Social Networking on E-Learning," Computers & Education (75), pp. 82-91.
de-Marcos, L., García-López, E., García-Cabot, A., Medina-Merodio, J.-A., Domínguez, A., Martínez-
Herráiz, J.-J., and Diez-Folledo, T. 2016. "Social Network Analysis of a Gamified E-Learning Course:
Small-World Phenomenon and Network Metrics as Predictors of Academic Performance," Computers
in Human Behavior (60), pp. 312-321.
Deci, E. L., and Ryan, R. M. 1980. "The Empirical Exploration of Intrinsic Motivational Processes," in
Advances in Experimental Social Psychology. Elsevier, pp. 39-80.
Deterding, S., Sicart, M., Nacke, L., O'Hara, K., and Dixon, D. 2011. "Gamification. Using Game-Design
Elements in Non-Gaming Contexts," CHI'11 extended abstracts on human factors in computing
systems: ACM, pp. 2425-2428.
Dichev, C., and Dicheva, D. 2017. "Gamifying Education: What Is Known, What Is Believed and What
Remains Uncertain: A Critical Review," International Journal of Educational Technology in Higher
Education (14:1), pp. 1-36.
Ding, L., Kim, C., and Orey, M. 2017. "Studies of Student Engagement in Gamified Online Discussions,"
Computers & Education (115), pp. 126-142.
Ding, L., and Orey, M. 2018. "An Exploratory Study of Student Engagement in Gamified Online
Discussions," Computers & Education).
Dweck, C. S. 1999. Self-Theories: Their Role in Motivation, Personality, and Development. psychology
Elliot, A. J. 1999. "Approach and Avoidance Motivation and Achievement Goals," Educational psychologist
(34:3), pp. 169-189.
Filsecker, M., and Hickey, D. T. 2014. "A Multilevel Analysis of the Effects of External Rewards on
Elementary Students' Motivation, Engagement and Learning in an Educational Game," Computers &
Education (75), pp. 136-148.
Fitz-Walter, Z., Johnson, D., Wyeth, P., Tjondronegoro, D., and Scott-Parker, B. 2017. "Driven to Drive?
Investigating the Effect of Gamification on Learner Driver Behavior, Perceived Motivation and User
Experience," Computers in Human Behavior (71), pp. 586-595.
Fredricks, J. A., Blumenfeld, P. C., and Paris, A. H. 2004. "School Engagement: Potential of the Concept,
State of the Evidence," Review of educational research (74:1), pp. 59-109.
Frost, R. D., Matta, V., and MacIvor, E. 2015. "Assessing the Efficacy of Incorporating Game Dynamics in a
Learning Management System," Journal of Information Systems Education (26:1), pp. 59-70.
Galbis-Córdova, A., Martí-Parreño, J., and Currás-Pérez, R. 2017. "Higher Education Students' Attitude
Towards the Use of Gamification for Competencies Development," Journal of E-Learning & Knowledge
Society (13:1), pp. 129-146.
Goel, L., Johnson, N. A., Junglas, I., and Ives, B. 2011. "From Space to Place: Predicting Users' Intentions
to Return to Virtual Worlds," MIS Quarterly (35:3), pp. 749-A745.
Greene, B. A. 2015. "Measuring Cognitive Engagement with Self-Report Scales: Reflections from over 20
Years of Research," Educational Psychologist (50:1), pp. 14-30.
Greene, B. A., Miller, R. B., Crowson, H. M., Duke, B. L., and Akey, K. L. 2004. "Predicting High School
Students' Cognitive Engagement and Achievement: Contributions of Classroom Perceptions and
Motivation," Contemporary educational psychology (29:4), pp. 462-482.
Hamari, J., and Koivisto, J. 2013. "Social Motivations to Use Gamification: An Empirical Study of
Gamifying Exercise," ECIS 2013 Completed Research).
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 13
Hamari, J., Koivisto, J., and Pakkanen, T. 2014a. "Do Persuasive Technologies Persuade?-a Review of
Empirical Studies," International conference on persuasive technology: Springer, pp. 118-136.
Hamari, J., Koivisto, J., and Sarsa, H. 2014b. "Does Gamification Work?--a Literature Review of Empirical
Studies on Gamification," System Sciences (HICSS), 2014 47th Hawaii International Conference on:
IEEE, pp. 3025-3034.
Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., and Edwards, T. 2016. "Challenging
Games Help Students Learn: An Empirical Study on Engagement, Flow and Immersion in Game-Based
Learning," Computers in Human Behavior (54), pp. 170-179.
Hamzah, W. M. A. F. W., Ali, N. H., Saman, M. Y. M., Yusoff, M. H., and Yacob, A. 2015. "Influence of
Gamification on Students' Motivation in Using E-Learning Applications Based on the Motivational
Design Model," International Journal of Emerging Technologies in Learning (10:2), pp. 30-34.
Handelsman, M. M., Briggs, W. L., Sullivan, N., and Towler, A. 2005. "A Measure of College Student Course
Engagement," The Journal of Educational Research (98:3), pp. 184-192.
Hanus, M. D., and Fox, J. 2015. "Assessing the Effects of Gamification in the Classroom: A Longitudinal
Study on Intrinsic Motivation, Social Comparison, Satisfaction, Effort, and Academic Performance,"
Computers & Education (80), pp. 152-161.
Henning, M., Hagedorn-Hansen, D., and von Leipzig, K. H. 2017. "Metacognitive Learning: Skills
Development through Gamification at the Stellenbosch Learning Factory as a Case Study," South
African Journal of Industrial Engineering (28:3), pp. 105-112.
Hew, K. F., Huang, B., Chu, K. W. S., and Chiu, D. K. W. 2016. "Engaging Asian Students through Game
Mechanics: Findings from Two Experiment Studies," Computers & Education (92), pp. 221-236.
Huang, X., and Mayer, R. E. 2018. "Adding Self-Efficacy Features to an Online Statistics Lesson," Journal
of Educational Computing Research), p. 0735633118771085.
Kankanhalli, A., Taher, M., Cavusoglu, H., and Kim, S. H. 2012. "Gamification: A New Paradigm for Online
User Engagement," ICIS 2012 Proceedings).
Keller, J. M. 2009. Motivational Design for Learning and Performance: The Arcs Model Approach.
Springer Science & Business Media.
Khan, A., Ahmad, F. H., and Malik, M. 2017. "Use of Digital Game Based Learning and Gamification in
Secondary School Science: The Effect on Student Engagement, Learning and Gender Difference,"
Education & Information Technologies (22:6), pp. 2767-2804.
Kuh, G. D. 2009. "What Student Affairs Professionals Need to Know About Student Engagement," Journal
of college student development (50:6), pp. 683-706.
Kyewski, E., and Krämer, N. C. 2018. "To Gamify or Not to Gamify? An Experimental Field Study of the
Influence of Badges on Motivation, Activity, and Performance in an Online Learning Course,"
Computers & Education (118), pp. 25-37.
Lambert, J. 2017. "An Examination of the Relationship between Higher Education Learning Environments
and Motivation, Self-Regulation, and Goal Orientation," Technology, Instruction, Cognition &
Learning (10:4), pp. 289-312.
Lamborn, S., Newmann, F., and Wehlage, G. 1992. "The Significance and Sources of Student Engagement,"
Student engagement and achievement in American secondary schools), pp. 11-39.
Landers, R. N. 2014. "Developing a Theory of Gamified Learning: Linking Serious Games and Gamification
of Learning," Simulation & Gaming (45:6), pp. 752-768.
Landers, R. N., and Armstrong, M. B. 2017. "Enhancing Instructional Outcomes with Gamification: An
Empirical Test of the Technology-Enhanced Training Effectiveness Model," Computers in Human
Behavior (71), pp. 499-507.
Landers, R. N., and Landers, A. K. 2014. "An Empirical Test of the Theory of Gamified Learning: The Effect
of Leaderboards on Time-on-Task and Academic Performance," Simulation & Gaming (45:6), pp. 769-
Lee, H., and Doh, Y. Y. 2012. "A Study on the Relationship between Educational Achievement and
Emotional Engagement in a Gameful Interface for Video Lecture Systems," Ubiquitous virtual reality
(isuvr), 2012 international symposium on: IEEE, pp. 34-37.
Liu, D., Li, X., and Santhanam, R. 2013. "Digital Games and Beyond: What Happens When Players
Compete?," MIS Quarterly: Management Information Systems (37:1), pp. 111-124.
Liu, D., Santhanam, R., and Webster, J. 2017. "Toward Meaningful Engagement: A Framework for Design
and Research of Gamified Information Systems," MIS quarterly (41:4).
Looyestyn, J., Kernot, J., Boshoff, K., Ryan, J., Edney, S., and Maher, C. 2017. "Does Gamification Increase
Engagement with Online Programs? A Systematic Review," PLoS ONE (12:3), pp. 1-19.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 14
Lowry, P. B., Gaskin, J. E., and Moody, G. D. 2015. "Proposing the Multimotive Information Systems
Continuance Model (Misc) to Better Explain End-User System Evaluations and Continuance
Intentions," Journal of the Association for Information Systems (16:7), pp. 515-579.
Lowry, P. B., Gaskin, J. E., Twyman, N. W., Hammer, B., and Roberts, T. L. 2013. "Taking "Fun and Games"
Seriously: Proposing the Hedonic-Motivation System Adoption Model (Hmsam)," Journal of the
Association for Information Systems (14:11), pp. 617-671.
Luzzo, D. A., Hasper, P., Albert, K. A., Bibby, M. A., and Martinelli Jr, E. A. 1999. "Effects of Self-Efficacy-
Enhancing Interventions on the Math/Science Self-Efficacy and Career Interests, Goals, and Actions of
Career Undecided College Students," Journal of Counseling Psychology (46:2), p. 233.
MacKenzie, S. B., Podsakoff, P. M., and Podsakoff, N. P. 2011. "Construct Measurement and Validation
Procedures in Mis and Behavioral Research: Integrating New and Existing Techniques," MIS quarterly
(35:2), pp. 293-334.
Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., and Urdan, T. 2000.
"Manual for the Patterns of Adaptive Learning Scales," Ann Arbor (1001), pp. 48109-41259.
Nel, D., van Niekerk, R., Berthon, J.-P., and Davies, T. 1999. "Going with the Flow: Web Sites and Customer
Involvement," Internet research (9:2), pp. 109-116.
Ngan, O. M. Y., Tang, T. L. H., Chan, A. K. Y., Chen, D. M., and Tang, F. M. K. 2017. "Blended Learning in
Anatomy Teaching for Non-Medical Students: An Innovative Approach to the Health Professions
Education," Health Professions Education).
O'Brien, H. L., and Toms, E. G. 2010. "The Development and Evaluation of a Survey to Measure User
Engagement," Journal of the Association for Information Science and Technology (61:1), pp. 50-69.
Ortiz-Rojas, M., Chiluiza, K., and Valcke, M. 2017. "Gamification and Learning Performance: A Systematic
Review of the Literature," European Conference on Games Based Learning: Academic Conferences
International Limited, pp. 515-522.
Pechenkina, E., Laurence, D., Oates, G., Eldridge, D., and Hunter, D. 2017. "Using a Gamified Mobile App
to Increase Student Engagement, Retention and Academic Achievement," International Journal of
Educational Technology in Higher Education (14:1), pp. 1-12.
Pintrich, P., Smith, D., Garcia, T., and McKeachie, W. 1991. "A Manual for the Use of the Motivated
Strategies for Learning Questionnaire (Technical Report 91-B-004)," The Regents of the University of
Rachels, J. R., and Rockinson-Szapkiw, A. J. 2018. "The Effects of a Mobile Gamification App on
Elementary Students’ Spanish Achievement and Self-Efficacy," Computer Assisted Language Learning
(31:1/2), pp. 72-89.
Reychav, I., and Wu, D. 2015. "Are Your Users Actively Involved? A Cognitive Absorption Perspective in
Mobile Training," Computers in Human Behavior (44), pp. 335-346.
Rose, J. A., meara, J. M. O., Gerhardt, T. C., and Williams, M. 2016. "Gamification: Using Elements of Video
Games to Improve Engagement in an Undergraduate Physics Class," Physics Education (51:5), pp. 1-1.
Ryan, R. M. 1982. "Control and Information in the Intrapersonal Sphere: An Extension of Cognitive
Evaluation Theory," Journal of personality and social psychology (43:3), p. 450.
Ryan, R. M., and Deci, E. L. 2000. "Self-Determination Theory and the Facilitation of Intrinsic Motivation,
Social Development, and Well-Being," American psychologist (55:1), p. 68.
Ryan, R. M., Koestner, R., and Deci, E. L. 1991. "Ego-Involved Persistence: When Free-Choice Behavior Is
Not Intrinsically Motivated," Motivation and emotion (15:3), pp. 185-205.
Ryan, R. M., Mims, V., and Koestner, R. 1983. "Relation of Reward Contingency and Interpersonal Context
to Intrinsic Motivation: A Review and Test Using Cognitive Evaluation Theory," Journal of personality
and Social Psychology (45:4), p. 736.
Sailer, M., Hense, J. U., Mayr, S. K., and Mandl, H. 2017. "How Gamification Motivates: An Experimental
Study of the Effects of Specific Game Design Elements on Psychological Need Satisfaction," Computers
in Human Behavior (69), pp. 371-380.
Santhanam, R., Liu, D., and Shen, W.-C. M. 2016. "Research Note-Gamification of Technology-Mediated
Training: Not All Competitions Are the Same," Information Systems Research (27:2), pp. 453-465.
Santhanam, R., Sasidharan, S., and Webster, J. 2008. "Using Self-Regulatory Learning to Enhance E-
Learning-Based Information Technology Training," Information Systems Research (19:1), pp. 26-47.
Schunk, D. H., Hanson, A. R., and Cox, P. D. 1987. "Peer-Model Attributes and Children's Achievement
Behaviors," Journal of Educational Psychology (79:1), p. 54.
Sinatra, G. M., Heddy, B. C., and Lombardi, D. 2015. "The Challenges of Defining and Measuring Student
Engagement in Science." Taylor & Francis.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 15
Skinner, E. A., Kindermann, T. A., Connell, J. P., and Wellborn, J. G. 2009. "Engagement and Disaffection
as Organizational Constructs in the Dynamics of Motivational Development," Handbook of motivation
at school), pp. 223-245.
Smith, T. 2017. "Gamified Modules for an Introductory Statistics Course and Their Impact on Attitudes and
Learning," Simulation & Gaming (48:6), pp. 832-854.
Su, C.-H., and Hsaio, K.-C. 2015. "Developing and Evaluating Gamifying Learning System by Using Flow-
Based Model," EURASIA Journal of Mathematics, Science & Technology Education (11:6), pp. 1283-
Tellegen, A., and Atkinson, G. 1974. "Openness to Absorbing and Self-Altering Experiences (" Absorption"),
a Trait Related to Hypnotic Susceptibility," Journal of abnormal psychology (83:3), p. 268.
Tomaselli, F., Sanchez, O., and Brown, S. 2015. "How to Engage Users through Gamification: The Prevalent
Effects of Playing and Mastering over Competing," ICIS 2015 Proceedings).
Trevino, L. K., and Webster, J. 1992. "Flow in Computer-Mediated Communication: Electronic Mail and
Voice Mail Evaluation and Impacts," Communication research (19:5), pp. 539-573.
Tsay, C. H.-H., Kofinas, A., and Luo, J. 2018. "Enhancing Student Learning Experience with Technology-
Mediated Gamification: An Empirical Study," Computers & Education).
Vallerand, R. J., Pelletier, L. G., Blais, M. R., Briere, N. M., Senecal, C., and Vallieres, E. F. 1992. "The
Academic Motivation Scale: A Measure of Intrinsic, Extrinsic, and Amotivation in Education,"
Educational and psychological measurement (52:4), pp. 1003-1017.
Webster, J., and Ahuja, J. S. 2006. "Enhancing the Design of Web Navigation Systems: The Influence of
User Disorientation on Engagement and Performance," Mis Quarterly), pp. 661-678.
Webster, J., and Ho, H. 1997. "Audience Engagement in Multimedia Presentations," ACM SIGMIS
Database: the DATABASE for Advances in Information Systems (28:2), pp. 63-77.
Webster, J., Trevino, L. K., and Ryan, L. 1993. "The Dimensionality and Correlates of Flow in Human-
Computer Interactions," Computers in human behavior (9:4), pp. 411-426.
Webster, J., and Watson, R. T. 2002. "Analyzing the Past to Prepare for the Future: Writing a Literature
Review," MIS quarterly), pp. xiii-xxiii.
Weniger, S., and Loebbecke, C. 2011. "Researching Cognitive Absorption in the Context of Fun-Oriented
Information Systems Usage: An Exploratory Study," ECIS, p. 135.
Yee, N. 2006. "Motivations for Play in Online Games," CyberPsychology & behavior (9:6), pp. 772-775.
Yildirim, I. 2017. "The Effects of Gamification-Based Teaching Practices on Student Achievement and
Students' Attitudes toward Lessons," The Internet and Higher Education (33), pp. 86-92.
Zeldin, A. L., Britner, S. L., and Pajares, F. 2008. "A Comparative Study of the SelfEfficacy Beliefs of
Successful Men and Women in Mathematics, Science, and Technology Careers," Journal of Research
in Science Teaching: The Official Journal of the National Association for Research in Science Teaching
(45:9), pp. 1036-1058.
Zimmerman, B. J., and Cleary, T. J. 2006. "Adolescents’ Development of Personal Agency: The Role of Self-
Efficacy Beliefs and Self-Regulatory Skill," Self-efficacy beliefs of adolescents (5), pp. 45-69.
Zweig, D., and Webster, J. 2004. "Validation of a Multidimensional Measure of Goal Orientation,"
Canadian Journal of Behavioural Science/Revue canadienne des sciences du comportement (36:3), p.
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 16
Appendix A
Scale (1)
Theory (2)
18 items, 4 dimensions,
5 Likert points
Attention, relevance, confidence and satisfaction
(Hamzah et al. 2015)
Motivating tasks, autonomy support, mastery
evaluation, perceived instrumentality, self-
efficacy, mastery goals, performance goals and
study strategies
(de Sousa Monteiro et
al. 2016)
22-45 items, 4-7
dimensions, 5 Likert
Interest/enjoyment, perceived competence, effort,
value/usefulness, felt pressure and tension,
perceived choice and relatedness
(Hanus and Fox 2015;
Lambert 2017)
13-21 items, 3-4
dimensions, 7 Likert
Competence, autonomy in regard to decision
freedom, autonomy in regard to task
meaningfulness, social relatedness
(Frost et al. 2015; Sailer
et al. 2017)
28 items, 7 dimensions,
5 Likert points
Intrinsic motivation (to know, to accomplish
things, to experience stimulation), external
motivation, introjected motivation, identified
regulation and amotivation
(Buckley et al. 2017)
6 items, 2 dimensions,
7 Likert points
Autonomous and controlled motivation
(Ding et al. 2017)
23 items, 4 dimensions,
5 Likert points
Skills, emotional, participation, performance
(Barata et al. 2014)
questionnaire; 8 items,
unidimensional, 5
Likert points
Emotional engagement and behavioral
(Hamari et al. 2016;
Khan et al. 2017)
24 items, 3 dimensions,
1 with 2
subdimensions, 7
Likert points
Emotional engagement: enjoyment and perceived
relatedness, behavioral engagement (student data)
and cognitive engagement
(Ding and Orey 2018)
Using Gamification in Education: A Systematic Literature Review
Thirty Ninth International Conference on Information Systems, San Francisco 2018 17
28 items, 3 dimensions
(1 with 2
subdimensions, 5-7
Likert points
Behavioral engagement, emotional engagement
(enjoyment and perceived relatedness) and
cognitive engagement
(Ding et al. 2017)
8 items, 4 dimensions,
5 Likert points
Interest in the activity, value for completing the
activity, perceived competence during the activity,
and effort completing the activity
(Filsecker and Hickey
5 items,
unidimensional, 5
Likert points
(Rachels and
4 items,
unidimensional, 0-100
(Banfield and Wilkerson
7 items,
unidimensional, 7
Likert points
(Santhanam et al. 2016)
25 items, 8 dimensions,
5 Likert points
Sense of control, concentration, clear goal,
challenge and skills, feedback, immersion,
knowledge improvement and interactivity.
(Su and Hsaio 2015)
20 items, 5 dimensions,
7 Likert points
Temporal dissociation, focused immersion,
heightened enjoyment, control and curiosity
(Santhanam et al. 2016)
Table 3. Experiential Outcome Scales
(1) Scale acronyms: IMMS = Instructional Materials Motivation Survey (Keller 2009); EQ = Motivational variables from Engagement Questionnaire (Greene et al. 2004); IMI = Intrinsic
Motivation Inventory (Ryan 1982; Ryan et al. 1991); IMN = Intrinsic motivation needs (Baard et al. 2004; Ryan et al. 1983); AMS = Academic Motivation Scale (Vallerand et al. 1992);
LSRQ = Learning Self-Regulation Questionnaire (Black and Deci 2000); SCE = Student Course Engagement (Handelsman et al. 2005); SES = School Engagement Scale (Fredricks et al.
2004); MSL = Motivated Strategies for Learning (Pintrich et al. 1991); CES = Cognitive Engagement Scale (Fredricks et al. 2004); PALS/AES = Pattern of Adaptive
Learning Scales’ (PALS) Academic Efficacy subscale (Midgley et al. 2000); SELFS = Self-Efficacy Scale (Bandura 2006); CLSE = Computer Learning Self-Efficacy (Zweig and Webster
2004); FBS = Flow-Based Scale (Su and Hsaio 2015) and CAS = Cognitive Absorption Scale (Agarwal and Karahanna 2000).
(2) Theory acronyms: AGT = Achievement Goal Theory (Elliot 1999); SDT = Self-Determination Theory (Ryan and Deci 2000); CET = Cognitive Evaluation Theory (Deci and Ryan 1980);
ARCS = Attention, Relevance, Confidence and Satisfaction (Keller 2009); IEST = Incremental and Entity Self-Theories (Dweck 1999); SET = School Engagement (Fredricks et al. 2004);
SE = Self Efficacy (Bandura 1977); SOFT = State of Flow Theory (Csikszentmihalyi 1990); CE = Cognitive Engagement (Webster and Ho 1997) and TOA = Traits of Absorption (Tellegen
and Atkinson 1974).
... Salah satu metode pembelajaran alternatif untuk mata kuliah praktik yang dapat diterapkan dalam kondisi luar biasa seperti ini adalah pendekatan gamifikasi, dimana peserta didik diberikan kesempatan mempelajari pengetahuan dan keterampilan melalui penerapan elemen-elemen permainan dalam konteks non-permainan (Inocencio, 2018;Kiryakova et al., 2014) tanpa harus bertatap muka secara langsung. ...
... Penerapan pendekatan gamifikasi merupakan salah satu pendekatan pembelajaran yang secara umum telah diterapkan pada konteks keadaan normal (non-pandemi) (Fui-Hoon Nah et al., 2014;Hanus & Fox, 2015;Inocencio, 2018;Kiryakova et al., 2014). Namun demikian, dalam konteks keadaan pandemi, penerapan solusi gamifikasi sebagai pendekatan pembelajaran praktik pada institusi pendidikan vokasi perlu diuji untuk mendapatkan gambaran persepsi peserta didik terhadap motivasi belajar setelah berinteraksi di dalam sistem gamifikasi. ...
... Motivasi merupakan salah satu konstruk yang umum dipelajari dalam berbagai riset mengenai gamifikasi (Fui-Hoon Nah et al., 2014;Inocencio, 2018). Menurut Deci & Ryan (2012) motivasi merupakan sesuatu yang menyatu dalam diri manusia sejak lahir, bukan merupakan sesuatu yang dipelajari, namun dapat berkembang atau terhambat sebagai efek dari lingkungan sosial manusia. ...
Full-text available
Penerapan gamifikasi pendidikan adalah salah satu solusi yang secara umum telah diterapkan pada konteks keadaan normal (non-pandemi). Dalam penelitian ini penerapan solusi gamifikasi diuji sebagai alternatif pendekatan pembelajaran praktik pada institusi pendidikan vokasi dalam masa pandemi. Perangkat interaksi berupa purwarupa aplikasi web gamifikasi peralatan konstruksi dibuat dan diujicobakan kepada 82 mahasiswa Politeknik Pekerjaan Umum yang telah mendapatkan mata kuliah Peralatan Konstruksi. Instrumen ukur dikembangkan dengan skala Likert 7-poin untuk mengukur manfaat gamifikasi terhadap motivasi belajar. Hasil uji validitas butir menunjukkan 33 dari 34 butir dalam instrumen valid untuk digunakan (r-hitung r-tabel=0.244), sedangkan hasil uji reliabilitas terhadap 7 (tujuh) dimensi instrumen ukur penelitian menunjukkan bahwa 6 (enam) dimensi reliable untuk digunakan dalam instrumen ukur penelitian (Alpha Cronbach ≥ 0.6 yaitu Interest/Enjoyment=0.823, Perceived Competence=0.804, Pressure/Tension=0.737, Value/Usefullness=0.812, dan Outcome=0.785). The application of educational gamification is a solution that has generally been applied in the context of normal (non-pandemic) circumstances. In this study, the application of gamification solutions was tested as an alternative approach to practical learning in vocational education during the pandemic. An interaction device in the form of a gamification web application prototype for construction equipment was made and tested on 82 Polytechnic of Public Works students who had received the Construction Equipment course. The measuring instrument was developed with a 7-point Likert scale to measure the benefits of gamification on learning motivation. The results of the item validity test show that 33 of the 34 items in the instrument are valid to use (r-count r-table=0.244), while the results of the reliability test on 7 (seven) dimensions of the research measuring instrument show that 6 (six) dimensions are reliable to be used in research measuring instrument (Cronbach's Alpha ≥ 0.6, namely Interest/Enjoyment=0.823, Perceived Competence=0.804, Pressure/Tension=0.737, Value/Usefullness=0.812, and Outcome=0.785).
... Scope Items Databases 1 Year [18] Higher education 1029 WoS 2019 [19] (unrestricted) 313 WoS 2018 [20] Management 244 Scopus, WoS 2017 [21] (unrestricted) 139 WoS 2014 [22] Empirical research 128 Scopus 2015 [23] (unrestricted) 119 Scholar, WoS, Scopus, ResearchGate, Academia 2014 [24] (unrestricted) 95 EBSCO, ScienceDirect, AISeL 2018 [25] Statistics 49 Scopus, AISeL 2019 [26] Engineering 48 IEEE 2019 [27] Tailored gamification 42 ACM, IEEE, ScienceDirect, Scopus, Springer 2019 [28] Empirical research 41 ACM, IEEE, ScienceDirect, Scopus, ERIC, Scholar 2015 [29] Higher education 41 ACM, EBSCO, ASME, IEEE, PsychINFO, Scopus 2017 [30] Information Systems 41 AISeL, ACM 2016 [31] Peer review 39 ACM, IEEE, ScienceDirect, Springer, Scopus, WoS, ERIC 2018 [32] Empirical research 34 ACM, IEEE, ScienceDirect, Scopus, Springer, ERIC, Scholar 2014 [33] MOOCs 34 ACM, IEEE, ScienceDirect, Scopus, Springer, 2017 [34] Higher education/STEM 30 WoS 2016 [17] (unrestricted) 26 ACM, ScienceDirect, IEEE, Scopus, Springer 2013 [35] Empirical research 24 EBSCO, Proquest, WoS, Scopus, ScienceDirect, Scholar, ACM, AISeL 2013 [36] Software engineering 21 ACM, IEEE, Scopus, ScienceDirect, WoS 2017 [37] Adaptive gamification 20 ACM, IEEE, ScienceDirect, Springer, Scholar 2019 [38] Computer Science 16 ACM, IEEE, ProQuest, Web of Science 2017 ...
... Looking at the contents of Table 1, most of the listed surveys are focused on a specific education level [18,29,34], subject [20,[25][26][27]30,31,33,34,[36][37][38] or reported research type [22,28,32,35], with the remaining ones were either outdated [17,21,23] or using too restricted search criteria [19] and/or selection of sources [24] to achieve an adequate coverage of the state of research on gamification in education. We therefore identify a research gap in the lack of an up-to-date survey of the scientific output in this field, not restricted to its particular subdomain or type of research. ...
Full-text available
Recent years have brought a rapid growth of scientific output in the area of gamification in education. In this paper, we try to identify its main characteristics using a bibliometric approach. Our preliminary analysis uses Google Scholar, Scopus, and Web of Science as data sources, whereas the main analysis is performed on 2517 records retrieved from Scopus. The results comprise the cross-coverage of databases, geographic distribution of research, forms of publication, addressed research areas and topics, preferred publishing venues, the most involved scientific institutions and researchers, collaboration among researchers, and research impact. The main conclusions underline the sustained growth of the research output in the area for at least seven years, the widespread interest in the area across countries and branches of science, and an effective research communication in the area documented by the number of citations and the map of co-citations.
... Student engagement is enhanced by overcoming boredom associated with certain activities and fostering engagement in learning activities, which positively affect learning outcomes. Many recent studies such Filomena and Maria (2015), Inocencio (2018), Öztürk and Korkmaz (2019) and Legaki et al. (2020) have suggested that gamification can add the elements of enjoyment and novelty in teaching and learning sessions by motivating and encouraging students to collaborate with other students, express emotions, and participate actively in learning activities. This study aims to explore ways to encourage effective collaboration among students in the teaching and learning of building morphology topics. ...
... In the contemporary digital era, introducing computational thinking concepts is considered an imperative need at all stages of schooling, since they are inextricably linked to skills applicable and beneficial in everyday life [4]. Researchers and educators around the world have found that play education is by far the best way for a child to acquire skills [5,6]. ...
Full-text available
In Romania of 2021, where society is constantly tempted by numerous changes in methodology and constrained by the development of the didactic act mainly online, as a result of the pandemic, education through play can be the long-awaited answer. The game itself can take various forms, sometimes the most interesting. The present paper seeks to present such a form. Starting from education through play, the paper aims to outline another possible dimension of the way of conducting the act of teaching-learning assessment and obtaining feedback. Thus through a series of worksheets, education through play is punctuated and finely delimited, as a bridge between mobile applications and philately. A series of exercises are proposed, which can be carried out in the mixed work variant. They can successfully provide lessons involving fun math or ecology, but the range of uses can be expanded. They can be used as materials in teaching new knowledge and in evaluating existing ones. This study certifies that education through play knows no limitations as long as the trainers has several key skills and has access to various resources.
... Gamification uses ludic activities to achieve a goal. It is widely used in education and the use and results has been widely documented by different authors [11]. Gamification is used here as an educational tool to design and fabricate a boomerang. ...
Full-text available
Engineering competences are characteristics, knowledge and skills that allow engineers to create in a challenging and dynamic environment. Higher education institutions must develop the competences that prepare engineering students to face modern society challenges. Here, it is presented a study case to develop engineering competences using low cost computational tools in combination with 3-d printing. A ludic activity is developed by undergrad students majoring Mechanical and Mechatronics Engineering. The design and fabrication of a boomerang by a group of students is analyzed and how they develop relevant competences during this process. The students are exposed to computational tools used during the project. First, databases for aerodynamic profiles are introduced. The databases used include curves of drag and lift coefficients performance for different attack angles. Students develop the understanding and the handling of those databases to create an aerodynamic profile for a boomerang. The computational tool used for numerical software is matlab and the g-code generation for 3-d printing used is ultimaker Cura. The methodology to design a boomerang using the tools mentioned is explained. Then, the development of engineering competences by the students during the activity is analyzed and discussed. It is observed that using a ludic educational activity in a short period using open access databases and numerical analysis computational tools is of great use to develop competences.
... Gamification uses ludic activities to achieve a goal. It is widely used in education and the use and results has been widely documented by different authors [11]. Gamification is used here as an educational tool to design and fabricate a boomerang. ...
Full-text available
Engineering competences are characteristics, knowledge and skills that allow engineers to create in a challenging and dynamic environment. Higher education institutions must develop the competences that prepare engineering students to face modern society challenges. Here, it is presented a study case to develop engineering competences using low cost computational tools in combination with 3-d printing. A ludic activity is developed by undergrad students majoring Mechanical and Mechatronics Engineering. The design and fabrication of a boomerang by a group of students is analyzed and how they develop relevant competences during this process. The students are exposed to computational tools used during the project. First, databases for aerodynamic profiles are introduced. The databases used include curves of drag and lift coefficients performance for different attack angles. Students develop the understanding and the handling of those databases to create an aerodynamic profile for a boomerang. The computational tool used for numerical software is matlab and the g-code generation for 3-d printing used is ultimaker Cura. The methodology to design a boomerang using the tools mentioned is explained. Then, the development of engineering competences by the students during the activity is analyzed and discussed. It is observed that using a ludic educational activity in a short period using open access databases and numerical analysis computational tools is of great use to develop competences. 
Full-text available
The research on educational gamification spans many topics of interest. As the total volume of research in this area has greatly increased in the last 10 years, it is interesting to see how the interest in the respective topics has changed over the same period. In this paper, we answer this question by means of keyword analysis performed on 7572 unique keywords extracted from 2203 papers. The obtained results reveal (1) the high popularity of keywords that are non-obviously relevant to gamification, (2) vast disproportions in the volume of research dedicated to different aspects of the same research sub-area, and (3) differing patterns of popularity among the most frequent keywords, as well as keywords introduced and abandoned in recent years (4). The presented findings bear a number of implications for the future of research on educational gamification.
Full-text available
This article is a journey through the advances and advantages of the gamification in education, based on the research and scientific studies identified as the most relevant. It focuses on collecting the most significant articles, texts, books and conferences, and making this state of the art as up-to-date as possible.
Full-text available
South Africa has a high-cost, low-performance education system, which ultimately leads to unemployment and a skill shortage in the country. In order to bridge the skill shortage gap at a tertiary level, the Stellenbosch Learning Factory (SLF) was established. Learning factories involve experiential learning in a production environment through 'learning by doing'. Gamification, one of the teaching methods used in the SLF, is investigated as a possible answer to South Africa's educational problems. Learning factories can be used to train employees: the knowledge transfer resulting from real production conditions is favoured because process improvements can be implemented or practised without any real production downtime. The aim of this study was to determine the learning contribution of the games implemented at the SLF. This was accomplished by developing a three-dimensional matrix that employs a revised version of Bloom's taxonomy to measure the learning success of the educational games at the SLF.
Full-text available
We evaluated the use of gamification to facilitate a student-centered learning environment within an undergraduate Year 2 Personal and Professional Development (PPD) course. In addition to face-to-face classroom practices, an information technology-based gamified system with a range of online learning activities was presented to students as support material. The implementation of the gamified course lasted two academic terms. The subsequent evaluation from a cohort of 136 students indicated that student performance was significantly higher among those who participated in the gamified system than in those who engaged with the nongamified, traditional delivery, while behavioral engagement in online learning activities was positively related to course performance, after controlling for gender, attendance, and Year 1 PPD performance. Two interesting phenomena appeared when we examined the influence of student background: female students participated significantly more in online learning activities than male students, and students with jobs engaged significantly more in online learning activities than students without jobs. The gamified course design advocated in this work may have significant implications for educators who wish to develop engaging technology-mediated learning environments that enhance students’ learning, or for a broader base of professionals who wish to engage a population of potential users, such as managers engaging employees or marketers engaging customers.
Full-text available
Purpose Anatomy is fundamentally an essential curriculum in health professions education. There are various commercial platforms providing learning materials in anatomy; the contents covered in great details are, however, not specifically designed for students majoring in non-medical programmes, such as nursing and pharmacy. To support Anatomy education, this study explored the feasibility of applying blended learning approach composed of narrative animation, interactive revision guide, and gamified quiz in the development of a courseware called electronic Professional Study (ePS). Method The Cardiovascular system was selected as the pilot theme as it is one of the most commonly diagnosed disorders in the local population. Under the central theme, three micro-modules were developed, including Heart Structure Investigation, Coronary Circulation, and Histology of Blood Vessels. This paper in two parts describes the development and components of the ePS courseware and presents preliminary findings of the evaluation conducted among courseware users. Results ePS was successfully launched in the university-wide learning platform, Blackboard Learn, where access is available to Pharmacy students attending the Anatomy course. Student's opinion about the courseware was surveyed at the end of the term. Study findings reported that blended learning with gamify components could function as positive reinforcement encouraging self-learning. Conclusions This study shows that the gamification design elements included in ePS are advocated to address students' needs in Anatomy learning and could be applied in other science-related learning and teaching in the Faculty of Medicine.
Conference Paper
Full-text available
Over the last 10 years, research on gamification, the use of game elements in non-game contexts, has increased in the field of education, due to its potential to enhance learning performance. Yet, the majority of available research rather focuses on the evaluation of motivation and engagement as key dependent variables. Hence, the purpose of this study is to review available studies on gamification, with an exclusive focus on learning performance as the key dependent variable. Through a systematic search and selection process, building on “Web of Science” articles and by considering studies between 2000 and 2016 related to gamification and learning, 582 articles were identified. Further inclusion and exclusion criteria, regarding setting (education), study focus (empirical), journal access (full access) and dependent variables (learning performance), resulted in a review of 23 articles meeting the criteria. The analysis of these articles showed how gamification could be linked to a direct increase in learning performance of students. Nevertheless, some studies also reflect weaker statistical differences between being involved or not in a gamified environment. The review analysis results are especially helpful to define a future agenda for gamification research, addressing the following gaps in the literature. First, include mediating and moderating variables to find more empirical research that can prove an indirect effect of gamification on learning performance. Second, carry out additional research that empirically underpins the direct linkage between gamification and learning performance. Third, include specific individual gamification elements to be able to determine explicit differential effects of these elements on learning performance. Fourth, conduct research in a broader range of knowledge fields to develop empirical evidence in the context of other knowledge domains next to computer sciences. Finally, consider involving larger sample and setting up longer experimental interventions, to avoid novelty effects and risks of lack of generalization.
This study investigated the effectiveness of adding four self-efficacy features to an online statistics lesson, based on Bandura’s four sources of self-efficacy information. In a randomized between-subjects experiment, participants learned statistical rules in an example-based online environment with four self-efficacy features added (treatment group) or not (control group). Results of analyses of variance showed that the treatment group performed better on practice (d = 0.36), retention (d = 0.39), and transfer (d = 0.42) tests as well as reporting higher self-efficacy (d = 0.44) and lower task anxiety (d = −0.45). Further, mediation analyses revealed that the effect of treatment group on performance was fully mediated by task anxiety and self-efficacy. The results support the inclusion of self-efficacy features in online mathematics lessons, when the goal is to improve learning outcomes by reducing anxiety and increasing self-efficacy. The results show the utility of applying Bandura’s model of self-efficacy to technology-based learning environments.
Over the last few years, the implementation of game elements like badges in non-game environments has become increasingly popular (Butler, 2014). In this study, we tested whether badges, which could be received for successful task performance and specific activities within an e-learning course in a higher education setting, had an impact on students' motivation and performance. In a between-subjects experimental field study, students were randomly assigned to three different conditions (no badges, badges visible to peers, badges only visible to students themselves). The results show that badges have less impact on motivation and performance than is commonly assumed. Independent of condition, students’ intrinsic motivation decreased over time. Contrary to expectation, the badges that could only be viewed by the students themselves were evaluated more positively than those that could also be viewed by others.
A quasi-experimental, pretest-posttest, non-equivalent control group design was used to examine the effect of a mobile gamification application on third and fourth grade students’ Spanish language achievement and student academic self-efficacy. In this study, the treatment group's Spanish language instruction was through the use of Duolingo®, a computer and mobile app that uses gamification and adaptive learning technology to teach foreign languages. Students in the control group received their regularly scheduled English L1/Spanish L2 class learning activities. The study was 12 weeks in duration. Students were assessed with a 50-question, multiple-choice English to Spanish and Spanish to English pretest covering vocabulary and grammar to control for prior Spanish language achievement. Students were assessed with the Pattern of Adaptive Learning Scales’ (PALS) Academic Efficacy subscale to control for prior academic self-efficacy. The same two instruments were used as posttests. An analysis of covariance showed no significant difference in students’ Spanish achievement or in academic self-efficacy between students who used Duolingo® and students who were taught with traditional face-to-face instruction. This demonstrates that Duolingo® is a useful tool for teaching Spanish to elementary students.
Background. The theory of gamified learning (Landers, 2014) posits that gamified approaches positively impact students’ attitudes, and in turn this change in attitudes impacts learning; however, research is needed to examine the role of attitude change in gamified approaches (Seaborn & Fels, 2014). A strong negative relationship between students’ attitudes towards statistics and their performance in statistics has been well documented. The need to help students have positive attitudes towards statistics, and therefore be more likely to achieve in the course, makes using gamified learning, which targets attitudes, an ideal domain to test the effects of gamification on attitudes.