Gamification of Adult Learning: Gamifying Employee Training and Development

Abstract

Gamification is now commonly used in adult learning contexts, but its effects remain unclear. This has happened in part because of the initially trendy and faddish nature of gamification leading to high rates of adoption without significant critical evaluation. This was most problematic in the years leading up to peak hype in 2013, at which point “gamification” was used as a catchall faddish buzzword that did not refer to any particular construct or approach, instead being used primarily as a marketing strategy. Since then, as gamification has been explored in the academic research literature, these broader problems have been largely addressed via carefully considered theoretical and empirical studies. Nevertheless, empirical work in particular is still relatively sparse; the last major published summary of such work only identified nine empirical gamification studies in the learning context as of 2014. Since then, the literature has grown, but there are still many unanswered questions among both gamification practitioners and academics. Among non-specialists, there is still substantial construct confusion stemming directly from gamification’s initially faddish definition. Given this, the purpose of this chapter is threefold. First, we define gamification and provide a comprehensive introduction to it, contrasting it with existing approaches. Second, we explore which theories describe its known and potential effects. Third, we provide extensive practical literature-driven recommendations for those seeking to gamify training themselves.

Full text

Title: Gamification of Adult Learning: Gamifying Employee Training and Development
Authors: Richard N. Landers, Elena M. Auer, Adrian Helms, Sebastian Marin, Michael B. Armstrong
Abstract
Gamification is now commonly used in adult learning contexts, but its effects remain unclear. This has
happened in part because of the initially trendy and faddish nature of gamification leading to high rates of
adoption without significant critical evaluation. This was most problematic in the years leading up to
peak hype in 2013, at which point “gamification” was used as a catchall faddish buzzword that did not
refer to any particular construct or approach, instead being used primarily as a marketing strategy. Since
then, as gamification has been explored in the academic research literature, these broader problems have
been largely addressed via carefully considered theoretical and empirical studies. Nevertheless, empirical
work in particular is still relatively sparse; the last major published summary of such work only identified
nine empirical gamification studies in the learning context as of 2014. Since then, the literature has
grown, but there are still many unanswered questions among both gamification practitioners and
academics. Among non-specialists, there is still substantial construct confusion stemming directly from
gamification’s initially faddish definition. Given this, the purpose of this chapter is threefold. First, we
define gamification and provide a comprehensive introduction to it, contrasting it with existing
approaches. Second, we explore which theories describe its known and potential effects. Third, we
provide extensive practical literature-driven recommendations for those seeking to gamify training
themselves.
Citation
Landers, R. N., Auer, E. M., Helms, A. B., Marin, S., & Armstrong, M. B. (2019). Gamification of adult
learning: Gamifying employee training and development. In R. N. Landers (Ed.), Cambridge
Handbook of Technology and Employee Behavior (pp. 271-295). New York, NY: Cambridge
University Press.

Gamification is now commonly used in adult learning contexts, but its effects remain unclear.
This has happened in part because of the initially trendy and faddish nature of gamification leading to
high rates of adoption without significant critical evaluation. This was most problematic in the years
leading up to peak hype in 2013 (Gartner, 2013), at which point “gamification” was used as a catchall
faddish buzzword that did not refer to any particular construct or approach, instead being used primarily
as a marketing strategy (Bogost, 2011). Since then, as gamification has been explored in the academic
research literature, these broader problems have been largely addressed via carefully considered
theoretical and empirical studies. Nevertheless, empirical work in particular is still relatively sparse; the
last major published summary of such work only identified nine empirical gamification studies in the
learning context as of 2014 (Hamari, Kovisto & Sarsa, 2014). Since then, the literature has grown, but
there are still many unanswered questions among both gamification practitioners and academics. Among
non-specialists, there is still substantial construct confusion stemming directly from gamification’s
initially faddish definition. Given this, the purpose of this chapter is threefold. First, we define
gamification and provide a comprehensive introduction to it, contrasting it with existing approaches.
Second, we explore which theories describe its known and potential effects. Third, we provide extensive
practical literature-driven recommendations for those seeking to gamify training themselves.
Defining Gamification
The most critical issue to address when a new research literature forms around a seemingly new
organizational construct is to develop a formal definition. As noted earlier, the lack of an agreed-upon
definition of gamification for the early years of its popularity harmed both public perception of its value
and researcher progress in studying it. In short, anything branded “gamification” was immediately
suspect, first because gamification was itself not considered a new or unique concept, and second,
because many of the gamification implementations getting press coverage were at best gimmicky and at
worst actively harmful. The term gamification was most often used to capitalize on hype in the
consulting marketplace as a means of generating profit (see Bogost, 2011). Since that hype subsided, it
has been replaced with an understanding that gamification has significant potential but is easy to conduct

poorly. In the academic community, a relatively strong consensus has emerged around definitions
developed by a small set of researchers, and the term gamification now means something much more
specific than it did previously. This view of gamification has even become the center of a new
gamification science (Landers, in press). We will explore definitions of gamification in the remainder of
this section.
Distinguishing Games, Serious Games, and Gamification
Despite their apparent similarity, games, serious games, and gamification are distinct concepts
focused on different objectives and principles. Games and serious games are most closely related.
Though researchers do not agree upon a singular definition, games can be generally defined as “a
voluntary activity, obviously separate from real life, creating an imaginary world that may or may not
have any relation to real life and that absorbs the player’s full attention” (Michael & Chen, 2005, p. 8).
Michael and colleagues (2005) further clarified that “games are played out within a specific time and
place, [and] are played according to established rules” (p. 8). Such games are typically created for
entertainment purposes as a composite of many game elements working in combination. Game elements
can be defined as features or mechanics of play typically found in games (e.g., fantasy, control,
environment, etc.; Deterding, Dixon, Khaled & Nacke, 2011). In contrast, serious games, also referred to
as educational games or games for learning, are “game[s] in which education (in its various forms) is the
primary goal, rather than entertainment” (Michael & Chen, 2005, p.10). Serious games can be used to
directly facilitate learning in several educational and training contexts, including, military, government,
education, business, and healthcare (Garris, Ahlers, & Driskell, 2002). For example, the game America’s
Army (http://www. americasarmy.com) is a video game created by the US military that trains players on
military tactics using single-player and multi-player missions as a way to teach them about military
combat and to encourage them to enlist. In the context of employee training and development, both games
and serious game can be considered an instructional method, where individual game elements can be
designed to teach the player some targeted knowledge or skill. In this model, the game acts as a virtual
instructor, providing information to the learner. In America’s Army, for example, the user learns about

military tactics by playing the game. Without playing America’s Army, there would be no instruction.
Both games and serious games have been shown to be effective methods in improving learning outcomes
(e.g., Sitzmann, 2011) and, given their similarity in composition, can inform gamification research and
practice. However, gamification is fundamentally distinct from games and serious games.
Instead, gamification in learning contexts is an instructional design process, and not an
instructional method, applied to existing instructional methods to improve target outcomes. More
generally, gamification is the process of adding game design elements to non-game contexts (Deterding,
Dixon, Khaled & Nacke, 2011), and learning can be considered a non-game context. Serious games and
gamification are unified by their use of game elements to improve learning outcomes. However, gamified
instructional methods, instead of being standalone game experiences with many game elements, only
include one or a few game elements added to an existing instructional program. Existing instructional
methods, such as an online training videos, are what deliver instructional content to the learner, and the
addition of game elements to those methods is intended to alter intermediary proximal learning behaviors
or attitudes to improve learning outcomes more distally. In summary, a user can learn directly from a
game but not from gamification. Instead, gamification is used to improve learning that is already
occurring or to overcome some psychological roadblock preventing learning in a system that is otherwise
functional (Landers, 2015). For example, a gamification intervention could be as simple as adding a
progress bar to an employee leadership training slide presentation. In this case, a progress bar would not
be the method by which a user learned, and consequently could not improve the training if the existing
content was inadequate. However, if the instructional content facilitated learning, the progress bar could
indirectly improve learning by increasing a user’s motivation to learn by providing progress feedback.
Theory of Gamified Learning
The theory of gamified learning can be used to understand the potential impact of gamification
and consists of two components: game element attribute categories and a process model (Landers, 2015).
In summary, game element attribute categories (see Table 1), originated by Bedwell and colleagues
(2012) in the context of serious games, provide a theoretically based yet practical framework for

implementing individual game elements in the learning context. These categories organize game elements
that have been previously linked to learning outcomes for application to non-game instructional methods.
The process model (see Figure 1), the second component of the theory, explains the indirect effect of
these elements on learning outcomes (Landers, 2015). In this model, gamification affects learning through
learning-related behaviors and attitudes by way of either mediating or moderating processes. In the
mediating process, game elements drive learning-related behaviors and attitudes, which are the underlying
mechanism for improving learning outcomes. In the moderating process, game elements strengthen or
weaken the existing relationship between learning-related behaviors and learning outcomes. In
combination, the game element attribute categories and process model provide a parsimonious theoretical
framework of gamification in a training and development context.
Table 1
Theory of Gamified Learning Game Element Attribute Categories and Definitions
Game Element Attribute
Categories
Definition
Action Language
The method and interface by which communication occurs between a
player and the game itself.
Assessment
The method by which accomplishment and game progress are tracked.
Conflict/Challenge
The problems faced by players, including both the nature and difficulty
of those problems.
Control
Degree to which players are able to alter the game, and the degree to
which the game alters itself in response.
Environment
The representation of the physical surrounding of the player.
Game Fiction
The fictional game world and story.
Human Interaction
The degree to which players interact with other players in both space
and time.
Immersion
The affective and perceptual experience of a game.
Rules/Goals
Clearly defined rules, goals, and information on progress toward those
goals, provided to the player.
Note. Table excerpted from Landers (2015).

Figure 1. Causal path model of the theory of gamified learning (Adapted from Landers, 2015).
Game element attribute categories can be defined as broad groupings of learning-related game
elements, organized by shared psychological attribute (Landers, 2015). Using Wilson and colleague’s
(2009) list of game elements, Bedwell and colleagues (2012) taxonomized game elements into nine
categories of attributes that facilitate learning related behaviors or attitudes. The categories of elements
include action language, assessment, conflict/challenge, control, environment, game fiction, human
interaction, immersion, and rules/goals. This taxonomy, while not comprehensive or exhaustive, was
empirically derived from existing research on serious games to provide guidance on which attribute
categories are most likely to affect learning outcomes (Bedwell et al., 2012). For example, adding
assessment to a training activity might include adding points that track correct answers in an existing
training module to track a learner’s progress.
The application of each of these elements to a learning context can also be described in terms of a
variety of psychological theories (Landers, Armstrong & Collmus, 2017). For example, according to self-
determination theory, which posits that motivation is rooted in the fulfillment of autonomy, competence,
and relatedness needs (Ryan & Deci, 2000), control game elements may satisfy a learner’s need for
autonomy and ultimately improve their motivation (Landers, Armstrong & Collmus, 2017). Similarly,
according to the theory of test-enhanced learning (Roediger & Karpicke, 2006), adding assessment game
elements may improve learning by triggering cognitive retrieval of previously learned content (Landers,
Armstrong & Collmus, 2017).
These learning-related elements can also be adopted individually or in meaningful combinations.
Game Element Learner
Behavior/Attitude
Existing Instructional
Content/Method Learning Outcome

In practice, adding combinations of elements may increase the change in learning behaviors or attitudes
over any particular element in isolation. However, in research, it is important isolate elements
individually or in meaningful combinations so that specific guidance can be provided on which
gamification elements lead to which behaviors. For example, Landers and Callan (2011) added several
game elements to instructional material, including assessment, challenge, human interaction, and
rules/goals. These elements were intentionally chosen to motivate students to complete optional training
tests that would increase time spent on learning materials and ultimately improve learning outcomes
(Landers & Callan, 2011). Although students reported favorable reactions to the gamified system when
compared to the non-gamified system, the specific elements that led to favorable reactions could not be
determined because the effect of each element could not be differentiated. It is impossible to determine
post hoc if the effect was caused by particular elements (i.e., main effects) or their combination (i.e., an
interaction).
The process model in the theory of gamified learning presents several types of causal
relationships among instructional content, learning-relevant behaviors and attitudes, learning outcomes,
and game characteristics. One fundamental causal relationship in the model is of the effect of instructional
content on behaviors/attitudes (Landers, 2015). Improved instructional content has repeatedly been shown
to alter learner behaviors and attitudes (e.g., Kulik, Kulik, & Cohen, 1980; Norris & Ortega, 2000;
Arthur, Bennett, Edens, & Bell, 2003; Graham & Perin, 2007; Seidel & Shavelson, 2007). This
relationship is fundamental because gamification cannot replace instructional content and therefore will
not improve learning outcomes in cases of completely ineffective existing content. For example, if a
leadership training model only covers material trainees are already familiar with, adding game elements
to the model will not improve learning behaviors or attitudes. Instead, training needs should be re-
evaluated, and content should be altered to fit those needs. Another causal relationship is between
behaviors/attitudes and learning outcomes, meaning learning attitudes affect learning outcomes (e.g.,
Zhao & Kuh, 2004; Paas, Tuovinen, van Merrienboer, & Darabi, 2005; Carini, Kuh, & Klein, 2006). For
example, increased student engagement can improve performance on critical thinking tests (Carini, Kuh,

& Klein, 2006) and increased motivation to learn can lead to improved skill acquisition (Colquitt, LePine,
& Noe, 2000). A third causal relationship exists between game characteristics and behaviors/attitudes
(e.g., Wilson et al., 2009; Tay, 2010; Bedwell et al., 2012; Hamari, Koivisto & Sarsa, 2014). For example,
the use of more specified goals can improve motivation, and ability-dependent adaptation of content may
improve learner cognitive strategies (Wilson et al., 2009). Because of this causal relationship, improving
learner behaviors and attitudes is the primary goal of theoretically-derived gamification interventions. All
three of these direct causal relationships exist in both the mediating and moderating process, which
describe how game elements indirectly affect learning outcomes.
The model presents two specific paths by which game elements affect learning. The first is the
mediation of the relationship between game elements and learning outcomes by learning-relevant
behaviors and attitudes. Mediation refers to sequential causal effects between constructs (Baron & Kenny,
1986). Here, game characteristics affect learning outcomes via behaviors and attitudes, the causal
mediator (Landers, 2015). For example, adding game elements that increase the amount of time a trainee
spends on the training can ultimately improve learning outcomes, but only because time-on-task causes
increased learning (Landers, 2015). In the case of Landers and Callan’s (2011) study, gamified practice
tests were meant to improve learning by increasing time spent on learning material, which has been
shown to improve learning outcomes (Brown, 2001). In this example, the relationship between game
characteristics (assessment, challenge, human interaction, and rules/goals) and learning outcomes is
mediated by time spent on the training (behavior). In a training context, gamification is most effective via
mediation when game element(s) specifically encourage a behavior/attitude that will improve learning
outcomes. Targeting a behavior or attitude with gamification that does not relate to a learning outcome
would likely be ineffective or result in an unexplained improvement in learning outcomes. For example,
using gamification to make training more fun would likely not improve learning outcomes if increased
fun did not itself improve learning outcomes. Furthermore, if learning outcomes did improve despite a
lack of relationship between fun and learning outcomes, this implies the designer’s understanding of why
their gamification worked is incomplete; another, unmeasured mediational variable has been affected.

Similarly, choosing game elements that do not influence target desired behaviors or attitudes would also
likely lead to a failed intervention. For example, if fun did improve learning yet gamification did not lead
to fun, gamification would likely not improve learning outcomes (Landers, 2015). Finally, it is also
important to note that instructional content simultaneously affects behaviors and attitudes, leading to
learning outcomes. So, it is important to consider how instructional content is beneficially contributing to
behaviors and attitudes and where gamification can approve upon behaviors and attitudes.
The second process by which game elements affect learning is the moderation of the relationship
between instructional content and learning outcomes by learning-relevant behaviors and attitudes altered
by game characteristics. Moderation occurs when the effect of one construct on another varies based on
the value of a third, the moderator (Baron & Kenny, 1986). In this moderating process, a game element
affects an attitude/behavior, which strengthens or weakens the relationship between instructional content
and learning outcomes. For example, gamifying a training module with fantasy may increase trainee
engagement, strengthening the relationship between instructional content and learning outcomes
(Landers, 2015). Gamification is most effective as a moderator when the game element(s) encourages a
behavior/attitude that will increase learning outcomes by improving upon instruction materials. Targeting
a behavior or attitude that does not moderate the relationship between instructional content and learning
outcomes would likely render the gamification intervention ineffective or, in the case of an effective
intervention, result in an unexplained improvement. For example, if narrative is added to an existing
training module to improve learner motivation, it should already be known that learner motivation is
linked to improved learning outcomes. If not, increasing motivation with a gamification intervention may
not lead to improved learning outcomes. Similarly, when there is ineffective instructional content,
incorporating a game element to improve behavior and attitudes would not be useful; it merely focuses
learner attention on something already known to be ineffective. Without sound instructional content,
gamification cannot improve learning outcomes through moderating effects.
Common Outcomes of Interest
The ultimate outcome of a gamification intervention is whatever change the practitioner chooses,

which in a training context is most commonly learning or transfer. Learning is typically defined as the
learning outcomes produced by experience and practice, which can be a change in cognitive, affective or
skill capacities (Kraiger, Ford & Salas, 1993). Transfer is the application of learning, how well a trainee
applies the knowledge, attitudes, or skills they learned in the program to their task or job (Burke,
Hutchins, 2007). In the context of the process model, learning and transfer are distal learning outcomes.
This means that although both learning and transfer are common targets of gamification interventions, it
is difficult to target those outcomes directly. Instead, the practitioner must target a mediating or
moderating process that is related to learning and transfer. For example, to indirectly target learning, an
action language can be added to a training module to improve trainee engagement. In this scenario,
trainee engagement, which is improved by the gamification intervention, is intended to strengthen the
relationship between instructional material and learning outcomes. Similarly, to target transfer, a
mediating process could be targeted. For example, to improve training transfer, a driving training module
could be gamified by incorporating challenge via time pressure. This game element may serve to increase
training fidelity, ultimately improving transfer of driver training.
Training reactions is another outcome frequently targeted by gamification interventions. Training
reactions are post-training opinions regarding the training program, including affective reactions,
perceptions of the training’s utility, and difficulty in justifying the effort required to perform the training
well (Kirkpatrick, 1959; Warr & Bunce, 1995). Training reactions are important for organizational
decision making, feedback, and marketing (Kraiger, 2002), and similar to learning and transfer, they are a
distal outcome that can be indirectly affected by gamification interventions. For example, Armstrong and
Landers (2017) examined the effects of adding narrative to a training module and found that trainees were
more satisfied with gamified training than a non-gamified version and that declarative knowledge gained
was similar between conditions; however, the narrative version appeared to adversely affect procedural
knowledge. Thus, prioritizing target outcomes before gamifying becomes critical. Other learning
outcomes may be unintentionally affected by targeting trainee satisfaction, which may or may not be
acceptable depending on the priorities of the designer. Prior to using a gamification intervention, it is also

important to differentiate which type of reaction is the target (i.e., affective, utility, or justification
reactions) to better match game elements to it. For example, if trainees find the training content itself to
be of low value, it might be more impactful to redevelop the instructional material than to intervene with
gamification. In contrast, if trainees have negative affective perceptions of the training, an intervention
like Armstrong and Landers’ narrative intervention may be useful.
Training motivation is a commonly targeted mediator in gamification interventions to improve
learning or transfer. Motivation explains the variation in intensity, persistence, quality, and direction of
behavior, and in a training context reflects the direction, intensity, and persistence of learning directed-
behavior (Mitchell, 1982; Kanfer, 1991). Numerous game elements have been shown to affect motivation
(e.g., Malone, 1981; Sailer, Hense, Mandl, & Klevers, 2013). For example, Malone (1981) examined the
effect of a variety of game elements, including assessment, game fiction, and immersion, on student
motivation to learn. He found that game fiction motivated students the most, followed by immersion and
assessment. Sailer and colleagues (2013) also identified specific game elements that are most likely to
increase motivation in gamification interventions: Points, badges, leaderboards, progress bars, quests,
meaningful stories, and avatars. For example, they suggested that badges cause motivation by fulfilling a
player’s need for success, by acting as a status symbol, by having a goal-setting function, and by fostering
a player’s feeling of competence (Sailer et al., 2013). Similarly, they suggested that avatars allow players
to have choices, which can foster feelings of autonomy and ultimately drive motivation (Sailer et al.,
2013). Increasing motivation using game elements has also been demonstrated to ultimately improve
learning outcomes (e.g., Parker & Lepper, 1992). Parker and Lepper (1992) applied fantasy elements to
children’s instructional materials to target motivation and found improved motivation and learning and
transfer. Because motivation typically makes pre-existing instruction better, rather than explicitly improve
learning, in the context of the process model, motivation is typically a moderator of the existing
relationship between instructional content and learning outcomes. In addition to the theory of gamified
learning, Landers, Bauer, Callan, and Armstrong (2015) identified four additional motivational
frameworks that explain how gamification effects training motivation including classic learning theories,

expectancy theory, goal-setting theory, and self-determination theory. Each of these motivational
frameworks will be discussed in more depth during later sections of the chapter.
Common Moderators of Interest
Moderators in gamified learning affect how gamification interventions affect outcomes
differently across people because of trait or situational context variation. Person-level moderators are
psychological constructs, or proxies for those constructs, that affect how well gamification interventions
work across different people. In other words, trainee characteristics can influence the effectiveness of
gamification. In the gamification literature, person-level moderators typically include experience with
games and game attitudes (e.g., Landers & Callan, 2012; Landers and Armstrong, 2015; Landers &
Armstrong, 2017), as well as proxy variables such as gender (e.g., Greenberg, Sherry, Lachlan, Lucas &
Holmstrom, 2010; Shen, Liu, Santhanam & Evans, 2016) and age (e.g. Koivisto & Hamari, 2014; Thiel,
Reisinger & Röderer, 2016). Experience with games and game attitudes have been shown to moderate the
relationship between gamified instructional content and the anticipated value of gamified instruction,
which is likely to persist to some degree post-training given the relationship between pre-training and
mid-training motivation (Landers & Armstrong, 2015). For learners with high game experience and
positive game attitudes, gamification led to improved anticipated learning outcomes. For learners with
little game experience and more negative game attitudes, however, gamification led to diminished
outcomes. Proxy variables, which include gender and age, are not causally related to the effectiveness of
gamification but correlated with psychological constructs that are. For example, age moderates the
effectiveness of gamification interventions in part because older adults tend to find gamified instruction
more difficult to use than non-gamified instruction (Kovisto & Hamari, 2014). In this case, age is not
necessarily causing gamified instruction to be more difficult to use; instead, generational differences in
traits related to technology may be contributing to this effect. Gender, another proxy variable, tends to
affect gaming preferences, in that males on average are more motivated by achievement game elements,
and females on average are more motivated by social game elements (Greenberg, Sherry, Lachlan, &
Holmstrom, 2010). Ultimately, because gamification can be helpful to some but harmful to others, it is

critical to consider the particular characteristics of any targeted trainee population.
Situational and contextual level moderators affect how well gamification interventions work
across different situations and in different organizational contexts. When implementing gamification, it is
important to consider a variety of situational moderators of training effectiveness, including
climate/culture, supervisor support, and employee buy-in. Further, it is important to understand how each
of these contextual influences is uniquely affected by gamification interventions. Climate and culture,
which include factors like organizational commitment for training and transfer (Darden, Hampton, &
Howell, 1989) and the opportunity or need for training (Ford, Quinones, Sego & Sorra, 1992), can affect
non-gamified training effectiveness and should be considered when implementing any training
intervention (Ostroff, Kinicki & Muhammad, 2013). When gamifying, consideration of organizational
climate for gamification is critical (Landers & Goldberg, 2014; Landers & Armstrong, 2015). Perceived
supervisor support is another contextual-level moderator that affects the success of a training
interventions (e.g., Noe & Schmitt, 1986; Ford, Quinones, Sego & Sorra, 1992; Foxon, 1997; Lim, 2001).
If supervisors view gamification negatively, the trainee will likely be less motivated or have less
favorable reactions towards the training (Landers & Callan, 2012). Lastly, employee buy-in and consent
are critical to the success of gamfied training interventions (Mollick & Rothbard, 2014; Heeter, Lee,
Magerko & Medler, 2011). When implementing gamified training, consent can moderate the response to
the gamification such that consent to gamification can increase positive affect while lack of consent can
decrease positive affect (Mollick & Rothbard, 2014). Given the potential for situational and contextual
variables to impact the effectiveness of a gamified training intervention, it is important to consider the
impact of these variables prior to gamifying existing training.
Relevant Psychological Theory
Although gamification is the process of modifying existing training using game elements and
design approaches derived from game science, this does not imply that the psychology underlying
gamification is new or unexplored. Most motivational concepts in gamification science are themselves
derived from psychology and contextualized to the context of playing games. Because of this, the games

and gamification literatures have explored contextualized psychological theories sometimes to a greater
extent than psychology has, and it is this literature that gamification leans on. In short, games researchers
often know more about how to use psychology to influence people’s behaviors in the context of a gameful
experience than psychologists do. Thus, understanding how to gamify training effectively requires
knowledge of relevant psychological theory as well as how such theory can be operationalized in a
gameful way. The remainder of this section will explore and provide an example of each of these
perspectives for each major set of gamification-relevant psychological theories.
Operant Conditioning
Operant conditioning is enacting a desired response by modifying behavior through two types of
“operants”: reinforcers and punishers (Skinner, 1953). Reinforcers, which are stimuli either added (i.e.,
positive reinforcers) or removed (i.e., negative reinforcers) from a baseline situation, increase the
probability that a behavior will be repeated. Positive reinforcement refers to the strengthening of behavior
through consequences that an individual believes to be rewarding, whereas negative reinforcement
strengthens behavior by removing an unpleasant stimulus. In contrast, punishers decrease the probability
of repeated behavior or discourage tendencies of behaving in targeted ways (Skinner, 1953).
Reinforcement schedules are specific plans for implementing positive or negative reinforcement
(Staddon, & Cerutti, 2003). There are four types of reinforcement schedules: fixed-interval, fixed-ratio,
variable-interval, and variable-ratio. A fixed-interval schedule issues a response to a reinforcing stimulus
after a specific time interval has passed, while in a fixed-ratio schedule issues a fixed number of responses
per stimulus (Ferster & Skinner, 1957). In a variable-interval schedule, the intervals for reinforcement
occur periodically, within randomized time-frames, whereas variable-ratio schedules issue reinforcers
randomly after a specific number of responses (Zieler, 1968; Ferster & Skinner, 1957). Implementing
reinforcement schedules can reinforce a desired set of behaviors, which in the case of gamification, can
lead to improved training outcomes if target behaviors, reinforcers, and punishers are chosen carefully.
Game elements like points and badges can be used as reinforcers to drive an individual to
complete a specific task or engage in a target behavior (Kapp, 2012). For example, Antin and Churchill

(2011) explained how badges could be utilized as reinforcers by attributing status, reputation, and group
identification to their attainment. Thus, if a badge can be designed to be perceived as a reward,
individuals will be more likely to engage in behaviors necessary for badge collection. Badge possession
may be viewed as rewarding if it elevate others’ perceptions of the badge-holder, provides information
about the badge-holder’s skills, expertise, and accomplishments, or creates a shared experience amongst
badge-holders. In addition to designing game elements as reinforcers, reinforcement schedules can be
used to improve the effectiveness of gamification (Linehan, Kirman, & Roche, 2015). For example,
points, badges, and other gamified elements appear to evoke behavior based on the kind of reinforcement
schedule given (Ferster & Skinner, 1957). For example, variable ratio schedules elicit high and steady
response rates and can be the most economical but have been criticized because the work produced by
them may be disproportional to the rewards offered, which could be viewed as exploitative (Linehan,
Kirman, & Roche, 2015).
By adding game elements intended to reward learners, a learning designer can modify learner
perceptions and potentially increase engagement on a subject. Stansbury and Earnest (2016), for example,
gamified an Industrial-Organizational psychology course. Students were randomly assigned to gamified
or traditional courses consisting of the same course material; however, the gamified course added game
elements of leveling up, feedback, exposition, and choice. These game elements reinforced students’
participation by associating these external motivators, which generally take the form of public
achievement recognition, with exposure to course material. The gamified course led to increased
perceptions of course content understanding, reinforcement of key concepts, and increased enjoyment of
course content compared to the traditional condition, suggesting some success from this gamification
intervention.
Expectancy-based Theories
Expectancy theories describe how people are motivated based upon their beliefs regarding
behaviors. The most well-known expectancy theory, proposed by Vroom (1964), describes how the
interaction between an individual’s expectancy, instrumentality, and valence leads to behavior (Lawler &

Suttle, 1973; Parijat & Bagga, 2014). Expectancy is the perceived probability that effort will result in an
immediate behavioral outcome. Instrumentality is the perceived probability that the behavior will result in
a reward. Valence is the attractiveness, value, or the liking of that reward. In most VIE models, these
three beliefs are quantified and multiplied to determine motivation. The expectancy-value model, which is
distinct from Vroom’s expectancy theory, is comprised of three slightly different factors of motivation:
expectancy, value, and an affective component (Pintrich & de Groot, 1990). In an expectancy-value
framework, expectancy is the individual’s belief in their ability to accomplish a task, value is the
individual’s thoughts of the importance of the task, and the affective component refers to the emotional
reaction of the individual to a task (Pintrich, Smith, Garcia, & McKeachie, 1993). Although they differ in
a few key ways, the expectancy-value model and Vroom’s expectancy theory both utilize expected
outcomes of engaging in a particular behavior as a motivator of that behavior.
By leveraging expectancy theories, game elements can be used to encourage motivation to
partake in specific tasks (Richter, Raban, & Rajaeli, 2015). Badges and leaderboards can be motivating by
eliciting feelings of status and reputation, achievement, and accomplishment. By these theories, game
elements are effective in motivating learners if the learner has high expectancies, instrumentality, and
valence for the behaviors and outcomes associated with the gamification intervention. For example,
badges and leaderboards can invoke competition by putting the behaviors of an individual in reference to
others (Blohm & Leimeister, 2013). An individual that succeeds in obtaining badges or climbing a
leaderboard may gain a sense of status and recognition, motivating them if they have high valence for
status and recognition (McNamara, Jackson, & Graesser, 2010). Points can also be understood through
the lens of expectancy theory. Points can provide a more evident connection between effort, performance,
and outcomes, increasing the point-associated behavior (Von Ahn and Dabbish, 2008). Therefore, game-
elements can increase the motivation of an individual by creating distinct relationships between targeted
behaviors such as collecting badges, earning points, or ranking on leaderboards, with the benefits of that
behavior. However, this is dependent upon the individual viewing these rewards as important or valuable.
Expectancy theory suggests that the connection between the expectancy of a desired outcome is

what motivates an individual; therefore, the addition of game-elements can be used to fortify this
connection. Using this approach, Browne, Anand, and Gosse (2014) explored the impact of gamifying a
learning application on low literacy adults’ ability to differentiate various homophones. The application
consisted of six different groups of homophones (e.g., it’s/its, your/you’re), which participants needed to
accurately differentiate. They gamified this process by incorporating badges (green/gold check marks),
levels, and goals. Participants in the experiment were individuals that wished to enhance their literacy;
therefore, they expected that their efforts would influence their behaviors on the gamified tasks. In regard
to instrumentality, gold and green check marks were immediately awarded after the completion of a level;
therefore, there was a clear connection between completing tasks and the rewards associated with them.
Self-Regulatory Theories
Self-regulation is defined as the maintenance and modification of personal goals, where goals are
internal representations of desired states (Vancouver & Day, 2005; Vancouver, 2008). An individual
regulates behavior to reduce the discrepancy a goal creates between actual performance and a desired
state of performance (Latham & Locke, 1991). Thus, self-regulation can act as a mediator between set
goals and performance (Kanfer & Ackerman, 1989). One self-regulatory theory that captures this
relationship is goal-setting theory, which states that difficult, specific goals prompt action because they
direct attention and action, inspire effort, increase persistence, and motivate the pursuit of improved
performance strategies (Locke & Latham, 1990, 2002). There are four key moderators of the goal-
performance relationship: commitment, feedback, task-complexity, and situational constraints (Locke &
Latham, 2006). People are only committed to a difficult goal when it is of personal importance and they
believe in their own ability to achieve said goal (i.e., self-efficacy). Feedback enables people to neatly
track their progress toward goal attainment. As tasks advance in complexity, the effectiveness of a goal is
dependent on the effectiveness of one’s performance strategies given the complexity of a task. Goals can
drive performance when there are appropriate amounts of time and resources to achieve those goals.
Leaderboards, progress bars, and badges are elements of games that align well with goal-setting
theory (Antin & Churchill, 2011; Hsu, Chang & Lee, 2013). More specifically, leaderboards have been

shown to elicit motivation to regulate behavior similarly to goal-setting; when presented with a
leaderboard containing scores corresponding to impossible, difficult, moderate, and easy goals,
participants tended to perform at the level of a difficult goal (Landers, Bauer, & Callan, 2017), as would
be expected in a traditional goal-setting intervention. Progress bars serve as useful feedback that regularly
tracks performance outcomes (Hsu, Chang & Lee, 2013). As for badges, they are identifiable and
quantifiable signs of accomplishment that vary is task-complexity and personal importance. Badges are
specifically awarded to an individual who has performed and completed an explicit task or goal.
To utilize goal-setting theory in gamification, game elements should be accompanied with, or
serve as a function of, the four key moderators of the goal-performance relationship, which should
maximize goal attainment. Using this approach, Singer and Schneider (2012) gamified a computer
science course to elicit best practice behavior when developing software. They implemented
“milestones”, which were given to students if they achieved a certain number of goals on software
development projects. Earning milestones became increasingly difficult over time as project tasks became
more complex, which encouraged more effort throughout the course. Weekly reports were also given to
students, enabling them to gauge their progress. In quantitative analyses, the researchers revealed patterns
as predicted by goal-setting theory: the increasing difficulty of earning milestones, accompanied with
weekly feedback, proved to be motivating, regardless of a student’s valence toward the way in which he
or she was being motivated. Hamari and Koivisto (2013) conducted an exploratory field experiment to
determine whether the use of badges affected user activity of an online trading service. They found that
badges themselves did not automatically yield significant increases in activity from users. However, users
who were committed to the personal goal of badge attainment, and who regularly monitored badge count
as a form of feedback, showed increased activity in the trading service.
Self-Determination Theory
Self-Determination Theory (SDT; Deci & Ryan, 2000b) states that humans regulate their
behavior depending on their intrinsic and extrinsic motivations to fulfill three basic psychological needs:
competence, autonomy, and relatedness. On its most basic level, SDT differentiates between intrinsic

motivation and extrinsic motivation to explain the personal will to act (Gagne & Deci, 2005) and
contrasts these from amotivation, defined as a lack of intention and self-determination (Ryan & Deci,
2000b). Intrinsic motivation is defined as the drive to engage in an activity for its enjoyment and inherent
satisfaction, and according to previous work on self-determination, situational factors that facilitate or
undermine self-motivated autonomy and competence can have a lasting impact intrinsic motivation (Deci
& Ryan, 1985). Extrinsic motivation is defined as the performance of an action in order “to attain some
separable outcome” (Ryan & Deci, 2000a) that does not prioritize the enjoyment of performing an action
itself. The degree to which someone internalizes regulatory processes, and integrates those processes into
his or her sense of self, distinguishes regulatory styles of motivation that varies in level of autonomy. For
example, relevant regulatory processes of extrinsic motivation include external rewards and punishments,
self-control, or maintenance of identity and values.
There have been several efforts to establish Ryan and Deci’s work on SDT as a theoretical
foundation for gamification. There has been increasing evidence that game elements can be used to
encourage enjoyment and increase intrinsic motivation by satisfying basic psychological needs of
autonomy, competence, and relatedness (Ryan, Rigby, & Przybylski, 2006; Sheldon & Filak, 2008).
Providing learners with positive feedback on their performance and consistent, achievable challenges
satisfies a need for competence (Ryan, Rigby, & Przybylski, 2006). This could include increasing level
difficulty and providing positive feedback in the form of badges. Giving learners opportunity for self-
direction and acknowledging their feelings satisfies a need for autonomy, which could include choosing
avatar features and different storylines. Team play and shared social attitudes toward gamified systems
satisfies the need for relatedness (Ryan, Rigby, & Przybylski, 2006; Hamari & Koivisto, 2013). An
important effect of enjoyment is that it promotes a higher quality of learning and creativity (Ryan & Deci,
2000b), implying that training can potentially be more enjoyable when game elements are designed
effectively. The integration of gamification and SDT has been a burgeoning line of research, but few
studies have determined exactly which gaming elements are most appetitive to our basic psychological
needs (Seaborn & Fels, 2015).

Overall, researchers have been attempting to devise ways to appropriately operationalize SDT
within a gamified system. A group of researchers have validated the conceptualization of enjoyment
under SDT by presenting evidence that specific antecedents accounted for 51% of the variance in
enjoyment during a gaming task (Tamborini, Bowman, Eden, Grizzard, & Organ, 2010). They found that
perceived game skill predicted autonomy, intuitive mapping of the controller predicted autonomy and
competence, and co-playing predicting relatedness. More recently, Mekler, Brühlmann, Tuch, and Opwis
(2017) examined the nature of intrinsic and extrinsic motivations within gamified systems. Mekler et al.
showed that specific game elements (i.e., points, levels, and leaderboards) increased overall efforts to
perform an image annotation task. Despite this evidence, they could not conclude there were changes in
intrinsic motivation. However, the gamified task itself may have not been intrinsically motivating in
general, and thereby preventing researchers from accurately capturing intrinsic motivation under a
gamified system. That is, the image annotation task employed might have not been inherently satisfying
enough to motivate participants intrinsically, regardless of whether the task was gamified.
Practical Recommendations for Gamifying Training and Development
Because gamification is inspired by the psychology of games and the psychology of games is
inspired by psychology itself, it can be difficult to identify which aspects of the literature are most useful
when creating gamified training. At first glance, it may appear that basic psychological theories alone are
sufficient; however, the serious game design literature and growing gamified learning literatures provide a
wealth of information regarding how these theories play out in authentic learning contexts. In the
remainder of this chapter, we will summarize the current lessons of these literatures for those seeking to
gamify their own training programs.
Implementation into the Training Design Process
Critically, gamification of training should not be attempted unless there is a specific, identifiable
problem with a training as it currently exists. It is assumed that the instructional designer has already
conducted a training needs assessment to identify performance gaps, developed and implemented training
intended to close those gaps, and conducted a training evaluation study. Gamification is most effective

when used to enhance training when the results of a training evaluation suggest specific motivational or
affective deficits. Although a growing research literature has demonstrated the merits of games and
gamification in learning (Bedwell et al., 2012), this pales in comparison to the vast pre-existing training
literature. When unsure of how to improve a training design, it is recommended to thoroughly explore the
recommendations already firmed established in the training literature before attempting to use novel
approaches like gamification. The empirical evidence supporting gamified learning is still sparse, and the
recommendations within the literature may not be complete or well-validated. It is possible that learning
may be improved by using simple techniques rather than attempting to develop a complex game-like
learning context. For example, in addition to post-training learning measures, practice tests may be used
throughout the training to enhance recall (Roediger & Karpicke, 2006). This addition is easily
implemented and supported by the literature (Rowland, 2014). In general, if your training is not “broken”,
do not try to “fix” it with gamification. Gamifying an already effective training could possibly worsen
outcomes, as not all gamification leads to improved learning (Armstrong & Landers, 2017).
Gamification of training is a process in which training content and methods are modified using
game elements. This process is incremental in nature, whereby the instructional designer modifies pieces
of the training bit by bit to improve learning outcomes. The instructional designer may choose to
incrementally modify the training content with game elements or the training method. For example, the
training content could be modified by adding elements of narrative, or game fiction. Armstrong and
Landers (2017) modified company laptop security training content by incorporating a narrative
component to relay the training material. Instead of presenting training content via slideshow or webpage,
the content was woven into a storyline in which trainees learned the content in the process of reading a
story. This gamified training was incremental in that the learning material was only slightly changed so
that the material could be expressed as a story. The training method may also be enhanced with game
elements. For example, a self-paced online training method may be gamified by the addition of progress
bars or badges for completing different modules. Again, this change is incremental over the original
training method. Trainees learn by reading or viewing material, but the process is gamified to provide

feedback to the trainee on their progress. It is possible that with the addition of enough game elements,
the training design might eventually become a game. As more game elements are added to the training,
such as images, sounds, stories, challenges, feedback, and social aspects, the training certainly becomes
more game-like. However, this is not necessarily the end goal of the gamification of training. It should
only be a by-product of modifying existing training to improve specific outcomes.
To gamify an underperforming training program, the instructional designer must first understand
what needs to be improved about the existing training process. This necessitates a training evaluation
study, where outcomes like reactions, learning, behavioral transfer, organizational-level performance
outcomes, or return-on-investment are measured. Once it is known which outcome is lacking, the
instructional designer can work backwards to determine what changes to make to the training. For
example, if reactions to the training are poor, designers should try to hone in on which type of reaction is
lacking. If trainees dislike the training (i.e., poor affective reactions), it suggests a different remedy than if
they find the training to be useless (i.e., poor utility reactions; Alliger, Tannenbaum, Bennett, Traver, &
Shotland, 1997). If trainees are not demonstrating learning, designers should try to hone in on the type of
learning that the training targets. A deficit in demonstrated declarative knowledge of facts would suggest
a different remedy is needed compared to a deficit in procedural knowledge or skill. If trainees’
motivations or attitudes are not changed by the training as intended, (i.e., affective learning; Kraiger,
Ford, & Salas, 1993), this would suggest another different remedy. Once the specific outcome
discrepancy has been identified, psychological theory and training theory can be used to continue to
identify the root of the problem. Once the root of the problem has been identified, it is time for the
instructional designer to consider whether game elements can be used to solve that problem. For example,
if trainees are not demonstrating declarative knowledge learning, the designer might consider what
mediators may be at play influencing learning. Colquitt, LePine, and Noe (2000) found meta-analytic
evidence suggesting that motivation to learn is a key mediator in the training-learning relationship. Game
elements intended to improve motivation to learn would be most appropriate in this instance (e.g., game
fiction, conflict/challenge, rules/goals, control; Bedwell et al., 2012). To give another example, if

behavioral transfer is not occurring post-training, the instructional designer may need to consider what
moderators may be influencing the effects of the training design on the outcome. Blume, Ford, Baldwin,
and Huang (2010) found meta-analytic evidence suggesting that transfer of skill-based training may
depend on the type of skill being trained (i.e., open skills like leadership vs. closed skills like computer
programming). In this case, game elements would not necessarily be helpful in improving transfer
because the training material itself may impact the likelihood of it being transferred to the workplace.
Targeting Psychological Mediators with Game Elements
To target psychological mediators with game elements, the instructional designer must conduct a
needs analysis (Surface, 2012) for gamification. This needs analysis starts by identifying the
psychological characteristic that is problematic. To do this, the instructional designer may use surveys or
focus groups to collect data about the training, which can be part of the training evaluation process
mentioned previously. The data collection effort should try to investigate the typical culprits of ineffective
training, while also including open-ended components to gather more contextual information and catch
any unexpected findings. The typical problem constructs in a training design are the learning outcomes
and mediators between training and those outcomes. Thus, in conducting a training evaluation, it would
be prudent to collect data on learning retention, learning application (i.e., transfer), and reactions or
satisfaction with the training. In addition, measures of motivation to learn or attitudes regarding the
training content or method are good to include, as motivation and attitudes likely impact the success of
most trainings. Asking open-ended questions to trainees about what they liked or did not like, what they
found to be effective, what they learned, or what they believed the purpose of the training to be can help
round out a full picture of the effectiveness of the training in the event the problem construct is out of the
ordinary.
After the data are collected, the instructional designer should try to identify which criterion
construct is most likely the problem. Each criterion measured should have an ideal point or level at which
the instructional designer or organization desires trainees to be. This desired point or level could be a
percent of answers correct on a training test (e.g., 80% correct to pass a test), familiarity with a given

training topic where the maximum possible level is desirable (e.g., 5.0 out of 5.0 on familiarity with
Microsoft Office products), or a motivational or attitudinal construct where the maximum possible level is
desirable (e.g., 5.0 out of 5.0 on a motivation to learn scale). To determine problematic criterion
constructs, criterion scores should be measured and subtracted from desired levels to assess training gaps.
These training gaps can then be prioritized based on a variety of criteria and resolved accordingly
(Watkins, Meiers, & Visser, 2012). Assuming all gaps in training criteria are equally important, the
largest gap may take priority and gamification may be implemented to improve that gap. However,
training outcomes are not always equally important. In these situations, the gaps that result in the largest
costs to the organization or the greatest consequences if left unaddressed should take precedence, even if
the gap itself is not the largest among all constructs measured. Once gaps have been prioritized, each gap
can be investigated in turn, exploring the root causes of the gap. For a given training gap, the root cause
may be any number of issues: the psychological measure may be unreliable or invalid, the training
content may be confusing, the trainees may be lacking attention or motivation, etc. Root cause analysis
may require further surveys, interviews, focus groups, or data analysis, but will lead the instructional
designer to the exact cause of the gap, yielding a psychological construct to target for improvement. Once
a criterion construct has been targeted for improvement, the instructional designer should select game
elements that are theoretically tied to that construct. The theory of gamified learning game attribute
taxonomy (Landers, 2014; see Table 1) provides a list of possible game element categories to consider
adding to a training context to improve outcomes, and ties between these elements and target constructs
vary in both quality of evidence and strength of tie. This taxonomy provides a good start for reasoning
through a solution to the problematic training construct. Next, we provide several examples of game
elements that are theoretically or logically tied to training-related constructs to demonstrate the process of
gamifying training. First, if post-training knowledge retention is deficient, the instructional designer may
conclude that trainees need to spend more time reviewing and practicing the material during the training
session. According to the testing effect (Roediger & Karpicke, 2006), practicing recall is more effective
than large amounts of studying at improving knowledge retention. In order to promote practicing recall,

the instructional designer could integrate a practice quiz into the training by creating a review game (e.g.,
Jeopardy! or any other popular game show format). Adding game elements like rules/goals, feedback via
points or a score, and social aspects like teamwork can incrementally improve on the practice quiz
training method to provide a more memorable and enjoyable experience during the training process. To
give a second example, trainees’ may be inattentive during a lecture-style training. To keep trainees
attentive, the instructional designer could create a task to keep trainees focused on the presenter and
materials. Different images and symbols or specific topics and keywords may be included within a lecture
slideshow and trainees could be tasked with finding these images or keywords and marking them on a
bingo card. This game-like task may not improve learning per se, but the activity will keep trainees
engaged with the presentation and paying attention to the slideshow to complete the task. For a third
example, perhaps training motivation is the root cause of the problem. Adding media to the training may
make the training more fun and enjoyable, improving trainee motivation. Using humorous but relevant
images, videos, activities, and discussions can improve the intrinsic motivation to participate in the
training session. This example may not appear to be as game-like as the previous examples, but this
approach would still be considered gamification, as media, humor, and other activities are still pieces of
games that make them enjoyable.
Once game elements have been implemented into training, data must be collected once again to
evaluate the new training design. If a problem persists, the instructional designer will need to repeat the
entire process. As with any training, a needs assessment or gap analysis must be conducted, training must
be designed with the intention to close prioritized gaps, the training must again be implemented, and
evaluation data must be collected. As the training development cycle continues, iterations can be made
based on what is working or failing to meet the goals of the training. For example, perhaps motivation to
learn is initially identified as the issue and a point and leaderboard system is implemented to improve
motivation via competition. If the evaluation data do not indicate improved motivation after implementing
the gamified training, perhaps a different game element should be used which better align with the
theoretical target construct. Alternatively, if motivation improves, but another learning outcome

decreases, the instructional designer may need to conduct a more thorough needs assessment as another
construct may be at play in affecting learning besides motivation. Once the problem has been finally
resolved, the scientist-practitioner should publish his or her work so that other instructional designers do
not make the same mistakes. In this manner, a scientific literature can be built, which is especially
important for the nascent field of gamified learning.
Conclusions
Although gamification is sometimes presented as a “new” approach to training, this chapter
describes how the techniques used in gamification often involve existing training design techniques.
Instead, the “new” aspects of gamification are 1) the systematic, science-based targeting of meaningful
outcomes using this game element toolkit and 2) the acceptance of both psychological and games research
as valuable sources of information regarding how to make learning more engaging and compelling.
Specifically, psychological theory serves as an effective and sizable foundation on which to build
gamification interventions, and games research provides a record of how psychological theory has already
been implemented to create fun and compelling experiences. This knowledge is what is harnessed in
gamification. To enable this for the reader, we provide specific guidance on how to translate this advice
into training design practice. With this chapter, we hope that training designers will be able to gamify
their training content in a cost-effective and impactful way to better meet their training goals.

References
Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of
the relations among training criteria. Personnel Psychology, 50, 341-358. doi: 10.1111/j.1744-
6570.1997.tb00911.x
Antin, J., & Churchill, E. F. (2011). Badges in social media: A social psychological perspective. In CHI
2011 Gamification Workshop Proceedings (pp. 1-4). New York, NY: ACM.
Armstrong, M. B., & Landers, R. N. (2017). An evaluation of gamified training: Using narrative to
improve reactions and learning. Simulation & Gaming, 48, 513-538.
doi:10.1177/1046878117703749
Arthur Jr, W., Bennett Jr, W., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in
organizations: a meta-analysis of design and evaluation features. Journal of Applied Psychology,
88, 234-245. doi: 10.1037/0021-9010.88.2.234
Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social
psychological research: Conceptual, strategic, and statistical considerations. Journal of
Personalitypersonality and Social Psychologysocial psychology, 51, 1173.
Bedwell, W. L., Pavlas, D., Heyne, K., Lazzara, E. H., & Salas, E. (2012). Toward a taxonomy linking
game attributes to learning: An empirical study. Simulation & Gaming, 43, 729-760.
doi:10.1177/1046878112439444
Blohm, I., & Leimeister, J. M. (2013). Gamification. Business & Information Systems Engineering, 5,
275-278. doi: 10.1007/s12599-013-0273-5
Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of training: A meta-analytic
review. Journal of Management, 36, 1065-1105. doi: 10.1177/0149206309352880
Bogost, I. (2011). Gamification is bullshit. In S. P. Waltz & S. Deterding (Eds.), The Gameful World (pp.
65-79). Cambridge, MA: The MIT Press.
Brown, K. G. (2001). Using computers to deliver training: Which employees learn and why? Personnel
Psychology, 54, 271–296. doi:10.1111/j.1744-6570.2001.tb00093.x

Browne, K., Anand, C., & Gosse, E. (2014). Gamification and serious game approaches for adult literacy
tablet software. Entertainment Computing, 5(3), 135-146.
Burke, L. A., & Hutchins, H. M. (2007). Training transfer: An integrative literature review. Human
resource development review, 6, 263-296. doi: 10.1177/1534484307303035
Canada. doi: 10.1145/2181037.2181040
Carini, R. M., Kuh, G. D., & Klein, S. P. (2006). Student engagement and student learning: Testing the
linkages. Research in higher education, 47, 1-32. doi: 10.1007/s11162-005-8150-9
Christy, K. R., & Fox, J. (2014). Leaderboards in a virtual classroom: A test of stereotype threat and
social comparison explanations for women's math performance. Computers & Education, 78, 66-
77. doi: https://doi.org/10.1016/j.compedu.2014.05.005
Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A
meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85,(5), 678-
707. doi: 10.1037//0021-9010.g5.5.678
Darden, W. R., Hampton, R., & Howell, R. D. (1989). Career Versus Organizational Commitment:
Antecedents And Con. Journal of Retailing, 65, 80.
Deci, E. L. and Ryan, R. M. (2010). Self-Determination. Corsini Encyclopedia of Psychology. 1–2. doi:
10.1002/9780470479216.corpsy0834
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-(2010). Self
‐
determination in human
behavior. New York: Plenum. John Wiley & Sons, Inc..
Deterding, S., Sicart, MKhaled, R., Nacke, L., O’Hara, K. E., & Dixon, D. (2011). Gamification: Toward
a definition. Proceedings of thefrom CHI 2011 Gamification Workshop, Vancouver, BC,.
Ferster, C. B. Skinner, B. F. (1957). Schedules of reinforcement, (pp. 44-137). East Norwalk, CT, US:
Appleton-Century-Crofts, vii, 744. doi: http://dx.doi.org/10.1037/10627-004
Ford, J. K., Quiñones, M. A., Sego, D. J., & Sorra, J. S. (1992). Factors affecting the opportunity to
perform trained tasks on the job. Personnel psychology, 45, 511-527.
Foxon, M. (1997). The influence of motivation to transfer, action planning, and manager support on the

transfer process. Performance Improvement Quarterly, 10, 42-63.
Gagné, M., & Deci, E. L. (2005). Self‐determination theory and work motivation. Journal of
Organizational Behavior, 26, 331-362. doi: 10.1002/job.322
Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice
model. Simulation & gaming, 33,(4), 441-467. doi: https://doi.org/10.1177/1046878102238607
Gartner. (2012). Gartner Says by 2014, 80 Percent of Current Gamified Applications Will Fail to Meet
Business Objectives Primarily Due to Poor Design [Press Release]. Retrieved from
http://www.gartner.com/newsroom/id/2251015
Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of
Educational Psychology, 99, 445-476. doi: http://dx.doi.org/10.1037/0022-0663.99.3.445
Greenberg, B. S., Sherry, J., Lachlan, K., Lucas, K., & Holmstrom, A. (2010). Orientations to video
games among gender and age groups. Simulation & Gaming, 41, 238-259. doi:
10.1177/1046878108319930
Hamari, J., & Koivisto, J. (2013, June). Social Motivations To Use Gamification: An Empirical Study Of
Gamifying Exercise. In ECIS 2013 Completed Research. http://aisel.aisnet.org/ecis2013_cr/105
Hamari, J., Koivisto, J., & Sarsa, H. (2014, January). Does gamification work? -- A?--a literature review
of empirical studies on gamification. In System Sciences (HICSS), 2014 47th Hawaii International
Conference on System Sciences (pp. 3025-3034). IEEE.
Heeter, C., Lee, Y. H., Magerko, B., & Medler, B. (2011). Impacts of forced serious game play on
vulnerable subgroups. International Journal of Gaming and Computer-Mediated Simulations
(IJGCMS), 3, 34-53. doi:10.4018/jgcms.2011070103
Hsu, S. H., Chang, J. W., & Lee, C. C. (2013). Designing attractive gamification features for collaborative
storytelling websites. Cyberpsychology, Behavior, and Social Networking, 16, 428-435.
https://doi.org/10.1089/cyber.2012.0492
Kanfer, R. (1991). Motivation theory and industrial and organizational psychology. In M. D. Dunnette &
L. M. Hough (Eds.), Handbook of industrial and organizational psychology, (Vol. 1, pp. 75-

170.). Palo Alto, CA: Consulting Psychologists Press.
Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-
treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74(4), 657-
690. http://dx.doi.org/10.1037/0021-9010.74.4.657
Kapp, K. M. (2012). The gamification of learning and instruction: game-based methods and strategies for
training and education. John Wiley & Sons.
Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society
of Training Directors, 13, 3–9.
Koivisto, J., & Hamari, J. (2014). Demographic differences in perceived benefits from
gamification. Computers in Human Behavior, 35, 179-188. doi:
https://doi.org/10.1016/j.chb.2014.03.007
Kraiger, K. (2002). Creating, implementing, and managing effective training and development: State-of-
the-art lessons for practice. San Francisco: Jossey-Bass.
Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of
learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78,
311-328. http://dx.doi.org/10.1037/0021-9010.78.2.311
Kulik, J. A., Kulik, C. L. C., & Cohen, P. A. (1980). Effectiveness of computer-based college teaching: A
meta-analysis of findings. Review of educational research, 50 525-544.
Landers, R. N. (2015). Developing a theory of gamified learning: Linking serious games and gamification
of learning. Simulation & Gaming, 45, 752-768. doi: 10.1177/1046878114563660(6), 752-768.
Landers, R. N., Auer, E. M., Collmus, A. B. & Armstrong, M. B. (in press). Gamification science, its
history and future: Definitions and a research agenda. Simulation & Gaming.
Landers, R. N., & Armstrong, M. B. (2015). Enhancing instructional outcomes with gamification: An
empirical test of the Technology-Enhanced Training Effectiveness Model. Computers in Human
Behavior, 71, 499-507. https://doi.org/10.1016/j.chb.2015.07.031
Landers, R. N., & Callan, R. C. (2011). Casual social games as serious games: The psychology of

gamification in undergraduate education and employee training. In M. Ma, A. Oikonomou, &
L. C. Jain (Eds.), Serious games and edutainment applications (pp. 399–423). New York:
Springer. doi: 10.1007/978-1-4471-2161-9_20
Landers, R. N., & Callan, R. C. (2012). Training evaluation in virtual worlds: Development of a model.
Journal of Virtual Worlds Research, 5, 1-20. doi:10.4101/jvwr.v5i3.6335
Landers, R. N., & Goldberg, A. S. (2014). Online social media in the workplace: A conversation with
employees. In M. D. Coovert & L. F. Thompson (Eds.), Psychology of Workplace Technology.
New York: Routledge Academic. Psychology of workplace technology, 284-306.
Landers, R. N., Armstrong, M. B., & Collmus, A. B. (2017). How to use game elements to enhance
learning: Applications of the theory of gamified learning. In M. Ma, A. Oikonomou, & L. C. Jain
(Eds.), Serious Games and Edutainment Applications (Vol. 2). Surrey, UK: Springer. doi:
10.1007/978-3-319-51645-5_21
Landers, R. N., Bauer, K. N., & Callan, R. C. (2017). Gamification of task performance with
leaderboards: A goal setting experiment. Computers in Human Behavior, 71, 508-515.
https://doi.org/10.1016/j.chb.2015.08.008
Landers, R. N., Bauer, K. N., Callan, R. C., & Armstrong, M. B. (2015). Psychological theory and the
gamification of learning. In T. Reiners & L. Wood (Eds.), Gamification in Education and
Business (pp. 165-186). Cham, Switzerland: Springer.
Latham, G. P., & Locke, E. A. (1990). A theory of goal setting and task performance. Eaglewood Cliffs,
NJ: Prentice Hall.
Latham, G. P., & Locke, E. A. (1991). Self-regulation through goal setting. Organizational behavior and
human decision processes, 50, 212-247. https://doi.org/10.1016/0749-5978(91)90021-K
Lawler, E. E., & Suttle, J. L. (1973). Expectancy theory and job behavior. Organizational behavior and
human performance, 9, 482-503. https://doi.org/10.1016/0030-5073(73)90066-4
Lim, D. H. (2001). The effect of work experience and job position on international learning transfer.
International Journal of Vocational Education and Training, 9, 59–74.

Linehan, C., Kirman, B. and Roche, B. (2015) 'Gamification as behavioral psychology' in Walz, S.P. and
Deterding, S. (eds.) The Gameful World: Approaches, Issues, Applications. Cambridge, MA,
USA : MIT Press, pp. 81-105.
Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task
motivation: A 35-year odyssey. American Psychologist, 57, 705-717.
http://dx.doi.org/10.1037/0003-066X.57.9.705
Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current directions in
psychological science, 15, 265-268. doi: 10.1111/j.1467-8721.2006.00449.x
Malone, T. W. (1981). Towards a theory of intrinsically motivation instruction. Cognitive Science, 4, 333-
369. doi:10.1016/S0364-0213(81)80017-1
McNamara, D. S., Jackson, G. T., & Graesser, A. (2010). Intelligent tutoring and games (TaG). In Y.
Baek (Ed.) Gaming for classroom-based learning: Digital role playing as a motivator of study
(pp. 44–65). Hershey, PA: IGI Global.
Mekler, E. D., Brühlmann, F., Tuch, A. N., & Opwis, K. (2017). Towards understanding the
effects of individual gamification elements on intrinsic motivation and performance. Computers
in Human Behavior, 71, 525-534.
Michael, D., & Chen, S. (2005). Serious games: Games that educate, train, and inform. Boston:
Thomson Course Technology.
Mitchell, T. R. (1982). Motivation: New Direction for Theory, Research, and Practice. Academy of
Management Review, 81. doi: 10.5465/AMR.1982.4285467
Mollick, E. R., & Rothbard, N. (2014). Mandatory fun: Consent, gamification and the impact of games at
work. The Wharton School Research Paper Series. Retrieved from:
http://ssrn.com/abstract=2277103
Noe, R. A., & Schmitt, N. (1986). The influence of trainee attitudes on training effectiveness: Test of a
model. Personnel psychology, 39, 497-523. doi: 10.1111/j.1744-6570.1986.tb00950.x
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative

meta‐analysis. Language learning, 50, 417-528. doi:10.1111/0023-8333.00136
Ostroff, C., Kinicki, A. J., & Muhammad, R. S. (2013). Organizational culture and climate. In I.B.
Weiner (Ed.), Handbook of Psychology (2nd ed., pp. 643-676). Hoboken, N.J.: John Wiley &
Sons, Inc. doi: 10.1002/0471264385.wei1222
Paas, F., Tuovinen, J. E., Van Merrienboer, J. J., & Darabi, A. A. (2005). A motivational perspective on
the relation between mental effort and performance: Optimizing learner involvement in
instruction. Educational Technology Research and Development, 53, 25-34.
Parijat, P., & Bagga, S. (2014). Victor Vroom’s expectancy theory of motivation–An evaluation.
International Research Journal of Business and Management (IRJBM), 7, 1-8.
Parker, L. E., & Lepper, M. R. (1992). Effects of fantasy contexts on children’s learning and motivation:
Making learning more fun. Journal of Personality and Social Psychology, 62, 625-633.
doi:10.1037/0022-3514.62.4.625
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of
classroom academic performance. Journal of Educational Psychology, 82(1), 33.
Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of
the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological
Measurement, 53, 801-813.
Richter, G., Raban, D. R., & Rafaeli, S. (2015). Studying gamification: the effect of rewards and
incentives on motivation. In T. Reiners, L.C. Wood (Eds.), Gamification in Education and
Business. Springer International Publishing. doi: 10.1007/978-3-319-10208-5_2
Roediger, H. L., III, & Karpicke, J.D. (2006). Test-enhanced learning: Taking memory tests improves
long term retention. Psychological Science, 17, 249–255. doi:10.1111/j.1467-9280.2006.01693.x
Rowland, C. A. (2014). The effect of testing versus restudy on retention: A meta-analytic review of the
testing effect. Psychological Bulletin, 140, 1432-1463. doi:10.1037/a0037559
Ryan, R. M., & Deci, E. L. (2000a). Intrinsic and extrinsic motivations: Classic definitions and new
directions. Contemporary educational psychology, 25, 54-67.

https://doi.org/10.1006/ceps.1999.1020
Ryan, R. M., & Deci, E. L. (2000b). Self-determination theory and the facilitation of intrinsic motivation,
social development, and well-being. American psychologist, 55, 68.
http://dx.doi.org/10.1037/0003-066X.55.1.68
Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The motivational pull of video games: A self-
determination theory approach. Motivation and emotion, 30, 344-360. doi: 10.1007/s11031-006-
9051-8
Sailer, M., Hense, J., Mandl, H., & Klevers, M. (2013). Psychological Perspectives on Motivation
through Gamification. IxD&A, 19, 28-37.
Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal of
Human-Computer Studies, 74, 14-31. https://doi.org/10.1016/j.ijhcs.2014.09.006
Seidel, T., & Shavelson, R. J. (2007). Teaching effectiveness research in the past decade: The role of
theory and research design in disentangling meta-analysis results. Review of educational
research, 77, 454-499. doi: 10.3102/0034654307310317
Sheldon, K. M., & Filak, V. (2008). Manipulating autonomy, competence, and relatedness support in a
game‐learning context: New evidence that all three needs matter. British Journal of Social
Psychology, 47, 267-283. doi:10.1348/014466607X238797
Shen, W. C. M., Liu, D., Santhanam, R., & Evans, D. A. (2016, June). Gamified Technology-Mediated
Learning: the Role of Individual differences. In PACIS 2013 Proceedings (p. 47).
Singer, L., & Schneider, K. (2012, June). It was a bit of a race: Gamification of version control. In Games
and Software Engineering (GAS), 2012 2nd International Workshop on (pp. 58). IEEE. doi:
10.1109/GAS.2012.6225927
Sitzmann, T. (2011). A meta‐analytic examination of the instructional effectiveness of computer‐based
simulation games. Personnel psychology, 64, 489-528. doi: 10.1111/j.1744-6570.2011.01190.x
Skinner, B. F. (1953). Science and human behavior. Simon and Schuster.
Staddon, J. E., & Cerutti, D. T. (2003). Operant conditioning. Annual review of psychology, 54, 115-144.

https://doi.org/10.1146/annurev.psych.54.101601.145124
Stansbury, J. A., & Earnest, D. R. (2017). Meaningful Gamification in an Industrial/Organizational
Psychology Course. Teaching of Psychology, 44, 38-45.
https://doi.org/10.1177/0098628316677645
Surface, E. A. (2012). Training needs assessment: Aligning learning and capability with performance
requirements and organizational objectives. In M. A. Wilson, W. Bennett, Jr., S. G. Gibson, & G.
M. Alliger (Eds.), Series in applied psychology. The handbook of work analysis: Methods,
systems, applications and science of work measurement in organizations (pp. 437-462). New
York, NY: Routledge/Taylor & Francis Group.
Tamborini, R., Bowman, N. D., Eden, A., Grizzard, M. and Organ, A. (2010), Defining Media Enjoyment
as the Satisfaction of Intrinsic Needs. Journal of Communication, 60: 758–777.
doi:10.1111/j.1460-2466.2010.01513.x
Tay, L. (2010). Employers: Look to gaming to motivate staff. itnews for Australian Business. Retrieved
from http://www.itnews.com.au/News/169862,employers-look-to-gaming-to-motivate-staff.aspx
Thiel, S. K., Reisinger, M., & Röderer, K. (2016, December). I'm too old for this!: influence of age on
perception of gamified public participation. In Proceedings of the 15th International Conference
on Mobile and Ubiquitous Multimedia (pp. 343-346). ACM. doi: 10.1145/3012709.3016073
Vancouver, J. B. (2008). Integrating self-regulation theories of work motivation into a dynamic process
theory. Human Resource Management Review, 18, 1-18.
https://doi.org/10.1016/j.hrmr.2008.02.001
Vancouver, J. B., & Day, D. V. (2005). Industrial and organisation research on self‐regulation: from
constructs to applications. Applied Psychology, 54, 155-185. doi: 10.1111/j.1464-
0597.2005.00202.x
Von Ahn, L., & Dabbish, L. (2008). Designing games with a purpose. Communications of the ACM, 51.
58-67. doi: 10.1145/1378704.1378719
Vroom, V. H. (1964). Work and motivation. New York: John Wiley and Sons, Inc.

Warr, P., & Bunce, D. (1995). Trainee characteristics and the outcomes of open learning. Personnel
psychology, 48, 347-375. doi: 10.1111/j.1744-6570.1995.tb01761.x
Watkins, R., Meiers, M. W., & Visser, Y. (2012). A guide to assessing needs: Essential tools for
collecting information, making decisions, and achieving development results. Washington, DC:
The World Bank.
Wilson, K. A., Bedwell, W. L., Lazzara, E. H., Salas, E., Burke, C. S., Estock, J. L., … Conkey, C.
(2009). Relationships between game attributes and learning outcomes. Simulation & Gaming, 40,
217-266. doi:10.1177/1046878108321866
Zeiler, M. D. (1968). Fixed and Variable Schedules of Response-Independent Reinforcement. Journal of
the Experimental Analysis of Behavior, 11, 405-414. doi:10.1901/jeab.1968.11-405
Zhao, C. M., & Kuh, G. D. (2004). Adding value: Learning communities and student
engagement. Research in higher education, 45, 115.