Technical ReportPDF Available

Learning Mechanics and Assessment Mechanics for Games for Learning

Authors:

Abstract and Figures

In this paper, we will discuss the approach for assessment of learning and related learner variables taken by the Games for Learning Institute (G4LI). We will first describe game mechanics in general, and then introduce the concepts of learning mechanics and assessment mechanics and describe criteria for their design and requirements of how they can inform the design of related game mechanics.
Content may be subject to copyright.
Learning Mechanics and Assessment Mechanics
for Games for Learning
Institute for Games for Learning
New York University
The Graduate Center of the City University of New York
Teachers College Columbia University
White Paper # 01/2011
Version 0.1 September 30, 20011
Jan L. Plass, NYU
Bruce D. Homer, CUNY GC
Charles Kinzer, TC/Columbia
Jonathan M. Frye, NYU
Ken Perlin, NYU
!
! ! !
"!
Learning Mechanics and Assessment Mechanics
for Games for Learning
In this paper, we will discuss the approach for assessment of learning and related learner
variables taken by the Games for Learning Institute (G4LI). We will first describe game
mechanics in general, and then introduce the concepts of learning mechanics and
assessment mechanics and describe criteria for their design and requirements of how they
can inform the design of related game mechanics.
Game Mechanics
We begin with a discussion of the concept of game mechanics, a term that is central to an
understanding of games. Game studies scholars and developers have offered definitions for
this term that range from finely detailed to more broad and conceptual approaches. In general
terms, the mechanics of the game consist of what the game allows players to do – they are
the rule systems that players must abide by and therefore shape the interactions and the
behaviors of players. Game mechanics are “the various actions, behaviors and control
mechanisms afforded to the player within a game context” (Hunicke, LeBlanc, & Zubek, 2004,
p. 3). In a narrower sense, the core mechanic of a game is “the essential play activity players
perform again and again and again” (Salen & Zimmerman, 2003, p. 316). In other words, the
core mechanic of the game contains the moment-to-moment actions and interactions in
which the player is engaging while playing the game. A definition proposed by Cook (2006)
describes game mechanics as “rule based system/simulations that facilitate and encourage
a user to explore and learn the properties of their possibility space through the use of
feedback mechanisms” (p. 1). Game mechanics are therefore a means to guide the player into
particular behaviors by constraining the space of possible plans to attain goals (Järvinen,
2008). For instance, mechanics involve reward systems such as points or stars are feedback
mechanisms that shape behaviors and interactions that the player performs. The same is
!
! ! !
#!
true for failure mechanics that are a way for the designer to playfully communicate what
actions the player should and should perform.
This paper focuses on games for learning, yet it is clear that learning is a fundamental part
of all games (Gee, 2008). At a minimum, players must learn the basics of a game’s
mechanics in order to play. Additionally, players must uncover the goals and purpose of these
mechanics; and what actions the game designer was aiming to facilitate for the player (Cook,
2006). Feedback mechanisms are an example of how game designers encourage (reward) or
discourage (punish) a particular action taken by the player. Game mechanics for learning must
incorporate all of these aspects, from the moment-to-moment activities in which players
engage to meaningful incentive systems.
The mechanics of the game therefore not only define the behaviors and actions players
take, but directly facilitate players’ understanding of the game and what the game may be
representing or aiming to teach. A good example of this is what Bogost (2008) refers to as
procedural rhetoric;
“Following the contemporary model, procedural rhetoric entails expression—to convey
ideas effectively. Procedural rhetoric is a subdomain of procedural authorship; its arguments
are made not through the construction of words or images, but through the authorship of
rules of behavior, the construction of dynamic models.” (p. 125).
In other words, the rules or mechanics of the game can be representative of arguments
or models. Through the representations and rules of the game system, players can form
mental models or understandings of similar real world systems, concepts, or formulas.
Players/learners can gain deeper insights by not only learning the variables involved in a
system, but also by understanding how these variables interact with each other.
Learning Mechanics
When games are designed with the explicit goal of facilitating learning, game mechanics
must go beyond making a game fun and engaging–they must engage players in meaningful
!
! ! !
$!
learning activities. The game mechanic becomes an integral part of the learning activity. Game
designers have long seen this connection and have argued that new mechanics are needed in
order to engage the player in a way that facilitates learning (Isbister, Flanagan, & Hash,
2010). Most importantly, the designers interviewed by Isbister et al. made a strong case that
learning needs to be embedded in the core mechanics of a game rather than added on to
existing mechanics. Game play cannot be used as a reward for answering questions about
facts; and vice versa, factual quizzes cannot be forced into unrelated game play.
In order to emphasize this qualitative difference of mechanics that are designed for
learning, we offer the following definition of what we call
Learning Mechanics,
based on Salen
& Zimmermann’s (2004) definition of game mechanics:
Learning mechanics are patterns of behavior or building blocks of learner interactivity,
which may be a single action or a set of interrelated actions that form the essential learning
activity that is repeated throughout a game.
The relationship between game mechanics and learning mechanics is that learning
mechanics are design patterns or meta-mechanics that need to be instantiated in order to
become a game mechanic. We use the term
design pattern
in the tradition of Christopher
Alexander’s definition as general solutions to commonly occurring problems (Alexander,
1977). Learning mechanics adapt the moment-to-moment activity of a game mechanic into a
meaningful learning activity. The learning aspects of a game are integrated in a way that they
become an integral part of the game play and not merely an addendum to the game
mechanic. An example of a poor integration of learning into a game is when a learning game
uses an established game mechanic such as a racing mechanic or a shooter mechanic, and
the learning mechanic consists of a popup question that players are asked to answer before
they can resume the race or shooter activities.
!
! ! !
%!
Player choice can play a significant role in learning mechanics. After all, the relationship
between the player’s agency and the game’s rule systems are what make the game fun and
challenging. All games offer players series of choices and then react to those choices with
new challenges. Learning mechanics can push game mechanics further to offer player
choices that help the student learn as well as facilitate gameplay.
Let’s review an example of a learning mechanic to illustrate this concept. Figure 1 shows a
learning mechanic from a game we developed as part of our research at G4LI, entitled
Noobs
v. Leets
. The goal of the game is to teach middle school geometry, in particular, rules that
describe relations of angles in quadrilaterals, as included in the common core math standards
for grades 6–8. These rules include the complementary angle rule, supplementary angle rule,
opposite angle rule, as well as the sum of angles in a triangle rule.
Figure 1. Learning Mechanic: Apply Rules to solve Problems
The learning mechanic for this game is Apply Rules to solve Problems: Learner selects
among different rules and indicates for which problems they apply. This mechanic was
chosen to engage the learner in higher-level thinking: rather than solving angles and
responding with the numeric answer of the missing angle, we wanted the focus of the learning
activity to be on the conceptual level of the rules to be learned. This approach is grounded in
aspects of Schoenfeld’s (1985) approach to mathematical problem solving, Lave’s (1988)
situated learning approach, and schema theory (Anderson, 1977). During our work on
!
! ! !
&!
mechanics such as this one as well as others (see case examples below), we began compiling
a list of criteria for good learning mechanics.
Criteria for Learning Mechanics
Learning mechanics have to meet a series of criteria. Most importantly, learning
mechanics are grounded in Learning Sciences and Learning Theory. Over the past several
decades, many such theories and frameworks have been developed that can be used as the
basis for the design of learning mechanics. Examples of these are
Cognitive Flexibility Theory
(Spiro, Coulson, Feltovich, & Anderson, 1988; Spiro & Jehng, 1990),
Cognitive Apprenticeship
(Collins, 1988; Liu, 1998),
Anchored instruction
(CTGV, 1990, 1991, 1992, 1993), and
Situated Learning
(Lave, 1988; Lave & Wenger, 1990). From these and other, related
theories, designers choose activities that engage the learner in meaningful interaction with a
specific subject. These interactions should follow a theoretical model of interactivity such as
the INTERACT model, which distinguishes three types of interactivity: behavioral interactivity,
cognitive interactivity, and emotional interactivity, and describes their relation in learning
processes such as feedback and guidance (Domagk, Schwartz, & Plass, 2010). Learning
designers use these interactions to describe tools that allow learners to generate solutions to
the learning problems that are designed to facilitate learning. If the subject matter allows for
different but equally appropriate solutions to problems, the mechanic should enable the
learner to generate their own solutions.
Based on this selection of a learning mechanic, game designers have a choice among
different game mechanics to instantiate the learning mechanic in the game. In the case of our
learning mechanic for Noobs v. Leets described above, possible game mechanics included the
Fling
mechanic from Angry Birds, the
Drag
Mechanic from Implode!, and the
Bounce
mechanic from Doodle Jump. In order to preserve the original learning goal, however, there
are several requirements that a game mechanic has to meet in order to be an appropriate
implementation of a learning mechanic.
!
! ! !
'!
Requirements for Designing Game Mechanics based on a Learning Mechanic
Learning mechanics describe activities that have the expressed goal of facilitating learning
and are grounded in the learning sciences and learning theory. But learning mechanics are
design patterns of mechanics, or meta-mechanics, not playable mechanics. They describe the
functions of the tools available to players to solve problems, but they don’t describe the actual
tools themselves. For example, the learning mechanic might specify that the learner/player
should be able to apply rules to solve problems, but it does not describe whether this is done
by flinging objects representing the rules at the problems, dragging those objects, or putting a
jet pack on them to guide them to the problem.
Learning mechanics need to be instantiated as game mechanics to describe the concrete
tools and moves the players have in the system that the game represents. Playing the game is
learning these tools and moves, getting facile with them, and having the satisfaction of solving
challenges, of ‘beating’ the game (Juul, 2003). In addition, game designers need to add the
game feel, the feel of engaging the core mechanics through interactive elements, visual
elements, emotional elements and sound elements that are added to the game mechanic
(Swink, 2008).
However, in this creative process, game designers have to make sure that the learning
goal is preserved. To that end, when selecting a game mechanic to implement a particular
learning mechanic, designers need to consider the following requirements:
(1)
Game Mechanic must not introduce excessive amounts of extraneous cognitive load
.
The instantiation of a learning mechanic as a game mechanic, and the addition of playful
elements that this may involve, are by their very nature introducing demands on learners’
cognitive processing that are not directly related to the processing of the essential learning
content. Although traditional cognitive load researchers would argue that such extraneous
cognitive load should always be removed (Kirschner, Sweller, & Clark, 2006), the success of
many games seems to indicate that elements such as a narrative, the requirement of
!
! ! !
(!
resource management, or incentive systems, can have a positive impact on learning. This
suggests that the under some conditions motivational benefits of such features can outweigh
the negative impact of the additional cognitive processing they require. It is therefore
necessary to design game mechanics that do not introduce excessive amounts of extraneous
cognitive load that would turn this advantage into a disadvantage. For example, the game
Dimenxian X
requires learners to perform a series of task that are only peripherally related to
learning goals, such as in the scene depicted in Figure 2, where the player has to retrieve data
packets from an underwater cavern.
Figure 2. Extraneous Cognitive Load in
Dimenxian X
.
In cases such as this it needs to be determined whether these additional game tasks
enhance or suppress learning, which can often only be decided through empirical research.
(2)
Game Mechanic must not reduce the amount of the required mental effort by too
much
. Another requirement related to cognitive load is that game mechanics do not reduce
the task demands imposed on the learner too much, i.e., that the mechanic does not provide
the results of the processing of the information or problem solving to the learner but instead
requires the learner to introduce mental effort to generate a solution. Research has shown
that such reduction of germane cognitive load can have a negative impact on learning
(Schnotz, Böckler, & Grzondziel, 1999).
!
! ! !
)!
Figure 3. Low Germane Cognitive Load in the Algebra Game
AlgebraActor
For example, in the Algebra game depicted in Figure 3, the game mechanic is designed to
show the learner how the term (b) from the right side has to be added as (-b) on the left side
when resolving for x. This eliminates the need for the learner to decide where to place the
term (b) on the left side, and to specify that the sign of b has to change to (-), which
unnecessarily reduced the amount of useful cognitive load required to solve this problem.
Unless a later level removes this scaffolding, it is less likely that the learner will be able to solve
a similar question by themselves and in a different context, i.e., as transfer task.
(3)
Game Mechanic must not introduce unnecessary confounds.
Anytime a learning
mechanic is instantiated as a game mechanic, there is a possibility that additional knowledge
and skills are introduced that the learner has to master in order to succeed. Examples are
requirements on fine motor skills, content knowledge, or content-related skills.
Figure 4. Game Mechanic in
Angry Birds
requires fine motor skills and physics knowledge to solve problem
!
! ! !
*+!
For example, the game mechanic in the popular game
Angry Birds
involves moving the
correct bird to the appropriate part of the structure containing pigs via a sling mechanic that
requires players to determine the angle and force of the bird trajectory (Figure 4). If a learning
game were to use this mechanic, then a learner who might know which bird should hit which
part of the structure would also need to have basic knowledge of Newtonian mechanics and
the necessary fine motor skills to use the sling mechanic to be able to move the bird to this
spot. In this case, the game mechanic would introduce the skill of using a sling and physics
knowledge to the solution of the problem, which adds unnecessary confounds from a learning
perspective.
Library of Learning Mechanics
In order to provide game designers of learning games with a set of learning mechanics
and associated instances of game mechanics that they can use for their own game design, we
have begun to compile a library of mechanics. This library includes a variety of options of game
mechanics for each learning mechanic; see Figure 5 (see http://g4li.org for updates).
Figure 5. Library of Learning Mechanics and Associated Game Mechanics
It is important to note that since learning mechanics are design patterns, there is a one-to-
many relationship of learning mechanics to game mechanics, and that each of the different
game mechanics that instantiates a learning mechanic may only be suitable under specific
conditions. Our ongoing research is concerned with adding new learning mechanics and
!
! ! !
**!
associated game mechanics, and with demonstrating their viability and usefulness through
empirical research.
Assessment Mechanics
In addition to engage learners in meaningful activities that facilitate learning and assist in
the creation of mental models, games also have the ability to provide educators and designers
as well as the players/learners themselves with insight into players’ learning processes and
advancements. The rules systems created by game mechanics can also be used for
assessment of a variety of variables including, but not limited to, learning outcomes.
Learning objectives and learning processes of interest can be operationalized into specific
actions within the game that allow for an assessment of their level of achievement by the
player/learner. Player actions can be captured in log files and can be analyzed to reveal what
players learned and how they learned it. Game mechanics for assessment must therefore be
designed to elicit relevant behaviors that can be observed through the user log and
interpreted to reveal learning process, outcome, and learner variables. We call mechanics
designed for this purpose
Assessment Mechanics
and define them, based on Salen &
Zimmerman (2004), as follows:
Assessment mechanics are patterns of behavior or building blocks of diagnostic
interactivity, which may be a single action or a set of interrelated actions that form the
essential diagnostic activity that is repeated throughout a game.
Similar to learning mechanics, assessment mechanics are design patterns, or meta-
mechanics, that describe the tools or activities but are not playable mechanics. They describe
the functions of the tools available to players to demonstrate their knowledge and skills or
expressions of other variables of interest, but they don’t describe the actual tools or
assessment tasks themselves. For example, the assessment mechanic might specify that the
learner/player should group related items in time or space, but it does not describe whether
!
! ! !
*"!
this is done by shifting items on the screen like in
Bejeweled
, dropping them in specific
locations like in
Drop Seven
, or placing them like in a
tower defense
game.
There are several variables that are of interest in the design of personalized or adaptive
games. These variables can be grouped as general trait variables, general state variables, and
situation-specific state variables. Some of these variables can be reliably assessed with valid
traditional instruments, but for many variables only methods with low reliably and validity are
available, often involving self-reported data, which are susceptible to learner biases and other
response sets.
For example, learners’ self-regulation is of interest because it describes whether players
establish learning goals, monitor their goal achievement, and change strategies when they are
not able to achieve their goals. Another example is the assessment of specific aspects of
learning. In the game
Noobs vs. Leets
, we were interested in understanding how well the
learner comprehends rules related to angles in quadrilaterals, such as the complementary
angles rule, and opposite angles rule. We therefore chose an assessment mechanic that
required learners to drag the correct rule to the angle to be solved (Figure 6, right). An
alternative assessment mechanic that would have required the learner to drag or enter the
correct numeric value for each angle (Figure 6, left) could not have revealed the source of any
possible errors, which could have been conceptual (lack of rule knowledge) or arithmetic (lack
of subtraction skills).
Figure 6. Two Assessment Mechanic options in Noobs vs. Leets
!
!
! ! !
*#!
This reasoning is based on the Evidence-Centered Design (ECD) Framework (Mislevy,
Almond, & Steinberg, 2003). This framework provides a formal approach to the essential
questions related to assessment design: What should be assessed (Student/ Competency
Model); What kinds of learner behaviors can be used to reveal these constructs (Evidence
Model), and, What tasks and activities can be designed to elicit these behaviors (Task Model).
The ECD model can be user to compile a list of criteria that mechanics have to meet in order
to be useful assessment mechanics.
Criteria for Assessment Mechanics
Similar to learning mechanics, assessment mechanics have to meet a series of criteria to
assure they engage the player in meaningful and valid assessment activities. The overall goal
of assessment mechanics is to elicit relevant behaviors that can be observed through the
user log and interpreted to reveal learning process, learning outcomes, and learner variables.
In order to be useful for this purpose, i.e., to be a valid measure producing reliable scores, a
first criterion is that assessment mechanics have to be based on models such as ECD. Based
on the student model of target competencies, described in relation to the changes in skills,
knowledge, identity, values, and epistemology of interest (Rupp, Gushta, Mislevy, & Shaffer,
2010), assessment designers need to construct an ECD Evidence Model. This means that
they have to specify the salient features of learner behavior and the rules for scoring and
interpreting these features for the purpose of assessment. This includes another important
criterion for the design of assessment mechanics, which is the consideration of Test
Theoretical Concerns. For example, since we cannot assume that individual test items can be
independent of one another, the statistical model of the assessment has to reflect these
possible dependencies (Rupp et al., 2010).
The next criterion is that the mechanics should to be designed based on aspects of the
ECD Task Model, i.e., the description of the key tasks and activities in which the learner will
engage for the purpose of assessment. These tasks form the assessment mechanic, and it is
!
! ! !
*$!
essential that they are designed in a way that the task execution can be captured through the
instrumentation of the game. This requires, for example, that the mechanics require the
learner to make explicit the steps learners used for problem solving rather than simply
provide the answer to the problem. The mechanics should also allow for repeated exposures
to similar problems to allow for multiple observations of the behavior of interest.
Depending on the decisions made by the designers, the assessment character of the
assessment mechanic may or may not be obvious to learner. We describe assessments in
which learners are likely aware of the fact that they are being assessed as Embedded
Assessment, and those where they are not aware of this fact as Stealth Assessment (Shute,
2010).
Once measurement experts have designed an assessment mechanic, game designers
can design corresponding instances of game mechanics. However, in this process, several
design requirements have to be met.
Requirements for Designing Game Mechanics based on Assessment Mechanics
One of the most common problems in designing game mechanics based on assessment
mechanics is the introduction of various confounds that make it difficult to determine whether
variability in learning scores among learners can be attributed to their different knowledge
and skills, or whether it is caused by other factors.
One such confound is the addition of new sources of extraneous cognitive load. For
example, game mechanics such as in
Flight Control,
where players have to determine the
approach patterns of airplanes for landing, are fun and engaging because the fast succession
of a high number of planes to land puts high demands on players’ processing. This would be
appropriate to assess speed of processing, but not to assess conceptual knowledge or higher
level thinking.
A related confound is the addition of scaffolding or guidance that reduces the cognitive
task demands for some learners but not for others. For example, if key information in an
!
! ! !
*%!
adventure game is hidden so that only some players will find it then assessment of knowledge
will be confounded by learners’ exploration strategies.
Another typical problem for assessment is when the game mechanic introduces
confounds though demands on fine motor skills, which is highly variable in learners. For
example,
MotionMath
asks learners to tilt their tablet device to direct a ball to the correct
answer. Success in this task does not only depend on learners’ knowledge, but also of their
fine motor skills in being able to move the ball to the correct location.
Likewise, many game mechanics include activities that require the learner to have
additional content knowledge or skills from unrelated subject matter areas. For example, an
assessment of algebra may be confounded by the need to know about Newtonian physics.
Although the integration of different subject matter areas is a desirable design feature for
learning mechanics, the use of the same strategy in the design of assessment mechanics
may confound results.
A final confounding variable to consider is emotion. During game play, learners will likely
experience a series of emotional responses that would impact learning outcomes (Um, Plass,
Hayward, & Homer, 2011). Designers of assessment mechanics need to consider learners’
emotions and design mechanics aimed at assessment in a way that the learners’ emotional
response does not interfere with their ability to solve the problems presented. A particularly
problematic situation would be a mechanic in which different people respond emotionally in
different ways. If such a situation is expected, the assessment model should include learners’
emotional state as a variable.
Our current work aims to compile a library of assessment mechanics with corresponding
game mechanics that meet our requirements.
Library of Assessment Mechanics
In order to provide game designers of learning games with a set of assessment
mechanics and associated game mechanics that they can use for their own game design, we
!
! ! !
*&!
have begun to compile a library of mechanics. This library lists a variety of options of game
mechanics for each assessment mechanic, see Figure 7 (see http://g4li.org for updates).
Figure 7. Library of Assessment Mechanics and Associated Game Mechanics
It is important to note that there is a one-to-many relationship of assessment mechanics
to game mechanics, and that each of the different game mechanics that can instantiate an
assessment mechanic may only be suitable under specific conditions. Our ongoing research is
concerned with adding new assessment mechanics and associated game mechanics, and
with demonstrating their viability and usefulness in empirical research.
Summary and Conclusion
Our approach suggests that Game Mechanics, the essential game play activity, should be
distinguished from Learning mechanics and Assessment mechanics. We define Learning
Mechanics as patterns of specialized activities, grounded in the learning sciences, that have
learning as the primary objective. In contrast, Assessment Mechanics are patterns of
specialized activities, grounded in test theory, that have assessment as the primary objective.
Learning and assessment mechanics are design patterns, or meta-mechanics, that can be
instantiated onto corresponding game mechanics, following criteria we outlined above to
preserve their intended teaching or assessment objective.
Variables related to learning that can be measured through game metrics include learning
outcomes (cognitive and skills), trait variables, general state variables, and situation-specific
!
! ! !
*'!
state variables. Supplementing log data of game events and user behavior with observational
data extends the ECD model and results in more valid assessments of these variables.
Our approach serves two related but separate goals. One goal is a measurement goal –
embedded assessment allows for more detailed insights into learning than many traditional
instruments, both with respect to the process of learning and learning outcomes. This has
implications for research as well as learner competency testing. The other goal is related to
improving the game play. By using assessment mechanics to measure a series of learner
variables, a learner model can be compiled that allows for the design of games that are
individualized and adaptive to a learner’s specific needs and characteristics. This has
implications for the design of effective games for learning by making games more adaptive
and personalized, and, hopefully, more effective.
In summary, we described an approach that, grounded in theory and tested in several
game design projects, has implications both for research and practice of the design of games
for learning.
!
! ! !
*(!
References
,-./012.34!56!7*)''86!A"Pattern"Language:"Towns,"Buildings,"Construction6!9/:;32!<1=>.3?=@A!B3.??6!!
,12.3?;14!C6!56!7*)''86!DE.!1;@=;1!;:!?FE.G0@0!012!@E.!.2HF0@=;10-!.1@.3I3=?.J!K.1.30-!2=?FH??=;1!;:!@E.!
F;1:.3.1F.6!L1!C6!56!,12.3?;14!C6!M6!NI=3;4!O!P6!Q6!R;1@0SH.!7Q2?68!Schooling"and"the"acquisition"of"
knowledge!7II6!$*%T$#*86!U=--?20-.4!VMJ!W0X3.1F.!Q3-Y0HG!,??;F=0@.?4!BHY-=?E.3?6!!
Z;S;?@4!L6!7"++(86!DE.!3E.@;3=F!;:!>=2.;!S0G.?6!L1![6!N0-.1!7Q2684!The"Ecology"of"Games:"Connecting"Youth,"
Games"and"Learning!7II6!**'T*$+86!50GY3=2S.!R,J!RLD!B3.??6!
5;--=1?4!,6!7*)((86!Cognitive"Apprenticeship"and"Instructional"Technology6!U=--?20-.4!VMJ!W0X3.1F.!Q3-Y0HG6!
5;;\4!]6!7"++&86!PE0@!03.!S0G.!G.FE01=F?^6!-;?@S032.16F;G4!C.@3=.>.2!R0A!"#324!"+*+!:3;G!
E@@IJ__-;?@S032.16F;G_"++&_*+_XE0@T03.TS0G.TG.FE01=F?6E@G-6!
];G0S\4!N64!NFEX03@`4!C64!O!B-0??4!M6!W6!7"+*+86!].:=1=1S!=1@.30F@=>=@A!=1!GH-@=G.2=0!-.031=1S6!Computers"in"
Human"Behavior,"264!*+"$a*+##6!!
K..4!M6!B6!7"++(86!What"video"games"have"to"teach"us"about"learning"and"literacy4!3.>=?.2!012!HI20@.26!
Z0?=1S?@;\.J!B0-S30>.!R0FG=--016!
UH1=F\.4!C64!W.Z-01F4!R64!O!bHY.\4!C6!7"++$86!R],J!,!:;3G0-!0II3;0FE!@;!S0G.!2.?=S1!012!S0G.!3.?.03FE6!L1!
Proceedings"of"the"Challenges"in"Game"AI"Workshop,"19th"National"Conference"on"Artificial"Intelligence!
7,,,L!c+$4!N01!M;?.4!5,84!,,,L!B3.??6!
L?Y=?@.34![64!d-010S014!R6!O!U0?E4!56!7"+*+86!].?=S1=1S!S0G.?!:;3!-.031=1SJ!L1?=SE@?!:3;G!F;1>.3?0@=;1?!X=@E!
2.?=S1.3?6!Proceedings"of"CHI"75;1:.3.1F.!;1!EHG01!:0F@;3?!=1!F;GIH@=1S8!"+*+4!,@-01@04!K,4!<N,6!
Me3>=1.14!,6!7"++(86!Games"without"Frontiers:"Theories"and"Methods"for"Game"Studies"and"Design6!D0GI.3.J!
D0GI.3.!<1=>.3?=@A!B3.??6!
MHH-4!M6!7"++#86!DE.!K0G.4!@E.!B-0A.34!@E.!P;3-2J!W;;\=1S!:;3!0!U.03@!;:!K0G.1.??6!L1!Level"Up:"Digital"Games"
Research"Conference"Proceedings4!.2=@.2!YA!R03=1\0!5;I=.3!012!M;;?@!C0.??.1?4!#+T$%6!<@3.FE@J!
<@3.FE@!<1=>.3?=@A6!
W0>.4!M6!7*)((86!Cognition"in"Practice:"Mind,"Mathematics,"and"Culture"in"Everyday"Life6!50GY3=2S.4!<[J!
50GY3=2S.!<1=>.3?=@A!B3.??6!
W0>.4!M64!O!P.1S.34!Q6!7*))+86!Situated"Learning:"Legitimate"Peripheral"Participation6!50GY3=2S.4!<[J!
50GY3=2S.!<1=>.3?=@A!B3.??6!
R=?-.>A4!C6!M64!N@.=1Y.3S4!W6!N64!O!,-G;124!C6!K6!7"++#86!91!@E.!?@3HF@H3.!;:!.2HF0@=;10-!0??.??G.1@?6!
Measurement:"Interdisciplinary"Research"and"Perspectives,"14!#T&'6!
[=3?FE1.34!B6!,64!NX.--.34!M64!O!5-03\4!C6!Q6!7"++&86!PEA!G=1=G0-!SH=201F.!2H3=1S!=1?@3HF@=;1!2;.?!1;@!X;3\J!,1!
010-A?=?!;:!@E.!:0=-H3.!;:!F;1?@3HF@=>=?@4!2=?F;>.3A4!I3;Y-.GTY0?.24!./I.3=.1@=0-4!012!=1fH=3ATY0?.2!
@.0FE=1S6!Educational"Psychologist4!$&7"84!'%T(&6!
!
! ! !
*)!
CHII4!,6!,64!KH?E@04!R64!R=?-.>A4!C6!M64!O!NE0::.34!]6!P6!7"+*+86!Q>=2.1F.TF.1@.3.2!2.?=S1!;:!.I=?@.G=F!S0G.?J!
R.0?H3.G.1@!I3=1F=I-.?!:;3!F;GI-./!-.031=1S!.1>=3;1G.1@?6!Journal"of"Technology,"Learning,"and"
Assessment,"87$86!!
N0-.14![64!O!b=GG.3G014!Q6!7"++#86!Rules"of"Play:"Game"Design"Fundamentals6!50GY3=2S.4!R,J!RLD!B3.??6!
NFE1;@`4!P64!ZgF\-.34!M64!O!K3`;12`=.-4!U6!7*)))86!L12=>=2H0-!012!F;T;I.30@=>.!-.031=1S!X=@E!=1@.30F@=>.!
01=G0@.2!I=F@H3.?6!European"Journal"of"Psychology"of"Education4!*$4!"$%T"&%6!!
NFE;.1:.-24!,6!7*)(%86!Mathematical"Problem"Solving6!V.X!h;3\J!,F02.G=F!B3.??6!
NEH@.4!i6!M6!7"+*+86"Innovative"Assessment"for"the"21st"Century:"Supporting"Educational"Needs6!V.X!h;3\4!V6h6J!
NI3=1S.36!
NI=3;4!C6!M64!5;H-?;14!C6!W64!d.-@;>=FE4!B6!M64!O!,12.3?;14!]6!7*)((86!5;S1=@=>.!:-./=Y=-=@A!@E.;3AJ!,2>01F.2!
\1;X-.2S.!0FfH=?=@=;1!=1!=--T?@3HF@H3.2!2;G0=1?6!L1!i6!B0@.-!7Q2684!Proceedings"of"the"10th"Annual"
Conference"of"the"Cognitive"Science"Society6!U=--?20-.4!VMJ!Q3-Y0HG6!
NX=1\4!N6!7"++(86!K0G.!d..-J!,!K0G.!].?=S1.3j?!KH=2.!@;!i=3@H0-!N.1?0@=;16!R;3S01![0H:G0116!
<G4!Q64!B-0??4!M6W64!U0AX0324!Q6964!O!U;G.34!Z6]6!7=1!I3.??86!QG;@=;10-!].?=S1!=1!RH-@=G.2=0!W.031=1S6!Journal"
of"Educational"Psychology.!
... The development team must decide upon the main learning mechanics that will be implemented in the serious game. We agree with the definition given by Plass et al [56], who defines learning mechanics as "patterns of behavior or building blocks of learner interactivity, which may be a single action or a set of interrelated actions that form the essential learning activity that is repeated throughout a game." Learning mechanics can include activities such as remembering, understanding, applying, analyzing, evaluating, or creating. ...
... Learning mechanics can include activities such as remembering, understanding, applying, analyzing, evaluating, or creating. Learning effectiveness increases when learning and game mechanics are aligned with learning objectives [42,[56][57][58]. This leads participants to develop and exercise their cognitive abilities throughout the game to reach its ultimate goal. ...
... In Figure 2, the left side of the framework corresponds to the learning design, and the right side corresponds to the game design. As in most design models [44,56], this vision emphasizes the inclusion of game and learning designs in serious game design. The Mechanics category can be seen as linking them together. ...
Article
Full-text available
Background Serious games are increasingly used at all levels of education. However, research shows that serious games do not always fulfill all the targeted pedagogical objectives. Designing efficient and engaging serious games is a difficult and multidisciplinary process that requires a collaborative approach. Many design frameworks have been described, most of which are dedicated to the development of specific types of serious games and take the collaborative dimension into account only to a limited extent. Objective Our aim was to create a generic serious game design framework that could be adapted to all kinds of serious games and implemented in a collaborative web platform. Methods We combined the results of a literature review with our experience in serious game design and development to determine the basic building blocks of a collaborative design framework. We then organized these building blocks into categories and determined the features that a generic design framework should include. Finally, based on the paradigm of complex systems and systemic modelling, we created the co.LAB generic design framework and specifications to allow its implementation in a collaborative web platform. Results Based on a total of 10 existing design methodologies or frameworks, 23 building blocks were identified and represent the foundation of the co.LAB framework. These blocks were organized into 5 categories: “context and objectives,” “game design,” “mechanics,” “learning design,” and “assessment.” The arrangement by categories provides a structure that can be visualized in multiple and complementary ways. The classical view links game and learning design while other views offer project, systemic, and process visualizations. For the implementation of the co.LAB framework in a web platform, we propose to convert the building blocks into “cards.” Each card would constitute a collaborative working space for the design of the corresponding block. To make the framework adaptive, cards could be added, adapted, or removed according to the kind of serious game intended. Enhancing the visualization of relationships between cards should support a systemic implementation of the framework. Conclusions By offering a structured view of the fundamental design elements required to create serious games, the co.LAB framework can facilitate the design and development of such games by virtue of a collaborative, adaptive, and systemic approach. The different visualizations of the building blocks should allow for a shared understanding and a consistent approach throughout the design and development process. The implementation of the co.LAB framework in a collaborative web platform should now be performed and its actual usability and effectiveness tested.
... Arnab et al. (2015) proposed the LM-GM framework as an analytic tool to understand how a game produces learning outcome (see also Kato et al., 2008). Basically, part of the framework holds that learning games are effective when the game designer considers how every game mechanics affects learning (Plass et al., 2011). The goal is to create an experience in which each of game mechanics facilitate the learning process, such that the learner's repeated interaction with the game results in learning (Salen et al., 2004). ...
... As an essential part of any game, feedback provides performance information to the player (Cezarotto & Battaiola, 2016;Zichermann & Cunningham, 2011) and shapes player's behaviors and interactions within the game (Plass et al., 2011). In the MR app, the player gets immediate clear feedback after each action (i.e., tapping where to dig) which both reinforces the correct association between a number and its size relative to another number and motivates them to continue playing. ...
Article
The present paper documents the design and development of a mobile mathematics appli- cation targeted to improve magnitude representation skills. Educational experts worked together with an app developer with the goal of creating an educational app as a math learning tool for children 5–8 years old. The description of the app design processes includes five core elements that we believe are central to the creation of a theory driven educational app. Creating a theory driven educational app is a difficult task; it involves a set of complex decisions as illustrated in this article.
... "Learning mechanics are patterns of behavior or building blocks of learner interactivity, which may be a single action or a set of interrelated actions that form the essential learning activity that is repeated throughout a game" (Plass, Homer, Kinzer, Frye, & Perlin, 2011, p. 3). In designing for learning, Plass et al. (2011) make the case that learning mechanics must further be intrinsically and meaningfully connected with game mechanics. They argue that the learning mechanic must be grounded in the learning sciences or learning theory. ...
... Learning mechanics describe which kinds of functions and scaffolds are needed in the environment, though not the actual game mechanics involved, which can vary by game design. An example of an ineffective learning mechanic would involve interrupting a racing or shooting game with popup "educational" questions before play could continue (Plass et. al., 2011). An example of an effective learning mechanic might be having a learner select or integrate related objects, though how they select or integrate them through game mechanics could vary by game or interface. For instance, a learner could drag one object onto the other, such as in a simple matching game, or break objects apart and put them ...
... Cook (2006) says the mechanics are rule-based systems and simulations that facilitate and encourage the players to explore and learn in the game environment. Plass et al. (2011) follow the same approach, stating that the mechanics enforce the players' behaviors and actions, directly facilitating the players understanding of the game system and enabling learning, which leads to direct relations between learning mechanics and game mechanics. If we consider that serious games are the games created or adapted with a defined objective in mind (learning and/or simulation), maintaining the fun of the play (Dörner et al. 2016;Michael and Chen 2006;Winn 2009), it is mandatory to analyze their game mechanics according to their objectives. ...
... If we consider that serious games are the games created or adapted with a defined objective in mind (learning and/or simulation), maintaining the fun of the play (Dörner et al. 2016;Michael and Chen 2006;Winn 2009), it is mandatory to analyze their game mechanics according to their objectives. For a learning mechanic to help players to learn in a serious game, it needs to be implemented in a game system as a game mechanic, relating to the activity of playing, bounded by the rules of the learning theories (Plass et al. 2011). When searching for MBG mechanics literature, difficulties establishing a connection with the previous definitions arise. ...
Conference Paper
Board games are thriving in the post-digital era. Digitalization is present in almost all dimensions of contemporary life and it is expected to be even more prevalent in the future. However, a growing number of players are choosing to play analog games, mainly influenced by modern board games and hobby game designs. These games are different from mass-market games and show that analog platforms still have much to offer in terms of gaming innovation. These new designs have not yet been adequately studied by the serious game research field, and the filling of this research gap can provide new solutions for learning and simulation games of all types. This paper proposes a new framework, relating learning mechanics to the tabletop mechanisms (LM-TM) of the modern board game designs. The case study of the adapted version of Steam board game, tested during a lecture with MBA students, is explained as an example of the application of the LM-TM framework. This case study proved to be useful to unravel future development paths in this field of research.
... jumping and collecting objects) with learning actions (e.g. solving problems), since this is an alignment that characterizes effective games for learning (Plass et al., 2011). ...
Conference Paper
Full-text available
This article presents a literature review on the state of the art of research on students' learning through educational game design. A growing number of researchers are studying students' design of educational games as a means of learning. Students are given active roles as educational game designers and learn by applying knowledge about their academic subjects to game learning and game mechanics, as well as by discussing and playing their games with peers. The role of game designers requires students to be innovative, make complex choices and apply creativity to reach their academic learning goals. The teacher plays the active role of co-creator in educational game development processes. Although this learning approach has great potential, it is complex for individual teachers to apply. Additionally, it can be difficult to assess and measure students' learning outcomes. The present systematic review investigates successful approaches as well as gaps in this research area. A total of 17 articles met the inclusion criteria and were coded for the literature analysis. The findings of the articles were extracted and synthesized to identify the dominating and essential themes and elements that contribute to the success of this way of learning. The reviewed articles shed light on (1) recommended pedagogical approaches, (2) examples of learning design frameworks, (3) methods for applying formal learning goals to students' educational games, (4) methods for measuring learning outcomes and (5) design of teacher and student support in the learning process. For each of these subtopics, future directions are proposed for improving the research area in the best possible way.
... Another key point of tension had to do with the integration of purposeful support for the learning objectives of the game. As others have noted, writing on the design of educational games (Ke, 2016;Plass et al., 2011), a good balance between the educational and, respectively, fun aspects of such games is of paramount importance. ...
Article
Full-text available
Given the need for innovative, engaging, and youth-centered approaches to media literacy, as well as the potential of active pedagogies to facilitate youth civic education and efficacy, games emerge as a particularly promising and under-utilized avenue for news literacy education. Our research asks, how might we use game-based learning to tackle fake news and stimulate news literacy among a youth audience? Here, we reflect on the process of designing LAMBOOZLED!, a news literacy game for middle school and high school students, based on a multilevel game design framework that allowed us to articulate learning objectives, consider suitable mechanics, dynamics and aesthetics, and integrate relevant instructional principles along multiple learning dimensions. Positioning this work at the nexus of game design and media literacy education, we discuss our key decision points and the larger stakes of adopting a game-based approach to news literacy education in the current political climate.
... The term "game mechanics" refers to methods invoked by agents for interacting with the game world [24]. "Learning mechanics" are patterns of behavior of learner interactivity [25]. Matching game mechanics and learning mechanics is a concise means to relate ludic elements and teaching objectives within the gameplay [26]. ...
Conference Paper
Full-text available
Serious games (SGs) are motivational and practical pedagogical tools that have been widely used in design education. SGs seem to be an efficient way to give instructions on innovation processes (IPs), offering unique and attractive environments to support situated learning. While there has been much interest in SGs of the IPs type, there is very little research about the design framework to reduce the complexity and time consumption of their design process. This paper presents the preliminary results of our ongoing study: a design framework adapted to innovation SGs. The framework integrates eight general design frameworks/models/methodologies for SGs. Besides, it introduces a new stage “analysis of traditional teaching experience,” which conducive to the early phases of the design. We use a case study to prove the value of this stage. First, it aids designers in defining the teaching objectives of innovation SGs, that is, choosing required competencies from innovation competency frameworks. More importantly, it helps identify game mechanics that may contribute to the realization of teaching objectives. This stage should support designers successfully making the transition from traditional innovation teaching towards SGs.
... The researchers and designers need to have clear understanding of what their design questions are and reach consensus on how to define metrics. Given the field of game analytics is fairly young [4] as is the idea of using data to inform game design [10], it remains unclear how game analytics can be used to balance among three possibly conflicting goals-learning, game, and assessment [11]. Kim and Shute [9], for example, demonstrated in an A/B testing that changing one design choice (linear puzzle sequence), significantly influences learning and enjoyment. ...
Conference Paper
Full-text available
There is increasing interest in using data to design digital games that serve the purposes of learning and assessment. One game element, difficulty, could benefit vastly from applying data-driven methods as it affects both players’ overall enjoyment and efficiency of learning and qualities of assessment. However, how difficulty is being defined varies across the learning, assessment, and game perspectives, yet little is known about how educational difficulty can be balanced in educational games for each of the potentially conflicting goals. In this paper, we first review varying definitions of difficulty and then we discuss how we came up with a difficulty metric and used it to refine our game-based assessment Shadowspect. The design guidelines, metrics and lessons learned will be useful for designers of learning games and educators interested in balancing difficulty before they implement these tools in the classroom.
Conference Paper
Full-text available
The knowledge and the implementation of patients' rights are fundamental in health professional practices. Courses for healthcare students are often taught using a lecture format. Yet, students find it difficult to understand the importance of legal concepts and apply it to their future practice. In order to promote learner centered teaching practices and encourage knowledge acquisition and practical skills development, the "Patient's Rights & Innovative Teaching Strategy (PRITS)" project aimed at developing a serious game dedicated to patients' rights education. To design and develop this serious game and the associated learning concept, we set up a multidisciplinary team of experts from the fields of healthcare, law, education and engineering. A collaborative methodology was used to ensure the coherent development of all games and learning elements. The PRITS serious game integrates knowledge about patients' rights into dialogues with virtual patients. It also provides additional resources and exercises related to the topics. The learning experience intends to challenge students' mental models and support a transition from "quality of care only "to "quality of care and respect of patients' rights".
Chapter
For any given Game Based Learning (GBL) project to be successful, the player must learn something. Designers may base their work on pedagogical research, but actual game design is still largely driven by intuition. People are famously poor at unsupported methodical thinking and relying so much on instinct is an obvious weak point in GBL design practice. Cognitive Walkthrough (CW) is a user-interface design technique that helps designers model how a type of user will understand an interface. The authors suggest that CW should be extended for use in any context where a designer must model a user’s thinking. They present an extension of CW that is suitable for constructivist GBL and apply it to a previously evaluated game to understand why one section of the game was more successful than another. The CW extension explains hitherto puzzling results and suggests further development of CWs for designer support may be beneficial.
Article
Full-text available
Computer-based learning environments provide the possibility to present interactive animated pictures, which can be manipulated for active exploratory learning and which allow to display the dynamic behavior of a complex subject matter. Due to the large range of possibilities of exploratory interaction, such learning environments seem to be well suited for cooperative learning, where different learners analyse a subject matter from different perspectives. Knowledge acquisition from interactive animated pictures was compared with knowledge acquisition from static pictures in two empirical studies under the conditions of individual learning (Study I) and of cooperative learning (Study II). In Study I, learning with interactive animated pictures resulted in a better encoding of detail information, but did not have positive effects on performance in mental simulation tasks. In Study II, learning with interactive animated pictures resulted both in lower encoding of detail information and poorer results in mental simulations. These findings and the analysis of discourse protocols of the co-operation suggest that exploratory learning with interactive animated pictures is associated with extraneous cognitive load, which can be further increased by the co-ordination demands of co-operative learning. Although animated pictures provide external support for mental simulations, they seem to be not generally beneficial for learning, as they can prevent individuals from performing relevant cognitive processes.
Article
Full-text available
Evidence for the superiority of guided instruction is explained in the context of our knowledge of human cognitive architecture, expert–novice differences, and cognitive load. Although unguided or minimally guided instructional approaches are very popular and intuitively appealing, the point is made that these approaches ignore both the structures that constitute human cognitive architecture and evidence from empirical studies over the past half-century that consistently indicate that minimally guided instruction is less effective and less efficient than instructional approaches that place a strong emphasis on guidance of the student learning process. The advantage of guidance begins to recede only when learners have sufficiently high prior knowledge to provide “internal” guidance. Recent developments in instructional research and instructional design models that support guidance during instruction are briefly described.
Article
Full-text available
In educational assessment, we observe what students say, do, or make in a few particular circumstances and attempt to infer what they know, can do, or have accomplished more generally. A web of inference connects the two. Some connections depend on theories and experience concerning the targeted knowledge in the domain, how it is acquired, and the circumstances under which people bring their knowledge to bear. Other connections may depend on statistical models and probability-based reasoning. Still others concern the elements and processes involved in test construction, administration, scoring, and reporting. This article describes a framework for assessment that makes explicit the interrelations among substantive arguments, assessment designs, and operational processes. The work was motivated by the need to develop assessments that incorporate purposes, technologies, and psychological perspectives that are not well served by familiar forms of assessments. However, the framework is equally applicable to analyzing existing assessments or designing new assessments within familiar forms.
Book
Innovative Assessment for the 21st Century Supporting Educational Needs Valerie Shute and Betsy Becker, editors Assessment continues to take center stage in contemporary education, not merely for the data themselves, but for what they can tell us about students and their instructors—and even more important, about educational domains that need upgrading. Success in the technology-driven modern world increasingly depends on new competencies (many of which have yet to be fully identified), requiring new methods for their accurate measurement. Innovative Assessment for the 21st Century asks readers to rethink the way assessment is conducted and competencies are defined, placing the process in the context of lifelong learning across subject domains. This forward-looking dialogue between contributors in education research, practice, and policy examines a range of specific assessment issues, technologies to address these needs, and larger policy concerns. Chapters focus on the most pressing goals and challenges facing the field today, including: • Using assessment to strengthen learning. • Recognizing the “natural” role of assessment in everyday life. • Creating an assessment culture in the schools. • Improving assessment through evidence-based methods. • Assessing competencies based in new media and technologies. • Making assessment relevant to students and faculty. Innovative Assessment for the 21st Century is a finely detailed blueprint for researchers in education and cognition, as well as for psychometricians in private and government agencies.