Content uploaded by Carlo Perrotta
Author content
All content in this area was uploaded by Carlo Perrotta on Nov 18, 2014
Content may be subject to copyright.
National Foundation for Educational Research,
The Mere, Upton Park, Slough, Berkshire SL1 2DQ
www.nfer.ac.uk
© National Foundation for Educational Research 2013
Registered Charity No. 313392
How to cite this publication:
Perrotta, C., Featherstone, G., Aston, H. and Houghton, E. (2013). Game-based Learning:
Latest Evidence and Future Directions. Slough: NFER.
Contents
Executive summary i
1. Introduction and aims 1
1.1 Review aims and questions 1
1.2 Methods 2
2. Definitions 5
2.1 What does this section cover? 5
2.2 What literature did we consider? 5
2.3 What does the literature say? 5
2.4 Moving towards a definition 7
3. Impact and potential impact of game-based learning 10
3.1 What does this section cover? 10
3.2 What literature did we consider? 10
3.3 What does the most up to date review of the evidence look like? 11
3.4 The type and extent of impact 14
3.5 The role of independent variables 17
3.6 The new landscape 19
4. Implications for future research 21
4.1 What does this section cover? 21
4.2 What literature did we consider? 21
4.3 Limitations of the reviewed evidence 21
5. Implications for teachers and schools 27
5.1 What can teachers safely ‘take home’? 27
5.2 Additional points for senior leaders who wish to support the use of game-based
learning in their schools 27
References 28
Appendix 1: Search strategy and the review process 31
Appendix 2: The evidence base for the review 35
Game-based Learning: Latest Evidence and Future Directions i
Executive summary
This review is the first output in the Innovation in Education strand of NFER’s research
programme. This strand will provide evidence about new approaches to education, teaching
and learning and aims to identify rewarding learning experiences that will inspire, challenge
and engage all young people, equipping them with the essential skills and attitudes for life,
learning and work in the 21st Century. Interest around the use of video games in education
is high, and following the emergence of new trends like ‘gamification’, Futurelab@NFER felt
that it was timely to provide educators, industry and researchers with an up-to-date analysis
of the literature.
To achieve this, we conducted a rapid review of the latest available evidence, seeking to
answer these research questions:
• What is game-based learning?
• What is the impact and potential impact of game-based learning on learners’
engagement and attainment?
• What is the nature and extent of the evidence base?
• What are the implications for schools?
The research questions are mainly concerned with the notion of ‘gameplay’ (playing games)
rather than ‘making games’ (how the prospect of creating original video games can be used
to interest young people in complex activities like software programming).
What is game-based learning in education?
• ‘Game-based learning’ broadly refers to the use of video games to support teaching and
learning. Although it is a relatively established notion, it is hard to define precisely. We
have arrived at a definition by extracting the key principles and mechanisms involved,
which Figure 1 shows.
Figure 1: The principles and mechanisms of game-based learning
Game-based Learning: Latest Evidence and Future Directions ii
• ‘Gamification’ is a much newer concept than game-based learning. It is about using
‘elements’ derived from video-game design, which are then deployed in a variety of
contexts, rather than about using individual video games.
• ‘Gameplay’ is the treatment of topics and ideas as rules, actions, decisions and
consequences, rather than as content to be communicated or assimilated. Video games
can be seen to allow learners to engage with topics and ideas through interaction and
simulation, rather than through the conventional materials and formats of schooling:
textbooks, lessons, assignments and so forth.
Impacts and potential impacts
• The literature was split on the extent to which video games can impact upon overall
academic performance. Where studies expressly sought to measure ‘academic
achievement’, five calculated some degree of improvement. Further, a meta-analysis of
studies observed significant, but undefined, cognitive gains across studies utilising
games versus traditional teaching methods. However, four studies found no impact on
academic achievement.
• The studies consistently found that video games can impact positively on problem
solving skills, broader knowledge acquisition motivation and engagement. All five studies
that specifically focused on problem solving skills found some degree of improvement,
and the majority of the studies examining the impact of video games on student
motivation and engagement found positive results. However, it was unclear whether this
impact could be sustained.
• Relatively few studies explored attitudes to learning as a possible outcome and their
findings varied. One found that games promoted a more positive attitude to maths
learning. A further study explored mathematics or academic self-concept (the set of
beliefs an individual holds about themselves as a mathematician) and found no
improvement. However, a meta-analysis found that significantly better attitudes towards
learning were yielded for subjects using interactive games or simulations, compared to
those using traditional methods for instruction.
• Despite some promising results, the current literature does not evidence adequately the
presumed link between motivation, attitudes to learning and learning outcomes. Overall,
the strength of the evidence has been affected by the research design or lack of
information about the research design.
Some recommendations and ‘take-home’ points for
teachers
• The evidence suggests that game-based learning can improve engagement and
motivation, but don’t rely on games to improve attainment - there is still a lot we don’t
know about the impact of video games on learning.
• The best way of integrating gaming into teaching is by using it within a clear pedagogic
process. In particular:
Game-based Learning: Latest Evidence and Future Directions iii
• Place learning activities and academic content within the video game’s fictional and
entertainment context, maintaining a balance between fun and learning.
• Make the academic content integral to the game rather than an add-on. Content-specific
tasks work better when embedded in the fictional context and rules (‘mechanics’) of the
game.
• Carefully plan the roles that you and your learners will take on in the game. Teachers
should play roles that allow them to mediate the experience for learners: providing
guidance when needed; ensuring that rules are followed; and maintaining a respectful
atmosphere.
• Don’t try to divorce decontextualized components of a game (such as badges, scores or
leaderboards) from the fictional context and rules of the game (the ‘mechanics’)
Implications for future research
• It is important we develop a more analytic approach that considers how the different
elements that operate within video games impact in an educational setting.
• We noted a complete lack of evidence about ‘gamification’, that is, the use of techniques
and mechanisms derived from video games, rather than actual games. More research is
needed on the impact of such techniques and mechanisms on learning outcomes. Can
such elements be detached from the technology, without losing their meaning or
potency? Can they be used effectively to add value to traditional teaching and learning?
• Opening up the ‘black box’ of video games would enable us to focus on specific
principles or mechanisms. This finer grained approach could unlock a more rewarding
research agenda.
• We have identified three research challenges:
1. Working towards a consensus about the relationship between academic
achievement and game-based learning. Efforts are needed to articulate clear
relations between game elements and a range of outcome from a broad level
(e.g. platform on which the game runs, single player, multiplayer, and so
forth), gradually narrowing down to specific gameplay mechanics.
2. Unpacking further the relationship between gaming and academic
achievement in the context of educational assessment. Games seem to allow
more powerful and ‘always-on’ forms of assessment in which all actions,
interactions, successes and failures can be constantly tracked and logged.
This raises a number of empirical and ethical issues.
3. Developing research into the potential of video games that accounts for the
realities of schools. In particular, more research is needed into the social,
cultural and economic factors that influence attitudes towards the use of
game-based learning in our increasingly diverse and multicultural schools.
Methods
• We carried out a rapid review of key literature to identify relevant theoretical contributions
and evidence. This involved systematic searching and a consistent, best evidence,
Game-based Learning: Latest Evidence and Future Directions iv
approach to the selection of the literature. We focused on a range of sources, including
empirical, practice-based evidence and more speculative literature, published from 2006
onwards.
• The process led to a shortlist of 31 items, which we appraised. This involved extracting
key data about the items and assessing their quality and relevance to the study, based
on a structured appraisal of their full texts.
Game-based Learning: Latest Evidence and Future Directions 1
1. Introduction and aims
The role of video games in teaching and learning is a source of debate among many
educators, researchers and in the popular press. Detractors and advocates have been
discussing the influences and the potentials of video games for quite some time, and we feel
that sound evidence and informed advice on these topics is still very much needed. Against
this background, Futurelab@NFER felt that it was timely to provide practitioners, industry
and researchers with an up-to-date account of what the evidence tells us about game-based
learning and its potential impact on learning and teaching.
This review is the first output in the ‘innovation in education’ strand of NFER’s research
programme. This strand will provide evidence about new approaches to education, teaching
and learning and aims to identify rewarding learning experiences that will inspire, challenge
and engage all young people, equipping them with the essential skills and attitudes for life,
learning and work in the 21st Century.
The review aims to bridge academic and non academic domains, to provide insights that will
be of interest to educators, educational researchers, industry and others seeking to engage
in a more thoughtful debate about the types of educational values that can be attached to
gaming. In particular, we provide accessible advice for practitioners, in the belief that
innovation in education is always underpinned by informed and critical teaching.
1.1 Review aims and questions
This review aims to:
• develop a better understanding of the impact and potential impact of game-based
learning for school-aged learners;
• identify the implications for schools and learners;
• identify unanswered research questions.
To achieve these aims, in so far as we can from the available evidence, we seek to answer
four research questions:
• What is game-based learning?
• What is the impact and potential impact of game-based learning on learners’
engagement and attainment?
• What is the nature and extent of the evidence base?
• What are the implications for schools?
The research questions listed above are mainly concerned with the notion of ‘gameplay’
(playing games) rather than ‘making games’ (how the prospect of creating original video
games can be used to interest young people in complex activities like software
programming). We will explore the relationship between gameplay and learning from an
Game-based Learning: Latest Evidence and Future Directions 2
empirical (impacts and effects) point of view. We will also consider promising research
areas that could be further explored.
1.2 Methods
We carried out a rapid review of key literature to identify relevant theoretical contributions
and evidence. This involved systematic searching and a consistent, best evidence, approach
to the selection of the literature. We focused on a range of sources, including empirical,
practice-based evidence and more speculative literature, published from 2006 onwards.
To identify relevant items we looked for relevant sources across a number of databases and
search engines. We developed a search strategy by using the controlled vocabulary
pertinent to each source. The keyword terms included combinations of: ‘game-based
learning’, ‘gamification’, ‘game design’, ‘attainment’, ‘good practice’, ‘outcomes’ and others
(see appendix for the full search strategy). Table 1 presents the main criteria for the
identification of relevant literature.
Table 1: Selection criteria for the inclusion of literature
Publication date:
Work published from the year 2006
Geographical scope:
United Kingdom and international
Language:
Published in English
Study type:
Empirical research and/or evaluation; good practice examples;
theoretical items; published literature (peer and non-peer
reviewed)
As expected, the search returned a large number of items (485). We reviewed all abstracts
to select contributions that were of greatest relevance to the research questions. During the
screening and coding phases we also took into account additional criteria:
• Distinctions between different kinds of evidence, such as: quantitative evidence and
qualitative evidence; well-established trends and emerging findings.
• The validity or trustworthiness of individual studies’ findings according to the research
design, sample size, methods of data collection and data analysis, theoretical approach,
and relationship between claims made and evidence presented.
The process led to a shortlist of 31 items, which we appraised. This involved extracting key
data about the items and assessing their quality and relevance to the study, based on a
structured appraisal of their full texts. We applied the following quality criteria:
• High: large scale quantitative studies, meta-analyses or in-depth qualitative case studies
covering a range of settings and stakeholders where views are triangulated; systematic
reviews have also been included in this category .
• Medium: quantitative or qualitative studies with smaller samples; qualitative studies not
covering a range of settings or stakeholders; non-systematic reviews. We also included
in this category more speculative contributions that are based on existing theories and
indirect evidence.
Game-based Learning: Latest Evidence and Future Directions 3
NB: In accordance with the ‘best evidence’ approach, we have adopted a relativistic
approach, considering the best available evidence we could identify. As such, the ‘high’
criterion does not necessarily equate to ‘gold standard’ evidence, such as Randomised
Controlled Trials (RCTs).
1.3 The evidence base
Table two summarises the evidence base in relation to the quality criteria considered:
Table 2: Extent and nature of the core evidence
High
Medium
Qualitative N/A Marković et al., 2007; Salen,
2008
Quantitative Chuang and Chen, 2009;
Delacruz, 2011; Huizenga et al.,
2009; Miller and Robertson,
2011
Annetta et al., 2009; Bourgonjon
et al., 2010; Brom et al., 2011;
Fengfeng, 2008a; Kenny and
McDaniel, 2011; Kolovou and
Heuvel-Panhuizen, 2010; Liu et
al., 2011; Papastergiou, 2009;
Schaaf, 2012; Spires et al., 2011;
Vos et al., 2011; Ya-Ting, 2012
Mixed methods N/A Fengfeng, 2008b; Kebritchi et al.,
2010
Review and
Meta-analysis
Chiu et al., 2012; Vogel et al.,
2006; Young et al., 2012
De Freitas and Griffiths, 2008;
Kapp, 2012; Williamson, 2009
Speculative N/A Bogost, 2011; Buckingham, 2007;
Gee, 2008; McGonigal, 2011;
Shaffer, 2008
As you can see, Table 2 shows some basic descriptive information about the items that we
selected for inclusion in the review. Most notably, just over half of the items we selected to
include in the review (16) were quantitative studies. Of these, four items were judged as
being of ‘high’ quality and 12 of ‘medium’ quality. It is also worth noting that five speculative
items were included. These publications focused mainly on theoretical discussion, citing
indirect evidence to support claims or trying to suggest a research agenda for the future. We
have taken the view that these contributions are important, as they have shaped the
theoretical debate about game-based learning and gamification.
Chapter Two, which covers key definitions, draws on eight items of literature: one
quantitative study; one edited book (Salen, 2008), classified as qualitative as it features case
studies and ethnographic research; one review and five speculative items. The evidence for
this chapter is therefore less robust than for the chapter on impact (Chapter Three). Applying
Game-based Learning: Latest Evidence and Future Directions 4
lower inclusion criteria enabled us to discuss highly relevant and influential items that would
otherwise have been excluded from the review. Chapter Three, which explores the impact
and potential impact of video games on learning, draws on 21 of the items that we
systematically selected for inclusion. Of these, 18 items are primary studies (qualitative,
quantitative or mixed methods), and three are meta-analyses or reviews. Six of the items are
‘high’ quality, while the remaining 15 are ‘medium’ quality. Chapter Four, which focuses on
possible research areas for the future, takes into account all of the reviewed literature
Game-based Learning: Latest Evidence and Future Directions 5
2. Definitions
• Game-based learning’ broadly refers to the use of video games to support teaching and
learning. Although it is a relatively established notion, it is hard to define precisely. We
have arrived at a definition by extracting the key principles and mechanisms involved.
• ‘Gamification’ is a much newer concept than game-based learning. It is about using
‘elements’ derived from video-game design, which are then deployed in a variety of
contexts, rather than about using individual video games.
• ‘Gameplay’ is the treatment of topics and ideas as rules, actions, decisions and
consequences, rather than as content to be communicated or assimilated.
2.1 What does this section cover?
This chapter sets out the definitions of the key terms used in this review: game-based
learning, gamification and gameplay.
2.2 What literature did we consider?
We based the definitions on our review and synthesis of eight items of literature, six of which
are non-empirical items, as Table 3 shows. In selecting which literature to draw on, we
prioritised influential contributions, rather than the most robust studies. The definitions and
concepts first introduced in this more speculative and theoretical literature have informed
many empirical studies, and are frequently cited as background and inspiration.
Table 3: Extent and nature of literature considered for Chapter Two
High Medium
Qualitative Salen, 2008
Quantitative Annetta et al., 2009
Review De Freitas and Griffiths, 2008
Speculative Bogost, 2011; Gee, 2008;Kapp,
2012; McGonigal, 2011;
Shaffer, 2008
2.3 What does the literature say?
The contributions we reviewed suggested that game-based learning broadly refers to the
use of video games to support teaching and learning. Different studies articulate and
interpret this broad definition in different ways.
Game-based Learning: Latest Evidence and Future Directions 6
All the items reviewed in this chapter share the view that game-based learning is less about
the computer game as a specific entity and more about the complex social dynamics that
surround it. Game-based learning may or may not involve ‘educational’ video games (those
with an explicit focus on learning and improving attainment) rather than ‘leisure-use’ video
games. According to some authors, game-based learning encompasses technological
developments that are blurring the lines between formats, spaces, languages and practices
associated with video games, leading to ‘blended’ experiences, which are not just confined
to the video game itself (De Freitas and Griffiths, 2008).
In terms of the social dynamics of game-based learning, a common theme is that through
video games young people cultivate interests and join ‘affinity groups’ that operate across
contexts, as part of their projects of personal development. In these groups, players engage
in sophisticated forms of learning fuelled by the shared passion for gaming. They include
forums where players share ‘cheats’; wikis1
We also found the work of Bogost (2011) particularly useful to explore a definition of game-
based learning. Bogost is concerned with understanding the distinctive properties of video
games as a medium, in order to examine the implications in a range of social contexts,
including education and learning. He stresses that games are first and foremost
representations that simulate certain behavious and experiences, and use rulesets, role
playing and reward mechanisms to motivate and engage. They offer learners a way to more
fully engage with topics or ideas than they might be able to in the ‘real’ world. They can
exercise choice and control over (virtual) reality, without incurring any real consequences.
This means that a video game allows, at least in theory, learners to experience a certain
school subject through interaction and simulation, rather than through the conventional
materials and formats of schooling: textbooks, lessons, assignments and so forth. Bogost
contends that video games treat topics and ideas not as content to be communicated or
assimilated, but as rules, actions, decisions and consequences – that is, gameplay.
that clarify elements of the game universe; and
’modding’ groups who use game development skills to modify how games are played and
experienced (Gee, 2008). A similar, and equally popular, theme is that video games provide
virtual worlds which are effective contexts for learning, because acting in such worlds allows
learners to develop social practices and take on the identities of actual professional
communities. These soft learning outcomes are seen by many commentators as more useful
and worthy than the ‘outdated’ forms of knowledge acquired through traditional schooling
(Shaffer, 2008).
One item focused instead on ‘gamification’. Kapp describes gamification as ‘the careful and
considered application of game thinking to solving problems and encouraging learning using
all the elements of games that are appropriate’ (Kapp, 2012, p.12).
This notion seems to move away from much of the literature on game-based learning as
described in this review, where there is a clear distinction between video-games on the one
hand, and the learning dynamics that they may or may not enable on the other. As the word
1Collaboratively developed repositories of knowledge, the most popular of which is Wikipedia
(www.wikipedia.org).
Game-based Learning: Latest Evidence and Future Directions 7
suggests, gamification is more concerned with how certain situations or processes (including
learning) can be turned into playful experiences that, indeed, resemble video games.
Gamification is therefore interested in ‘gaming’ as a set of principles, tools and techniques,
which are born in video games but spill over into several other contexts, including education.
This spill-over has also been described as ‘adding a game layer on top of the world’2
2.4 Moving towards a definition
, thus
suggesting that a range of human endeavours could benefit from being viewed as games. In
this ‘gamified’ reality, rules and principles like those used by successful game designers
could make children and young people eager to engage in educational activities they used to
find boring, unrewarding or too difficult. For teachers, this raises the possibility of using
‘gamification’ as a strategy to improve teaching, learning and assessment. At its simplest,
this could mean using game-based techniques to encourage competition and self-
improvement in the classroom: badges, high scores, leaderboards and so forth. Similarly, it
could mean employing video game language to reframe traditional school activities – for
instance, assignments could become ‘missions’ or ‘quests’.
Taking into account all of the reviewed literature, we could further describe game-based
learning as a form of experiential engagement in which people learn by trial and error, by
role-playing and by treating a certain topic not as ‘content’ but as a a set of rules, or a
system of choices and consequences.
In curricular terms, this means translating an element of a subject (such as a law of physics
or the law of supply and demand) into the mechanics of a game, which operates within a
self-contained system based on choices and consequences. For instance, in the popular
series of simulation games SimCity3
‘if you really think about playing the game, it’s more like gardening. So you’re kind of
tilling the soil, and fertilizing it, and then things pop up and they surprise you, and
occasionally you have to go in and weed the garden, and then you maybe think about
expanding it, and so on’ (OP. CIT).
, players are tasked with running a virtual city, managing
a number of concurrent activities and priorities to keep the population happy. According to
the creator of the series (Pearce, 2002), the gameplay in SimCity reflects a specific logic (a
‘model’) of city-wide management. This could be likened to gardening.
In SimCity, players act according to the rules determined by this internal logic, which means
that each choice you make in the game has a specific effect. As such, ‘learning through
gameplay’ in SimCity means exploring the possibilities and the limits that are offered by the
underlying model: if the population is starved, then chaos and riots will ensue; if the city is to
be expanded, then sources of income are needed; and so forth. All this is performed in a
safe environment where failure is not only contemplated but actively encouraged. In fact,
2This is the title of a talk filmed for the website TED:
http://www.ted.com/talks/seth_priebatsch_the_game_layer_on_top_of_the_world.html
3A SimCity ‘EDU’ version hasjust beenannounced. The result of a partnership between the video
games company Electronic Arts and the Glass lab. See http://signup.simcityedu.org/
Game-based Learning: Latest Evidence and Future Directions 8
part of the fun is the awareness that mistakes have no lasting or real consequences, but are
simply necessary to improve performance.
Complementing the definition proposed above, we suggest a number of principles and
mechanisms that we have inferred from the literature considered for this chapter. These are
shown in Figure 1 (below). By ‘principles’ we mean underlying assumptions and concepts;
by ‘mechanisms’ we mean processes and dynamics that help us understand how video
games can, in theory, assist the pursuit of educational goals. The principles and
mechanisms involved in game-based learning are interdependent.
Some readers may be struck by the commonsensical nature of some of the elements
considered. Arguably, this goes some way to illustrating how innovations work best when
they bring into sharper relief what we know already about education and learning.
Game-based Learning: Latest Evidence and Future Directions 9
Figure 1: The principles and mechanisms of game-based learning
Principles
1. Intrinsic motivation.Gaming is intrinsically motivating
because by and large it’s avoluntary activity.Therefore,
gaming for learning works best in the context of invitation
and persuasion, rather than compulsion.
2. Learning through intense enjoyment and “fun”.
Several authors suggest that games can be avehicle for
engaging students in a“flow”.Flow is astate of
consciousness during which an individual is in control of
his actions and completely absorbed in the task at hand.
3. Authenticity.Authenticity means aconcern for the
real nature of learning, which is supposedly different from
the “artificial” or decontextualised forms of learning that
take place in schools.In the name of authenticity,
contextual skills are prioritised over the abstract notions
and facts valued in traditional instruction.Therefore,
“good” gaming reflects actual learning processes, which
are always grounded in specific settings and practices.
These can be actual professions, but also extravagant,
fantastic roles and endeavours.
4. Self-reliance and autonomy.Gaming encourages
independent inquiry and exploration;interests and
passions can branch off from the individual game,
towards aspects of the “ecosystem” that surrounds it.
These aspects include technical and artistic skills like
programming, writing, drawing, making music;but also
the desire to find out more about certain topics , e.g.
about science, history or mythology.
5. Experiential learning.The notion of experiential
learning is avery old and influential one in education,
dating back to the seminal work of John Dewey.Many
claim that gaming provides acost effective alternative to
learning by doing in real settings.
Mechanisms
1. Rules. At their most basic level, video games are sets of rules.These rules can be more or less complex depending on the
choices they elicit and the related consequences.For instance, rules can be simple and binary (if/then);or multifaceted and
accommodating abroad range of decision making processes.
2. Clear but challenging goals.The presence of clearly defined, demanding activities which, although might appear arbitrary and
unnecessary, allow people to see the direct impact of their efforts.
3. Afictional setting or “fantasy” that provides acompelling background.This is an essential but easily misunderstood
aspect of gaming.Notwithstanding the tendency to indulge in escapism and compensatory fantasies, classic studies on playing
suggest that pretence can also be adeliberate and conscious strategy that assists learning. A consensual and transparent
adherence to afictional setting or role allows players to experiment with skills and identities without suffering the consequences of
failure in real life.
4. Progressive difficulty levels ,underpinned by understandable criteria for progression.Over the years game developers have
devised mechanisms for progression and “levelling up” to aconsiderable degree of refinement.Not all of these mechanisms may
be appropriate in an educational context, but they raise interesting questions for educators -not least the fact that players are
allowed to tackle challenges and tests (like exams) as many times as necessary -and with no lasting consequences -in order to
progress from one level to another.
5. Interaction and high degree of student control.This mechanism is strongly related to the notion of “agency”:the feeling of
being in control of one’s destiny through actions and choices.Most importantly, it concerns the certainty that effort and dedication
will be acknowledged and rewarded.
6. Adegree of uncertainty and unpredictability. As long as it does not contradict point 5, a measured injection of uncertainty
into tasks is consistent with many game developments tenets.For example, in the Quest schools students are not given
assignments as such, but asked to choose between number of possible “missions”.The scope and purpose of the missions
become clear only when certain clues have been gathered and deciphered.
7. Immediate and constructive feedback .One of the most powerful mechanisms of video games is the ability to provide
feedback in real time, not only as evaluation, but more often as guidance to facilitate and correct performance.This is largely in
line with the idea of formative assessment in education.
8. Asocial element that allows people to share experiences and build bonds. As mentioned several times in this chapter, a
game is not simply aproduct or tool which may or may not have arelationship with learning.The ecosystem surrounding the game
is just as important.In fact, it is probably even more relevant from an education perspective, because it provides players with a
range of opportunities to share, interact and pursue interests and passions.
Game-based Learning: Latest Evidence and Future Directions 10
3. Impact and potential impact of game-
based learning
Key findings
• The literature was split on the extent to which video games can impact upon overall
academic achievement. However, the studies consistently found that video games can
impact positively on problem solving skills and knowledge acquisition.
• The majority of the studies examining the impact of video games on student motivation
and engagement found positive results. However, it was unclear whether this impact
could be sustained over time.
• Few studies explored whether video games can affect attitudes to learning. Of those that
did, including one meta-analysis, a positive relationship was found.
• Despite the presumed link between motivation, attitude to learning and learning
outcomes, the current literature on the subject of video games does not evidence this or
adequately advance our knowledge and understanding of the mechanisms at play.
• While teachers are generally positive about the use of video games in the classroom,
they would require very strong evidence of their impact before they replaced more
traditional learning styles. Until this time, they are more likely to use video games to
supplement existing practice.
• We found no studies into the impact of gamification - i.e. using video-game elements,
rather than video-games - on learning and achievement.
3.1 What does this section cover?
This chapter focuses on the empirical studies included in our review list. It will draw out key
methodological approaches (section 3.3), and go on to examine the type and extent of the
impact of video games on learning (section 3.4) and possible independent variables on the
extent and type of impact (section 3.5). In doing so, we aim to bring readers up to date on
the current landscape and level of evidence available in relation to video games (section 3.6
concludes the chapter).
3.2 What literature did we consider?
Table 4: Extent of literature considered for Chapter Three
High Medium
Qualitative N/A N/A
Quantitative Chuang and Chen, 2009;
Delacruz, 2011; Huizenga
et al., 2009; Miller and
Annetta et al., 2009; Bourgonjon et
al., 2010; Brom et al., 2011;
Fengfeng, 2008a; Kenny and
McDaniel, 2011; Kolovou and Heuvel-
Game-based Learning: Latest Evidence and Future Directions 11
High Medium
Robertson, 2011 Panhuizen, 2010; Liu et al., 2011;
Papastergiou, 2009; Schaaf, 2012;
Spires et al., 2011; Vos et al., 2011;
Ya-Ting, 2012
Mixed methods N/A Fengfeng, 2008b; Kebritchi et al.,
2010
Review and
Meta-analysis
Chiu et al., 2012; Vogel et
al., 2006
Williamson, 2009
As Table 4 suggests, this chapter synthesises 21 studies, mostly consisting of primary
research into game-based learning or gamification in the classroom.
3.3 What does the most up to date review of the evidence
look like?
3.3.1 What was the focus of the studies?
The studies were wide ranging in their focus and aimed to test a variety of hypotheses. Many
of the studies acknowledge limitations in the current evidence base and, in designing their
studies, seek to plug these gaps or test assumptions implicit in the literature and gaming
theory. Most commonly, however, the studies sought to test the impact of gaming on
learning outcomes as measured by academic achievement, cognitive performance,
knowledge gain or skills and performance, such as capacity for problem solving or critical
thinking.
A number of the studies also explored the impact of gaming on motivation and engagement
in classroom activity . A few studies examined the impact of gaming on student attitudes to
learning (in the wider context of school life or in relation to a specific subject).
The studies also sought in many cases to test or discuss the possible impact of a range of
variables that may or may not mediate the impact of gaming in the classroom. Most
commonly, these included student and/or teacher attitudes to gaming, type of game and the
type of learning experience promoted by the game, and gender. Only one study explored the
impact of previous gaming experience (Miller and Robertson, 2011).
It is notable that we found no studies into the impact of gamification - i.e. using video-game
elements, rather than video-games - on learning and achievement. This is a clear gap in the
evidence base.
3.3.2 How were studies designed?
Mathematics was the most common subject domain to feature in primary studies (six
studies: Delacruz, 2011; Fengfeng, 2008a and b; Kebritchi et al., 2010; Kolovou and Heuvel-
Panhuizen, 2010; Miller and Robertson, 2011), whilst two featured science (Annetta et al.,
Game-based Learning: Latest Evidence and Future Directions 12
2009; Spires et al., 2011). The remainder individually focused on computer science
(Huizenga et al., 2009), language (Vos et al., 2011) and civics and society (Ya-Ting, 2012).
Three of the studies did not specify subject domain. One study (Chuang and Chen, 2009)
intentionally avoided curriculum subjects and instead focused on the issue of ‘fire fighting’ so
as to limit the impact of (and possibly avoid the need to control for) prior subject knowledge
or attainment levels on the study.
It was apparent that students of secondary school age were most likely to have been
selected for studies (although some studies did not specify the age range of students
involved). One study focused on first year university students, and only one study featured
pupils of primary school age (Schaaf, 2012).
It is also important to note that not all studies compared video games to traditional
classroom instruction, but instead sought to examine video games in comparison to other
‘alternative learning strategies’ or other ‘media rich learning materials’ (Brom et al., 2011;
Schaaf, 2012), computer based (non-game) learning tools or instruction packages (Chuang
and Chen, 2009; Papastergiou, 2009) or different games played on different platforms (e.g.
in Delacruz, 2011 where a game was played on the Ipod Touch as opposed to a laptop). In
around four studies it was not apparent whether or not a control group had been used
(Bourgonjon et al., 2010; Fengfeng, 2008b; Kolovou and Heuvel-Panhuizen, 2010; Liu et al.,
2011).
From the information available, it was apparent that the majority of studies sought to test the
impact of games whose structure reflected one or more of the main principles of
game-based learning (for example, rules, goals, an element of fantasy, difficulty levels, and
feedback; more information on the ‘mechanisms’ of gameplay-based learning are discussed
in Chapter Two, section 2.4.1). For example, five studies (Chuang and Chen, 2009; Kebritchi
et al., 2010; Liu et al., 2011; Spires et al., 2011; Ya-Ting, 2012) involved games that could
be defined as simulation games, therefore strongly reflecting the notion of fantasy in their
design and could therefore be purported to support constructionist and experiential learning
theories. These games were described variously by authors as ‘supporting a narrative-
centred learning environment’ (Spires et al., 2011), a ‘microworld’ (Liu et al., 2011), or a ‘3D
immersive environment’(Kebritchi et al., 2010). Where named, examples include Tycoon
City: New York and Sim Cities Societies (Ya-Ting, 2012), Crystal Island (built on Valve
Software’s Source engine, the 3D game platform for Half Life 24
Some studies chose games which aimed to isolate one principle of gaming for specific
review. For example Kolovou and Heuvel-Panhuizen (2010) used an online archery game to
explore game-generated feedback to support problem solving processes, whilst Delacruz
) and Fire Department 2:
Fire Captain (developed by ‘Monte Cristo Games’) (Chuang and Chen, 2009).
4 A game engine is a software framework used to design video games. The same engine can be used
to create a limitless number of games. Valve Software developed Half Life 2 in 2004 using the Source
engine and made the engine available to their customers so they could modify the main game or
develop new ones. See http://en.wikipedia.org/wiki/List_of_Source_engine_mods for a list of games
created using this engine.
Game-based Learning: Latest Evidence and Future Directions 13
(2011) used a maths focused laptop game to explore the impact of feedback as a form of
‘formative assessment’.
Some studies, which sought to measure motivation and engagement, selected games which
they considered to be compatible with Keller’s (1987) attention, relevance, confidence, and
satisfaction (ARCS) model of motivation to facilitate children’s learning (Chuang and Chen,
2009; Kebritchi et al., 2010).
One study found that the engaging potential of video games could be slightly problematic,
especially because students may be distracted by the entertainment elements of the game if
these are not effectively meshed with the instructional component (Fengfeng, 2008a). For
instance, a video game to support mathematics must strive to be integral rather than
ancilliary to the specific nature of the subject matter. In such a game, subject-specific tasks
work better when embedded in the fictional context and consistent with other game
mechanics. For instance, players may be required to compute distances to help a likable
game character jump over certain obstacles (e.g. Delacruz, 2011).
There were various common factors across games, but given the varying focus and
conceptual frameworks that grounded studies, it is unsurprising that the games were often
quite different in nature. The level of detail about the games involved in studies was variable;
whilst most focused on the features of the games (such as content and design of the game),
the literature often lacked detail about who developed the game or why and for whom. For
example, it was often difficult to determine if the game had been built with the express
intention of meeting educational outcomes, or whether researchers were appropriating
existing games to test their possible educational impact. It was also difficult in some cases to
understand what hardware or consoles were necessary to facilitate play and whether the
game was multi- or single-player. Overall, therefore, this lack of information and
acknowledgement of game features could be considered to have implications for the overall
strength of evidence around gaming in the classroom.
In order to better illustrate the contrasts apparent in games and the varied aims of studies,
we have included the following three vignettes which detail the variety of games and
methodologies used to explore the impact of video games on the subject domain
mathematics.
Study 1: The effects of modern mathematics computer games on mathematics
achievement and class motivation, (Kebritchi et al., 2010).
In this study the research treatment group used a game called DimensionM™. This game
was adopted in this study because it included a) advanced three-dimensional (3-D) graphics
and interfaces, b) learner centred approach and encouragement of interaction with the
environment, and c) multi-player options. The game teaches algebra by involving players in
completing mathematics related missions within a 3-D immersive environment designed with
advanced graphics. The teachers were provided with material to support lesson delivery
using the game. Quantitative data was collected through student motivation surveys, surveys
to collate student and teacher demographics and through the school-district benchmark
exams. In addition, a series of qualitative interviews were conducted.
Game-based Learning: Latest Evidence and Future Directions 14
Study 2: Educational benefits of using game consoles in a primary classroom: a
randomised controlled trial (Miller and Robertson, 2011).
In this study an experimental class of children used a games console every day for 20
minutes: a Nintendo DS Lite, playing Dr. Kawashima’s Brain Training while a control group
continued with their regular classroom programme. The study was designed to test the
impact of the game on students’ speed and accuracy of mental computation. This was
measured through a 100-item test (designed to reflect the students’ current stage in the
Scottish Curriculum). Questionnaires were used to measure students’ ‘self-perception’ of
their own mathematics ability and attitude to school. Previous experience of computer use at
home was also recorded.
Study 3: Online game-generated feedback as a way to support early algebraic
reasoning (Kolovou and Heuvel-Panhuizen, 2010).
This study tested the impact of an online game on students’ problem solving processes in
relation to number problems. The study featured an online archery game which directly
provided students with the results of their shooting actions. No control group was used.
Instead, students were given a paper and pencil test on algebra – during and after which
students received no feedback. A week later students could log into an online environment
and play the computer game at home (to purposely remove input from teachers). The
computer game provided feedback and monitored performance. After four weeks the paper
and pencil test was administered again.
3.4 The type and extent of impact
3.4.1 To what extent did gaming impact on learning outcomes?
Given the complex nature of assessment and measuring academic achievement, it is
perhaps unsurprising that there was not one, definitive measure against which the studies
sought to explore impact on learner performance. Instead a range of outcomes were
explored across studies. For example, some studies generally sought to measure the extent
of knowledge gain amongst students (Brom et al., 2011; Chuang and Chen, 2009; Huizenga
et al., 2009; Papastergiou, 2009), while others explored the impact of gaming on specific skill
sets such as problem solving or critical thinking (often positing that this was predictive of
student learning, but the extent to which this link was tested was variable) (Kolovou and
Heuvel-Panhuizen, 2010; Liu et al., 2011; Spires et al., 2011; Ya-Ting, 2012). Academic
achievement was often referenced as an outcome but generally without detailing the
measures explored. Although this offers important evidence about the extent of impact, it
makes it somewhat difficult to fully understand the nature of the outcomes. Similarly, the
method of assessment varied widely; some studies adopting standardized tests commonly
used to measure traditional teaching practice while, in other cases, the researcher had
Game-based Learning: Latest Evidence and Future Directions 15
employed tests developed for the purposes of academic study or tests and measures
integrated into the gaming application itself.
Overall, where studies sought to measure ‘academic achievement’, five calculated some
degree of improvement (Chuang and Chen, 2009; Delacruz, 2011; Kebritchi et al., 2010;
Miller and Robertson, 2011). However, four studies found no impact on academic
achievement (Annetta et al., 2009; Fengfeng, 2008b; Spires et al., 2011; Ya-Ting, 2012).
Therefore the literature is split on the extent to which video games can impact on overall
learning outcomes. One explanation for this, and perhaps some of the less positive results
around academic achievement, could be a possible mismatch between traditionally
measured outcomes and those (such as problem solving) facilitated by games. Furthermore,
most studies were not conducted over substantial periods of time, and it could therefore be
argued that learning gains were not advanced enough to be reflected in standardised
achievement tests. Outcome comparisons across studies may also be aided through some
level of agreement and consistency on how academic achievement should be measured in
relation to video games.
Perhaps more promising were the results for problem solving and knowledge acquisition; all
five studies that specifically focused on problem solving skills (Chuang and Chen, 2009;
Kolovou and Heuvel-Panhuizen, 2010; Liu et al., 2011; Spires et al., 2011; Ya-Ting, 2012)
found some degree of improvement. For example, Ya-Ting (2012) credits the simulation
provided by a digital game with providing an ‘authentic and relevant context for problem
solving’ (p. 17). Likewise, all four studies (Brom et al., 2011; Chuang and Chen, 2009;
Huizenga et al., 2009; Papastergiou, 2009) which specifically explored knowledge
acquisition also all found positive results.
Secondary evidence
In addition to the primary evidence considered above, a meta-analysis was also pertinent to
our consideration of the impact of gaming on learning outcomes. (Vogel et al., 2006)
observed significant cognitive gains across studies utilising games versus traditional
teaching methods (although it must be acknowledged that this piece of literature makes no
attempt to define ‘cognitive gains’ or indeed how these have been measured across studies).
3.4.2 To what extent did gaming impact on motivation and
engagement?
Eight of the studies explored the impact of video games on student motivation and
engagement (Annetta et al., 2009; Fengfeng, 2008a; Kebritchi et al., 2010; Liu et al., 2011;
Papastergiou, 2009; Schaaf, 2012; Vos et al., 2011; Ya-Ting, 2012). This factor is identified
as a key principle defining gaming for learning in Chapter Two. Studies often measured this
factor through student surveys which collected self-efficacy measures or students’ perceived
level of focus on the activity, while some also incorporated student observations or
measured ‘time on task’ as an indicator of engagement.
It is important to note that, again, the measures used across studies to explore this outcome
varied widely. The results were generally positive, with six of the studies finding that video
games increased student motivation or engagement (Annetta et al., 2009; Fengfeng, 2008a;
Game-based Learning: Latest Evidence and Future Directions 16
Papastergiou, 2009; Liu et al., 2011; Schaaf, 2012; Ya-Ting, 2012). The two that did not find
this relationship included important caveats; in one (Kebritchi et al., 2010) the authors, with
hindsight, felt that the test did not adequately match the design of the experiment; while in a
the second study (Vos et al., 2011) the authors had compared the impact of gameplay and
game construction, finding that the latter did indeed have an impact on motivation and
engagement, while the former actually resulted in decreased motivation. This was the only
study in our review that compared ‘playing games’ with ‘making games’. Its findings point to
interesting implications which are, however, beyond the scope of the review.
A study by Huizenga et al. (2009) experienced technical issues with the game used by the
students. They report that this issue had an impact on some students’ engagement with the
game, but not all. However, this does highlight the need for games to be technically sound
before they are introduced into the classroom. Further, some students in this study were
found to be more engaged in the game than others, with a number of students becoming
distracted by other websites.
In exploring the reasons for game-based learning having an impact on student motivation,
Ya-Ting (2012) suggests that by immediately providing students with praise, encouragement
and reinforcement, gaming software helps students develop confidence and motivation to
continue with the task. The author also perceived the game to have elicited learner curiosity.
In one of the studies (Papastergiou, 2009) a student is quoted as saying, ‘it [the game] is
more enjoyable and active. You never get bored as in traditional teaching because you
concentrate on a goal. This helps you to retain elements in your memory easily and
understand concepts that are difficult in order to advance in the game. (p.10). This
perception was also reflected in Kebritchi et al. (2010), where teacher interviews revealed
that ‘when students play the game, they want to learn more and pay more attention because
they liked to pass the game missions. (p. 435). The same study reported that teachers
perceived the game to have ‘changed students’ state of mind about mathematics. The
students became aware of the relationship between mathematics and real life and their
mathematics phobia was diminished.’ (p. 435). Furthermore, the students themselves
reported that the games ‘took them out of class, changed their mood, and were entertaining’
and that they enjoyed the aspects of the game which included adventure, exploration and
challenge. Interestingly, this was one of the few studies which explored the views of students
through qualitative study, as most studies were quantitative in nature. However, this data
sheds important light on some of the features that students found most engaging.
In another study, (Liu et al., 2011), the authors explain that amongst students who learnt
through video games, appeared to be a link between the observed increase in both the level
of challenge posed by the task, and the level of skill and effort required to complete it. The
authors have surmised therefore that a ‘flow state’ was reached amongst some students5
5The ‘flow’ experience refers to an intense, sustained and focused engagement in an activity that
leads to rewarding feelings of outstanding productivity. The concept was first introduced in the context
of ‘positive psychology’ (Csíkszentmihályi,1996).
.
This is believed to support student engagement in learning tasks. However, despite the
assumption implicit in much of the literature that motivation and engagement created by
Game-based Learning: Latest Evidence and Future Directions 17
gaming directly impacts on learning outcomes, this was one of very few studies that actually
examined both factors. Only this and two other studies (Papastergiou, 2009; Ya-Ting, 2012)
reported both an overall improvement in motivation or engagement and skills acquisition
(such as problem solving), while two other studies that measured improved motivation or
engagement found no overall improvement in academic achievement measures (Annetta et
al., 2009; Fengfeng, 2008b). This could add further weight to the suggestion that academic
achievement measures do not adequately measure the learning outcomes supported by
gaming and that the intervention may need to run for longer lengths of time in order to
impact on overall academic outcomes. However, further study of the motivation and learning
outcomes is necessary to make more solid conclusions and to better understand the
dynamic between both factors.
Despite the positive findings regarding motivation and engagement, however, the concern
that motivation associated with digital gaming may be short lived and due to a ‘novelty effect’
also remains largely untested. Although one study (Ya-Ting, 2012) asserts that motivation
was found to have been sustained, this outcome was measured over the course of a
semester, and it is questionable whether this is a long enough period to be confident of
continued impact.
3.4.3 To what extent did gaming impact on students’ attitude
to learning?
Relatively few studies explored this possible outcome. For example, Fengfeng (2008a and ;
b) found that games promoted a more positive attitude to mathematics learning. Further,
Fengfeng (2008b) also found that a cooperative structure (where learners worked together in
the gamespace) was better than an individualistic or competitive structure at facilitating
attitudes towards mathematics learning. Miller and Robertson (2011) took a slightly different
focus and explored mathematics or academic self-concept, defined as the set of beliefs an
individual holds about themselves as a mathematician, finding no improvement.
Secondary evidence
However, Vogel et al’s (2006) meta-analysis found that significantly better attitudes towards
learning were yielded for subjects utilising interactive games or simulations compared to
those using traditional methods for instruction. This meta-analysis is also widely cited
because it observed higher cognitive gains in the interactive games scenario. Given this
finding was based on a number of studies, the authors conclude that this makes the finding
unlikely to be due to chance. It also suggests a possible correlation between both attitudes
to learning and learning outcomes. We should, however, reiterate that Vogel et al. make no
attempt to define ‘cognitive gains’ or how this measure has been measured across the
studies they included in their meta-analysis.
3.5 The role of independent variables
It appears possible that the impact of gaming can be mediated by the prior experience or
skill levels of students, student and teacher attitude to gaming in the classroom and the type
of game or learning experience promoted by the game. A few studies explored student
Game-based Learning: Latest Evidence and Future Directions 18
and/or teacher attitudes to gaming. In exploring student ‘acceptance’ of videogames for
learning through a survey (although not the possible relationship between this and learning
outcomes), Bourgonjon et al. (2010) reported that students’ preference for using video
games in the classroom was affected by their perception of: how useful the game would be
in relation to learning outcomes; its ease of use; the opportunities it presents for learning;
and their own personal experience of gaming outside of the classroom. Therefore, the
authors argue that it cannot be assumed that outcomes for students will be consistent, given
that they may commence a game-based lesson with very difference attitudes and starting
points. Further to this, Williamson (2009) found that not all students automatically see the
social and developmental value of the skills associated with computer games and this may
subsequently impact on the extent to which they are willing to engage with the game.
Although Bourgonjon et al. (2010) found that male students have a more positive attitude to
gaming than female students, only one study found a more positive effect of video games on
outcomes for males (Brom et al., 2011), while the remaining four studies which explored
impact by gender (Fengfeng, 2008b; Miller and Robertson, 2011; Papastergiou, 2009; Vos et
al., 2011) found no difference in outcomes for males and females.
Secondary evidence
The meta-analysis carried out by Vogel et al. (2006) also found no significant difference
between genders, suggesting that the effects of games and interactive simulations sustained
across male and female participants, but also across situations (in terms of learner control,
level of realism, and individual/group usage).
In terms of teacher attitudes to gaming, Kenny and McDaniel (2011) found that there is a
relationship between what teachers believe and their behavioural patterns in the classroom.
The study argues that teachers can be reluctant to spend efforts learning the gaming
medium when they are not convinced of its impact. The authors believe that a more
convincing argument needs to be constructed before teachers will change their views
towards video games and integrate them into existing classroom practice. An exploratory
study by Williamson (2009), found that 35 per cent of teachers (in English state primary and
secondary schools) had already used computer games in their teaching and 60 per cent
would consider using games in the future. The most commonly cited reason was to enhance
motivation and engagement. The majority of teachers also believed that gaming could help
support children’s cognitive development, their ICT development, and their higher-order
thinking skills. However, the results also suggested that teachers are more interested in the
‘pragmatic’ benefits of gaming, for example, to assist teachers in relating formal schooling to
everyday lives and creating social interaction in the classroom – rather than for pursuing
educational ‘big ideas6
There was also some evidence that the type of game can have an impact on outcomes.
Most commonly, computer simulation games were compared to simpler drill and practice
.’
6The author cites these ‘big ideas’ as being: games as a persuasive medium that can affect young
people’s thinking; games as a constructionist technology; games as providing environments for
authentic activity; and games in the context of ‘media literacy’.
Game-based Learning: Latest Evidence and Future Directions 19
games. Fengfeng (2008b) found, for instance, that games are more motivating and engaging
than drills, albeit not more effective.
Secondary evidence
A meta-analysis by Chiu et al. (2012) found that the type of game used has an impact on
learning as far as language learning is considered. Meaningful and engaging games, in
which learners are given opportunities to explore, interact and engage with a complex game
world, are more effective than simpler games that rely on drill and practice. The meta-
analysis by Vogel et al. (2006) found no alteration in results when controlling for the level of
‘realism’ in the presentation of the game.
As already mentioned, Fengfeng (2008b) found that regardless of application (gaming or
drills), individualistic goal structures facilitate mathematics performance more than other goal
structures (competitive and cooperative), whereas a cooperative structure was better than
the other two at facilitating more positive attitudes towards mathematics learning. Similarly,
Ya-Ting (2012) felt that computer games often emphasise collaborative and active learning,
as their design is largely based on social constructivist principles.
As stated, studies were focused on individual subject domains and so it would be
impossible to determine the extent to which the outcomes could be replicated in different
subject domains without understanding the mechanisms behind the impact. There is little
explanation in the studies as to why one subject domain has been selected for study over
any other. One could assume that researchers have most commonly selected maths as the
subject domain because, arguably, it is more straightforward to collect hard, measurable
data about achievement for this subject when compared to, for example, humanities
subjects. Alternatively, games developers or researchers may consider gaming in the
classroom to better lend itself to certain subjects. Whatever the reason, it will be important
that future studies take this area of inquiry a step further to explore what replicable factors, if
any, exist across subject domains.
Few studies examined the possible mediating role of previous gaming experience on
impacts in the classroom. Only two formally explored this variable (Kebritchi et al. 2010;
Miller and Robertson, 2011) and found no difference in outcomes dependent on prior level of
computer skill. This may seem surprising as, with anything, we might expect to see some
advantage of having prior experience and this may be an area for further exploration in the
future.
3.6 The new landscape
Overall, the results of the studies included in this review appear to demonstrate a positive
relationship between gaming in the classroom, learning outcomes and motivation and
engagement. However, the overall strength of the evidence has been affected by the
research design or lack of information about the research design. For example, the origin
and design of the game was not always clear and it was sometimes difficult to know whether
or not a control group had been used. There was also some detail lacking overall about how
various outcomes or variables had been measured. The findings around motivation and
engagement are further called into question when we consider that only one study (Ya-Ting,
Game-based Learning: Latest Evidence and Future Directions 20
2012) sought to explicitly test the ‘sustainability’ of outcomes. Because the majority of
studies were based on a small sample from one school or age group, there was also little
scope for testing how replicable the outcomes would be in different contexts.
A number of the studies tested elements of learning theory or the impact of independent
variables in isolation, which provides some interesting evidence of correlating factors and a
basis for further exploration. However, the studies do not give much clarity as to whether or
not it is the technology on which the game is played that adds value, or if the same results
might be achieved through a lesson which is ‘gamified’ using more traditional resources
(such as pen and paper). It may be argued that authors would have better plugged the gaps
in the pre-existing literature had they further examined the mechanisms at play supporting
the positive relationship that so many of the studies found.
Game-based Learning: Latest Evidence and Future Directions 21
4. Implications for future research
Key points
• It is important we develop a more analytic approach that considers how the different
elements that operate within video games impact in an educational setting.
• Opening up the ‘black box’ of video games would enable us to focus on specific
principles or mechanisms. This finer grained approach could unlock a more rewarding
research agenda.
• Three research challenges have been identified:
1. Working towards a consensus about the relationship between academic
achievement and game-based learning. Efforts are needed to articulate clear
relations between game elements and a range of outcomes, from a broad level (e.g.
platform on which the game runs, single player, multiplayer, and so forth), gradually
narrowing down to specific gameplay mechanics.
2. Unpacking further the relationship between gaming and academic achievement in
the context of educational assessment. Games seem to allow more powerful and
‘always-on’ forms of assessment in which all actions, interactions, successes and
failures can be constantly tracked and logged. This raises a number of empirical and
ethical issues.
3. Developing research into the potential of video games that accounts for the realities
of schools. In particular, more research is needed into the social, cultural and
economic factors that influence attitudes towards the use of game-based learning in
our increasingly diverse and multicultural schools
4.1 What does this section cover?
This chapter is a more ‘future facing’ section that looks at the patterns identified thus far and
identifies research areas to tackle. The chapter summarises the issues noted in the review of
empirical studies (section 4.3), and uses it as background to define three research
challenges to inform future research (sections 4.3.1; 4.3.2; 4.3.3).
4.2 What literature did we consider?
We looked across all the literature included in our review. This section will try to interrogate
the literature more thoughtfully, in order to hone in on areas that warrant further exploration.
Firstly we identified the issues with the evidence base, both in terms of the quality of the
evidence and gaps in its coverage.
4.3 Limitations of the reviewed evidence
• A lack of clarity about the types of games investigated, in terms of design features,
scope and purposes. A game expressly built for education is likely to have some
distinctive features that sets it apart from games created for entertainment purposes.
Other aspects liable to influence the research process are the platform needed to run the
video game for example a console or a portable device?, and the specific nature of
gameplay, such as single player or multiplayer.
Game-based Learning: Latest Evidence and Future Directions 22
• The limited timescale of most studies, which might explain some of the uneven results
obtained. We cannot rule out the possibility that changes in performance may require a
longer and more sustained engagement with video games, or game-based design
elements, than what we observed in most studies.
• One of the key assumptions we noted in the more ‘speculative’ literature in Chapter Two
was concerned with the relationship between motivation, attitudes and learning
outcomes. In this respect, the empirical studies provided some promising findings, but
nothing conclusive. The main issue, which mirrors the timescale problem noted above, is
that we cannot rule out the possibility that increases in motivation and engagement, with
a knock-on effect on outcomes, are caused by a ‘novelty effect’.
• A lack of insight about the role of individual and social characteristics. While gender was
sometimes considered, and found of little significance, several equally important aspects
were missing or only fleetingly accounted for. These include personality profiles,
cognitive styles and previous gaming experience.
• A lack of insights about the role of subject domains other than mathematics. As noted in
Chapter Three, mathematics was often the subject of choice because it lends itself to
easy measurement and comparisons. We do not know whether different subject
domains may require different forms of ‘gameplay’, thus leading to different types of
learning outcomes.
• As suggested in Chapter Two, the ‘gamification’ trend is mostly concerned with the use
of elements and mechanics derived from video games, rather then with actual games.
However, we noted in the literature a complete lack of evidence - and a more general
lack of clarity - about the impact of gamification on learning outcomes and relations
between technology and video game design elements. Can such elements be detached
from the technology without losing their meaning or potency? Can they be used
effectively to add value to traditional teaching and learning?
Rather than addressing each evidence gap or issue individually, we used the whole list as
background. We then extrapolated three specific research challenges that might inform
future research efforts in game-based learning. Exploring these challenges will contribute to
a more realistic, evidence-based and ultimately more effective, integration of game-based
learning in education. This will be beneficial to all those interested in unlocking the
educational potential of video-games: teachers, learners and the video-games industry.
4.3.1 Research challenge 1: working towards a consensus
about the relationship between academic achievement and
video games
It appears that empirical research may have to move beyond the search for linear
relationships between video games and academic achievement. Instead, opening up the
black box of video games would enable us to focus on specific principles or mechanisms
that operate within them. This finer grained approach could unlock a more rewarding
research agenda.
Game-based Learning: Latest Evidence and Future Directions 23
The long term objective of this agenda would be to create a taxonomy of game-based
learning; an evolving and collaborative effort that will clarify relations between game
elements and a range of outcomes. This taxonomy would begin from a broad level (e.g.
equipment required to run the game, single player, multiplayer, and so forth), gradually
narrowing down to specific gameplay mechanics.
A research question might be: what outcome measures or criteria best reflect (or correlate
with) proficiency in the ‘gameplay’ that underpins simulation games like SimCity, in which
goal-oriented actions lead to expected and unexpected consequences.
To an extent, this is what Gentile et al. (2009) did in order to study the relationship between
video games and human behaviour; although they were focused on aggression and
prosocial tendencies and not on learning. Gentile and his co-authors provided robust
evidence that exposure to specific forms of content in video games has short-term, as well
as long-term influences. They looked at the effects of video game content under very
specific circumstances, clearly distinguishing between violent content (shooting, killing, etc.)
and pro-social content in which ‘players and game characters help and support each other in
nonviolent ways’ (p. 752). As such, the study’s results point to a relationship between
specific actions performed during gameplay and a range of psychological constructs
(‘priming scripts’) that regulate attitudes and behaviours.
Research on the educational potential of video games could indeed learn from these efforts,
and begin to analyse specific gameplay actions: what people actually do when they play
video games. An important distinction will be between in-game actions, and actions
performed in the social contexts that surround gameplay. Through a mix of quantitative and
qualitative methods, an observer would focus on the relationships between content and
design elements embedded during the development process - the ‘intended’ game – and the
actual behaviours and interactions that unfold at specific moments and over time. The aim
would be to explore whether those actions may be related to a range of benefits or
outcomes. Some of these benefits and outcomes may easily overlap with official measures
of formal schooling, while others will require additional efforts to be defined.
4.3.2 Research challenge 2: unpacking further the relationship
between video games and academic achievement in the
context of educational assessment
This research challenge follows on directly from the previous one, but is directly concerned
with the complex issue of educational assessment.
Video games seem to allow more powerful and ‘always-on’ forms of assessment in which all
actions, interactions, successes and failures can be constantly tracked and logged. This
generates large amounts of data that, if correctly analysed, would allow educators to make
important inferences about learners’ capabilities, needs and potentials.
Game-based Learning: Latest Evidence and Future Directions 24
On the one hand, this is part of a ‘data trend’ in educational assessment. There is, in fact, a
growing interest in the use of automated techniques to make sense of large datasets about
different aspects of measurable performance. It is also safe to assume that students may
benefit from a more adaptive and fast remedial action, which can be facilitated by the
computer-assisted analysis of achievement and behavioural data.
Game-based learning offers the possibility of moving beyond traditional forms of standards-
based assessment, to consider alternative dimensions of performance and achievement. For
example, the nature of gameplay described in Chapter Two, section 2.4 highlights an
important aspect: the priority of process over content. In most video games mastery is not
achieved through knowledge of textbooks or manuals. Instead, what counts is the increasing
familiarity with a ‘proceduralised’ system of choices and consequences. Some authors argue
that focusing on how learners negotiate such choices could lead to fairer and more accurate
forms of assessment (Schwartz and Arena, 2013). These authors suggest that the game-
based assessment of choices would allow educators to evaluate more effectively learning
strategies and identify ‘malpractices’. For instance, when a student repeatedly makes the
wrong choice when defining a sequence of actions to solve a problem.
Although these emerging research areas are promising, there are issues and risks as well.
To begin with, powerful analyses of gameplay data are only applicable in digital settings.
There are undoubtedly implications when important inferences are based on actions
performed in fictional, digitalised contexts - not least the ethical ramifications of using forms
of ‘stealth’ assessment of which learners are largely unaware. A digital-only approach is also
incompatible with the idea of using game-derived elements, instead of actual games (i.e.
gamification), in the context of otherwise ‘traditional’ teaching.
Moreover, if the ‘choices’ that learners make in games is used to formally assess them in
schools, then we must consider whether they may be used to create quantitative social
indicators for social decision making (for example, access to higher education). This raises a
whole range of additional ethical issues, such as:
• types of choices may be tacitly imposed on learners as the ‘right’ ones;
• once social and educational value is attached to ‘choices’, they are no longer the by-
product of natural behaviour. We cannot rule out the emergence of more sophisticated
and hard-to-track forms of cheating and ‘gaming the system’. For example, people could
learn how to display artificial choice patterns, whilst teachers could, paradoxically, ‘teach
to the choice’.
4.3.3 Research challenge 3: developing research into the
potential of video games that accounts for the realities of
schools
We noticed in Chapter Three that research endeavours might be undermined at the outset
by the incompatibility between video games for learning and formal instructional practices.
The reviewed evidence responds to this incompatibility in different ways. Some argue that
the key issue is not ‘whether or not to use computer games’, but ‘how to better design an
educational computer game’, or ‘how to better apply game-based classroom instructional
Game-based Learning: Latest Evidence and Future Directions 25
strategies’ (Fengfeng, 2008b, p. 554). Others see the gap between school and game-based
learning contexts as hard to bridge. For instance, Gee (2007) suggests that enthusiastic
gameplayers share norms, values and beliefs about what counts as worthwhile knowledge,
what is good and what is not in terms of performance. This is generally in contrast with the
norm of schoolwork and classroom values, which makes it impossible for teachers to fully
appreciate the potential of gaming.
It could be argued that this tension should be acknowledged and explored, without falling
into the ‘trap’ of portraying schools as being too conservative. For instance, for this review
we also considered literature critical of the enthusiasm with which video games are
described in many educational technology circles. In particular, Buckingham (2007) argues
that bringing leisure time experiences and practices into the formal educational domain may
not always be possible or even desirable. He reasons: ‘we need to be wary of simply
celebrating children’s informal experiences of media and technology […] there are good
reasons to be cautious about the idea of simply extending those experiences into the more
‘formal’ context of the school’ (p. 101). Indeed, Buckingham suggests that an ‘anti-school’
rhetoric seems to underlie many accounts of game-based learning. Instead, he articulates a
defence of the distinctive role of schools, where important forms of learning take place that
cannot be replicated in interest-driven, self-selecting contexts.
While it is essential to keep in mind these more critical viewpoints, they should not
discourage educators and researchers from exploring ways in which video games can be
beneficial to young people. In this respect, a few authors are seeking to address this
contradiction without renouncing critical reflection. For instance, Young et al. (2012) are
aware of the hype that surrounds game-based learning and gamification, but also invite
more complex research questions that move beyond linear relationships between
achievement and video games use.
As noted earlier on, evidence tentatively suggests that video games are motivating and
support a more positive attitude to learning and school. However, these findings are not
conclusive and require, most of all, a more sustained and longitudinal research effort. From
a school-based perspective, this means studying the long-term motivational impact of
gameplay-based approaches, when used regularly within specific curricula. School curricula
are extremely diverse, with degrees of prescription and guidance changing over time and
across countries - sometimes across individual schools. However, they all share the
fundamentally correct assumption that meaningful learning takes place over a reasonable
amount of time, generally a full academic year. As such, school curricula underpin school life
and shape routines, times and priorities. Any research effort that ignores this framework is
bound to be marginal, short-term and surrounded by an aura of novelty.
Acknowledging the specificity of formal learning settings also means accepting that not all
students would choose to play games in their spare time, let alone at school. Enthusiastic
accounts based on informal, largely self-selecting samples may in fact feed an incorrect
perception that all young people engage with video games in the same, positive way. Some
of the studies we reviewed suggest that not all students (and teachers) display positive
attitudes and perceptions in relation to video games in education (Bourgonjon et al., 2010;
Kenny and McDaniel, 2011; Williamson, 2009). While resistance to using video games in the
Game-based Learning: Latest Evidence and Future Directions 26
classroom could be a threat to the uptake of game-based learning, it cannot be dismissed
simply as a barrier to overcome.
More research is needed into the social, cultural and economic factors that influence
attitudes towards the use of video games and ‘gameplay’ in our increasingly diverse and
multicultural schools. A possible hypothesis to guide future research is that such factors
make some individuals and groups better positioned than others to reap the benefits of
technological innovations. Therefore, attention might need to shift from the search for effects
of video games on learning, to how the social and economic contexts in which people live
can support or undermine a positive use of technological innovations for personal or social
development.
Game-based Learning: Latest Evidence and Future Directions 27
5. Implications for teachers and schools
5.1 What can teachers safely ‘take home’?
This section lists the main ‘messages from the review for ‘teachers and other educators.
• The evidence suggests that game-based learning can improve engagement and
motivation, but don’t rely on games to improve attainment - there is still a lot we don’t
know about the impact of video games on learning.
• The best way of integrating gaming into teaching is by using it within a clear pedagogic
process. In particular:
- Place learning activities and academic content within the video game’s fictional and
entertainment context, maintaining a balance between ‘fun’ and ‘learning’.
- Make the academic content integral to the game rather than an add-on. Content-
specific tasks work better when embedded in the fictional context and rules
(‘mechanics’) of the game. For example, in a maths game, asking learners to
compute distances to help a likable game character jump over obstacles will be more
engaging than asking them to complete traditional maths tests in order to make a
story advance.
- Carefully plan the roles that you and your learners will take on in the game. Teachers
should play roles that allow them to mediate the experience for learners: providing
guidance when needed; ensuring that rules are followed; and maintaining a
respectful atmosphere.
• Don’t try to divorce decontextualized components of a game (such as badges, scores or
leaderboards) from the fictional context and rules of the game (the ‘mechanics’). Using
badges and medals can work for certain simple tasks, but actual game-based learning
will require using those techniques in the context of rule-sets and role-playing.
5.2 Additional points for senior leaders who wish to
support the use of game-based learning in their schools
• Ensure that teachers are not left on their own when trying to enable game-based
learning. Beyond the video game itself, teachers should have the time and the resources
for offline activities to support learning. These include time to organise collaborative
tasks, and the ability and the skills to provide timely guidance while students play the
game.
• If you are trying to bring game-based learning into your school, ‘buy in’ from teachers is
needed to ensure that video games are fully integrated. To achieve integration, you’ll
probably need to invest in in-depth and sustained professional development.
• Acknowledge and, if possible, address the barriers that may stop your teachers from
engaging with game-based learning. These include lack of preparation time, poor
technical support, outdated technologies and lack of opportunities for collaboration due
to the rigid structure and time constraints of formal instruction.
Game-based Learning: Latest Evidence and Future Directions 28
References
Annetta, L. A., Minogue, J., Holmes, S. Y., and Cheng, M.T. (2009). ‘Investigating the impact
of video games on high school students’ engagement and learning about genetics’,
Computers and Education, 53, 1, 74–85.
Bogost, I. (2011). How to Do Things with Video Games. Minneapolis, MN: University of
Minnesota Press.
Bourgonjon, J., Valcke, M., Soetaert, R., and Schellens, T. (2010). ‘Students’ perceptions
about the use of video games in the classroom’, Computers and Education, 54, 4, 1145–
1156.
Brom, C., Preuss, M., and Klement, D. (2011). ‘Are educational computer micro-games
engaging and effective for knowledge acquisition at high-schools? A quasi-experimental
study’, Computers and Education, 57, 3, 1971–1988.
Buckingham, D. (2007). ‘Playing to learn? Rethinking the educational potential of computer
games.’ In: Buckingham, D. Beyond Technology: Children’s Learning in the Age of Digital
Culture. Cambridge: Polity Press.
Chiu, Y.H., Kao, C.W. and Reynolds, B.L. (2012). ‘The relative effectiveness of digital
game‐based learning types in English as a foreign language setting: A meta‐analysis’, British
Journal of Educational Technology, 43, 4, 104–107.
Chuang, T.Y. and Chen W.F. (2009). ‘Effect of computer-based video games on children: an
experimental study’, Educational Technology and Society, 12, 2, 1–10.
Csíkszentmihályi, M. (1996), Creativity: Flow and the Psychology of Discovery and
Invention, New York, NY: Harper Perennial.
De Freitas, S. and Griffiths, M. (2008). ‘The convergence of gaming practices with other
media forms: what potential for learning? A review of the literature’, Learning, Media and
Technology, 33, 1, 11–20.
Delacruz, G.C. (2011). Games as Formative Assessment Environments: Examining the
Impact of Explanations of Scoring and Incentives on Math Learning, Game Performance,
and Help Seeking (CRESST Report 796). Los Angeles, CA: University of California, National
Center for Research on Evaluation, Standards, and Student Testing.
Fengfeng, K. (2008a). ‘A case study of computer gaming for math: engaged learning from
gameplay?’ Computers and Education, 51, 4, 1609–1620.
Fengfeng, K. (2008b). ‘Computer games application within alternative classroom goal
structures: cognitive, metacognitive, and affective evaluation’, Educational Technology
Research and Development, 56, 5–6, 539–556.
Fling, S., Smith, L., Rodriguez, T., Thornton, D., Atkins, E. and Nixon, K. (1992).
‘Videogames, aggression and self-esteem’, Social Behavior and Personality: an
International Journal, 20, 1, 39–45.
Gee, J.P. (2007). What Video Games Have to Teach Us About Learning and Literacy. New
York, NY: Palgrave Macmillan.
Game-based Learning: Latest Evidence and Future Directions 29
Gee, J.P. (2008). ‘Learning and games.’ In: Salen, K. (Ed) The Ecology of Games:
Connecting Youth, Games, and Learning (The John D. and Catherine T. MacArthur
Foundation Series on Digital Media and Learning). Cambridge, MA: The MIT Press.
Gentile, D.A., Anderson, C., Yukawa, S., Ihori, N., Saleem, M., Ming, L.K. and Sakamoto, A.
(2009). ‘The effects of prosocial video games on prosocialbehaviors: International evidence
from correlational, experimental, and longitudinal studies’, Personality and Social
Psychology Bulletin, 35, 752–763.
Huizenga, J., Admiraal, W., Akkerman, S. and Dam, G.T. (2009). ‘Mobile game‐based
learning in secondary education: engagement, motivation and learning in a mobile city
game’, Journal of Computer Assisted Learning, 25, 4, 332–344
Kapp, K.M. (2012). The Gamification of Learning and Instruction: Game-based Methods and
Strategies for Training and Education, San Francisco, CA: Pfeiffer.
Kebritchi, M., Hirumi, A. and Bai, H. (2010). ‘The effects of modern mathematics computer
games on mathematics achievement and class motivation’, Computers and Education, 55, 2,
427–443.
Keller, J.M. (1987). ‘Development and use of the ARCS model of motivational design’,
Journal of Instructional Development, 10, 3, 2–10.
Kenny, R.F.K. and McDaniel, R. (2011). ‘The role teachers’ expectations and value
assessments of video games play in their adopting and integrating them into their
classrooms’, British Journal of Educational Technology, 42, 2, 197–213.
Kolovou, A. and Heuvel-Panhuizen, M.V.D. (2010). ‘Online game-generated feedback as a
way to support early algebraic reasoning’, International Journal of Continuing Engineering
Education and Life Long Learning, 20, 2, 224–238.
Liu, C.C., Cheng, Y.B. and Huang, C.W. (2011). ‘The effect of simulation games on the
learning of computational problem solving’, Computers and Education, 57, 3, 1907–1918.
Marković, F., Petrovic, O., Kittl, C. and Edegger, B. (2007). ‘Pervasive learning games: A
comparative study’, New Review of Hypermedia and Multimedia, 13, 2, 93–116.
McGonigal, J. (2011). Why Games Make Us Better and How They Can Change the World.
London: Jonathan Cape.
Miller, D.J. and Robertson, D.P. (2011). ‘Educational benefits of using game consoles in a
primary classroom: a randomised controlled trial’, British Journal of Educational Technology,
42, 5, 850–864.
Papastergiou, M. (2009). ‘Digital game-based Learning in high school computer science
education: impact on educational effectiveness and student motivation’, Computers and
Education, 52, 1, 1–12.
Pearce, C. (2002). ‘Sims, battleBots, cellular automata god and go: a conversation with Will
Wright’, Game Studies, 2, 1 [online]. Available: http://www.gamestudies.org/0102/pearce/ [4
March, 2013].
Game-based Learning: Latest Evidence and Future Directions 30
Salen K. (Ed) (2008). The Ecology of Games, Connecting Youth, Games, and Learning (The
John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning).
Cambridge, MA: The MIT Press.
Salen, K. (2008). ‘Towards an ecology of gaming.’ In: Salen, K. (Ed) The Ecology of Games:
Connecting Youth, Games, and Learning (The John D. and Catherine T. MacArthur
Foundation Series on Digital Media and Learning). Cambridge, MA: The MIT Press.
Schaaf, R. (2012). ‘Does digital game-based learning improve student time-on-task behavior
and engagement in comparison to alternative instructional strategies?’ Canadian Journal of
Action Research, 13, 1, 50–64.
Schwartz, D.L. and Arena, D. (2013). Measuring What Matters Most. Choice-Based
Assessments for the Digital Age. Massachussets, MA: MIT Press.
Shaffer, D.W. (2008). How Computer Games Help Children Learn. New York, NY: Palgrave
Macmillan.
Spires, H.A., Rowe, J.P., Mott, B.W., and Lester, J.C. (2011). ‘Problem solving and game-
based learning: effects of middle grade students' hypothesis testing strategies on learning
outcomes’, Journal of Educational Computing Research, 44, 4, 453–472.
Vogel, J.J., Vogel, D.S., Cannon-Bowers, J., Bowers, C.A., Muse, K. and Wright, M. (2006).
‘Computer gaming and interactive simulations for learning: a meta-analysis’, Journal of
Educational Computing Research, 34, 3, 229–243.
Vos, N., van der Meijden, H. and Denessen, E. (2011). ‘Effects of constructing versus
playing an educational game on student motivation and deep learning strategy use’,
Computers and Education, 56, 1, 127–137.
Williamson, B. (2009) Computer Games, Schools and Young People: a Report for Educators
on Using Games for Learning. Bristol: Futurelab [online]. Available:
http://archive.futurelab.org.uk/resources/documents/project_reports/becta/Games_and_Lear
ning_educators_report.pdf [28 February, 2013].
Ya-Ting, C.Y. (2012). ‘Building virtual cities, inspiring intelligent citizens: digital games for
developing students’ problem solving and learning motivation’, Computers and Education,
59, 2, 365–377.
Young, M., Slota, S., Cutter, A.B. Jalette, G., Mullin, G., Lai, B., Simeoni, Z., Tran, M. and
Yukhymenko, M. (2012). ‘Our princess is in another castle : a review of trends in serious
gaming for education’, Review of Educational Research, 82, 1, 61–89.
Game-based Learning: Latest Evidence and Future Directions 31
Appendix 1:
Search strategy and the review process
This appendix provides information on the:
• review parameters
• search strategies used to identify the literature that the review team considered for
inclusion in the review
• review process that the team used to select the literature included in the review.
Search parameters
Publication date:
Work published from the year 2006
Geographical scope:
United Kingdom and international
Language:
Published in English
Study type: Empirical research and/or evaluation; good practice examples;
theoretical items; published literature (peer and non-peer
reviewed)
Inclusion criteria
• Conforms to search parameters
• Pertinence / relevance (to research questions)
• Research quality / use of ‘best evidence’ approach (i.e. reliance on the best evidence
available to answer any of the key research questions).
In addition, we applied the following quality criteria:
• High: large scale quantitative study, meta-analyses or in-depth qualitative case studies
covering a range of settings and stakeholders where views are triangulated; systematic
reviews have also been included in this category .
• Medium: quantitative or qualitative studies with smaller samples; qualitative studies not
covering a range of settings or stakeholders; non-systematic reviews. We also included
in this category more speculative contributions that are based on existing theories and
indirect evidence.
Search strategy
This search strategy sets out the keywords used (and also their combination) with each of
the bibliographic databases. All searches were limited to publication years 2006-2012
Game-based Learning: Latest Evidence and Future Directions 32
Databases
The following databases were searched.
Source
Items selected
for
consideration
Australian Education Index
(BEI)
133
British Education Index (BEI)
88
Education Resources
Information Center (ERIC) 264
Australian Education Index (AEI)
AEI is Australia’s largest source of education information covering reports, books, journal
articles, online resources, conference papers and book chapters.
British Education Index (BEI)
BEI provides information on research, policy and practice in education and training in the
UK. Sources include over 300 journals, mostly published in the UK, plus other material
including reports, series and conference papers.
Education Resources Information Center (ERIC)
The ERIC database is sponsored by the US Department of Education to provide extensive
access to education-related literature.
These sources were queried using key words (used individually or in various combinations),
which included:
Video Games, Gamification ,Gaming, Game design, Game mechanics, Game theory ,Game-
based learning, Gameplay, Educational games, Game-based technologies, Game elements,
Learning, Assessment, Best practice, Educational outcomes, Attainment, Outcomes of
education, Curriculum, Engagement, Motivation, Feedback, Engage, Collaborate, Role-
playing games, Online Multiplayer, Console, PC, Electronic games, Achievements.
Websites
A range of websites were also searched and included the following
List details
Number of
sources
London Knowledge lab
0
OU-CRC
1
OECD Centre for Educational Research and
Innovation (CERI)
0
Futurelab
0 (duplicates
included in
databases
search results)
Becta
0
Game-based Learning: Latest Evidence and Future Directions 33
List details
Number of
sources
Department for Education
0
Ofsted
0
Institute of Education
2
EPPI Centre
0
Digital Cultures Research Centre
0
Gamification.co
0
Partnership for 21st Century Skills
0
American Educational Research Association
0
Gamification Research Network
4
MacArthur Foundation
0 (duplicates
included in
databases
searches)
ACM digital library
8
Institute of Play
37
Google scholar
17
Review process
We used a four-stage process to filter the search results, so that only the most relevant and
best quality studies available were included within the review. The three stages were i)
coarse-gained screening; ii) fine-grained screaning iii) appraising; and iv) synthesising.
These are explained below.
i) ‘Coarse grained’ screening of the literature
The initial, large number of items were screened using broad criteria of relevance and
quality. Items were excluded using the following broad criteria, applied to abstracts:
- wrong age group (e.g. the item focuses on post-secondary setting);
- not specific about education (e.g. the item focuses on games to develop leadership,
or to support autistic children);
- about education in non-formal settings (e.g., the item focuses on game-based
learning to improve performance in a nutrition course - or in a museum setting - or
non-research items like practitioner guides);
- not-research based and/or too explicitly based on opinion.
This stage led to a ‘Long list’ of 64 items.
ii) ‘Fine-grained’ screening of the literature
This stage was based on a thorough analysis of the ‘long list’ abstracts, seeking to exclude
all items that did not meet the agreed inclusion criteria. The review team then selected the
most relevant and best quality items to appraise and synthesise. We selected 31 items of
literature.
Game-based Learning: Latest Evidence and Future Directions 34
iii) Appraising the literature
We then appraised the full text of each selected item, using a template, to extract the key
research questions(s) and findings from each study, as well as assessing the quality and
relevance of each item.
iv) Synthesising the literature and making sense of patterns
Having appraised the key literature items, we synthesised the findings. This involved
analysing the reviewed evidence to draw out emerging themes and key messages.
Game-based Learning: Latest Evidence and Future Directions 35
Appendix 2: the evidence base for the review
This appendix provides a brief description of all relevant items of literature included in the
main body of the review, together with the table (also in Chapter One) reporting the inclusion
ratings of each item.
High Medium
Qualitative Marković et al.,2007; Salen, 2008
Quantitative Chuang and Chen, 2009;
Delacruz, 2011; Huizenga et al.,
2009; Miller and Robertson,
2011
Annetta et al.,2009; Bourgonjon
et al., 2010; Brom et al., 2011;
Fengfeng, 2008a; Kenny and
McDaniel, 2011; Kolovou and
Heuvel-Panhuizen, 2010; Liu et
al., 2011; Papastergiou, 2009;
Schaaf, 2012; Spires et al.,2011;
Vos et al.,2011; Ya-Ting, 2012
Mixed methods Fengfeng, 2008b; Kebritchiet
al.,2010
Review and
Meta-analysis
Chiu et al.,2012; Vogel et al.,
2006; Young et al., 2012
De Freitas and Griffiths, 2008;
Kapp, 2012; Williamson, 2009
Speculative N/A Bogost, 2011; Buckingham, 2008;
Gee, 2008; McGonigal, 2011;
Shaffer, 2008
Full references
Brief description
Annetta, L. A., Minogue, J., Holmes, S. Y., and
Cheng, M.T. (2009). ‘Investigating the impact of
video games on high school students’ engagement
and learning about genetics’, Computers and
Education, 53, 1, 74–85.
This quasi-experimental study
evaluated a teacher created
videogame on genetics in terms
of its affective and cognitive
impact on student users.
Bogost, I. (2011). How to Do Things with Video
Games. Minneapolis, MN: University of Minnesota
Press
The book proposes some key
definitions and theoretical
assumptions of game-based
learning and gamification. Ian
Bogost is one of the most
prominent authors who are
shaping the emerging field of
‘game studies’. He is mainly
interested in applying insights
from a range of disciplines,
including critical social theory,
politics and philosophy, to the
Game-based Learning: Latest Evidence and Future Directions 36
Full references
Brief description
analysis of video games or game-
based experiences.
Bourgonjon, J., Valcke, M., Soetaert, R.,
andSchellens, T. (2010). ‘Students’ perceptions
about the use of video games in the classroom’,
Computers and Education, 54, 4, 1145–1156.
In this study, a path model to
examine and predict student
acceptance of videogames is
proposed, and empirically tested
by involving 858 secondary
school students.
Brom, C., Preuss, M., and Klement, D. (2011). ‘Are
educational computer micro-games engaging and
effective for knowledge acquisition at high-
schools? A quasi-experimental study’, Computers
and Education, 57, 3, 1971–1988.
Curricular schooling can benefit
from the usage of educational
computer games, but it is difficult
to integrate them in the formal
schooling system. Here, the
authors investigate one possible
approach to this integration.
Buckingham, D. (2007). ‘Playing to learn?
Rethinking the educational potential of computer
games.’ In: Buckingham, D. Beyond Technology:
Children’s Learning in the Age of Digital Culture.
Cambridge: Polity Press.
In the sixth chapter of Beyond
Technology – children’s learning
in the age of digital culture,
Buckingham (2007, p.99)
examines the notion supported by
many computer games advocates
that more authentic kinds of
learning occur through children’s
informal interactions with new
media and technology.
Chiu, Y.H., Kao, C.W., and Reynolds, B.L. (2012).
‘The relative effectiveness of digital game
‐
based
learning types in English as a foreign language
setting: A meta
‐
analysis’, British Journal of
Educational Technology, 43, 4, 104–107.
This study synthesizes relevant
studies which investigate overall
effects of digital game based
learning. It also refers to some
meta analysis
Chuang, T.Y. and Chen W.F. (2009). ‘Effect of
computer-based video games on children: an
experimental study’, Educational Technology and
Society, 12, 2, 1–10.
This experimental study
investigated whether computer-
based video games facilitate
children's cognitive learning. One
hundred and eight third-graders
from a middle/high socio-
economic standard school district
in Taiwan participated in the study
De Freitas, S. and Griffiths, M. (2008). ‘The
convergence of gaming practices with other media
forms: what potential for learning? A review of the
literature’, Learning, Media and Technology, 33, 1,
11–20.
This article reviews literature for
evidence of these trends of
convergent media forms as a
starting point for a wider debate
for using games technologies and
practices to support learning
practices.
Delacruz, G.C. (2011). Games as Formative
Assessment Environments: Examining the Impact
of Explanations of Scoring and Incentives on Math
Learning, Game Performance, and Help Seeking
This report examines how
different
levels of detail about a game’s
scoring rules affect math learning
and performance.
Game-based Learning: Latest Evidence and Future Directions 37
Full references
Brief description
(CRESST Report 796). Los Angeles, CA:
University of California, National Center for
Research on Evaluation, Standards, and Student
Testing.
Fengfeng, K. (2008a). ‘A case study of computer
gaming for math: engaged learning from
gameplay?’ Computers and Education, 51, 4,
1609–1620.
This case study examined the in
situ use of educational computer
games in a summer math
program to facilitate 4th and 5th
graders' cognitive math
achievement, metacognitive
awareness, and positive attitudes
toward math learning.
Fengfeng, K. (2008b). ‘Computer games
application within alternative classroom goal
structures: cognitive, metacognitive, and affective
evaluation’, Educational Technology Research and
Development, 56, 5–6, 539–556.
The study found that a
cooperative structure (where
learners worked together in the
gamespace) was better than an
individualistic or competitive
structure at facilitating attitudes
towards maths learning
Gee, J.P. (2007). What Video Games Have to
Teach Us About Learning and Literacy. New York,
NY: Palgrave Macmillan.
This is a ‘seminal’ text about
game-based learning that
explored several theoretical and
empirical issues.
Huizenga, J., Admiraal, W., Akkerman, S. and
Dam, G.T. (2009). ‘Mobile game
‐
based learning in
secondary education: engagement, motivation and
learning in a mobile city game’, Journal of
Computer Assisted Learning, 25, 4, 332–344.
The study investigates the effects
of a mobile city game .A quasi-
experimental design was used
with 458 pupils from 20 classes
from five schools.
Kapp, K.M. (2012). The Gamification of Learning
and Instruction: Game-based Methods and
Strategies for Training and Education, San
Francisco, CA: Pfeiffer.
This book focuses on the design
opportunities of ‘gamification’,
mainly in terms of increasing
learning, retention, and
productivity in the workplace.
Kebritchi, M., Hirumi, A. and Bai, H. (2010). ‘The
effects of modern mathematics computer games
on mathematics achievement and class
motivation’, Computers and Education, 55, 2, 427–
443.
The paper reports two studies,
measuring learning gains and
time on task in different
experimental conditions.
Kenny, R.F.K. and McDaniel, R. (2011). ‘The role
teachers’ expectations and value assessments of
video games play in their adopting and integrating
them into their classrooms’, British Journal of
Educational Technology, 42, 2, 197–213.
This paper presents the results of
a pilot study with a group of
preservice teachers, investigating
their perceptions about video
games.
Game-based Learning: Latest Evidence and Future Directions 38
Full references
Brief description
Kolovou, A. and Heuvel-Panhuizen, M.V.D. (2010).
‘Online game-generated feedback as a way to
support early algebraic reasoning’, International
Journal of Continuing Engineering Education and
Life Long Learning, 20, 2, 224–238.
This study explores the potential
of game-generated feedback to
support primary school students'
problem solving processes.
Liu, C.C., Cheng, Y. B. and Huang, C.W. (2011).
‘The effect of simulation games on the learning of
computational problem solving’, Computers and
Education, 57, 3, 1907–1918.
This study analyzed the feedback
and problem solving behaviors of
117 students in a simulation
game, designed to assist them to
learn computational problem
solving.
Marković, F., Petrovic, O., Kittl, C. andEdegger, B.
(2007). ‘Pervasive learning games: A comparative
study’, New Review of Hypermedia and
Multimedia, 13, 2, 93–116.
This paper investigates how
pervasive games can be used for
an efficient transfer of knowledge
in learning situations. The paper
presents the design of a new
pervasive learning game, which
was compared with a
conventional case-study
approach in an empirical study
with 100 students in respect to
long-term learning results and
learning efficiency.
McGonigal, J. (2011). Why Games Make Us Better
and How They Can Change the World. London:
Jonathan Cape.
This is an influential book that
popularised the idea of
‘gamification’: the use of game
design elements to aid a range of
individual and social enterprises.
Miller, D.J. and Robertson, D.P. (2011).
‘Educational benefits of using game consoles in a
primary classroom: a randomised controlled trial’,
British Journal of Educational Technology, 42, 5,
850–864.
Randomised Control Trial of
games consoles in classroom,
based on large scale quantitative
study.
Papastergiou, M. (2009). ‘Digital game-based
Learning in high school Computer science
education: impact on educational effectiveness and
student motivation’, Computers and Education, 52,
1, 1–12.
The aim of this study was to
assess the learning effectiveness
and motivational appeal of a
computer game for learning
computer memory concepts. The
sample was 88 students, who
were randomly assigned to two
groups.
Game-based Learning: Latest Evidence and Future Directions 39
Full references
Brief description
Salen K. (Ed.) (2008). The Ecology of Games,
Connecting Youth, Games, and Learning (The
John D. and Catherine T. MacArthur Foundation
Series on Digital Media and Learning). Cambridge,
MA: The MIT Press.
Gee, J.P. (2008). ‘Learning and games.’ In:
Salen, K. (Ed) The Ecology of Games: Connecting
Youth, Games, and Learning (The John D. and
Catherine T. MacArthur Foundation Series on
Digital Media and Learning). Cambridge, MA: The
MIT Press.
Salen, K. (2008). ‘Towards an ecology of gaming.’
In: Salen, K. (Ed) The Ecology of Games:
Connecting Youth, Games, and Learning (The
John D. and Catherine T. MacArthur Foundation
Series on Digital Media and Learning). Cambridge,
MA: The MIT Press.
This volume looks at games as
systems in which young users
participate, as gamers, producers,
and learners. It’s a collection of
influential views about ‘game
studies’ including game-based
learning. Specific chapters which
have been reviewed more in
detail are cited in full in the left
column.
Schaaf, R. (2012). ‘Does digital game-based
learning improve student time-on-task behavior
and engagement in comparison to alternative
instructional strategies?’ Canadian Journal of
Action Research, 13, 1, 50–64.
The study examined Digital
Game-Based Learning activities
in comparison with effective,
research-based learning
strategies to observe any
difference in student engagement
and time-on task behaviour.
Shaffer, D.W. (2006). How Computer Games Help
Children Learn. New York, NY: Palgrave
Macmillan.
Another influential book that
popularised the notion of ‘game-
based learning’. In particular, the
idea that, through role play and
simulation, video games help
young people think and act like
actual professionals.
Spires, H.A., Rowe, J.P., Mott, B.W., and Lester,
J.C. (2011). ‘Problem solving and game-based
learning: effects of middle grade students'
hypothesis testing strategies on learning
outcomes’, Journal of Educational Computing
Research, 44, 4,453–472.
In this study, middle grade
students achieved significant
learning gains from gameplay
interactions that required solving
a science mystery based on
microbiology content.
Vogel, J.J., Vogel, D.S., Cannon-Bowers, J.,
Bowers, C.A., Muse, K. and Wright, M. (2006).
‘Computer gaming and interactive simulations for
learning: a meta-analysis’, Journal of Educational
Computing Research, 34, 3, 229–243.
A widely cited meta-analysis that
examines the available evidence
about the relationship between
video-games, academic
achievement and other learning
gains.
Game-based Learning: Latest Evidence and Future Directions 40
Full references
Brief description
Vos, N., van der Meijden, H. and Denessen, E.
(2011). ‘Effects of constructing versus playing an
educational game on student motivation and deep
learning strategy use’, Computers and Education,
56, 1, 127–137.
In this study the effects of two
different interactive learning
tasks, in which simple games
were described with respect to
student motivation and deep
strategy use.
Williamson, B. (2009) Computer Games, Schools
and Young People: a Report for Educators on
Using Games for Learning. Bristol: Futurelab
[online]. Available:
http://archive.futurelab.org.uk/resources/document
s/project_reports/becta/Games_and_Learning_edu
cators_report.pdf [28 February, 2013].
This report focuses on the use of
games as resources to support
the educational aims, objectives
and planned outcomes of
teachers who understand that
games are an important medium
in contemporary culture and
young people’s experiences,
providing an assessment of
game-based learning in UK
schools.
Ya-Ting, C.Y. (2012). ‘Building virtual cities,
inspiring intelligent citizens: digital games for
developing students’ problem solving and learning
motivation’, Computers and Education, 59, 2, 365–
377.
This study investigates the
effectiveness digital game-based
learning (DGBL) on students’
problem-solving, learning
motivation, and academic
achievement.
Young, M., Slota, S., Cutter, A.B. Jalette, G.,
Mullin, G., Lai, B., Simeoni, Z., Tran, M. and
Yukhymenko, M. (2012). ‘Our princess is in
another castle : a review of trends in serious
gaming for education’, Review of Educational
Research, 82, 1, 61–89..
In a review of literature, where the
authors identified 300+ articles
whose descriptions related to
video games and academic
achievement.
