ArticlePDF Available

Current Practices in Serious Game Research: A Review from a Learning Outcomes Perspective


Abstract and Figures

Despite scant empirical substantiation, serious games are in widespread use. The authors review 28 studies with empirical data from a learning outcome perspective to outline the effectiveness of serious games (compared to other learning approaches and specific game features). They conclude that serious games potentially improve the acquisition of knowledge and cognitive skills. Moreover, they seem to be promising for the acquisition of fine-grid motor skills and to accomplish attitudinal change. However, not all game features increase the effectiveness of the game. To further advance game research the chapter proposes recommendations including the alignment of learning outcome(s) and game type, the alignment of the game complexity and human cognitive processes, attention for cognitive and motivational processes, research on specific mitigating factors like gender on game effectiveness and, finally, developing new ways of assessing game effectiveness. (
Content may be subject to copyright.
Pieter Wouters
Erik van der Spek
Herre van Oostendorp
Institute of Information and Computing Sciences, Utrecht University, the Netherlands
Despite scant empirical substantiation, serious games are in widespread use. We review 28
studies with empirical data from a learning outcome perspective to outline the effectiveness of
serious games (compared to other learning approaches and specific game features). We conclude
that serious games potentially improve the acquisition of knowledge and cognitive skills.
Moreover, they seem to be promising for the acquisition of fine-grid motor skills and to
accomplish attitudinal change. However, not all game features increase the effectiveness of the
game. To further advance game research we propose recommendations including the alignment
of learning outcome(s) and game type, the alignment of the game complexity and human
cognitive processes, attention for cognitive and motivational processes, research on specific
mitigating factors like gender on game effectiveness and, finally, developing new ways of
assessing game effectiveness.
Keywords: computer games; video games; PC games; digital games; serious games; virtual
reality; instructional technology; learning outcome; computer-based training; cognitive theory;
cognitive skills, attitudinal change; game types; simulations; game-based learning
The use of games in learning and instruction, often referred to as serious games, has been
propagated by many researchers. Serious games are hypothesized to address both the cognitive
and affective dimensions of learning (O’Neil, Wainess & Baker, 2005), to enable learners to
adapt learning to their cognitive needs and to provide motivation for learning (Malone, 1981).
However, reviews focusing on serious games have revealed little substantiation for these claims
(Fletcher & Tobias, 2008; Kirriemuir & McFarlane, 2004; Leemkuil, de Jong & Ootes, 2000;
O’Neil et al; Vogel et al., 2006).
This review focuses on the learning outcomes for two reasons. First, typically serious games
aim at specific learning goals and consequently for specific learning outcomes as well. Therefore
it seems obvious to conduct the review from this perspective. Secondly, many studies on serious
games have focused particularly on cognitive learning outcomes: learning of knowledge and
problem solving skills. Consequently, previous reviews have focused on particular types of
learning outcomes and have neglected other types. We contend that a comprehensive taxonomy
of learning outcomes will not only reveal in which situations serious games improve learning, but
also uncover dimensions of learning that have been neglected thus far in reviews.
In the remainder of this chapter we first define games. Next, we present a taxonomy of
learning outcomes. For each learning outcome we then review relevant studies and draw some
conclusions. Finally, we present some directions for future research and draw a final conclusion.
A serious game is a computer based game with a primary purpose other than entertainment,
ranging from anywhere between advertisements to military training exercises (Michael & Chen,
2005). Naturally in this review we will concern ourselves mainly with games that aim at the
aforementioned learning outcomes. Many definitions exist that describe a game (cf. Garris,
Ahlers & Driskell, 2002; Vogel et al., 2006), but mostly a definition along the following lines is
chosen: that it is goal-directed, a competitive activity (against the computer, another player, or
oneself) and conducted within a framework of agreed rules (Lindley, 2004). In addition, games
constantly provide feedback to enable players to monitor their progress towards the goal
(Prensky, 2001).
There are many classifications of learning outcomes. Traditionally, researchers have focused on
the cognitive dimension of learning outcomes (Bloom, 1956; Gagné, 1977). Others have included
affect-oriented objectives such as appreciation (Krathwohl, Bloom & Massai, 1964). More recent,
other classifications have emerged identifying factors such as collaboration/teamwork,
communication and self-regulation as potential outcomes of learning (Baker & Mayer, 1999). An
interesting classification of learning outcomes has been provided by Kraiger, Ford and Salas
(1993), who distinguish between cognitive outcomes (e.g., problem solving), skill-based
outcomes concerning the development of technical or motor skills, and affective outcomes
including attitude and motivation. Drawing from the two latter classification schemes, we propose
a taxonomy consisting of four categories of learning outcomes: cognitive, motor skills, affective
and communicative. Figure 1 presents an overview of these learning outcomes and their
constituent parts. Insert figure 1 here
Cognitive learning outcomes can be divided into knowledge and cognitive skills. Knowledge
refers to encoded knowledge reflecting both text-oriented (e.g., verbal knowledge) and non text-
oriented knowledge (e.g., knowledge in the form of an image). Several types of encoded
knowledge can be discerned such as declarative (explicit knowledge of facts) and procedural
(knowledge of how to perform a task). A cognitive skill pertains to more complex cognitive
processes. In problem solving, for example, learners have to apply knowledge and rules to solve
new problems. In complex and dynamic situations people are sometimes forced to make decisions
under time-pressure. Such decision making skills require situational awareness, that is, the ability
to attend to and perceive the relevant information in a situation, comprehend this information and
predict how the situation may develop (O’Brien & O’Hare, 2007).
The second type of learning outcome, learning motor skills, involves several stages. Initially a
learner has to acquire the skill by making a transition from declarative knowledge to procedural
knowledge. In subsequent stages the learner practices the motor behavior and in this way
compiles the behavior, that is, make the motor behavior faster, less error-prone and independent
of verbal rehearsal.
With affective learning outcomes we can differentiate two subtypes. To start with, learning
may focus on a change in the attitude of the learner. Attitudes refer to internal states that
influence the choices or actions of an individual (Gagné, 1977). This may pertain to a change
from a negative to a positive learning attitude towards (subjects at) school, but also to a change in
behavior that is exhibited in daily live (e.g., driving cautiously) or for therapeutic purposes (e.g.,
overcoming fear of spiders). The second subtype, motivation, is a prerequisite for learning to
commence. Motivation reflects the willingness to pay attention to learning material and to spent
cognitive resources to process information.
The last type comprises communicative learning outcomes. Although collaborative learning is
claimed to lead to a deeper level of understanding and long term retention of the learned material,
it also emphasizes the opportunities for developing social and communication skills, and building
social relationships and group cohesion (Kreijns, Kirschner & Jochems, 2003). In environments
where teams have to work together on tasks that go beyond the capabilities of one individual
(e.g., firefighters, cockpit crews), the training of communication and collaboration skills can be
the primary purpose of an instructional intervention.
In general the performance of (complex) tasks will involve different types of learning
outcomes. For example, learning to drive may comprise knowledge (e.g., traffic rules), motor
skills (e.g., changing gears) and attitudinal change (e.g., driving cautiously). This example also
illustrates that the learning outcomes are sometimes hierarchical: before the changing gears motor
skill can be performed the procedural knowledge has to be learned.
For the purpose of this review we searched several databases (PsychINFO, ERIC) with terms
including ‘game-based learning’, ‘PC games’, ‘video game’, computer video game’, ’serious
games’, ‘educational games’, ‘simulation games’, ‘virtual environments’, ‘virtual reality’. If
necessary these terms were combined with ‘learning’, ‘instruction’ and ‘training’. In addition, we
consulted the proceedings of relevant conferences (CHI, AERA and so forth) with the same
terms. The review was conducted in Summer 2008 and covered the last 10 years. Studies were
only considered when empirical data were available. Table 1 classifies the serious game studies in
the taxonomy of learning outcomes. We classified each study according to the learning outcome
that the research was primarily aimed at.
Insert table 1 here
First we discuss the relevant studies for each type of learning outcome. We discuss studies that
compare serious games with other instructional methods as well as studies that investigate the
effectiveness of specific game features.
Cognitive learning outcomes
Encoded knowledge
First, studies comparing a game group with a control group are discussed. In the multi-user
virtual environment River City two gaming groups (with a focus on respectively ‘learning-by-
doing’ and modeling) were compared with a control condition (Dede, Clarke, Ketelhut, Nelson &
Bowman, 2005). The task for all participants was to discover why residents of a virtual town were
getting ill. In this process they learned about biology. The results showed that the two gaming
groups gained more biology content knowledge than the control group. In learning about
electromagnetism Squire, Barnett, Grant and Higginbotham (2004) compared learners engaging
in the Supercharged game with a group engaging in guided inquiry. The game places students in
a three dimensional environment where they must navigate a spaceship by controlling the electric
charge of the ship. Learners in the Supercharged game performed better on knowledge than
guided inquiry learners. In training Navy electronic technicians, Parchman, Ellis, Christinaz and
Vogel (2000) found that trainees engaging in an adventure game (King’s Quest V-based) in which
they had to visit a series of compartments in a battleship and perform exercises, were
outperformed by trainees who received a computer-based practice-and-drill or an enhanced
computer-based instruction on a posttest measuring knowledge of definitions. However, no
differences were found on knowledge of symbols. In the domain of biology the game Metalloman
was compared with a hypertext and a text instruction group in knowledge on physiology concepts
(Wong et al., 2007). It was observed that the game and the hypertext group yielded higher
learning gains than the text instruction group.
The effectiveness of the game features for relevance of information, interactivity, instructional
guidelines, level of stress, game task and game type have been investigated as well. In the game
America’s Army people learn about the values and history of the army. A study with military
academy recruits revealed that information that was relevant for the task was better recalled than
non-relevant information (Belanich, Sibley & Orvis, 2004). Regarding the game feature for
interactivity Wong et al. (2007) surprisingly found that an interactive version of Metalloman
yielded no higher learning gains than a non-interactive version of the game. In KMQuest, learners
run a commercial organization in which they have to make decisions in order to make the
organization more efficient. Leemkuil (2006) investigated several variations of the game feature
for instructional guidelines (advice vs. no advice, extra assignments vs. no extra assignments,
advice with hints vs. advice without hints vs. no advice), however, no difference on the
knowledge learning outcome was found in a transfer test that was administered after the training
in KMQuest. In a training with the game Delta Force no effect of the game feature for level of
stress was found on recall of knowledge of military tactics content (e.g., use of equipment)
between trainees who were exposed to either a low or a high level of stress (Morris, Hancock &
Shirkey, 2004). In geography Virvou, Katsionis and Manos (2005) investigated the effect of the
game feature for game task. For this purpose they compared a group engaging in a 3D virtual
reality game (based on Doom) with a group working in a hypertext environment. Both groups
differed in the fact that the virtual reality group had a game task, that is, they had to navigate
through a virtual 3D world with the explicit mission to find the missing pages of the ‘book of
wisdom’. The game group yielded higher learning gains than the hypertext group. Closely related
to the game task is the type of game that is used. In the game Re-Mission young people engage in
missions in 3-D virtual bodies of cancer patients and learn about the mechanisms underlying
cancer. Participants engaging in Re-Mission showed larger knowledge gains than participants
who received a commercial adventure game (Beale, Kato, Marin-Bowling, Guthrie & Cole,
The fact that three out of four studies report higher performance for groups learning with
games provides some evidence that the new generation of serious games support the acquisition
of knowledge. The effectiveness of specific game features is mixed. The game features for
relevance of information, game task and game type ameliorate the acquisition of knowledge,
whereas the game features for level of stress, instructional guidelines, and interactivity failed to
have an impact on learning.
Cognitive skills
First, we describe studies comparing a game group with a control group. In the multi-user
environment QuestAtlantis (QA) learners travel to virtual worlds and engage in educational
activities (quests). It was used by Barab, Warren and Ingram-Goble (2006) to compare the
performance in writing skills of a group receiving traditional instruction with a group working as
an investigative reporter for the local newspaper in QA. In writing tasks similar to the tasks
performed during instruction the QA group showed significantly more improvement. In the
aforementioned River City environment cognitive skills were measured in two ways: with open-
ended questions and with a letter to the mayor in which the participants discussed their
hypothesis, the results and the interpretations. Interestingly only a difference in favor of the game
groups was found when the cognitive skills were measured by the letter to the mayor (Dede et al.,
2005). In the military, games have been used to investigate the effect on learning specific
cognitive skills during a flight mission such as task management, decision making, and assessing
the situation. In the training of cockpit crews, commonly known as crew resource management
(CRM), the comparison of a group training on a PC-based simulator and a control group revealed
that the former group performed better on task management and situational awareness, but not on
other cognitive skills, such as decision making and planning (Nullmeyer, Spiker, Golas, Logan &
Clemons, 2006). Problematic in this study was that no information was provided with respect to
the training in the control condition. In the aforementioned training of Navy electronic
technicians, Parchman et al. (2000) found that the two computer-based groups performed better
on the application of principles than the game and classical instruction groups. For the application
of rules no differences between the groups were found. Finally, Ke and Grabowski (2007)
compared two game groups (students were either assigned to a cooperative or competitive version
of the game ASTRA Eagle) with a no game group on the acquisition of mathematical problem
solving skills. It was found that both game groups outperformed the no game group on
mathematical skills.
Other researchers have studied the effect of the game features for instructional guidelines,
level of stress and game type. Leemkuil (2006) varied the guideline advice (advice vs. no advice
and advice with hints vs. advice without hints vs. no advice) in the KMQuest environment, but
found no differences in a transfer task that was administered after the learning phase in KMQuest.
Nelson (2007) used River City to investigate whether guidance would improve science inquiry
and hypothesis formation skills. It appeared that groups with either extensive or moderate
guidance did not use this guidance and consequently did not gain a better command of these
cognitive skills than a group without such guidance. However, further analyses revealed higher
learning gains for learners who did use the provided guidance. Also the effect of extra
assignments was studied in the KMQuest learning environment, but no differences were found on
the transfer task that was administered afterwards (Leemkuil, 2006). The game feature for level of
stress was investigated in the game Delta Force in which the participants had to engage in an
arctic mission that required cognitive skills. The trainees in the group with a high level of stress
during training were more successful in completing the mission than trainees who were exposed
to a low level of stress (Morris et al., 2004). Finally, the impact of the game feature for game
type was investigated by comparing the effect of an action game (Medal of Honor: Pacific
Assault) with a non action game (Balance) on spatial cognition (Feng, Spence & Pratt, 2007). It
was found that only the action game enhanced spatial cognition. Interestingly, they also found
that the initial superiority of males over females in spatial cognition was much reduced after
working with the action game.
Although four out of five studies substantiate the claim that serious games are more effective
in training cognitive skills than traditional instructional methods, the results in the River City
study also pose the question how cognitive skills should be measured. The contextualized type of
learning that takes place in serious games may not be detected by traditional measurements, but
all the more with alternative measurements (i.e., essays). Also for learning cognitive skills the
game feature for instructional guidelines failed to be effective.
Motor skill learning outcomes
Much research has focused on the effect of video game experience on screen-mediated surgery
skills (i.e., the surgeon operates via a monitor). Researchers have compared groups that practiced
with games with groups that did not practice with games (Rosenberg, Landsittel & Averch, 2005;
Waxberg, Schwaitzberg & Cao, 2005). These studies did not show that video game experience
yielded better surgery skills. For instance, in laparoscopic surgery Waxberg et al. (2005)
hypothesized that practicing with the video game James Bond 007: Goldeneye
for a week
lead to better performance on several tasks on a surgery skills trainer. The results showed that this
was true for some tasks, but that the no-game group performed better on other tasks.
Other researchers have correlated video game experience with surgery skills and reported
video game experience predicted surgery performance (Enochsson
et al., 2004; Grantcharov,
Bardram, Funch-Jensen & Rosenberg, 2003; Rosser et al., 2007). Rosser et al. (2007), for
instance, showed that surgeons with video games experience made less errors and showed faster
completion times in a learning environment for laparoscopic surgery. Moreover, the video games
skills of these surgeons, demonstrated during three different video games, appeared to be
significant predictors of laparoscopic surgery skills.
The results in the domain of surgery are still inconclusive: whereas experimental designs fail
to show a beneficial effect, correlation studies seem to confirm the predictive power of game
experience. Given the ample evidence indicating that experience in video games enhances the
ability for visual search (e.g., Castel, Pratt & Drummond, 2005; Green & Bavelier, 2003, 2006)
and dual-tasking (Satyen & Ohtsuka, 2001), we believe that the use of serious games is promising
in learning fine-grid motor skills that require excellent hand-eye coordination.
Affective learning outcomes
Increasingly, virtual reality systems are used to support people in desensitizing a large range of
fears and phobias by allowing them to confront frightening situations without the danger of
possible (physical) harm. Since these systems are rather expensive, researchers have investigated
whether realistic games can be used for this purpose as well. Their focus was on the effect of
games on attitude and not the comparison of a game therapy with a traditional therapy. Bouchard,
Côte, St-Jacques, Robillard and Renaud (2006) designed a therapy requiring participants with fear
of spiders to engage in a Half-Life based environment where they were increasingly exposed to
spiders. Before the treatment, the majority stayed 2 metres from a bowl with spiders. After the
treatment the majority was able to stand next to the bowl. A similar positive effect was obtained
with participants who were diagnosed as having an accident phobia. Before and after the
intervention comprising a 12 hours program of game racing (e.g., London Racer) they were
assessed with several ratings such as distress and severity of fear of driving. It was found that the
participants showed posttest reductions on all measures (Walshe, Lewis, Kim, O’Sullivan &
Wiederhold, 2003).
The effectiveness of serious games on attitudinal change towards school topics was
investigated in the aforementioned study of Ke and Grabowski (2007), which revealed that
students engaging in a cooperative game developed a more positive attitude towards mathematics
than students who engaged in a competitive game or in paper-and-pencil lessons. This effect was
stronger for socio-economically disadvantaged students.
Regarding attitudinal change the effect of the features game task and game type have been
studied. In a simulator environment for training driving skills a group receiving a game task (i.e.,
they had to follow an ambulance without losing sight of it) was compared with a group without
such game task. It was found that the group with the game task showed safer traffic behavior on
dimensions such as looking in rearview windows and lane changing than the group without a
game task (Backlund, Engström, Johanneson & Lebram, 2007). The effect of the feature game
type on attitudinal change was investigated in a series of experiments conducted by Fischer,
Kubitzki, Guter and Frey (2007). They found that participants who played race games (e.g.,
Burnout) exhibited a less cautious driving behavior in terms of risk-taking and excitement than
participants who played neutral games (e.g., Fifa 2005). Apparently, not only is engaging in a
game task important, but so is the game type in which the task is performed.
Tentatively it can be concluded that serious games facilitate attitudinal change. Game features,
such as a game task and the game type also have an impact on attitude. The findings of both race
games studies also confirm that individual characteristics should be taken into account when
using serious games for attitudinal change. The traumatized participants in the Walshe et al.
(2003) study benefitted from the race game and became less fearful for driving, whereas the non-
traumatized participants in the Fisher et al. (2007) study became more reckless drivers after
engaging in the race game.
Previous reviews have claimed that serious games motivate players to continue and subsequently
it is alleged that this feature can be useful for the purpose of learning (cf. Garris et al., 2002), but
recent research on motivation is scant. Dede et al. (2005) argued that the large drop in absentee
rate (50%) during learning in River City environment may have indicated an increased
engagement during the implementation of River City. However, the absentee rate for the
traditional instruction group was not reported. In the aforementioned training of military cadets in
the Parchman et al. study (2000) a motivation questionnaire based on Keller’s ARCS model was
used to compare trainees’ motivation in the four groups (a game group, classical instruction, and
computer-based practice-and-drill or enhanced instruction). Although the game group participants
were more attentive to the contents than the classical instruction and computer-based practice-
and-drill group, no differences were observed between the game and computer-based enhanced
instruction groups. A qualification of the motivational aspects of games comes from Tuzun,
Yilmaz Soylu, Karakus, Inal and Kizilkaya (2008) who compared a game group (Quest Atlantis)
with traditional school learning on intrinsic and extrinsic motivation. They found some evidence
that students in the game group were more intrinsically motivated, whereas students in the
traditional school setting were more extrinsically motivated.
In short, no recent convincing evidence was found for the assumed motivational pull of
serious games.
Given the popularity of playing games among adolescents, it seems obvious that
games are motivating. It is not clear to what extent this pertains to serious games. In Quest
Atlantis, for example, Lim, Nonis and Hedberg (2006) reported that learners were less motivated
than the researchers had expected. It is remarkable that characteristics of games such as
immersion and interactivity that are considered motivating in entertainment games, refrained
students in Quest Atlantis from full engagement in the learning task. This indicates that a better
understanding is required about the underlying motivational processes in serious games. We will
return to this issue in the Discussion section at the end of the chapter.
Communicative learning outcomes
In training communicative skills of cockpit crews (CRM training), Brannick, Prince and Salas
(2005) compared the communication skills of trainees receiving CRM-simulator training with a
group receiving group exercises and video games (Asteroids). The CRM-simulator group showed
better communicative skills than the group with exercises and video games in an assessment task
requiring the trainees to contact the air traffic controller (ATC) in order to obtain the information
that was deliberately omitted by the ATC. However, these results were not confirmed in a similar
study comparing a PC-based simulator and a control group (Nullmeyer et al., 2006), although a
problem with this study was that it did not report what kind of training the control group received.
The impact of a game feature called the awareness tool was investigated in the game
SpaceMiners where dyads have to collect minerals located in asteroids by launching drones and
bring them to a space station. The players can use tools to manipulate the direction of the drone,
but they have to negotiate where to position these tools in space. An awareness tool helps players
to understand the activities of other players in the game. It appeared that pairs with an awareness
tool outperformed pairs without an awareness tool in collecting minerals (Nova, Dillenbourg,
Wehrle, Goslin & Bourquin, 2003). The game feature for level of immersion was investigated in
the game DOOM II in which dyads had to collaborate in order to find their way through a virtual
maze. It appeared that the quality of collaboration in the immersive (head-mounted display) and
the nonimmersive (monitor) conditions was comparable, with the exception that the immersive
group took more time to complete the task (Galimberti, Ignazi, Vercesi, & Riva, 2001).
Research investigating the effect of serious games on communicative skills is still
undeveloped. Recently, massive multiplayer online games (MMOGs) have become very popular.
These MMOGs (e.g., World of Warcraft) are graphical 3D videogames allowing players, by
means of self-created digital characters or ‘avatars’, to interact with the game world and with
other players’ avatars as well. Research on interaction and collaboration in these games is very
limited. There is some research on social interaction in these MMOGs, but these studies focus on
the characteristics of game players (e.g., Seay, Jerome, Lee & Kraut, 2004). Hopefully, the
increasing popularity of MMOGs will become an encouragement for more research into the
impact of online gaming on communicative skills.
It should be noted that the number of studies is too low to make definite conclusions. In order to
substantiate the claims regarding the learning potential of serious games more research is required
and, even more important, data and results have to be reported. We recommend five directions for
follow-on research that may further advance learning with serious games.
1. Alignment of game type and learning outcome
First, the mixed results of this review pose the question whether the appropriate game design was
selected for obtaining the specified learning outcome(s). As different game types can elicit very
different cognitive and affective responses in the player (e.g., Ravaja et al., 2004), designers of
serious games should carefully consider the implications of the game design on the possible
learning outcomes. For this reason, we propose a framework that categorizes the games according
to their level of cognitive and affective complexity (CALC). Most taxonomies that have been
introduced (cf. Lindley, 2003; Björk, Lundgren, & Holopainen, 2003) approach the categorization
from a design standpoint. As our review approached the research from a user-centered viewpoint,
namely the learning outcomes, we also propose a more user-centered framework for the
categorization of games (Figure 2). Insert Figure 2 here
The scale we currently propose consists of four different layers, although in the future more
differentiation is possible. Furthermore, a level of cognitive and affective complexity is defined as
the corresponding layer together with the previous underlying layers. As increasing levels of
complexity open up new possibilities for training while maintaining those of previous levels, the
levels are said to work cumulatively in possible learning outcomes.
The first level comprises games that are textual or symbolic in outlook often with simple and
explicit mechanics. In cognitive terms, players have to create a mental model of the game rules
and consequences of the actions they perform. At this basic level, games can be used to train
problem solving skills, decision making and teach verbal and conceptual knowledge.
The second level comprises games that are situated in a spatial environment. Here spatial
dimensions have to be interpreted and the spatial interrelationships between different objects are
thus added to the mental model constructed in the first layer. Some basic situational awareness
can be trained with these kinds of games, for instance in assessing the distance and route to the
nearest exit in case of fire, as well as hand-eye coordination and motor skills.
A layer on top of this is the presence layer, where players not only have to navigate a virtual
world, but strongly feel that they are a person immersed in this world. This feeling of ‘presence’
opens up a range of affective responses that could be part of a training exercise, for example
stress control or anxiety alleviation. Different presentation types generate different degrees of
presence (Nunez & Blake, 2003), but First Person 3D games are probably best suited.
Lastly, because the feeling of being in an environment opens up possibilities for social
interactions with other beings inside the virtual world, and these virtual communities add to the
complexity of the game, multiplayer or MMO games make up the top level on our cognitive and
affective complexity scale. These games can be used to study and train a person’s social skills in
a group or large scale community setting.
As all taxonomies trying to cope with the highly diffuse area of games, it is not perfect; a
3D game with no social interaction may be more perceptually rich, and therefore cognitively
demanding, than a text based multi-user dungeon. However, while tentative, we maintain that
it can provide a guideline for choosing the right game design to achieve the desired learning
goals; situational awareness may not transfer well when trained with a 2D game, while overly
complex designs may compromise the learning outcomes that can also be achieved with
simpler games.
2. The role of human cognitive architecture
The second recommendation pertains to the question how to (further) optimize the effectiveness
of serious games. As this review has shown game features can be manipulated to improve the
effectiveness of the serious game. In Table 2 an overview is presented of the game features that
were discussed in four types of learning outcomes. It shows that the investigated instructional
guidelines failed to increase the effectiveness of the serious game. In other cases the effectiveness
was only increased for one type of learning outcome, but not for another learning outcome.
Insert Table 2 here
Playing a serious game is a complex task, even when an appropriate design was chosen for the
intended learning outcomes: Players have to visually attend different locations on the screen,
coordinate this with mouse or joystick movement, interpret verbal cues, and solve problems that
occur during the game play. We contend that the effectiveness of a game feature is contingent on
the ability of designers to align the complexity of the serious game with the limitations of human
processing capacity. From a cognitive theory perspective it can be argued that without support
novice game players can easily become overwhelmed by all the information that has to be
processed. For example, a relevant game task may limit the amount of irrelevant information that
the player has to process. In this way cognitive capacity can be effectively used for processing
information that fosters learning from the serious game.
It would be interesting to see whether instructional guidelines that have been successful in
learning from animations pertain to the design of serious games as well (cf. Wouters, Tabbers &
Paas, 2007). Potential instructional guidelines that may reduce information overload in serious
games include pacing (i.e., regulating the speed of information presentation), focusing attention
and activating relevant domain knowledge (e.g., by providing knowledge gaps). The challenge for
designers would be to implement these instructional guidelines without losing the power of
attraction that games have.
The purpose of these guidelines is to enable learners to engage in cognitive processes that
contribute to learning. However, little is known about the types of cognitive processes that occur
during serious gaming. Therefore we recommend more research be carried out that extends the
understanding of effective and ineffective cognitive processes in learning with serious games.
example, cognitive theories consider the use of trail-and-error methods in learning how to
solve problems to generate ineffective cognitive processes. It would be valuable to see under
which conditions such ineffective cognitive processes occur. One of these conditions, the
game structure, was investigated by Pillay (2002) who observed that linear cause-and-effect
oriented games yielded a trial-and-error problem solving behavior in the game, whereas
adventure games encouraged more inferential and proactive thinking. Apart from the
implication for cognitive processes, the Pillay study also emphasizes the importance of the
structure of serious games.
3. The role of mitigating factors
The third recommendation is related to the lack of understanding on factors that mitigate the
effect of serious games on learning, and three factors in particular. The first factor pertains to the
gender of learners. Some researchers have reported notable differences in results between male
and female participants. For example, in the River City study Nelson (2007) found girls to be
more effective in the use of guidance and Feng et al. (2007) reported that on spatial cognition
female students benefitted more from action games than male students.
The second factor concerns training time. If it is true that players immerse themselves in
games and consequently spend more time on the task, then the question arises whether the higher
performance can be ascribed to the extra time spent on the task in the game or to the
characteristics of the games that support learning.
The last mediating factor to be discussed is age. One of the central findings in cognitive aging
research is that the efficiency of working memory deteriorates with aging. This may be particular
relevant for complex serious games. Elderly learners may have problems with discerning between
relevant and irrelevant information in the game or their processing speed can not keep up with the
progress in the game. Without instructional support, a serious game that may be effective for
young learners may be ineffective for elderly learners.
4. Understanding motivation(al) processes
The fourth recommendation concerns the assumed motivational impact of games. Apart from
theoretical accounts of game characteristics that motivate players to sustain playing a game, we
also need a better understanding of the psychological mechanisms underlying these motivation
processes. Of particular note for this type of study are those conducted by Ryan, Rigby and
Przybylski (2006). Drawing from self-determination theory, they hypothesized that perceived
autonomy (i.e., feeling uncontrolled when pursuing an activity) and competence (i.e., a need for
challenge and feelings of effectance) would enhance motivation to play games. In the studies
participants played games like Super Mario 64, Zelda and A Bugs Life. The results showed that
experiences of competence and autonomy while playing accounted for gaming motivation and
enjoyment. Another promising avenue of research is the relation between ‘flow’ and learning
from serious games. Flow has been described as such an extent of involvement in a task that
nothing else seems to matter (Garris et al., 2002). Although it is undeniable that players fully
engage in popular games and forget the real world around them, it is still unclear how such full
engagement relates to learning. With this focus research may begin to establish links between
game features, motivational processes, and learning outcomes.
5. Assessment of learning outcomes
The final recommendation concerns the validity of the learning outcomes, that is, did the
assessment test that was used really measure the learning outcome that was aimed at? Dede et al.
(2005) demonstrated a better command of cognitive skills for a game group when measured with
an evaluation letter to the mayor, but not with traditional test items. Most serious games are
situated in specific contexts that may yield learning outcomes that are contextualized as well.
Assessment methods that take the context of learning into account (e.g., an evaluation letter to the
mayor) may reveal differences in performance that would be undisclosed with traditional
assessment methods.
An additional argument for reconsidering the traditional assessment methods follows from the
results of Belanich et al. (2004) who found that items with visual information were better recalled
than written information. Video games are highly visual and may favor the acquisition of visually
encoded knowledge. In that case visually-oriented assessment may reveal learning of knowledge
that would probably not have been found with a text-based assessment method.
Another promising direction for assessment in serious games comes from a study by Day,
Arthur and Gettman (2001) who measured the learning of complex skills with the game Space
Fortress by assessing the knowledge structures that the players constructed. In knowledge
structures the information is mentally organized in concepts, the features that define them and the
relationships between the concepts. It appeared that the degree of similarity between the
knowledge structures of trainees and those of experts was correlated with complex skill
acquisition and a good predictor of skill retention and transfer. Altogether, future research on the
effectiveness of serious games should also consider other techniques to measure learning.
We gave an outline of the current practices in serious games research by reviewing 28 studies
with empirical data from the perspective of learning outcomes. We discerned cognitive, motor
skills, affective and communicative learning outcomes. In general, serious games seem to be
effective when it comes to cognitive learning outcomes. Serious games for training motor skills
and attitudinal change is promising. Finally, little recent substantiation was found for the
effectiveness on motivation and communicative learning outcomes. With respect to the
effectiveness of game features, especially, the implementation of the investigated instructional
guidelines did not improve learning. Although
the number of studies is too low to make definite
conclusions, the review provides an indication of the current practices. We believe that
serious games are promising, but that more research is required that should also consider the
alignment of learning outcomes and game type, the limited cognitive capacity, specific mitigating
factors (e.g., gender), motivational processes and new assessment methods.
This research has been supported by the GATE project, funded by the Netherlands Organization
for Scientific Research (NWO) and the Netherlands ICT Research and Innovation Authority (ICT
Backlund, P., Engström, H., Johannesson, M., & Lebram, M. (2007). Games and traffic safety
an experimental study in a game-based simulation environment. Proceedings of the International
Conference Information Visualization, 11, 908–916.
Baker, E. L., & Mayer, R. E. (1999). Computer-based assessment of problem solving. Computers
in Human Behavior, 15, 269–282.
Barab, S., Warren S., & Ingram-Goble, A. (2006, April). Academic play space: Designing games
for education. Paper presented at the meeting of the American Educational Research Association,
San Francisco, CA.
Beale, I. L., Kato, P. M., Marin-Bowling, V. M., Guthrie, N., & Cole, S. W. (2007). Improvement
in cancer-related knowledge following use of a psychoeducational video game for adolescents
and young adults with cancer. Journal of Adolescent Health, 41, 263–270.
Belanich, J., Sibley, D. E., & Orvis, K. L. (2004). Instructional characteristics and motivational
features of a PC-based game (Research Report No. 1822). Alexandria, VA: U.S. Army Research
Institute for the Behavioral and Social Sciences.
Björk, S., Lundgren, S., & Holopainen, J. (2003). Game Design Patterns. In M. Copier and J.
Raessens (Eds.), Proceedings of the Level Up: Digital Games Research Conference. Utrecht, The
Bloom, B. (1956). Taxonomy of educational objectives: The cognitive domain. New York:
Donald McKay.
Bouchard, S., Côte, S., St-Jacques, J., Robillard, G., & Renaud, P. (2006). Effectiveness of virtual
reality exposure in the treatment of arachnophobia using 3D games. Technology and Health Care,
14, 19–27.
Brannick, M. T., Prince, C., & Salas, E. (2005). Can PC-based systems enhance teamwork in the
cockpit? The International Journal of Aviation Psychology, 15, 173–187.
Castel, A. D., Pratt, J., & Drummond, E. (2005). The effects of action video game experience on
the time course of inhibition of return and the efficiency of visual search. Acta Psychologica, 119,
Day, E. A., Arthur Jr., W., & Gettman, D. (2001). Knowledge structures and the acquisition of a
complex skill. Journal of Applied Psychology, 86, 1022–1033.
Dede, C., Clarke, J., Ketelhut, D. J., Nelson, B., & Bowman, C. (2005, April). Students’
motivation and learning of science in a multi-user virtual environment. Paper presented at the
meeting of the American Educational Research Association, Montréal, Quebec.
Enochsson, L., Isaksson, B., Tour, R., Kjellin, A., Hedman, L., Wredmark, T., & Tsai-Fellander,
L. (2004). Visuospatial skills and computer game experience influence the performance of virtual
endoscopy. Journal of Gastrointestinal Surgery, 8, 874–880.
Feng, J., Spence, I., & Pratt, J. (2007). Playing an action video game reduces gender differences
in spatial cognition. Psychological Science, 18, 850–855.
Fischer, P., Kubitzki, J., Guter, S., & Frey, D. (2007) Virtual driving and risk taking: Do racing
games increase risk-taking cognitions, affect, and behaviors? Journal of Experimental
Psychology: Applied, 13, 22–31.
Fletcher, J.D., & Tobias, S. (2008). What research has to say (thus far) about designing
computer games for learning. Paper presented at the American Educational Research
Association, New York, NY.
Gagné, R. M. (1977). The conditions of learning. New York: Holt, Rinehart & Winston.
Galimberti, C., Ignazi, S., Vercesi, P., & Riva, G. (2001). Communication and cooperation in
networked environments: An Experimental Analysis. CyberPsychology & Behavior, 4, 131–146.
Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and
practice model. Simulation & Gaming, 33, 441–467.
Grantcharov,T. P. , Bardram, L., Funch-Jensen, P., & Rosenberg, J. (2003). Impact of hand
dominance, gender, and experience with computer games on performance in virtual reality
laparoscopy. Surgical Endoscopy, 17, 1082–1085.
Green, C. S., & Bavelier, D. (2003). Action video game modifies visual selective attention.
Nature, 423, 534–537.
Green, C. S., & Bavelier, D. (2006). Effect of action video games on the spatial distribution of
visuospatial attention. Journal of Experimental Psychology: Human Perception and
Performance, 32, 1465–1478.
Ke, F., & Grabowski, B. (2007). Gameplaying for maths learning: cooperative or not? British
Journal of Educational Technology, 38, 249–259.
Kirriemuir, J., & McFarlane, A. (2004). Literature review in games and learning (Futurelab Series
Report 8). Retrieved March 17, 2008 from Futurelab Website
Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective
theories of learning outcomes to new methods of training evaluation. Journal of Applied
Psychology, 78, 311–328.
Krathwohl, D. R., Bloom, B. S., & Massai, B. B. (1964). Taxonomy of educational objectives:
The classification of educational goals. White Plains, NY: Longman.
Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction
in computer-supported collaborative learning environments: A review of the research. Computers
in Human Behavior, 19, 335–353.
Leemkuil, H. (2006). Is it all in the game? Learner support in an educational knowledge
management simulation game. Unpublished doctoral thesis. University of Twente, Enschede, The
Leemkuil, H., de Jong, T., & Ootes, S. (2000). Review of educational use of games and
simulations (IST-1999-13078 Deliverable D1).University of Twente, Enschede, The
Lim, C. P., Nonis, D., & Hedberg, J. (2006). Gaming in a 3D multiuser virtual environment:
engaging students in Science lessons. British Journal of Educational Technology, 37, 211–231.
Lindley, C.A. (2003). Game Taxonomies: A High Level Framework for Game Analysis and
Design. Retrieved March 17, 2008 from Gamasutra Website.
Lindley, C.A. (2004). Narrative, game play, and alternative time structures for virtual
environments. In S. Göbel, U. Spierling, A. Hoffman, I. Iurgel, O. Schneider, J. Dechau & A.
Feix (Eds.), Lecture Notes in Computer Science: Vol. 3105. Technologies for Interactive Digital
Storytelling and Entertainment (pp. 183-194). Heidelberg: Springer Berlin.
Malone, T. (1981). Toward a theory of intrinsically motivating instruction. Cognitive Science, 4,
Michael, D., & Chen, S. (2006). Serious games: Games that educate, train, and inform. Boston,
MA.: Thomson Course Technology.
Morris, C. S., Hancock, P. A., & Shirkey, E. C. (2004). Motivational effects of adding context
relevant stress in PC-based game training. Military Psychology, 16, 135–147.
Nelson, B. C. (2007). Exploring the use of individualized reflective guidance in an educational
multiuser virtual environment. Journal of Science Education and Technology, 16, 83–97.
Nova, N., Dillenbourg, P., Wehrle, T., Goslin, J., & Bourquin, Y. (2003). The impacts of
awareness tools on mutual modelling in a collaborative video-game. In J. Favela & D.
Decouchant (Eds.), Lecture Notes in Computer Science: Vol. 2806. Groupware Design,
Implementation, and Use (pp. 99–108). Heidelberg: Springer Berlin.
Nullmeyer, R. T., Spiker, V. A., Golas, K. C., Logan R. C., & Clemons, L. (2006, December).
The effectiveness of a PC based C-130 crew resource management aircrew device. Paper
presented at the meeting of the Interservice/Industry training, simulation, and Education
Conference, Orlando, FL.
Nunez, D. & Blake, E. (2003). A direct comparison of presence levels in text-based and graphics-
based virtual environments. Proceedings of the 2nd international Conference on Computer
Graphics, Virtual Reality, Visualisation and Interaction in Africa, 53-56.
O’Brien, K. S., & O’Hare, D. (2007). Situational awareness ability and cognitive skills training
in a complex real-world task. Ergonomics, 50, 1064–1091.
O’Neil, H. F., Wainess, R., & Baker, E. L. (2005). Classification of learning outcomes: Evidence
from the computer games literature. The Curriculum Journal, 16, 455–474.
Parchman, S. W., Ellis, J. A., Christinaz, D., & Vogel, M. (2000). An evaluation of three
computer-based instructional strategies in basic electricity and electronic. Military
Psychology, 12, 73–87.
Pillay, H. (2002). An investigation of cognitive processes engaged in by recreational
computer game players: An implication for skills of the future. Journal of Research on
Technology in Education, 34, 336–350.
Prensky, M. (2001). Digital Game-Based Learning. New York: McGraw-Hill
Ravaja, N., Salminen, M., Holopainen, J., Saari, T., Laarni, J., & Järvinen, A. (2004). Emotional
response patterns and sense of presence during video games: Potential criterion variables for
game design. Proceedings of the Third Nordic Conference on Human-Computer interaction,
Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The motivational pull of video games: A
self-determination theory approach. Motivation and Emotion, 30, 347–363.
Rosenberg, B. H., Landsittel, D., & Averch, T. D. (2005). Can video games be used to predict or
improve laparoscopic skills? Journal of Endourology, 19, 372–376.
Rosser, J. C., Lynch, P. J., Cuddihy, L., Gentile, D. A., Klonsky, J., & Merell, R. (2007). The
impact of video games on training surgeons in the 21
century. Archives of Surgery, 142, 181–
Satyen, L., & Ohtsuka, K. (2001). Strategies to develop divided attention skills through video
game training. In D. Harris (Ed.), Engineering psychology and cognitive ergonomics (Vol. 6) (pp.
191 – 199). Aldershot, UK: Ashgate.
Seay, A. F., Jerome, W. J., Lee, K. S., & Kraut, R. E. (2004). Project massive: A study of online
gaming communities. Extended abstracts of the Conference on Human Factors in Computing
Systems (pp. 1421-1424), Vienna, Austria.
Squire, K., Barnett, B., Grant, J. M., & Higginbotham, T. (2004). Electromagnetism
Supercharged! Learning physics with digital simulation games. Proceedings of the international
conference on Learning sciences, 6, 513–520.
Tuzun, H., Yilmaz Soylu, M., Karakus, T., Inal Y., & Kizilkaya, G. (2008, March). The effects of
computer games on primary school students’ achievements and motivation in geography
learning. Paper presented at the meeting of the American Educational Research Association, New
York, New York.
Virvou, M., Katsionis, G., & Manos, K. (2005). Combining software games with education:
Evaluation of its educational effectiveness. Educational Technology & Society, 8(2), 64–65.
Vogel, J. J., Vogel, D.S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006).
Computer gaming and interactive simulations for learning: A meta-analysis. Journal of
Educational Computing Research, 34, 229–243.
Walshe, D. G., Lewis, E. J., Kim, S. I., O’Sullivan, K., & Wiederhold, B. K. (2003). Exploring
the use of computer games and virtual reality in exposure therapy for fear of driving following a
motor vehicle accident. CyberPsychology & Behavior, 6, 329–334.
Waxberg, X. L., Schwaitzberg, S. D., & Cao, C. G. L. (2005). Effects of videogame experience
on laparoscopic skill acquisition. Proceedings of the Human Factors and Ergonomics Society, 49,
Wong, W. L., Shen, C., Nocera, L., Carriazo, E., Tang, F., Bugga, S., & Narayanan, H. (2007).
Serious video game effectiveness. Proceedings of the international conference on Advances in
computer entertainment technology, 49–55.
Wouters, P., Tabbers, H. K., & Paas, F. (2007). Interactivity in video-based models. Educational
Psychology Review, 27, 327–342.
Table 1. Classification of studies in the taxonomy of learning outcomes
Learning outcome Study Game Domain Results Effect
Game vs. control group)
Dede et al. (2005) River City Biology Game > Text +
Squire et al. (2004) Supercharged Science Game > Guided inquiry +
Parchman et al. (2000) King’s Quest V Electronics Game < Computer-based +/–
Wong et al. (2007) Metalloman Biology Game and Hypertext > Text +
Game features
Belanich et al. (2004) America’s Army Military Relevant knowledge better recalled +
Wong et al. (2007) Metalloman Biology Interactivity has no effect
Leemkuil (2006) KMQuest Economy Instructional support not effective
Morris et al. (2004) DeltaForce Military Stress level has no effect on recall
Virvou et al. (2004) VR Engage Geography Game mission more effective +
Beale et al. (2007) Re-Mission Medicine Game type has an effect +
Cognitive skills
Game vs. control group
Barab et al. (2006) QuestAtlantis Writing Game > Traditional +
Dede et al. (2005) River City Biology Game > Text +/
Nullmeyer et al. (2006) Unknown CRM Game > Control +/
Parchman et al. (2000) King’s Quest V Electronics Computer-based > Game = Text +/–
Ke et al. (2007) Astra Eagle Math Game > No game +
Game features
Leemkuil (2006) KMQuest Economy Instructional support not effective
Nelson (2007) River City Biology Guidance not effective
Morris et al. (2004) Delta Force Military High stress yields better mission success +
Feng et al. (2007) MoH, Balance Spatial Cogn. Action game yields better spatial cognition +
Motor skills
Game effect
Rosenberg et al. (2005) Top Spin Surgery Game experience has no effect
Waxberg et al. (2005 Goldeneye Surgery Game experience has no effect
Enochsson et al. (2004) Unknown Surgery Game experience yields better performance +
Grantcharov et al. (2003) Unknown Surgery Game experience yields better performance +
Rosser et al. (2007) Silent Scope Surgery Game experience yields better performance +
Learning outcome Study Game Domain Results Effect
Game effect
Bouchard et al. (2006) Half-Life Phobia Reduction of fear of spiders +
Walshe et al. (2003) London Racer Phobia Reduction on fear of driving +
Ke et al. (2007) Astra Eagle Math Cooperative game > Competitive/ No game +
Game features
Backlund et al. (2007) Unknown Driving Safe driving behavior +
Fischer et al. (2007) Burnout/Fifa Driving Race gamers less cautious than non race gamers +
Game vs. control group
Clarke et al. (2006) River City Biology Game > traditional +
Parchman et al. (2000) King’s Quest V Electronics Enhanced = Game > Drill-practice = Text +/–
Tuzun et al. (2008) Quest Atlantis Geography Intrinsic: Game > traditional +
Extrinsic: Game < Traditional
Game vs. control group
Branninck et al. (2005) Asteroids CRM PC based simulator > Game + Exercises
Nullmeyer et al. (2006) unknown CRM Game = Control
Game features
Nova et al. (2003) Spaceminers Science task performance better with awareness tool +
Galimberti et al. (2001) DOOM II Maze Collaboration: Immersion = nonimmersion +/–
Note: + = an effect (positive or negative) is reported, – = no effect is reported, +/– = results are inconclusive
Table 2. Overview of results by game feature
Game feature Description Study Results Effect
Awareness tool Helping player
activities of other
Nova et al. (2003) An awareness tool
yields higher task
performance than no
awareness tool
Game task A specific task or
mission is involved
or not
Virvou et al. (2005)
Backlund et al.
The task/mission has
a positive effect on
knowledge or attitude
Game type The specific game
type that is involved
(e.g., an action
game or not)
Feng et al. (2007)
Fischer et al. (2007)
Beale et al. (2007)
Game type causes
cognitive processes
to occur or not
Advice/Guidance Advice or guidance
with or without
hints is offered
Leemkuil (2006)
Nelson (2007) Advice or guidance
have no effect
without additional
Assignments Extra assignments
(tasks) are
implemented or not
Leemkuil (2006) Extra assignments
have no effect
Interaction Game allows
choices of the
player or not
Wong et al. (2007) Interaction has no
Level of immersion Effect of head-
mounted display vs.
Galimberti et al.
(2001) No effect, except
immersion took more
time for the task
Level of stress A high or low level
of stress is brought
into the game
Morris et al. (2004) The level of stress
has a positive effect
on cognitive skills,
but no effect on
Relevance The information is
relevant or not for
the game
Belanich et al.
Relevant information
has a positive effect +
Note: + = an effect (positive or negative) is reported, – = no effect is reported,
+/– = results are inconclusive
Figure 1. A taxonomy of learning outcomes.
Learning outcomes
- textual
- non-textual
- problem solving
- decision making
- situational awareness
Motor skills
Figure 2. Cognitive and Affective Level of Complexity (CALC).
... Gameplay participant feedback is also widely used, as it is a relatively inexpensive and quick method for rapid assessment of the immediate impact of the game, and, indeed, is suitably robust when collated over a large number of game workshops (see in [14]). A multi-method evaluation approach is also recognised to fit well with serious gaming in general [15,51]. ...
... While the participants largely enjoyed the engagement process, assessing their personal learning was a more difficult task because of the limitations of the methodology used. To assess the bioeconomy learning experience among the participants, see [14,15,51]. However, the response rate to the follow-up surveys was significantly lower (10%) than the baseline surveys and varied among missions (121 answers to the baseline survey and 41 to the follow-up ones were received in the co-creation phase, for example, the J&C group had the best balance (39/21) and the F&T group had the most baseline surveys (48) but the fewest follow-up surveys filled in (9)). ...
... Moreover, there were some changing participants throughout the process who missed either the baseline or follow-up survey. Considering all this, qualitative feedback received from participants in formative evaluation served as the main input for drawing conclusions, being aware of the limitation that the method was relying on self-reflection of the participants' knowledge, which can give only their own perceived assessment of their learning [14,15,51]. ...
Full-text available
In serious game design, there is a growing trend towards involving players more substantially in the development process. However, several scholars and practitioners agree that still more research and empirical data are needed to guide such participatory prototyping processes. So far, less focus has been put on the influence of co-creation and co-design on participants in the process. This article explores the co-creation and co-design process in creating a serious game called “Mission BioHero”, with a focus on sustainability and the bioeconomy. The game aims to engage citizens in the bioeconomy, help them learn in a fun and interactive way, and aggregate their ideas to shape bioeconomy policies. This serious game was fully co-created and co-designed with citizens and stakeholders from eight regions in five European countries. This research investigates the nature and impact of co-creation and co-design on game development and its driving success factors, explores educational capacities of this participatory process on participants, and evaluates its impact on the co-designed serious game. The findings draw upon empirical data gathered from the co-creation and co-design phases, participants’ feedback, and external game players’ input. The study confirms the value of the participatory design process in serious games design and offers methodologies, tools and lessons learned for similar future processes.
... One of the central challenges of educational video games is helping learners transform intuitive learning into generative learning during game play (Fiorella & Mayer, 2012;ter Vrugte et al., 2015). A possible solution to this challenge is adding learning support (Lee & Ke, 2019;O'Neil et al., 2014;Wouters et al., 2009) that refers to assistance, guidance, or instruction that stimulates or facilitates students' game-based learning (GBL) processes (Tobias 2009;Tobias et al., 2011). For example, in-game references, as well as cues and hints are the two frequently used approaches to delivering the learning support in GBL (Pan et al., 2022). ...
... Students can get stuck and feel frustrated when playing the game. Providing learning support can reduce the cognitive load during information processing in a complex gaming environment, and consequently foster learning (Wouters et al., 2009). ...
... A number of studies provided evidence on the effectiveness of modeling in classroom instruction (e.g., Bentley & Yates, 2017;Cooper & Sweller, 1987). Learners, especially novices, could develop a mental representation of the problem by observing an expert performing a complex task (e.g., Collins et al., 1989;Wouters et al., 2009). Modeling is reported to be most effective in well-structured domains such as mathematics and programming (Bentley & Yates, 2017;Renkl & Atkinson, 2003). ...
Full-text available
Adopting a pretest–posttest experimental design with repeated measures, this study examined the effects of three types of game-based learning supports in the form of modeling on knowledge development that contributed to successful math problem solving and students’ perceived game flow. Forty-one sixth-grade students participated in the study and played a 3D architecture game that aims to promote mathematical conceptual understanding and problem solving skills, and presented with different types of learning supports (i.e., conceptual knowledge only, procedural knowledge only, and the combination of the two). No significant effect of the types of supports was found on participants’ perceived game flow. However, there was a significant impact of support type on participants’ performance on two posttests. The results indicated that the learning support with procedural knowledge only, compared with the learning support with both conceptual and procedural knowledge as well as that with conceptual knowledge only, was significantly more effective in promoting students’ knowledge acquisition related to mathematical problem solving.
... However, while the evidence regarding their educational prowess seems promising (De Freitas, 2018), SGs are not effective by default (Watt & Smith, 2021). For example, meta-analytic research has revealed insignificant study effect sizes when using some SGs in a university context in the past (Lamb et al., 2018), and there is also evidence that they are not necessarily effective in developing motivation and communicative learning outcomes (Wouters et al., 2009), although other studies investigating single SGs have found them to be highly motivating (e.g., Treviño-Guzmán & Pomales-García, 2014) and helpful in terms of developing communication skills (e.g., . Given that SGs are very costly to develop , it is thus important to design them in a way that they achieve their intended learning outcomes (Watt & Smith, 2021), which requires integrating pedagogical considerations into the design process (Ravyse et al., 2017). ...
Full-text available
Serious games (SGs) represent promising digital learning tools. However, SG design frameworks frequently lack a comprehensive integration of pedagogical considerations, which might explain the observed variance in SG effectiveness. To address this research gap, we use a design science research approach to develop a mid-range design theory that translates pedagogy into SG design. We exemplify and empirically evaluate the application of our design theory with an SG designed for intercultural competency development with good results.
... According to Michael and Chen (2006), serious games are a tool for promoting learning and are potential promoters of behavioral change, as they can engage players in interactive and immersive experiences that enhance their motivation and interest in the topic. Wouters et al. (2009) also suggested that serious games can be used to promote the mobilization and/or development of different dimensions of various competencies (e.g., knowledge, skills, attitudes, values, dispositions). Furthermore, serious games can positively impact learners' motivation and engagement. ...
Full-text available
Quality education is an indispensable element for the successful implementation of the 2030 Agenda for Sustainable Development, as it equips all children with the essential skills to promote sustainable development within the context of their lifestyles, in line with the fourth Sustainable Development Goal (SDG4). This will have to be an innovative education, not only in the contents and guidelines to be followed but also in the educational strategies and resources to be used. This article aims to present and describe the methodology used to develop the digital educational resource (DER) "Sustainability at Play," a serious game intended for primary education and based on the concept of the ecological footprint. This DER was developed through educational design research (EDR) by a multidisciplinary team within a project to create Digital Educational Resources for Primary Education. The EDR approach was organized in four phases-Phase I-Problem Analysis, Phase II-Exploration of Possible Solutions, Phase III-Design, and Phase IV-Reflection-allowing for an iterative DER development process shared by different participants (researchers, illustrators, web designers, primary school teachers and students). As a result, this study enabled not only the development of an innovative DER to work on issues related to education for sustainability in primary school but also the understanding and validation of the suitability of the EDR methodology for the development of this type of educational resource.
... Nella ricerca "Current Practices in Serious Game Research: A Review from a Learning Outcomes Perspective" condotta nel 2009 da Wouters, van der Spek e van Oostendorp dell'Institute of Information and Computing Sciences dall'Università di Utrecht nei Paesi Bassi è stata valutata l'efficacia nell'apprendimento dei Serious Games tramite la comparazione con altre metodologie e si è stabilito che i Serious Games migliorano l'acquisizione di conoscenze, le abilità cognitive e motorie, ma è necessario considerare alcune peculiarità del gioco come gli stimoli, il feedback, le difficoltà del gioco stesso (Wouters, 2009). ...
Conference Paper
Full-text available
Il presente contributo affronta i temi della formazione docenti e della esperienza sul campo di docenti di discipline tecniche, legate all’informatica, in istituti di istruzione superiore, settore tecnico tecnologico. Nello specifico vengono trattate le tecnologie e le piattaforme utilizzate durante l’emergenza Covid19 e viene fatto un confronto con le metodologie e tecnologie utilizzate prima dell’emergenza. Verranno trattati gli applicativi utilizzati durante l’emergenza sanitaria e condivise le esperienze didattiche. In conclusione viene presentata una nuova esperienza di didattica legata alla realtà virtuale.
... À l'inverse, il n'est pas rare d'utiliser des jeux dits « sérieux » (ou serious games en anglais) comme leviers des apprentissages des enfants et adolescents. Ces jeux sérieux ont été reliés à des résultats prometteurs concernant les apprentissages académiques (Wouters, van der Spek, & van Oostendorp, 2009) tandis que leurs effets sur les comportements de santé des enfants et adolescents semblent plus nuancés (Bossen et al., 2020 ;Scarle et al., 2011). Dans cette perspective, les JVA ont progressivement émergé en contexte scolaire de la même manière qu'ils ont pu être utilisés à l'armée ou dans les écoles de commerce (Kooiman & Sheehan, 2015). ...
Les outils numériques sont omniprésents dans notre société et occupent aujourd’hui une place centrale à l’école. Dans ce contexte, les jeux vidéo actifs (JVA) peuvent constituer une ressource pour l’enseignement de l’éducation physique et sportive (EPS). L’objectif de cette revue systématique est de synthétiser les études proposant des JVA dans le contexte du cours d’EPS afin d’explorer les différents effets de leur utilisation sur les élèves. Les normes PRISMA ont été utilisées pour identifier les études qui proposaient des JVA en cours d’EPS via les moteurs de recherche Science-Direct, Education Resources Information Center (ERIC), Web of science, Pubmed, PsycINFO, CAIRN et OpenEdition jusqu’à octobre 2021. Les résultats issus des 28 études retenues soulignent que les JVA sont majoritairement reliés à des résultats bénéfiques sur les élèves, sur des effets psychologiques ( e.g. , plaisir) et l’activité physique en classe, en particulier à court terme. Pour autant, certains aspects de la littérature demeurent peu explorés, comme l’effet des JVA sur les apprentissages transversaux, les capacités cognitives, et sur les élèves de degré secondaire. Finalement, l’ensemble de ces résultats confirme que les JVA offrent une opportunité supplémentaire pour les enseignants d’EPS. À la condition d’une utilisation d’outils et de protocoles adaptés, les JVA semblent permettre aux enseignants de répondre à des problématiques précises et ciblées.
Full-text available
Digital educational games exhibit substantial promise in advancing STEM education. Nevertheless, the empirical evidence on both the efficacy of digital game-based learning and its designs in STEM education is characterized by notable inconsistencies. Therefore, the current study aimed to investigate (1) the general effect of digital game-based STEM learning over STEM learning without digital game, and (2) the enhancement effect of added game-design elements against base game versions in STEM learning. Two meta-analyses were conducted in this study. Based on the 136 effect sizes extracted from 86 studies, the first meta-analysis revealed a medium to large general effect of digital game-based STEM learning over conventional STEM learning (g = 0.624, 95% CI [0.457, 0.790]). In addition, digital game-based STEM learning appeared to be differentially effective for different learning outcome, different types of game, and different subject. A total of 44 primary studies and 81 effect sizes were identified in the second meta-analysis. The results revealed a small to medium enhancement effect of added game-design elements over base game versions (g = 0.301, 95% CI [0.163, 0.438]). Furthermore, our results indicated that the game-design elements added for content learning were more effective than those added for gaming experience. Possible explanations for these findings, as well as the limitations and directions for future research were discussed.
This work shows an analysis of the perception and intention to use serious games as teaching-learning strategies in academic processes in higher education. For the analysis, the adaptation of a TAM model is used, analyzing aspects such as ease of use, usefulness, attitude, confidence, and intention of use. As a result, it is evident that there is a low real use of the tools categorized as innovative and that the attitude towards the use of the serious game is mainly explained by the perceived confidence and the ease of use, being the attitude, in turn, the most important antecedent of the intention to adopt these strategies.
Full-text available
This NSF-funded project utilizes graphical multi-user virtual environments (MUVEs) as a vehicle to study (1) classroom-based situated learning and (2) the ways in which virtual environments may aid the transfer of learning from classroom contexts into real world settings. In the project's River City curriculum, teams of middle school students are asked to collaboratively solve a simulated 19 th century city's problems with illness, through interaction with each others' "avatars," digital artifacts, tacit visual and auditory clues, and computer-based "agents" acting as mentors and colleagues in a virtual community of practice. In this paper, we provide an overview of results from a large-scale implementation of the River City environment and curriculum in Spring 2004. Our findings show that students and teachers were highly engaged, that student attendance improved, that disruptive behavior dropped, and that interesting patterns are emerging about which students do best under our various pedagogical conditions. A major goal for education in the 21st century is to create scientifically literate citizens who are able to think critically, make sense of complex data, and solve problems (AAAS, 1993; NRC, 1996). Research suggests that, if all students are to become scientifically literate, science instruction must convey greater engagement and meaning to them. To achieve this, science instruction in secondary schools must provide students with opportunities to explore the world, to apply scientific principles, to sample and analyze data and to make connections among these explorations, their personal lives, and their communities. However, given the constraints of classroom settings, real world data collection is challenging to orchestrate. Due to safety issues and lack of equipment, laboratory experiments are also difficult to conduct. It is no surprise that educators report teaching higher order inquiry skills (such as hypothesis formation and experimental design) is among the most difficult challenges they face with students who have a history of low achievement in and engagement with science. With NSF funding, we are creating and studying graphical multi-user virtual environments (MUVEs) to enhance middle school students' motivation and learning about science and society ( MUVEs enable multiple simultaneous participants to access virtual contexts, to interact with digital artifacts, to represent themselves through "avatars," to communicate with other participants and with computer-based agents, and to enact collaborative learning activities of various types. Our "River City" MUVE is centered on higher order scientific inquiry skills, as well as on content related to national standards in biology and ecology.
Full-text available
Following up on an earlier issue of The Curriculum Journal (Vol. 16, No. 1), this article focuses on learning outcomes in the context of video games. Learning outcomes are viewed from two theoretical frameworks: Kirkpatrick's levels of evaluation and the CRESST model of learning. These are used to analyse the outcomes claimed in journal articles that report empirical work, indicating the usefulness of the frameworks, and the necessity to consider the role of affective learning. The article ends with some comments on the relationship of instructional design to effective games and learning outcomes.
Full-text available
In 1999, Doug Church proposed the use of formal abstract design tools for game design [3]. Part of Church's suggestion,was,to develop,a common,design,vocabulary.,It's ironic that while the game,design,community,has,started,to develop,these,more,rigorous,design,principles,for games, there is much confusion even about the most basic of questions, such as what a game is, compared to a story or a simulation. This confusion only increases when we start to consider new and emerging forms like mobile games, location-based games and pervasive games. It's obvious that we need some basic distinctions and definitions at the highest level, so that more detailed,methods,can,be sorted,into their appropriate,areas,of application. Developing,a basic language,for describing,different,types,of games,requires,different
Full-text available
This work was designed to examine the effects of contextually relevant stress on personal computer (PC)-based game training. Off-the-shelf PC-based games are being applied to many training situations because of their affordability, flexibility, and teaming capabilities. The ultimate purpose of training is to transfer superior performance to the real world. In this respect, 1 of the major drawbacks to using games as training tools, especially for military applications, is the absence of the surrounding context. In response to this omission, we examined the effects of adding context-relevant stress to infantry game-based training by exposing 1 group of participants to a graphically intense and stressful experience while the control group viewed an unstressful analog. Pre-post self-reported stress levels confirmed the efficacy of this manipulation. The stress condition produced significantly higher scores on "mission success"; however, no differences were evident in participants' use of trained tactics or game functions. Supplementing context-relevant stress in game training shows promise for enhancing individuals' motivation to succeed.
Full-text available
Job requirements for US Navy electronic technicians include extensive knowledge of basic electricity and electronics (BE/E) fundamentals. Historically the BE/E material has proved difficult for trainees to learn and has resulted in high setback and attrition rates. This study evaluates alternative computer-based instructional strategies for teaching complex technical content with 4 instructional conditions: computer-base drill and practice (CBDP), enhanced computer-based instruction (ECBI), a computer-based adventure game (GAME), and the existing classroom instruction (CI). The authors evaluated trainees on completion of the instruction with a schoolhouse comprehensive test, a specially designed cognitive skills test, and a motivation questionnaire In addition, the authors recorded and analyzed the time participants took to complete the instruction. In general, the CBDP and ECBI groups outperformed the CI and GAM groups on all measures. The GAME condition performed no better than the CI condition. When differences existed between the ECBI and CBDP groups, the ECBI group performed better. (PsycINFO Database Record (c) 2013 APA, all rights reserved)
This exploratory study investigated the influence of two recreational computer games on children’s subsequent performance on computer-based instructional tasks. Children were assigned to three groups: two were invited to play their respective recreational computer games, and the third acted as a control group. All three groups then worked on a common set of educational tasks from environmental education software. The three groups’ performances on a set of educational tasks were compared using quantitative analysis for speed and correct solutions, and then qualitatively for the cognitive manoeuvres engaged in to accomplish the tasks. The findings suggest that playing recreational computer games may influence children’s performance on subsequent computer-based educational tasks. However, the extent of this influence depended on how closely the recreational computer game types matched the design of the tasks in the educational software. The cognitive manoeuvres used by game players also depended on the types of games played during the learning phase. Linear cause-and-effect games tended to encourage means–end analysis strategy, whereas adventure games encouraged inferential and proactive thinking. Though the findings of this study are encouraging, further studies need to be undertaken to replicate the results.
The apprenticeship model used to teach surgical residents is no longer adequate, especially in laparoscopic surgery training. The other alternatives available, such as simulators or animal models, can be expensive and difficult to implement. This study was conducted to explore the effect of videogame experience on surgical skill acquisition. We hypothesized that a week of videogame playing would improve performance on a surgical skills trainer and that performance on the videogame would reflect performance on the trainer. Thirty participants were tested in a between-subjects mixed design. Results were inconclusive. However, the use of videogames for training may be justified given the minimal cost. The idea of training surgeons using an inexpensive technology that is familiar, and that is fun and engaging at the same time, has considerable potential for the field of training in surgery.