Conference PaperPDF Available

When Playing Meets Learning: Methodological Framework for Designing Educational Games

Authors:

Abstract and Figures

Game-based learning builds upon the idea of using the motivational potential of video games in the educational context. Thus, the design of educational games has to address optimizing enjoyment as well as optimizing learning. Within the EC-project ELEKTRA a methodological framework for the conceptual design of educational games was developed. Thereby state-of-the-art psycho-pedagogical approaches were combined with insights of media-psychology as well as with best-practice game design. This science-based interdisciplinary approach was enriched by enclosed empirical research to answer open questions on educational game-design. Additionally, several evaluation-cycles were implemented to achieve further improvements. The psycho-pedagogical core of the methodology can be summarized by the ELEKTRA’s 4Ms: Macroadaptivity, Microadaptivity, Metacognition, and Motivation. The conceptual framework is structured in eight phases which have several interconnections and feedback-cycles that enable a close interdisciplinary collaboration between game design, pedagogy, cognitive science and media psychology.
Content may be subject to copyright.
When Playing Meets Learning: Methodological
Framework for Designing Educational Games
Stephanie B. Linek
1
, Daniel Schwarz
2
, Matthias Bopp
3
,
Dietrich Albert
1
1
Institut of Psychology, University of Graz, Universitätsplatz 2,
8010 Graz, Austria
stephanie.linek@uni-graz.at, dietrich.albert@uni-graz.at
2
Takomat GmbH, Neptunplatz 6b,
50823 Cologne, Germany
dan@takomat.com
3
Center for Advanced Imaging, Brain Research Institut, University of Bremen,
Hochschulring 18,
28359 Bremen, Germany
Matthias.Bopp@t-online.de
Abstract. Game-based learning builds upon the idea of using the motivational
potential of video games in the educational context. Thus, the design of
educational games has to address optimizing enjoyment as well as optimizing
learning. Within the EC-project ELEKTRA a methodological framework for
the conceptual design of educational games was developed. Thereby state-of-
the-art psycho-pedagogical approaches were combined with insights of media-
psychology as well as with best-practice game design. This science-based
interdisciplinary approach was enriched by enclosed empirical research to
answer open questions on educational game-design. Additionally, several
evaluation-cycles were implemented to achieve further improvements. The
psycho-pedagogical core of the methodology can be summarized by the
ELEKTRA’s 4Ms: Macroadaptivity, Microadaptivity, Metacognition, and
Motivation. The conceptual framework is structured in eight phases which have
several interconnections and feedback-cycles that enable a close
interdisciplinary collaboration between game design, pedagogy, cognitive
science and media psychology.
Keywords: Game-based learning, methodology, microadaptivity, media
psychology.
1 Introduction
Game-based learning is a relatively new research area and so far there exist no
concrete systematic recommendations for the conceptualization of an integrated
design of educational games.
In the following, a newly developed conceptual framework for the creation of
educational (adventure-)games will be outlined and illustrated by several concrete
examples and empirical (evaluation) studies. The proposed methodology was
developed and successfully used in the EC-project ELEKTRA (Enhanced Learning
Experience and Knowledge Transfer). The described process can serve as a model for
other contexts of game based-learning as well as the creation of serious games.
1.1 When Playing Meets Learning: The Appeal of Game-Based Learning
Game-based learning rests upon the idea of using the motivational and immersive
potential of conventional video games in the educational context. Even though there
are several publications on games [1], game-play [2], and game-based learning [3],
the divers contributions are often rather unconnected and an overall framework for the
creation of educational games is still missing.
The main problem in this context is the seldom collaboration between psycho-
pedagogical scientist and industrial game designers. For an appropriate serious game
design, both, the creativity of game designers as well as the expertise of psycho-
pedagogical scientists are necessary. A first step in this direction was made within the
EC-project ELEKTRA which will be described in the next subchapter.
1.2 Interdisciplinary Research-Project ELEKTRA
ELEKTRA (Enhanced Learning Experience and Knowledge Transfer) was an EC-
project under FP6 on game-based learning. The aim of this interdisciplinary research
project was twofold: On the one hand it aimed at the development of a state-of-the-art
educational adventure-game to make learning as exciting as leading-edge computer
games. For this practical aim the so-called ELEKTRA-demonstrator was developed
which comprises the first chapter of an educational adventure-game on the learning
domain physics/optics. On the other hand a general methodology about the conceptual
design and production of digital learning games should be established. This second
aim was accomplished by the ELEKTRA methodology which will be described in
this article.
The core idea of producing effective and motivating digital game-based e-learning
experiences for young children relies on an interdisciplinary approach which
combines state-of-the-art research in cognitive science, pedagogical theory and
neuroscience with best industrial practice in computer game design. The developed
methodology builds not only a framework for structuring and supporting the
interdisciplinary cooperation, but also inherent several interrelated phases and
evaluation-cycles that enable continuous improvements and enhancements of the
educational game design.
1.3 The ELEKTRA Methodology: Overview
On a general level, the ELEKTRA methodology does not reinvent the wheel but
shares a lot of elements with usual instructional design models that many readers
might be familiar with (e.g., [4]). In particular the proposed methodology can be seen
as an adaption of the Dick and Carey System Approach Model [5] revised for the
purpose of making a state-of-the-art digital learning game.
The base of the developed methodology can be summarized by the ELEKTRA’s
4Ms: Macroadaptivity, Microadaptivity, Metacognition, and Motivation. Within the
ELEKTRA-project we identified these 4Ms as the pivotal elements of an (exciting)
educational game (independent of the concrete learning content and storyline/genre of
the game). In order to manage the workflow within the interdisciplinary collaboration
a framework with eight phases was developed:
Phase 1: Identify instructional goals
Phase 2: Instructional analysis
Phase 3: Analyze learners and context of learning
Phase 4: Write performance objectives and overall structure of the game
Phase 5: Learning game design
Phase 6: Production and development
Phase 7: Evaluation of learning
Phase 8: Revise instruction
RES 422
Identify
instructional
goals
Instructional
analysis
- Create Knowledge
structure
Analyse
learners and
context of
learning
- User requirements
-User preferences
- User‘s entry skills
Write performance
Objectives + overall
structure of the game
Evaluation of
learning
- User validation
- Scientific
validation design
- Testing
(functional +
pedagogical)
Revise instruction
- Analyse user validation
- Recommendations
- Revise and update:
Instructional goals, user requirements, instructional analysis, Learning Game Design, Production and Development via RAD-approach
Learning Game Design
Didactic
Design
- LeS
(Macro-
adaptivity,
Metacognition,
Microadaptivity
(pedagogical rules,
adaptive elements))
Design story-
based game
world
- Game world and
mechanics
- GpS
- Background story
- Characters
- StS
- Assessment +
Validation
instruments
(e.g. Logfiles)
- Microadaptivity
(Update learner
model, Assess-
ment, adaptivity)
What to learn How to learn How to make
CONCEPTION
How to
evaluate
DESIGN PRODUCTION
DEVELOPMENT VALIDATION
CONTENT LEARNING TECHNOLOGY
T
L
C
1
2
3
4
5
Production and
Development
- Technology
development
- Content production
- Game release
67
8
M1 M4
Design In-Game
Assessment
M2
M3
- Game chapters
- Categorize learning
Objectives
- Triple consistency
- Learning methods
- Performance structure
RES 422
Identify
instructional
goals
Instructional
analysis
- Create Knowledge
structure
Analyse
learners and
context of
learning
- User requirements
-User preferences
- User‘s entry skills
Write performance
Objectives + overall
structure of the game
Evaluation of
learning
- User validation
- Scientific
validation design
- Testing
(functional +
pedagogical)
Revise instruction
- Analyse user validation
- Recommendations
- Revise and update:
Instructional goals, user requirements, instructional analysis, Learning Game Design, Production and Development via RAD-approach
Learning Game Design
Didactic
Design
- LeS
(Macro-
adaptivity,
Metacognition,
Microadaptivity
(pedagogical rules,
adaptive elements))
Design story-
based game
world
- Game world and
mechanics
- GpS
- Background story
- Characters
- StS
Design story-
based game
world
- Game world and
mechanics
- GpS
- Background story
- Characters
- StS
- Assessment +
Validation
instruments
(e.g. Logfiles)
- Microadaptivity
(Update learner
model, Assess-
ment, adaptivity)
What to learn How to learn How to make
CONCEPTION
How to
evaluate
DESIGN PRODUCTION
DEVELOPMENT VALIDATION
CONTENT LEARNING TECHNOLOGY
TT
LL
CC
11
22
33
44
55
Production and
Development
- Technology
development
- Content production
- Game release
6
Production and
Development
- Technology
development
- Content production
- Game release
66 77
88
M1M1 M4M4
Design In-Game
Assessment
M2M2
M3M3
- Game chapters
- Categorize learning
Objectives
- Triple consistency
- Learning methods
- Performance structure
Fig. 1. Overview on the eight phases of the model.
Even though these phases are numbered from one to eight, they do not follow a
linear order but have several interconnections and feedback cycles. Figure 1 illustrates
the workflow within the eight phases of the model.
The ELEKTRA’s 4Ms are mainly addressed in phase 5 which can be suggested as
the core of the methodology: the learning game design. But also the other phases
relate to the 4Ms in an implicit way: The phases before feed in the learning game
design, the succeeding phases rely on the learning game design and its
implementation and improvements, respectively.
In the following, first the ELEKTRA’s 4Ms will be characterized. Second, the
eight phases will be described; thereby the focus lies on the psychological
contribution within this framework. Several practical and empirical examples from
the ELEKTRA-project will be given. Finally, a short resume will be provided.
2 Base of the Methodology: ELEKTRA’s 4Ms
The ELEKTRA’s 4Ms include the pivotal features of a successful educational game.
The headwords Macroadaptivity, Microadaptivity, Metacognition, and Motivation are
only rough catch phrases for various elaborated concepts, models and findings.
Within the ELEKTRA project the main psychological contributions regard to
microadaptivity and motivation. The work on macroadaptivity and metacognition was
mainly part of the pedagogical partners.
2.1 M1 - Macroadaptivity
Macroadaptivity deals with the adaptive pedagogical sequencing of alternative
learning situations for one learning objective. Thereby macroadaptivity refers to the
instructional design and management of the available learning situation. It addresses
the adaptivity between different learning situations and refers also to a diversification
of learning based on Bloom’s taxonomy [6].
The macroadaptive process leads to the creation of a learning path which
represents a specific combination of divers learning situations. The diversity of
learning situations should strengthen deeper cognitive processing and foster long-term
knowledge gain.
2.2 M2 - Microadaptivity
Microadaptivity regards to adaptive interventions within a learning situation. It
involves a detailed understanding of the learner’s skills and a set of pedagogical rules
that determine the interventions given to the learner. Within ELEKTRA the idea
behind the concept of microadaptivity [7] is to develop a system that provides hints
adapted on the user’s (current) knowledge and competence state.
Whereas macroadaptivity refers to traditional techniques of adaptation such as
adaptive presentation and adaptive navigation on the level of different learning
situations microadaptivity deals with the adaptivity within a single learning situation.
The basis of the microadaptive skill assessment and the non-invasive interventions
is a formal model for interpreting a learner’s (problem solving) behavior. To realize
the non-invasive skill-assessment and the adaptive interventions, ELEKTRA relies on
the formal framework of the Competence-based Knowledge Space Theory (CbKST;
[8], [9], [10]). Originating from conventional adaptive and personalized tutoring, this
set-theoretic framework allows assumptions about the structure of skills of a domain
of knowledge and to link the latent skills with the observable behavior.
Microadaptivity in this context means that the intervention/hint was selected on the
basis of knowledge assessment via CbKST in a modified version. This microadaptive
assessment procedure assigned a set of required skills as well as a set of missing (non-
activated) skills to each feasible action of the player. Given the player’s action (e.g.,
positioning an object), the likelihoods of the competence states which contain the
associated required skills are increased (one updating step for each skill in the set of
required skills). Analogously, the likelihoods of the competence states which required
the associated missing (non-activated) skills are decreased (one updating step for each
skill in the missing skill set). Thereby, in the microadaptive assessment routine each
player’s action is followed by a sequence of updates. The likelihoods of the skill
states are modified consecutively for each required skill and each missing skill. These
probabilities of the required skills and missing skills build up the basis for the
microadaptive intervention/hint.
LearningSituation
Position_Category
Objects
Competence
CompetenceState
Learner
CompetenceSet
Related_Objects* in_les* SkillSets_Required*
position_category* poscat_related_object*
poscat_skills_missing* poscat_skills_required*
Includes_Skills*
Skills_Taught*
Has_Skill*
Has_Prerequisite*
incl_skills*
has_skillstate*
Fig. 2. Microadaptivity – integrated model.
The chosen hint provides either the necessary information to solve the problem (to
learn a missing skill) or the affective support (e.g., motivating or activating feedback)
fitting the current progress state of the learner assessed by his/her action history. In
order to provide the appropriate kind of intervention, not only the knowledge
assessment but also pedagogical rules are taken into account.
Within the ELKTRA-demonstrator the microadaptive interventions are presented
by a non-player character (NPC) named Galileo in order to merge microadaptivity
with the storyline and the overall game play. The NPC Galileo provides the
intervention, usually in the form of a comment or a question, in accordance with the
game story. If for example, the learner needs motivational support, Galileo
encourages him/her by use of the story elements like the rescue of the female pretty
NPC named Lisa.
Fig. 3. Using the non-player character named Galileo for providing microadaptive interventions
2.3 M3 - Metacognition
Regarding Flavell [11 (p. 232)] “Metacognition refers to ones knowledge concerning
one‘s own cognitive processes or anything related to them, e.g., the learning-relevant
properties of information or data”. Even though there exists slightly different
interpretations of this original definition, metacognition is agreed to involve
knowledge about one’s own knowledge as well as knowledge about one’s own
cognitive processes. The ability of the ELEKTRA-demonstrator to foster
metacognitive development was considered as a major challenge and an important
differentiator compared to traditional educational games.
The integration of a reflective pause in the game-based learning process seems at
first sight contradicted to storytelling and the flow of game play. Within ELEKTRA
the resolution to this dilemma is based on two pillars: First, the implementation of
certitude degrees, i.e., while performing a task, the learner has to indicate the
prudence and confidence he/she has in his/her performance. Second, a firm support of
this kind of metacognition by the storytelling, i.e., the prudence and confidence
estimation were made in a close parasocial dialog with the NPC Galileo.
The metacognitive reflections are therefore tightly bound to the gaming process.
Thereby the ELEKTRA-demonstrator contributes to develop not only the ability to
perform, but also to understand the conditions of success, and thus, having cognitive
and metacognitive goals in addition to the pure performing goal.
2.4 M4 - Motivation
The fourth M named Motivation comprises several motivational concepts and related
approaches used for enjoyment and learning. Motivation in this sense is only a
keyword for different aspects of the storyline, the challenges and skills (flow-
experience), the intrinsic motivation of the gamer, the parasocial interaction and
empathy with the NPCs as well as the identification with the avatar.
In general, motivation is a phrase used to refer to the reason(s) for engaging in
certain activities. In the context of learning games, the creation of motivation to
engage in and perform learning activities is a core element of good game design and
can be suggested as the major advantage of educational games compared to other
ways of e-learning.
There are many aspects of games which are suggested to contribute to the gamers
motivation [1], e.g., competition, parasocial interaction with the NPCs, fantasy,
escapism, suspense or curiosity as well as the balance between challenges and skills
(enabled by different game-levels) which in turn fosters the so-called flow-experience
[12]. Within ELEKTRA we mainly focussed on the storyline and the game characters
as motivational tools for learning. This includes the creation of a story that adds
“sense” to specific learning activities, i.e., the learning activities are an integrative
part of the story itself. Thereby the story confronts the player with certain game-
challenges/problems (e.g., riddles) that he/she can only solve when he/she first learns
certain skills and the story makes it worth to do so.
Fig. 4. Riddle within the ELEKTRA-demonstrator: Solution requires knowledge on optics.
A typical example would be that the learning activities influence the fate of the
avatar or the good and bad NPCs. The crucial issue is to merge learning activities and
storyline in a playful way. The usage of the storyline (including game-characters) as a
motivational tool comprises several subtasks: designing a setting and a general plot,
an avatar with which the players can easily identify as well as designing interesting,
authentic good and bad game-characters with which the players can have an
immersive parasocial interaction. Especially the latter point is of crucial importance
since the game-characters are not only essential for the storyline but also a pivotal
source of motivation. Thus, in ELEKTRA, we paid special attention to this issue. On
the one hand we conducted several multimedia-studies on the design of the game-
characters (which will be partly described in section 3). On the other hand for each of
the involved game-characters a biographical background and a personality description
were created in order to have authentic and credible game-characters that behave in a
(psychological) coherent way throughout the whole game/storyline.
3 Description of the Eight Phases
In the following the eight phases will be described. Like mentioned above it is
important to note, that these phases don’t follow a simple linear order but rather
comprise several interconnections and feedback cycles (see also figure 1).
The psycho-pedagogical contributions within ELEKTRA regarded mainly to the
phases of instructional analysis, the analysis of the learners and the context of
learning, the learning game design and the evaluation phase. For these phases
concrete practical and empirical examples will be given to illustrate the important part
of cognitive science and media psychology in the conception of educational games.
3.1 Phase 1: Identify Instructional Goals
In this early stage, pedagogy clearly prevails the overall game design by setting some
fundamental pedagogical and didactical decisions with respect to the chosen learning
goals, the basic areas of learning content and the general pedagogical approach. The
context of the game has to be outlined as well: Should the learning game be deployed
in a class-room situation at school or should it be played at home as a spare-time
activity? This decision is another important cornerstone for the general conditions of
the whole design of the learning game.
After the definition of learning goals, topic, target group, learning content,
pedagogical approach and the context, the general framework of the game is settled.
This pedagogical framework not only constitutes the learning experience in the game,
but also has got a fundamental impact on the overall concept of the game design. The
choice of the game genre is the first crucial design decision which is directly
dependent on the learning objectives. If someone like to create for example a strategic
simulation game, he/she would perhaps choose different types of learning goals than
for a racing game.
3.2 Phase 2: Instructional Analysis
In phase 2 the learning objectives and the related learning content are transferred into
a formal knowledge structure which is called knowledge space. The theoretical
background and mathematical-formal framework is delivered by the already
mentioned CbKST. In this context, the main advantage of the CbKST is the clear
distinction between observable behavior and the underlying skills and their
interrelationships. Thereby the prerequisite relations between skills as well as between
overt behaviors enable the adaptation to the actual available skills of the learner as
well as the adaptation to the ongoing learning progress.
In the established knowledge space all of the learning objectives are represented as
an ontology of skills. Thereby the accordingly skills are structured as a map that
allows analyzing the developing knowledge state of the learner and thus a learner
model. In addition, it allows adapting the game environment to the individual learning
needs of the player. This can take place on different levels, e.g., on the level of
macroadaptivity or on the level of microadaptivity.
3.3 Phase 3: Analyse Learners and Context of Learning
Phase 3 contributes to the detailed analysis of the learners and the context of learning.
Thereby the characteristics of the learner group concerning entry skills, learning
problems, preferences and attitudes are determined. In a learning game, these areas
refer to the learning process as well as to the game play [13]. Thereby the twofold role
of the target user has to be taken into account: he/she is both, a learner and a player.
Entry skills for the learner could be known difficulties in the chosen learning
topic. Additional, entry skills for the player could be the state of his/her game literacy.
The learner analysis serves as input for a variety of game decisions: For example
the NPC design, the visual style of the game, and the provision of specific learning
methods. It is also used to determine the initial state of the learner model. These
decisions could be partly made by help of existing literature and research findings.
However, with respect to the concrete game design partly additional empirical studies
might be necessary.
For example within ELEKTRA a focused multimedia study on the NPC-design
(regarding his friendliness, the naturalism of the graphics, and the role of color) was
conducted. The results of this so-called NPC-study indicate a clear preference for a
colored, naturalistic NPC-design. For the NPC’s friendliness the pupils favor a NPC
that was similar to their own, indicating similarity-attraction [14]. However, regarding
that last finding, the data indicate also gender differences in a way, that similarity-
attraction hold true for female players only, whereas male players showed a general
preference for the more unfriendly version of the NPC.
3.4 Phase 4: Write Performance Objective and Overall Structure of the Game
On the basis of phases 1 to 3, performance objectives are laid out and, closely linked
to this, the overall pedagogical structure of the game is written. This basic scenario is
a kind of working paper which will go through various changes throughout the
continuing revising process of the creation for the game. In particular the overall
pedagogical structure should include a general description of the story of the game
(including the setting, the characters, and the plot), the game-chapters as well as
various situations of the game that build up the chapters. They are described in a
rough way which includes their main functionality within the game and their possible
sequences which can include adaptive branches.
3.5 Phase 5: Learning Game Design
Phase 5 is the very core of the ELEKTRA methodology and the accordingly design of
a learning game. It is the central work phase where the successful integration of
learning and gaming takes place and everything comes together. The main task in this
phase is to develop detailed descriptions of each situation in the game: Learning
situations (LeS), gameplay situations (GpS), and storytelling situations (StS). Every
situation must be described in terms of stage, possible actions, and events that happen
in the environment in reaction to the player’s activities. The output is a “Game Design
Document” which gives programmers (development) and artists (content production)
precise instructions for the development and production of the educational game.
The challenge of this design process is to design those three types of situations in
such a manner that they constitute pedagogical valid learning activities that are
embedded in a meaningful and exiting learning game experience for the player. In an
ideal learning game experience the three essential situation types work together as
ingredients of a new experience which would arrange a superior game situation from
games, learning, and storytelling. This ideal is not always achievable but at least the
gameplay situations, learning situations and storytelling situations should motivate,
amplify and legitimate each other by embedding them into a meaningful context.
1
Game authoring
Pedagogy
Cognitive Science
Game Design
Design + Arts
An
Ideal user experience
in a digital learning game would
comprise a mutual pervasion of the
three identified situational
components (Learning situation,
Story situation, Gameplay situation)
in ONE LEARNING GAME-Situation
(LGS):
Learning
Situation
Story
Situation
Game play
Situation
Digital learning game
as a whole
In Game
Assess-
ment
NEURO-
SCIENCE
USER-
TESTING
New Learning experience Validation of Learning
HOME-
WORK
SCHOOL
Transfer of Learning
Learning
Situation
Fig. 5. The ideal learning game situation.
The conceptual tools for the design of the situations and their sequencing are
based on the already described ELEKTRA’s 4Ms. Thereby Macroadaptivity,
Microadaptivity, and Metacognition are mainly concepts of the instructional strategy
of the learning situations and the appending in-game assessments, while Motivation is
rather the objective of the story-based game world.
3.6 Phase 6: Production and Development
There are two main work areas in the production and development phase: On the one
hand programmers develop the various technologies required for the game, on the
other hand, artists and producers create all the media assets that are necessary to build
the game world. Roughly spoken, one can say, that the development team works on
the logic of the game while the production team creates the data for it.
The necessary input for the development team and the content production team are
the pedagogical scenarios written in phase 4 and the Game Design Document of phase
5. During phase 6 there is a vivid exchange between the programmers of the
development team and the artists and producers of the content production team. The
outcome of this phase is a published release version of the game that can be tested,
played and evaluated.
3.7 Phase 7: Evaluation of Learning
There are two different forms of evaluation: the formative evaluation and the
summative evaluation of the game.
The formative evaluation is also called testing and is closely connected with the
development and production work in phase 6. Ideally, the formative evaluation should
take place in (monthly) timeboxes when a new testable version of the game-prototype
with the latest implementations and improvements is delivered (as output of phase 6).
This iterative timebox releases will undergo each time a functional and psycho-
pedagogical testing. The formative evaluation can concentrate on single game-
elements like background-music or game characters or might deal with the
implementation of a new approach like microadaptivity in ELEKTRA [15]. The
evaluation results of this testing will directly feed back into early phases. Thereby, the
report on technical testing describes functional bugs that manifest themselves in
mistakes of the game system. The programmers then have to correct or change the
according software components. The report of the psycho-pedagogical testing relates
to gaming and learning experiences of the target end user. The results of the psycho-
pedagogical evaluation forces sometimes even to go back to the design phase 5.
The summative evaluation can be described as a general evaluation of the
developed game and the whole process. It takes place when the iterative technical
testing leads to a stable running and psycho-pedagogical meaningful version of the
game. In order to analyze the learning behavior and success of the pupils in the game
and their evaluation of the gaming experience as a whole, a science-based
methodology is applied, using standardized questionnaires as well as logfile-
information. In this context not only control variables and pre-questionnaires are
considered, but also long-term effects of the learning-game experience should be
assessed (e.g., to assess the long-term knowledge gain).
For both, the formative as well as the summative evaluation, it is advantageously
to use an integrative assessment approach by using logfile-information as well as
questionnaire data (like it was done in ELEKTRA). Since these methods deliver
complementary data of diverse informational impact their combination provides a
more holistic view of the evaluated game and the associated psychological processes
[15].
3.8 Phase 8: Revise Instructions
Subsequent to the game testing and the empirical summative evaluation, the next
essential step is to interpret and exploit the evaluation results for providing
recommendations for improvements and enhancements of the learning game as a
whole. These recommendations have to be feed in all preceding phases, affecting all
previous tasks and activities and hence, might resulting in a revision and update of the
instructional goals (phase 1), instructional analysis (phase 2), user requirements and
preferences (phase 3), learning game design (phase 5) as well as production and
development (phase 6).
Moreover, also the implementation of the evaluation itself might befall revision,
e.g., in case of an emerging need for improving the assessment instruments or
enhancement of the questionnaires. This in turn requires a close collaboration
between scientific research and evaluation. Accordingly, research partners are
responsible for selecting scientific proven evaluation instruments as well as for
proposing an adequate methodology and data-analysis.
4 Conclusions
The proposed methodology delivered a general conceptual framework for the creation
of a broad spectrum of educational games. The applicability and validity of the
methodology was firstly proven within the EC-project ELEKTRA. The ELEKTRA-
demonstrator was evaluated empirically and proved its effectiveness for enjoyment as
well as for learning. Besides this first positive evidence of the effectiveness of the
proposed methodology, also the newly developed micropadaptivity-formalism was
successfully tested in several empirical pilot-studies (Linek, Marte, & Albert, 2008).
The proposed ELEKTRA methodology can be suggested as a first framework for
designing a broad spectrum of educational games. The framework is flexible and open
for new technical developments and possibilities and bears the potential to integrate
new scientific psycho-pedagogical concepts. Accordingly, the described methodology
can be suggested as an open framework that can be adapted to the concrete needs and
aims of game-designers, scientists and the target end users.
Acknowledgments. This paper is part of the ELEKTRA-project funded by the Sixth
Framework Programme of the European Commission’s IST-Programme (contract no.
027986). The author is solely responsible for the content of this paper. It does not
represent the opinion of the European Community.
Thanks to the ELEKTRA-team for the inspiring interdisciplinary work!!!
References
1. Vorderer, P., Bryant, J.: Playing video games. Motives, responses, and consequences. New
Jersey: Lawrence Elbaum Associates (2006).
2. Salen, K., Zimmerman, E.: Rules of play. Game design fundamentals. Cambridge, MA:
MIT Press (2004).
3. Prensky, M.: Computer games and learning: Digital game-based learning. In J. Raessens &
J. Goldstein (Eds.). Handbook of computer game studies, pp. 97-122. MIT Press Cambidge,
Massachusetts London, England (2005).
4. Brown, A., Green, T. D.: The essentials of instructional design. Connecting fundamental
principles with process and practice. Upper Saddle River NJ: Pearson/Merrill Prentice Hall
(2006).
5. Dick, W., Carey, L., Carey, J. O.: The systematic design of instruction. 6
th
ed. Boston:
Pearson/Allyn & Bacon (2005).
6. Bloom, B.: Taxonomy of educational objectives, Handbook I: The cognitive domain. New
York: David McKay Co Inc (1956).
7. Albert, D., Hockemeyer, C., Kickmeier-Rust, M. D., Pierce, N., Conlan, O.:
Microadaptivity within complex learning situations – a personalized approach based on
competence structures and problem spaces. Poster presented on the International
Conference on Computers in Education (2007).
8. Albert, D., Lukas, J. (Eds.): Knowledge spaces: theories, empirical research, and
applications. Mahwah: Lawrence Erlbaum Associates (1999).
9. Doignon, J. P., Falmagne, J. C.: Knowledge spaces. Berlin: Springer (1999).
10. Korossy, K.: Extending the theory of knowledge spaces: A competence-performance
approach. Zeitschrift für Psychologie, Vol. 205, pp. 53-82 (1997).
11. Flavell, J. H.: Metacognitive aspects of problem solving. In L. B. Resnick (ed.), The nature
of intelligence. (pp. 231-236). New Jersey: Lawrence Erlbaum Associates (1976).
12. Csikszentmihalyi, M.: Flow: The psychology of optimal experience. New York: Harper &
Collins (1990).
13. Linek, S.: Creating flow in game-based learning: Threefold conception of challenges and
skills. Proceedings of the International Technology, Education and Development
Conference (2007).
14. Linek, S., Schwarz, D., Hirschberg, G., Kickmeier-Rust, M., Albert, D.: Designing the non-
player character of an educational adventure-game: the role of personality, naturalism, and
color. Proceedings of the International Technology, Education and Development
Conference (2007).
15. Linek, S. B., Marte, B., Albert, D.: The differential use and effective combination of
questionnaires and logfiles. Proceedings of the International Conference on Interactive
Computer Aided Learning, Special Track “Computer-based Knowledge & Skill
Assessment and Feedback in Learning Settings” (2008).
... But, without a basic understanding of how to teach a skill effectively, designing a game based on skill development can be unsuccessful. In this paper, we incorporate skill development theory ( Romizowski,2009) with the educational game development methodology proposed by Linek, Schwarz, Bopp and Albert (2010), which provides a framework for game design. The following questions guided the approach to this topic: ...
... • How may skills development theory and the 4M's Method of designing Game by Linek et al. (2010) be integrated in order to build a framework for teaching skills using instructional games? ...
... In response to the lack of an overall design framework for educational games in research literature, Linek et al (2010) proposed a new interdisciplinary approach. They proposed an overall methodology that incorporates ideas from Dick and Carey's instructional design model, while focusing primarily on the unique qualities of educational game design. ...
... The most frequently reported approach was 'Based on existing methods and guidelines', which comprise previous methodological frameworks or guidance and published methodology. Whilst some studies did not explain how the existing methods formed the foundations of the framework being developed, most did expand this further: adapting the methods [19,24], integrating methods, building on the existing methods [4,37], based on the framework [20-22, 27, 30, 33], combined well established guidelines which comprised the same stages [16], and the framework was basic inspiration [28]. Only one study specified how the frameworks or guidance was identified; Squires and colleagues used a literature review [5]. ...
... Approaches taken to refining and validating comprised; piloting the framework [35], trialling identified stages and using the results of the trial to further develop the framework [25], using a case study or Delphi panel to evaluate and refine the framework [5,8,33], using a case study to validate the framework [17,29] and testing the framework [20]. Two studies did not report details of the case study [18,24]. ...
Article
Full-text available
Background: Although the benefits of using methodological frameworks are increasingly recognised, to date, there is no formal definition of what constitutes a 'methodological framework', nor is there any published guidance on how to develop one. For the purposes of this study we have defined a methodological framework as a structured guide to completing a process or procedure. This study's aims are to: (a) map the existing landscape on the use of methodological frameworks; (b) identify approaches used for the development of methodological frameworks and terminology used; and (c) provide suggestions for developing future methodological frameworks. We took a broad view and did not limit our study to methodological frameworks in research and academia. Methods: A scoping review was conducted, drawing on Arksey and O'Malley's methods and more recent guidance. We systematically searched two major electronic databases (MEDLINE and Web of Science), as well as grey literature sources and the reference lists and citations of all relevant papers. Study characteristics and approaches used for development of methodological frameworks were extracted from included studies. Descriptive analysis was conducted. Results: We included a total of 30 studies, representing a wide range of subject areas. The most commonly reported approach for developing a methodological framework was 'Based on existing methods and guidelines' (66.7%), followed by 'Refined and validated' (33.3%), 'Experience and expertise' (30.0%), 'Literature review' (26.7%), 'Data synthesis and amalgamation' (23.3%), 'Data extraction' (10.0%), 'Iteratively developed' (6.7%) and 'Lab work results' (3.3%). There was no consistent use of terminology; diverse terms for methodological framework were used across and, interchangeably, within studies. Conclusions: Although no formal guidance exists on how to develop a methodological framework, this scoping review found an overall consensus in approaches used, which can be broadly divided into three phases: (a) identifying data to inform the methodological framework; (b) developing the methodological framework; and (c) validating, testing and refining the methodological framework. Based on these phases, we provide suggestions to facilitate the development of future methodological frameworks.
... We identi ed eight different approaches used for developing methodological frameworks (Table 2), these are also summarised by study in Additional File 4 The most frequently reported approach was 'Based on existing methods and guidelines', which comprise previous methodological frameworks or guidance and published methodology. Whilst some studies did not explain how the existing methods formed the foundations of the framework being developed, most did expand this further: adapting the methods (19,24), integrating methods, building on the existing methods (4, 37), based on the framework (20-22, 27, 30, 33), combined well established guidelines which comprised the same stages (16), and the framework was basic inspiration (28). Only one study speci ed how the frameworks or guidance was identi ed; Squires and colleagues used a literature review (5). ...
... Approaches taken to re ning and validating comprised; piloting the framework (35), trialling identi ed stages and using the results of the trial to further develop the framework (25), using a case study or Delphi panel to evaluate and re ne the framework (5,8,33), using a case study to validate the framework (17,29) and testing the framework (20). Two studies did not report details of the case study (18,24). ...
Preprint
Full-text available
Background A methodological framework is a structured guide to completing a process or procedure. Although the benefits of using methodological frameworks are increasingly recognised, to date, there is no formal definition of what constitutes a ‘methodological framework’, nor is there any published guidance on how to develop one. This study’s aims are to: (a) map the existing landscape on the use of methodological frameworks; (b) identify approaches used for the development of methodological frameworks and terminology used; and (c) provide suggestions for developing future methodological frameworks. We took a broad view and did not limit our study to methodological frameworks in research and academia. Methods A scoping review was conducted, drawing on Arksey and O’Malley’s methods and more recent guidance. We systematically searched two major electronic databases (MEDLINE and Web of Science), as well as grey literature sources and the reference lists and citations of all relevant papers. Study characteristics and approaches used for development of methodological frameworks were extracted from included studies. Descriptive analysis was conducted. Results We included a total of 30 studies, representing a wide range of subject areas. The most commonly reported approach for developing a methodological framework was ‘Based on existing methods and guidelines’ (66.7%), followed by ‘Refined and validated’ (33.3%), ‘Experience and expertise’ (30.0%), ‘Literature review’ (26.7%), ‘Data synthesis and amalgamation’ (23.3%), ‘Data extraction’ (10.0%), ‘Iteratively developed’ (6.7%) and ‘Lab work results’ (3.3%). There was no consistent use of terminology; the studies included a range of terms for ‘methodological framework’, which were also used interchangeably within studies. Conclusions Although no formal guidance exists on how to develop a methodological framework, this scoping review found an overall consensus in approaches used, which can be broadly divided into three phases: (a) identifying data to inform the methodological framework; (b) developing the methodological framework; and (c) validating, testing and refining the methodological framework. Based on these phases, we provide suggestions to facilitate the development of future methodological frameworks.
... We identi ed eight different approaches used for developing methodological frameworks (Table 2), these are also summarised by study in Additional File 4 The most frequently reported approach was 'Based on existing methods and guidelines', which comprise previous methodological frameworks or guidance and published methodology. Whilst some studies did not explain how the existing methods formed the foundations of the framework being developed, most did expand this further: adapting the methods (19,24), integrating methods, building on the existing methods (4,37), based on the framework (20-22, 27, 30, 33), combined well established guidelines which comprised the same stages (16), and the framework was basic inspiration (28). Only one study speci ed how the frameworks or guidance was identi ed; Squires and colleagues used a literature review (5). ...
... Approaches taken to re ning and validating comprised; piloting the framework (35), trialling identi ed stages and using the results of the trial to further develop the framework (25), using a case study or Delphi panel to evaluate and re ne the framework (5,8,33), using a case study to validate the framework (17,29) and testing the framework (20). Two studies did not report details of the case study (18,24). ...
Preprint
Full-text available
Background Although the benefits of using methodological frameworks are increasingly recognised, to date, there is no formal definition of what constitutes a ‘methodological framework’, nor is there any published guidance on how to develop one. For the purposes of this study we have defined a methodological framework as a structured guide to completing a process or procedure. This study’s aims are to: (a) map the existing landscape on the use of methodological frameworks; (b) identify approaches used for the development of methodological frameworks and terminology used; and (c) provide suggestions for developing future methodological frameworks. We took a broad view and did not limit our study to methodological frameworks in research and academia. Methods A scoping review was conducted, drawing on Arksey and O’Malley’s methods and more recent guidance. We systematically searched two major electronic databases (MEDLINE and Web of Science), as well as grey literature sources and the reference lists and citations of all relevant papers. Study characteristics and approaches used for development of methodological frameworks were extracted from included studies. Descriptive analysis was conducted. Results We included a total of 30 studies, representing a wide range of subject areas. The most commonly reported approach for developing a methodological framework was ‘Based on existing methods and guidelines’ (66.7%), followed by ‘Refined and validated’ (33.3%), ‘Experience and expertise’ (30.0%), ‘Literature review’ (26.7%), ‘Data synthesis and amalgamation’ (23.3%), ‘Data extraction’ (10.0%), ‘Iteratively developed’ (6.7%) and ‘Lab work results’ (3.3%). There was no consistent use of terminology; diverse terms for methodological framework were used across and, interchangeably, within studies. Conclusions Although no formal guidance exists on how to develop a methodological framework, this scoping review found an overall consensus in approaches used, which can be broadly divided into three phases: (a) identifying data to inform the methodological framework; (b) developing the methodological framework; and (c) validating, testing and refining the methodological framework. Based on these phases, we provide suggestions to facilitate the development of future methodological frameworks. Trial registration: Not applicable
... The most frequently reported approach was 'Based on existing methods and guidelines', which comprise previous methodological frameworks or guidance and published methodology. Whilst some studies did not explain how the existing methods formed the foundations of the framework being developed, most did expand this further: adapting the methods (20,25), integrating methods, building on the existing methods (4,38), based on the framework (21-23, 28, 31, 34), combined well established guidelines which comprised the same stages (17), and the framework was basic inspiration (29). Only one study specified how the frameworks or guidance was identified; Squires and colleagues used a literature review (5). ...
... Approaches taken to refining and validating comprised; piloting the framework (36), trialling identified stages and using the results of the trial to further develop the framework (26), using a case study or Delphi panel to evaluate and refine the framework (5,9,34), using a case study to validate the framework (18,30) and testing the framework (21). Two studies did not report details of the case study (19,25). ...
Preprint
Full-text available
Background: A methodological framework is a structured guide to completing a process or procedure. Although the benefits of using methodological frameworks are increasingly recognised, to date, there is no formal definition of what constitutes a ‘methodological framework’, nor is there any published guidance on how to develop one. This study’s aims are to: (a) map the existing landscape on the use of methodological frameworks; (b) identify approaches used for the development of methodological frameworks and terminology used; and (c) provide suggestions for developing future methodological frameworks. We took a broad view and did not limit our study to methodological frameworks in research and academia. Methods: A scoping review was conducted, drawing on Arksey and O’Malley’s methods and more recent guidance. We systematically searched two major electronic databases (MEDLINE and Web of Science), as well as grey literature sources and the reference lists and citations of all relevant papers. Study characteristics and approaches used for development of methodological frameworks were extracted from included studies. Descriptive analysis was conducted. Results: We included a total of 30 studies, representing a wide range of subject areas. The most commonly reported approach for developing a methodological framework was ‘Based on existing methods and guidelines’ (66.7%), followed by ‘Refined and validated’ (33.3%), ‘Experience and expertise’ (30.0%), ‘Literature review’ (26.7%), ‘Data synthesis and amalgamation’ (23.3%), ‘Data extraction’ (10.0%), ‘Iteratively developed’ (6.7%) and ‘Lab work results’ (3.3%). There was no consistent use of terminology; the studies included a range of terms for ‘methodological framework’, which were also used interchangeably within studies. Conclusions: Although no formal guidance exists on how to develop a methodological framework, this scoping review found an overall consensus in approaches used, which can be broadly divided into three phases: (a) identifying data to inform the methodological framework; (b) developing the methodological framework; and (c) validating, testing and refining the methodological framework. Based on these phases, we provide suggestions to facilitate the development of future methodological frameworks. Trial registration: Not applicable
... The conceptual design of COMPCAT has made use of the methodology adopted in the ECproject ELEKTRA (Enhanced Learning Experience and Knowledge Transfer) as elucidated by Linek et al. (2009). The same authors suggested that the methodology used can be taken as a base for other game-based learning modules as well as serious games. ...
Article
Full-text available
The selection of career paths and making of academic choices is a difficult and often confusing task for young people. The impact on their lives, however, is enormous as it can determine entire future career possibilities. In India, a general remedy to this stress is that instead of choosing a field of study tailored to individual preferences and strengths, topics are chosen that align with the choices of the students’ families or their friends. This can have the effect of entrenching patterns of intergenerational inequity. The aim of this research is to give students greater access to the knowledge capital which will help them make better choices. This is achieved by engaging students in the career planning process, in order to convey information in a likeable and credible way. The COMPCAT (Competency and Career Assessment Tool) game engine combines the use of learning analytics and real time, interactive computer simulations designed to gain insights into the students’ engagement in the making of these complex decisions. This paper presents the conceptual architecture of the game and demonstrates its role in enhancing the learning effectiveness of the students.
... Dessa forma, os métodos existentes para produção de jogos sem enfoque educacional , podem não ser suficientes para identificar as reais necessidades dos usuários, pois a maioria dos processos para o desenvolvimento de JEDs não contempla os requisitos de propósito educacional [Gomes and Wanderley 2003, Linek et al. 2010]. Na literatura há uma carência de métodos e técnicas específicos para produção de JEDs, e os existentes [Godoy and Barbosa , Rodrigues et al. 2010, Santos et al. 2012, Posvolski et al. ] não possuem uma descrição detalhada quanto ao levantamento de requisitos. ...
Conference Paper
Full-text available
The use of digital educational games (DEGs) in educational institu- tions presents evidence of a new society, that believes in game efficiency to the process of teaching and learning. Thus, the requirements elicitation for produc- tion of these DEGs should be accurately performed. This paper presents a te- chnique of requirement elicitation for production of DEGs, called ERJED. This technique has as main characteristic the effective participation of the specialist teacher in the development process. ERJED was evaluated using documents inspection and the results indicate that this technique collaborated improving in 82% the quality of specified requirements. These results are promising and show that the technique can contribute to the development of JEDs.
Article
Full-text available
The basic idea of game-based learning is the fusion of fun and learning by using the motivational potential of video games in the educational context. Thus, the design of an educational game should be adaptive not only to the learner's cognitive abilities, but also to the individual preferences of the player. Especially the design of the non-player character (NPC) that takes the role of a teacher by providing guidance, hints, and companionship seems of crucial importance. The presented study addresses to three design question with respect to an optimal NPC-design, namely the NPC's personality as well as the role of naturalism and color. Based on media equation and related research several hypotheses were derived. Independent variables were the NPC's personality (funny & friendly vs. severe & unfriendly), color (black & white vs. colored design), and naturalism (naturalistic vs. comic-like appearance), resulting in a 2 x 2 x 2-design. By means of eight different NPC-versions the players had to rate and indicate which of the versions they prefer for an NPC that provides guidance and companionship in an educational adventure-game. Dependent measurements include the selection of the preferred NPC as well as an NPC-rating. The results of the used χ²-tests indicate a clear preference for a colored, naturalistic NPC-design. For the NPC's personality the pupils favor a personality that was similar to their own, indicating similarity-attraction. Further results and implications for the design of educational games will be discussed.
Conference Paper
In the context of e-learning the assessment of user-data can be accomplished by different methods. On the one hand, conventional questionnaires can be used; on the other hand, there are also other technical possibilities available, namely the tracking of user-activities by logfiles. These methods deliver complementary data of diverse informational impact. The following article outlines the pros and cons of these methods and propagates a combined assessment approach. On the basis of a practical example (evaluation of a new concept called Microadaptivity) the advantages of the combination of questionnaires and logfiles will be illustrated.
Conference Paper
Game-based learning is based on the idea of using the motivational and immersive potential of video games in the educational context in order to provide an enjoyable form of learning. Thereby one core question regards to the optimal conditions for enjoyable experiences in games. To address this issue one can regard to the flow theory of Csikszentmihalyi according to which the balance between the perceived challenges of a situation and the perceived skills of a person determines the quality of subjective experience. The highest quality of subjective experience will be reached when challenges and skills are equally high. This so-called flow-experience is a hedonic state that can be characterized by being immersed in the activity. Since immersion is often though as the heart of enjoyable games the balance between challenges and skills seems to be of essential importance for (educational) video games, too. However, the challenges within a video game are manifold. Thus, a more fine-grained view is proposed whereby cognitive, socio-affective, and physical challenges are differentiated. Accordingly, these different challenges address to the respective cognitive, socio-affective, and physical skills. Cognitive challenges are, e.g., solving a riddle or dealing with the learning content. To master these cognitive challenges the accordingly cognitive skills like logical thinking or problem-solving abilities are required. Socio-affective challenges regards to the (para)social interaction with game characters and the handling of highly emotional situations. To address these socio-affective challenges, socio-affective skills like social competencies or stress coping strategies are needed. Physical challenges refer to the speed and the perceptual complexity of the game. To manage these physical challenges, the respective physical skills like spatial abilities or perceptual speed are necessary. Especially (educational) adventure-games often provide cognitive, socio-affective, and physical challenges simultaneously, e.g., when solving a riddle (cognitive challenge) while interacting with a beautiful female game-character (socio-affective challenge) in a situation with exciting background music (physical challenge). Thus, the question arises how these different challenges are interrelated. Three different kinds of interrelationships, an independent, an additive, and an interactive one will be disputed and exemplarily described with respect to their impact of an optimal, adaptive game-design. The different scenarios will be discussed in the face of the already existing related research. Additionally, possible empirical approaches for a more systematic investigation of the interrelationship of cognitive, socio-affective and physical challenges will be provided.