ArticlePDF Available

Abstract and Figures

Most existing educational games cannot compete with their non-educational counterparts in terms of visual and narrative quality, gameplay, or adaptability. Amongst the most advanced approaches is ELEKTRA, a European project targeting on producing a 3D adventure game teaching physics. The project developed a scientifically sound framework for intelligent and adaptive tutoring, enabling the game to adapt learning/gaming activities to individual learning progress and pedagogical strategies. A crucial aspect, and a weak spot of present educational games, is the individualized assessment of knowledge. Existing approaches frequently rely on typical quiz-like methods, failing to adapt to individual learners and, most likely, they break the game's narrative, what in turn weakens the "natural" advantages of educational games by compromising immersion and motivation to play and learn. In ELEKTRA, assessment occurs in integrated and individualized game situations within which learners have to accomplish adapted and tailored physics-related tasks, for example to hit a light sensor with a narrow beam of light, created with different optical devices, in order to open a door. ELEKTRA's methodology allows providing individualized game situations on the basis of the same pool of game assets. For example, a high performer will be provided with fewer but more complex situations than an underachiever. The set of possible actions and action sequences is modeled in terms of problem spaces. Problem solution states are determined and linked with a skill structure established by prerequisite relations between skills. An ontology holds both information, enabling a "learning engine" to reason about the learner's skills and increase or decrease their probabilities, approaching the true skill state. On this basis, the skills and therefore the learning progress can be assessed without compromising the learner's immersion with the game and, furthermore, subsequent learning and assessment situations can be adapted to the learners' needs. 1. What do you want to play/learn today? The majority of current approaches to technology-enhanced learning are based on traditional, unexciting 2D user interfaces. This perspective is compounded by the proliferation of immersive recreational computer games. In addition, traditional interfaces for educational applications have distinct weaknesses from the perspectives of learning psychology and didactics. For example, they are not intrinsically motivational and it is difficult to retain a learner's interest, to provide a meaningful context throughout learning episodes, or to activate prior knowledge as a basis for learning. Moreover, it is not always possible to provide real-world problems for practicing new knowledge and a purposeful application of new knowledge is difficult without a meaningful and engaging context. Immersive digital educational games (DEGs) offer a highly promising approach to make learning more engaging, satisfying, inspiring, and probably more effective. Thus, it is not surprising that currently there is significant hype over game-based learning (cf. Kickmeier-Rust et al. 2006).
Content may be subject to copyright.
1
Not Breaking the Narrative: Individualized Competence Assessment in Educational Games
Michael D. Kickmeier-Rust, Dietrich Albert, Cord Hockemeyer, Thomas Augustin
University of Graz, Austria
michael.kickmeier@uni-graz.at
dietrich.albert@uni-graz.at
cord.hockemeyer@uni-graz.at
thomas.augustin@uni-graz.at
Abstract: Most existing educational games cannot compete with their non-educational counterparts
in terms of visual and narrative quality, gameplay, or adaptability. Amongst the most advanced
approaches is ELEKTRA, a European project targeting on producing a 3D adventure game teaching
physics. The project developed a scientifically sound framework for intelligent and adaptive tutoring,
enabling the game to adapt learning/gaming activities to individual learning progress and pedagogical
strategies. A crucial aspect, and a weak spot of present educational games, is the individualized
assessment of knowledge. Existing approaches frequently rely on typical quiz-like methods, failing to
adapt to individual learners and, most likely, they break the game’s narrative, what in turn weakens
the “natural” advantages of educational games by compromising immersion and motivation to play
and learn. In ELEKTRA, assessment occurs in integrated and individualized game situations within
which learners have to accomplish adapted and tailored physics-related tasks, for example to hit a
light sensor with a narrow beam of light, created with different optical devices, in order to open a door.
ELEKTRA’s methodology allows providing individualized game situations on the basis of the same
pool of game assets. For example, a high performer will be provided with fewer but more complex
situations than an underachiever. The set of possible actions and action sequences is modeled in
terms of problem spaces. Problem solution states are determined and linked with a skill structure
established by prerequisite relations between skills. An ontology holds both information, enabling a
“learning engine” to reason about the learner’s skills and increase or decrease their probabilities,
approaching the true skill state. On this basis, the skills and therefore the learning progress can be
assessed without compromising the learner’s immersion with the game and, furthermore, subsequent
learning and assessment situations can be adapted to the learners’ needs.
Keywords: Adventure game, micro-adaptivity, competence assessment, non-invasive interventions
1. What do you want to play/learn today?
The majority of current approaches to technology-enhanced learning are based on traditional,
unexciting 2D user interfaces. This perspective is compounded by the proliferation of immersive
recreational computer games. In addition, traditional interfaces for educational applications have
distinct weaknesses from the perspectives of learning psychology and didactics. For example, they
are not intrinsically motivational and it is difficult to retain a learner’s interest, to provide a meaningful
context throughout learning episodes, or to activate prior knowledge as a basis for learning.
Moreover, it is not always possible to provide real-world problems for practicing new knowledge and a
purposeful application of new knowledge is difficult without a meaningful and engaging context.
Immersive digital educational games (DEGs) offer a highly promising approach to make learning more
engaging, satisfying, inspiring, and probably more effective. Thus, it is not surprising that currently
there is significant hype over game-based learning (cf. Kickmeier-Rust et al. 2006).
Many of the potential advantages of DEGs (e.g., interactivity, feedback, problem solving) are
considered to be important for successful and effective learning (Merrill 2002). Moreover, DEGs serve
the needs of the “Nintendo generation” or the “digital natives” who grew up on “twitch speed”
computer games, MTV, action movies, and the Internet. Marc Prensky (2001) argues that the
exposure to such media has emphasized certain cognitive aspects and de-emphasized others, thus,
the demands on education have changed.
Still, DEGs have major disadvantages such as difficulties in providing an appropriate balance
between gaming and learning activities or between challenge and ability, in aligning the game with
national curricula, or the extensive costs of developing high quality games (Van Eck 2006). Thus,
DEGs most often cannot compete with commercial counterparts in terms of gaming experience,
immersive and interactive environments, narrative, or motivation to play. Moreover, most educational
games do not rely on sound instructional models, leading to a separation of learning from gaming;
often they provide gaming actions only as reward for learning. Existing DEGs do not differ significantly
from other multimedia learning objects and applications and there is considerable debate regarding
the power of games for educational purposes, the advantages, disadvantages, costs, and risks.
At the same time, computer games are tremendously successful and game industry constantly
increases sales to several billions of Euros. A significant number of young people spend many hours
a week playing computer games and most often these games are the preferred play. Adventure
games like Myst sold 6 million copies; simulations like The Sims 2 sold 1 million copies in the first ten
days after publications; the new generation game consoles (i.e., Microsoft Xbox 360, Nintendo Wii,
and Sony Playstation 3) sold approximately 17 million units until February 2007.
In conclusion, the attempt to utilize - at least parts of - gaming activities for educational purposes and
to utilize the educational potential of computer games is a highly promising approach to facilitate
learning and to make it a more pleasant task. The very nature of utilizing (computer) games for
learning is that playing games is one of the most natural forms of learning. Children start learning to
talk by playing with noises or they learn collaboration and strategic thinking when playing Cowboys
and Indians. Since the 1990s research and development has increasingly addressed learning aspects
of playing recreational games and also the realization of computer games for primarily educational
purposes. Kickmeier-Rust et al. (2006) or Mitchell & Savill-Smith (2004) provide an overview of
existing DEGs.
From a psycho-pedagogical viewpoint, the state-of-the-art in game-based learning is at an early
stage. Most existing DEGs are rather small and often simple games, focusing on insight in processes
and complex issues (e.g., the Palestine conflict) or addressing particular sets of skills (e.g., job
application trainings). They generally do not related to school curricula or do not attempt to enable
learning related to school-related subject matter. More importantly, existing games do not provide
sound assessment methods and generally there is an imbalance between learning and gaming.
Finally, while game intelligence is well developed, educational games do not include adaptation to the
learner in terms of knowledge, learning progress, motivation, or individual preferences. Thus, they
cannot compete with their commercial counterparts and they cannot utilize the full potential of
immersive digital games with respect to learning efficacy and learning experience.
2. The ELEKTRA project
The ELEKTRA project (www.elektra-project.org), funded by the European Commission, has the
ambitious and visionary goal to fully utilize the advantages of computer games and their design
fundamentals for educational purposes and to address and eliminate the disadvantages of game-
based learning as far as possible. Nine interdisciplinary European partners contribute to the
development of a sound methodology for designing educational games and the development of a
comprehensive game demonstrator based on a state-of-the-art 3D adventure game teaching physics
according to national curricula. Furthermore, ELEKTRA will address important research questions
concerning game design, didactic design, or adaptive interventions.
The linchpin of successful DEG is motivation to play and therefore to learn. So, an appropriate
balance of challenges by the game and the learner’s abilities is required. Thus, from the perspective
of cognitive science and computer science, an adaptive and individualized approach to DEG
technology is the focus. This is true for pure gaming activities but in particular for learning activities.
As attempted by conventional adaptive and personalized approaches to technology-enhanced
education (Brusilovsky 1999, De Bra 1997), a learner must not be overcharged by subject matter in
order to avoid frustration but at the same the learner must not be subchallenged in order to avoid
boredom. Only if such balance can be achieved, some sort of flow experience can rise, enthralling
and captivating the learner.
In contrast to conventional adaptive tutoring and knowledge testing, adaptive assessment and
interventions within a DEG are restricted by the game’s narrative and the game flow. Existing
approaches to assessment frequently rely on typical quiz-like methods, failing to adapt to individual
learners and, most likely, they break the game’s narrative, what in turn weakens the “natural”
advantages of educational games by compromising immersion and motivation to play and learn. On
the other hand, within an educationally adaptive game such as ELEKTRA the learning tasks are so
integrated with the games narrative that the reordering of learning tasks in order to personalize
learning experience would result is a corresponding reordering of narrative plot elements. With a
linear narrative this would result in a nonsensical narrative that is implausible. The challenge of
creating dynamic yet plausible adaptive narratives is considerable and requires arduous manual
editing of branching narratives. Experimental systems such as Façade (Mateas & Stern 1998)
exemplify the difficulties of creating adaptive narratives.
Within the field of adaptive hypermedia, adaptation is limited by the presentation medium and so
adaptation is manifested through intermittent curriculum ordering and adaptive presentation. Due to
the nature of 3D immersive games adaptation needs to be continuous and less periodic; it needs to
occur at a greater frequency than on a task by task level. Considering this with the existing difficulties
associated with generating adaptive narratives, the ELEKTRA game provides micro-adaptivity, that is,
assessment by interpreting the learner’s behavior and adaptation within learning situations (LeS) as
opposed to around them.
3. Micro-adaptivity
The very basis of micro-adaptive skill assessment and non-invasive interventions is a formal model for
interpreting a learner’s (problem solution) behavior within learning and assessment situations in an
educational game. As an example, a learner might be confronted with a torch, a number of blinds, and
a screen. The learner’s task might be to reduce the cone of the torch’s light into a narrow beam of
light using the blinds (contributing to the understanding that light propagates in a straight line).
To obtain a formal model, we describe such a game situation and its current status at a certain point
of time by a set of props (e.g., torch, blinds, and screen) and their current properties (e.g., location or
alignment). For each of a situation’s status a number of admissible actions can be performed by the
learner (e.g., to turn on the torch or to position a blind). Each action, in turn, is interpreted regarding
its correctness or appropriateness for accomplishing the task (e.g., narrowing the light cone). These
interpretations of behavior enable conclusions (in a probabilistic sense) about the presence of certain
skills and, in some cases, also the absence of certain skills. The probabilistic assessment of skills is
the very basis for micro-adaptive interventions, either in terms of generating LeS tailored for an
individual learner or in terms of providing a learner with non-invasive educational interventions within
a LeS, for example giving the learner hints.
To realize non-invasive assessment of skills and adaptive educational interventions, ELEKTRA relies
on the formal framework of Competence-based Knowledge Space Theory (CbKST). Originating from
conventional adaptive and personalized tutoring, this set-theoretic framework allows assumptions
about the structure of skills of a domain of knowledge and to link the latent skills with observable
behavior.
3.1 Skill structures and performance structures
To address the challenges for research and development and to incorporate a separation of latent
skills and observable performance, ELEKTRA utilizes the framework of CbKST to provide the game
with a methodology for suitable adaptive interventions. It offers an internal cognition-based logic that
is quite similar to the logic of ontologies: well-defined entities (the skills) are in a well-defined
relationship (a so-called prerequisite relation). Skills are defined as distinct entities of ability or
knowledge. The term “competence” is often used synonymously.
CbKST is an extension of the originally behavioral Knowledge Space Theory (KST, Doignon &
Famagne 1985, 1999) where a knowledge domain Q is characterized by a set of problems or test
items. The knowledge state of an individual is identified on the subset of problems this person is
capable of solving. Due to mutual dependencies between the problems captured by prerequisite
relations, not all potential knowledge states will occur. The collection of all possible states is called a
knowledge structure
Κ
. To account for the fact that a problem might have several prerequisites (i.e.,
and/or-type relations) the notion of a prerequisite function was introduced. The basic idea of CbKST is
to assume a set E of abstract skills underlying the problems and learning objects of the domain. The
relationships between the skills and problems are established by a skill function. Such function
assigns a collection of subsets of skills (i.e., skill states) to each problem, which are relevant for
solving it and it assigns the skills to each learning object taught. By associating skills to the problems
of a domain, a knowledge structure on the set of problems is induced. The skills, which are not
directly observable, can be uncovered on the basis of a person’s observable performance. A further
extension is to assume prerequisite relationships between the skills, inducing a skill structure C on the
set of skills (Korossy 1999). To illustrate this approach, assume that a knowledge domain is
represented by Q={a, b, c, d}. Consider the set E={V, W, X, Y, Z} of skills that are relevant for solving
them. A prerequisite function that might exist among these skills is demonstrated in Figure 1a. For
example, this function reads that if a student has skill X we can assume that this student also
possesses either skill V or W, or both; the corresponding skill structure is shown in Figure 1b. It
includes only 13 possible skill states from a total of 25 = 32 states.
Figure 1: The left panel illustrates a prerequisite function (the bended line below skill X indicates a
logical or). The right panel shows the corresponding skill structure. The bolded line indicates one of
several meaningful learning paths.
This approach entails several advantages. Given the performance, that is, the subset of problems a
student could master, the latent skills underlying that problem solving performance can be identified.
Due to the utilization of representation and interpretation functions no one-to-one mapping of
performance to skills is required and meaningful learning paths can be identified.
3.2 Problem spaces
In addition to the formal model of the knowledge domain, its skills, and the prerequisite relations
between those skills, a formal model of tasks and problem definitions within a LeS must be defined;
the so-called problem spaces.
Each LeS is characterized by a set of props or objects (e.g., torch, blinds, and screen) the learner can
manipulate in order to achieve a certain goal. For example, the torch, two blinds, and the screen must
be aligned in a row to narrow the torch’s light cone (Figure 2).
Figure 2: Blinds in a row
Formally, let O be a set of props that can be used to define a certain LeS. For simplicity of notation,
we assume that O={o1,…,oN}. Furthermore, for 1 n N let Pn be a non-empty set such that ∅∈ Pn,
which contains the properties of the n-th object on. These properties can be of quite different character
(e.g., a six-dimensional vector describing position and orientation of an object in the virtual space or
simply two values “on” and “off” for a switch). The definitions of such properties for each object by
location and alignment, unfortunately, would result in an almost infinite number of combinations. To
make the properties manageable or computable, we define categories for the objects’ properties. For
example, there might be four “location categories” for a blind, each having a certain value of
correctness (Figure 3). This allows us to describe a problem state as the N-tuple of all objects’
properties, that is, (po1,…,poN), where pon Pon (for simplicity of notation, we write pi and Pi respectively).
If pn=, then the n-th object does not appear in the problem situation. If, on the other hand, pn≠∅, then
the n-th object on appears in the problem situation and can be manipulated by the learner. The set S of
all problem states is called the problem space: S=P1 x…xPN. Finally, to specify a problem situation, we
have to fix an initial state sS and, for a fixed initial state sS, a set SsS of solution states.
Figure 3: Four categories of a blind’s locations; category 1 is the most correct location.
To solve a certain problem, the learner can perform different actions to modify the objects and
therefore change the problem state. Additionally, we assume that any problem can be solved in a
finite number of steps. For a problem space S=P1 x…xPN, let us assume an initial state sS, a set SsS
of solution states. Furthermore, let A be a non-empty set of actions a user may perform. Furthermore,
let RSxA denote a “compatibility relation”, that is, (s,a)R if and only if action a is performable in
problem state s. Furthermore, let f:R
S be a “transition function” in the following sense: If a learner
performs action a when the problem state s is given, then the problem state f(s,a) results. Finally, a
finite sequence (s1,a1), (s2,a2), …,(sm,am) is called a problem solution process if the following conditions
are satisfied: (1) s1=s; (2) (st,at)R for all t=1,…,m; (2) st+1=f(st,at) for all t=1,…,m-1; (4) st S
s for all
t=1,…,m; and (5) f(sm,am)Ss.
3.3 Interpreting learner behavior – continuous skill assessment
The combination of both skill structure and problem spaces allows the continuous interpretation of the
learner’s behavior / actions within a LeS in terms of present and absent skills. This interpretation of
course cannot be deterministic but rather probabilistic. For example, if a learner does not turn on the
torch, we can assume - with a certain probability - that this learner lacks the skill to “know that the task
requires a light source”.
Let us assume a finite set E of skills associated with a given problem state. Furthermore, let C be a
family of subsets of E, containing at least E, and the empty set . For simplicity of notation, we
assume that C=(C0,…,CM), where C0= and CM=E. The elements CC are referred to as skill states
and the tuple (E,C) is denoted as skill structure.
We assume that at a certain point of time, any learner is in exactly one of the skill states in C.
However, since the skill state of a person is not directly observable, the problem solution process of a
person is analyzed to obtain evidence about the person’s skill state.
Let us assume a skill structure (E,C), and a problem solution process
(s1,a1), (s2,a2), …,(sm,am)
. Then,
for 0 t m, let assume a probability distribution Lt:C[0,1], with the following interpretation in mind:
Lt(C) denotes the likelihood that a person who has performed the actions a1 (in state s1), a2 (in state
s2), …, and at (in state st), is in skill state C. Similarly, L0 is the initial distribution at the beginning of the
solution process (i.e., before action a1 is observed).
Note that there are different options to obtain the initial distribution L0. At the beginning of the learning
process (i.e., before the first problem situation is presented to the user), either the initial distribution
can be estimated from an entry test or the skill states are assumed to be uniformly distributed.
Alternatively, let us assume that the user has already solved some of the problems. If, at last, we have
observed the problem solution process (s1,a1), (s2,a2), …,(sm,am), then the final likelihood function
Lm:C[0,1], is used as initial distribution for the next problem situation. Similarly, we assume that the
initial state of this next problem situation depends on the likelihood function LM. This general idea can
be formalized by introducing a function
00
:{( ,..., ) : ... 1}
MM
gp p p p S++ = →,
which assigns to each probability distribution on C, a problem state in S, which can be used as initial
state of the next problem situation.
The important question remains how to update the likelihood of the skill states. In the following, the
multiplicative updating rule by Falmagne & Doignon (1988) is adapted to our needs: Let us assume
the problem solution process (s1,a1), (s2,a2), …,(sm,am). Then the likelihood of the skill states is updated
according to the following idea:
If the action at in state st provides evidence in favor of c E, then increase the likelihood
of all skill states containing c, and decrease the likelihood of all skill states not containing
c.
If the action at in state st provides evidence against the elementary skill c E, then
decrease the likelihood of all skill states containing c, and increase the likelihood of all
skill states not containing c.
Formally, let us assume a skill structure (E,C), a problem solution process (s1,a1), (s2,a2), …,(sm,am)
and, for 0 t m a likelihood function Lm:C[0,1]. Furthermore, let us assume two “skill assignment”
functions fs:RE and fu:RE with the following interpretations in mind: if action a is performed in
problem state s, then we can surmise that the user has all the skills in fs(s,a) (“supported skills”), but
does not have the skills in fu(s,a) (“unsupported skills”). Furthermore, to recalculate the likelihood of
the skill state C, let us fix two input parameters
ζ
0 and
ζ
1 with
1
ζ
0 and
1
ζ
1. Then we update the
likelihood function iteratively for each of the supported und unsupported skills c according to following
formula.
1
'
() ()
() (') (')
t
t
t
C
CLC
LC CLC
ζ
ζ
+
=
C
with a parameter function
ζ
(C) defined as
0
1
,, (,)
() , , (, )
1, otherwise
ut t
s
tt
cCc fsa
CcCcfsa
ζ
ζζ
∉∈
=∈
.
3.4 Non-invasive, individualized, adaptive interventions
On the basis of the probabilistic assessment of skills/skill states, several methods exist to provide the
learner with tailored educational interventions without compromising the game’s narrative and the
game flow.
3.4.1 Generating and adapting LeS
ELEKTRA’s methodology allows providing individualized game situations on the basis of the same
pool of game assets. For example, a high performer will be provided with fewer but more complex
situations than an underachiever. Moreover, based on the presence or absence of certain skills,
specific props can be presented or not and tasks can be adjusted to the learner’s needs. In the same
way, a specific LeS can be presented repeatedly if necessary, for example with an increasing level of
difficulty.
3.4.2 Non-invasive interventions
In addition to tailoring an entire LeS, the learner can be educationally supported by interventions (e.g.,
hints) when necessary. The conditions under which a certain adaptive intervention is given are to be
developed on the basis of pedagogical rules; however, these rules will apply the micro-adaptivity
framework and utilize the learner model obtained through the assessment within the framework.
Types of interventions are:
A skill activation adaptive intervention may be applied if a learner gets “stuck” in some area of
the problem space and some skills are not used although the user model assumes that the
user masters these skills.
A skill acquisition adaptive intervention may be applied in a similar situation where, however,
the user model assumes that the user does not master the unused skill.
Basically independent of the model is the application of motivational adaptive interventions.
These might be applied, for example if the learner does not act at all for a certain, unexpectedly
long time.
Assessment clarification adaptive interventions may be applied, for example if the learner’s
actions give contradicting support for and against the assumption of a certain skill state.
4. Technical realization
The introduced framework for micro-adaptive skill assessment and non-invasive interventions is
currently implemented in a game demonstrator within the ELEKTRA project.
The architecture consists of four modules or engines (Figure 4). The learner is connected to the
ELEKTRA system through the game engine (GE). It provides the non-adaptive parts of the game, and
as such it is also the user interface to the system. The GE provides information on the learner’s action
in the game to the skill assessment engine (SAE). The SAE updates the learner model (i.e., the skill
state likelihoods) according to the procedure proposed in Section 3.3 and the information it has in the
ELEKTRA ontology. This ontology serves as a database, containing various information, particularly
the skills assigned to objects and their properties as well as the prerequisite relations between those
skills (Kickmeier-Rust & Albert, in press). The resulting information about the learner’s skill state and
its changes are then forwarded to the Educational Reasoner (ER), the pedagogical part of micro-
adaptivity. Based on pedagogical rules and learning objectives, the ER gives recommendations on
adaptive interventions to the adaptation realization (AR) module which maps the abstractly formulated
educational recommendations onto more concrete game recommendations. In this mapping process,
data on game elements and information on previously given recommendations are considered. The
game recommendations are then forwarded to the GE which realizes them as concrete adaptive
interventions in the game.
Figure 4: ELEKTRA’s architecture for micro-adaptive assessment and interventions
5. Conclusions
The aim of micro-adaptivity is to enable an assessment of skills and learning progress during the
game, which does not compromise the game flow and therefore does not negatively impact intrinsic
motivation. The probabilistic assessment on the basis of interpreting the learner’s behavior and
actions within the game is supplemented with more “significant” test items, for example the
accomplishment of a certain task in order to reach a new level of the game. On the basis of this
assessment, non-invasive adaptive interventions can be triggered in order to support the learning
process.
Based on sound psychological models for problem solving and for skill structures, we have outlined a
framework for micro-adaptivity within complex learning objects. However, micro-adaptivity is still in an
early stage of research and development. The underlying framework uses some simplifying
assumptions like the identity of properties (or position categories) and actions. For example, with each
action only a single object can be manipulated. Based on the experiences in the ELEKTRA project,
the framework will be generalized within and beyond the domain of game-based learning. Future work
will also address the integration of meta-cognitive aspects such as confidence ratings into the
assessment procedure In future projects also the realization of adaptive storytelling is envisaged in
order to enable educational game technology even a broader range of individualization and
adaptation to specific learners.
6. Acknowledgements
The research and development introduced in this work is funded by the European Commission under
the sixth framework programme in the IST research priority, contract number 027986.
References
Brusilovsky, P. (1999) “Adaptive and intelligent technologies for web-based education”. In C. Rollinger
& C. Peylo (Eds.), Special Issue on Intelligent Systems and Teleteaching, Künstliche Intelligenz, Vol.
4, pp. 19-25.
De Bra, P. (1997) “Teaching through adaptive hypertext on the WWW”, International Journal of
Educational Telecommunications, Vol. 3, pp. 163-180.
Falmagne, J.-C. and Doignon, J.-P. (1988) “A class of stochastic procedures for the assessment of
knowledge”, British Journal of Mathematical and Statistical Psychology, Vol. 41, pp. 1–23.
Doignon, J.-P. and Falmagne, J.-C. (1985) “Spaces for the assessment of knowledge”, International
Journal of Man-Machine Studies, Vol. 23, pp. 175-196.
Doignon, J.-P. and Falmagne, J-C. (1999) Knowledge spaces, Springer-Verlag, Berlin.
Kickmeier-Rust, M.D. and Albert, D. (in press). The ELEKTRA ontology model: A learner-centered
approach to resource description.
Kickmeier-Rust, M.D., Schwarz, D., Albert, D., Verpoorten, D., Castaigne, J.-L., and Bopp, M. (2006)
“The ELEKTRA project: towards a new learning experience”. In M. Pohl, A. Holzinger, R. Motschnig,
& C. Swertz (Eds.), M3 – Interdisciplinary aspects on digital media & education, Österreichische
Computer Gesellschaft, Vienna, pp. 19-48.
Korossy, K. (1999) “Modelling knowledge as competence and performance”. In D. Albert &
J. Lukas (Eds.), Knowledge Spaces: Theories, Empirical Research Applications, Lawrence Erlbaum
Associates, Mahwah, pp. 103–132.
Mateas, M. and Stern, A. (2007) “Façade, an artificial intelligence-based art/research experiment in
electronic narrative”, [online], Procedural Arts, http://www.interactivestory.net.
Merrill, M.D. (2002) “First principles of instruction”, Educational Technology, Research and
Development, Vol. 50, pp. 43-59.
Mitchell, A. and Savill-Smith, C. (2004) The use of computer and video games for learning: A review
of the literature, Learning and Skills Development Agency, London.
Prensky, M. (2001) Digital game-based learning, McGraw-Hill, New York.
Van Eck, R. (2006) “Digital game-based learning: It's not just the digital natives who are restless”,
Educause Review, Vol. 41, pp. 16-30.
... More importantly, ELEKTRA addressed research questions concerning data model design as basis for adaptivity and resource description enabling interoperability of systems as well as the data model itself [2]. In the course of the project, an approach to adaptivity, that is, micro adaptivity, was developed that allows assessing learning performance and cognitive states in a non-invasive way by interpreting the learners' behaviour within the game and by responding on the conclusions drawn from their behaviour [3]. Attuned to the assessed competencies (or lack of competencies), meaningful feedback, for example hints, suggestions, reminders, critical questions, or praise, can be triggered, without destroying the gaming experience. ...
... Each of those problem solution states is mapped, through an ontology, to one of a set of possible competence states. By this means, the game can interpret the behaviour of the learner in terms of available knowledge, un-activated knowledge, or missing knowledge, by mapping the actions of the learner to competence states [3]. ...
Article
Full-text available
The idea of utilizing computer games for educational purposes is not new and grounds on the simple fact that playing is one of the most natural forms of learning. Advantages of digital games are that they offer a meaningful context, rich visualizations, and interactivity. Successful educational games, however, require a subtle balance between learning and gaming as well as challenge and ability. Thus, an AI is required that can assess knowledge, learning progress, and motivational-emotional states without compromising the flow of the game. Moreover, non-invasive interventions and feedback is necessary to support and guide the learner. The present paper describes the effects, based on empirical research, of such individualized guidance and feedback on problem solving and learning behaviour.
... Ever since psychologists, educationists, and technicians attempted to develop technology that is able to take the role of a private teacher and to intelligently provide individual learners with suitable tutoring. In the context of educational games, concepts of adaptivity on macro and micro levels, which are tailored to learning environments with large degrees of freedom, were introduced [5] [6]. Macro adaptivity refers to traditional techniques of adaptation such as adaptive presentation and adaptive navigation on the level of learning objects (or learning situations in a DEG). ...
... 80Days combines this story and learning by linking competence spaces with story plots (Figure 1), which generates game paths, possible and meaningful paths through the game accounting for story model, learning objectives, and pedagogical interventions. From a technical perspective, this linkage grounds on an ontological approach, which was developed in the context of DEG [5], extending it by story elements and by a mapping of competences / knowledge and story. Similar to the competence-performance separation introduced in CbKST, we realize a competence-performance-story separation based on mathematical interpretation and representation functions. ...
Article
Full-text available
The present paper introduces the 80Days project, an inter-disciplinary European research project endeavoring after pushing the state-of-the-art in digital educational games. The main objectives of the project are enabling curriculum-related education with competitive computer games, realizing non-invasive and educationally meaningful support of the learner, and combining adaptive tutoring with interactive digital storytelling. 80Days' solution to those challenges is an ontology-based linkage between so-called knowledge spaces and atomic narrative elements. On this basis, an intelligent adaptation of storyline, story pace, and game play to the learning progress and the preferences of the learner can be achieved.
... Most of these methods and frameworks for adaptation and personalisation were developed in the context of conventional e-learning. The underlying concepts and ideas are currently extended and adjusted to the requirements of the rich virtual gaming worlds, particularly to maintain an immersive gaming experience and high levels of motivation, curiosity, and flow experience [8]. A method, which is highly interesting for interaction design in general, is an approach to non-invasive assessment of knowledge and learning progress in the open virtual worlds of computer games and a corresponding adaptation by personalised psycho-pedagogical interventions. ...
... A distinct characteristic of adaptive DEGs is that gathering the necessary information from the user cannot occur in a conventional form (e.g., by test items, questions, or tasks). A DEG requires an assessment that does not harm motivation, immersion, flow experience, or the game's storyline [8]. The very basis of micro adaptivity is a formal psychological model for interpreting the behaviour within the virtual environment. ...
Chapter
Full-text available
Software that intelligently interprets the goals and needs of its users on the basis of their behaviors without interrupting the work flow and consequently disturbing concentration and software that can support the users in a personalized, smart, yet unostentatious way is a desirable vision, for sure. One attempt to such support system was Microsoft’s famous paperclip. The underlying logic, unfortunately, was rather simple and the users did not accept the feature very well. This paper introduces a psychologically and formally sound approach to a non-invasive, hidden assessment of very specific needs of the users as well as their competencies and corresponding tailored support and feedback. The approach was developed in the context of adaptive digital educational games and is based on the concepts of Competence-based Knowledge Space Theory as well as that of Problem Spaces. The purpose of this paper is to broaden the concept and elucidate a possible bridge from computer games to regular software tools. KeywordsEmbedded Assessment-Micro Adaptation-Support Methods-Feedback-User Model
... This is often a point of contention between game developers and ISDs: while game experts generally advocate for covert, discovery-based assessments, ISDs may push for overt assessments such as criterionbased exams. It is imperative to merge assessments naturally with existing game mechanics, as breaking player immersion to administer a multiple-choice test often does more harm than good (Kickmeier-Rust, Albert, Hockemeyer, & Augustin, 2007). These two groups can compromise by fluidly assessing learners through problem-solving exercises or performance checklists, and implementing data collection to track learners' concrete progress towards instructional milestones. ...
Conference Paper
Full-text available
The field of applied game development is highly interdisciplinary, requiring collaboration from many expert groups including game developers, instructional designers, and researchers. Although this collaboration is necessary to create a successful product, it is often difficult for experts to unify their diverse knowledge, opinions, and goals. A comprehensive literature review illustrates that there are many barriers to interdisciplinary collaboration, which frequently lead to communication issues between various expert groups. In an applied game development setting, this often results in experiences that lack effective learning content, engaging mechanics, or proper data collection methodologies. This article examines the roles of various expert groups involved in the development of applied games and identifies the gaps between goals, ideas, and understanding of these groups. The authors propose a solution framework that is designed to close these gaps in order to promote more effective practices for the research and development of applied games.
... based on the open source Nebula 2 engine and the Managalore framework. The GE handles gameplay mechanics such as steering a UFO, displays the results of interaction and abstracts the information about the gameplay into discrete events called game evidence (as described in [10] in the context of the ELEKTRA project). ...
Article
The overall aim of the European research project 80Days situated in the field of Technology-enhanced Learning is to combine adaptive learning, Storytelling and gaming technology in order to build intelligent, adaptive and exciting learning environments in the form of Storytelling-based digital educational games (DEGs). This paper presents the major results of the Serious Gaming group at TU Darmstadt achieved in the first development cycle of 80Days: Based on a short introduction in section 1 providing a brief overview of the 80Days approach and key challenges being addressed, section 2 introduces methods and concepts of an adaptive digital storytelling framework and indicates how this contributes to intelligent personalisation and adaptation in DEGs. Section 3 describes practical results in terms of a first technical platform of 80Days integrating an adaptive learning engine, story engine and game engine plus a content repository and StoryTec as authoring environment to create such Story-based DEGs. Section 4 summarizes the current status and main results of the 80Days approach achieved so far including first evaluation feedback, and points out further research and development activities.
... designed for use with existing learning management systems), and therefore , they cannot be easily transferred to the context of DEGs. The underlying concepts and ideas must be extended and adjusted to the requirements of the rich virtual gaming worlds, particularly to maintain an immersive gaming experience and high levels of motivation , curiosity and flow experience (Rust et al. 2007a). In the present paper, we introduce an approach to non-invasive assessment of knowledge and learning progress in the open virtual worlds of computer games and a corresponding adaptation by personalized psycho-pedagogical interventions. ...
Article
The idea of utilizing the rich potential of today's computer games for educational purposes excites educators, scientists and technicians. Despite the significant hype over digital game-based learning, the genre is currently at an early stage. One of the most significant challenges for research and development in this area is establishing intelligent mechanisms to support and guide the learner, and to realize a subtle balance between learning and gaming, and between challenge and ability on an individual basis. In contrast to traditional approaches of adaptive and intelligent tutoring, the key advantage of games is their immersive and motivational potential. Because of this, the psycho-pedagogical and didactic measures must not compromise gaming experience, immersion and flow. In the present paper, we introduce the concept of micro-adaptivity, an approach that enables an educational game to intelligently monitor and interpret the learner's behaviour in the game's virtual world in a non-invasive manner. On this basis, micro-adaptivity enables interventions, support, guidance or feedback in a meaningful, personalized way that is embedded in the game's flow. The presented approach was developed in the context of the European Enhanced Learning Experience and Knowledge TRAnsfer project. This project also realized a prototype game, demonstrating the capabilities, strengths and weaknesses of micro-adaptivity.
Article
Full-text available
Using computer games for educational purposes is a compelling idea that is increasingly adopted by researchers, developers, and educators. Still, digital educational games are at an early stage. A crucial factor that must be increasingly addressed by future research is a personalization of learning and gaming experiences in the rich virtual worlds of computer games. In the present paper we introduce an approach to combine frameworks of psycho-pedagogical adaptation, interactive storytelling, and emergent game design in order to provide the individual learners with tailored learning experiences without corrupting the game's storyline and without requiring massive content production.
Technical Report
Full-text available
This research report outlines a preliminary definition and design of a concept relationship type (CRT) tool. Objectives and requirements of a CRT tool for representing relationship types based on the domain model (DM) and for defining adaptation rules towards a conceptual adaptation model (CAM) are investigated. Functionalities and components constituting the building blocks of the CRT tool and their initial design are delineated. Keyword list: authoring, concept relationship types, CRT tool, domain model, conceptual adaptation model D3.2a -Integrated model of adaptation on learning with specifications (v1.0), 19-12-2008 Integrated model of adaptation on learning with specifications Page 2 (62) Summary This research report outlines a preliminary definition and design of a concept relationship type (CRT) tool. Objectives of a CRT tool for representing relationship types based on the domain model (DM) and for defining adaptation rules towards a conceptual adaptation model (CAM) are analysed and documented. Existing software tools that are related to the definition of a CRT tool are reviewed and analysed in order to identify how they can inspire and feed into the CRT tool design and implementation. The requirements towards a CRT tool that result from the state of the art and literature as well as from the requirements analysis carried out in university and industry settings are investigated and specified from technological as well as from a psycho-pedagogical perspective. Specifics with regard to requirements and functionalities in the context of simulations and virtual reality are elaborated and reported. Based on these steps of analysis functionalities and components constituting the building blocks of the CRT tool and their initial design are delineated.
Article
Full-text available
A crucial factor for successful digital educational games, particularly for older children and adolescents, is an appropriate balance; balance between learning and gaming and balance between challenge and ability. These factors are important to maintain fun, immersion, flow experience, and motivation – the motivation to play and therefore to learn. Moreover, it is important to realize a gaming experience that can compete with that of commercial, non-educational games. A special challenge in this context arises from the need for pedagogical support during learning -and therefore during gaming. At many staves of the learning ladder, from a psych-pedagogical perspective, support and feedback is necessary in order to ensure successful, effective, and complacent learning. Considering the importance of not destroying immersion with the game, the assessment of the learning progress and psycho-pedagogical feedback must occur in a non-invasive way. This, however, requires an intelligent system that is capable of assessing individual competencies and learning progress by observing and interpreting the learner's behaviour in the learning situations within the game. In ELEKTRA, a project funded by the European Commission and aiming at developing a sound psycho-pedagogical framework for immersive educational games, we developed a formal cognitive framework for the non-invasive assessment and interventions within complex learning situations, that is, micro adaptivity. Attuned to the assessed competencies or lack of competencies, meaningful feedback, for example hints, suggestions, reminders, critical questions, or praise, can be triggered, without destroying the gaming experience. Two questions arise with respect to feedback. First, does feedback, although designed to be non-invasive, on educational issues impair gaming experience? Second, can feedback in gaming situations facilitate the learning progress or does it increase the learner's cognitive load, which was suggested be several researchers. In the context of the ELEKTRA project, we implemented the theoretical framework of micro adaptivity in the game demonstrator. This demonstrator is a state-of-the art 3D adventure game teaching physics in relation to national school curricula for the age group of 12 o 14 years. For evaluation purposes, log files of the gaming sessions were recorded and, in addition, questionnaires and performance tests were presented. In this work, we present results from an evaluation session. The results indicate that (micro) adaptive interventions (i.e., appropriate and meaningful interventions/feedback for an individual learner, his/her knowledge and learning progress) are superior to neutral (i.e., non-individualized but semantically correct interventions) and inappropriate interventions (i.e., non-individualized, unsuited interventions) in terms of learning and gaming measures. In addition, we analysed the relationships between learning progress and socio-emotional variables. The results indicate that adaptive feedback not only facilitates learning but also attitude and immersion.
Article
Full-text available
Note: This article was UPDATED and revised in 2015 in a new article entitled "DGBL: Still Restless After All These Years" which can be found in Research Gate and at Educause Review. What follows are BOTH abstracts: 2006 Abstract: After years of research and proselytizing, the proponents of digital game-based learning (DGBL) have been caught unaware. Like the person who is still yelling after the sudden cessation of loud music at a party, DGBL proponents have been shouting to be heard above the prejudice against games. But now, unexpectedly, we have everyone’s attention. The combined weight of three factors has resulted in widespread public interest in games as learning tools. 2015 Abstract: Nearly a decade ago, I wrote an article for EDUCAUSE Review about digital game-based learning (DGBL) and the challenges it faced.1 I suggested that once proponents of DGBL were successful in convincing people that games could play a role in education, they would be unprepared to provide practical guidance for implementing DGBL. Just as when the person shouting to be heard at a party is suddenly the center of attention at the moment there is a lull in the conversation, we DGBL proponents had everyone's attention—but not much to say. In the article I also suggested that our sometimes overzealous defense of videogames (hereafter often referred to as "digital games") ran the risk of overselling the benefits (and underreporting the challenges) of using digital games in formal education. Digital games, I said then and still believe today, are effective as embodiments of effective learning theories that can promote higher-order outcomes. Our inability to provide guidance in doing so a decade ago was ceding the DGBL front to digital games as tools for making didactic, instructivist learning (i.e., lectures) more "engaging." DGBL, I suggested, was effective not as a means for making learning "fun" or for "tricking" students into learning; DGBL was effective because it supported powerful learning strategies such as situated learning, authentic environments, and optimized challenge and support (scaffolding). What was needed was a renewed focus on (1) research about why DGBL is effective and (2) guidance on how, when, for whom, and under what conditions to integrate digital games into formal education. I was not the only one with these ideas, but my timing and the venue combined to reach many people. That 2006 article has been cited more than 1,000 times since then.2 Yet though these ideas continue to resonate with many people, much has changed in terms of research, practice, and to some extent, my own beliefs about the future of DGBL.
Conference Paper
Full-text available
Abstract Digital game-based ,learning is a ,hot topic of research and development. Since the advent of computer and video games, educators were inherently interested in utilizing the beneficial aspects of computer games for educational purposes. These factors are primarily the intrinsic motivation of games, immersive environments, engaging stories, and an artful balance between challenges and continuously,growing ,abilities. Proponents of computer ,games ,delivered a large ,number ,of empirical investigations revealing that games may ,foster the development ,of abilities ,and competencies. Besides the advantageous aspects of computer games, a variety of problems were reported by researchers. Due to the high costs of professional game development, many educational games ,are technologically poor and cannot compete ,with entertainment ,games ,in terms of visual design, possibilities for interactions, or storytelling. Moreover, many current educational games do not incorporate a sound psychological, pedagogical, or didactic background; instead they are focusing on transmission ,or rehearsal ,of isolated ,facts or skills. Finally, such games lack the ability to adapt to individual competencies failing to balance challenge and abilities regarding knowledge or skills. The ELEKTRA project, introduced in this article, aims for addressing these problems relying on an interdisciplinary approach of cognitive science, neuroscience, pedagogy, game design, and game development. The project will develop an adventure,game ,that can ,keep ,up with ,commercial games ,and ,that focuses on primarily curriculum-related educational,purposes by incorporating a sound,psychological and pedagogical framework. Moreover, the project will prove the outcomes of research and development by a comprehensive,game demonstrator.
Conference Paper
Full-text available
There is little doubt that intelligent and adaptive educational technologies are capable of providing personalized learning experiences and improving learning success. Current challenges for research and development in this field concern, for example, the design of comprehensive data models for adaptive systems as well as the interoperability of systems and the re-usability of learning material across different systems. In the present work we introduce an ontology model, basically developed in the context of immersive digital games, which attempts to provide a solution to existing problems in resource description. On the one hand, comprehensive data models for adaptive systems are supported by separating static information from adaptive systems as far as possible. On the other hand, the ontology model offers a potential solution to precise and, above all, learner-centered resource description by separating latent competencies from observable performance (in learning objects or test items).
Article
Full-text available
The paper provides a review of adaptive and intelligent technologies in a context of Web-based distance education. We analyze what kind of technologies are available right now, how easy they can be implemented on the Web, and what is the place of these technologies in large-scale Web-based education.
Article
This chapter develops an extension of Doignon and Falmagne's knowledge struc-tures theory by integrating it into a competence-performance conception. The aim is to show one possible way in which the purely behavioral and descriptive knowledge structures approach could be structurally enriched in order to account for the need of explanatory features for the empirically observed solution behav-ior. Performance is conceived as the observable solution behavior of a person on a set of domain-specific problems. Competence (ability, skills) is understood as a theoretical construct accounting for the performance. The basic concept is a mathematical structure termed a diagnostic, that creates a correspondence be-tween the competence and the performance level. The concept of a union-stable diagnostic is defined as an elaboration of Doignon and Falmagne's concept of a knowledge space. Conditions for the construction and several properties of union-stable diagnostics are presented. Finally, an empirical application of the competence-performance conception in a small knowledge domain is reported that shall illustrate some advantages of the introduced modeling approach.
Article
Defines a Markovian class of stochastic assessment procedures and investigates the properties of such a system. The knowledge state of an individual with respect to a particular body of information is conceptualized as the set of all the questions that the S is capable of solving. The goal of an assessment procedure is to identify, by a sequence of questions, the S's state among all possible ones. A deterministic procedure is conceivable, but not realistic, in that it does not account for possible inconsistencies in the observed responses. A stochastic framework is proposed, in which an individual state is formalized as a distribution on the set of all possible knowledge states. A central problem is to describe conditions ensuring that the latent distribution corresponding to the S's state can be estimated. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
The information regarding a particular field of knowledge is conceptualized as a large, specified set of questions (or problems). The knowledge state of an individual with respect to that domain is formalized as the subset of all the questions that this individual is capable of solving. A particularly appealing postulate on the family of all possible knowledge states is that it is closed under arbitrary unions. A family of sets satisfying this condition is called a knowledge space. Generalizing a theorem of Birkhoff on partial orders, we show that knowledge spaces are in a one-to-one correspondence with AND/OR graphs of a particular kind. Two types of economical representations of knowledge spaces are analysed: bases, and Hasse systems, a concept generalizing that of a Hasse diagram of a partial order. The structures analysed here provide the foundation for later work on algorithmic procedures for the assessment of knowledge.
First principles of instruction”, Educational Technology
  • M D Merrill
Merrill, M.D. (2002) “First principles of instruction”, Educational Technology, Research and Development, Vol. 50, pp. 43-59
The use of computer and video games for learning: A review of the literature, Learning and Skills Development Agency Digital game-based learning
  • A Mitchell
  • C M Smith
Mitchell, A. and Savill-Smith, C. (2004) The use of computer and video games for learning: A review of the literature, Learning and Skills Development Agency, London. Prensky, M. (2001) Digital game-based learning, McGraw-Hill, New York.