Content uploaded by Daniel Schwarz
Author content
All content in this area was uploaded by Daniel Schwarz
Content may be subject to copyright.
Immersive Digital Games: The Interfaces for
Next-Generation E-Learning?
Michael D. Kickmeier-Rust1, Neil Peirce2, Owen Conlan2,
Daniel Schwarz3, Dominique Verpoorten4, and Dietrich Albert1
1 Department of Psychology, University of Graz,
Universitätsplatz 2 / III, 8010 Graz, Austria
{michael.kickmeier, dietrich.albert}@uni-graz.at
2 Department of Computer Science, Trinity College Dublin, Ireland
{peircen, Owen.Conlan}@cs.tcd.ie
3 Laboratory for Mixed Realities, Cologne, Germany
schwarz@lmr.khm.de
4 Support Laboratory for Telematic, University of Liège, Belgium
dominique.verpoorten@ulg.ac.be
Abstract. The intrinsic motivation to play, and therefore to learn, that might be
provided by digital educational games teases researchers and developers.
However, existing educational games often fail in their attempt to compete with
commercial games and to provide successful learning. Often some learning is
added to digital games or some gameplay is added to educational applications.
Successful education games, however, require merging professional game
design with sound pedagogical strategies. This merge creates a new and hybrid
format that truly deserves the denotation being a learning game. Moreover, a
methodology is required that allows continuously balancing gaming and
learning challenges and the learner’s abilities and knowledge in order to retain
an immersive gaming experience. In this article we introduce approaches to
game design and didactic design, as well as a framework for adaptive
interventions in educational games.
Keywords: Digital educational games, Game Design, Didactic Design
Adaptive intervention, Microadaptivity, Competence-based Knowledge Space
Theory
1 Introduction
Over the past decades e-learning and technology enhanced learning and teaching, has
become more and more popular. Consequently, a vast number of e-learning platforms
and multimedia learning objects are available and widely accepted for the use in
schools as well as for continuing education. E-learning has high potential to distribute
high quality learning objects, facilitating learning and enabling new perspectives and
insights to learning content.
However, the majority of e-learning and multimedia learning objects are based on
traditional 2D user interfaces, e.g. website interfaces, Flash animations, Java applets,
2
or PDF files. Provocatively speaking, current e-learning offers have all more or less
the same unexciting look and feel. This perspective is compounded by the
proliferation of immersive computer games that learners are exposed to outside of
learning experiences. Moreover, independent of usability and accessibility, traditional
interfaces for educational applications have serious weaknesses from the perspectives
of learning psychology and didactics. For example, they are not intrinsically
motivational and it is difficult to retain a learner’s interest, it is difficult to provide a
meaningful context throughout learning episodes, it is difficult to activate prior
knowledge as a basis for learning, it is not always possible to provide real-world
problems for practicing new knowledge, and meaningful application of new
knowledge is difficult without a meaningful and engaging context [1].
Immersive educational computer games offer a highly promising approach to
overcome the mentioned weaknesses and to make learning more engaging, satisfying,
and probably more effective. Currently, there is significant hype over game-based
learning, ranging from edutainment to games for primarily educational purposes (see
[2] for an overview). The major strength of digital games in education is a high level
of intrinsic motivation to play and to proceed in the game and, thus, to learn within
the context of a meaningful and continuous storyline and the according para-scoial
dimension for both gaming and learning provided by game characters. According to
Malone [3], the factors forming that strength and making games fun are challenge,
fantasy, and curiosity. Educational games provide clear goals and rules, a meaningful
learning context, an engaging storyline, immediate feedback, a high level of
interactivity, challenge and competition, random elements of surprise, and rich and
appealing learning environments [3], [4]. These factors determine motivation to play
and to learn but are also considered to be important for successful and effective
learning, e.g. interactivity, feedback, problem solving, or context effects (for reviews
see [1], [5]). On the other hand, educational games have also major disadvantages like
difficulties in providing appropriate balance between gaming and learning activities,
providing a continuous balance between challenge and ability, aligning the game with
national curricula, or the extensive costs of developing high quality games (for a
review see [6]). Due to these problems, most of today’s educational games cannot
compete with their commercial counterparts in terms of gaming experience,
immersive and responsive environments and storytelling, or intrinsic motivation to
play. Moreover, most educational games do not rely on sound instructional models
thus leading to a separation of learning from gaming and often they provide gaming
actions only as reward for learning. Therefore, such games do not differ significantly
from other traditional multimedia learning applications.
The ELEKTRA project (www.elektra-project.org) has the ambitious and visionary
goal to fully utilize the advantages of computer games and their design fundamentals
for educational purposes and to address and eliminate the disadvantages of game-
based learning as far as possible. Nine interdisciplinary European partners contribute
to the development of a sound methodology for designing educational games and the
development of a comprehensive game demonstrator based on a state-of-the-art 3D
adventure game teaching physics according to national curricula. Furthermore,
ELEKTRA will address important research questions concerning game design,
didactic design, or adaptive interventions.
3
2 Game Design
In the point of view of many game designers, educationists, and psychologists, the
genre of digital games offers a wide range of means of expressions to enhance the
presentation of educational contents and to provide learners with an exciting and new
interface to knowledge.
In the disciplines of game design and development exists a long tradition and
experience in the creation of immersive narratives and story lines as well as in the
design of appealing virtual environments. Design domains include game play design,
story design, visual design, information design, sound design, interface design, and
interaction design, and each is a fine art. When seriously aiming for integrating
educational purposes to digital games and developing primarily educational games
whose main goal is to convey educational content, the mentioned design domains are
facing completely new challenges and demands. On the other hand didactic designers
for educational software are confronted with the design potential of a highly
interactive spatiotemporal game world that goes far beyond the interfaces of
traditional e-learning applications. Therefore the establishment of a new genre is
required within which both worlds, recreational games and psycho-pedagogical aims
and methods, are meld into a new hybrid format. From an educational perspective,
digital games offer (a) a real-time 3D virtual environment that serves as space for
learning and is (b) perceived by the learner through a user-centric perspective (“Ego-
perspective”). Such environment is determined by (c) situations. In the format of
digital educational games basically three types of situations exist: (a) learning
situations (LeS) with the particular aim of teaching learning matter, (b) gameplay
situations that provide interactions with the game environment, objects, and
characters requiring a skilful usage of tools and artfully accomplished game moves,
and (c) story situations that are required to combine learning and gameplay situations
in a meaningful context that propels the learning game experience with the motivation
of a enthralling story and its characters. The different situations and types of
situations must be joined, motivated, and legitimated by a global narrative that
provides a meaningful context.
Still, a large number of open questions remain to be addressed by research in the
ELEKTRA project and beyond the project. Just to mention a few, these questions
concern an appropriate balance of gameplay and learning, the translation of game
elements into a pedagogically sound learning methods, or the role of para-social
interaction play (either with real mates or with non-player characters; NPC).
3 Didactic Design
Besides an immersive and motivating game design, a sound pedagogical approach is
required to develop a successful educational game. Up-front choice of reference
pedagogical frameworks allowing instructionally informed decisions is a key success
factor when developing any kind of virtual learning environments [7]. This emphasis
is taken up by authors in the field of digital game-based learning [4], [8], [9]. Without
such educational beacons, risks are high of lacking vocabulary for describing and
4
implementing pedagogical endeavors, ultimately loosing instructional focus. This
section briefly presents the dominant conceptual tool used in ELEKTRA for dealing
with methods of learning. The model is tentatively connected to Bloom's taxonomy of
educational objectives [10]. The articulation of methods, objectives, and evaluation is
conceptually scrutinized according to the principle of "triple consistency" which is
seen, in ELEKTRA's interdisciplinary perspective, as an apparently basic but major
concern and contribution of traditional pedagogy to the whole project.
Created in a context of teacher professional development ([11], [12], the 8
Learning Events Model (8LEM) is intended to trigger reflections of practitioners
about methods of learning and their diversification. This emphasis on methods is
grounded in an educational philosophy which postulates that any educational action
cannot restrict itself to "products of learning" but must also look at "processes of
learning". Conceived for inviting teachers to broaden their repertoire of methods,
8LEM helps managing in a more systematic way the tricky theoretical and practical
issue of designing equivalent but different (inter-individual diversification) or
complementary (intra-individual diversification [13]) learning/gaming paths and
events, regardless whether they are instantiated along an adaptation or an adaptivity
mode. The learning events specified in 8LEM are exploration, imitation, reception,
creation, experimentation, practice, debate, and meta-reflection.
Methods, adapted or not, do not take place in a vacuum. They become meaningful
once they are connected to learning objectives [14]. In this respect, and still in order
to making the instructional design and its rationale [15] to the designer and possibly
to users, each 8LEM-labelled activity is attached to the level of Bloom's taxonomy
[10] it is deemed to serve. The taxonomy includes learning objectives like knowledge,
understanding, application, analysis, synthesis, and evaluation. The resulting
combinations of learning events and objectives result in 48 learning activities. This
gives a rough but complete overview of what is pedagogically at stake in the virtual
learning environment. Diversification of methods can be identified as well as levels of
cognitive objectives targeted.
An additional important component of ELEKTRA's pedagogical conceptual tool-
kit supporting game development is the notion of "triple consistency". This
cornerstone principle of valid pedagogy stresses the mandatory solidarity which must
exist between objectives, methods and evaluation. Tyler [16] demonstrated that
failures in learning system are related to a lack of consistency in between learning
objective, method used to teach, and evaluation of the level of objective’s knowledge
reached.
4 Adaptive Interventions
An immersive digital game for educational purposes requires a subtle balance of
challenge and ability. Thus, the game must be able to adapt to the learner’s
knowledge, skills, and abilities, motivation, and also pedagogical implications.
Generally, adaptive approaches to e-learning contest the one-size-fits-all approach of
traditional learning environments, trying to tailor the learning environment according
to individual needs and preferences. Adaptivity refers to navigation, curriculum
5
sequencing, and presentation. For example, an adaptive system may only provide
learning objects which are suitable for an individual’s learning progress - too difficult
and also too easy learning objects might not be displayed in order to avoid visual and
cognitive load and to suggest an appropriate learning path through the learning
content. In the context of immersive digital games, existing approaches to adaptivity
must be extended in order to retain an immersive gaming experience, motivation, and
probably flow experience by suitable adaptive interventions.
4.1 Adaptive Influencing Learning Situations in a Game
(and not breaking the narrative)
Narratives and storylines have long been known to be a major motivating factor in
video games. Contemporary games frequently employ a game narrative as a
motivating factor. Examples can found through out all video game genres and include
many popular game franchises such as Tomb Raider, Command & Conquer, Grand
Theft Auto, Super Mario, and Final Fantasy1 to name but a few. Evidently the
presence of a plausible narrative within a video game can lead to an engrossing and
rewarding experience. It is for this reason that the ELEKTRA demonstrator has an
embedded narrative that has been crafted to appeal to the target audience of 13-15
year of school pupils. Whereas an embedded narrative has major advantages for
motivating a learner to engage, it also poses considerable challenges for educational
adaptation within the game. Educationally adaptive systems to date have focused
largely on adaptively ordering and presenting learning tasks. Within an educationally
adaptive game such as ELEKTRA the learning tasks are so integrated with the games
narrative that the reordering of learning tasks would result is a corresponding
reordering of narrative plot elements. With a linear narrative this would result in a
nonsensical narrative that is implausible. The challenge of creating dynamic yet
plausible adaptive narratives is considerable and requires arduous manual editing of
branching narratives. Experimental systems such as Façade [17] exemplify the
difficulties of creating adaptive narratives.
Within the field of adaptive hypermedia adaptation is limited by the presentation
medium and so adaptation is manifested through intermittent curriculum ordering and
adaptive presentation (e.g., [18]). Due to the nature of 3D immersive games the
adaptation within the ELEKTRA demonstrator needs to be continuous and less
periodic; it needs to occur at a greater frequency than on a task by task level.
Considering this with the existing difficulties associated with generating adaptive
narratives, the ELEKTRA game provides microadaptivity, that is, adaptation within
learning tasks as opposed to around them.
Microadaptivity removes the challenges of adaptive narratives yet creates
challenges of its own due to the nature of the experience of game play, and the impact
that game world changes can have on a player’s experience. Games are considered to
be intrinsically rewarding to play, and it is factors such as self-governance,
1 Tomb Raider, Core Deign (1996); Command and Conquer, Westwood Studios (1995); Grand
Theft Auto, DMA Design (Rockstar North) (1997); Super Mario Bros., Nintendo (1983);
Final Fantasy, Square Enix (1987)
6
immersion, clear goals, immediate feedback, and game feasibility that contribute to
rewarding play. Any adaptation within the game world must thus be achieved in such
a manner so that it is not to the detriment of the play experience. Essentially the
experience of Flow [19] must be maintained, where the learner is immersed, self-
governing, aware of their ability to succeed, and clearly able to see the game’s goal.
From an educational perspective the experience of Flow is rarely catered for in
adaptive eLearning systems. There is evidence however to suggest that Flow can be
beneficial to learning in classroom environments [20] and may prove beneficial to
learning experiences in immersive educational games.
4.2 Non Invasive Microadaptivity
Achieving microadaptivity without compromising the learner’s gaming experience
requires that any adaptation must be achieved in consideration of the gaming
experience. To accomplish this within ELEKTRA the Learning Engine, the system
responsible for adaptation, does not directly intervene with the game engine but
provides recommendations for changes that are desirable. These recommendations are
always contextually specific to a learning situation and are manifested in such a way
that they are in line with the game’s narrative and the learner’s sense of self-
governance. The intention of the Learning Engine as to what needs to be adapted is
embodied by the recommendations. However, it is at the ultimate discretion of the
game engine as to whether or not to enact a recommendation, this is provided as an
extra insurance so that no adaptation is detrimental to the learner’s gaming
experience.
It is through these appropriate recommendations that the Learning Engine can
effectively influence the game engine to adapt without negatively impacting the
game’s narrative. The ultimate realization of the recommendations is determined by
the game engine and they can take the form of hinting or guidance from a NPC,
motivational support from an NPC, throttling the learning situation difficulty, or
appropriate direct intervention by an NPC where deemed necessary.
4.3 Abstraction within the Learning Engine
The ELEKTRA game consists of a number of LeS each of which may be adapted to
the learner’s performance. The generation of adaptive rules for each learning situation
independently would be both time consuming and lead to many rules that are similar
in nature yet sufficiently LeS specific to prevent their reuse. This is undesirable from
an authoring perspective as it is both arduous and difficult to author adaptation for
each learning situation; additionally the reuse of adaptive pedagogical strategies is
hindered. The ELEKTRA demonstrator overcomes this through a system of
abstracted pedagogical adaptation. Whereas with many adaptive educational systems
the adaptation for a learning situation is specific to that situation only, the Learning
Engine provides generic abstracted pedagogical adaptation that is applied to all
learning situations. This is then mapped into recommendations that are specific to a
learning situation. It is in this mapping from generic adaptation to specific game
7
changes that allows the Learning Engine adaptively influence a learning situation
without adversely affecting the game’s narrative.
The learning engine’s adaptation methodology can be broken into four separate
components, broadly described as Evidence, Inferring, Recommendation, and
Realization. The first component, Evidence, is that information coming from the
game engine that is deemed pertinent to the learner’s performance within a learning
situation. The second component, Inferring, embodies an assessment of the Evidence
information and the generation from this of generic semantically meaningful
performance information. The Recommendation component consumes the output of
the Inferring component in order to generate a generic pedagogical adaptation goal.
The final component the Realization consumes generic pedagogical goals and
produces learning situation specific adaptation within the game engine to achieve the
generic goal.
The advantages of this specific-generic-specific cycle can be found in the ability of
the system to reuse pedagogical rules, reduce the specific per learning situation
adaptation authoring, and aid the independent authoring of both pedagogical
adaptation and game learning situations.
In ELEKTRA the above cycle uses skills assessment at the Inferring component as
the key means for moving from the specifics of a LeS to the generic capability to
reason pedagogically about the learner’s performance. For example, based on the
learner’s interactions with a learning situation evidence is produced for how they are
progressing. This evidence is translated into their acquisition of skills pertinent to the
domain in which the LeS is set. With no knowledge of the specific LeS,
recommendations are made based on this inferred progress. These recommendations
are pedagogically informed and indicate the type and level of influence required. It is
then the responsibility of the Game Engine to determine how this recommendation
will be realized.
4.4 The Cognitive Framework Behind Adaptivity
The cognitive framework underlying adaptivity in ELEKTRA is CbKST. In its
original formulation, Knowledge Space Theory (KST) [21] provides a set-theoretic
framework for organizing domain of knowledge and for representing the knowledge
of individuals. The basic idea behind KST is to assume prerequisite relations among
a finite set of problems (e.g., test-items). If a problem a is a prerequisite for a problem
b, from mastering problem b also the mastering of problem a can be inferred. Based
on the set of problems and the corresponding prerequisite relation a knowledge
structure is established which includes all admissible or meaningful knowledge states.
Each individual can be assigned to a specific knowledge state.
The behavioral KST was extended by explicitly referencing the latent skills and
competencies underlying the observable behavior [22], [23], [24]. On of the most
successful approaches to CbKST is the Competence Performance Approach (CPA) by
Klaus Korossy [25], [26]. The basic idea of CPA is to assume a basic set
Ε
of abstract
cognitive skills that are relevant for mastering the problems of a domain. The skill
state of an individual is the collection of all available skills, which is not directly
observable but can be uncovered on the basis of the observable performance on the
8
problems representing the domain. As in KST, prerequisite relations are defined on
the set of skills establishing a skill structure C, which contains all possible skill states.
Utilizing skill and problem functions, families of subsets of skills can be mapped to
problems, which can be mastered with the given set of skills and vice versa. By the
assignment of skills to the problems of a domain, also a performance structure or
problem structure on the set of problems is induced.
An example is given in Fig. 1. If we assume that skill s1 is a prerequisite for skill
s2, skill states, which include skill s2 but not all of its prerequisites, in this example
s1, are not admissible, at least from a deterministic perspective. This procedure allows
us to substantially reduce the number of possible skills states. Without prerequisite
relation, six skills establish a skill structure that includes 64 skill states, i.e. the power
set of all skills (26). The prerequisite relation in Fig. 1 establishes a skill structure that
includes only eleven possible skill states. A major advantage is that CPA allows
multiple, individual, not necessarily linear learning paths. This offers a sound model
of learning objectives which, in turn, establishes the very basis for game design as
well as learning design. Moreover, by an assignment of latent skills to LeS and single
objects (e.g., tools, switches, laboratory equipment, or books) within LeS using
problem and skill functions, the skill structure serves a basis for reasoning about the
skill state of a learner on the basis of the performance in LeS.
Adaptively providing a learner with suitable learning objects is a major mechanism
of existing approaches to adaptive e-Learning. This principle can also be utilized for
providing adaptivity on a macro-level in digital educational games. In the context of
game design, branched storylines for interactive, personalized storytelling could be
realized, characterized by multiple paths through the game environment. As discussed
in section 4.1, in the context of educational games the principle of macroadaptivity
has major drawbacks, however. First, it might be highly expensive to implement the
requirements of adaptive storytelling, which is a demand of an adaptive ordering of
LeS. Second, a plausible re-ordering might not be possible in a linear storyline and
third, a theoretical model for adaptive storytelling still has to be developed. For these
reasons, in the ELEKTRA project we pursue an approach to adaptivity on a micro-
level.
Fig. 1. Prerequisite relation (left) and the corresponding skill structure (right).
In order to meet the requirements of non invasive adaptive interventions,
traditional CbKST-based approaches to the assessment of skills and competencies
must be complemented with an approach of probabilistic competence diagnosis. In
contrast to traditional approaches to adaptive knowledge assessment, which is mostly
9
based on typical test items (e.g., a multiple choice task), adapting to a learner’s skills
on a micro-level requires a probabilistic assessment of knowledge based on the
learner’s actions within a LeS. Such LeS include a number of objects, which can be
manipulated by the learner in order to accomplish a more or less complex problem
solution task. For example, a task might be to focus a cone of light using available
objects such as an electric torch and a set of convex and concave lenses. An efficient,
probabilistic assessment of skills in this context requires interpreting the learner’s
actions and, thus, the sequence of positions (and alignments) of all objects. The
positions of all objects of a LeS constitute a problem solution state that can be
evaluated with a specific utility function, determining the correctness of the current
problem solution state, or in other terms, the learner’s approach to the task’s solution.
In order to assess the available and lacking skills of the learner, the skills, which are
necessary to correctly manipulate an object, must be assigned to each object. The
probabilistic skill assessment occurs by updating an initial probability distribution
over all skill states based on the utility value of the evaluation of the problem solution
state. For example, if the learner uses the correct lens in order to focus a cone of light
(i.e., a convex lens), the probabilities for skill states including the skill of knowing
that convex lenses focus light, are increased. This procedure enables detecting the
available and lacking skills of a learner. Moreover, on this basis also misconceptions
(e.g., using a concave lens to focus light) or incorrect loops in the problem solution
process (e.g., if the learners tries the same incorrect manipulations over and over
again) can be detected. On this basis the system can determine suitable interventions
(e.g., hints) for a specific problem solution state or sequence, which are directly
related to the underlying latent skills.
5. Conclusion
The ELEKTRA project has the visionary goal of making a significant step towards
educational games that can compete with commercial games in terms of immersion,
engagement, and motivation while being a full educational option, utilizing
appropriate balance between learning and gaming and relying on national curricula
and sound psychological, pedagogical, and instructional theories. Developing a
successful educational game, especially for older children, cannot be accomplished by
adding some extra learning to a recreational game and it cannot be accomplished by
adding some gaming to a traditional learning object. This aim requires the generation
of a new genre within which both worlds, recreational games and psycho-pedagogical
aims and methods, are meld. The adaptive technology which already exists and which
is extended will help to fully incorporate the advantages of digital games for
educational purposes and to make a significant step towards developing the interfaces
for next-generation e-learning.
Acknowledgments. The research and development introduced in this work is funded
by the European Commission under the sixth framework programme in the IST
research priority, contract number 027986.
10
References
1. Merrill, M.D.: First principles of instruction. Educational Technology Research and
Development 50 (2002), 43-59
2. Mitchell, A., Savill-Smith, C.: The use of computer and video games for learning: A review
of the literature. Learning and Skills Development Agency, London (2004)
3. Malone, T.W.: Toward a theory of intrinsically motivating instruction. Cognitive Science 4
(1981) 333–369
4. Prensky, M.: Digital game-based learning. McGraw-Hill, New York (2001)
5. Schulmeister, R.: Didaktisches Design aus hochschuldidaktischer Sicht - Ein Plädoyer für
offene Lernsituationen. In Rinn, U., Meister, D.M. (eds.): Didaktik und Neue Medien.
Konzepte und Anwendungen in der Hochschule. Medien in der Wissenschaft 21 (2004) 19
– 49
6. Van Eck, R.: Digital game-based learning: It's not just the digital natives who Are restless.
Educause Review 41 (2006) 16-30
7. Koper, R., Olivier, B.: Representing the learning design of units of learning. Educational
Technology & Society 7 (2004) 97-111
8. Squire, K.: Cultural framing of computer/video games. Game Studies. The International
Journal of Computer Game Research 1 (2002)
9. Gee, J.P.: What video games have to teach us about learning and literacy. Palgrave
Macmillan, New York (2003)
10. Bloom, S.: Taxonomy of educational objectives, the classification of educational goals –
Handbook I: Cognitive Domain. McKay, New York (1956)
11. Leclercq, D., Poumay, M.: The 8 learning events Model. Release 2005. [Retrieved online
February 5, 2007 from : http://www.labset.net/media/prod/8LEM.pdf]
12. Verpoorten, D.: The 8 learning events model: a pedagogic conceptual tool supporting
diversification of learning methods. In: Proceedings of the 6th IEEE International
Conference on Advanced Learning Technologies - (ICALT 2006) - Kerkrade, The
Netherlands (2006)
13. Clark, J.M., Paivio, A.: Dual coding theory and education. Educational Psychology Review
3 (1991) 149-170
14. Crawford, C.: The art of computer game design. McGraw-Hill, Berkley (1984)
15. Ip, A., Morrison, I.: Learning objects in different pedagogical paradigms. In: Kennedy, G.,
Keppell, M., McNaught, C., Petrovic, T. (eds): Meeting at the crossroads: Proceedings 18th
ASCILITE Conference. The University of Melbourne, Melbourne (2001) 289-298
16. Tyler, R.W.: Basic principles of curriculum and instruction. University of Chicago Press,
Chicago (1950)
17. Mateas, M., Stern, A.: Façade, an artificial intelligence-based art/research experiment in
electronic narrative [Retrieved February 5, 2007 from http://www.interactivestory.net/]
18. DeBra, P., Calvi, C.: AHA! An open Adaptive Hypermedia Architecture. The New Review
of Hypermedia and Multimedia 4 (1998) 115-139
19. Csikszentmihalyi, M.: Flow: The psychology of optimal experience. Harper and Row, New
York (1990)
20. Shernoff, D.J, et al.: Student engagement in high school classrooms from the perspective of
flow theory. School Psychology Quarterly 18 (2003) 158–176
21. Doignon, J.-P., Falmagne, J.-C.: Knowledge spaces. Springer-Verlag, Berlin Heidelberg
New York (1999)
22. Albert, D., Held, T.: Component based knowledge spaces in problem solving and inductive
reasoning. In: Albert, D., Luks, J. (eds.): Knowledge Spaces: Theories, Empirical Research
Applications. Lawrence Erlbaum Mahwah, NJ (1999) 15-40
11
23. Doignon, J.-P.: Knowledge spaces and skill assignments. In Fischer, G., Laming, D. (eds.):
Contributions to Mathematical Psychology, Psychometrics and Methodology. Springer-
Verlag, Berlin Heidelberg New York (1994) 111-121
24. Düntsch, I., Gediga, G.: Skills and knowledge structures. British Journal of Mathematical
and Statistical Psychology 48 (1995) 9–27
25. Korossy, K.: Extending the theory of knowledge spaces: a competence-performance
approach. Zeitschrift für Psychologie 205 (1997) 53-82
26. Korossy, K.: Modelling knowledge as competence and performance. In: Albert, D., Luks, J.
(eds.): Knowledge Spaces: Theories, Empirical Research Applications. Lawrence Erlbaum
Mahwah, NJ (1999) 103-132