Conference PaperPDF Available

Using Knowledge Space Theory to support Learner Modeling and Personalization

Figures

Content may be subject to copyright.
ELEKTRA | Report on scientific publication
Page 1|14
SIXTH FRAMEWORK PROGRAMME
Information Society Technologies (IST)
Project acronym:
Project Full title:
Enhanced Learning Experience and Knowledge Transfer
Contract number 027986
Instrument Specific targeted research or innovation project
Start date of project: 01 MARCH 2006
Duration: 24 months
End date of project: 29 FEBRUARY 2008
Project coordinator name: Daniel Schwarz
Project coordinator organisation name: LMR | GFKI e.V. (Cologne, Germany)
Report on publication in scientific journals / magazines
Author(s) Owen Conlan, Cormac Hampson, Ian O’Keeffe (Trinity College,
Dublin, Ireland ), Jürgen Heller (University of Graz, Austria)
Title of publication / article
Using Knowledge Space Theory to support Learner Modeling and Personalization
Book / journal / magazine "Using Knowledge Space Theory to support Learner Modeling and
Personalization" at the E-Learn 2006, World Conference on E-Learning
in Corporate, Government, Healthcare, and Higher Education, Honolulu,
Hawaii, USA, October 13-17.
Date of publication October 2006
Copyright status
Summary publishable YES
Article publishable YES
Conlan, O., O'Keeffe, I. ., Hampson, C., & Heller, J. (2006). Using Knowledge Space Theory to
support Learner Modeling and Personalization. In T. Reeves & S. Yamashita (Eds.),
Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and
Higher Education 2006 (pp. 1912-1919). Chesapeake, VA: AACE.
ELEKTRA | Report on scientific publication
Page 2|14
Abstract
A learners knowledge is often the key aspect towards which personalized eLearning systems
attempt to adapt. However, the assessment of their knowledge usually involves tedious and
time consuming questionnaires or making stereotypical assumptions about what they know. The
Knowledge Space Theory (KST) [Doignon and Falmagne, 1985; Albert and Held, 1999] offers a
means of efficiently and effectively determining the current knowledge of a learner. By applying
this theory to the analysis and determination of a learners knowledge, highly configurable
adaptive systems, such as APeLS [Conlan et al. 2002], can provide highly dynamic event driven
personalized adaptations based on up-to-date information about the learner. This paper
describes the culmination of collaborative research that has been carried out by Knowledge and
Data Engineering Group of Trinity College, Dublin and the Cognitive Science Section of the
University of Graz under the auspices of several European Commission funded projects.
ELEKTRA | Report on scientific publication
Page 3|14
Article
Starting with next page as submitted by the author
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 4|14
Using Knowledge Space Theory to support
Learner Modeling and Personalization
Owen Conlan1, Cormac Hampson1 , Ian OKeeffe1, Jürgen Heller2
1 Knowledge and Data Engineering Group,
Trinity College, Dublin, Ireland
E-mail: {Owen.Conlan, hampsonc, Ian.OKeeffe}@cs.tcd.ie
2 Cognitive Science Section, Department of Psychology,
University of Graz, Austria
E-mail: juergen.heller@uni-graz.at
Abstract: A learners knowledge is often the key aspect towards which personalized
eLearning systems attempt to adapt. However, the assessment of their knowledge usually
involves tedious and time consuming questionnaires or making stereotypical assumptions
about what they know. The Knowledge Space Theory (KST) [Doignon and Falmagne,
1985; Albert and Held, 1999] offers a means of efficiently and effectively determining the
current knowledge of a learner. By applying this theory to the analysis and determination
of a learners knowledge, highly configurable adaptive systems, such as APeLS [Conlan et
al. 2002], can provide highly dynamic event driven personalized adaptations based on up-
to-date information about the learner. This paper describes the culmination of
collaborative research that has been carried out by Knowledge and Data Engineering
Group of Trinity College, Dublin and the Cognitive Science Section of the University of
Graz under the auspices of several European Commission funded projects.
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 5|14
1 Introduction
Personalized eLearning systems [Brusilovsky, 1996; De Bra, 2001; Conlan et al. 2002] attempt to
reconcile several pieces of information about a learner in order to produce a learning experience that is
tailored towards their particular needs. For personalized eLearning or any learning experience to be
effective, appropriate pedagogical and educational theories must be the fundamental drivers in designing
and developing that experience. In contrast to this requirement, personalized eLearning systems are
often developed from a purely technical perspective often leading to well engineered, but educationally
inappropriate, or worse ineffective, systems. By developing the Adaptive Personalized eLearning Service
(APeLS), and its predecessor the Personalized Learning Service (PLS), through cooperation with
cognitive scientists and pedagogues, the Knowledge and Data Engineering Group (KDEG) of Trinity
College, Dublin has produced a system that is both effective and pedagogically flexible. This paper
describes the relationship between the Knowledge Space Theory (KST) and the development of APeLS.
The relationship between KST and APeLS is traceable through a number of European
Commission funded Information, Society and Technologies (IST) projects from 2000 to the present.
Namely there are three projects in which KDEG have collaborated with the Cognitive Science Section
(CSS) of the University of Graz EASEL (2000-2003), iClass (2004-present) and ELEKTRA (2006-
present). During this time CSS have evolved their theoretical constructs for the Knowledge Space Theory,
principally introducing the notion of skills [Heller et al., 2006] and confidence degrees [Leclercq et al.,
1993] into the theory
In their own right neither KST nor APeLS are solutions for personalized eLearning KST is a
theory that describes and models how knowledge and skills are learned and related, while APeLS is a
pedagogically flexible service for the reconciliation of multiple models towards producing tailored
eLearning experiences. When combined, however, these approaches become the fundamental building
blocks for a highly effective and flexible personalized eLearning solution. Pedagogical flexibility is still
maintained as neither KST nor APeLS prescribe a pedagogical approach.
This paper describes the parallel evolution of both of these approaches, tracing their growth
through a number of successful and innovative European Commission projects. The paper starts with an
introduction and overview of the Knowledge Space Theory, which describes the fundamental elements of
the theory. This is followed by a section that highlights the evolution of the relationship between KST and
APeLS by describing case studies that snapshot their evolution over the last 6 years including their
current state of the art use.
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 6|14
2 Overview of Knowledge Space Theory
Knowledge Space Theory provides a theoretical framework within which the knowledge or competence
state of a learner can be determined through an efficient adaptive assessment procedure, presenting the
learner with only a subset of all possible problems. A personalized learning system requires the
availability of a framework that allows for the formal representation of a whole body of knowledge while
providing a representation of the learners current state of knowledge. Based on such a framework,
methods for adaptive knowledge assessment and for suggesting a personalized learning path can then
be developed. Knowledge Space Theory, introduced by Doignon and Falmagne [1985], is proposed as a
basic framework that meets these requirements. A state-of-the-art report on Knowledge Space Theory is
presented in Doignon and Falmagne [1999], for an introduction into the theory and its applications we
refer the reader to Falmagne, et al. [1990].
The fundamental approach taken in KST is to reduce the number of possible questions asked of
a learner to an optimal set. In this way the Knowledge State of a learner may be assessed through the
minimum number of queries, thus achieving maximum efficiency. This is only possible by examining the
domain in which the questioning is occurring and identifying the underlying prerequisite relationships that
exist between concepts. For example, in the domain of algebraic multiplication it may be assumed that
the concept of multiplying whole numbers is a prerequisite to multiplying decimal numbers. If it is
determined that a learner cannot multiply whole numbers then it may be extrapolated that they cannot
multiply decimal numbers without questioning them further. Continuing the example, the knowledge
domain Q = {a, b, c, d, e} consists of the problems
Table 1
a 378 x 605 = ?
b 58.7 x 0.94 = ?
c 1/2 x 5/6 = ?
d What is 30% of 34?
e Gwendolyn is 3/4 as old as Rebecca. Rebecca
is 2/5 as old as Edwin. Edwin is 20 years old.
How old is Gwendolyn? Figure 1
The empirically observed solution behavior on a given knowledge domain Q will exhibit some
dependencies. One way to identify these kinds of dependencies is by drawing upon domain knowledge,
for example, by querying an expert (possibly a mathematics teacher or professional curricula developer in
the above example). A surmise relation is a binary relation on the set Q. Referring to the above
example, the expression p q means that whenever problem q is solved correctly then we can surmise a
correct solution to problem p. In other words, the mastery of problem q implies the mastery of p. Any
surmise relation on a given knowledge domain can be illustrated by a so-called Hasse diagram, in which
the mutual relationships between the problems are depicted in an economical way. The Hasse diagram in
figure 1 presents a surmise relation defined on the knowledge domain Q from the above example. The
relation is depicted by ascending sequences of line segments. For instance, from a correct solution to
problem b a correct answer to problem a can be surmised. The mastery of problem e implies correct
answers to problems a, b, and c, and the mastery of problem d implies the mastery of problems a and b.
From a correct solution to either problem c or problem a no inferences regarding the solution of the
remaining problems can be made.
From the above we have a formal framework for introducing pre-requisites in a knowledge
domain. The central question of how to represent the learners knowledge within this framework still
remains. The state of knowledge of an individual is identified with the subset of problems of the
knowledge domain Q, which this individual is capable of solving. This means that for a knowledge domain
of n problems there exist no less than 2
n potential knowledge states. Due to the mutual dependencies
between the problems, however, not each of the subsets of the set Q is a plausible knowledge state.
c
e
b
d
a
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 7|14
Whenever a problem b is contained in a knowledge state K Í Q, and from solving b we can surmise
solving problem a (i.e. we have the surmise relation a b), then a should be contained in the knowledge
state K, too. For the surmise relation of figure 1 the set {a, b, c} Í Q is a possible knowledge state, while
the subset {b, c, d} Í Q is not a possible knowledge state (b is in {b, c, d}, and we have a b, but a is not
in {b, c, d}). A collection K of knowledge states for a given knowledge domain Q is called a knowledge
structure, whenever it contains the empty set Æ and the set Q. In other words the collection K contains all
of the possible knowledge states that a learner may be in for a given knowledge domain. The knowledge
structure K, consisting of the knowledge states induced by the surmise relation of Figure 1, is given by
K = {Æ, {a}, {c}, {a, c}, {a, b}, {a, b, c}, {a, b, d}, {a, b, c, e}, {a, b, c, d}, Q }.
This representation of knowledge only focuses on the actual solution of the problems, and does
not refer to any underlying latent constructs (skills, competencies, problem demands, etc.) that may be
responsible for the observable behavior. There have been various extensions that integrate the
consideration of latent constructs into Knowledge Space Theory. By assigning to a problem the skills or
competencies that are relevant for mastering it, the solution behavior is linked to some underlying
cognitive constructs. Moreover, any such assignment completely specifies the possible knowledge states
in the considered knowledge domain. Various approaches have been devised that differ in their
assumptions concerning what are the necessary and sufficient skills for solving a problem. A number of
approaches are outlined in Falmagne et al. [1990] and developed further in Doignon [1994], or later in
Düntsch and Gediga [1995]. Korossy [1997; 1999] proposes an independent but similar skill-based
approach, the so-called competence-performance approach. Albert and Held [1994; 1999] devised a
method for constructing problems from components that are based on skills and demands the problems
pose to an individual.
Case Studies in Modeling Learner Knowledge using KST
There are two main approaches to developing adaptive systems; the first is through the use of embedded
strategies, which are either embedded in the engine and/or the media it operates across. This was the
approach taken in early Intelligent Tutoring Systems (ITS) in the domain of eLearning. The resulting
systems, while possibly effective in their adaptation, were very difficult to modify or repurpose. The
second approach, and that advocated in the design and implementation of APeLS, is the separation of all
of the constituent parts of the adaptation process into discrete models. Significantly, this includes the
model of strategy, referred to as ‘narrative’ in this paper. This second approach requires a generic
adaptive engine that is capable of reconciling the narrative with other models. This reconciliation is at the
core of the adaptation process.
The Knowledge and Data Engineering Group of Trinity College, Dublin has been developing such
a generic adaptive engine that supports the execution of adaptive strategies [OKeeffe et al., 2006] for the
past six years. The initial versions of the adaptive engine (AE) were tightly coupled to their use in the
eLearning domain. However, in the last couple of years this domain dependency has lessened to the
point where the current version of the engine, Adaptive Engine 3 (AE3), is independent to any specific
application domain. AE3 is the generic adaptation engine at the core of the Adaptive Personalized
eLearning Service (APeLS), while its predecessor, the original AE, was the engine integrated at the
centre of the Personalized Learning Service (PLS).
2.1.1.1.1 Case Study 1: Personalized Course on Mechanics
The first iteration of a KST informed adaptive system produced by the Knowledge and Data Engineering
Group, developed in 2001, was based on the PLS and developed as part of the EASEL [EASEL] project.
This project examined the discovery and integration of eLearning content and services into consolidated
offerings. One such service offered was an adaptive course to teach the Physics subject of Mechanics.
The course adapted to a learners prior knowledge by only offering material they were capable of learning
based on a pre-requisite analysis of their understanding. The adaptivity in the course was divided into the
following phases
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 8|14
1. Pre-test: When a learner first entered the system they were asked to answer four basic
questions about Mechanics. These questions did not use KST-based optimization and as such
the same questions were presented to every learner. Correct answers given to these questions
were used to build a basic model of the learners knowledge.
2. Dynamic Personalization: Once the pre-test had been completed the learner was presented
with learning material that the system determined they were capable of learning. This decision
was based on the surmise relationship between the concepts in the content. For example, if a
learner had knowledge of collisions in 2D spaces then they have the pre-requisite competency
necessary to learn about collisions in 3D spaces. The narrative executed in the PLS would
determine that content covering collisions in 3D spaces was now suitable to add to the learners
personalized course (but does not add it yet).
3. Dynamic Modeling: Modeling of the learners evolving competencies was performed by making
the assumption that once a learner had accessed and read content pertaining to a competency
that they knew it. This is obviously a major simplification. This information about the new
competency is added to the learners model.
4. Learner chooses to expand the course: The course does not change unless the learner
explicitly decides they are ready to learn more. When they are ready all of the dynamically
modeled information is processed and any further content they are capable of learning is
presented.
Phases 2-4 may be repeated continuously enabling the learner to gradual expand the set of content to
learn. The technical approach taken in implementing this adaptive service violated one of the rules of
model driven adaptivity [Conlan et al., 2002]. The systems used three models learner, content metadata
and narrative. The narrative embodied the generic rules for Knowledge Space Theory, i.e. that a piece of
content should not be added until all of its pre-requisite concepts had been met and the learner model
kept track of the learners competencies. It was the content metadata model that violated one of the key
concepts of the model driven approach separation of concerns.
The metadata representing the individual pieces of content contained pre-requisite information
that identified which concepts needed to be understood in order to understand this piece of content. In
this way the domain model, or Knowledge Space, was distributed across all of the content metadata. This
did not impact the effectiveness of the course or the effectiveness of KST as an approach to
personalization. It did, however, adversely impact the ability of course authors and designers to expand
the course. As part of the EASEL project the Mechanics course was extended using material supplied by
the Open University [OU]. In order to add more content the course designer needed an intricate
knowledge of all of the content metadata to ensure the pre-requisites of the new content could be met.
For example, the designer needed to ensure that cyclical dependencies weren’t introduced, i.e. that one
piece of content was a pre-requisite of another and vice versa, without there being an appropriate piece
of content that fulfilled a pre-requisite that would act as entry point into the cycle.
2.1.1.1.2 Case Study 2: Personalization in iClass
Personalization technologies are a central theme in the iClass project [iClass] with particular
consideration given to the complete life cycle of a personalized experience. At the centre of the project
are a number of services that support the personalization of concepts, activities and content as well as
services that monitor and profile the learner. The adaptive engine has seen its most dramatic evolution
through the course of this project. So too has the implementation of the Knowledge Space Theory.
The key challenges with respect to knowledge assessment in iClass were to incorporate the
notion of confidence degrees [Leclercq, D. et al. 1993; Leclercq and Poumay, 2003] and to examine the
importance of assessing skills (as against concepts). The former involves including an important meta-
cognitive check point into the assessment procedure. For every question asked of a learner, using the
KST approach, an associated confidence degree is also solicited. For example, if a learner is asked What
is the capital city of Australia? and they answer Sydney the previous knowledge assessment techniques
would only see this as an incorrect answer. In iClass the learner would also be asked to determine their
confidence in this answer. Continuing the above example, if the learner stated that they were highly
confident in their answer then this highlights a serious misconception. If, however, they stated that they
had little confidence in their answer then it may indicate that they merely guessed the answer. From the
KST perspective this information can be incorporated into the procedures for determining the knowledge
state of the learner by giving an associated probability of them being in a particular knowledge state. This
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 9|14
probability is based on the confidence expressed in the evidence used to determine the knowledge state.
From a personalization perspective the main benefit of this approach is that the personalization
mechanisms can compensate for low probabilities by providing additional learning material.
The second challenge regarding knowledge assessment addressed in iClass was to examine
knowledge not only in terms of the concepts known by the learner, but also in terms of the skills they had
acquired. This challenge arose from a pragmatic consideration many national curricula in Europe are
expressed both in terms of concepts and the associated skills acquired. For the iClass context, a skill was
defined as a concept with an action verb. In an attempt to bring a common understanding to the set of
verbs used Blooms Taxonomy [Bloom, 1956] is being investigated as a means of constraining the
vocabulary and understanding. The structures and mechanisms of KST are not heavily impacted by the
adoption of skills instead of concepts alone. The onus is very much on the monitoring of the learners
interaction with the system and on the types of questions asked in order to determine the learners skill
level. Again confidence degrees may be used.
From the personalization perspective the tailoring of an educational experience is split between
two different services the Selector service and the LO Generator service [OKeeffe et al., 2006]. Both of
these services are based on the third iteration of the adaptive engine, evolved as part of iClass, yet adapt
different things; the Selector is responsible for adapting the concepts and activities presented to a learner
based on their knowledge and preferences while the LO Generator is responsible for selecting or
assembling new learning objects from atomic content assets. It is the Selector, therefore, that has the
most interaction with the knowledge assessment service of iClass, referred to as the Monitor
[Muehlenbrock et al., 2005].
The primary advancement made through iClass has been a further separation of concerns with
respect to personalized eLearning. This is manifest in the creation of the Selector and LO Generator
services as the central services for personalization. This separation may also be seen in the
disaggregation of all of the other elements of personalized eLearning, such as the modeling of learner
knowledge, maintenance of learner preferences, the separate storage of the modeled information and the
distributed nature of content and activities [OKeeffe et al., 2006]. This separation has enabled the
evolution of the knowledge assessment to be carried out independently to the evolution of the
personalized eLearning, thus enabling different pedagogical approaches to be adopted for different
learners while still using the same knowledge assessment facilities. This contrasts with the approach
highlighted in Case Study 1 where the adaptation mechanism and knowledge assessment approach were
inextricably tied.
2.1.1.1.3 Case Study 3: Generic Strategy for Assessing Knowledge State
With the evolution of the Adaptive Engine to its third iteration (AE3) the opportunity arose to create a
narrative that could handle any Knowledge Space and assess learner knowledge generically. By
generically it is meant that the assessment performed is not tied to a specific knowledge domain. This
possibility came about with the inclusion of service handling facilities into AE3. The engine would no
longer have to rely on an embedded domain model, as it did in Case Study 1, or on a completely
separate and specialized service as was the case in Case Study 2.
With the capacity to call external services AE3 could now utilize the advantages of ontology
reasoning services. The role of the narrative in this version of the engine, referred to as the Knowledge
Assessment Engine (KAE) in the ELEKTRA project [ELEKTRA], is to produce meaningful queries to pass
to an ontology reasoning service. The functionality of the narrative may be extended, but currently it can
perform three functions
· Add/remove a concept to/from the model of the learners current knowledge
· Enquire about the possible knowledge states of a learner based on their current knowledge
· Ask what the next concept that should be questioned of the learner
The KAE, again because of the AE3 implementation, can be offered as a service itself, thus allowing
external services to utilize the operations it exposes. As part of the ELEKTRA project the KAE is invoked
by another adaptive engine that is attempting to ensure that the educational experience of a learner offers
a sufficient challenge. This separate Adaptive Engine invokes the KAE as a service to determine the
current possible knowledge states of the learner.
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 10|14
Architecturally this approach is very similar to that taken in the production of the Personalized
New Service [Conlan et al., 2006], indeed both implementations required no code changes to the AE3
and just needed additional narratives reflecting the different forms of adaptivity required to be developed.
Figure 2, below, shows the architecture of the KAE with a sample workflow for how a learner may be
questioned
1. A third party learning service invokes one of the operations offered by the KAE; in this case it is
asking for an appropriate question to ask the learner. In the ELEKTRA project the third party
learning service is also based on AE3.
2. The adaptive engine at the core of the KAE retrieves the known current knowledge for the
learner. Building on the example from the Overview of Knowledge Space Theory section the
learner model shows that the learner has the knowledge state of {a, c}.
3. The narrative for asking for the next appropriate question is triggered (based on the invocation).
This complex and highly involved narrative executes in the AE3 and assembles an appropriate
SPARQL [SPARQL] query by including information about the learners current known
knowledge state. This is where the true intelligent capabilities of AE3 are used. It is combining
model information with the complicated processing rules of the query language in order to
create a bespoke and appropriate query.
4. The Ontology Reasoning Service, which is based on Jena [Jena], is invoked and the SPARQL
query is passed to be processed. This service may either have the appropriate ontology loaded
into memory; if not it will load the ontology for reasoning.
5. The SPARQL query is processed and the result set is returned as a response to the AE3. The
response is in the form of one or more concepts related to the ontology. In this example the
concept relating to b is returned as the next most appropriate concept to be questioned.
6. Based on this response the AE3 calls a third party questioning service (such as QuizPACK
[Sosnovsky et al., 2003]). The only proviso on this invocation being successful is that the
metadata associated with the questions must use the same vocabulary as the ontology. The
appropriate question may then be returned to the learning service.
Figure 2
AE3
4) SPARQL
Query
Ontology Reasoning Service & KST Ontology
5) Response
Learner
Model
2) Learner
Knowledge
3rd Party
Questioning
Service
Knowledge Assessment Engine
Narrative
1) Invocation
3)
6)
3rd Party
Learning
Service
Learner
Internet
AE3
4) SPARQL
Query
Ontology Reasoning Service & KST Ontology
5) Response
Learner
Model
2) Learner
Knowledge
3rd Party
Questioning
Service
Knowledge Assessment Engine
Narrative
1) Invocation
3)
6)
3rd Party
Learning
Service
Learner
Internet
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 11|14
The primary benefit of this approach is that different ontologies may be loaded into the Jena service
enabling the knowledge of learners to be assessed across a wide variety of domains. The narratives used
in the KAE are both generic and extensible, making them agnostic to the domain for which they are
assembling queries. The extensibility enables more functionality to be added to the KAE as it is required.
The use of SPARQL and ontologies described using OWL [OWL] ensures that the approach is
conformant with current best practice in knowledge representation. However, since the ontology
reasoning is performed by a service the possibility to replace it with an alternative exists. The example in
this section has concentrated on assessing the concepts known by the learner, but the approach is
equally applicable to the skills assessment as described in Case Study 2.
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 12|14
3 Conclusion
This paper has described the evolution of the application of the Knowledge Space Theory for
personalized eLearning through three collaborative research projects since 2000. It has highlighted the
significant steps made in assessing learner knowledge both in terms of how the theory has evolved and in
terms of how the technical implementations have progressed. These steps have been shown through a
series of illustrative case studies.
The Knowledge Space Theory has been shown as a useful mechanism for efficiently assessing
the knowledge state of a learner. The case studies showed different mechanisms for realizing the theory
culminating in a generic service driven approach that enables third party learning services to utilize the
power of KST. This approach builds upon the most up to date technologies coming from the Semantic
Web community and it is envisaged that as these technologies mature so to will their associated tools.
For example, in the case on ontologies the authoring tools available are progressing all of the time. With
the capacity to author ontologies that represent knowledge spaces the generic approach applied by the
Knowledge Assessment Engine has much potential.
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 13|14
4 Acknowledgements
The authors gratefully acknowledge the European Commissions ongoing support for this work as
witnessed by the funding received under the IST FP5 and FP6 Framework Programmes in the guise of
the EASEL, iClass and ELEKTRA projects.
References
Albert, D. & Held, T. (1994). Establishing knowledge spaces by systematical problem construction. In D.
Albert (Ed.), Knowledge Structures (pp. 78–112). New York: Springer Verlag.
Albert, D., & Held, T. (1999). Component Based Knowledge Spaces in Problem Solving and Inductive
Reasoning. In D. Albert & J. Lukas (Eds.), Knowledge Spaces: Theories, Empirical Research, and
Applications (pp. 15–40). Mahwah, NJ: Lawrence Erlbaum Associates.
Bloom, B.S. (Ed.) (1956). Taxonomy of educational objectives: The classification of educational goals:
Handbook I, cognitive domain. New York, Toronto: Longmans, Green.
[Brusilovsky, 1996] Brusilovsky, P. Methods and Techniques of Adaptive Hypermedia, User Modeling and
User-Adapted Interaction, 6, pp. 87-129, 1996.
[Conlan et al., 2002] Conlan, O.; Wade, V.; Bruen, C.; Gargan, M. Multi-Model, Metadata Driven
Approach to Adaptive Hypermedia Services for Personalized eLearning. Second International Conference
on Adaptive Hypermedia and Adaptive Web-Based Systems, Malaga, Spain, May 2002
[Conlan and Wade, 2004] Conlan, O., Wade, V. Evaluating the Multi-model, Metadata-driven Approach to
producing Adaptive eLearning Services, Third International Conference on Adaptive Hypermedia and
Adaptive Web-Based Systems (AH2004) Proceedings, Eindhoven, The Netherlands (2004)
[Conlan et al., 2006] Conlan, O., O'Keeffe, I., Tallon, S., Combining Adaptive Hypermedia Techniques
and Ontology Reasoning to produce Dynamic Personalized News Services, Proceedings of the Fourth
International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH2006), Dublin,
Ireland (2006)
[De Bra, 2001] De Bra, P. AHA! Adaptive Hypermedia for All, project proposal.
http://www.nlnet.nl/project/aha/200106-aha-proposal.html
[Doignon, 1994] Doignon, J. (1994). Knowledge spaces and skill assignments. In G. H. Fischer & D.
Laming (Eds.), Contributions to Mathematical Psychology, Psychometrics and Methodology (pp. 111–
121). New York: SpringerVerlag.
[Doignon and Falmagne, 1985] Doignon, J.-P., & Falmagne, J.-C. (1985). Spaces for the assessment of
knowledge. International Journal of Man-Machine Studies, 23, 175-196.
[Doignon and Falmagne, 1999] Doignon, J.-P., & Falmagne, J.-C. (1999). Knowledge spaces. Berlin;
Heidelberg ; New York : Springer.
[Düntsch and Gediga 1995] Düntsch, I., & Gediga, G. (1995). Skills and Knowledge Structures. British
Journal of Mathematical and Statistical Psychology, 48, 9–27.
[EASEL] EASEL, Educator Access to Services in the Electronic Landscape, IST Project 10051.
[ELEKTRA] ELEKTRA IST FP6 Project, http://www.elektra-project.org/
[Falmagne et al., 1990]., J.-C., Koppen, M., Villano, M., Doignon, J.-P., & Johannesen, L. (1990).
Introduction to knowledge spaces: how to build, test and search them. Psychological Review, 97, 201-
224.
[Heller et al., 2006] Heller, J., Steiner, C., Hockemeyer, C., & Albert, D. Competence-Based Knowledge
Structures for Personalised Learning. International Journal on E-Learning, 5(1), 75-88 (2006)..
[iClass] Intelligent Distributed Cognitive-based Open Learning System for Schools (iClass), European
Commission FP6 IST Project. http://www.iclass.info.
[Jena] Jena Semantic Web Framework, http://jena.sourceforge.net/
ELEKTRA | Report on scientific publication
© THE ELEKTRA CONSORTIUM 2006-2008 Page 14|14
[Korossy, 1997] Korossy, K. (1997). Extending the theory of knowledge spaces: a competence-
performance approach. Zeitschrift für Psychologie, 205, 53-82.
[Korossy, 1999] K orossy, K. (1999). Modelling Knowledge as Competence and Performance. In D. Albert
& J. Lukas (Eds.), Knowledge Spaces: Theories, Empirical Research, and Applications (pp. 103–132).
Mahwah, NJ: Lawrence Erlbaum Associates.
[Leclercq, 1993] Leclercq, D. et al. (1993). Validity, reliability and acuity of self-assessment in educational
testing, in D. Leclercq & J. Bruno (Eds). Item banking: self-assessment and interactive testing, NATO
ARW, F112, Berlin: Springer Verlag, 114-131.
[Leclercq and Poumay, 2003] Leclercq D. & Poumay, M. (2003). Analyses édumétriques et indices
métacognitifs appliqués aux questios des 10 check-up MOHICAN. In D. Leclercq (2003), Diagnostic
cognitive et métacogntif au seuil de l’universi. Editions de l’université de Liège.
[Muehlenbrock et al., 2005] Muehlenbrock, M., Winterstein, S., Andres, E., & Meier, A. (2005).
Continuous Learner Modelling in iClass. In Proceedings of the World Conference on Educational
Multimedia Hypermedia & Telecommunications ED-MEDIA 2005.
[OKeeffe et al., 06] O'Keeffe, I., Brady, A., Conlan, O., Wade, V. (2006) "Just-in-time Generation of
Pedagogically Sound, Context Sensitive Personalized Learning Experiences", International Journal on E-
Learning (IJeL), Special Issue: Learning Objects in Context, Volume 5, Issue 1, pp 113-127. Chesapeake,
VA: AACE. (2006)
[OU] The Open University, http://www.open.ac.uk
[OWL] Web Ontology Language, http://www.w3.org/TR/owl-features/
[Sosnovsky et al., 2003] Sosnovsky, S., Shcherbinina, O., and Brusilovsky, P., Web-based parameterized
questions as a tool for learning. In: A. Rossett (ed.) Proceedings of World Conference on E-Learning, E-
Learn 2003, Phoenix, AZ, USA, November 7-11, 2003, AACE, pp. 309-316.
[SPARQL] SPARQL Query Language for RDF, http://www.w3.org/TR/rdf-sparql-query/
... Falmagne's model by incorporating the student's competence beyond performance, hence presenting the "Competency-based Knowledge Space Theory" (CbKST). Heller et al. [43] further CbKST by correlating skills required with the problem's nature. ...
... Knowledge Space Theory and its competence-based approaches have been successfully applied in the context of technology-enhanced learning for realising personalised learning paths and adaptive assessment (e.g. Albert, Hockemeyer, & Wesiak, 2002;Conlan, O'Keeffe, Hampson, & Heller, 2006;Kickmeier-Rust, Mattheiss, Steiner, & Albert, 2011). This theoretical framework enables the creation of tailored learning experiences that are characterised by an appropriate level of challenge for the learner and by didactically meaningful learning sequences based on the consideration of a knowledge domain's inherent structure (e.g. ...
Article
Full-text available
Ontologies play an important role as knowledge domain representations in technology-enhanced learning and instruction. Represented in form of concept maps they are commonly used as teaching and learning material and have the potential to enhance positive educational outcomes. To ensure the effective use of an ontology representing a knowledge domain it needs to be validated. In this paper a previously presented validation methodology for concept maps is exemplified. Two different types of concept map validity are distinguished, referring to the correctness of the concept map’s content (content validity) and to the applicability of the concept map for its designated purpose (application validity), like its use in intelligent tutoring. To demonstrate the usefulness of the two validation types and approaches, they are illustrated by an empirical study. The content validity of a concept map on elementary geometry has been investigated by comparing it with empirically collected criterion maps through s...
... It has been successfully applied in technology-enhanced learning (e.g. Albert, Hockemeyer, Kickmeier-Rust, Nussbaumer, & Steiner, 2012;Conlan, O'Keeffe, Hampson, & Heller, 2006). (Nussbaumer et al., 2015) used Competence-based Knowledge Space Theory to provide a systematic approach to model decision-making competences, adapting the above mentioned reviews of decision-making, and drawing comparisons between the self-regulation process, which includes critical thinking, metacognitive regulation, communication and teamwork and the decision-making process. ...
Article
Full-text available
Errors in decision-making worldwide highlight the need for training in decision-making. The unpredictability and complexity of emergencies makes training in every possible emergency impossible. Rather than training in specific examples of major emergency events, training in a decision-making skill set will provide a method of response that will be transferable to all emergencies. Various scenarios will support the training as a decision needs a context for application. The resulting educational tool will focus on emergency services at the strategic and tactical levels in the response stage of an emergency. The continual engagement of stakeholder should result in a purpose-built training course. Design science research approach will be utilised, investigating connections between theories of cognitive load and expert performance. Key aspects of the developed training course will include the concepts of metrics, deliberate practice and proficiency based progression, to ensure an appropriate training programme rather than a mere educational experience.
... The work on Competence-based Knowledge Space Theory (CbKST) and its applications as presented in this paper constitutes a continuous elaboration and evolution of this theoretical framework towards new directions in the field of technology-enhanced learning (see alsoConlan, O'Keeffe, Hampson, & Heller, 2006;Pilato, Pirrone, & Rizzo, 2008).This is an ongoing process of taking up current and new trends in education, in general, and technology-enhanced learning, in particular, and is reflected by the work inother projects, like MedCAP (http://www.medcap.eu/),TARGET (http://www.reachyourtarget.org/), ...
Chapter
The 21st century is challenging the future educational systems with ‘twitch-speed’ societal and technological changes. The pace of (technological) innovations forces future education to fulfill the need of empowering people of all societal, cultural, and age groups the acquire competences and skills in real-time for demands and tasks we cannot even imagine at the moment. To realize that, we do need smart novel educational technologies that can support the learners on an individual basis and accompany them during a lifelong personal learning and development history. This paper gives some brief insights in approaches to adaptive education based on sound psycho-pedagogical foundations and current technologies.
... If the learner exhibits evidence of being able to multiply decimal numbers it may be assumed that they can also multiply whole numbers. Such probabilistic reasoning enables a system to infer a learner's skill state based on partial evidence (Conlan 2006). ...
... In [27], beginning college chemistry students' understanding of stoichiometry is assessed using the knowledge space theory, with the results suggesting that there is a need for teaching students how to integrate their knowledge. A survey is provided in [28] on how people use the knowledge space theory to assess students' background knowledge in order to provide personalized learning. The students' thinking patterns are identified by using the knowledge space theory in [29] so the teachers can effectively guide the students along the critical learning paths suggested by the students themselves. ...
Article
In this paper, an automatic lesson generation system is presented which is suitable in a learning-by-mimicking scenario where the learning objects can be represented as multiattribute time series data. The dance is used as an example in this paper to illustrate the idea. Given a dance motion sequence as the input, the proposed lesson generation system automatically generates the lesson plan for students. It first extracts patterns from the input dance sequence to form the learning objects. The prerequisite structure is then built by considering the relations between the learning objects. Afterward the knowledge structure is constructed from the prerequisite structure based on the knowledge space theory. Finally, the learning path is derived according to an easy-to-complex manner while respecting the prerequisite relations. A user study that involved 40 students was conducted to evaluate the proposed work. The average learning time required for the treatment group (learning with the proposed system) was found to be lower than that of the control group (learning by free browsing) thus demonstrating the learning efficiency of the proposed system. The feedback from the questionnaires indicated that a majority of the subjects showed positive response toward the usefulness and rationality of our proposed system.
Chapter
Computing machinery allows the creation of intelligent, personalized, adaptive systems and programs that consider the characteristics, interests, and needs of individual users and user groups. In the field of serious games, storytelling and gaming approaches are used as motivational instruments for suspenseful, engaging learning, or personalized training and healthcare. This chapter describes models and mechanisms for the development of personalized, adaptive serious games with a focus on digital educational games (DEG). First, the term adaptation is defined—both in general and in the context of games—and basic mechanisms such as the concept of flow are described. Then, player and learner models are analyzed for classification of player characteristics. For the control of serious games, adaptive storytelling and sequencing mechanisms are described. In particular, the concept of Narrative Game-based Learning Objects (NGLOBs) is presented, which considers the symbiosis of gaming, learning, and storytelling in the context of an adaptive DEG. Finally, the presented theoretical concepts, models, and mechanisms are discussed in the course of the 80Days project as a DEG best-practice example—which considers authoring, control, and evaluation aspects, and its practical implementation in 80Days using the authoring framework StoryTec.
Article
Knowledge production and the demand for new curriculaChallenges to GIS&T curriculum developmentThe GIScience curricula development model - GISc-CDMConclusions AcknowledgmentReferences
Article
Full-text available
This article describes the narrative approach to personalisation. This novel approach to the generation of personalised adaptive hypermedia experiences employs runtime reconciliation between a personalisation strategy and a number of contextual models e.g. user and domain. The approach also advocates the late binding of suitable content and services to the generated personalised pathway resulting in an interactive composition that comprises services as well as content. This article provides a detailed definition of the narrative approach to personalisation and showcases the approach through the examination of two use-cases: the personalised digital educational games developed by the ELEKTRA and 80Days projects; and the personalised learning activities realised as part of the AMAS project. These use-cases highlight the general applicability of the narrative approach and how it has been applied to create a diverse range of real-world systems.
Article
Learning is a process that is associated with a lot of effort and perseverance. In learning theories, motivation can be observed as a key factor. In some cases learning can become playing if the learning experience is so intrinsically satisfying and rewarding that external pressures or rewards for learning are of secondary importance. Serious games are able to increase motivation for learning by realizing diverse approaches which can address cognitive as well as affective learning. By using a variety of elements such as visual environments, story-lines, challenges, and interactions with non-player characters, serious games can be optimal learning environments. Even though, they have such motivational power, several studies have shown that there are no known forms of education as effective as a professional human tutor. This paper explores the interaction of human tutors with learners in a serious games with the focus on ‘Social Development Theory’. It will present results that show how human tutors observe players in executing learning tasks, and interacting with the game environment in serious games. Based on the results of this studies we provide a definition of adaptivity for serious games.
Article
Full-text available
The just-in-time generation of personalized learning experiences requires the assembly of atomic learning assets into coherent learning activities for a learner, based on his/her preferences and requirements. Through the appropriate application of pedagogical strategy to a learner's learning activities the effectiveness and efficiency of his/her learning can increase significantly. The strategies behind this process should be pedagogically informed to ensure the learning experience is suitable for the learner and the environment in which they are carrying out his/her learning. By utilizing appropriate pedagogical strategies in the personalization process, learning objects generated for a learner will not only be appropriate to what they wish to learn, but also to how they should learn it. This article describes the Selector and LO Generator services of the iClass IST project and the approach taken to producing pedagogically sound personalized learning experiences using a standards-based approach.
Article
Full-text available
Adaptive hypermedia is a new direction of research within the area of adaptive and user model-based interfaces. Adaptive hypermedia (AH) systems build a model of the individual user and apply it for adaptation to that user, for example, to adapt the content of a hypermedia page to the user's knowledge and goals, or to suggest the most relevant links to follow. AH systems are used now in several application areas where the hyperspace is reasonably large and where a hypermedia application is expected to be used by individuals with different goals, knowledge and backgrounds. This paper is a review of existing work on adaptive hypermedia. The paper is centered around a set of identified methods and techniques of AH. It introduces several dimensions of classification of AH systems, methods and techniques and describes the most important of them.
Conference Paper
Full-text available
Applying traditional Adaptive Hypermedia techniques to the personalization of news can pose a number of problems. The first main difficulty is the fact that news is inherently dynamic, thus producing an ever shifting pool from which content can be sourced. The second difficulty arises when trying to model a users interests and how they may be related to the available news items. This paper investigates the use of ontologies as a means of providing semantic bridges between available news items from RSS [1] news feeds and the interests of a user. Specifically, it investigates the combination of AH techniques with the ideas of loose and strict ontologies as the basis for personalization. This combination is highlighted through the design, development and evaluation of the Personalized News Service (PNS), which is based on the APeLS architecture [2].
Chapter
Procedures which are to test a subject’s knowledge concerning a specific domain obviously require (in addition to other prerequisites) a set of problems.
Chapter
When teachers use confidence marking, they should be aware that confidence estimation and confidence expression are influenced by a series of factors. Some of them have been studied in detail, such as the general human capacity to estimate one’s knowledge (how far can people be sensitive, reliable and valid in appreciating their uncertainty). This paper indicates how some of these factors have been studied, the results and the implications for designing test Instructions, proper scoring rules and indices of the quality of self assessment.
Article
This chapter develops an extension of Doignon and Falmagne's knowledge struc-tures theory by integrating it into a competence-performance conception. The aim is to show one possible way in which the purely behavioral and descriptive knowledge structures approach could be structurally enriched in order to account for the need of explanatory features for the empirically observed solution behav-ior. Performance is conceived as the observable solution behavior of a person on a set of domain-specific problems. Competence (ability, skills) is understood as a theoretical construct accounting for the performance. The basic concept is a mathematical structure termed a diagnostic, that creates a correspondence be-tween the competence and the performance level. The concept of a union-stable diagnostic is defined as an elaboration of Doignon and Falmagne's concept of a knowledge space. Conditions for the construction and several properties of union-stable diagnostics are presented. Finally, an empirical application of the competence-performance conception in a small knowledge domain is reported that shall illustrate some advantages of the introduced modeling approach.
Article
Suppose that Q is a set of problems and S is a set of skills. A skill function assigns to each problem q i.e. to each element of Q — those sets of skills which are minimally sufficient to solve q; a problem function assigns to each set X of skills the set of problems which can be solved with these skills (a knowledge state). We explore the natural properties of such functions and show that these concepts are basically the same. Furthermore, we show that for every family K of subsets of Q which includes the empty set and Q, there are a set S of (abstract) skills and a problem function whose range is just K. We also give a bound for the number of skills needed to generate a specific set of knowledge states, and discuss various ways to supply a set of knowledge states with an underlying skill theory. Finally, a procedure is described to determine a skill function using coverings in partial orders which is applied to set A of the Coloured Progressive Matrices test (Raven, 1965).
Article
The information regarding a particular field of knowledge is conceptualized as a large, specified set of questions (or problems). The knowledge state of an individual with respect to that domain is formalized as the subset of all the questions that this individual is capable of solving. A particularly appealing postulate on the family of all possible knowledge states is that it is closed under arbitrary unions. A family of sets satisfying this condition is called a knowledge space. Generalizing a theorem of Birkhoff on partial orders, we show that knowledge spaces are in a one-to-one correspondence with AND/OR graphs of a particular kind. Two types of economical representations of knowledge spaces are analysed: bases, and Hasse systems, a concept generalizing that of a Hasse diagram of a partial order. The structures analysed here provide the foundation for later work on algorithmic procedures for the assessment of knowledge.