Towards a Web Service for Competence-based Learning and Testing
Department of Psychology, Cognitive Science Section
University of Graz, Austria
Institute for Information Systems and Computer Media
Graz University of Technology, Austria
Department of Psychology, Cognitive Science Section
University of Graz, Austria
Abstract: Adaptation to the learner’s knowledge level and competence-based course creation
have been a major goal of e-learning research projects. However, besides the research projects
there is a powerful and emerging development of many open source learning management
systems with little or no adaptation functionality. In order to bring together widespread and
popular learning management systems and scientific-oriented approaches of competence-based
personalisation, this paper presents a solution how a personalised e-learning approach can be
applied to arbitrary Web-based learning management systems. A Web service approach is
presented which enables and controls adaptation of learning management systems. The scientific
foundation is realised by implementing the Competence-based Knowledge Space Theory which is
a framework for representing and connecting domain ontologies and knowledge and competence
levels of learners.
The concept of adaptation has often been addressed in literature (Brusilovsky 1996; Conlan 2002; De Bra 1999)
over the last two decades. In an early paper about adaptive hypermedia (Brusilovsky 1996) gives an overview about
methods and possible application areas of adaptation. One of the most important and interesting area is the
educational application domain. Based on differences with respect to learners' knowledge states and preferred
learning styles, adaptive learning systems can support the learning process by adapting to such parameters and
characteristics. Also known as personalisation, this might be one of the most important system behaviours of an
adaptive e-learning system. This paper presents a new system design which is based on a flexible system
architecture and which makes use of competences for personalisation purpose.
In the field of cognitive psychology there has been done much research to model knowledge domains of curricula
and knowledge states of persons. Knowledge Space Theory (KST) is a behaviouristic theory which provides a set-
theoretic framework for representing the knowledge of a learner. The knowledge state is a set of problems which a
person is able to solve. Due to psychological dependencies between problems, not all combinations of knowledge
states are possible. The collection of the possible knowledge state is called the knowledge structure. KST provides
mathematical-based methods for testing the learner’s knowledge state. Since KST takes into account only
observable behaviour and does not refer to learning objects, skills, and competencies, approaches have been made to
extend this theory in this direction. The Competence-based Knowledge Space Theory (CbKST) incorporates
underlying cognitive constructs in order to explain observable behaviour. In this sense skills and competencies can
be formally modelled and assigned to learning objects and to learners (Heller 2006; Korossy 1999).
In order to use CbKST in e-learning applications, it is necessary to model and to build machine readable relations
between content, learning objects, learners, and skills. The skills (and competencies) are the cognitive constructs
which are assumed that learners have if problems can be solved. They can be assigned to learners as well as to
learning objects. In the following the term learning object is used for both digital objects which teach content and
test objects which are problems posed to learners for the reason of assessing their knowledge states. To include the
curriculum in this model, knowledge domains are modelled as concept maps and concepts are related to skills by
combining concepts with actions verbs, such as understanding or knowing a concept. The relation to a learner is
established by relating skills to learners. All relations and model components together form an ontology which is
machine readable and can be used for information processing (Görgün 2005).
Several implementation approaches have been made, which outline diverse use cases of CbKST. Examples are the
commercial application ALEKS (ALEKS, 2007) which implements KST, as well as the research projects RATH
(Hockemeyer 1999) and APeLS (Hockemeyer 2003) which make use of CbKST. The proposed solution of this
paper aims at being more general in order to cover all these approaches and to simulate them.
Beside the scientific research stream of technology-enhanced learning and teaching many Web-based learning
management systems (LMS) have emerged which already pervade teaching organisations. A lot of these LMS are
open source systems, such as Moodle (Moodle 2007) or Ilias (Ilias 2007), which advantages their spreading. There
is a huge community of people including even teachers who develops those platforms. However, in contrast to
systems developed in scientific environments, they mostly have poor or even no adaptation functionality in terms of
the learners' knowledge levels.
To take into account the widespread usage of those popular LMS, the proposed solution approach aims at utilising
the CbKST framework for personalisation purpose and to apply it on arbitrary LMS. Therefore a Web service is
being created which learning platforms can be connected to. This Web service implements the models and methods
of CbKST and controls the personalisation behaviour of the connected LMS. The user connects to the platform in
the usual way and benefits from the additional behaviour without having knowledge of the Web service in the
The next section discusses the educational approach which is based on the CbKST and which is implemented in a
system consisting of both an arbitrary learning platform and the CbKST Web service. Then a section depicts the
technical realisation of the Web service and how it is connected to an arbitrary LMS. After that first implementation
results are explicated together with the context where the research and development is done. Finally, an overview of
possible future work in terms of technical realisation and theoretical concepts is given.
2 Educational Approach
To explain the basic educational approach, this section starts with two scenarios followed by arguments for the basic
idea and an explanation which problems can be solved and which functionality is provided.
The first scenario describes a typical situation where a teacher in a school has already chosen Moodle as LMS. After
having analysed the curriculum, learning objects and a course is being made for the class. Since Moodle does not
provide personalised learning in terms of competences, the teacher decides to uses the CbKST Web service.
Competencies and skills have to be formally defined which are taught by the learning objects. Then the skills have
to be assigned to the respective objects, which implicitly spans a knowledge space. The learners' first task is to make
a knowledge assessment to determine their individual competence state. After that a personalised course can be
offered to the pupils who take into account their individual knowledge level. The teacher makes the pedagogical
choice that the degree of freedom in the course should be rather narrow, so the pupils have little possibility to
influence the learning path.
The second scenario describes the situation of an in-firm training program. A project manager defines a competence
level in a specific field which should be achieved by employees. The company has already a LMS in used of which
the usage is well known by the employee. To tackle the task of bringing numerous project collaborators to a specific
knowledge level, the CbKST Web service is employed. The necessary skills which define the competence level have
to be modelled and stored in the system and learning objects are created and skills are assigned to them. Each
employee can start the course after having made a knowledge assessment to automatically determine the pre-
knowledge. Then a course based on the individual competence level is offered by the system, whereby the
employees can choose among a list of appropriate learning objects. Since the employees are supposed to be highly
qualified, it makes sense to choose a didactical approach with a broad degree of freedom. After having done the
course, a further assessment is conducted to ensure that each employee has achieved the necessary skills and reached
the required competence level.
As denoted in the scenarios, many teachers and learners are used to specific LMS and have already created and
stored content and test items in their preferred LMS. Moreover, they have registered and stored their profile and
other data which their LMS can process. Therefore it makes sense to leave this information in the particular LMS
and to use it by a separated adaptation service. The Web service stores and manages competency information and
meta-data of the learning objects, which together is described by ontologies. Furthermore it controls the personalised
learning process of the LMS which the learner is connected to. Apart from the personalisation functionality provided
by the Web service the users can benefit from all features of the LMS in their habitual way.
The possibilities of this approach extend ordinary LMS by determining competence levels and by creating
personalised learning paths based on these individual levels. Since this approach is based on the formal CbKST
model, scientifically proven personalisation behaviour is ensured.
Knowledge Assessment has often been discussed in the literature (Doigon 1999). Applying this method to arbitrary
LMS is a new opportunity to improve platforms without personalisation features. If learning objects which test the
learners' knowledge and which are already available in an LMS can be assigned with skills and they can be used for
the competence-based assessing. When determining the pre-knowledge of an individual learner, the course can be
adapted to the learner's competence level, so learners do not have to complete learning objects which teach
competencies they already have.
The CbKST framework builds on a model which includes and connects learning objects, competences, and
concepts. The mesh of these components can be described by ontologies which brings them into a standardised
form. These relations offer several didactical possibilities how to create learning paths and how to define learning
objectives. The sequence of learning objects can be determined along the skills they require and they teach. Since
there is a prerequisite structure on the skills, it can be ensured that learning objects which have fewer prerequisites in
terms of pre-knowledge are sequenced before learning objects with more prerequisites. In this case the learning
objective can be defined as a set of competencies or skills. The CbKST framework also allows sequencing learning
objects along concept relations which are covered by learning objects. In this case the learning objective would be
defined as a set of concepts which should be taught.
Though the CbKST framework provides algorithms for learning and testing, it allows possibilities for self-regulated
learning. The learner can choose the degree of freedom with respect to influencing the learning path and adaptive
assessment. Regarding the ontological connections between learning objects, skills, and concepts, multiple
possibilities are often available how to sequence learning objects. Therefore different didactical approaches can be
chosen, if the learner is guided more or less. A fully guided course would mean that the learner can not select among
learning objects, while a less guided course enables the learner to choose among a list of learning objects.
Determining the competence state of a learner can also be seen as locating it on the ontological knowledge map.
Consequently, visual reports can be given to the learner, which gives a clear feedback of the learning progress.
Observing competence state and progress brings great benefit for learner and teacher, because in this way meta-
cognition and self-reflection are fostered.
3 System Design
This section discusses the solution approach of the implementation of the psychological concepts described above.
First, the network aspect and architecture is outlined to show how the system is distributed over the Internet. Second,
the system behaviour, its components, and its functions are explained in order to give a detailed picture of the
The system and network architecture consists of three main parts which are connected over the Internet, as it is
depicted in Figure 1. The first part is the Web browser which is the interface to the learning system for the user. It is
connected to the LMS over HTTP, since the LMS is a Web-based application which runs within a Web server. The
core system of the LMS is the original application which has to be extended by CbKST personalisation functions.
To extend the core system, an extension module is needed which is tailored to the individual core LMS. This module
establishes the connection to the Web service and is therefore responsible for the connection between the core LMS
and the Web service. The Web service as the third part is responsible for the personalisation and individualisation
functionality as described above.
As shown in Figure 1, several LMS of different types can be connected to one single CbKST Web service. Typically
the LMS is located in an educational institution in order to provide learning content for their pupils. It is the choice
of this institution, if an own CbKST Web service or a service located somewhere on the Internet is used. In any case
the user does not need to have any knowledge about this Web service.
Figure 1: System and network architecture. The diagram shows the main components which
are distributed over the Internet. Different LMS can be connected to the same CbKST Web service.
The particular modules and the functionality of the system are graphically represented in Figure 2. This diagram
allows a view inside the main parts of the system for the reason of showing the modules and their relations to each
The core LMS stores the learning objects (digital objects for teaching and testing) as well as user profile data, which
is a requirement of the LMS that it can store those data. The extension module is a plug-in for the core LMS and has
its main purpose in applying the CbKST functionality on the core LMS. Therefore it must have access to the
learning objects and the profile data of users in order to send information about these data to the Web service.
Second an event handling must be implemented for the purpose of getting informed if data have been modified.
Third, an ontology authoring module is needed which provides the Web interface for the user to modify the
ontological representation (see below).
The adaptation service is designed as a Web service which gets the necessary information about content and user
and which controls the adaptation of the course to the learners' needs. It is designed to be independent from any
LMS and to communicate via a communication protocol with the LMS extension module. Requirements for this
protocol has been described in (Nussbaumer 2007). In this way it controls the adaptation in terms of conducting
assessment and learning path creation. In other words, it selects learning and assessment objects to be presented to
The CbKST Web service stores and manages the ontologically represented data as described in the introduction
section. These are meta-information of the learning objects, a list of all skills and competencies, the prerequisite
structure on the skills, the concepts, the knowledge and competence states of learners and the relations among all
them. Consequently, there is a clear separation of digital content which is managed by the LMS and the meta-data
which are managed by the Web service. Learning objects are modelled by assigning a type (testing and teaching)
and by assigning skills which are taught by a learning object and skills which are required to understand a learning
Furthermore, the ontological representation acts as a database of skills and their prerequisite structure which can be
successively extended on the long term. Then the database can be reused by other content creators. In that sense, this
service not only provides adaptation functionality, but also valuable information about competencies.
The CbKST Web service has two modules, one which has implemented the algorithm for knowledge and
competence assessment, and the other for creating personalised learning paths. For this purpose, appropriate
algorithms have to be chosen and implemented. The CbKST framework already provides such algorithms which can
rather easily be implemented, since they are mathematically formulated. They make use of both the ontologies and
the competence and knowledge states of the learners.
Teacher and Learner
States of Users
Figure 2: Modules architecture. The diagram shows the modules and data used in the system
and their functional relations to each other.
4 First Implementation Results
The described Web service is currently being implemented and is expected to be demonstrated by end of 2007.
Furthermore, two different LMS are being adapted in order to be connected to this Web service. First, the popular
LMS Moodle is being extended by using the genuine extension mechanism of Moodle. Second, the mathematical
tutoring system ISAC (ISAC 2007) is being modified, which will demonstrate the flexible interface approach of the
Parts of this work are done in the context of the iClass research project (iClass 2004) and it should demonstrate the
feasibility of the concepts and methods researched for iClass. It should demonstrate that they can be implemented in
a system and used by learners and teachers.
Creating the ontology, which includes the tasks of creating skills and a structure on them and assigning them to
documents, may be a tedious work if it is done through a conventional Web interface. Since the ontology can be
graphically represented, information visualisation research can offer various methods to support the content and
ontology author in creating and editing the ontology. A future version of this system is intended to have respective
A further field where information visualisation brings benefit is the reporting of a learner's knowledge state and
progress. CbKST offers great possibilities to represent graphically knowledge and competence state. Theses states
could be identified and marked on knowledge and competence maps. This offers possibilities for learners and
teachers to trace the learning progress on a graphical map and to watch the gap between the current competence state
and the learning objective.
Beyond competence-based learning and testing the Web service can also undertake tasks, such as managing access
permissions of content, billing the used content, and dealing with intellectual properties.
ALEKS (2007). http://www.aleks.com/
Brusilovsky, P. (1996). Methods and techniques of adaptive hypermedia. Journal on User Modeling and User-Adapted
Interaction, 6, 87-129.
Conlan, O., Hockemeyer, C., Wade, V., & Albert, D. (2002). Metadata driven approaches to facilitate adaptivity in personalized
eLearning systems. Journal of Information and Systems in Education, 1, 38-44.
De Bra, P., Brusilovsky, P., and Houben, G. (1999). Adaptive hypermedia: from systems to framework. ACM Computing
Doigon J.P., Falmagne, J.C. (1999). Knowledge Spaces. Berlin: Springer
Heller, J., Steiner, C., Hockemeyer, C., & Albert, D. (2006). Competence-Based Knowledge Structures for Personalised
Learning. International Journal on ELearning, 5, 75-88.
Görgün, I., Türker, A., Ozan, Y., & Heller, J. (2005). Learner Modeling to Facilitate Personalized E-Learning Experience.
Cognition and Exploratory Learning in Digital Age 2005 (CELDA'05), pp. 231-237.
Hockemeyer, C., & Albert, D. (1999). The Adaptive Tutoring System RATH. In M. E. Auer & U. Ressler (Eds.), ICL99
Workshop Interactive Computer aided Learning: Tools and Applications. Villach, Austria: Carinthia Tech Institute.
Hockemeyer, C., Conlan, O., Wade, V., & Albert, D. (2003). Applying competence prerequisite structures for eLearning and skill
management. Journal of Universal Computer Science, 9, 1428-1436.
iClass (2004), http://www.iclass.info/
Ilias (2007). http://www.ilias.de/
ISAC (2007). http://www.ist.tu-graz.ac.at/projects/isac/
Korossy, K. (1999). Modelling knowledge as competence and performance. In D. Albert & J. Lukas (Eds.), Knowledge Spaces:
Theories, empirical research, applications (pp. 103-132). Mahwah, NJ: Lawrence Erlbaum.
Moodle (2007). http://www.moodle.org/
Nussbaumer, A., Gütl, C., Hockemeyer C. (2007). A Generic Solution Approach for Integrating Adaptivity into Web-based E-
Learning Platforms. International Conference on Interactive Mobile and Computer Aided Learning 2007 (IMCL'07).
The work presented in this paper is partially supported by European Community under the Information Society Technologies
(IST) program of the 6th FP for RTD - project iClass contract IST-507922. The authors are solely responsible for the content of
this paper. It does not represent the opinion of the European Community, and the European Community is not responsible for any
use that might be made of data appearing therein.
Parts of this paper are based on experiences gained in the AdeLE project. This project is partially funded by the Austrian
ministries BMVIT and BMBWK, through the FHplus impulse program. The support of the Department of Information Design,
Graz University of Applied Sciences (FH JOANNEUM) and Institute for Information Systems and Computer Media (IICM),
Faculty of Computer Science at Graz University of Technology as well as individuals involved in the AdeLE project are