Learning from Experience: Towards a Formal Model for Contextualization of Experiences for Quality Development

Abstract

Quality development in e-learning becomes more and more important. A variety of quality approaches have been developed to improve educational processes in this field. The implementation of these approaches, however, often lacks success. One important reason is that experiences and recommendations are not systematically utilized for the implementation processes. The following article suggests a formal approach to collect and share experiences and recommendations to help users of quality approaches enhance their success by reusing experiences of others. Based on knowledge management principles, the contextualization of experiences will help to transfer them from one specific situation to a different context. The suggested approach is a result of the European Quality Observatory, an analysis and information platform for quality approaches in European e-learning.

Full text

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience: Towards a Formal
Model for Contextualization of Experiences for
Quality Development
Markus Bick, Ulf-D. Ehlers, Jan M. Pawlowski,
Heimo H. Adelsberger
University of Duisburg-Essen
Abstract: Quality development in e-learning becomes more and more important. A
variety of quality approaches have been developed to improve educational proc-
esses in this field. The implementation of these approaches, however, often lacks
success. One important reason is that experiences and recommendations are not
systematically utilized for the implementation processes. The following article
suggests a formal approach to collect and share experiences and recommenda-
tions to help users of quality approaches enhance their success by reusing experi-
ences of others. Based on knowledge management principles, the contextualiza-
tion of experiences will help to transfer them from one specific situation to a dif-
ferent context. The suggested approach is a result of the European Quality Obser-
vatory, an analysis and information platform for quality approaches in European
e-learning.
Keywords: Experience Sharing, Quality Management, Knowledge Management,
Quality Decision Process
1 Introduction
The number of quality approaches that have been developed for the different edu-
cational fields, along with the large number of methods (such as criteria lists,
guidelines, management approaches), has grown to an unmanageable amount (a
comprehensive overview can be found in [EhPG2003]). However, it is not just the
number which makes it difficult to select a suitable approach, but the often diver-
gent intentions and recommendations of the approaches as well. On the European
level, this problem becomes even more severe because of different cultural con-
texts, languages, and educational systems come in, resulting in a vast multitude of
diverse regional and differentiated sectoral approaches.
Therefore, the decision which quality approach is suitable for an educational or-
ganization is a highly complex task – which envisioned as a decision process – can

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
768 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
be called the Quality Decision Process. This paper describes how experiences
made by users of quality approaches can be described and structured in a standard-
ized way and thus become searchable and accessible. We show how the quality
decision process as well as the process to use and implement quality approaches
can be supported by using knowledge management principles: the formalization
and reuse of experiences to improve quality development.
The proposed method for this ambitious aim is to construct a formal model which
allows to analyze experiences, and to specify recommendations against the back-
ground of their contexts. The difficulties and the boundaries of such an approach
are quite clear from beginning: can there be something like a generalization of ex-
periences – defined as perceptions in a specific context – at all? The methodology
we propose to enhance quality development is based on formal description models
to enhance decision processes and to make recommendations according to certain
characteristics. We provide solutions to analyze and compare quality approaches
and thus improve the fit between organizational and individual requirements and
the selection of quality approaches resp. strategies.
With this work we reach beyond a methodology for choosing and recommending
quality approaches – developments which are already implemented in the “Euro-
pean Quality Observatory (EQO)”, a European portal for e-learning quality. The
mechanisms already implemented there provide support for quality development
in e-learning through profile based recommendation mechanisms [MaSa2004].
With this paper we are also looking into combinations of knowledge management
approaches and quality development because we believe that apart from finding
the appropriate solution, the implementation process is of critical importance for
the success of every quality project in organizational contexts.
2 Knowledge Management for Quality Development
in E-Learning
2.1 Quality Development in E-Learning – a Learning Cycle
Building on Experiences
Quality in e-learning decides over success or failure for this form of learning. This
is quite clear today – on a national level as well on an international level. How-
ever, because of the nature of the concept “quality”, what it means is highly de-
pending on perspectives and contexts, and how it is achieved is a field of great de-
bates. Today, it is apparent that quality on the one hand is a very important factor
on all educational levels in European education, but on the other hand is a moving
target without clear cut definitions, understandings, and ways to achieve it. We

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 769
have described reasons and dimensions for that in [Ehle+03b, Ehle2004] which
state that quality in e-learning in Europe is a field of great diversity: Many differ-
ent approaches on different levels, for different educational sectors, contexts, and
target groups compete with each other in the area of quality management, assur-
ance, and assessment. Not only different techniques and methods to enhance qual-
ity in e-learning are used here, but also different levels of educational quality in
the various approaches, and different concepts of quality itself can be recognized
(e.g., pedagogical quality, technological quality, economical quality, etc.). Ap-
proaching this field on a European level (i.e., beyond the borders of national dis-
courses) makes it even more complex.
The problem with finding a suitable approach for a specific educational purpose,
e.g., improving the learner support in an e-learning course for secretaries, is no
longer that there are too few approaches to choose from, but rather that it is diffi-
cult to structure the divers field of approaches properly, and to map a set of re-
quirements against the approaches available. The European Quality Observatory
(http://www.eqo.info) addresses this problem by providing a formal model for de-
scribing quality approaches and thus making them comparable and searchable in a
database.
However, it becomes more and more evident that quality development is not only
a matter of finding a suitable approach but also strongly depends on an appropri-
ate implementation process, and moreover on aspects which vary from context to
context: Staff trainings, motivational conditions, organizational restructuring
processes, costs, and alike. Depending on how much quality development focuses
on changing, resp. redefining individual work patterns, believes, and values, it can
be envisioned as a learning process itself. For organizations, we consider three
possible groups of strategies for quality development. The first group of strategies
we call official external strategy, the second we call official internal strategy and
the third groups of strategies we call implicit strategies:
• Official External Strategy: Quality strategies or instruments coming from ex-
ternally adopted approaches (e.g., ISO, EFQM, BAOL Quality Mark)
• Official Internal Strategy: Quality strategies that are developed within an or-
ganisation but still apply to the organisation as a whole or parts of it specifi-
cally
• Implicit strategies: Quality development which is not part of an official strat-
egy but rather left to individuals’ professional activities.
For the two official strategies, quality development is part of the official organiza-
tional policy. Quality in these cases can be envisioned as a learning cycle rather
than an isolated single occasion. It usually aims at changing organizational proc-
esses and addresses actors to change their “traditional” patterns of behavior. Qual-
ity development in those cases deals with applying new rules and proceedings but
it will only be successful if it aims at stimulating awareness for quality improve-

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
770 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
ment on the side of the organizations’ members. Therefore, quality development
in an organization always builds on changing behavior, competencies, and be-
lieves of its members. At this point it becomes quite obvious that mechanisms
which apply to learning and knowledge management processes also apply to or-
ganizational change processes in form of quality development. To enable organ-
izational actors to learn from others’ experiences, e.g., of previous implementation
processes, therefore bears important potential to raise success of quality improve-
ment processes in education. It becomes clear that one important factor of quality
development – apart from a methodology and a value model – is knowledge man-
agement which aims at stimulating learning processes in actors’ competence de-
velopment processes.
Our research in this field addresses aspects of success and failure in implementa-
tion of quality improvement strategies. It means to explore, classify, and assess the
impact which quality approaches have on the various processes of the educational
environment, and to gain knowledge about implementation processes. Therefore –
apart from describing quality approaches – we are concerned with finding formal,
generic description categories for experiences and recommendations which users
of quality approaches have. In order to make such experiences reusable, they have
to be analyzed according to the related context. The identification of such attrib-
utes enables transfer of experiences into different contexts.
This leads to the conclusion, that research on the experiences which organizations’
individual actors make, during and after the implementation process of quality ap-
proaches, is necessary. More knowledge is needed about the context factors which
determine success or failure of quality approaches, and about how quality ap-
proaches can/should be adapted according to those contexts. In addition research
about the possibility of transferring relevant experiences from one to another con-
text is necessary. The aim is to derive generalizable knowledge from the analysis
of subjective perceived effects of the implementation and use of quality ap-
proaches in specific contexts. Learning from experiences thus means to find rec-
ommendations from implementation and usage experiences.
2.2 Learning to Share Experiences from Knowledge
Management
Reuse and transfer of knowledge is one important element for improvement – as
we have seen both in the field of knowledge management as well as in the field of
quality development. As described above, the reuse of experiences can become an
important factor for quality development in education as well. However, espe-
cially in the field of education, reusing and assessing experiences for quality de-
velopment purposes is fairly new ground. In the following we show how knowl-
edge management concepts can be applied to the above mentioned learning and
decision processes. This work is based on previous work as described in

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 771
[ABLP2004]. During the past decade, knowledge management has emerged as
one of the most important and widespread management issues. Knowledge man-
agement finds its origins in a desire to learn from mistakes and to hinder the “re-
invention of the wheel” in organizations [ReRa2001]. In the past decade, the im-
portance of knowledge as a key resource has become well established (cf., e.g.,
[Druc94; Maie2002]).
We use a definition of knowledge management by MAIER which is on the one
hand general enough to support all kinds of different knowledge areas and on the
other hand regards management in a functional sense:
“Knowledge management is defined as the management function responsible for
the regular selection, implementation, and evaluation of goal-oriented knowledge
strategies that aim at improving an organizations’ way of handling knowledge in-
ternal and external to the organization in order to improve organizational per-
formance. The implementation of knowledge strategies comprises all person-
oriented, organizational, and technological instruments suitable to dynamically
optimize the organization-wide level of competencies, education, and ability to
learn of the members of the organization as well as to develop collective intelli-
gence.” [Maie2002, 55]
Although MAIERS’ definition has a slightly technocratic notion and it could be de-
bated wheather it is possible at all to stimulate individual competency develop-
ment through external strategies, we still want to use an important distinction. Ac-
cording to this definition, two approaches to knowledge management exist: hu-
man-oriented (personalization strategy) and technology-oriented (codification
strategy) [HaNT1999; Lehn2000]. This terminology shows the two sides of
knowledge management which can be differentiated between (see also Table 1):
• Human-oriented/personalization strategy: Knowledge is closely tied to the
person who constructed it. Knowledge is mainly shared through direct person-
to-person contacts. Information and communitcation technology (ICT) just
supports people to communicate knowledge, not to store it. One example in
our context is experience sharing within a community or within an organiza-
tional context in which activity patterns change and people share their experi-
ences because of quality development processes.
• Technology-oriented/codification strategy: This strategy addresses the com-
puter technology resp. ICT: Information is (carefully) codified and stored in
‘databases’ where it can be accessed and used easily [HaNT1999]. The formal
model of experiences is an example which could be used in the frame of such a
strategy to supply people with a standardized set of infomration.
Besides this, more recent knowledge management approaches suggest to follow a
holistic approach of knowledge management, bridging the gap between human-
oriented and technology-oriented knowledge management [Albr1993; Lehn2000].
Hansen et al. [HaNT1999] identified certain strategy-mixes within their survey: A

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
772 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
company pursues one strategy predominantly and uses the second strategy to sup-
port the first.
human-oriented technology-oriented
knowledge man-
agement strategy Personalization Codification
comprehension of
knowledge
knowledge is contained in
peoples head
Knowledge is rather
understood as stored,
documented informa-
tion, detached from em-
ployees
actors/roles
knowledge worker, net-
works, and communities
of interest
authors, experts, infor-
mation broker
knowledge man-
agement systems
(KMS)
interactive knowledge
management systems
integrative knowledge
management systems
important knowl-
edge management
system functions
communication and coop-
eration, allocation of ex-
perts, community-support,
human captial manage-
ment
publication, structuring
and integration, search,
presentation and visu-
alization of information
elements
EQO component Community, expert net-
work, experience sharing
Formal experiences,
analysis model
Table 1: Classification of Knowledge Management Approaches and Systems based on
[MaHä2001]
ICT is the enabler for knowledge management activities, especially technology-
oriented knowledge management. Correspondingly, a great variety of so called
knowledge management systems (KMS) arose (cf., e.g., [EpSe2000; Lehn2000;
Klos2001; Maie2002]). With regard to the above mentioned knowledge manage-
ment approaches (see Table 1), we can derive a distinction between integrative
knowledge management systems, focussing on method-, product-, and process
knowledge, and interactive knowledge management systems, focussing person-
oriented knowledge [MaHä2001; Maie2002b]:
• Integrative knowledge management systems: support the codification of infor-
mation search and retrieval as well as the administration of information reposi-
tories and the organization of knowledge structures. The European Quality Ob-
servatory provides methods for codification, search, and retrieval.
• Interactive knowledge management systems: support location of experts, their
communication and collaboration, provides shared homespaces for communi-
ties, and modern e-learning instruments. The European Quality Observatory

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 773
provides a community in this context, including possibilties to improve com-
munication and cooperation. People can therefore get in contact about their
quality development strategies and challenges, offer their knowledge, and re-
flect on experiences.
Furthermore, MAIER [Maie01] identifies functions bridging the gap between
knowledge integration and knowledge interaction, especially for supporting
knowledge intensive (business) processes. In our context, the specific (business)
process is the choice and implementation of quality approaches for educational
organizations (see also next section). In this respect, we link business processes to
knowledge management processes in our concept. In our view, this link is crucial
to every qality development process – including the subprocesses of choosing the
right strategy and implementing/introducing a quality strategy into an organiza-
tion. The term process is used with respect to knowledge management in three
connotations [Remu02]:
• knowledge-intensive operative (business) process
denotes a (business) process that relies substantially on knowledge and relates
to organizations core competencies on the operative level: e.g., choice and im-
plementation of a quality approach for an organization.
• knowledge process
refers to a dedicated service- or support process which supports the flow of
knowledge within and between knowledge-intensive operative (business)
processes: e.g., search, analysis, etc.
• knowledge management process
is a ‘meta’-process that is responsible for the extensive implementation of the
knowledge management initiative: e.g., organizational instruments, ICT in-
struments, controlling, etc.

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
774 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
Figure 1 depicts the link between knowledge-intensive (operative) business proc-
esses and knowledge processes.
processes
knowledge base
content/
topic
processes
strategy
instruments/
systems
knowledge
life cycle
Figure 1: Process orientation according to [Remu02]
As we have shown in this chapter, the complex Quality Decision Process as a
knowledge intensive (business) process can be supported by different knowledge
management strategies, such as codification and personalization. Moreover the
introduction or implementation process relates to such processes as well.
3 European Quality Observatory – Providing a
Decision Cycle for Quality Strategies
The European Quality Observatory (EQO) is an information, knowledge, and sup-
port source for educational institutions and other organizations, as well as for us-
ers. In the following, EQOs’ contribution to the previously presented Quality De-
cision Process is shown and the decision process – envisioned as a cycle – itself
will be described.
The Quality Decision Process consists of four phases (see Figure 2). In the first
phase, quality apporaches are collected. These approaches are analyzed using the
so called EQO Model – the EQO model is a conceptual model to describe, com-
pare, and analyze quality approaches (for EQO model: [EHPT2004] or

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 775
http://www.eqo.info). After this analysis, usually several quality approaches are
chosen as alternative solutions for an organization – this decision process is based
on preference profiles which we provide for assesment. Finally, a decision for one
apporach is made – the approach then needs to be adapted and in parts be rede-
fined, leading to an organization-specific approach. Figure 2 shows the decision
process and the results of the harmonization.
Figure 2: Quality Decision Process [EhPa2004]
This decision cycle is supported through the different information and community
services EQO offers:
• Structuring and comparing quality approaches for the field of e-learning:
The variety of quality initiatives (e.g., ISO 900x:2000, EFQM) and quality ap-
proaches (e.g., different abstraction levels, national/regional/local approaches,
domain specific approaches, process-orientation / product-orientation / compe-
tency-orientation) leads to a increasing confusion in the community. There is
no accepted quality-mark for organizations or products on a European or
global level. Therefore, the approaches are compared, classified, and transpar-
ently structured in the EQO repository – which is internet accessible – in order
to provide a survey of actual approaches. The repository provides information
for decision makers in organizations and users in the field of E-learning. Dif-
ferent levels of abstraction are be covered: Quality management, quality assur-
ance, and quality assessment. By providing structured, context-related infor-
mation, the analysis phase of the decision process is supported.
• Building a community of practice in order to reach a common understanding
of and increase expertise on the concept of “E-Learning Quality”: The first

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
776 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
parts are part of the codification strategy. However, EQO integrates these ap-
proaches into a personalization strategy by providing a community and meth-
ods to improve communication and cooperation.
Currently, various initiatives, projects, and experts deal with the issue of qual-
ity for E-learning separately. There is no central platform for the exchange of
information and expertise. By building an interactive community of practice,
users and experts from different contexts can start the exchange of informa-
tion, experiences, and expertise.
The next step towards a knowledge management approach for quality develop-
ment in e-learning would be to provide information retrieval, recommendation,
and communication possibilities which are individually designed for specific or-
ganisational contexts in the field of e-learning.
4 The Difficulty of Finding a Formal Model for the
Evaluation of Experiences
In this chapter we will present a solution for the above discussed problem of stan-
dardizing the description of experiences. A formal model for such a description
has to meet four requirements which will be discussed first. In a second step the
formal model itself will be presented, consiting of three sections: general informa-
tion, recommendations, and assessment.
4.1 Conditions and Constraints in Classifying Experiences
Learning from experiences is a difficult task because experiences are usually
highly context specific. Therefore it is difficult to map existing experiences to
other specific contexts. The method we propose to use in order to make experi-
ences reusable is to develop a formal model which specifies standardized catego-
ries for the description of experiences. Information on experiences could then be
mapped to certain contexts, could be searched by certain keywords, and can alto-
gether be examined in a systematic way to find generalized structures. To map
those experiences to specific contexts, we need to structure and formalize the de-
scription of this information.

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 777
We know that experiences on quality approach implementation processes cannot
easily be generalized because of their specific, related context. Each of those ex-
periences on the other hand could contain valuable information for (other) organi-
zations which may find themselves in a similar situation and face the same or
similar problems.
The challenge in finding a description model therefore is twofold: a) to find ap-
propriate description categories that are restrictive enough to make experiences
comparable and b) to find a suitable way of being open enough in description, so
that individual characteristics can still be integrated into such a description. Such a
model therefore has to be open at certain parts and clearly structured into pre-
defined categories in other parts, to meet theses challenges. This goal can only be
achieved against the background of four conditions/ restrictions:
1. Restrictive in description but holistic in coverage: The model which we are
looking for, has to be capable of describing experiences concerning the whole
educational process; in this sense it has to be holistic. It can not be reduced to cer-
tain processes only, the learning phase or the evaluation phase in the end of a
course, for example. It must allow a description of experiences for all processes
which a quality approach can possibly affect. On the other hand it has to be able to
structure actors’ experiences in a way which allows others to learn from them.
They must be transferable, applicable, structured, and searchable.
The solution for this, on first sight contradictory requirement, is the integration of
a well structured process model with a model for describing experiences. It allows
to describe experiences or recommendations for the use of quality approaches for
specifically those educational processes which are affected. Collections of data
about what is important and what has to be avoided can thus in particular be re-
lated to a certain process of education. As an example, actors concerned with cur-
riculum development can directly learn from recommendations and experiences
specifically for this process and do not have to deal with overall recommendations
for implementing a guideline for instructional design in general. To restrict de-
scriptions of experiences and recommendations to a certain model of process cate-
gories that is capable of covering the whole educational scenario therefore allows
searching and comparing, but also assessing the impact which a certain quality
approach has on certain educational processes.
2. Being aware of the difference between intentional and external effects: An
educational organization, a course with its actors, teachers, learners, technical and
administration staff can be conceptualized as an organic working system. Imple-
menting a quality approach in this system or in specific parts of this system (e.g.,
guidelines for course authors, screen designers, or tutors) usually changes the
whole system. There is no such thing like partial improvement of e-learning sce-
narios without affecting other interconnected processes and actors without produc-
ing so-called “external effects” or “side effects”. In previous works, we described
the interdependencies between technological processes, economical conditions,

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
778 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
and educational services in e-learning environments [Ehle+03a]. A formal model
for describing experiences can only cover incidents related to intentional im-
provement processes. Unintentional side effects are too context specific and
rooted in singular specialties of individual constellations to be of specific value for
other organizational contexts. Therefore, we are aware of the boundaries: To con-
struct a model that is able to cover experiences for all educational contexts and
processes in principal needs to be of a more generic nature. Of course, such a
model has to allow for the possibility to describe external effects as well, but it can
not specifically concentrate on these effects. To acknowledge this weakness of a
formal model for describing individual experiences does not reduce the impor-
tance of such a model. It still leaves enough space to be of a helpful nature to other
users of quality approaches.
3. Accepting the paradigm of co-production in the pedagogical scenario: Al-
though of its prefix ‘e’, e-learning still remains learning. Learning processes are
not led through producer driven structures only but rather constitute a process of
co-production between the learner/the client and the educational environment.
Quality approaches generally aim at improving processes on the side of the educa-
tional environment to – that’s the philosophy – enhance the possibilities for the
learner to improve in his/her own learning processes. The concept of empower-
ment describes this development and will eventually lead to a better understanding
of what quality in e-learning for the learner could be. To accept the impossibility
to direct the whole learning process and to steer it from the producer side, means
also to give the floor to uncertainty. The learner has to be enabled, but will he also
use the provided structures for a ‘better’ learning process? And, what if not? In
exactly this field, experiences gain their importance for quality development be-
cause they provide knowledge that goes beyond the officially documented con-
cepts. Actors in an educational scenario can often learn a lot more from these ex-
periences than from their official quality-handbook or their guidelines because
they get to know that the pure implementation of a quality approach alone is no
guaranty for improvement processes. There still have efforts to be undertaken to
learn how the learners react to the newly structured learning environments. These
are very valuable experiences that are important to all educational professionals.
4. Accepting the difficulty of generalizing the singularity of events: To collect
experiences, to structure them accordingly and thus make them comparable and
applicable does not automatically mean that they are transferable. Experiences are
highly dependent on a specific context in which they are made. They also are
highly dependent on the view of the actor who contributes the experience, on the
language he uses, and on the cognitive/affective structure and background of those
who are perceiving the contribution. We are aware of this ‘transfer problem’.
Learning from experiences therefore does not mean that the plain transfer of ex-
periences works. However, it is indisputable that the perception of reference
knowledge – may it contradict or enforce ones own experience – can lead to a
greater reflection of a situation and thus a greater competency. It is important to

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 779
acknowledge that the pure provision of experiences can only stimulate learning
processes on the users’ side but that experiences can not be transferred directly to
a different context. For our model we therefore suggest to ask users for their rec-
ommendations rather for an account of their experiences. It will become clear that
the scope of recommendations is much more directed at providing usable knowl-
edge to other users than the scope of experiences. Therefore it seems reasonable to
aim at building a concept that provides recommendation for usage and implemen-
tation where users can learn from other users’ experiences.
Against the background of these four conditions we suggest the following model
to describe experience based recommendations for quality approach implementa-
tion processes.
4.2 A Formal Model for Experience-Based Recommendations
The European Quality Observatory has proposed a conceptual model which con-
tains a section for the categorization of experiences and which serves as the basis
for the following scheme [EQO03]. The proposed model therefore is strongly
based on the EQO model for describing quality approaches and experiences.
However, some of the categories proposed here are new to the original model and
some categories of the original model are changed into similar or slightly different
ones. Basically, we suggest to analyze experiences according to three different
sections:
1. The first section collects general information about the context, where the ex-
perience took place and about the actors, who are reporting the experience. We
believe that these background factors are constitutional for the assessment and for
recommendations someone states. It will be interesting to analyze how actors from
different contexts report about their experience with a quality approach in differ-
ent ways. Research in this field can follow the question, how a certain context in-
fluences the impact that a quality approach has.
2. In the second section we want to collect the recommendations which actors
have on basis of their experiences. In our model we propose a three step approach
for this analysis. In a first step the user can choose a process from the process
model for which he wants to state a recommendation (we use the process model of
the CEN/ISSS WS LT, [CENI2002]). This concept follows the assumption that
quality approaches are usually not affecting all processes in an educational sce-
nario but are directed at a few specific ones. Therefore we want to give the user a
specific option to choose a process which was affected by the quality approach
and for which he/she then would like to contribute recommendations. In a second
step we provide the possibility to specify what has been done resp. how the quality
approach was instantiated for this specific process. With this description we lay
the foundation for the next step in which we ask the user for a recommendation
which is rooted in the experiences he/she has made. As stated above, we believe

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
780 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
that asking the user not for an experience but for a recommendation leads to a
more goal oriented, more specific information.
3. In a third section we then ask the user for an assessment of the overall quality
approach. There are basically two methodological approaches to assess quality
approaches: an open and qualitative way and a standardized, more quantitative
way. The advantages of collecting assessments in a standardized and quantitative
way are obvious because then they can be rated and ranked against users require-
ments. The question which quality approach is best against the background of a
certain standardized item can be easily answered. But there are also problems with
a standardized way of assessment because it follows the assumption that all qual-
ity approaches can be assessed on basis of the same criteria. This seems odd – not
only because of the variety of contexts we already described above but also be-
cause of the even greater diversity of implementation contexts and actors. We
therefore suggest a methodological mixture of approaches. In a first step, the user
is asked to rate his assessment against a set of criteria we provide (e.g., “economi-
cal improvement”: 60% success). In a second step he/she then can contribute
his/her own criteria and provide an assessment for these (e.g., “comprehensiveness
of the used expressions in the quality approach”: 20% success). It is very impor-
tant that users are able to refine the standardized categories by specifying their
own because it can well be that an analysis category for “economical success” is
plainly too broad for description.
In the retrieval interface, the individually specified assessment categories should
be provided to users who are searching through the experiences. They also should
be collected in a special index and connected to the search and browse options.
Those keyword-lists will be a growing pool of evaluation criteria from the users
perspective, on which basis the standardized items should regular be refined.
Table 2 shows the proposed formal model for describing experience based rec-
ommendations for quality approach implementation processes.
A. General Information and Context: Overall information of the quality approach
application process
Identifier The quality approach description in the EQO database to
which the described experience relates
1. Description of the organizational context of the experience.
1.1 Name The name of the organization where the experience was
made
1.2 Country The Country of the organization in which the experience
with the QA was made

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 781
1.3 Size of the organization where the experience took place
1.3.1 Size Number of Employees
1.3.2 Project size Number of People involved in the quality approach imple-
mentation process
1.4 Educational
Level
Educational level of entity in which the quality approach
was implemented (e.g., school)
2. Contributor: Who contributed the experience/recommendation?
2.1 Status First contribution
Refined contribution
2.2 Function Function of contributor within organization
2.3 Entity Name, Address, etc.
2.4 Date Date of contribution of experience
2.5 Function of re-
sponsible actor
Function of responsible actor for the current implementation
process within organization?
3. Prior experience with quality approach implementations
3.1 Description Specifies experiences with usage of quality approaches in
the past
3.2 Date Date (year) of the prior experience
4. Time span Duration of the current implementation process
4.1 Start Date Approx. start date of implementation process
4.2 End Date Approx. end date of implementation process
5. Description Short account of implementation process
6. Target Group Who was/were the target group(s) of the quality approach
implementation?
7. Cost: Total implementation cost
7.1 Cost Model Direct (e.g., a certain amount for a certification process or
an external audit) or indirect costs (e.g., internal staff train-
ings, etc.)
7.1.1 Amount Amount of money
7.1.2 Currency €, US-$, etc.
7.1.3 Description Description of costs, especially indirect costs

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
782 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
B. Recommendation
1. Process: This category describes to which (educational) processes the quality
approach was applied and specifies recommendations
1.1 Name Name of the educational Process (CEN Model)
1.2 Relation Name of the educational process(es) from the originating
model (other process models)
1.3 Description Description of how the user instantiated this process with
regard to the quality approach (What was done?)
1.4 Recommenda-
tion
The user specifies recommendations considering his/her ex-
periences with the usage of the quality approach
C. Assessment: Assessment of the experience and the actual effect of the quality
approach
1. Quality concepts The user specifies how quality is defined in his/her view
(Quality is… faultlessness, conformance to standards, ex-
cellence in performance and perfection, the best value for
money, defined through pedagogical transformation and
learning achievements)
2. Assessment: The user assesses the quality approach
2.1 Name of stan-
dard criteria
User chooses from Standard-Criteria for assessment (Qual-
ity of… institutional and executive commitment, techno-
logical infrastructure, student services, instructional design
and course development, instruction and instructors, pro-
gram delivery, financial health, regulatory and legal com-
pliance, evaluation and assessment procedures)
2.1.1 Assessment
Value
This element describes the contributors expert assessment
regarding the enhancement of the quality of the specific cri-
teria.
2.1.2 Importance
Weight
This element describes the importance that the specific cri-
teria had for the application of the specific quality approach.
2.2 Name of user
specific criteria
Description of users’ own assessment criteria

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 783
2.2.1 Assessment
Value
This element describes the contributors expert assessment
regarding the enhancement of the quality of the specific
criteria
2.2.2 Importance
Weight
This element describes the importance that the specific cri-
teria had for the application of the specific quality ap-
proach
3. Success Factors Free text description of what have been proved to be key
factors of success
4. Failure Factors Free text description of what have been proved to be key
factors of failure
Table 2: Formal model for experiences
5 Conclusion
We have shown that quality development as a knowledge intensive (business)
process can be supported by different knowledge management strategies, such as
codification and/or personalization. The described approach shows how experi-
ences from one specific context (e.g., quality approach implementation in a high
school) can be reused for other contexts in form of recommendations. We showed
that knowledge management concepts can be used in the area of quality develop-
ment. This integration of concepts becomes fruitful because quality development
in organizational contexts implies individual actors’ learning processes which can
be improved by sharing experiences. The suggested formal model represents a
codification strategy by specifying information categories for substantially impor-
tant fields concerning the topic of implementation, context, and assessment for
quality approach implementation processes.
The implementation of the model in form of a database will lead to a growing pool
of structured data that allows users to reuse other users’ knowledge for their own
purposes and contexts. In the field of experiences and reusing of knowledge, it is
important to acknowledge certain conditions, stated above, that constitute the
boundaries of such an approach. Nevertheless, it is important to proceed with the
development of knowledge tools that allow to provide structured access to rec-
ommendations and thus help in decision situations [EhPa2003]. Experiences in a
database which are structured according to the above suggested formal model
could be directly mapped to certain context variables, and therefore provide valu-
able knowledge for specific requirements, e.g., if a user is searching for specific
recommendations for the university level. The research in the field of experiences

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
784 M. Bick, U.-D. Ehlers, J. M. Pawlowski, H. H. Adelsberger
in the implementation of quality approaches for educational contexts will deliver
important knowledge on impact and effects that will lead to an overall improve-
ment of quality development in education.
However, we made clear that quality development also needs a personalization
strategy for sharing experiences. This strategy must be supported by the codifica-
tion strategy in order to enable people to build their success on experiences. This
can be integrated into the quality development in form of communities for ex-
change and reflection but also in form of more organized organizational forms of
information and knowledge exchange.
References
[ABLP2004] Adelsberger, H.H.; Bick, M., Lodzinski; T., Pawlowski, J.M.: Electronic Per-
formance Support Systems in Production and Operations Management – State of the
Art and Future Development. Submitted: FACES 3 (2004) 5.
[Albr1993] Albrecht, F.: Strategisches Management der Unternehmensressource Wissen
[Strategic Management of Knowledge]. Peter Lang, Frankfurt 1993.
[CENI2002] CEN/ISSS Workshop Learning Technologies: CWA14644 Quality Assurance
Standards, CEN, Brussels 2002.
[Druc1994] Drucker, P. F: Post-Capitalist Society. Harper Business, New York 1994.
[EGHJ2003a] Ehlers, U.-D.; Gerteis, W., Holmer; T., Jung, H. (ed.): E-learning-Services
im Spannungsfeld von Pädagogik, Ökonomie und Technologie. L³-Lebenslanges Ler-
nen im Bildungsnetzwerk der Zukunft. W. Bertelsmann, Bielefeld 2003.
[Ehle2004] Ehlers, U.-D.: Qualität im E-Learning aus Lernersicht. Grundlagen, Empirie
und Modellkonzeption subjektiver Qualität. Verlag für Sozialwissenschaften, Wiesba-
den 2004
[EhPa2003] Ehlers, U.-D.; Pawlowski, J.M.: Qualitätsentwicklung im E-Learning: Ein Ent-
scheidungsmodell für die Auswahl von Qualitätsansätzen in Europa, In: Fietz, G., Go-
dio, C., Mason, R.: eLearning für internationale Märkte. Entwicklung und Einsatz von
eLearning in Europa. W. Bertelsmann, Bielefeld 2003.
[EhPG2003] Ehlers, U.-D.; Pawlowski, J.M.; Goetz, L.: Qualität von E-learning kontrollie-
ren, Die Bedeutung von Qualität im E-learning, In: Hohenstein, A., Wilbers, K.: Hand-
buch E-learning, Expertenwissen aus Wissenschaft und Praxis: Strategien, Instrumente,
Verlag Deutscher Wirtschaftsdienst, Köln 2003.
[EHPT2004] Ehlers, U.-D.; Hildebrandt, B.; Pawlowski, J.M.; Teschler, S.: The European
Quality Observatory. Enhancing Quality for Tomorrow’s Learners. Conference Pro-
ceedings of 3. EDEN Research Workshop 03/2004, Oldenburg 2003.

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg
Learning from Experience 785
[EQO2004] European Quality Observatory: European Quality Observatory: The EQO
Model, Draft Paper version 1.2, University of Duisburg-Essen, http://www.eqo.info,
2004, Extracted on 01-07-2004.
[HaNT1999] Hansen, M. T.; Nohria, N.; Tierney, T.: What’s Your Strategy for Managing
Knowledge? In: Harvard Business Review, 77, (1999), 3-4, pp. 106-116.
[Klos2001] Klosa, O.: Wissensmanagementsysteme in Unternehmen: State-of-the-Art des
Einsatzes [Knowledge Management Systems in Organizations: State-of-the-Art]. Deut-
scher Universitäts-Verlag, Wiesbaden 2001.
[Lehn2000] Lehner, F.: Organisational Memory. Hanser, Munich 2000.
[MaHä2001] Maier, R.; Hädrich, T.: Modell für die Erfolgsmessung von Wissensmanage-
mentsystemen [A model to measure success of knowledge management systems]. In:
WIRTSCHAFTSINFORMATIK 43 (2001) 5, pp. 497-509.
[Maie2001] Maier, R.: State-of-Practice of Knowledge Management Systems: Results of an
Empirical Study. In: INFORMATIK/INFORMATIQUE (2001) 1, pp. 14-22.
[Maie2002] Maier, R.: Knowledge Management Systems: Information and Communication
Technologies for Knowledge Management. Springer, Berlin 2002.
[MaSA2004] Manouselis, N.; Sampson, D.: A Multi-criteria Model to Support Automatic
Recommendation of e-Learning Quality Approaches, Proceedings of the 16th World
Conference on Educational Multimedia, Hypermedia and Telecommunications (ED-
MEDIA 2004), Lugano, Switzerland, June 2004.
[Remu2002] Remus, U.: Prozessorientiertes Wissensmanagement [Process-oriented
Knowledge Management]. PhD-Thesis. University of Regensburg 2002.
[ReRa2001] Reeves, T.C.; Raven, A.: Performance-Support Systems. In: Adelsberger,
H.H., Collis, B., Pawlowski, J.M. (Eds.): Handbook on Information Technologies for
Education & Training. International Handbook on Information Systems Series,
Springer, Berlin 2001.

7. Internationale Tagung Wirtschaftsinformatik (WI2005)
23.-25. Februar 2005 in Bamberg