Conference PaperPDF Available

The Evaluation Framework for Impact Assessment

Authors:

Abstract and Figures

This article introduces the Evaluation Framework EFI for the Impact Measurement of learning, education and training: The Evaluation Framework for Impact Measurement was developed for specifying the evaluation phase and its objectives and tasks within the IDEAL Reference Model for the introduction and optimization of quality development within learning, education and training. First, a description of the Evaluation Framework for Impact Measurement will be provided, followed by a brief overview of the IDEAL Reference Model. Finally, an example for the implementation of the Evaluation Framework for Impact Measurement within the ARISTOTELE project is presented.
Content may be subject to copyright.
The Evaluation Framework
for Impact Assessment
by Christian M. Stracke (2013)
Citation:
Stracke, C. M. (2013). The Evaluation Framework for Impact Assessment. In Proceedings of
6th International Conference of Education, Research and Innovations (ICERI) (pp. 4654-
4663).
[also online available at: http://www.opening-up.education]
Contact:
Dr. Christian M. Stracke
ICDE Chair in OER
Associate Professor for Open Education and Innovation
Open University of the Netherlands
Adjunct Professor, Korean National Open University
Advisory Professor, East China Normal University
http://www.ou.nl/web/welten-institute
Christian.Stracke@OU.NL
http://www.opening-up.education
http://www.learning-innovations.eu
http://www.ICORE-online.org
© Christian M. Stracke
This article is published under the Creative Commons licence "BY-NC-ND 4.0" (Attribution Non-Commercial No Derivate 4.0).
The full licence (legal code) can be read online here:
<http://creativecommons.org/licenses/by-nc-nd/4.0/legalcode>
You are free to share the work, i.e. to copy and redistribute the material in any medium or format, under the following conditions:
1. Attribution
2. NonCommercial
3. NoDerivates
THE EVALUATION FRAMEWORK FOR IMPACT ASSESSMENT
Christian M. Stracke
University of Duisburg-Essen (GERMANY)
Abstract
This article introduces the Evaluation Framework EFI for the Impact Measurement of learning,
education and training: The Evaluation Framework for Impact Measurement was developed for
specifying the evaluation phase and its objectives and tasks within the IDEAL Reference Model for the
introduction and optimization of quality development within learning, education and training. First, a
description of the Evaluation Framework for Impact Measurement will be provided, followed by a brief
overview of the IDEAL Reference Model. Finally, an example for the implementation of the Evaluation
Framework for Impact Measurement within the ARISTOTELE project is presented.
Keywords: Evaluation Framework EFI, Impact Measurement, IDEAL Reference Model, Learning,
Education and Training, Quality Development, ARISTOTELE.
1 INTRODUCTION
The Evaluation Framework for Impact Measurement (EFI) focuses the important purpose and demand
to assess the internal and external impact by learning, education, and training: Impact measuring is
becoming more and more crucial due to economic cost pressures and international competition. The
Evaluation Framework for Impact Measurement was developed for specifying the evaluation phase
and its objectives and tasks within the IDEAL Reference Model for the introduction and optimization of
quality development within learning, education and training. First, a description of the Evaluation
Framework for Impact Measurement will be provided by explaining the combination of the two targets
for impact measurement: the internal development and the external impact. Then, the IDEAL
Reference Model will be introduced to explain the broader picture and background for the
development of the Evaluation Framework for Impact Measurement. Afterwards the example for the
introduction and usage of the Evaluation Framework for Impact Measurement within the ARISTOTELE
project is described.
2 QUALITY DEVELOPMENT IN LEARNING, EDUCATION AND TRAINING
Evaluation is an important and necessary part for the holistic approach of Quality Development to
assure and improve the quality of learning, education and training: Quality development is a crucial
task for any learning, education and training (LET). A long-term debate on quality development
regarding the different quality issues, aspects and approaches has taken place (cf. Deming 1982;
Juran 1951 and 1992; and for an overview Stracke 2006a). This article focuses on the special support
that evaluation can provide and thus, we can only highlight the main characteristics of quality
development and its relevance in learning, education and training.
Quality development in its broad sense can be defined as follows (cf. Stracke 2006b):
Quality development covers every kind of strategy, analysis, design, realisation,
evaluation, and continuous improvement of the quality within given systems.
Quality development needs a long process to be established and integrated throughout a whole
organisation. Once started, it has to be a continuous on-going circle to be successful. Quality cannot
be described and fixed by a simple definition, because in itself quality is too abstract to have any
impact. Therefore, quality has to be defined and specified according to the given context and situation
considering the perspectives of stakeholders involved. It is important to identify the relevant aspects
and to specify the suitable criteria. And it is necessary to find a consensus among the different views
and perspectives to gain a common understanding of quality for the given context and situation due to
different and sometimes contradictory needs and definitions of quality by all stakeholders (Crosby
1980; Deming 1986; Donabedian 1980). Evaluation plays a key role for the quality development and
Proceedings of ICERI2013 Conference
18th-20th November 2013, Seville, Spain
ISBN: 978-84-616-3847-5
4654
the continuous improvement cycle, in particular in learning opportunities and processes as intangible
products and services.
3 THE EVALUATION FRAMEWORK FOR IMPACT MEASUREMENT EFI
The Evaluation Framework for Impact Measurement EFI was developed to close a gap for assessing
and optimizing the holistic quality development within learning, education and training. It combines the
traditional (internal) evaluation of the processes and developed products with the (external) evaluation
concerning the strategic objectives and impact that is becoming more and more crucial due to
economic cost pressures and international competition. Through this connection, the Evaluation
Framework for Impact Measurement EFI offers an adaptable model for the definition and specification
of indicators for both, the internal lifecycle and the external relations.
The Evaluation Framework for Impact Measurement EFI is combining the measurement of two
dimensions:
1. (Internal) Impact of Pilot Implementations and
2. (External) Impact of Outcomes.
Using the Evaluation Framework for Impact Measurement EFI, the following theoretical procedure has
to be applied in general:
First, the impact of pilot implementations will be measured by operative indicators. Within one given
project or process the operative indicators will be related to the planned products of the project or
process. The measurement of the operative indicators has to focus the two dimensions of the pilot
implementations: (1) the internal processes and activities (within the pilot implementation) and (2) the
(implemented internal) results (to be tested).
Second, the impact of outcomes will be measured by strategic indicators. They will be related to the
strategic objectives of a given project or process: The measurement of the strategic indicators has to
focus the two dimensions of the given project or process: (1) the external processes and activities
(within the whole organization and external relations) and (2) the (developed and improved) outcomes
and their impact.
The following figure presents the overview of the Evaluation Framework for Impact Measurement EFI
and demonstrates its relations between the two dimensions of impact measurement (internal impact of
pilots assessed by the operative indicators and external impact of outcomes assesses by the strategic
objectives):
Fig. 3-1: The Evaluation Framework for Impact Measurement EFI
4655
Through the combination to measure the internal impact of the internal processes and results by
operative indicators and the external impact of the external processes and impact by strategic
objectives, the Evaluation Framework for Impact Measurement offers a holistic approach for assessing
and optimizing the tasks of quality development.
Thus, the Evaluation Framework for Impact Measurement can be used to fulfil the third phase
("Evaluate") of the generic IDEAL Reference Model for quality development in learning, education and
training that will be introduced in the following chapter.
4 THE IDEAL REFERENCE MODEL
The IDEAL Reference Model is a generic approach for the introduction and improvement of quality
development in learning, education and training (cf. Stracke 2010a and 2010b): It can be used e. g. for
the implementation of the Reference Process Model of ISO/IEC 19796-1 to establish a sustainable
quality management including a continuous improvement cycle. Moreover the Evaluation Framework
for Impact Measurement EFI was developed and designed to cover one specific phase of the IDEAL
Reference Model: the third phase "Evaluate". Therefore a short overview of the IDEAL Reference
Model for quality development will be provided in the following.
To achieve a holistic quality development the needs and requirements of all stakeholders of the
current learning scenario have to be considered (cf. Feigenbaum 1986; Ishikawa 1985; Soin 1992).
This assumption is also valid for the adoption and introduction of the IDEAL Reference Model: A
systematic plan is needed for its adaptation to a specific organisation including all stakeholders (like
the Evaluation Framework for Impact Measurement EFI is providing for the evaluation phase).
The following figure shows the IDEAL Reference Model in an overview:
Fig. 4-1: The IDEAL Reference Model
The abbreviation IDEAL stands for the four main phases and in addition for the learner who should be
in centre of all activities and objectives for quality development.
The IDEAL Reference Model consists of four main phases to introduce and optimize quality
development (according to our definition above):
1. Initiate!
2. Do!
3. Evaluate! and
4656
4. Act!
The four phases and their tasks were developed according to the Deming cycle and the ISO standard
family ISO 9000ff. (cf. Stracke 2006a). They have to be fulfilled for the implementation of quality
development in LET: These phases and their tasks can be realized and applied according to the
specific needs and the given situation of the organization including feedback loops, individually
adapted sequences and parallel implementation.
The four phases and their tasks can be described and defined as follows:
1. Phase: Initiate!
First the raising of the awareness of all stakeholders and their full involvement and
participation is needed. In transparent procedures the vision for the quality development
should be defined based on a common and shared understanding of quality for learning,
education, and training and the innovations that should be achieved by the implementation of
quality development. The long-term policies and strategies will be discussed and agreed upon
the approved vision.
2. Phase: Do!
For the starting implementation the quality model has to be selected (here in our case: RFDQ)
and to be adapted as an application profile. Concerning RFDQ that means the selection of the
appropriate and relevant processes in a quality profile. Then the specific definitions and
success criteria has to be described, for RFDQ those are the attributes of the descriptive
model, mainly the methods, actors, metrics and criteria. The preparation of the implementation
will include all needed decisions and developments for the organizations. And finally the
realization of all planning for the quality development and its integration into organizational
processes is completing this task.
3. Phase: Evaluate!
The evaluation will focus on three distinctive objects: First, the realization of the
implementation of quality development itself as the main outcome of phase 2. Second, the
adaptation of the quality model selected and adapted at the beginning of phase 2. And third,
the evaluation of the initiation phase 1 including the revision of the vision, strategy and policies
for the quality development.
4. Phase: Act!
The fourth phase is dedicated to the sustainability and long-term impact of the adaptation and
implementation of the quality development. The communication and further discourse with all
stakeholders will guarantee the ongoing debate and consensus building on the quality
definitions and common understandings. Based on the evaluation results, the adaptation of
the quality model will be revised and the vision and strategy will be reviewed. Finally it should
lead the whole organization to the establishment of a continuous improvement cycle for the
quality development related to all phases and their tasks.
The following figure shows the overview of the four phases of the IDEAL Reference Model and their
tasks:
4657
Fig. 4-2: The Phases of the IDEAL Reference Model and their tasks
The IDEAL Reference Model can be used for the introduction and adaptation of process-oriented
quality management and quality development: It could be demonstrated by using the example of
introducing the ISO quality standard RFDQ (ISO/IEC 19796-1) that the IDEAL Reference Model is an
appropriate model for establishing a continuous improvement cycle based on the principles of ISO
9000ff. and the Deming cycle, in particular for the implementation of sustainable quality development
in learning, education and training by adapting RFDQ, the unique ISO quality standard for LET, to the
specific organization and given situation (cf. Stracke 2010a and 2010b).
5 THE USAGE OF EFI IN THE ARISTOTELE PROJECT
The Evaluation Framework for Impact Measurement EFI that has been selected and adapted for the
ARISTOTELE project1 by developing the ARISTOTELE Evaluation Framework that is combining the
measurement of two dimensions:
1. Impact Measurement of Pilot Implementations by outcome indicators and
2. Impact Measurement of Outcomes by usage indicators.
First, the impact of pilot implementations will be measured by so called "outcome indicators". Within
ARISTOTELE the outcome indicators were related to the planned products of the project and its
processes. The measurement of the outcome indicators has to focus the two dimensions of the pilot
implementations: (1) the activities (within the pilot implementation) and (2) the (implemented interim)
outcomes (to be tested).
Second, the impact of outcomes will be measured by so called "usage indicators". They were related
to the strategic objectives of ARISTOTELE: The measurement of usage indicators has to focus the
two dimensions of the project and its processes: (1) the activities (within the whole organization or
within specific processes) and (2) the (developed and improved) outcomes.
The following figure shows the adaptation of the Evaluation Framework for Impact Measurement EFI
to the ARISTOTELE project and demonstrates its relations between the two dimensions of impact
measurement (usage impact and product validation):
1 For more information about the ARISTOTELE project cf. www.aristotele-ip.eu.
4658
Fig. 5-1: The Adaptation of EFI for the ARISTOTELE project
In the following the structure and design of the ARISTOTELE Evaluation Framework is described in
brief providing a general overview how the Evaluation Framework for Impact Measurement EFI was
adapted for ARISTOTELE.
The ARISTOTELE Evaluation Framework combines four different evaluation categories and methods
to be used and assessed:
1. Conceptual Validation through Experts' Reviews & Focus Groups;
2. Usability Check of ARISTOTELE tools through Cognitive Walkthrough;
3. User Validation of the ARISTOTELE tools through Functional Testing & Quantitative Survey
(Online Questionnaire);
4. Software Validation and Impact Measurement - Usability of the Platform and Impact of whole
project through Functional testing (verification and validation), Qualitative Semi-Structured
Interviews & Quantitative Survey (Online Questionnaire).
The methods and tools applied in each of the evaluation levels have been chosen according to the
principles of "What is state of the art in evaluation?" as well as to efficiency and applicability
considerations ("What is possible to measure within the application partners in the given timeframe
and with the given resources?"). The mix of methods, the different aspects to be evaluated and their
order follow the principles of the IDEAL Reference Model and the EFI Evaluation Framework for
Impact Measurement.
For the evaluation of the satisfaction of the target groups with ARISTOTELE tools, models and
methodologies as well as for the evaluation of the integrated ARISTOTELE Platform and Tools and
their impact, the following Evaluation Levels, Pilot Trials and Iteration Cycles were distinguished and
defined within ARISTOTELE:
Evaluation Level 1: Conceptual Validation,
Evaluation Level 2: Usability Study,
Evaluation Level 3: User Validation of ARISTOTELE Tools,
Evaluation Level 4: Software Validation and Impact Measurement.
4659
For the evaluation of the satisfaction of the target groups with ARISTOTELE tools, models and
methodologies as well as for the evaluation of the ARISTOTELE platform and project impact, the
following evaluation methods are used within ARISTOTELE:
Level 1: a two phase process including individual experts' assessment of the concepts and a
SWOT Analysis based on the feedback by the internal and external experts (using SWOT
analysis),
Level 2: a usability study using a cognitive walkthrough approach for representatives of both
application partners,
Level 3: a standardized online survey and functional testing for tool users from the APs in the
1st evaluation iteration cycle,
Level 4: functional testing, a semi-structured interview on usability and on impact and a
standardized online survey for ARISTOTELE platform users by APs in the 2nd evaluation
iteration cycle.
The following table presents the overview of the ARISTOTELE Evaluation Phases, their Evaluation
Levels and Methodologies.
Tab. 5-1: Overview of ARISTOTELE Evaluation Phases, Levels and Methodologies
Phases
Level
Name
Methodologies
Conceptual
Evaluation
Level 1
Concept Validation
(qualitative evaluation)
Phase 1: Individual Experts Reviews
(internal and external experts)
Phase 2: SWOT Analysis from Focus
Group
Pilot
Trial 1
Level 2
Usability Study
(qualitative evaluation)
Cognitive Walkthrough
Level 3
User Validation of
Tools
(qualitative and
quantitative evaluation)
Functional Testing
Quantitative Survey (Online
Questionnaire)
Pilot
Trial 2
Level 4
Software Validation
and Impact
Measurement
(qualitative and
quantitative evaluation)
Phase 1: Software Validation
Consists of:
Functional Testing
(Validation and Verification)
Extended Oral Feedback on
Usability Aspects (if needed)
Phase 2: Impact Measurement
Consists of:
Semi-Structured Interviews on
Impact (Platform and Tools)
Quantitative Survey
(Online Questionnaire)
The first three evaluation categories and ARISTOTELE Evaluation Levels 1 - 3 were selected to cover
and ensure the broadest evaluation and validation of the ARISTOTELE tools and platform.
Within ARISTOTELE, not only the impact of the interim results during the Pilot Trials is evaluated, but
also the impact of the Integrated ARISTOTELE Platform as the final ARISTOTELE result (the
"outcome" according the Evaluation Framework EFI). Therefore the ARISTOTELE Evaluation Level 4
was defined for the impact measurement of Integrated ARISTOTELE platform through In-depth
Interviews and Online Survey.
4660
Through their careful combination the different Evaluation Levels and their methodologies and
instruments, ARISTOTELE is covering both categories of the Evaluation Framework for Impact
Measurement EFI: (1) the internal evaluation, that are the processes and outcomes for measuring the
product validation within ARISTOTELE (in the Evaluation Levels 1, 2, and 3) as well as (2) the
external evaluation that are the activities and outcomes for measuring the usage impact within
ARISTOTELE (in the Evaluation Level 4).
ARISTOTELE has selected a strong mixture of different methodologies within the four Evaluation
Levels to ensure rich and diverse evaluation perspectives and results: In particular the mix of
quantitative and qualitative methodologies was addressed and established to achieve both: baseline
data as well as quality feedback.
By using the Evaluation Framework for Impact Measurement EFI as the basis for the ARISTOTELE
Evaluation Framework, ARISTOTELE could assess and optimize the internal development processes
and results as well as the external impact. Thus, ARISTOTELE could prove that the introduction and
adaptation of the Evaluation Framework for Impact Measurement EFI is not only feasible but leading
to a valuable and sustainable evaluation and validation.
6 SUMMARY AND FUTURE PROSPECTS
This article has presented and analysed the Evaluation Framework for Impact Measurement EFI for
the combination of internal and external evaluation and for the realization of the third phase of the
IDEAL Reference Model for sustainable quality development as a valuable instrument for its
introduction and implementation.
Quality development is always depending on the given situation. The definition of quality development
and the benefits of quality standards were presented in particular relation to LET. The establishment of
a continuous improvement cycle is one main important benefit as the involvement of all stakeholders
is one main aim for the introduction of quality development. It has been shown that the Evaluation
Framework for Impact Measurement EFI is offering a valuable support for the adoption and
implementation of quality development. It is an appropriate means for assessing and optimizing the
quality and impact and in consequence, for supporting the adoption and implementation of a
sustainable quality development that is covering all learning and business processes. We have
pointed out the main tasks and potentials for its adaptation that is always needed for gaining a long-
term impact leading to overall quality development with continuous improvement.
For the facilitation and improvement of the broad application of quality development, the IDEAL
Reference Model was introduced and presented as an instrument for adopting and implementing
quality development by establishing a continuous improvement cycle.
Finally, the adaptation of the Evaluation Framework for Impact Measurement EFI within the
ARISTOTELE project for the ARISTOTELE Evaluation Framework was explained demonstrating its
feasibility and benefits for internal and external evaluation.
In summary, it can be stated that the Evaluation Framework for Impact Measurement EFI is a suitable
and valuable instrument for the introduction and implementation of internal and external evaluation to
assess and optimize the quality development and impact within learning, education and training. EFI is
a holistic evaluation approach facilitating the adaptation and implementation of the IDEAL Reference
Model as a helpful instrument for its objective as well as supporting sustainable quality development in
learning, education, and training in general.
REFERENCES
[1] ARISTOTELE (2012). The ARISTOTELE Evaluation Framework: Evaluation Methodologies and
Pilot Trials Definitions (D9.2).
[2] Berg, B. L. (1989). Qualitative Research Methods for the Social Sciences. Allyn and Bacon,
Boston.
[3] Chermack, T. J.; Kasshanna, B. K. (2007). The Use and Misuse of SWOT Analysis and
Implications for HRD Professionals. In: Human Resource Development International 10 (4),
383-399.
4661
[4] Chesbrough, H.W.; West, J.; Vanhaverbeke, W. (2006). Open Innovation: Researching a New
Paradigm. Oxford: Oxford University Press.
[5] Crosby, P. B. (1980). Quality is Free. The art of making quality certain. New York: McGraw-Hill.
[6] Deming, W. E. (1986). Out of the Crisis. Cambridge, MA: MIT.
[7] Deming, W. E. (1982). Quality, productivity and competitive position. Cambridge, MA: MIT.
[8] Donabedian, A. (1980). The Definition of Quality and Approaches to Its Assessment
[= Explorations in Quality Assessment and Monitoring, vol. 1]. Ann Arbor: Health Administration
Press, 1980.
[9] Drucker, P. F. (1999). Management Challenges of the 21st Century. Harper Business, New
York.
[10] Feigenbaum, A. (1986). Total Quality Control. Engineering and management. New York:
McGraw-Hill.
[11] Gassmann, O.; Enkel, E; Chesbrough, H.W. (2010). The future of open innovation. R&D
Management, 40(3), 213221.
[12] Ishikawa, K. (1985). What is Total Quality Control? The Japanese Way. Englewood Cliffs, NJ:
Prentice-Hall.
[13] ISO 9000:2005 (2005). Quality Management Systems. Fundamentals and Vocabulary. Geneva:
International Organisation of Standardization (ISO).
[14] ISO 9001:2008 (2008). Quality Management Systems. Requirements. Geneva: International
Organisation of Standardization (ISO).
[15] ISO/IEC 19796-1:2005 (2005). Information Technology - Learning, Education, and Training
Quality Management, Assurance and Metrics Part 1: General Approach. Geneva:
International Organisation for Standardization (ISO).
[16] Juran, J. M. (1992). Juran on quality by design. The new steps for planning quality into goods
and services. New York: Free Press.
[17] Juran, J. M. (Ed.) (1951). Quality Control Handbook. New York: McGraw-Hill.
[18] Patton, M. Q. (2002). Quantitative evaluation and research methods. Newbury Park: Sage
Publications.
[19] Singh, K. (2007). Quantitative Social research methods. New Delhi: Sage Publication.
[20] Soin, S. S. (1992). Total Quality Essentials. New York: McGraw-Hill.
[21] Stracke, C. M. (2013). "Open Learning: The Concept for Modernizing School Education and
Lifelong Learning through the Combination of Learning Innovations and Quality"; in: Stracke,
Christian M. (Ed.): Learning Innovations and Quality: The Future of Digital Resources, Berlin:
Logos. p. 15-28.
[22] Stracke, C. M. (2012). "Learning Innovations and Learning Quality: Relations,
Interdependences, and Future"; in: Stracke, Christian M. (Ed.): The Future of Learning
Innovations and Learning Quality. How do they fit together?, Berlin: Gito. p. 13-25.
[23] Stracke, C. M. (2010a). "Quality development and standards in learning, education, and
training: adaptation model and guidelines for implementations"; in: Информатизация
образования и науки [= Информике (Informika), ISSN 2073-7572]; Vol. 7 (3), 2010. Moscow
(Russian Federation): Informika. p. 136-146.
[24] Stracke, C. M. (2010b). "Quality and Standards in Learning, Education, and Training: The
Adaptation Model IDEA for the Introduction of Quality Development"; in: Proceedings of the
International Conference on the Past and Future of e-Learning Standards. Tokyo (Japan). p. 26-
36.
[25] Stracke, C. M. (2009). "Quality Development and Standards in e-Learning: Benefits and
Guidelines for Implementations"; in: Proceedings of the ASEM Lifelong Learning Conference: e-
Learning and Workplace Learning. Bangkok (Thailand) [online available at:
http://www.qualitydevelopment.eu/docs].
4662
[26] Stracke, C. M. (2006a). Process-oriented Quality Management. In Pawlowski J. & Ehlers, U.
(Eds.) European Handbook on Quality and Standardisation in E-Learning. Berlin: Springer, 79-
96.
[27] Stracke, C. M. (2006b). Interoperability and Quality Development in e-Learning. In Proceedings
of the Asia-Europe e-Learning Colloquy. Seoul: e-ASEM [online available at:
http://www.qualitydevelopment.eu/docs].
[28] Thompson, J. L.; Martin, F. (2005). Strategic management: Awareness and change. London:
Thomson learning.
4663
... For each of the six process categories and at each level that they are covering, we need appropriate services and instruments to support and improve the overall quality development in Open Education. Some instruments and tools are already developed and in practice such as the Quality Platform Learning (QPL, 2011), the Evaluation Framework for Impact Assessment (EFI, see Stracke, 2014bStracke, , 2013b. A general framework for the introduction of quality development also exists: The IDEA(L) framework (Stracke, 2010b), which consists of four phases "Initiate, Do, Evaluate, and Act" (adapted from PDCA cycle presented by Deming, 1982) and was also integrated into the international quality standard ISO/IEC 40180 (2017). ...
Article
Full-text available
This article discusses the need to innovate education due to global changes to keep its status as a human right and public good and introduces Open Education as a theory to fulfil these requirements. A systematic literature review confirms the hypothesis that a holistic quality framework for Open Education does not exist. For its development, a brief history and definition of Open Education are provided first. It is argued that Open Education improves learning quality through the facilitation of innovative learning designs and processes. Therefore, sources of learning quality and dimensions of quality development are discussed. To support the improvement of the learning quality and design of Open Education, the Reference Process Model of ISO/IEC 40180 (former ISO/IEC 19796-1) is introduced and modified for Open Education. Adapting the three quality dimensions and applying the macro, meso, and micro levels, the OpenEd Quality Framework is developed. This framework combines and integrates the different quality perspectives in a holistic approach that is mapping them to the learning design, processes, and results. Finally, this article illustrates potential adaptations and benefits of the OpenEd Quality Framework. The OpenEd Quality Framework can be used in combination with other tools to address the complexity of and to increase the quality and impact of Open Education. To summarize, the OpenEd Quality Framework serves to facilitate and foster future improvement of the learning design and quality of Open Education.
Conference Paper
Full-text available
While causing unprecedented disruption worldwide, COVID-19 has also stimulated the mainstreaming of digital technologies in the delivery of formal education. For most key stakeholders-organisations, educators, and students-this has been a new and challenging experience and has been described in policy terms as 'emergency remote education'. For many students, however, it has either exacerbated or marginalised their opportunity to access formal education. In probing this impact at a deeper level, an international collaboration involving the authors during 2020-2021 focused on reviewing contemporary practices and potentials of open education as a strategic and sustainable response. This paper highlights practices, case studies, and emerging issues from 13 diverse countries, to be globally representative, which include: Australia, and the United Kingdom. This collection of countries was selected based on researcher contexts and contributions. To date, findings indicate open education has demonstrable benefits for distance learning. More broadly, open educational practices are positioned to shape a 'new normal' that embraces 'global citizenship' while also being equitable and inclusive. Our aspirations are that such practices will lead to better formal education promoting and ensuring human rights, democracy, lifelong learning, safety, social justice, diversity, cultural sensitivity and inclusivity through strategic and long-term support by all stakeholders in both modes of educational delivery and access: face-to-face and distance learning.
Chapter
Full-text available
O presente capítulo discute a necessidade atual de mudanças educacionais devido a seu status de direito humano e bem comum, bem como introduz a Educação Aberta como um conceito que atende a tal demanda. Discute-se que a introdução da “abertura” (acessibilidade) promovida pela Educação Aberta tem melhorado seu elemento central, a qualidade da aprendizagem, ao facilitar e promover a inovação na aprendizagem e a mudança educacional. A análise de duas linhas concorrentes de inovação da aprendizagem e suas histórias leva à conclusão de que uma mescla é essencial para a qualidade da aprendizagem, que vai além de ambas as correntes por si só, o que é o mais importante para o sucesso da aprendizagem. Acesso Aberto, Recursos Educacionais Abertos (em inglês, Open Educational Resources – OER) e Cursos Online Abertos em Massa (em inglês, Massive Open Online Course – MOOCs) são conceitos proeminentes para Educação Aberta ao redor do mundo. Ao todo, nove dimensões de Abertura podem ser diferenciadas como base fundamental para a Educação Aberta e para o desenvolvimento de instrumentos e ferramentas a ela relacionados. Como exemplo de uma dessas dimensões (por exemplo, padrões abertos), as categorias de processo de normas de qualidade ISO/IEC 40180 (anterior ISO/IEC 19796-1) são modificadas para a Educação Aberta. Após esta definição, o breve panorama histórico da Educação Aberta apresenta seu progresso pelos últimos 4,000 anos. O objetivo deste artigo é analisar a Educação Aberta, de acordo com as três dimensões de qualidade adaptadas e aplicadas aos níveis macro, meso e micro. Essa análise tem levado ao desenvolvimento do quadro da Educação Aberta, integrando esses aspectos relevantes de qualidade, e usado como referência para categorias de processo modificadas. O Quadro da Educação Aberta pode servir como instrumento de guia para a complexidade da Educação Aberta e no desenvolvimento futuro de ferramentas relevantes. Finalmente, grandes movimentos e iniciativas em prática, bem como políticas de Educação Aberta e seus impactos são apresentados e discutidos em relação a seu impacto na pesquisa e na sociedade.
Article
Full-text available
This article discusses the need to change education due to global changes to keep its status as a human right and public good and introduces Open Education as a theory to fulfil these requirements. First, a brief history and definition of Open Education are provided. It is argued that Open Education is improving learning quality through the facilitation of innovative learning designs and processes. Hence, sources of learning quality and dimensions of quality development are discussed. Arguably, nine dimensions of openness can be distinguished and defined for the learning quality and design of Open Education and the development of supporting instruments, services and tools. As an example for one of the nine dimensions (i.e., open standards), the reference process model of ISO/IEC 40180 (former ISO/IEC 19796-1) is modified for Open Education. The article defines and analyses Open Education following the three adapted quality dimensions and applying the macro, meso and micro levels. As a result, the OpenEd Framework is developed which combines and integrates the different quality perspectives mapped to the learning design and processes. This article illustrates the potential adaptations and benefits of the OpenEd Framework. The OpenEd Framework can be used in combination with other tools such as the EFI and IDEA(L) frameworks to address the complexity and to increase the impact of Open Education. To summarize, the OpenEd Framework can facilitate and foster the future improvement of the learning quality and design of Open Education.
Conference Paper
Full-text available
In this paper, the author describes how learning analytics can be included within a general evaluation framework and be defined from the beginning of learning design for a learning opportunity. It is analyzed what part of a general evaluation framework will be covered by learning analytics and how learning analytics can contribute to the impact assessment using the generic Evaluation Framework for Impact Assessment (EFI) as an example. It will thereby be discussed how learning analytics can be addressed at the start of the design phase of the learning opportunity based on the reference process model from the international quality standard ISO/IEC 19796-1. Finally it is indicated how an extension of the learning design specification like PAS 1032-2 can be helpful for the introduction and support of learning analytics and summarized which further research is required.
Chapter
Full-text available
In this contribution process-oriented quality management, its emphasis and its relevance for educational organisations will be examined. Process-oriented quality management is a generic term for a lot of approaches with a variety of emphasis and range. Therefore, first the relevance of process orientation for a holistic quality management concept will be described. According to the requirements of a holistic quality management the four main quality management concepts (KAIZEN, BPR, Six Sigma, TQM) are analysed by the Integrative Management model. Total Quality Management (TQM) as the most comprehensive holistic approach will be explained more explicitly. The general philosophy and the multitude of facets of TQM will be described. In introducing and realizing the TQM ideas, quality standards and reference models can offer a decisive support. The standards family ISO 9000:2000ff. and the EFQM Excellence Model will be explained as the two most important instruments of introducing a comprehensive quality management system and the TQM idea. Finally, the introduction and the use of the quality management systems in educational organisations will be outlined.
Conference Paper
Full-text available
This article focuses on quality development in Learning, Education, and Training (LET) and on quality standards as appropriate instruments for corresponding activities. Quality standards are supporting the adoption and implementation of quality development throughout the whole organisation. Based on the introduced definition of quality development, the benefits of quality standards are presented. Especially the first ISO quality standard for learning, education, and training "RFDQ" (ISO/IEC 19796-1) offers a promising potential for the raise of quality awareness and the involvement of all stakeholders. The presented adaptation model IDEA for the implementation of the quality standard is the main focus looking at the specific given situation and organisation.
Conference Paper
Full-text available
Learning innovation and learning quality are very often addressed separately and solely. But in fact they are interdependent and have to be reflected both for achieving the best learning quality: This article discusses how to achieve the best appropriate learning quality as the core objective in learning, education and training by combining the three dimensions learning history, learning innovations and learning standards. Only their mix can ensure to meet the learners' needs and to provide the best and appropriate learning opportunities and learning quality: The Open Learning Concept is presented as combination of suitable open learning styles and open learning scenarios and adapted for school education and lifelong learning in the world of work. Open Learning aims at the right balance between learning innovation and quality for modernizing school education and lifelong learning fitting to the given situation and for a long-term and sustainable improvement across all sectors in learning, education and training, all communities, educational and training systems and societies in Europe and worldwide.
Book
https://www.cengage.co.uk/books/9781473726338/ Strategic Management: Awareness and Change provides a comprehensive introduction to modern strategy and is essential reading for those who will become future managers as well as managers in practice. Following the ‘analysis, choice, implementation’ model the book moves from examining the strategy process as a whole, through the three approaches to strategy, strategy formulation and growth and retrenchment issues, finally concluding with an exploration of strategy implementation.