![Example of QRF Process Model description from ISO [20]](https://www.researchgate.net/profile/Khitam-Shraim/publication/344458244/figure/fig1/AS:943348617576449@1601923245412/Example-of-QRF-Process-Model-description-from-ISO-20_Q320.jpg)
![Key articles on QOE standards -Modified from Esfijani [9]](https://www.researchgate.net/profile/Khitam-Shraim/publication/344458244/figure/tbl1/AS:943348617580545@1601923245463/Key-articles-on-QOE-standards-Modified-from-Esfijani-9_Q320.jpg)
Content uploaded by Khitam Yousuf Shraim
Author content
All content in this area was uploaded by Khitam Yousuf Shraim on Oct 06, 2020
Content may be subject to copyright.
Content uploaded by Khitam Yousuf Shraim
Author content
All content in this area was uploaded by Khitam Yousuf Shraim on Oct 05, 2020
Content may be subject to copyright.
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
Quality Standards in Online Education
The ISO/IEC 40180 Framework
https://doi.org/10.3991/ijet.v15i19.15065
Khitam Shraim
Palestine Technical University-Kadoorie, Kadoori, Palestine
k.shraim@ptuk.edu.ps
Abstract—As higher education institutions (HEIs) make growing use of
online education, enhancing and ensuring quality in online education (QOE)
have become increasingly important for their competitiveness. Researchers and
both national and international organizations have developed a variety of mod-
els, frameworks and guidelines for QOE. However, selecting from these a holis-
tic quality framework that meets the needs and requirements of HEIs is chal-
lenging. This study reviews current QOE frameworks, guidelines and bench-
marks used in diverse contexts, with reference to an analysis of 72 publications
between 2000 and 2019, then introduces the ISO/IEC 40180 framework for
quality assurance, quality management and quality improvement in IT-
enhanced learning, education and training. The findings show that while no ho-
listic quality framework for open education exists currently, ISO/IEC 40180 is a
flexible and adaptable framework for revolutionary organizational change,
meeting the needs of multiple stakeholders of educational organizations at the
macro, meso and micro levels. Therefore, HEIs seeking to foster growth, com-
petitiveness and international recognition are advised to consider adopting the
ISO/IEC 40180 framework, which should be integrated into national quality
education standards.
Keywords—Online education, Quality standards, Quality framework, ISO/IEC
40180, Higher education.
1 Introduction
The widespread adoption of online education makes it increasingly important for
higher education institutions (HEIs) to enhance and ensure quality in online education
(QOE) to maintain a competitive advantage. Delimiting QOE is fundamental to the
systematic monitoring of quality improvement and effective higher education reform
[1]. A European survey by Ehlers et al.[2], to analyze quality in e-learning in general
and participants’ experience of using quality instruments in e-learning, found that
quality plays a key role in the success of e-learning, that educational organizations
should treat quality development as a core process and that open quality standards
should be implemented widely. To this end, HEIs must implement certain quality
standards to ensure sustainable quality in education [3]. Researchers and organiza-
22
http://www.i-jet.org
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
tions, national and international, have developed various models, frameworks,
benchmarks and guidelines to enhance and assure QOE [4]. The multiplicity of these
approaches to QOE and their different scopes and objectives can cause confusion [5],
so a key challenge to attaining quality in practice is selecting the most appropriate one
to meet each HEI’s requirements [6]. According to Stracke [7], there presently exists
no holistic quality framework for open education adopting a total quality management
philosophy and addressing all educational levels (micro, meso and macro). Thus,
there is a need to derive a comprehensive QOE framework by reviewing existing
models and frameworks with a view to adapting international standards to the local
needs of HEIs [5,7,8].
Pawlowski [6] and Stracke [7] adapted and successfully implemented ISO/IEC
19796-1 and ISO/IEC 40180 respectively, concluding that it was important to take
advantage of ISO opportunities. The purpose of the present research was to review
several QOE frameworks, guidelines and benchmarks currently used in HEIs, then to
determine the value of ISO/IEC 40180 (formerly ISO/IEC 19796-1) for quality assur-
ance, quality management and quality improvement in IT-enhanced learning, educa-
tion and training, comparing it with other frameworks.
2 Literature Review
Defining quality in online education is increasingly challenging. There is broad
consensus in the literature that QOE is a complex and difficult concept which depends
on a number of factors related to students, the curriculum, educational design and the
technological means used, and to other organizational, planning and contextual factors
[1, 4, 5, 6, 7, 9]. There is no common understanding of the terminology or methodol-
ogy of quality, because it can be seen from a variety of perspectives and dimensions
[6, 7, 9]. QOE has various meanings for multiple stakeholders (learners, academics,
leaders, employers, and society) [9]. The variety of methods used to measure it in-
cludes commercial instruments, government and national standards and individual
frameworks, which all identify different quality criteria. Among the QOE indicators
considered are quality benchmarks, accreditation, measurement and standardization,
all of which can be evaluated at three levels- micro ( learning experiences at learner
level ); meso (individual courses at national level) and macro (online programs at
institutional or national or international level) [6, 7, 9]. Since QOE and quality stand-
ards for conventional education are not identical, it is imperative to integrate e-
Learning criteria into national quality assurance systems [10], which entail harmoniz-
ing stakeholders’ differing views of quality.
Researchers have proposed a variety of models, frameworks and guidelines for
QOE, including Khan’s E-Learning framework, Frydenberg’s e-Learning quality
standards and the e-Quality framework of Masoumi and Lindström, while those de-
veloped by organizations investigating the overall quality of online and e-learning in
diverse contexts include the Swedish E-Learning Quality model, the University of
Pennsylvania quality course design standards, the British Open University e-Learning
approach, the Norwegian Association for Distance Education model, the New Zealand
iJET ‒ Vol. 15, No. 19, 2020
23
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
e-learning Maturity Model (eMM), the E-xcellence quality benchmarking instrument,
the Online Learning Consortium (OLC) quality scorecard and the Hamdan Bin Mo-
hammad Smart University model. A summary of some of the frameworks is presented
in Table 2 and Table 3.
Ossiannilsson et al. [11] offer detailed descriptions of several quality models for
online and open education, arguing that they all suffer deficiencies such as restricted
applicability, failure to clarify which maturity levels they are best for, widely diver-
gent quality of reviews and of information provided, and poor response to change.
Having reviewed several QOE models, Esfijani [9] asserts that they remain fragment-
ed and lack coherence, focusing mainly on resources, input and processes, and reports
that there is no evidence for an output / outcome-oriented approach to identifying and
measuring quality factors. He also notes that the same quality framework or bench-
mark has often been used in different educational cultures without any modifications,
concluding that there needs to be a holistic approach which considers diverse aspects
of quality factors including inputs, resources, processes, outputs and outcomes. In the
same vein, Stracke [7] states that there is currently no holistic quality framework for
open education that follows the total quality management philosophy, with continuous
improvement cycles, applied at the micro, meso and macro levels. Furthermore, Farid
et al. [12] observe that existing QOL models have been designed in developed coun-
tries, where online education does not face the same problems as in developing coun-
tries. Accordingly, applying these models to other cultural contexts is questionable
[4].
In a globalized world, Ossiannilsson [5] emphasizes that any quality model for
online education needs to be flexible enough to embrace and empower the rapid
changes that institutions undergo, responsive to local context and globally recognized.
She recommends adopting international standards and incorporating their principles to
replace a mechanistic, tick-box understanding of quality assurance with a greater
emphasis on learning, engagement, analysis and outcomes. Similarly, Esfijani [9]
suggests that HEIs should ensure QOE by adopting a universal quality framework or
international standards, while responding to advanced technologies and techniques
within the requirements of their particular contexts. A holistic framework for QOE
would be beneficial for all open education, but it should be customized to each institu-
tion’s context specific. According to Ozbek [13], as exchange programs and collabo-
ration among universities increases rapidly, mutual compatibility grows in im-
portance, with many HEIs seeking internationally accepted quality standards in re-
sponse. By adopting international standards such as ISO, higher education institutions
can guarantee the professionalism of the diverse workforce in dealing with diverse
learners and stakeholders [13]. Several studies reveal that adopting ISO series stand-
ards helps HEIs, especially in developing countries, to advance their standards for
attaining the international recognition [14,15,16].
24
http://www.i-jet.org
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
3 The International Organization for Standardization
The International Organization for Standardization (ISO) was established in 1946
in Geneva, Switzerland. ISO has developed over 23117 quality international standards
for all types of organizations (iso.org). The ISO and the International Electrotechnical
Commission (IEC) have worked intensively on e-learning standardization since 2004
[10]. The main standards in the ISO series for education are shown in the table 1 be-
low.
Table 1. Descriptions of ISO series for education
Many universities worldwide use ISO series quality standards [13] and ISO/IEC
40180 is widely recognized as a framework for open education, because it can be
adapted to the needs of HEIs anywhere [7]. However, there are limited studies provid-
ing empirical evidence for its benefits for online education. One recent study
(Stracke[7] introduces the OpenEd Quality Framework, a modification for open edu-
iJET ‒ Vol. 15, No. 19, 2020
25
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
cation of the Reference Process Model of ISO/IEC 40180 which integrates the three
quality dimensions (Learning objectives, Learning realization and Learning achieve-
ments) and applies them at the macro, meso and micro levels. Stracke argues that this
framework combines the different quality perspectives in a holistic approach by map-
ping them to the learning design, processes and results, and that it can be combined
with certain other quality frameworks such as the Quality Reference Framework
(QRF) and the IDEA(L) (Initiate, Do, Evaluate, Act) framework. The OpenEd Quality
Framework can also be applied to MOOCs (massive open online courses) and (OER)
open education resources.
In an earlier study, Pawlowski [6] adapted ISO/IEC 19796-1 to develop the quality
adaptation model, identifying four main steps as necessary to implement it successful-
ly in response to the needs of stakeholders at the macro, meso and micro levels: con-
text setting, model adaptation, model implementation/adoption and quality develop-
ment. Each step should be performed with a broad range of interested parties to raise
awareness and reach consensus. Pawlowski recommends considering the cultural
factors of the different countries when adapting the model.
According to Ozbek [13], ISO standards are global and scalable, with inherent
flexibility that fosters creativity and efficiency for any HEI which adopts them. ISO
series provide generic standards which support common understanding and consistent
practices (Kezar & Eckel, 2002). To ensure that such generic standards can be applied
in diverse contexts, whatever the modes of teaching and learning, their operational
principles must be contextually sensitive [10].
4 Methodology
The main aims of this research are to review the QOE frameworks, guidelines and
benchmarks currently used in HEIs, to examine the applicability of the ISO/IEC
40180 framework for quality assurance, quality management and quality improve-
ment in IT-enhanced learning, education and training, and to compare it with other
frameworks.
4.1 Research questions:
1. What standards of QOE are identified in the literature?
2. What are the characteristics of the ISO/IEC 40180 framework relevant to the quali-
ty standards of online education in the context of HEIs?
These questions were addressed by conducting an extensive literature review of 72
studies examining quality of online education in higher education published between
2000 and 2019 in indexed and peer-reviewed journals, government reports, web pages
and books, to identify and review the available frameworks, models, guidelines,
benchmarks, etc. related to QOE in online education in HEI. The following key words
were used for this research: online learning quality, e-learning quality, online educa-
26
http://www.i-jet.org
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
tion quality, e-learning quality standards, quality of virtual learning, quality of tech-
nology-enhanced education and e-learning course design standards.
The selected publications were coded according to:
1. Stakeholders’ perspectives (learners, academics, librarians, administrators, techni-
cians, leaders, employers, administrators).
2. Level (macro, micro, meso).
3. Quality terms (standards, benchmark, framework, criteria).
4. Source (researchers, organizations).
5 Findings and Discussion
The reviewed literature reveals that many quality standard models have been de-
veloped for specific purposes, in different contexts. Table 1 gives examples of the
well-know QOE standards based on the literature, while Table 2 shows several studies
that contributed to QOE standards from different perspectives and for different online
education levels.
Table 2. Descriptions of several well-known QOE frameworks
iJET ‒ Vol. 15, No. 19, 2020
27
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
Table 3. Key articles on QOE standards - Modified from Esfijani [9]
Author
Stakeholders
Standards
Level
(Khan [27])
Educators
Pedagogical dimension
Technological dimension
Interface design dimension
Evaluation dimension
Management dimension
Resource support dimension
Ethical dimension
Institutional dimension
Macro
(Frydenber [28])
Educators
Executive commitment.
Technological infrastructure.
Student services.
Design and development.
Instruction and instructor services.
Program delivery.
Financial health.
Legal and regulatory requirements.
Program Evaluation
Macro
(McGorry [29])
Students
Flexibility
Responsiveness
Interaction
Student learning
Technical support
Technology
Student satisfaction
Micro
(Walker & Fraser, [30])
Students
Instructor support
Student interaction and collaboration
Personal relevance
Authentic learning
Active learning
Student autonomy
Micro
(Young & Norgard, [31])
Students
Course design
Course interactions
Course content
Course support
Course delivery
Meso
(Shelton [32])
Admins
Institutional support
Technology support
Course development & design
Course structure
Teaching and learning
Social and student &engagement
Faculty support
Student support
Evaluation and assessment
Meso
(Gordin & Hall [33])
Faculties
Learner support
Online organization and design
Instructional design and delivery
Assessment and evaluation
Innovative teaching with technology
Use of student feedback
Meso
(Agariya & Singh [34])
Students
Faculties
Course content
Design structure
Collaboration
Industry acceptance
Meso
28
http://www.i-jet.org
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
Author
Stakeholders
Standards
Level
Value addition
Transparency in assessment
Technical know-how
Engagement
(Masoumi &
Lindstrom [4])
Students
Faculties Admin
Institutional factor
Technological factor
Instructional design factor
Pedagogical factor
Faculty support
Student support
Evaluation factor
Meso
(Ashlaghi et al., 2013)
Students
Faculties
Admin
Institutional support
Teaching & learning
Course development
Course structure
Faculty support
Student support
Evaluation
Meso
The University of
Pennsylvania
e-learning model
Navigation,
Student Orientation,
Curriculum, Communication and Availa-
bility, Course Resources Requirements,
Technical Support,
Accessibility Requirements,
Learning Objectives,
Learning and Assessment Activities,
Copyright Requirements,
Functionality and Reports for course
improvement.
Meso
The Swedish e-Learning
Quality (eLQ) model. (eLQ
[35])
Students
Educators
Material/content,
Structure/virtual environment, Commu-
nication,
Cooperation and interactivity,
Student assessment, flexibility and
adaptability,
Support (student and staff),
staff qualifications and experience,
Vision and institutional leadership,
Resource allocation.
Meso
Hamdan Bin Mohammad
Smart University (MeLQ )
model
(OU-UK [36])
Strategic Dimension
Learning and Teaching Dimension
Organizational Dimension
Change Dimension
Economical Dimension
Technological Dimension
Macro
The reviewed literature reveals that most of these standards address online course
design and online program. Several studies conclude that it is essential to engage all
actors involved in distance/online education: Developers, administrators, govern-
ments, providers, teachers and learners [7,12]. The review also reveals a general focus
on input criteria, such as facilities and support for faculty or students, rather than on
outputs and outcomes like student learning and employment, as shown in Table 1.
This finding is consistent with those of Esfijani [8] and Ransom et al. [9]. The aspects
iJET ‒ Vol. 15, No. 19, 2020
29
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
of a quality experience in the online learning environment most commonly addressed
in the literature are:
1. Institutional support (vision, planning, and infrastructure)
2. Course design and development
3. Teaching and learning (instruction)
4. Student and faculty support
5. Technology support
6. Assessment
7. Security
However, there are significant differences in quality terminology and in the aggre-
gation of the quality criteria according to these aspects, making it difficult to compare
the various frameworks and models. This finding is similar to one reported by Butcher
and Wilson-Strydom [37].
Analysis of the literature indicates that none of these quality standards is compre-
hensive, each having been developed for a specific purpose in a given context. Indeed,
the quality criteria and standards reviewed were mostly developed in the West for use
there. This result is consistent with the assertion of Stracke [7], Esfijani [9] and Faried
et al. [12] that there is no holistic standard for open education that addresses all stake-
holders' needs at all three levels. Masoumi and Lindstrom [4] justifiably conclude that
these approaches can be applied only cautiously in other contexts. It is also noted that
most of the quality criteria and standards are based on theoretical findings and have
yet to be translated into practical principles or tested in different contexts. Other
shortcomings are that most of the frameworks and models are fairly broad and lack
details, and that neither validation nor guidelines are provided for their utilization.
The analysis reveals that none of these quality standards specifies how to respond
to changes in stakeholders’ interests or in HEIs’ internal and external environments;
nor do they take into account technological evolution or the different types of digital
media and resources that could be integrated into learning, despite the need for quality
standards to be continually adaptable and scalable to any such changes [5,13]. In par-
ticular, the globalization and international competitiveness of universities makes har-
mony with international standards a key requirement for any national quality stand-
ards in education. So, what is needed is the development of global standards and qual-
ity assurance frameworks that can improve pedagogy in diverse cultural, knowledge
and delivery platforms in the parallel worlds of reality and virtual reality [38].
5.1 ISO/IEC 40180
ISO/IEC 40180 is an international standard which "provides the fundamentals and
the reference framework for quality assurance, quality management and quality im-
provement in IT-enhanced learning, education and training (e-learning)”. Its principal
element is the quality reference model (QRF), "a common and generic framework to
describe, specify and understand critical properties, characteristics and metrics of
quality", consisting of three parts ISO [20]:
30
http://www.i-jet.org
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
1. The Process Model is a guide to the relevant processes for developing learning
scenarios within the whole life cycle of online education. Its seven process catego-
ries (Needs Analysis, Framework Analysis, Concept/Design, Develop-
ment/Production, Implementation, Learning Process/Realization and Evalua-
tion/Optimization) and 38 sub-processes are shown in Figure 1.
2. The Description Model is a scheme to describe and document quality approaches
in a transparency way such as guidelines, design guides and requirements. It pro-
vides processes to develop online education scenarios by specifying quality objec-
tives, methods to ensure quality, the actors involved, and relations with other pro-
cesses, evaluation methods, standards and references. Table 4 outlines an example.
3. A comprehensive list of context-specific Reference Quality Criteria (RQC) is pro-
vided for the evaluation of the quality of learning products based on the Process
Model. The list includes media- and learning psychology-related criteria, as well as
those related to data security and to national legislation.
Fig. 1. Example of QRF Process Model description from ISO [20]
iJET ‒ Vol. 15, No. 19, 2020
31
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
Table 4. Overview of the Process Model
ID
Category
Process
Description
Relation
CD.7
Conception/Design
Concept of media and
interaction design
Definition of media and interac-
tion design
NA.3; NA.4;
CD.1; CD.2;
CD.4; CD.6
Sub-processes/Sub-aspects
Media design
Interaction design
Objective
Representation of the design concept concerning all relevant fields in
consideration of existing templates/guidelines
Method
Development of screen and human-computer interaction design based on
specifications of software ergonomics, usability and corporate design
Result
Documentation of design principles (design concept, style guide)
Design prototype
Actor
Design experts, Experts media didactics
Metrics/Criteria
Usability test on the basis of the design prototype
Heuristic evaluation
Categories 2, 6, 8 of RQC
Standards
ISO 9241, ISO 13407, W3C-Accessibility-Guidelines
The main characteristics of ISO/IEC 40180 are as follows:
Harmonization: The QRF for e-learning serves to compare diverse standards and
to harmonize these towards a common quality model. The description model provides
considerable information and guidance to develop a harmonized scheme to describe
quality approaches. It provides sector-specific information useful in integrating man-
datory and facultative quality approaches at the organizational, local, regional, nation-
al and international levels. Therefore, HEIs can implement ISO/IEC 40180 in harmo-
ny with their institutional vision, mission, values, goals and objectives.
Flexibility/adaptability: The process model provides a general framework that
can be extended and adapted to the specific situation, organization, target group and
requirements. The potential advantage for an HEI is that in practice, quality standard
processes have to be selected and adapted to align with stakeholders’ needs and with
the organization’s mission, vision, objectives and action plans. The flexibility of the
QRF supports the development of quality profiles for organizations, meaning that the
generic standard is tailored to the HEI’s requirements [6]. Flexible and adaptable
instruments are fundamental to QOE, to accommodate rapid changes in technology
and both internal and external environments [7]. Thus, Stracke [7] proposes modifica-
tions to reflect the fact that two of the process categories in Figure1 may be performed
together, or that evaluation and optimization, for example, could be done separately.
In addition, we want to highlight the importance of optimization and the involvement
of learners in the crucial process of continuous quality development.
Contextualization: Importantly, the process model begins by analyzing the organ-
ization and its context, external and internal, as reflected in sub-processes FA1 and
FA4 (Figure 1). The external analysis should consider legal, political and economic
circumstances, social expectations, technological evolution, and international and
local competition, while analysis of the institutional and organizational context focus-
es on strategic items such as vision and mission, resources, campus facilities and
32
http://www.i-jet.org
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
stakeholders’ interests. One size does not fit all in education and an HEI adopting ISO
40108 can respond promptly to its changing context.
Stakeholders: Achieving QOE means evolving to suit all stakeholders, whose re-
quirements for quality standards differ greatly [12, 37]. By addressing harmonization
throughout the development process, beginning with stakeholder identification as an
element of needs analysis, ISO/IEC 40180 considers the roles, interests, expectations
and requirements of all key partners interested in improving QOE at all levels: learn-
ers, academics, trainers, tutors, designers, administrators, enterprises, organizations,
examination boards, regulators, universities, sponsors, cooperating institutions, cli-
ents, relevant social groups, national and international bodies. The HEIs must seek
consensus about quality among its stakeholders; in practice, this entails meeting their
diverse and changing needs and expectations, which the ISO/IEC 40180 enables ho-
listically [7].
Process orientation: The QRF is not prescriptive or systematic, specifying a se-
quence of processes or an outcome, but general, descriptive and process-oriented,
making no assumptions about required quality approaches. The process model is ge-
neric, covering all phases of online education and serving as a guideline to develop
quality concepts from conception to evaluation and optimization, while the descrip-
tion model allows all kinds of processes to be modeled and documented in a transpar-
ency manner. The ISO/IEC 40180 provides set of best practice examples that help to
understand and manage the interrelated processes of inputs and outputs and outcome
that operate as a coherent system.
Evaluation and optimization: ISO/IEC 40180 provide a comprehensive list of
reference criteria to be used in analyzing and evaluating online education for different
purposes. Only criteria which are suitable for a certain context should be used. Use of
the RQC makes the evaluation process more transparent and comparable, because
they relate to a standardized set of criteria.
Compatibility: ISO/IEC 40180 is compatible with and complementary to ISO
9000, ISO 9001 and other series including ISO 14001 and ISO 21001. Further, the
QRF can be used as a meta-model for online education incorporating other approach-
es such as Plan, Do, Check, Act. According to Pawlowski [6], a variety of existing
approaches can be used for different objectives and purposes, and the QRF provides
clear terminology and description formats to assemble specific quality concepts from
these. The common terminology of ISO/IEC 40180 facilitates such recombination; for
example, (AACSB) Association to Advance Collegiate School of Business or (ABET)
Accreditation Board for Engineering Technology quality standards can be combined
and integrated with international standards for application in specific cases. ISO/IEC
40180 ensures that the processes of university teaching and thus outcomes meet local
and international standards.
6 Conclusion
Quality in online education is critical for HEIs’ competitively. Despite the many
frameworks, models and tools proposed for QOE, no holistic quality framework for
iJET ‒ Vol. 15, No. 19, 2020
33
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
open education currently exists, but ISO/IEC 40180 is an international and scalable
standard adaptable to this context, since characteristics including harmonization, flex-
ibility, contextualization, process orientation and compatibility make it a holistic
framework for use which meet all stakeholders at all educational levels. Implementa-
tion of standards is particularly difficult in HEI settings and success depends on ana-
lyzing context and adaptation; therefore, a practical guide is needed.
With the emergence of assistive technologies such as artificial intelligence and ma-
chine learning, the process of analyzing the context of any HEIs at the macro, inter-
mediate and micro levels is more efficient and consistent. The adoption of ISO/IEC
40180 will help a university to be internationally recognized. The responsiveness of
ISO standards to rapid changes ensures future improvements.
7 Acknowledgements
The author would like to thank the Palestinian Higher Education and the Palestine
Technical University- Kadoorie for funding this research.
8 References
[1] Vlachopoulos, D. (2016). Assuring quality in online course design: The roadmap. The In-
ternational Review of Research in Open and Distance Learning ,17(6), 183-205. https://
doi.org/10.19173/irrodl.v17i6.2784
[2] Ehlers, U. D., Goetz, L., Hildebrandt, B., & Pawlowski, J. M. (2005). Quality in e-
learning. Use and dissemination of quality approaches in European e-learning. A study by
the European Quality Observatory. Luxembourg: Office for Official Publications of the
European Communities.
[3] Cömert N., Akgun I., Karaköy D. & Kaya P. (2019). Quality Management Approaches and
Their Comparison in Higher Education: Marmara University Faculty of Business Admin-
istration Quality Studies. The Online Journal of Quality in Higher Education, 6(3), 101-
111.
[4] Masoumi, D. and Lindström, B. (2012). Quality in e-learning: a framework for promoting
and assuring quality in virtual institutions. Journal of Computer Assisted Learning, 28: 27–
41. https://doi.org/10.1111/j.1365-2729.2011.00440.x.
[5] Ossiannilsson, E. (2018). Quality considerations for blended learning. Conceptual report
for the Technical Innovation in Blended Learning (TIBL), a KA202 - Strategic Partner-
ships for vocational education and training project, 2017-1-ES01-KA202-038256. https://
doi.org/10.21125/iceri.2018.1577
[6] Pawlowski, J. M. (2007). The Quality Adaptation Model: Adaptation and Adoption of the
Quality Standard ISO/IEC 19796-1 for Learning, Education, and Training. Educational
Technology & Society, 10 (2), 3-16.
[7] Stracke, C. M. (2019). Quality Frameworks and Learning Design for Open Education. The
International Review of Research in Open and Distributed Learning, 20(2), 180-203.
https://doi.org/10.19173/irrodl.v20i2.4213
[8] Ransom, T., E. Knepler & C. Zapata-Gietl (2018). New Approaches to Judging Quality in
Higher Education Profiles of Emerging Methods Apart from Traditional Accreditation.
(CHEA/CIOG) Publication Series. Council for Higher Education Accreditation Retrieved
from https://www.chea.org/new-approaches-judging-quality-higher-education https://doi.
org/10.1080/02602938.2014.960361
34
http://www.i-jet.org
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
[9] Esfijani, A. (2018) Measuring Quality in Online Education: A Meta-synthesis. American
Journal of Distance Education, 32(1), 57-73. https://doi.org/10.1080/08923647.2018.14176
58
[10] Grifoll J., Huertas E., Prades A., Rodríguez S., Rubin Y., Mulder F., Ossiannilsson E. 7.
(2010). Quality assurance of e-learning. ENQA Workshop report (14). http://www.enqa.
eu/indirme/papers-and-reports/workshop-and-seminar/ENQA_wr_14.pdf
[11] Ossiannilsson, E., Williams, K., Camilleri, A. F. & Brown, M. (2015). Quality models in
online and open education around the globe: State of the art and recommendations. Oslo:
International Council for Open and Distance Education-ICDE. https://www.pedocs.de/voll
texte/2015/10879/pdf/Ossiannilsson_et_al_2015_Qualitymodels.pdf
[12] Farid, S., Ahmad, R., Alam, M., Akbar, A. & Chang, V. (2018). A sustainable quality as-
sessment model for the information delivery in E-learning systems. Information Discovery
and Delivery, 46(1), 1-25. https://doi.org/10.1108/idd-11-2016-0047
[13] Ozbek, O. (2019). International Quality Standards Application to Administrative Processes
of Higher Education. In: Sony M., Karingada K.T., Baporikar N. (eds) Quality Manage-
ment Implementation in Higher Education: Practices, Models and Case Studies. IGI Glob-
al, USA 1-23. https://doi.org/10.4018/978-1-5225-9829-9.ch001
[14] Kagumba, A.M.& George, G.E. (2013). Quality Assurance Strategy on Organizational
Performance: Case of Kenyatta University. European Journal of Business and Manage-
ment, 5(2), 265-270
[15] El Abbadi, L., Bouayad, A. and Lamrini, M. (2014). ISO 9001 and the Field of Higher Ed-
ucation: Proposal for an Update of the IWA 2 Guidelines. Quality Approaches in Higher
Education, 4(2), 14-19
[16] Sangher, K.S., Noor, A., Kalyani, L., & Sharma, S.K. (2017). ISO 9001:2015 implementa-
tion in the e-learning based virtual teaching program. 2017 5th National Conference on E-
Learning & E-Learning Technologies (ELELTECH), 1-3. https://doi.org/10.1109/eleltech.
2017.8074994
[17] ISO (2005). ISO/IEC 19796-1:2005: Information technology - Learning, education and
training – Quality management, assurance and metrics - Part 1: General approach. Geneva:
ISO. https://www.iso.org/obp/ui/#iso:std:iso-iec:19796:-1:ed-1:v1:en. https://doi.org/10.34
03/30383754u
[18] ISO (2009) ISO/IEC 19796-3:2009: Information technology - Learning, education and
training – Quality management, assurance and metrics - Part 3: Reference methods and
metrics. . Geneva: ISO. https://www.iso.org/obp/ui/#iso:std:iso-iec:19796:-3:ed-1:v1:en.
https://doi.org/10.3403/30383754u
[19] ISO (2017a) ISO/IEC 40180:2017: Information technology - Quality for learning, educa-
tion and training -Fundamentals and reference framework. Geneva: ISO. https://www.iso.
org/obp/ui/%20#iso:std:iso-iec:40180:ed-1:v1:en. https://doi.org/10.3403/30310928u
[20] ISO (2017b): Description of ISO/IEC 40180:2017. http://www.iso.org/iso/en/Catalogue
DetailPage.CatalogueDetail?CSNUMBER=33934&COMM
[21] ISO (2018). ISO 21001:2018: Educational organizations - Management systems for educa-
tional organizations- Requirements with guidance for use. Geneva: ISO. https://www.iso.
org/obp/ui/#iso:std:iso:21001:ed-1:v1:en. https://doi.org/10.3403/01746157u
[22] Marshall, S. (2006). E-Learning Maturity Model Version Two: New Zealand Tertiary In-
stitution E-Learning Capability: Informing and Guiding E-Learning Architectural Change
and Development Project Report. Wellington NZ: New Zealand Ministry of Education.
[23] Ubachs, G. (2009). Quality Assessment for e-Learning a Benchmarking Approach. Euro-
pean Association of Distance Teaching Universities (EADTU), Heerlen.
[24] Kear, K., Rosewell, J., Williams, K., Ossiannilsson, E., Covadonga Rodrigo, C., Sánchez-
Elvira Paniagua, A., Mellar, H. (2016). Quality Assessment for E-learning a Benchmark-
ing Approach (third edition). Maastricht, The Netherlands: European Association of Dis-
tance Teaching Universities (EADTU).
iJET ‒ Vol. 15, No. 19, 2020
35
Paper—Quality Standards in online Education: The ISO/IEC 40180 Framework
[25] Online Learning Consortium (OLC) (2002). Five pillars of quality online education. Re-
trieved from https://onlinelearningconsortium.org/about/quality-framework-five-pillars
[26] Quality Matters (2019). K-12 Rubric Workbook Standards for Course Design (Fifth Edi-
tion). https://www.nsqol.org/the-standards/quality-online-courses/
[27] Khan, B. H. (2002). A framework for web-based learning. In B. H. Khan (Ed.), Web-based
Training (pp. 75–98). Englewood Cliffs, NJ, USA: Educational Technology Publications.
[28] Frydenberg, J. (2002). Quality standards in E-learning: A matrix of analysis. International
Review of Research in Open and Distance Learning, 3(2), 1–15.
[29] McGorry, S. Y. (2003). Measuring quality in online programs. The Internet and Higher
Education, 6(2), 159–177. https://doi.org/10.1016/s1096-7516(03)00022-8
[30] Walker, S. L., & Fraser, B. (2005). Development and validation of an instrument for as-
sessing distance education learning environments in higher education: The Distance Edu-
cation Learning Environments Survey (DELES). Learning Environments Research, 8,
289–308. https://doi.org/10.1007/s10984-005-1568-3
[31] Young, A., & Norgard, C. (2006). Assessing the quality of online courses from the stu-
dents’ perspective. The Internet and Higher Education, 9(2), 107–115. https://doi.org/10.10
16/j.iheduc.2006.03.001
[32] Shelton, K. (2011). A review of paradigms for evaluating the quality of online education
programs. Online Journal of Distance Learning Administration, 4(1), 1–11.
[33] Gordin, P., & Hall, B. (2012), October. Using a team approach to assess online course
quality and training needs with a scorecard called ROCS. 18th Annual Sloan Consortium
International Conference on Online Learning, Lake Buena Vista, Florida
[34] Agariya, A. K., & Singh, K. (2012). E-Learning quality: Scale development and validation
in indian context. Knowledge Management and E-Learning, 4(4), 500–517. https://doi.org/
10.34105/j.kmel.2012.04.036
[35] eLQ (2008). E-learning quality aspects and criteria for evaluation of e-learning in higher
education (Report 2008:11 R). Stockholm: Högskoleverkets. http://www.eadtu.nl/e-xcellen
celabel/files/0811R.pdf
[36] OU-UK (2018). Online Learning in Tertiary Education in the Middle East and North Afri-
ca. Open University – UK
http://www.open.ac.uk/research/sites/www.open.ac.uk.research/files/files/Documents/Onli
ne%20learning%20in%20tertiary%20education%20in%20the%20Middle%20East%20and
%20North%20Africa%20.pdf.https://doi.org/10.9737/hist.2018.644
[37] Butcher, N. & Wilson-Strydom, M. (2013). A Guide to Quality in Online Learning Dallas
TX: Academic Partnerships
[38] Rajasingham, L. (2009). Breaking Boundaries: Quality E-Learning for the Global
Knowledge Society. iJET, 4(1), 58-65.
9 Author
Khitam Shraim is Associate Professor of Educational Technology at Palestine
Technical University – Kadoorie, Palestine. Email: k.shraim@ptuk.edu.ps
Article submitted 2020-04-21. Resubmitted 2020-05-27. Final acceptance 2020-05-28. Final version
published as submitted by the authors.
36
http://www.i-jet.org
