ChapterPDF Available

Abstract and Figures

The hype surrounding MOOCs has been tempered by scepticism about the quality of MOOCs. The possible flaws of MOOCs include the quality of the pedagogies employed, low completion rates and a failure to deliver on the promise of inclusive and equitable quality education for all. On the other hand, MOOCs that have given a boost to open and online education have become a symbol of a larger modernisation agenda for universities, and are perceived as tools for universities to improve the quality of blended and online education—both in degree education and Continuous Professional Development. MOOC provision is also much more open to external scrutiny as part of a stronger globalising higher education market. This has important consequences for quality frameworks and quality processes that go beyond the individual MOOC. In this context, different quality approaches are discussed including possible measures at different levels and the tension between product and process models. Two case studies are described: one at the institutional level (The Open University) and one at a MOOC platform level (FutureLearn) and how they intertwine is discussed. The importance of a national or international quality framework which carries with it a certification or label is illustrated with the OpenupEd Quality label. Both the label itself and its practical use are described in detail. The examples will illustrate that MOOCs require quality assurance processes tailored to e-learning and open education, embedded in institutional frameworks. The increasing unbundling of educational services may require additional quality processes.
Content may be subject to copyright.
The manuscript is a pre-print version of chapter 14 published
In M. Jemni, Kinshuk & M.K. Khribi (Eds.),
Open Education: from OERs to MOOCs. Lecture Notes in Educational
Technology (LNET). pp. 261-281 Aug 18, 2016. Berlin: Springer. [LINK]
We explicitly appreciate the support and contribution
of the editors to this manuscript.
Quality Frameworks for MOOCs
Darco Jansen
programme manager at the
European Association of Distance Teaching Universities (EADTU)
darco.jansen@eadtu.eu
Jon Rosewell
Senior Lecturer at the UK Open University
Jon.Rosewell@open.ac.uk
Senior Lecturer at the UK Open University
karen.kear@open.ac.uk
Abstract
The hype surrounding MOOCs has been tempered by scepticism about the quality of
MOOCs. The possible flaws of MOOCs include the quality of the pedagogies
employed, low completion rates and a failure to deliver on the promise of inclusive
and equitable quality education for all. On the other hand, MOOCs that have given a
boost to open and online education have become a symbol of a larger modernisation
agenda for universities, and are perceived as tools for universities to improve the
quality of blended and online educationboth in degree education and Continuous
Professional Development. MOOC provision is also much more open to external
scrutiny as part of a stronger globalising higher education market. This has important
consequences for quality frameworks and quality processes that go beyond the
individual MOOC. In this context, different quality approaches are discussed including
possible measures at different levels and the tension between product and process
models. Two case studies are described: one at the institutional level (The Open
University) and one at a MOOC platform level (FutureLearn) and how they intertwine
is discussed. The importance of a national or international quality framework which
carries with it a certification or label is illustrated with the OpenupEd Quality label.
Both the label itself and its practical use are described in detail. The examples will
illustrate that MOOCs require quality assurance processes tailored to e-learning and
open education, embedded in institutional frameworks. The increasing unbundling of
educational services may require additional quality processes.
Content
Introduction ............................................................................................................................................. 3
Why Does Quality of MOOCs Matter? .................................................................................................... 4
Quality Pedagogy and Dropout Rates ................................................................................................. 4
MOOCs for Lifelong Learning and Continuous Professional Development ........................................ 5
Unbundling of MOOC Services ............................................................................................................ 5
Consequences for Quality Processes ................................................................................................... 6
Quality Frameworks and Quality Processes ............................................................................................ 6
Case Study: The Open University ........................................................................................................ 9
Strategic Management .................................................................................................................... 9
Curriculum Design ........................................................................................................................... 9
Course Design .................................................................................................................................. 9
Course Delivery.............................................................................................................................. 10
Staff Support .................................................................................................................................. 10
Student Support ............................................................................................................................ 10
Case Study: FutureLearn ................................................................................................................... 11
The Openuped Quality Label ................................................................................................................. 12
The OpenupEd Label in Practice ........................................................................................................ 15
Discussion .............................................................................................................................................. 15
Analysis of Case Study ....................................................................................................................... 16
General Reflection ............................................................................................................................. 17
Conclusion ............................................................................................................................................. 18
Acknowledgments ................................................................................................................................. 18
References ............................................................................................................................................. 19
Introduction
Goal number four of the UNESCO Sustainable Development Goals states: “Ensure inclusive
and equitable quality education and promote lifelong learning opportunities for all”
(UNESCO 2015a). In addition, the Education 2030 Declaration (UNESCO 2015b, point 43,
page 16) states “The provision of tertiary education should be made progressively free, in
line with existing international agreements”. MOOCs are generally seen as contributing to
these goals as they provide complete learning experiences without any costs for the
participants. However, this does not necessarily mean that MOOCs ensure quality education
for all.
In exploring this issue, we start with the question: what is a MOOC? Bates (2015) considers
MOOCs to share a combination of the four key characteristics related to the acronym
Massive Open Online Course. A collaboration of EU-funded MOOC projects extended this to
the following definition
1
: “an online course designed for a large number of participants that
can be accessed by anyone anywhere, as long as they have an internet connection, is open
to everyone without entry qualifications and offers a full/complete course experience online
for free”. This definition was recently validated amongst European institutions (Jansen et al.
2015).
This definition positions MOOCs as part of both online and open education. But what
openness means has been the subject of debate (Open Education Handbook 2014);
openness must not be associated only with “free”. In general, open education has the
primary goal of removing barriers to education (Bates 2015). Mulder and Jansen (2015)
examine whether MOOCs can be instrumental in opening up education. Their main
conclusion is that MOOCs cannot remove all barriers to learning, and hence can only
contribute, to a certain extent, to ensuring quality education for all. The main flaw is that
quality assurance and accreditation schemes are not yet equipped for MOOCs.
This raises question of the relation between MOOCs and formal education. Are MOOCs
essentially forms of non-formal education, with related flexible provision? Or are MOOCs a
pathway to higher education, helping to ensure inclusive and equitable quality education for
all? The latter option implies the need for similar quality assurance processes as in formal
education.
This chapter reviews current and emerging practice for the quality assurance and quality
enhancement of MOOCs. It stresses the importance of the use of international quality
frameworks for MOOCs, embedded in institutional quality processes. In addressing the issue
of how best to assure quality in MOOCs, the chapter considers the question of why quality
matters for MOOCs. Quality frameworks and processes are then discussed, and illustrated
with two case studies. In this context, the OpenupEd Quality Label for MOOCs is considered.
1
http://www.openuped.eu/images/docs/Definition_Massive_Open_Online_Courses.pdf
Why Does Quality of MOOCs Matter?
Starting from the perspective of MOOC participants, we can argue that learners are entitled
to a high quality learning experience, whether they are enrolled on a fee-paying, credit-
bearing course or a MOOC. On this basis, it is valuable to consider whether the quality of
MOOCs should be assessed in the same way as a university course with degree awarding
processes, a question posed by Ehlers et al. (2013).
Quality Pedagogy and Dropout Rates
MOOC have the promise to widen access to higher education to millions of people, including
the developing world, and ultimately enhance the quality of life for millions (Daniel 2012).
However, MOOCs generally attract only well-educated learners who already have higher
education qualifications, and are already in employment (Macleod et al. 2015). MOOC
provision is dominated by a handful of platforms supported by elite universities, and very
few MOOCs offer formal pathways to recognised academic qualifications. This poses a
potential threat of inequality of access (Schuwer et al. 2015).
There is widespread scepticism of the quality of MOOCs and the pedagogies employed, for
example those of xMOOCs (Gaisch and Jadin 2014). Evidence supporting this sceptical view
can be found in a study by Margaryan et al. (2015), which evaluated a sample of 76 MOOCs
using a checklist of 37 items based on existing instruments for instructional design quality.
The research included principles of effective learning activity, learning resources and
organisation. The MOOCs evaluated were a random sample from those available in the late
2013 across a variety of platforms. The authors found that, while all MOOCs were well-
packaged, they all scored poorly overall (median 9, range 028, on a scale from 0 to72)
indicating poor instructional quality. Lowenthal and Hodges (2015) reviewed six MOOCs
applying the Quality Matters rubric intended for traditional for-credit online courses. They
concluded that “two of the MOOCs could pass this review and, therefore, be considered high
quality online courses”.
Poor quality pedagogy is considered a threat that can damage the reputation of the
institution and counteract the vision of MOOCs as being the best that higher education has
to offer (Schuwer et al. 2015). However, alternative MOOC approaches exist, providing more
inclusive and social approaches. Examples are pedagogical approaches like the well-known
cMOOC (Siemens 2012) and the more recent sMOOCs model (Brouns et al. 2016). In
addition, inclusive MOOC partnerships have emerged, such as the ECO project (Osuna Acedo
et al. 2016) and the OpenupEd initiative (Mulder and Jansen 2015). These initiatives are
characterised by distinct criteria and quality processes related to common features, specific
pedagogical models, training of skilled (e-)teachers and scalability of re-using MOOCs and
MOOC content.
A controversial topic related to the quality of MOOCs is the reported low completion rate.
Neuböck et al. (2015) and Macleod et al. (2015) have confirmed earlier findings by Hollands
and Tirthali (2014, p. 42) that only “3–15 % of all enrollees” complete a course. Jordan
(2014) reported that the majority of MOOCs had completion rate of less than 10 % with a
median of 6.5 % (p. 150), although more recent data show some improvement to a median
of 12.6 % (Jordan 2015). For many commentators, high dropout rates are a sign of the poor
quality of MOOCs. But this may be only true in relation to the metrics of formal education
i.e., if MOOCS are a pathway to formal higher education, low completion rates are
disastrous. However, it is argued that many MOOC participants do not want to do the entire
course; they are interested in gaining information and knowledge, but do not intend to get a
certificate of completion. To make the personal learning objectives more visible,
experiments with digital badging systems can be applied (Schön et al. 2013), and the
motivations and intentions of participants can be measured (Kalz et al. 2014).
MOOCs for Lifelong Learning and Continuous Professional Development
MOOCs have prompted a broad discussion on the use of technology-based modes of
teaching and learning in formal higher education and continuous professional development
(CPD), as well as in initiatives to open up education. It is expected that new modes of
teaching and learning, including MOOCs, will have an impact on the further development of
these three areas of provision and will change the higher education landscape (CPL 2015).
MOOCs have become a symbol of a larger modernisation agenda for universities,
intertwined with the concept of “unbundling”, and with related economic imperatives about
the viability, scalability, and sustainability of higher education (Selwyn 2014). Institutions are
developing online variants based around their own range of programs in order to raise their
national and international visibility, while helping to improve internal quality (e.g. Manturuk
and Ruiz-Esparza 2015).
Unbundling of MOOC Services
The growth of the MOOCs movement raises issues relating to the function and practice of
quality assurance. Currently, universities consider the quality assurance of the MOOCs they
provide to be an internal matter. However, MOOCs and other new modes of teaching are
part of the move to unbundling of educational services. MOOCs are complete courses
consisting of educational content, assessments, peer-to-peer tutoring and/or some limited
tutoring by academics. All of these components can be outsourced by higher education
institutions to third parties, for example, video recording of lectures, automatic grading
programs, authentication services and exam centres. Partnerships are growing between
universities and for-profit education companies, including major educational publishers and
global testing services. Partnering allows universities to fast-track into MOOC provision
without the need to build internal capabilities. As a consequence, quality assurance systems
can no longer focus only on educational institutions. However, Ossiannilsson et al. (2015)
note that national higher education ‘quality assurance standards and other regulatory
instruments cannot easily be applied to partner organisations as they were not designed to
regulate’ such entities (p. 46). Up to now, national quality assurance agencies in Europe have
not considered the quality assurance of MOOCs to be within their remit (e.g. NVAO 2014).
This would need to change if MOOCs were to become considerable parts of degree
programs in the future.
Consequences for Quality Processes
Since MOOC provision is much more open to external scrutiny than is campus-based higher
education, the quality of what a country’s own universities offer as MOOCs is important to
the ‘national brand’ of its higher education system; MOOCs form a window into the quality
of the national HE system as a whole. The UK QAA recognised this in their 2014 position
statement which states that MOOC providers should “ensure that they reflect the
established reputation of UK higher education” (QAA 2014). MOOCs may therefore be part
of a general endeavour to maintain competitive position in an expanding global market.
These concerns will influence the degree of support of national governments for MOOCs and
open education.
But this raises questions about how to ensure good governance, quality and overall
responsibility for educational credentials. Assuring the quality of MOOCs should be seen as
the shared responsibility of MOOC platforms, cross-institutional partnerships and
institutions, possibly with guidance and oversight from national quality agencies. To consider
the balance between these stakeholders, an institutional and a MOOC platform perspective
will be studied later in this chapter. In addition the quality label of a pan-European MOOC
partnership (OpenupEd) is discussed in this context.
Quality Frameworks and Quality Processes
The previous section suggests that quality of MOOCs can be considered from the following
four perspectives.
1. Quality from the learner’s point of view.
MOOCs attract a diverse range of learners, who come from different backgrounds
and have wide ranging motivations for enrolling in a particular MOOC (e.g., Hill 2013;
Kizilcec et al. 2013). Considering quality from the perspective of learners requires
engaging with the diverse goals, expectations, learning behaviours, and abilities of
learners to facilitate their own learning.
2. Quality connected to the pedagogical framework of the MOOC.
The pedagogical model of MOOCs should be designed to scale gracefully to unlimited
numbers of participants, requiring the teaching and support effort to not increase
significantly as the number of participants increase. Current research is beginning to
examine qualitative indicators for dialogue and interaction that can guide the choice
of pedagogical model. For example, Downes (2013) has formulated four key success
factors in this area: autonomy, diversity, openness and interactivity.
3. Quality related to the input elements.
These may include aspects such as instructional design, the content and resources,
multiple choice questions and assessment, the technology employed, and the quality
of the teacher (e.g. Margaryan et al. 2015; Lowenthal and Hodges 2015). For example
Costello et al. (2016) found a number of flaws when analysing the multiple choice
questions of several MOOCs. These aspects fit with the conventional views of course
quality.
4. Quality based on outcome measures.
These might include the number of learners completing a MOOC or achieving
certification. These metrics are (relatively) easy to measure. However, we know that
not all learners intend to follow the instructional pathway of a MOOC. Taking
completion rate as a measure for the quality of a MOOC has therefore been criticised
(e.g. Weller 2013; Clark 2016). It is argued that low values of conventional measures,
such as retention and completion, may not signal poor quality.
Consequently, the concept of quality in online education, and particularly in MOOCs, is
complex. There are a variety of stakeholders involved: learners and educators, higher
education institutions (HEIs), MOOC platform providers, quality agencies, governments, and
potentially employers and others who might recognise achievement in a MOOC. Quality can
also be viewed at three levels: macro (national), meso (institution) and micro (course) level
(Nordkvelle et al. 2013).
Figure 1 provides a simple view of MOOC quality processes. A learner faced with a choice of
MOOCs will wish to be assured of their quality, and might wish to use reviews and
recommendations of other learners. However, despite the very large numbers of MOOC
learners, no MOOC rating website has become prominent and, given that many MOOCs are
presented only once or a few times and may be changed between presentations, this
approach may never bear fruit.
Fig. 1: A model for MOOC quality processes
A potential learner therefore only has available a notion of brand reputation attaching to the
MOOC platform, the originating institution, and possibly the course author. However, Daniel
(2012) cautions that university brand is a poor measure of quality in online teaching, since
reputations are gained primarily in research rather than teaching. Nevertheless, both HEI
and platform have a stake in maintaining their brand reputation. They can impose control by
acting as reviewer and final gatekeeper, and also ensuring that a quality process is followed
during course creation (This assumes that MOOCs remain predominantly products of HEIs
and are often related to core curriculum.)
One can see the system encapsulated in Fig. 1 as a quality system where improving quality
should be reflected in some measure. However, what should be optimised for a MOOC:
learner satisfaction, completion rate, or some other measure? These conventional measures
may not be appropriate if the intentions of MOOC learners differ from those of a
conventional university student (Ehlers et al. 2013). Butcher and Hoosen (2014) also
question whether tightly structured frameworks for quality assurance can be applicable to
MOOCs, because openness and flexibility are primary characteristics of these new
approaches. However, the authors also suggest that, since both conventional HEIs and
MOOCs offer higher education, quality principles developed for HE could be used to improve
the quality of MOOCs and OER.
One way of dealing with these tensions would be to use a national or international quality
framework which carries with it a certification or label. Such a visible recognition would act
as a reassurance to all the stakeholders in MOOCslearners, authors, institutions,
platforms, employers, and quality agencies. In this chapter, we focus on the OpenupEd
Quality Label as an example.
The question then arises whether such a MOOC quality label should focus on product or
process, and this echoes a long-running tension in the landscape of quality assurance in
education. Ossiannilsson et al. (2015) characterise this as a spectrum: from systems which
check compliance to norms and often focus on product, to systems that aim at quality
enhancement by focusing on process. They align this with a maturity model: low maturity
systems are characterised by externally set norms, whereas in high maturity systems
institutions have embedded processes aimed at quality enhancement towards their own
objectives.
Ossiannilsson et al. (2015) present a global survey of quality models for e-learning. They find
that most models take a holistic view of quality, recognising the need to address many
aspects of the enterprise. Although the models vary considerably in the detail and number of
quality indicators, most cover a consistent set of important dimensions. For example, the E-
xcellence framework uses six dimensions: Strategic Management, Curriculum Design, Course
Design, Course Delivery, Staff Support and Student Support (Kear et al. 2014). If there is a
consensus that this range of dimensions is appropriate for e-learning generally, it seems
appropriate to use a similar framework for MOOCs.
The following case studies illustrate these ideas, and explore how quality can be assured
during the development and presentation of MOOCs.
Case Study: The Open University
This first case study discusses the UK Open University (UKOU), and its processes for offering
MOOCs. This case study is presented broadly according to the six quality dimensions
mentioned above.
Strategic Management
The UK Open University (UKOU) has a mission to increase access to higher education. Its
courses and qualifications are open to all, regardless of prior qualifications. Most UKOU
courses require payment, but since 1992 the UKOU has offered some learning resources for
free. At the time of writing, it offers MOOCs in partnership with FutureLearn, as well as
offering online open courses via its OpenLearn OER repository
2
, some of which offer Mozilla
badges on completion. FutureLearn MOOCs have a definite start time, and are hence
presented to a cohort of learners; in contrast, OpenLearn courses can be studied at any time.
In both cases there is a well-structured process for the development of the course, and for
monitoring it in presentation, so that it can be improved.
The development of an open course follows a similar process to that used to develop all
UKOU modules, although at a smaller scale. It still involves a number of staff from across the
university, including academic faculties and the Learning and Teaching Solutions (LTS) unit
which carries out course production.
Curriculum Design
A central Open Media Unit (OMU) has a specific remit to oversee and support open access
developments, and each faculty has an Open Media Fellow whose role is to encourage the
development of open access resources within the faculty. The process for approving a new
course begins with a proposal from the faculty. This is then subject to institutional approval
by OMU. In the case of a FutureLearn MOOC, there is also an approval process by
FutureLearn, which depends on the fit with existing and proposed FutureLearn MOOCs from
all partners.
Course Design
One aim of the design stage is that the course should provide a mix of different media and
activities which will engage learners and support their learning. In the case of a FutureLearn
MOOC, each week’s study consists of a number of ‘steps’ of up to 20 min study time. The
steps include resources and activities, e.g. videos, animations, discussions. Interaction
between learners is encouraged by having a discussion thread associated with every step. At
the end of each study week there is a quiz so that learners can check their knowledge and
understanding. During the course development stage, any third-party resources will be
cleared for copyright; course authors are encouraged to use open educational resources or
other material available via a Creative Commons licence.
2
http://www.open.edu/openlearn
Course Delivery
After several stages of drafting, critical reading, editing and checking, the course is put onto
the platformFutureLearn for MOOCs or OpenLearn for UKOU open courses. There is then a
final check before it is signed off by the course authors as ready for presentation. For a
FutureLearn MOOC in presentation, UKOU trained online facilitators monitor the discussion
threads, engaging with learners in the discussions as appropriate. In addition, FutureLearn
moderates the discussions to minimise any offensive contributions (learners can identify
such contributions themselves).
Staff Support
Courses are typically developed during a short but intensive period by just one or two
experienced UKOU academics. Course authors are supported by critical readers (who are
often UKOU tutors) and colleagues from OMU and LTS, in particular an experienced OU
editor. At an early stage in the course development, a Learning Design workshop takes place,
based on a framework developed at the UKOU (Galley 2015; Conole 2013). The workshop
involves specifying the aims/learning outcomes for each week of study, together with the
learning resources and activities. Training is offered by the UKOU audio visual department
for any staff who are to appear in course videos.
Student Support
Once the course is in presentation, a number of quality metrics and processes come into
play. Learner activity is closely monitored and measured, and the data presented in detail
back to the course authors in the form of a dashboard. Various measures of learner
retention and activity are used as key parameters, both while the course is in presentation
and once it is finished. For example, in a FutureLearn MOOC it is possible to tell if learners
are struggling to complete a particular step; on this basis, the learning resources for that
step can be improved for later presentations and the facilitators can be briefed on how to
help learners in the current cohort.
At the end of the course, learners are invited to complete a feedback survey; or if they
decide to withdraw part way through the course, they are invited to give feedback at that
point. OMU also reviews the discussion threads, in order to investigate learners’ reactions to
different parts of the course. The survey data, together with retention data, student activity
data and feedback gathered via the discussion threads is used to carry out a review after the
first course presentation. On this basis, decisions can be made as to whether the course
should continue in presentation and how it could be improved for learners in the future.
Case Study: FutureLearn
FutureLearn is an organisation that partners with universities and other groups to provide
MOOCs on a wide range of topics. It is a limited company wholly owned by the UK Open
University (UKOU) and benefits from the UK OU’s long experience of online learning. The
initial 12 FutureLearn partners were high status UK universities. At the time of writing,
FutureLearn has 73 partners: the majority are universities in the UK and other countries, but
there are also partners such as the British Museum and the European Space Agency.
FutureLearn courses typically last 38 weeks, and require 25 h of study per week. The
largest course, on English as a Foreign Language, attracted 400,000 learners in early 2015.
FutureLearn has over 2.5 million registered users in more than 190 countries. In July 2015,
60 % of FutureLearn users were from outside the UK; 60 % were female; and the age range
was from 13 to 93 (JISC 2015). Most users already have a degree, but FutureLearn also has
resources aimed at school leavers, including those making the transition to university.
FutureLearn has its own MOOC platform and hosts the MOOCs from all partners. The MOOC
platform will operate on a range of devices, using different browsers. FutureLearn set out to
create a ‘modern, attractive, experience’ for the learner (Simon Nelson in Chung 2015) and it
won the UXUK award for best user experience in late 2015
3
. The pedagogical approach aims
to make the learning experience simple and well-structured. Learning resources (e.g. text
and videos) are organised into “steps”, which can be flagged as completed so that learners
(and FutureLearn) can easily keep track of their progress. A model of social learning also
informs the design; for example, discussion threads are closely integrated with the learning
resources in each step so that learners can share ideas and experiences related to the
material they are studying
The FutureLearn approach of combining a clear structure and navigation with opportunities
for discussion and debate appears to have led to high learner retention. An average of 22 %
of the people who begin a FutureLearn course are (to use FutureLearn’s term) ‘fully
participating learners’: they have carried out at least 50 % of the steps and all the
assessments (typically weekly quizzes). In terms of the number of people who sign up for a
FutureLearn course, 12 % are fully participating learners.
FutureLearn has a publicly available set of ‘Openness Principles’ which indicate its
philosophy
4
with regards to open education, intellectual property and privacy. FutureLearn
also has a detailed policy on ‘Accessibility and Inclusion’, which is used when reviewing
courses
5
. This specifies the responsibilities of both FutureLearn and of the partner
organisation providing the course material. The policy refers to FutureLearn’s compliance
with the World Wide Web Consortium’s web content accessibility guidelines
6
. For example,
the FutureLearn platform can be used via a keyboard and a screen reader; attention is paid
to suitable font sizes and use of colour.
3
http://uxukawards.com/
4
https://about.futurelearn.com/terms/openness/
5
https://about.futurelearn.com/terms/accessibility-policy/
6
http://www.w3.org/TR/WCAG/
Learners may pay for a ‘Statement of Participation’ to demonstrate that they have
completed a course, including the assessment. For some courses, and at a somewhat higher
cost, FutureLearn offers invigilated examinations, in collaboration with Pearson VUE
7
, which
lead to a more formal “Statement of Attainment”. No FutureLearn courses currently provide
credit points from the partner universities, although there is nothing to prevent this if the
partner considers it appropriate.
The Openuped Quality Label
The OpenupEd partnership is an alliance of institutional MOOC providers, brought together
by the European Association of Distance Teaching Universities (EADTU), who agree to follow
the quality principles and practices represented in the OpenupEd Quality Label. The partners
in OpenupEd have a commitment to opening up education through MOOCs to the benefit
both of learners and of wider society. To this end, partners endorse the eight distinctive
features described in Table 1 as guiding principles for their MOOC offering.
Table 1. The distinctive features of OpenupEd MOOCs
OpenupEd distinctive
features
Openness to learners
[OL]
Digital openness
[DO]
Learner-centred approach
[LC]
Independent learning
[IL]
Media-supported interaction
[MI]
Recognition options
[RO]
Quality focus
[QF]
Spectrum of diversity
[SD]
7
http://home.pearsonvue.com/
The OpenupEd Quality Label provides a process-based quality enhancement framework for
MOOCs and their providers. It was derived from the E-xcellence label
8
(mentioned earlier)
which provides a methodology for assessing the quality of e-learning in higher education. E-
xcellence has a review process that is based around a number of benchmark statements,
grouped according to the six dimensions of Strategic Management, Curriculum Design,
Course Design, Course Delivery, Staff Support and Student Support. E-xcellence has been
periodically updated in the light of feedback from its reviewers and to reflect the changing
nature of e-learning in HE; at the time of writing (2016) the current benchmarks and manual
(Williams et al. 2012) are being updated.
The OpenupEd quality label
9
(Rosewell and Jansen 2014) builds on E-xcellence by taking a
similar approach; however, it adopts a lighter weight process and adapts the benchmarks to
better suit MOOCs. The benchmarks are divided into two groups: one that applies at
institutional level and a second that applies to individual courses. The institution should be
considered against the full set of institutional-level benchmarks but only at intervals. Every
MOOC needs to be considered, but only against the much smaller number of course-level
benchmarks.
An outline of the OpenupEd Quality Label process is as follows. OpenupEd partners are
expected to be higher education institutions (HEI) that meet national requirements for
quality assurance and accreditation. The HEI should have an internal procedure to approve a
MOOC, typically a “light-touch” version of the procedure applied to formal courses. The HEI
should endorse the eight distinctive OpenupEd features listed in Table 1. New partners will
obtain the OpenupEd Quality Label by a self-assessment and review process that will
consider benchmarks both at institutional and course level (for two courses initially). The
label must be renewed periodically; between institutional reviews, MOOCs will be reviewed
at course level only. The HEI is expected to evaluate and monitor each MOOC in
presentation, including data on participation, completion and student satisfaction, and an
assessment of equality, quality, and diversity.
The self-assessment and review focus on the 21 institutional and 11 course-level
benchmarks. A “quick scan” checklist is provided (Fig. 2) which lists the benchmarks with an
accompanying grid to record two aspects. Firstly, an overall judgement on the extent to
which the benchmark is achieved is recorded using a four-point scale: not achieved, partially
achieved, largely achieved, or fully achieved (E in Fig. 2). Secondly, a mapping can be made
between each benchmark and the eight OpenupEd distinctive features; an initial mapping is
provided but this can be adapted where necessary (D in Fig. 2). For example, in Fig. 2
benchmark 22 “A clear statement of learning outcomes for both knowledge and skills is
provided” is mapped to the distinctive feature “IL—Independent learning” to suggest that
evidence gathered in relation to this benchmark is also likely to provide evidence of a course
suited to independent learning.
8
http://e-xcellencelabel.eadtu.eu/
9
http://www.openuped.eu/quality-label
Fig. 2 Part of the quick scan checklist. Key: A Benchmark number; B Benchmark
statement; C Cross-reference to E-xcellence manual; D Mapping to OpenupEd
features (Table 1 for abbreviations); E Grid for recording benchmark achievement
(NA, Not achieved) ; PA, Partially achieved ; LA, Largely achieved ; FA, Fully achieved)
The quick scan can be used to give an initial picture of areas of strength and weakness. It can
also highlight: where benchmarks may not be fully appropriate; where they may fail to
capture good practice in a particular HEI or MOOC; and where additional detailed indicators
might be helpful. The quick scan should then be fleshed out by a more detailed self-
assessment process, ideally including different stakeholders such as teachers, managers,
course designers and students. This should gather evidence for each benchmark, including
the extent to which the evidence also supports the distinctive OpenupEd features. A plan
detailing improvement actions is then prepared. The documented self-assessment and the
improvement plan form the basis of a final review and discussion with external assessors,
who then prepare a final report including their recommendation for the award of the
OpenupEd Quality Label.
A number of documents and templates support this process. Assessor’s notes are provided
that cross-reference the OpenupEd benchmarks to additional indicators and background
material in the E-xcellence manual (Williams et al. 2012), with supplementary material
provided for MOOC-specific aspects where necessary (Fig. 3).
Fig. 3: Example assessor’s note, with cross-references to the E-xcellence manual
The OpenupEd Label in Practice
The initial partners in OpenupEd were all members of EADTU. The consortium took the view
that MOOCs from these providers were already being created under institutional quality
processes that met the requirements of the OpenupEd label, and the initial portfolio of
OpenupEd MOOCs therefore were not required to go through an additional review process.
Rodrigo et al. (2014) report a self-assessment exercise of over 20 MOOCs on the UNED
platform using the OpenupEd benchmarks. The assessed MOOCs had all been developed by
experienced staff under a strong existing institutional quality framework for online learning;
they could therefore be expected to meet the OpenupEd benchmarks. However, the
exercise highlighted some benchmarks which could not confidently be scored as largely or
fully achieved; for example not all MOOCs gave a clear statement of learning outcomes, and
materials were published under a restricted rather than an open licence. These are aspects
that could be taken forward for discussion and perhaps inform institutional policy, leading to
quality enhancement.
The authors also report that additional and more specific indicators would improve the
benchmarking for their institution; these include specific academic roles (curator, facilitator),
a variety of certification (badges, ECTS credit), and flavours of MOOC pedagogy (c-MOOC, X-
MOOC, SPOC). The OpenupEd assessor’s notes do incorporate most of these issues (see Fig.
3 for example), but they were judged too specific to be included in the standard
benchmarks. Rodrigo et al. also report issues such as teacher’s workload and accessibility
issues which became apparent during a course-level exercise, but which are covered by
OpenupEd benchmarks at institutional rather than course level. The OpenupEd quality
process suggests that initial self-assessment can be used to highlight benchmarks that are
not fully appropriate to an HEI and to discover additional indicators needed to capture
aspects of good practice. Rodrigo and colleagues therefore conclude that the OpenupEd
Quality Label is a versatile tool, providing guidance with sufficient flexibility to meet an
institution’s aspirations without being a straitjacket.
Discussion
The two case studies in Sect. 3 presented aspects of MOOC quality from the perspective of
an institution (the UKOU) and of a MOOC platform provider (FutureLearn). In the discussion
that follows, we will focus on the joint enterprisea representative MOOC designed by the
UKOU and presented via FutureLearnand do so through the lens of the OpenupEd Quality
Label and its benchmarks (Rosewell and Jansen 2014). The discussion is mapped to
appropriate the OpenupEd benchmarks (for example #1) and OpenupEd features (for
example DO; see Table 1 for key). To complete a quick scan (Fig. 2) for a specific course
would require in addition a judgement on whether the benchmarks and features are fully
achieved or not.
Analysis of Case Study
Although we focus on this single example, it is likely that arrangements work similarly with
other HEIs and MOOC platforms. It is also clear that quality emerges from the joint
enterprise and is not solely the responsibility of one partner [#5, QF]. However, there is one
reasonably clear division between the originating institution and platform provider marked
by handover to the platform for publishingbefore that point the weight of quality
assurance falls on the HEI, with FutureLearn taking a greater role at and after handover [#6,
QF].
The OpenupEd Quality Label takes the view that MOOC quality is best approached
holistically, looking at the institutional processes as well as the completed product.
Ossiannilsson et al. (2015) find the same approach in most e-learning quality frameworks.
Both the UKOU and FutureLearn have clear strategies and processes for MOOC production
which are seen as essential to ensuring quality [#3, #5, QF]. These include commissioning
processes on both sides so that course proposals are scrutinised at an early stage, one
output of which is a course description [#18, #22, OL, IL]. This ensures that the course will
meet the needs of learners [LC], as well as contribute to a MOOC portfolio that meets the
strategic goals of both the HEI and platform [#1, #8, OL]. The UKOU delivers MOOCs on
FutureLearn (with certificates) and on OpenLearn (with badges) [RO], which also includes
access material [#8, OL, SD] and tasters for core non-MOOC curriculum [#7].
Both the UKOU and FutureLearn take very clear positions on aspects such as openness [#11,
#27, DO], accessibility and inclusion [#4, OL], and these values therefore permeate normal
work, helping to ensure that material is produced that conforms to accepted standards
without needing rework at a late stage.
Course design is mainly the responsibility of the HEI, but is supported by guidance
documents from FutureLearn [#9]. A strong steer is provided by the affordances of the
platform, which is directed to a particular pedagogical model [#13, #23, LC, IL, MI]. This
model appears to be successful, although it may limits the freedom of course authors to take
alternative approaches. At a practical level, this can be seen in the way that FutureLearn
currently only hosts a restricted set of resource types and activities [#13, #23], requiring the
author or HEI to make alternative arrangements for some resources; the result is that not all
FutureLearn courses are entirely self-contained [#5].
The UKOU process for course design follows the model used in development of their
standard non-MOOC provision [#6, QF], although with fewer staff and at an accelerated
pace. The early learning design workshop ensures that there is coherence between content,
teaching and learning strategy and assessment [#23, LC, IL]. This workshop, together with
guidelines from FutureLearn and the affordances of the platform itself (with its clear design
in “steps” and the emphasis on social learning [#20, #24, LC, SD]), also ensures that there is
interactivity (student-to-student and student-to-content) to encourage active engagement
[#29, LC, IL, MI]. Team writing and critical reading of drafts help to assure that content is
relevant, accurate and current [#25, QF]. The process of course approval, which includes
choice of authors, helps to ensure that staff have the required skills to develop material
suitable for the proposed audience [#26, QF]. The UKOU already has significant capability in
delivering online education with trained specialist support staff [#17, QF], but it has also
provided some specific MOOC and media training [#15, QF]. The UKOU also has institutional
structures and processes which promote educational research and innovation as important
activities, for example, its Institute of Educational Technology [#2, #16, QF]. FutureLearn
complements this with the FutureLearn Academic Network which exists to promote research
around the FutureLearn platform and its learners [#2, #16, QF].
A clear division of responsibility is seen in course delivery, with FutureLearn having
responsibility for providing the platform, which is effectively outsourced by the HEI,
presumably with clear service level agreements and financial arrangements in place [#5,
#12]. However, there is a shared responsibility for human input: FutureLearn provide
moderators and the UKOU provide the course facilitators who act in an academic role [#21,
IL]. The UKOU provides training for those undertaking the facilitator role, ensuring that staff
delivering the course have suitable skills [#15, QF]. FutureLearn publish policies and
guidelines for support that is available to participants [#19, #21, OL, IL]. There is a further
division of responsibility in assessment: UKOU authors create embedded self-assessment
and a final quiz [#29, #30, LC, IL]; FutureLearn handles certification [#31, RO].
Finally there is also a division of responsibility for monitoring and evaluating courses. The
FutureLearn platform provides analytic and survey data, which is fed back to the UKOU as a
dashboard during presentation [#14, QF]. UKOU course staff monitor the presentation and
are able to respond to issues raised in discussion threads, although there is limited scope for
changing the material itself during presentation. A thorough review by the UKOU after
presentation is used to decide whether to continue presentation and to identify changes
required to enhance quality [#32, QF]; since this is overseen by an institutional body there is
a mechanism to share experience more widely [#10, QF].
General Reflection
It should be clear from the above discussion that quality of MOOCs can only be measured
against their design principles. Quality is the result of the application of a systematic process
of design and evaluation, aimed at improvement over time. As such, quality enhancement
for MOOCs is an iterative process, and design methodology at different levels of granularity
can support this (e.g. see Dalziel et al. 2013, for learning design principles).
Quality needs to be thought about at both the institutional and course level, and the focus
must include process and not just the resulting product. Both FutureLearn and the UKOU
have invested in structures and processes that embed a concern with quality throughout the
development, delivery and evaluation of a MOOC in order to assure the quality of any
individual MOOC. Noticeably absent from the case study descriptions is any formal stage in
the process that is labelled “Quality assurance”: this is because a concern with quality
permeates the whole process.
The OpenupEd Quality Label and its benchmarks is sufficiently broad ranging that it can
capture the quality practices described in these two case studies. Clearly the contributions of
both parties (UKOU and FutureLearn) would have to be considered as part of the review and
label. Members of OpenupEd are expected to be HEIs and it would be the HEI and its MOOCs
that would be labelled, rather than the platform provider. An interesting boundary case
occurs when a MOOC is transferred from one platform to another; for example, MOOCs
presented by the UKOU on FutureLearn are later made available as self-paced open courses
on its OpenLearn site. In this case, the institution will need to check that the course still
complies with the OpenupEd features.
Conclusion
This chapter has explored the key issue of quality in relation to MOOCs. It has considered
how questions of quality are raised by MOOCs, and has proposed approaches for assuring
the quality of MOOCs. The chapter illustrated these ideas through two case studies of quality
assurance for MOOCs, one focussing on FutureLearna platform provider which supports
many institutionsand the other on the UKOUa single institution which uses multiple
platforms. These case studies illustrated the different quality processes involved.
It is concluded that MOOCs require quality assurance processes that are tailored to e-
learning, embedded in institutional frameworks. There are existing e-learning quality
approaches intended for use in formal, credit-bearing education that can be pressed into
service; Ossiannilsson et al. (2015) provide a useful overview and guide to the issues.
The chapter also introduced the reader to the pan-European OpenupEd framework for
enhancing quality in the development of MOOCs. The OpenupEd Quality Label is derived
from the E-xcellence label, an established approach to quality assurance of e-learning and
blended learning that has roots in the experience of open and distance learning institutions.
As HEIs increasingly collaborate on a global scale on their MOOC provision, additional quality
processes are required. This is related to the unbundling of educational services and
illustrated with FutureLearn and OpenupEd. These two examples demonstrate that this
unbundling introduces distinct quality processes at a cross-institutional level. The OpenupEd
Quality Label requires courses to address openness to learners and open licencing and is
thus firmly rooted in the Open Education movement. This international dimension is
expected to gain in importance as new kinds of partnership emerge (Osuna Acedo et al.
2016) and if MOOCs are to become considerable parts of degree programs in the future.
Acknowledgments
This research is partly conducted as part of the European Union-funded project
SCORE2020Support Centres for Open education and MOOCS in different Regions of
Europe 2020 (Ref. 2014-1-NL01-KA203-001309). We would like to thank all partners in
SCORE2020 and OpenupEd for their contributions. However, sole responsibility for this
article lies with the authors, and the Commission, SCORE2020 partners and OpenupEd
partners are not responsible for any use that may be made of the information contained
therein.
References
Bates, A. W. (2015). Teaching in a digital age. Guidelines for designing teaching and learning
for a digital age. Retrieved from http://opentextbc.ca/teachinginadigitalage/
Brouns, F., Teixeira, A., Morgado, L., Fano, S., Fueyo, A. & Jansen, D. (2016). Designing
massive open online learning processes: The smooc pedagogical framework. In M. Jemni,
Kinshuk & M.K. Khribi (Eds.), Open Education: From OERs to MOOCs. Berlin: Springer.
Butcher, N., & Hoosen, S. (2014). A guide to quality in post-traditional online higher
education. Dallas: Academic Partnerships.
Chung, C. (2015). The MOOC platform with a twist: The emergence of UK-based FutureLearn
(An Overview of the MOOC Provider Based on an Interview with CEO Simon Nelson,Online).
15 June 2015. Retrieved from https://www.class-central.com/report/futurelearn/ (Accessed
20 Nov 2015).
Clark, D. (2016) MOOCs: course completion is wrong measure. Donald Clark Plan B, 27
February 2016. Retrieved from http://donaldclarkplanb.blogspot.co.uk/2016/02/moocs-
course-completion-is-wrong-measure.html
Conole, G. (2013). Designing for learning in an open world. New York: Springer.
Costello, Brown & Holland. (2016). What questions are MOOCs asking? An evidence-based
investigation. In M.Khalil, M. Ebner, M. Kopp, A. Lorenz & M. Kalz (Eds.), Proceedings of the
European Stakeholder Summit on experiences and best practices in and around MOOCs (pp.
211221). Retrieved from http://emoocs2016.eu/wp-
content/uploads/2016/02/proceedings-emoocs2016.pdf
CPL. (2015). The changing pedagogical landscape. Retrieved from
http://www.changingpedagogicallandscapes.eu/publications/
Daniel, J. (2012). Making sense of MOOCs: Musings in a maze of myth, paradox and
possibility. Journal of Interactive Media in Education, 2012(3). Retrieved from
http://jime.open.ac.uk/article/view/2012-18
Dalziel, J., Conole, G., Wills, S., Walker, S., Bennett, S., & Dobozy, E., et al. (2013). The
Larnaca declaration on learning design2013. Retrieved from www.larnacadeclaration.org
Downes, S. (2013). The quality of massive open online courses. Retrieved from
http://mooc.efquel.org/week-2-the-quality-of-massive-open-online-courses-by-stephen-
downes/
Ehlers, U.D., Ossiannilsson, E., & Creelman, A. (2013). MOOCs and QualityWhere are we
where do we go from here…? MOOC Quality Project, May 6, 2013. Retrieved from
http://mooc.efquel.org/first-post-of-the-series/
Gaisch, M., & Jadin, T. (2014). Enhanced MOOCs for the conceptual age: A diversified lens on
the MOOCversity. In D. Jansen & A. Teixeira (Eds.). Position papers for European cooperation
on MOOCs (pp. 120129). Heerlen: EADTU. Retrieved from
http://home.eadtu.eu/images/Position_papers_for_European_cooperation_on_MOOCs.pdf
Galley, R. (2015). Learning Design at the Open University: Introducing methods for enhancing
curriculum innovation and quality. Retrieved from http://www.open.ac.uk/iet/learning-
design/sites/www.open.ac.uk.iet.learning-design/files/files/ecms/web-content/Learning-
Design-at-the-Open-University.pdf
Hill, P. (2013). Emerging student patterns in MOOCs: A (Revised) graphical view. Retrieved
from http://mfeldstein.com/emerging-student-patterns-in-moocs-a-revised-graphical-view/
Hollands, F., & Tirthali, D. (2014). Why Do Institutions Offer MOOCs? Online Learning, 18(3).
Retrieved from http://olj.onlinelearningconsortium.org/index.php/jaln/article/view/464
Jansen, D., Schuwer, R., Teixeira, A., & Aydin, H. (2015). Comparing MOOC adoption
strategies in Europe: Results from the HOME project survey. International Review of
Research in Open and Distributed Learning, Special Issue on European MOOCs, 16 (6). ISSN
1492-3831. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/2154
JISC. (2015). The free learning revolutionSimon Nelson, FutureLearn. Retrieved from
https://www.jisc.ac.uk/news/the-free-learning-revolution-simon-nelson-futurelearn-22-jul-
2015
Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses.
International Review of Research in Open and Distance Learning, 15(1). Retrieved from
http://www.irrodl.org/index.php/irrodl/article/view/1651
Jordan, K. (2015). Massive open online course completion rates revisited: Assessment,
length and attrition. International Review of Research in Open and Distributed Learning,
16(3), 341358.
Kear, K., Williams, K., & Rosewell, J. (2014). Excellence in e-learning: A quality enhancement
approach. In: EFQUEL Innovation Forum 2014/International LINQ Conference 2014: Changing
the TrajectoryQuality for Opening up Education, 79th May 2014, Crete. Retrieved from
http://oro.open.ac.uk/40207/
Kalz, M., Kreijns, K., Niellissen, G., Castaño-Muñoz, J., Guasch, T., & Espasa, A., et al. (2014).
MOOCKnowledge: Establishing a large-scale data collection about participants of European
Open Online Courses. In D. Jansen, & A. Teixeira (Eds.), Position papers for European
cooperation on MOOCs (pp. 113119). Heerlen: EADTU. Retrieved from
http://home.eadtu.eu/images/Position_papers_for_European_cooperation_on_MOOCs.pdf
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: Analyzing
learner subpopulations in massive open online courses. In Proceedings of the third
international conference on learning analytics and knowledge (pp. 170179). ACM.
Lowenthal, P. R., & Hodges, C. B. (2015). In search of quality: Using Quality Matters to
analyze the quality of massive, open, online, courses (MOOCs). The International Review of
Research in Open and Distributed Learning, 15(5). Retrieved from
http://www.irrodl.org/index.php/irrodl/article/view/2348/3411
Macleod, H., Haywood, J., Woodgate, A., & Alkhatnai, M. (2015). Emerging patterns in
MOOCs: Learners, course design and directions. TechTrends, 59(1), 5663.
http://dx.doi.org/10.1007/s11528-014-0821-y
Manturuk, K., & Ruiz-Esparza, Q. M. (2015). On-campus impacts of MOOCs at Duke
University. EDUCAUSE Review, August 3, 2015. Retrieved from
http://er.educause.edu/articles/2015/8/on-campus-impacts-of-moocs-at-duke-university
Margaryan, A, Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open
online courses (MOOCs). Computers & Education 80, 7783. Retrieved from
http://dx.doi.org/10.1016/j.compedu.2014.08.005
Mulder, F., & Jansen. D. (2015). MOOCs for Opening Up Education and the OpenupEd
initiative. In: C. J. Bonk, M. M. Lee, T. C. Reeves, T. H. Reynolds (Eds.). The MOOCs and Open
Education Around the World. New York: Routledge Tayler & Francis Group.
http://eadtu.eu/documents/Publications/OEenM/OpenupEd_-
_MOOCs_for_opening_up_education.pdf
Neuböck, K.; Kopp, M., & Ebner, M. (2015). What do we know about typical MOOC
participants? First insights from the field. In: EMOOCS2015. Proceedings Papers. European
Stakeholder Summit on Experiences and best practices in and around MOOCs (pp. 183190).
1820 May 2015. Université Catholique de Louvain, Mons (Belgium).
Nordkvelle, Y., Fossland, T., & Netteland, G. (Eds.). (2013). Kvalitet i höjre utdanning. Norges
universitet Oslo: Akademiska Forlag (In Norwegian). Retrieved from
http://norgesuniversitetet.no/files/kvalitet_i_fleksibel_hoyere_utdanning.pdf
NVAO. (2014). MOOCs and online HE, a survey. Retrieved from
https://www.nvao.net/system/files/pdf/NVAO%20MOOCs%20and%20Online%20HE%20A%
20Survey%202014.pdf
Open Education Handbook. (2014). Retrieved from http://booktype.okfn.org/open-
education-handbook-2014/what-is-open-education/
Osuna Acedo, S., Frau-Meigs, D., Camarero Cano, L., Bossu, A., Pedrosa, R. & Jansen, D.
(2016). Intercreativity and interculturality in the virtual learning environments of the ECO
MOOC project. In M. Jemni, Kinshuk & M.K. Khribi (Eds.), Open Education: From OERs to
MOOCs. Berlin: Springer.
Ossiannilsson, E., Williams, K., Camilleri, A. F., & Brown, M. L. (2015). Quality models in
online and open education around the globe: State of the art and recommendations.
Retrieved from http://icde.org/admin/filestore/Resources/Reports/ICDEQualitymodels2.pdf
QAA. (2014). Statement on massive open online courses. Retrieved from
http://www.qaa.ac.uk/en/Publications/Documents/QAA-position-statement-MOOCs.pdf
Rodrigo, C., Read, T., Santamaría, M., & Sánchez-Elvira, A. (2014). OpenupEdLabel for MOOC
quality assurance: UNED COMA initial self-evaluation. In: Actas del V Congreso Internacional
sobre Calidad y Accesibilidad en la Formación Virtual (CAFVIR 2014) (pp. 551555).
Rosewell, J., & Jansen, D. (2014). The OpenupEd quality label: benchmarks for MOOCs.
INNOQUAL: The International Journal for Innovation and Quality in Learning, 2(3), 88100.
Retrieved from http://oro.open.ac.uk/41173/
Schuwer, R., Gil-Jaurena, I., Hakan Aydin, C., Costello, E, Dalsgaard, C., & Brown, M., et al.
(2015). Opportunities and threats of the MOOC movement for higher education: the
European perspective. International Review of Research in Open and Distributed Learning,
Special Issue on European MOOCs. 16 (6), 2038. ISSN 1492-3831. Retrieved from
http://www.irrodl.org/index.php/irrodl/article/view/2153
Selwyn, N. (2014). Digital technology and the contemporary university: Degrees of
digitalisation. London: Routledge.
Siemens, G. (2012). ‘MOOCs are really a platform’, Elearnspace, July 25, 2012. Retrieved
from http://www.elearnspace.org/blog/2012/07/25/moocs-are-really-a-platform/
Schön, S.; Ebner, M.; Rothe, H.; Steinmann, R. & Wenger, F. (2013). Macht mit im Web!
Anreizsysteme zur Unterstützung von Aktivitäten bei Community- und Content-Plattformen.
Güntner, G. & Schaffert, S. (eds.). Salzburg: Salzburg Research.
UNESCO (2015a) Goal 4: Ensure inclusive and quality education for all and promote lifelong
learning. Retrieved from http://www.un.org/sustainabledevelopment/education/
UNESCO (2015b). Education 2030: Incheon Declaration and Framework for Action. Retreived
from http://www.uis.unesco.org/Education/Documents/incheon-framework-for-action-
en.pdf
Weller, M. (2013) MOOCs & Quality. MOOC Quality Project, June 19, 2013. Retrieved from
http://mooc.efquel.org/week-7-moocs-quality-by-martin-weller/
Williams, K., Kear, K., & Rosewell, J. (2012). Quality Assessment for E-learning: a
Benchmarking Approach (2nd ed.). Heerlen, The Netherlands: European Association of
Distance Teaching Universities (EADTU). Retrieved from http://E-
xcellencelabel.eadtu.eu/tools/manual
... An important issue about methods is the standards and quality criteria applied to the development and sustainability of MOOCs [Jansen et al. 2017]. In this study, the quality of MOOCs is evaluated in the context of the use of international quality assurance and measurement frameworks. ...
Preprint
Full-text available
Massive Open Online Courseware (MOOCs) appeared in 2008 and grew considerably in the past decade, now reaching millions of students and professionals all over the world. MOOCs do not replace other educational forms. Instead, they complement them by offering a powerful educational tool that can reach students that, otherwise, would not have access to that information. Nevertheless, designing and implementing a successful MOOC is not straightforward. Simply recording traditional classes is an approach that does not work, since the conditions in which a MOOC student learns are very different from the conventional classroom. In particular, dropout rates in MOOCs are, normally, at least an order of magnitude higher than in conventional courses. In this paper, we analyze data from 7 successful MOOCs that have attracted over 150,000 students in the past years. The analysis led to the proposal of a set of guidelines to help instructors in designing more effective MOOCs. These results contribute to the existing body of knowledge in the field, bring new insights, and pose new questions for future research.
... Our findings highlight the need for educating HR managers about available MOOC offers, as well as about quality control options MOOCs enable. MOOCs are open, traceable, trackable, accessible and evaluated by a massive number of users [56] and as such are more closely monitored than individual training done behind closed doors. In addition, automated quality assurance with learning analytics is making progress [57]. ...
Article
Full-text available
Despite the wide acknowledgement of the knowledge-based economy, the need for life-long learning and quickly growing open online resources, Massive Open Online Courses (MOOCs) are not common means of corporate learning and development programs. The aim of this study is to identify key factors determining the adoption of MOOCs in corporate workplace learning programs. In this exploratory research, the authors employ empirical data from 36 in-depth interviews with corporate managers directly responsible for learning and development practices. Findings provide potential explanations for the mismatch between a generally positive attitude towards MOOCs and their still low adoption rate by identifying expectations towards MOOCs, as well as major reservations. We find that while corporations recognize the opportunities MOOCs can introduce into workplace learning, elevated expectations, negative first-time experiences and objective barriers inhibit MOOC adoption in corporate learning and development programs. It is among the first to expose the perspective of organizations at an early stage of adopting MOOCs. The findings provide a novel contribution to both workplace learning scholarship as well as practical recommendations which can inform HR managers’ decisions in regard to adopting digital means in workplace learning.
... Criteria for the course structure guaranteeing the quality of the course content were partly taken from the OpenupEd suggested distinctive features [9], and some others were motivated by the Checklist for MOOC Accreditation in [10]. These criteria are requiring to clearly specify learning outcomes that can be achieved by the course content. ...
Chapter
Cyber security MOOCs (Massive Open Online Courses) can enable lifelong learning and increase the cyber security competence of experts and citizens. This paper contributes with a review of existing cyber security MOOCs and MOOC quality assurance frameworks. It then presents quality criteria, which we elicited for evaluating whether cyber security MOOCs are worthy to be awarded with a quality seal. Finally, an exemplary evaluation of six selected European MOOCs is presented to exercise the quality seal awarding process. Additionally, the evaluation revealed that criteria for assuring privacy, ethics, meeting professional expectations and openness were on average not clearly met.
... The accreditation issues are much-discussed topics in the ODL literature (Anderson, 2013;Eaton, 2012). Nevertheless, quality assurance and accreditation schemes are not yet equipped for MOOCs (Jansen, Rosewell, & Kear, 2017). Hence, such views suggest that those MOOCs which are accredited can be chosen by the preservice teachers. ...
Article
Full-text available
The main aim of this study is to identify the possible roles of massive open online courses (MOOCs) in faculty-based teacher professional development (TPD). In line with this aim, the MOOC experiences of preservice teachers and teacher trainers were examined. This study uses a qualitative single case holistic design. The participants of the study were 58 preservice teachers and 8 teacher trainers. The data were gathered through the semi-structured interviews in 2018-2019 Fall Semester. Before the interviews, a two-day briefing and orientation about MOOCs were held to inform the participants. After enrolling at least one of these upcoming MOOCs, interviews were carried out in the upcoming two months. The content analysis was used to examine the data collected. According to content analysis, TPACK acquisition and its sub-themes were found to be dominant with regards to participants' views. The preservice teachers mostly enrolled in MOOCs which were related to their "professional field and technical development" domains and mostly suggested TPD MOOC topic by teacher trainers is "effective communication and presentation techniques". It was noticed in the findings that preservice teachers have lack of confidence about the issues related to classroom management. Some negative views of preservice teachers about the MOOCs are concerned with two specific views; language constraints and costs of certain courses. Overall tendency concerning the use of MOOCs for TPD was found to be positive among the participants. However, most of the participants were uninformed or underchallenged about the MOOCs. Results of this study may help to open a new door for integrating MOOCs into teacher training programs.
Article
Full-text available
Nowadays, internet technology plays a vital role in all the fields of our daily lives ranging from the world economy, professional careers, higher education, and almost all the spheres that are deeply impacted. In the current situation, due to COVID19, the dependence on the Internet for almost everything, including learning, getting daily needs, etc., is heavily dependent on the Internet. Online learning is made possible by the Internet, and today most students, educators, researchers are leveraging online learning platforms to enhance their knowledge at their own pace. Generally, the quality of the E-learning courses is evaluated with the help of the courses’ review and rating mechanisms. In the present context, review systems are centralized, storing highly valuable information at one location and are liable to manipulation, hacking, and tampering. In this paper, the Blockchain-based Online Education Content Ranking system is proposed for an online review and ranking system that offers a decentralized trustworthy system, ensuring the integrity of the rating and the independence and integrity of content reviews by Subject Matter Experts (SME).
Conference Paper
Full-text available
Nowadays ICTs have significant roles and variety of applications in teaching. Such innovative technologies are special platforms for online learning. Recently social distancing became mandatory due to Covid-19 pandemic. This had imposed and accelerated changes in education towards digitalization, virtualization and online learning. In the paper the use of Google Classroom in teaching physics for the 3rd grade in the grammar school was described. In addition to that both digital simulations and virtual labs were used. Google Classroom was also applied in teaching of two courses (Electrical Engineering with Electronics, and Sensors and Actuators), to the students of advanced vocational studies in mechanical engineering. Further it was used to disseminate educational materials to students, provide access to them and exchange of various files (messages) between teachers and students. The listed applications have proven to be valuable in communication and classroom activities, while students were quite engaged in lectures and practices. Keywords: physics, electrical engineering, online learning; Google Classroom, simulation. https://www.metropolitan.ac.rs/files/2021/07/eLearning-Proceedings-2020-final.pdf
Article
Full-text available
The dropping out of students in the Massive Open Online Courses (Mooc) has been the subject of debate and concern on the part of educational researchers and practitioners during the last decade. Considering its growth as an educational trend and the emerging research generated on this topic, a systematic review of literature on 131 studies was conducted about attrition in both cMooc and xMoocs. The results highlight the role of collaboration, the sense of community, the need for certification and standardization as the main factors that affect attrition in Moocs.
Article
Full-text available
This paper examined assessment of Open Educational Resources (OER) by students in selected Academic Institutions in Southwest Nigeria. A descriptive research design was used for this study and the instrument used for data collection was the questionnaire. The population of this study comprised two hundred and fifty two respondents from selected academic institutions and a stratified sampling technique was used to select respondents from each of the nine institutions investigated. This study assessed the use of Open Educational Resources by students in nine academic institutions in Nigeria which comprised (Federal, State and Private Universities; Polytechnics; and Colleges of Education) in Nigeria. Findings illustrates that 40.5% of students do not use OER because they are not aware of OER. Male students use more OER than females. Finding indicates that 21.4% of students access OER through multimedia, while 32.5% of students access OER with their laptops because it is portable and convenient to move from one location to another. Analysis shows that Master degree students had 20.6% awareness of OER than other class of students. Result shows that students of government owned institutions are more knowledgeable in OER as University of Lagos have the highest with 15.1% through types of institutions. Analysis from Federal Government owned institutions such as (university, polytechnic and college of education) indicates a high percent of 41.3% which may be due to the fact that Federal Government institutions possess the bests of scholars because years of training and experience on the job, and interaction between lecturers and students from different states, culture and religion in Nigeria.
Article
Full-text available
İşletmeler sürdürülebilir başarı ve gelişmeyi elde etmek için sürekli çaba içindedir. Başarı ve gelişme bir seferlik olamayacak kadar değerlidir ve devamlılığın sağlanması elzemdir. Bunun farkında olan işletmeler çeşitli yönetim sistemlerini kullanmaktadır ve bu sistemler birbirlerinin destekler biçimde iç içe geçmiştir. Bunların en sık kullanılanları süreç yönetimi, kalite yönetimi, stratejik yönetim ve performans yönetimi olarak sayabiliriz. Bu yönetim modelleri birbirine entegre olabilmekte, birbirini desteklemekte ve performansı yükseltmeyi sağlamaktadır. Her işletmenin özellikleri ve kültürü dikkate alınarak organizasyonel sistem kurulduğunda sürdürülebilir başarıda elde edilmektedir. Bu makalede açık ve uzaktan esnek öğrenme ortamlarının yönetimini üslenen kurumların sürdürülebilir başarıyı sağlamaları için temel süreç yapısı önerisi sunulmuştur. Model oluşturulurken işletmelerin başarılarında önemli rol oynayan yönetim sistemleri dikkate alınarak açık ve uzaktan esnek öğrenme yönetimindeki ana süreçler oluşturulmuştur. Abstract: Businesses are constantly striving to achieve sustainable success and development. Success and development are too valuable to be a one-time and continuity is essential. Businesses aware of this use various management systems and these systems are intertwined to support each other. The most frequently used of these are process management, quality management, strategic management and performance management. These management models can be integrated, supporting each other and improving performance. When the organizational system is established considering the characteristics and culture of each enterprise, it is achieved in sustainable success. In this article, the basic process structure proposal is presented for the sustainable success of the institutions that manage the open and distance flexible learning environments. The main processes in open and distance flexible learning management have been formed by taking into consideration the management systems that play an important role in the success of the enterprises while creating the model.
Article
Full-text available
p>This analysis is based upon enrolment and completion data collected for a total of 221 Massive Open Online Courses (MOOCs). It extends previously reported work (Jordan, 2014) with an expanded dataset; the original work is extended to include a multiple regression analysis of factors that affect completion rates and analysis of attrition rates during courses. Completion rates (defined as the percentage of enrolled students who completed the course) vary from 0.7% to 52.1%, with a median value of 12.6%. Since their inception, enrolments on MOOCs have fallen while completion rates have increased. Completion rates vary significantly according to course length (longer courses having lower completion rates), start date (more recent courses having higher percentage completion) and assessment type (courses using auto grading only having higher completion rates). For a sub-sample of courses where rates of active use and assessment submission across the course are available, the first and second weeks appear to be critical in achieving student engagement, after which the proportion of active students and those submitting assessments levels out, with less than 3% difference between them.</p
Conference Paper
Full-text available
Multiple Choice Questions (MCQs) are a core building block of many MOOCs. In this exploratory study we analyze a sample of MCQs from a number of MOOCs and evaluate their quality. We conducted this analysis using a framework informed by a body of empirical research, which describes several common flaws that may occur in the way MCQs are written or phrased. Studies have shown that the presence of these flaws are likely to compromise the reliability and validity of tests containing these MCQs, potentially leading to poorer pedagogical outcomes. Through our study we contribute to the broad debate of whether MOOCs are a force that can enable enhanced and improved pedagogies or whether they will be susceptible to replicating existing poor pedagogies or practises at scale.
Chapter
Full-text available
The pan-European OpenupEd MOOCs initiative combines the ‘classical’ notion of openness attached to open universities with the digital openness now possible through Open Educational Resources. OpenupEd’s mission is to open up education. The chapter’s leading question is whether MOOCs in general are instrumental to open up education. To that end, all barriers to learning should be taken away, and learners should receive incentives towards success. As such, eleven barriers and three incentives are examined in this chapter. In the end, it appears that four barriers can be removed by MOOCs and their providers, for four others this is dependent on their readiness, and for the remaining three it is problematic. The three incentives can be offered through MOOCs but at the cost of substantial effort. The OpenupEd ‘score’ is better or at least equal to what is shown as indicative for the wide variety of MOOC initiatives. This could be expected because of the distinct mission of OpenupEd. Other MOOC providers might also consider to move into such a mission of opening up education and may find valuable ingredients in the chapter for a way forward.
Article
Full-text available
MOOCs (Massive Open Online Courses) are the educational buzzword of 2012. Media frenzy surrounds them and commercial interests have moved in. Sober analysis is overwhelmed by apocalyptic predictions that ignore the history of earlier educational technology fads. The paper describes the short history of MOOCs and sets them in the wider context of the evolution of educational technology and open/distance learning. While the hype about MOOCs presaging a revolution in higher education has focussed on their scale, the real revolution is that universities with scarcity at the heart of their business models are embracing openness. We explore the paradoxes that permeate the MOOCs movement and explode some myths enlisted in its support. The competition inherent in the gadarene rush to offer MOOCs will create a sea change by obliging participating institutions to revisit their missions and focus on teaching quality and students as never before. It could also create a welcome deflationary trend in the costs of higher education. Explanatory Note During my time as a Fellow at the Korea National Open University (KNOU) in September 2012 media and web coverage of Massive Open Online Courses (MOOCs) was intense. Since one of the requirements of the fellowship was a research paper, exploring the phenomenon of MOOCs seemed an appropriate topic. This essay had to be submitted to KNOU on 25 September 2012 but the MOOCs story is still evolving rapidly. I shall continue to follow it. 'What is new is not true, and what is true is not new'. Hans Eysenck on Freudianism This paper is published by JIME following its first release as a paper produced as part of a fellowship at the Korea National Open University (KNOU). Both the original and this republication are available non-exclusively under Creative Commons Attribution (CC-BY). Apart from this note and minor editorial adjustments the paper is unchanged. Normal 0 false false false EN-GB X-NONE X-NONE
Article
Full-text available
p style="margin: 0cm 0cm 12pt 36pt; text-align: justify; line-height: 15pt;"> Much of the literature and the academic discussion about the impact of Massive Open Online Courses (MOOC) in institutional strategic planning has been centred on the US context. However, data shows that although the US are responsible for the largest MOOC platforms and the most successful course provision, it is the European region which accounts for the highest percentage of global MOOC participation. Differently from the US Higher Education system framework, however, in Europe public policy and in particular the European Commission is now driving MOOC institutional uptake. Given the very different institutional, political and cultural contexts, it is interesting to analyse how in these two different regions Higher Education institutions are responding to the challenges of the MOOC phenomena and are integrating it in their own strategic planning. The current research presents the first attempt to conduct a benchmarking study of institutional MOOC strategies in Europe and the US. It's based on a survey launched by the EU-funded project HOME and compares results with a similar survey launched in the US. Results show that are significant differences in how US and European institutions understand the impact of massive forms of open education and also how they perceive the efficiency of digital education and online learning. </p
Research
In 2008, a new term emerged Massive Open Online Courses (MOOCs). Lifelong learners can now use a variety of tools to create and control their own learning groups, and MOOCs will offer chances to investigate such networks. MOOCs are free and open to anyone, with effectively unlimited enrollment. MOOC is a new concept that is generating extensive interest in the education system. MOOC are likely to have a well-built renovation power on the education structure. The enlightening technologies enable education institutions to carry programs and courses all around the world. Although the opening MOOCs originated recently in the USA, MOOCs can also be seen from a broader perception where they are a milestone towards the globalization of education.
Chapter
The European ECO project tests the hypothesis that intercreativity and interculturality in MOOCs (Massive Open Online Courses) are key factors for removing barriers to new teaching–learning processes. These two elements are understudied and undertheorized in most research about MOOCs. The ECO project proposes a new model named sMOOC (social MOOCs), based on constructivist and connectivist pedagogical theories that foster intercreativity and interculturality. Participants from six European countries and their attendant linguistic and cultural zones (Spanish-speaking and French-speaking world mostly) become the protagonists of their own learning, building their knowledge through collaboration and participation in different virtual platforms and social networks. The transnationally produced and shared sMOOC “Step by Step” serves as a learning practice experience which fosters interculturality in conjunction with intercreativity. It is analyzed as a case study to examine the main characteristics of online collaboration, in particular support, engagement, diversity, loss of control and internal policing. The results confirm the presence of such characteristics that are instrumental in the creation of collective intelligence in constant coordination. Additionally, strong cognitive processes are at work: interventional or accidental focus, empathy, tolerance to error, decentring, tolerance to ambiguity, presence to self and identity construction.
Chapter
Massive Open Online Courses (MOOCs) as provision of open and online education have become phenomena in higher education that cannot be dismissed. While MOOCs have orginated in Canada and the United States, the cMOOC and the xMOOC model used there does not fit entirely with the European take on education. This chapter describes an alternative, collaborative approach of MOOC design. This approach is based on a model already tested in practice and has been further elaborated and evaluated in the Elearning, Communication and Open-data (ECO) project. The pedagogical framework is based on the notion that MOOCs should be designed to accommodate the specific context of open online education with its heterogeneity of learner needs. It differs very much from a traditional classroom approach and needs to put the learner center-stage in a social networked learning environment. The characteristics of such a pedagogical framework are described and it is explained how digital inclusion, ubiquitous learning, and gamification can provide affordances for active participation of learners that meet the learners’ needs. Examples are given of implementation of these aspects in ECO MOOCs and initial reports of user evaluations indicate that learners liked this approach.
Book
Digital Technology and the Contemporary University examines the often messy realities of higher education in the ‘digital age’. Drawing on a variety of theoretical and empirical perspectives, the book explores the intimate links between digital technology and wider shifts within contemporary higher education - not least the continued rise of the managerialist ‘bureaucratic’ university. It highlights the ways that these new trends can be challenged, and possibly changed altogether. Addressing a persistent gap in higher education and educational technology research, where digital technology is rarely subject to an appropriately critical approach, Degrees of Digitization offers an alternative reading of the social, political, economic and cultural issues surrounding universities and technology. The book highlights emerging themes that are beginning to be recognised and discussed in academia, but as yet have not been explored thoroughly. Over the course of eight wide-ranging chapters the book addresses issues such as: • The role of digital technology in university reform; • Digital technologies and the organisation of universities; • Digital technology and the working lives of university staff; • Digital technology and the ‘student experience’; • Reimagining the place of digital technology within the contemporary university. This book will be of great interest to all students, academic researchers and writers working in the areas of education studies and/or educational technology, as well as being essential reading for anyone working in the areas of higher education research and digital media research.