Content uploaded by Martin Ebner
Author content
All content in this area was uploaded by Martin Ebner on Sep 22, 2015
Content may be subject to copyright.
PAPER
EVALUATION GRID FOR XMOOCS
Evaluation Grid for xMOOCs
http://dx.doi.org/10.3991/ijet.v10i4.4653
Mohammad Khalil, Hubert Brunner, Martin Ebner
Graz University of Technology, Graz, Austria
Abstract—Massive Open Online Courses, shortly MOOCs,
are a phenomenon nowadays. The number of courses is
worldwide steadily increasing since Sebastian Thrun has
offered a free online course for more than 100.000 students
[25]. Nowadays, decision makers and students as well as
lecturers are asking about the quality of such courses. After
a live experiment on 15 randomly chosen courses and a brief
literature review, we discuss the possibility of finding an
evaluation grid for xMOOCs. The finally suggested criteria
can be used now for future investigations.
Index Terms—Evaluation, MOOC, Online Courses
I. INTRODUCTION
There are several platforms with hundreds of courses
available – so called MOOCs – on the internet educating
students from all over the world. However, courses vary
arbitrarily on quality. Our study seeks to provide the min-
imum requirements that must be taken into account by
decision-makers in order to detect the quality with a spe-
cial eye on didactics of existing xMOOCs as well as the
requisites to develop a new massive online course. The
aim of this research is to gather experience with the didac-
tics of xMOOCs on several levels and to understand the
underlying structures and concepts. We will investigate
the temporal sequence of instructions, the interaction with
the learning materials, the recording of the activities in the
learning units and the interaction with the learning group
in xMOOCs.
A. Defining MOOCs
MOOCs (Massive Open Online Courses) are treated as
high-quality, affordable and integrated courses in every-
day life (higher) education [32]. From a technical point of
view, open and global online courses can be easily attend-
ed with rather little effort by the huge number of students
who are willing to participate. Students only need an in-
ternet connection to start learning. MOOC is the abbrevia-
tion of four letters. These letters stand for: Massive: Re-
fers to large in scale, amount or degree. In which the
number of participants exceeds the so-called Dunbar's
number [31]. The Dunbar number describes the cognitive
limit to the number of social relationships with other peo-
ple [7]. Open: The openness of MOOCs usually refers to
the free access to online courses and learning materials.
Learners can participate in a course without the fulfillment
of other formal requirements or other additional re-
strictions. Thus, learners can access the courses and the
education materials whenever and wherever they like [11].
Online: The management, the information system as well
as the course itself are exclusively online. The communi-
cation between the course participants and the learning
contents takes place via a specially accredited course that
is available online and introduced as a web page [31].
Course: The course can be summarized as a collection of
learning materials that are being introduced by teachers in
a form of a program. These courses have usually a prede-
termined start date and end date [27]. Courses could be
taught by more than one teacher according to the content
itself and the online course provider [31].
B. Types of MOOCs
On the Web, There are a variety of MOOC types avail-
able by different providers. Siemens distinguishes, for
example between cMOOCs, xMOOCs and quasi-MOOCs
[27]. The idea of cMOOCs is basically about knowledge
and knowledge construction by self-organized networks
[31]. cMOOCs are based on phases of an iterative process
"Aggregate, Remix, Repurpose & Feed Forward" [18].
Through this process, the learners in cMOOCs produce
and reflect their content and share their new knowledge
[1]. Moreover, the learning environment is created by the
learners themselves [24]. The "c" in the cMOOCs comes
from the roots of the underlying learning theory of con-
nectivism [26]. In contrast, xMOOC is an online mass
course with a strongly predetermined learning path, com-
munication tools and assignments [31]. The prefix "x"
finds its origin afford by the famous universities such as
Harvard and Stanford and serves as the abbreviation:
“extended” [5]. Online platforms providers started to
distribute additional information, learning resources and
activities to lectures, which made these courses open and
easily accessible by general users [22]. Unlike cMOOCs,
which focus on distributing information on networks,
xMOOCs are based on the traditional instruction-driven
principle. Information is made available via an online
learning platform for a large group of students [15]. The
study by Langer & Thillosen reveals that the main tool for
distributing information in xMOOCs is done by video
sequences. These follow often the model of traditional
lectures. Moreover, xMOOCs offer multiple-choice ques-
tions, asynchronous discussion forums and work with
essays [16]. In order to make the online courses more
encouraging, xMOOCs providers propose badges or cer-
tificates to students who successfully complete courses.
On the other hand, Siemens defined quasi-MOOCs. He
specified it as a loose collection of web-based tutorials or
Open Educational Resources (OER) elements. These have
neither an interaction as in cMOOCs, nor an instruction-
driven curriculum as xMOOCs [27]. There are obvious
common areas such as interaction between different types
of MOOCs. Figure 1 shows a scheme covering intersec-
tion points between the three types of Massive Open
Online Courses: a) xMOOCs, b) cMOOCs and c) the
Quasi-MOOCs.
40
http://www.i-jet.org
PAPER
EVALUATION GRID FOR XMOOCS
Figure 1. Intersection points between the three types of MOOCs
The criteria “asynchronous communications” can be
achieved in cMOOCs according to its definition, which
enhances the learners to share and reflect their learning
content [2]. In accordance with the studies by [24, 31],
cMOOC can have a high score of interaction criteria.
Quasi-MOOCs are courses which are authored by non-
certified authors [27]. Therefore, Quasi-MOOCs lack of
asynchronous communication and interaction.
II. FUNDAMENTAL ELEMENTS OF XMOOCS
In our study, we strongly concentrate on investigating
xMOOCs. Therefore, we considered an appropriate obser-
vation on xMOOCs based on various references and ex-
tracted the following crucial elements for further research
in-details: Curriculum, videos, self-testing units, accom-
panying material, asynchronous communication, assign-
ments, certificates, technical implementation.
A. Curriculum
Most of xMOOCs are offered as multi-week courses.
The typical duration is from 6 to 12 weeks [30]. The cur-
riculum is mainly introduced in weekly intervals [11, 31].
Within a boundary-timing curriculum, the concentration
among participants increases rapidly [30].
B. Videos
The most common way of transmitting information to
students, is through lecturing videos. In addition to videos,
short movies take the part of marketing the courses based
on the quality of the presentation. This can be seen across
different MOOCs providers.
C. Self-testing Units
Fundumental components of xMOOCs are quizzes and
multiple-choice tests [17]. These elements are referred to
as self-test units [11]. Some courses tend to provide fre-
quent quizzes after a predetermined set of information
units. Other courses, offer a full quiz after completing the
whole course [28]. To enhance the social element in
xMOOCs, some courses offer exchanging the quizzes
information and answer them among the discussion fo-
rums [5].
D. Accompanying Material
In addition to the video lectures, MOOC organizers of-
fer supplementary and accompanying material to achieve
voluntary deepening purposes [31]. Accompanying mate-
rial can be formed as simple texts, lecture notes, case
studies or simply hyperlinks that lead to external re-
sources. Kerres & Preußler confirmed that the additional
materials in xMOOC play a critical role and gives indi-
viduals a better support for their learning activities [11].
E. Asynchronous Communciation
Regarding the theory of media synchronicity [6], and
knowledge communication [10], information and factual
knowledge are well communicated through asynchronous
communication [8]. The social structures among MOOCs
providers are usually similar between each other. For
instance, the communication between the learners and/or
the teachers happens in discussion forums [31]. These
discussions are used to clarify questions regarding the
content of a MOOC [12]. Learners feel the positive effect
when they touch the cooperation between the tutor and
them.
F. Assignments
There are different methods to assess performance of
participants in xMOOCs. The learners process tasks week-
ly, which are commonly referred to as assignments [31].
Different types of assessments are available for xMOOCs;
a) Automatic assessment: This is an automated process of
evaluating quizzes provided by such as multiple choice
tests. b) Self-assessments: Here, the students evaluate
themselves and assess each other whether they achieved
the course goals. c) Peer-assessment: Here, students eval-
uate each other in small groups and provide feedback
about their experience [11].
G. Certificates
After achieving the minimum number of points re-
quired passing in a course, students can pay to get certifi-
cates. Certificate is a motivator for many course partici-
pants [20]. Unlike cMOOCs, where the participants are
motivated to extend the collective capabilities of the
course’s network, participants of xMOOCs are eager to
achieve a good score to be able to pursue a badge or a
certificate [14, 22]. Badges can be used as a proof of per-
formance. These were firstly introduced in order to satisfy
the demand for certificates in cMOOCs. With online
badges, student may show his/her achievements publicly
[4].
H. Technical Implementation
There are some requirements that must be met for the
technical implementation of xMOOCs such as; quizzes
functionality, navigation based on the weekly courses
principle, powerful search function in the discussion fo-
rums, availability of social media components, assurance
of videos accessibility on peak hours, as well as the repre-
sentation of the learning progress and the generation of
certificates [19].
III. RESEARCH DESIGN
In this study, the fundamental elements of xMOOCs are
refined. Through our active observations of different
xMOOCs from multiple providers, characteristics of the
basic instructional elements of xMOOCs are determined.
We divided them into subcategories and presented criteria
in order to clarify the main categories which exist in
xMOOCs. In this research study, different types of meth-
odologies are used to reveal and evaluate the upcoming
results. These methods are: qualitative content analysis,
personal observations, document analysis and experts
opinions. In the period of three months, an observation of
15 courses from 12 xMOOCs providers has been carried
out. We applied different types of criteria for examination
purposes.
iJET ‒ Volume 10, Issue 4, 2015
41
PAPER
EVALUATION GRID FOR XMOOCS
A. Document Analysis
Through document analysis, we examine the relevant
contents of xMOOCs [23]. xMOOC’s tutor provides in-
structions to the learners in different ways such as emails
with different instructions, motivational information and
reports before, during and after the course start-off. Dur-
ing courses, assignments and general discussions are ex-
changed using documents, forums, and video’s comments.
This data shed light on the entries made by learners about
the course subject and the learning environment in gen-
eral. Contents of forum threads and the main difficulties
learners post in discussions have been traced. Further-
more, analysis on the forum’s topics and their frequency
has been performed.
B. Observations
As test learners, we observed 15 courses of xMOOCs.
Through a brief looking at the learning contents, participa-
tion in forum discussions as well as resolved assignments
and tasks, all steps and difficulties faced during the obser-
vation were documented. Additionally, the interactions
and discussions between teachers and students in the at-
mosphere of learning environment were assessed. Passive
participation is used in some courses and observations of
similar courses were compared together to enhance the
results [21].
C. Data Collection
In order to collect data, we surveyed courses presented
by different xMOOCs providers on a weekly basis; this is
due to the common way of presenting courses. The study
includes 15 courses from 12 xMOOCs provider address-
ing different topics. Table I shows the studied courses and
their providers.
From our point of view, the courses design is similar
across all xMOOCs providers. Data was collected accord-
ing to curriculum and time constraints. We comprise all
the time limits, deadlines, dates and documented the com-
position of the video lectures, supplementary materials,
and quizzes. As a part of this survey, the interaction be-
tween course participants, course administrators, tutors
and assistants as well as forums activity were documented
and studied.
Being in a role of a student in these courses, we
watched the educational videos, attended courses and
resolved quizzes. Within the personal participation, we
compared between workload and the provided learning
content. We posted in forums and recorded the response
time from students, teachers and teacher’s assistants. Ad-
ditionally, we looked into visualizations and progress in
courses. The time and efforts needed for quizzes, and how
hard they were to solve have been all documented. An
important aspect for attending in xMOOCs is the temporal
components. We distinguished between courses in respect
to the time boundaries and the participation rate.
IV. FINDINGS
During our research studies on xMOOCs, we become
familiar with the fundamental elements of the provided
courses. This study comes up with different criteria to
differentiate between courses. We considered the common
dilemmas of xMOOCs and cMOOCs to classify our crite-
ria into categories and subcategories [29].
A. Categories
Table II lists these subcategories and the criteria we
specified after a careful literature study and observation of
different xMOOCs on different platform.
We categorized xMOOCs into three main parts accord-
ing to literature study and our observations. Some subcat-
egories are missing, in our case; we were not able to
measure some criteria, such as learner’s satisfaction and
the dropout rates [13].
B. Weights
Each criterion was scaled from 2 to 5 according to the –
didactic dimension- model by Baumgartner as follows [3]:
(a) Grade “1”: Very clear. (b) Grade “2”: Clear. (c) Grade
“3”: Sufficient. (d) Grade “4”: Unclear. (e) Grade “5”:
Non-existent. By comparing xMOOC with each other, we
introduce weights for the specified criteria as shown in
table III. The weights were determined in regards to their
importance and relevance. Each weight expresses how im-
TABLE I.
OVERVIEW OF COURSES’ NAMES AND XMOOCS PROVIDERS
Course Provider
Course Name
Moodle
Learn Moodle
Coursera
Learn to Program: The Fundamentals
Coursera
Foundations of Virtual Instruction3
OpenCourseWorld
Learn how to lead
Udacity
Introduction to Computer Science
edX
Introduction to Biology - The Secret of Life
Canvas Network College Foundations: Reading, Writ
ing, and
Math
Canvas Network
Exploring Engineering
openHPI
In-Memory Data Management 2013
NovoEd
Technology Entrepreneurship Part 1
Open2Study
Concepts in Game Development
Standford OpenEdX
SciWrite: Writing in the Sciences
Standford OpenEdX
Solar: Solar Cells, Fuel Cells and Batteries
Waikato University
Data Mining with Weka
University of Amsterdam
Introduction to Communication Science
TABLE II.
CATEGORIES, SUBCATEGORIES AND CRITERIA OF XMOOCS
Category
Subcategory
Criteria
System
General Course Content, Conditions of partic-
ipations, certificates
Information Requirements, target Group, learn
ing
objective, workload
User Interface
Courses clarity, availability and
durability forums searching feature
Interaction
Nature of Infor-
mation
Emails prior to the course, emails
during the course
Interactivity
Interactive elements, forums activi
ty,
replies intensity, course activity,
motivations
Asynchronous
communication
Average response time, teach
ers’
reply and assistance, invitations
Contents
Media elements
Video duration, scripts and docu-
ments, download feature
Evaluation
Self-study plans, self-
assessment,
quizzes level, transparency, assign-
ments level, learning strategy, learn-
ing experience integration
42
http://www.i-jet.org
PAPER
EVALUATION GRID FOR XMOOCS
portant and relevant one criterion is in a successful course.
Crucial elements were assigned with a very high weight;
medium weight was assigned for criteria that support the
fundamental elements. Low weights were assigned to
useful extensions criteria. During our observations, and
the literature review study, we categorized knock-out
criteria from the set of previous described subcategories in
the next section. In table III, Weights were assigned to
each subcategory criteria. The sum of weights per catego-
ry is 100 points. Weights range is between 3 to 16, the
higher its weight, the more important the criterion affects.
Knockout criteria were the highest in weights.
C. Knockout Criteria
Figure 2 shows the knockout criteria, which were
defined due to their crucial role in Table III.
Figure 2. Overview of the Knockout Criteria of xMOOCs
The definition of the target group is essential for each
xMOOC. Each xMOOC has to specify the target students
who will take apart [31]. A clear learning objective facili-
tates the right selection of the large xMOOC offer [5]. A
good planning of the workload and the required efforts
such as “This course needs 5 hours/week” is an important
issue. Courses with high workload have a higher dropout
[9]. Course clarity was added according to [19]. Assis-
tance to students who need help has been variously dis-
cussed as a crucial criterion in xMOOCs in [5, 16, 29].
Quick feedback in the forums is an important factor for all
participants [31]. All xMOOCs providers have to make
teaching materials such as scripts and documents accessi-
ble any time [11]. Finally, the ability to self-study and to
plan the learning activities is essential according to [22].
The knockout criteria are assigned a high priority when
grading each course of the xMOOCs in our experiment.
D. Final Results
Figure 3 shows the evaluation of the 15 courses from 12
different xMOOCs providers. The figure depicts a 3D
graph of the courses containing each category and the
final evaluation. The X-axis shows course names while
the Y-axis shows our grading evaluation. The grading
evaluation is the accumulative summation of points from
each subcategory criteria. The following formula is used
to get the percentage rate of each subcategory criteria:
Criterion Point = Criterion Weight – ((Grade - 1) *
25% * Criterion Weight)
All tested courses share high level system features such
as: offering certificates, defining the target group and
clarifying the courses’ content. We found four courses that
have a clear declivity within the interaction category;
“OpenCourseWorld: Learn how to lead” is one of the
examples, where no informative emails were sent to stu-
dents and a poor tutoring assistant. Additionally, it is no-
table that the Content category revealed an average high
score in all courses.
TABLE III.
CATEGORIES, SUBCATEGORIES AND CRITERIA OF XMOOCS
Criteria
Activity
Weight
System
Criteria
Learning Objective, Workload, Target Group
15
Course clarity
14
Forums searching
11
Requirements
10
Course content
8
Certificates
5
Availability & Durability 4
Conditions of participation 3
Interaction
Criteria
Average response time 16
Teacher assistance
14
Forums activity, Invitations 13
Emails during the Course
12
Replies intensity
9
Motivation, Interactive elements
7
Emails prior to course
5
Course activity
4
Content
Criteria
Self-study plan, Scripts & Docs
14
Transparency
13
Self-Assesment, Assignments level, Quizzes
level
10
Average video duration
8
Download feature
7
Learning strategy
6
Figure 3. Categories Evaluation of the Studied Courses from xMOOCs
E. The Evaluation Grid
As the result of this research study, we present an eval-
uation grid that combines all the criteria of all categories.
Table IV shows the grades for all criteria that have been
used for all the courses of xMOOCs in this experiment.
The table reveals the criteria of the main three categories:
System, Interaction and Contents. By using the previous
formula, a user can calculate each subcategory’s accumu-
lative points and therefore evaluate the courses. This eval-
uation grid supports decision makers to evaluate xMOOCs
and enable them to compare different courses.
!"
#!"
$!"
%!"
&!"
'!!"
())*+,-./"
0)123,24-.#5"
0)123,24-678"
9:,;0)123,-.<#."
=*4>?@A-8#0B"
CDE-8#F"
04;G43-06"
04;G43-CC"
9:,;<58-8(D("
H)G)C*-IC5"
9:,;#B@1*A-0JD"
9:,;C*E-BK"
9:,;C*L-BB"
K4?M4@)-D(K"
=/-80B"
BA3@,N" 8;@,24>O);" 0);@,;@3"
iJET ‒ Volume 10, Issue 4, 2015
43
PAPER
EVALUATION GRID FOR XMOOCS
TABLE IV.
EVALUATION GRID FOR ALL CRITERIA OF THE STUDIES COURSES OF XMOOCS
Criteria
MD_Learn
CO_FUPRO
CO_FOVIR
OC_LLEAD
UD_INIBI
Edx_INTBI
CN_COLRD
CN_EXENG
OH_DATMG
NE_TECENT
O2_CONGD
OX_SCIWR
OX_SOLCE
WA_DAMIN
UA_INCSI
Course content
1
1
2
1
2
1
1
1
1
1
1
1
1
1
1
Condition of participation
1
1
1
2
1
1
1
1
1
1
1
1
1
1
1
Certificates
3
5
2
2
3
2
2
4
2
5
2
2
5
5
2
Requirements
3
1
2
1
1
2
1
1
2
1
2
2
1
2
2
Target group 2 2 1 3 1 2 1 1 4 1 2 2 4 3 3
Learning objective
1
1
1
2
2
1
1
2
3
2
1
2
3
1
2
Workload 3 2 5 1 3 2 3 3 5 2 2 2 5 5 1
Courses clarity
3
1
2
2
1
1
2
1
1
3
2
2
2
2
2
Availability & Durability 2 1 2 4 2 1 2 1 2 1 2 2 2 2 2
Forums search feature
4
1
1
5
2
2
1
1
1
1
3
3
3
1
4
Emails prior to course
1
2
3
4
4
1
4
2
4
3
2
3
4
4
4
Email during course
1
1
2
5
4
4
5
1
2
3
3
1
4
2
4
Interactive elements
1
1
5
5
1
1
3
1
5
2
5
3
3
5
5
Forum activity
1
1
1
5
4
1
3
1
1
1
3
3
3
1
3
Teachers assistance
1
1
2
5
2
1
2
1
1
2
1
3
3
1
1
Average response time 1 1 1 5 3 1 3 2 3 1 3 2 2 2 2
Replies intensity
1
1
2
5
3
1
3
2
1
2
3
3
3
1
1
Course activity 1 1 1 2 3 3 2 1 2 1 2 2 2 1 2
Invitations
1
2
5
2
2
2
5
2
5
5
5
5
5
3
2
Motivation 2 1 2 3 4 2 3 2 2 2 3 3 3 2 2
Video duration 1 2 3 2 1 4 1 1 3 3 1 2 1 2 1
Documents and scripts
4
2
2
1
1
1
3
3
1
1
1
3
3
1
1
Download feature
2
1
5
2
2
1
2
2
2
2
2
1
1
1
2
Self-assessment
3
3
3
3
3
3
2
3
5
5
5
2
3
3
5
Self-study plan
4
1
1
1
1
1
1
1
1
3
1
1
1
1
3
Quizzes level
3
1
1
1
3
1
1
3
1
1
2
1
1
3
3
Assignments level
5
1
1
5
1
2
1
2
1
1
2
1
1
3
3
Learning strategy
3
3
2
2
2
1
1
1
1
1
2
2
1
2
2
Transparency 5 1 1 3 5 2 2 3 1 3 3 3 3 5 3
Learning exp. integration
1
2
1
2
3
1
2
1
2
2
2
2
2
1
1
V. CONCLUSION
Massive Open Online Courses, shortly MOOCs, are a
phenomenon in online education these days. Different
types of MOOCs are currently available. Furthermore,
there are many MOOCs providers who are offering cours-
es to the public. Nevertheless, these courses vary in quali-
ty and the way they are offered. Our research study intro-
duces a detailed definition of xMOOCs. We distinguished
xMOOCs with cMOOCs and Quasi-MOOCs based on
proposed criteria. Within a deep survey and an experiment
containing of 15 courses from 12 xMOOCs providers and
a deep browsing of literature study of xMOOCs, we listed
some fundamental elements and knockout criteria that
should exist in any online course of xMOOCs platform.
We categorized these elements and classified them into
subcategories. The studies courses were evaluated accord-
ing to the specified criteria and our categorization of these
elements. Certainly, he outcome list is not immutable, but
it can be seen as a proposed structure to adapt when estab-
lishing a course by any xMOOC provider. This paper
makes a significant contribution to the MOOCs research
field because: a) it provides a new grounded categoriza-
tion of the evaluation criteria of MOOCs. b) Presents an
evaluation grid than can be used to evaluate online courses
of xMOOCs. Our future intent is to revolutionize the re-
sults and the experiment into an automated evaluation grid
that can be used whenever xMOOCs are intended to be
assessed.
REFERENCES
[1] Ahn, J., Butler, B. S., Alam, A., & Webster, S. A. (2013). Learner
participation and engagement in open online courses: Insights
from the Peer 2 Peer University. MERLOT Journal of Online
Learning and Teaching, 9(2), 160-171.
[2] Baker, Thomas Jerome (2012). Connectivism and Connected
Knowledge: Participating in a MOOC.
http://www.amazon.co.uk/Connectivism-Connected-Knowledge-
Participating-
ebook/dp/B0088DQMUS/ref=sr_1_2?ie=UTF8&qid=1369412151
&sr=8-2&keywords=connectivism
44
http://www.i-jet.org
PAPER
EVALUATION GRID FOR XMOOCS
[3] Baumgartner, P. (2009). Developing a Taxonomy for Electronic
Portfolios. In The Potential of E-Portfolios in Higher Education
(S. 13-44). Innsbruck, Wien, Bozen: StudienVerlag.
[4] Baraniuk, R. (2012). Open education: One perfect storm yields
three revolutions. Visiones de Telefónica.
[5] Bremer, Claudia, & Anne Thillosen (2013). Der deutschsprachige
Open Online Course OPCO12. In D. Krämer (Hrsg.), E-Learning
zwischen Vision und Alltag, 64 Medien in der Wissenschaft (15–
27). Münster / New York / München / Berlin: Waxmann.
http://www.waxmann.com/?eID=texte&pdf=2953Volltext.pdf&ty
p=zusatztext
[6] Dennis, A. R., & Valacich, J. S. (1999, January). Rethinking
media richness: Towards a theory of media synchronicity. In Sys-
tems Sciences, 1999. HICSS-32. Proceedings of the 32nd Annual
Hawaii International Conference on (pp. 10-pp). IEEE.
[7] Dunbar, Robin (2010). How Many Friends Does One Person
Need?: Dunbar’s Number and Other Evolutionary Quirks. Faber &
Faber.
[8] Eppler, M. J. (2006). The Concept of Knowledge Communication
and Its Relevance to Management. Available on
http://www.knowledge-communication.org/pdf/research-note-
knowledge-communication.pdf
[9] Halawa, S., Greene, D., & Mitchell, J. (2014). Dropout prediction
in MOOCs using learner activity features. Proceedings of the Eu-
ropean MOOC Summit. Lausanne, Switzerland.
[10] Kastberg, Peter (2012). Knowledge Communication Theory
Revisited–from ‘communicatio’to ‘communis esse’: paper at the
IFSA conference.
[11] Kerres, Michael, & Anabell Preußler (2013). Zum didaktischen
Potenzial der Vorlesung: Auslaufmodell oder Zukunftsformat? In
G. Reinmann, S. Schön, & M. Ebner (Hrsg.), Hochschuldidaktik
im Zeichen der Heterogenität und Vielfalt (79–97). Norderstedt.
http://www.bimsev.de/n/userfiles/downloads/festschrift.pdf
[12] Khalil, Hanan, & Martin Ebner (2013a). Interaction Possibilities in
MOOCs – How Do They Actually Happen? (1–24). International
Conference on Higher Education Development, Egypt.
http://de.scribd.com/doc/134249470/Interaction-Possibilities-in-
MOOCs-%E2%80%93-How-Do-They-Actually-Happen
[13] Khalil, Hanan, & Martin Ebner (2013b). “How satisfied are you
with your MOOC?”-A Research Study on Interaction in Huge
Online Courses. World Conference on Educational Multimedia,
Hypermedia and Telecommunications (Bd. 2013, 830–839).
http://www.editlib.org/p/112057
[14] Khalil, H. & Ebner, M. (2014). MOOCs Completion Rates and
Possible Methods to Improve Retention - A Literature Review. In
Proceedings of World Conference on Educational Multimedia,
Hypermedia and Telecommunications 2014 (pp. 1236-1244).
Chesapeake, VA: AACE
[15] Lackner, E., Kopp, M., Ebner, M. (2014) How to MOOC? – A
pedagogical guideline for practitioners. Roceanu, I. (ed.). Proceed-
ings of the 10th International Scientific Conference "eLearning
and Software for Education" Bucharest, April 24 - 25, 2014. Pub-
lisher: Editura Universitatii Nationale de Aparare "Carol I”
[16] Langer, Volkmar & Anne Thillosen (2013). Freie Online-
Angebote für Selbstlernende - Lebenslanges Lernen mit dem In-
ternet. Lehrbuch für Lernen und Lehren mit Technologien, 0(0)
[17] Lipson, Kay (2013). Dealing with megaclasses in an online envi-
ronment. Available on http://www.statistics.gov.hk/wsc/IPS040-
P3-S.pdf
[18] Mackness, J., Waite, M., Roberts, G., & Lovegrove, E. (2013).
Learning in a small, task–oriented, connectivist MOOC: Pedagog-
ical issues and implications for higher education. The International
Review of Research in Open and Distance Learning, 14(4).
[19] Meinel, Christoph, & Christian Willems (2013). Das MOOC-
Angebot des Hasso-Plattner-Instituts. Technische Berichte des
Hasso - Plattner - Instituts für Softwaresystemtechnik an der Uni-
versität Potsdam, 34
[20] Nesterko, S. O., Dotsenko, S., Han, Q., Seaton, D., Reich, J.,
Chuang, I., & Ho, A. D. (2013). Evaluating the geographic data in
moocs. In Neural Information Processing Systems.
[21] Nonnecke, Blair, & Jenny Preece (2001). Why lurkers lurk. Amer-
icas Conference on Information Systems (1–10).
http://bacsy.wirtschaft.fhnw.ch/iwi/publications.nsf/ae2a39a43cc0
9951c12572180036eb5b/5daca7644c019593c125722e002922b5/$
FILE/Gg004.pdf
[22] O’Toole, Robert (2013). Pedagogical strategies and technologies
for peer assessment in Massively Open Online Courses (MOOCs).
http://wrap.warwick.ac.uk/54602/
[23] Punch, K. F. (1998). Introduction to social research: Quantitative
and qualitative approaches. London: Sage Ltd.
[24] Robes, Jochen (2012). Offenes und selbstorganisiertes Lernen im
Netz. Ein Erfahrungsbericht über den OpenCourse 2011 „Zukunft
des Lernens“. In E. Blaschitz, Brandhofer, Nosko, & Schwed
(Hrsg.), Zukunft des Lernens. Wie digitale Medien Schule, Aus-
und Weiterbildung verändern (219–244). Glückstadt: Werner
Hülsbusch.
[25] Salmon, F. (2012). Felix Salmon on Sebastian Thrun and Udacity.
Retrieved from http://delong.typepad.com/sdj/2012/01/felix-
salmon-on-sebastian-thrun-and-udacity.html
[26] Siemens, George (2006). Connectivism: Learning Theory or
Pastime of the Self-Amused?
http://www.elearnspace.org/Articles/Connectivism_response.doc
[27] Siemens, George (2012). Massive Open Online Courses: Innova-
tion in Education? Open Educational Resources: Innovation, Re-
search and Practice, 2012.
[28] Schulmeister, Rolf (2012). As Undercover Students in MOOCs -
Lecture2Go Videoportal. Universität Hamburg.
http://lecture2go.uni-hamburg.de/konferenzen/-/k/14447
[29] Schulmeister, Rolf (2013). MOOCs - Massive Open Online Cour-
ses: Offene Bildung oder Geschäftsmodell. Münster / New York /
München / Berlin: Waxmann.
http://www.waxmann.com/fileadmin/media/zusatztexte/2960Vollt
ext.pdf
[30] Sharples, Mike u. a. (2012). Innovating Pedagogy 2012. Innova-
tion Report. Open University.
http://www.open.ac.uk/blogs/innovating/
[31] Wedekind, Joachim (2013). MOOCs - eine Herausforderung für
die Hochschulen? In G. Reinmann, S. Schön, & M. Ebner (Hrsg.),
Hochschuldidaktik im Zeichen der Heterogenität und Vielfalt (45–
69). Norderstedt.
http://www.bimsev.de/n/userfiles/downloads/festschrift.pdf
[32] Yuan, L., & Powell, S. (2013). MOOCs and open education:
Implications for higher education. Retrieved from
http://publications.cetis.ac.uk
AUTHORS
Mohammad Khalil, Hubert Brunner, and Martin
Ebner are with Graz University of Technology, Graz,
Austria.
Submitted 24 April 2015. Published as resubmitted by the authors 20
August 2015.
iJET ‒ Volume 10, Issue 4, 2015
45
