Conference PaperPDF Available

Portraying MOOCs Learners: a Clustering Experience Using Learning Analytics

Authors:

Abstract and Figures

Massive Open Online Courses are remote courses that excel in their students' heterogeneity and quantity. Due to the peculiarity of being massiveness, the large datasets generated by MOOCs platforms require advance tools to reveal hidden patterns for enhancing learning and educational environments. This paper offers an interesting study on using one of these tools, clustering, to portray learners' engagement in MOOCs. The research study analyse a university mandatory MOOC, and also opened to the public, in order to classify students into appropriate profiles based on their engagement. We compared the clustering results across MOOC variables and finally, we evaluated our results with an eighties students' motivation scheme to examine the contrast between classical classes and MOOCs classes. Our research pointed out that MOOC participants are strongly following the Cryer's scheme of Elton (1996).
Content may be subject to copyright.
265
Portraying MOOCs Learners: a Clustering
Experience Using Learning Analytics
Mohammad KHALIL, Christian KASTL & Martin EBNER
Graz University of Technology, Educational Technology,
{mohammad.khalil, martin.ebner}@tugraz.at, kastl@sbox.tugraz.at
Abstract
Massive Open Online Courses are remote courses that excel in their students’
heterogeneity and quantity. Due to the peculiarity of being massiveness, the large
datasets generated by MOOCs platforms require advance tools to reveal hidden
patterns for enhancing learning and educational environments. This paper offers
an interesting study on using one of these tools, clustering, to portray learners’
engagement in MOOCs. The research study analyse a university mandatory
MOOC, and also opened to the public, in order to classify students into appropriate
profiles based on their engagement. We compared the clustering results across
MOOC variables and finally, we evaluated our results with an eighties students’
motivation scheme to examine the contrast between classical classes and MOOCs
classes. Our research pointed out that MOOC participants are strongly following
the Cryer’s scheme of ELTON (1996).
Keywords
MOOCs, Learning Analytics, Clustering, Engagements, Patterns
Research Track
Proceedings of the European MOOC Stakeholder Summit 2016
266
1 Introduction
In the last years, Technology Enhanced Learning (TEL) has been developed rapidly so
that now is including modern online classes in which they are called MOOCs
(MCAULEY et al., 2010). The word MOOCs is an abbreviation of four letters, ‘M’
which is Massive, and it means massive in the number of enrollees than what is in
regular classes. ‘O’ and this is Open, and that is an implication of a field that has no
accessibility limitations. Furthermore, openness also means that these massive courses
should be open to anyone. The second ‘O’ stands for Online where all courses are held
on the Internet without any borders. In the end, ‘C’ means courses, this represented a
structured learning material and is mostly embodied as filmed lectures, documents and
interactive social media such as discussion forums or even social media channels.
The first version of MOOCs was named cMOOCs, which were developed by George
Siemens and Stephan Downes back in 2008, and it adopted the connectivism theory
that is based on the role of social and networks of information (HOLLAND &
TIRTHALI, 2014). After that, other versions of MOOCs become available, but it was
noticeable that the extended MOOCs or so-called xMOOCs attracted the eyes of to-
day’s online courses learners.
One of the prominent and most successful activities of xMOOCs has been done by
Sebastian Thrun in 2011. He and his colleagues launched an online course called “In-
troduction to Artificial Intelligence” which attracted over 160,000 users from all over
the world (YUAN et al., 2013). xMOOCs follow theories that are based on guided
learning and the classical information transmission (RODRIGUEZ, 2012). FERGU-
SON & CLOW (2015) argued that xMOOC is an extended version of cMOOC with
additional elements of content and assessment as well as a larger-scale role of educa-
tors to be part of the content; in other words, an online course for hundreds of learners
simultaneously (CARSON & SCHMIDT, 2012).
The benefits of MOOCs are crystallized to be welfare in improving educational out-
comes, extending accessibility, and reducing costs. In addition, Ebner and his col-
leagues addressed the advantages that MOOCs can add to the Open Educational Re-
sources (OER) movement as well as lifelong learning experiences in TEL contexts
(Ebner, et al., 2014). Despite their advantages, MOOCs suffered from students who
Portraying MOOCs Learners: a Clustering Experience Using Learning Analytics
Mohammad Khalil, Christian Kastl & Martin Ebner
267
register and afterwards do not complete the courses. This has been cited in several
scientific researches and is now commonly named as “the dropout rate” (MEYER,
2012; JORDAN, 2013). Various investigations have been done to identify the reasons
behind the low completion rates, such as the research studies by KHALIL & EBNER
(2014; 2016), LACKNER et al. (2015). Furthermore, lack of interaction between learn-
ers and instructor(s), and the controversy argument about MOOCs pedagogical ap-
proach, are the negative factors that obstruct the positive advancement of MOOCs. In
addition to all this, recent research publications discussed the patterns of engagement
and the debates about categorizing students in MOOCs (KIZILCEC et al., 2013; FER-
GUSON & CLOW, 2015; KHALIL & EBNER, 2015a).
Since MOOCs include a large quantity of data that is generated by students who reside
in an online crucible, the heed toward what is so-called Learning Analytics steered the
wheel into an integration of both sectors (KHALIL & EBNER, 2016). KNOX (2014)
discussed the high promises behind Learning Analytics when it is applied to MOOCs
datasets for the principles of overcoming their constraints. The needs for Learning
Analytics emerged to optimize learning, and for a better students’ commitment in dis-
tance education applications (KHALIL & EBNER, 2015b).
In this research study, we employ Learning Analytics, using a clustering methodology,
on a dataset from one of the courses offered by the leading Austrian MOOC platform,
iMooX
1
. The sought objectives behind clustering are to portray the engagement and
behaviour of learners in MOOC platforms and to support decisions of following up the
students for purposes of increasing retention and improving interventions for a specific
subpopulation of students. In addition, this research study will contribute with an addi-
tional value to ease the grouping of MOOCs participants.
The publication is organized as follow: Section 2 covers the research methodology of
this research study. Section 3 gives an overview about the MOOC platform itself as
well as the demographics of the course. Section 4 covers in details the clustering meth-
odology and data analysis. Section 5 is the discussion and the comparison with the
Cryer’s scheme, while section 6 concludes the findings.
1
http://www.imoox.at (last visited October 2015)
Research Track
Proceedings of the European MOOC Stakeholder Summit 2016
268
2 Research Methodology
This research study is based on data collected by a formal Learning Analytics applica-
tion of the iMooX MOOC-platform. By tracking their traces, the application records
learners actions within the divergent MOOC indicators such as videos, files downloads,
reading in forums, posting in forums and the quizzes performance. In the present study,
a MOOC named “Social Aspects of Information Technology”, shortly GADI (abbrevi-
ated from the original German title), was chosen for further analysis and research.
The collected information after that, which takes the form of log files, was parsed to
filter the duplicated and unstructured data format. The data analysis was carried out
using the R software, and the clustering methodology was performed using an addi-
tional package called NbClust (CHARRAD et al., 2014). We followed content analysis
in which units of analysis get measured and benchmarked based on qualitative deci-
sions (NEUENDORF, 2002). These decisions are founded on sustained observations
on a weekly basis and examination of surveys at the end of the course by one of the
researchers.
3 Stats and Overview
3.1 The MOOC-Platform
iMooX is the leading Austrian MOOC platform founded by the cooperation of Graz
University of Technology and University of Graz (NEUBÖCK et al., 2015). The of-
fered courses vary in topics between social science, engineering and technology topics
and cope with lifelong learning and OER tracks. The target groups are assorted among
school children, high-school students and university degree holders. Additionally,
iMooX offers certificates and badges to successful students who fulfilled courses re-
quirements at no cost.
Portraying MOOCs Learners: a Clustering Experience Using Learning Analytics
Mohammad Khalil, Christian Kastl & Martin Ebner
269
3.2 Course Overview and Demographics
Our analysis of portraying learners is based on a summer course provided by Graz
University of Technology in 2015 called “Social Aspects of Information Technology”
abbreviated in German and in this research study as GADI. This course was selected
because it is specialized of being mandatory for the university students of Information
and Computer Engineering (Bachelor-6th semester), Computer Science (Bachelor-2nd
Semester), Software Development, Business Management (Bachelor-6th semester) and
for the Teacher’s Training Certificate of computer science degree (2nd Semester). Fur-
thermore, the course was also opened for external participants and not only restricted to
university students. The main content of the course is based on discussions about the
implications of information technology on society.
The course lasted 10 weeks. Every week includes 2 or 3 video lectures, a discussion
forum, further readings and a multiple choice quiz. Each quiz could be repeated up to
five times. The system is programmed to record the highest grade of these trials.
MOOC’s workload was predefined with about 3 hours/week, and the passing grade for
each quiz was set to be 75%. Students of Graz University of Technology gain 2.5
ECTS (credits) for completing the MOOC but they have also to do an additional essen-
tial practical work.
Finally, there were in summary 838 participants in the course, 459 of them were uni-
versity students, while 379 were voluntary external participants. Because this MOOC is
obligatory to pass the university class, the completion ratio was much higher compared
to other MOOCs. The general certification rate of this particular MOOC is 49%. The
certification ratio of the university students was 80%, and 11.35% of the external par-
ticipants.
Candidates, who successfully completed all quizzes, were asked to submit answers for
a predefined evaluation form. The collected data showed that most of the external par-
ticipants are from Austria and Germany. University students’ average age was 23.1
years old, while the average age of the external participants was 46.9 years old. Table 1
reports the course demographics based on the evaluation results.
Research Track
Proceedings of the European MOOC Stakeholder Summit 2016
270
Table 1: The GADI MOOC Demographics of completed participants
Gender (M/F)
High School
MSc & PhD
Others
Students
327/40
357
4
2
Others
23/20
13
3
4
4 Clustering and Analysis
The main goal behind clustering is to assign each participant in the MOOC to a suitable
group with common behaviours. Each group should be as distinct as possible to pre-
vent overlaps. The elements in these groups should fit tied to the defined group param-
eters. Therefore, clustering using the k-mean algorithm with the Euclidean distance
was selected as our tool of choice. In order to begin clustering, we labeled the variables
that will be referenced in the algorithm. The expected results should be clustered with
activities and characteristics that distinguish the MOOC participants.
Due to the relations between certain variables, we excluded the high correlated indica-
tors as this will not affect the grouping sequence. As a consequence, the used variables
in clustering were:
1. Reading Frequency: This indicates the number of times a user clicked on par-
ticular posts in the forum.
2. Writing Frequency: This variable determines the number of written posts in
the discussion forum.
3. Videos Watched: This variable contains the total number of videos a user
clicked.
4. Quiz Attempts: It calculates the sum of attempts that have been spent on all ten
quizzes.
Because of the structure of the examined MOOC, which is obligatory for university
students and opened for external participants, the clustering was done independently in
both groups. The intention of each group could vary. For example, are the university
Portraying MOOCs Learners: a Clustering Experience Using Learning Analytics
Mohammad Khalil, Christian Kastl & Martin Ebner
271
students attending the MOOC for learning purposes or are they only seeking for the
grade?
4.1 Case 1: University Students
In this case, the k value was assigned with a value from 3 to 6, as long as we do not
really want more than 6 groups. The suggested cluster, based on the variables value
and the NbClust package, resulted to four clusters. Figure 1 illustrates the four clusters
of the MOOC university students.
Figure 1: MOOC’s University Students Clusters
Figure 1 shows a cluster amount of four classes. Two of the groups, the blue and the
green are overlapping. The relation between components in x-axis and y-axis is valued
Research Track
Proceedings of the European MOOC Stakeholder Summit 2016
272
at 67.76%. This percentage means that we have nearly 70% of unhidden information
based on this clustering value
2
. The clusters are characterized as the following:
Cluster (1) with the pink oval shape contains 95 students. This group has low activity
among the four variables. Only 10 students are certified, and the dropout rate is high.
Cluster (2) with the blue oval shape contains 154 students. Most of the participants in
this group completed the course successfully. This cluster is distinguishable by their
videos’ watching.
Cluster (3) with the green oval shape has 206 participants. The certification rate was
94%. Both of cluster 2 and cluster 3 share a high certification rate, but differ in watch-
ing the videos.
Cluster (4) is the smallest cluster, containing 4 students. By observing the variables, we
noticed that the students in this cluster are the only ones that had been writing on the
forums. The amount of certified students in cluster 4 totals to 50%.
4.2 Case 2: External Learners
Figure 2 shows the proposed cluster solution of the external participants who do not
belong to the university class. Again, k value was set to be from 3 to 6. The point vari-
ability shows a competitive rate of 88.89%, which indicates a steep seclusion among
the three groups. The clusters of this case are characterized as the following:
Cluster (1) with the blue oval shape contains 42 participants. The certification rate of
this group is 76.20%. The social activity and specifically reading in forums are moder-
ate compared to the other clusters. Whiles the number of quiz trials is high.
Cluster (2) with the red oval shape holds only 8 participants. The certification rate in
this group is 100%. Participants from cluster 2 showed the highest number of written
contributions and the highest reading frequency in the forum.
2
Explanation: http://stats.stackexchange.com/questions/141280/understanding-cluster-plot-
and-component-variability (Last accessed, 15th October 2015).
Portraying MOOCs Learners: a Clustering Experience Using Learning Analytics
Mohammad Khalil, Christian Kastl & Martin Ebner
273
Figure: 2 MOOC’s External Learners Clusters
Cluster (3) with the pink oval shape includes all the other participants. This group
showed a high dropout rate and a completion rate of only 1%.
5 Discussion
Within the previous clustering results in both cases, we studied the values of each vari-
able in each cluster. The next step was to make a classification scale of “low”, “moder-
ate” and “high” that describes characteristics and the activity level of each group. Table
2 shows them for both of the cases, university and external participants.
Research Track
Proceedings of the European MOOC Stakeholder Summit 2016
274
Table 2: Characteristics of each cluster of both MOOC cases
Case: University Students
Reading
Freq.
Writing
Freq.
Watching
Videos
Quiz
Attempts
Certification
Ratio
Cluster 1
Low
Low
Low
Low
10.53%
Cluster 2
High
Low
High
High
96.10%
Cluster 3
Moderate
Low
Low
High
94.36%
Cluster 4
High
High
Low
Moderate
50%
Case: External Participants
Cluster 1
Moderate
Low
Moderate
High
76.19%
Cluster 2
High
High
High
High
100%
Cluster 3
Low
Low
Low
Low
1%
By analyzing the clusters, we think the opportunity to portray students’ behaviours in
the MOOC becomes possible nearby. However, a study by ELTON in (1996), which
examined the general strategies to motivate learners in the classes, meets a similar
scheme of our clustering results. Figure 3 illustrates the so-called Cryer’s scheme,
which shows student behavior within a course. The x-axis represents intrinsic factors,
which are achievements and subject. The y-axis includes the examination preparation,
which is named as the extrinsic factor. It must be stated that this scheme does not only
include the shown specific profiles, but it also contains other learners who reside be-
tween these four profiles.
The students, on the bottom left of the Cryer’s scheme, describe the ones who are not
interested in the course subject nor score positive results.
Portraying MOOCs Learners: a Clustering Experience Using Learning Analytics
Mohammad Khalil, Christian Kastl & Martin Ebner
275
Figure 3: Cryer’s Scheme Based on Levels of Student Commitment
This class represents Cluster (1) of our university students’ case, and Cluster (3) of
external participants’ case. An appropriate profile name of this cluster would be simply
“Dropout”. This profile shares common patterns of being inactive among all the
MOOC variables. The certification rate in this profile is low.
The class, on the top left in the scheme, describes learners who play the system. This
term comes from a case when students are treated and just doing what instructors want
to do for getting a grade. Using Learning Analytics, some students were determined
watching the learning videos with various skips, or even they start a quiz without
watching the weekly video. Such students were named as “Gamblers”. In spite of cer-
tain questions that are hard to answer without watching a video, some of them could
pass the exams. It should also be considered that the MOOC platform offers up to five
trials per week quiz, which might be the reasons behind a high percentage of gamblers
among university students.
Rebellions are those who show interest in the course, but fail because of bad exam
preparations. In the Cluster Analysis, this group was available in the university stu-
dents’ group, which is represented by Cluster (4). However, it was hard to detect in the
Research Track
Proceedings of the European MOOC Stakeholder Summit 2016
276
external participants’ group. Cluster (4) was distinct for being very active with the
social activities in the forums. We named them as the “Sociable Students”.
The last class is the students whom their commitment is high. “Perfect Students” might
be the appropriate name for them. Every MOOC platform looks to have such students.
With their high certification rate, Cluster (2) in both cases embodied this profile.
6 Conclusion
This research study examined learners’ behavior in a mandatory xMOOC offered by
iMooX. Because the course was also opened to the public, we studied patterns of the
involved students and separated them into two cases, internal and external participants.
Within our research study, we performed a cluster analysis, which pointed out partici-
pants in MOOCs, whether they did the course on a voluntary basis or not. Furthermore,
we found that the clusters can be applied on the Cryer’s scheme of ELTON (1996).
This leads to the assumption that tomorrow’s instructors have to think about the in-
crease of the intrinsic motivation by those students who are only “playing the system”.
Our research study also pointed out that online courses behave very similar to tradi-
tional face-to-face courses. Therefore, we strongly recommend researching on how
MOOCs can be more engaging and creating new didactical concepts to increase moti-
vational factors.
References
Carson, S., & Schmidt, J. (2012). The Massive Open Online Professor Academic Matter.
Journal of higher education.
Charrad, M., Ghazzali, N., Boiteau, V., & Niknafs, A. (2013). NbClust: An examination of
indices for determining the number of clusters: NbClust Package.
Ebner, M., Kopp, M., Wittke, A., & Schön, S. (2014). Das O in MOOCs – über die
Bedeutung freier Bildungsressourcen in frei zugänglichen Online-Kursen. HMD Praxis der
Wirtschaftsinformatik, 52(1), 68-80. Springer, December 2014.
Portraying MOOCs Learners: a Clustering Experience Using Learning Analytics
Mohammad Khalil, Christian Kastl & Martin Ebner
277
Elton, L. (1996). Strategies to enhance student motivation: a conceptual analysis. Studies in
Higher Education, 21(1), 57-68.
Ferguson, R., & Clow, D. (2015). Examining engagement: analysing learner
subpopulations in massive open online courses (MOOCs). In Proceedings of the Fifth
International Conference on Learning Analytics And Knowledge (pp. 51-58). ACM.
Hollands, F. M., & Tirthali, D. (2014). MOOCs: Expectations and reality. Center for Benefit-
Cost Studies of Education, Teachers College, Columbia University.
Jordan, K. (2013). MOOC Completion Rates: The Data. Retrieved October 2015, from
http://www.katyjordan.com/MOOCproject.html
Khalil, H., & Ebner, M. (2014). Moocs completion rates and possible methods to improve
retention-a literature review. In World Conference on Educational Multimedia, Hypermedia
and Telecommunications (Vol. 2014, No. 1, pp. 1305-1313).
Khalil, M., & Ebner, M. (2015a). A STEM MOOC for School Children – What Does Learning
Analytics Tell us? In Proceedings of 2015 International Conference on Interactive
Collaborative Learning, Florence, Italy. IEEE.
Khalil, M., & Ebner, M. (2015b). Learning Analytics: Principles and Constraints.
In Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications (pp. 1326-1336).
Khalil, M., & Ebner, M. (2016). What can Massive Open Online Course (MOOC)
Stakeholders Learn from Learning Analytics? Learning, Design, and Technology. An
International Compendium of Theory, Research, Practice, and Policy. Springer. Accepted, in
print.
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement:
analyzing learner subpopulations in massive open online courses. In Proceedings of the
third international conference on learning analytics and knowledge (pp. 170-179). ACM.
Lackner, E., Ebner, M., & Khalil, M. (2015). MOOCs as granular systems: design patterns
to foster participant activity. eLearning Papers, 42, 28-37.
McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). Massive Open Online
Courses Digital ways of knowing and learning. The MOOC model For Digital Practice.
Retrieved October 2015, from http://www.elearnspace.org/Articles/MOOC_Final.pdf
Meyer, R. (2012). What it’s like to teach a MOOC (and what the heck’s a MOOC?).
Retrieved October 2015, from http://tinyurl.com/cdfvvqy
Research Track
Proceedings of the European MOOC Stakeholder Summit 2016
278
Neuböck, K., Kopp, M., & Ebner, M. (2015). What do we know about typical MOOC
participants? First insights from the field. In M. Lebrun, I. de Waard, M. Ebner, & M. Gaebel
(Eds.), Proceedings of eMOOCs 2015 conference (pp. 183-190). Mons, Belgium.
Neuendorf, K. A. (2002). The content analysis guidebook (Vol. 300). Thousand Oaks, CA:
Sage Publications.
Rodriguez, C. O. (2012). MOOCs and the AI-Stanford Like Courses: Two Successful and
Distinct Course Formats for Massive Open Online Courses. European Journal of Open,
Distance and E-Learning.
Yuan, L., Powell, S., & Cetis, J. (2013). MOOCs and open education: Implications for
higher education.
... To understand learners' expectations, previous research grouped learners based on their behaviours (e.g. video clicking stream) and further studied their behaviours or learning strategies (Arora et al. 2017;Khalil, Kastl, and Ebner 2016;Kizilcec, Piech, and Schneider 2013;Poquet et al. 2018). These behavioural data were objective, and the quantity was large since MOOCs usually involve thousands of learners. ...
... Most relevant research grouped learners by their behaviours, such as interactions with video lectures, assessments (Kizilcec, Piech, and Schneider 2013), forum activity (Poquet et al. 2018), or both (Arora et al. 2017;Khalil, Kastl, and Ebner 2016). These objective data can be collected in a large amount without learners' active feedbacks, but from these data, we can hardly interpret learners' direct feelings and opinions. ...
Article
In massive open online courses (MOOCs), learners have diverse types of motivation. Learners with different motivations have different interaction behaviours, presence, and learning outcomes. However, scant research has investigated the moderating role of learner motivations in the associations between presence and learning outcomes. This study examined MOOC learner motivation and its moderating role by surveying 646 MOOC learners. By exploratory factor analysis, this study identified four types of motivation: interest in knowledge, curiosity and expansion, connection and recognition, and professional relevance. Based on motivation, the study clustered learners into high-motivation, low-motivation, and asocial learners. Both high-motivation and asocial learners reported strong interest in knowledge and professional relevance, but asocial learners reported the lowest level of connection and recognition among the three groups of learners. Despite the low social presence, the asocial learners still had high levels of cognitive and teaching presence and learning outcomes. In addition, learners with higher presence generally perceived higher cognitive learning, but asocial learners with higher social presence were less satisfied. The results highlight the impacts of specific types of motivation to enrol in MOOCs and suggest designing different environments for learners with different motivation types.
... Ainsi, bien que les MOOC soient des dispositifs récents, ils tendent à réactiver des problématiques plus anciennes marquant le champ de la formation à distance et du e-learning notamment en termes de manque d'engagement des apprenants dans ces dispositifs. Il est à noter que de plus en plus de chercheurs s'interrogent sur le taux de réussite et le lient à l'engagement des apprenants (Cassidy et al. 2014 ;Cheung, 2014 ;Khalil et Ebner, 2016) dans une perspective critique sur la qualité de l'apprentissage (Toven-Lindsey et al. 2015). ...
Thesis
Au cours de ces dernières années, les cours massifs ouverts en ligne (CLOM) ou les Massive Open Online Courses (MOOC) ont offert des possibilités d'apprentissage dans le monde entier dans divers domaines. Néanmoins, ces dispositifs sont critiqués vu leurs taux de réussite très bas. La recherche actuelle s’intéresse à la problématique de l’engagement et la réussite dans les MOOC. Comme pour de nombreuses technologies éducatives émergentes, il convient de mieux comprendre pourquoi et comment les apprenants réussissent aux MOOC et, surtout, quels sont les facteurs qui contribuent à améliorer leurs performances dans ces dispositifs. Cette thèse a pour objectif de contribuer à une meilleure compréhension des déterminants qui affectent l’engagement et les performances des apprenants dans les MOOC. Dans les environnements d'apprentissage en ligne, le niveau d’engagement semble être lié aux performances des apprenants. Cependant, l’engagement est une construction complexe et la recherche sur la façon dont il fonctionne dans les MOOC n'en est qu'à ses débuts. Nous avons cherché dans ce travail à proposer une modélisation théorique de l’engagement en tant que concept multidimensionnel qui joue le rôle d’une variable médiatrice qui s’intercale entre les variables individuelles et les performances des apprenants dans les MOOC. En invoquant un cadre théorique multi-référencés et à travers l’analyse des données empiriques et les traces log de 5904 apprenants dans un xMOOC, notre travail propose un modèle permettant d’une part de mesurer l’engagement des apprenants dans un MOOC et d’autre part d’interroger la nature des relations qui existent entre les variables individuelles, l’engagement et les performances des apprenants. Les résultats saillants de ce travail de recherche mettent en valeur :-Les effets positifs de l’âge de l’apprenant sur son engagement comportemental et cognitif ;-Les effets positifs du principal motif d’inscription sur l’engagement comportemental et cognitif et sur les performances de l’apprenant ;-Les effets positifs du contexte du suivi du MOOC sur l’engagement comportemental, cognitif, social et les performances de l’apprenant ;-L’effet négatif du genre de l’apprenant sur son engagement cognitif ;-Et l’absence de l’effet du niveau d’études de l’apprenant sur son engagement et sur ses performances.Les résultats de ce travail de recherche permettent ainsi de valoriser le rôle médiateur de l’engagement qui s’intercale entre les variables âge, principal motif d’inscription et contexte de suivi du MOOC et les performances et de mettre en exergue les effets positifs de l’engagement dans ses dimensions comportementale, cognitive et sociale sur les performances d’un apprenant dans le MOOC PRD5, avec un effet plus important de l’engagement comportemental, suivi de l’engagement cognitif et en fin de l’engagement social.
... Prior research leveraged cluster analysis to investigate the patterns of learner engagement in MOOCs. In general, researchers group learners based on the summative performance in MOOCs and provide insightful implications on persistently engaging learners (Khalil, Kastl, & Ebner, 2016;Kizilcec, Piech, & Schneider, 2013;Kovanović et al., 2016). Although their efforts speak to the issue of diversity in forum participation patterns, researchers neglected the trend of longitudinally decreased learner engagement in MOOCs (Halawa, Greene, & Mitchell, 2014). ...
Article
An effective experience in discussion forums is important for online learners to maintain their persistence in a MOOC. The purpose of this research is to identify learners’ meaningful participation patterns of topic-related forum posts through the temporal dimension and investigate how the longitudinal trajectory of online meaningful participation is associated with learner performance. Specifically, latent semantic analysis (LSA) and machine learning approaches were used to classify forum posts. Inferential statistic methods were then used to quantify the effect of the temporal dimension of meaningful forum participation on learner performance in MOOCs. The findings of this research provided significant implications on facilitating effective forum discussions and supporting learner performance in MOOCs.
... Similarly, traditional questionnaires are also time-consuming, and may not accurately reflect the real difficulty of videos for students. Clustering analysis, such as the clustering of learners [5], [6] and learning resources [7], [8], is a hot topic in the EDM area. Learning resources clustering techniques help find the relation between the resources for learners and teachers [9], [10]. ...
Article
Full-text available
The pervasive application of Small Private Online Course (SPOC) provides a powerful impetus for the reform of higher education. During the teaching process, a teacher needs to understand the difficulty of SPOC videos for students in real time to be more focused on the difficulties and key points of the course in a flipped classroom. However, existing educational data mining techniques pay little attention to the SPOC video difficulty clustering or classification. In this paper, we propose an approach to cluster SPOC videos based on the difficulty using video-watching data in a SPOC. Specifically, a bipartite graph that expresses the learning relationship between students and videos is constructed based on the number of video-watching times. Then, the SimRank++ algorithm is used to measure the similarity of the difficulty between any two videos. Finally, the spectral clustering algorithm is used to implement the video clustering based on the obtained similarity of difficulty. Experiments on a real data set in a SPOC show that the proposed approach has better clustering accuracy than other existing ones. This approach facilitates teachers learn about the overall difficulty of a SPOC video for students in real time, and therefore knowledge points can be explained more effectively in a flipped classroom.
... As previous authors, they have identified 5 behavioural patterns which described learners as viewers (primarily watching videos; submitting only few assignments); solvers (primarily submitting assignments; watching few if any videos); all-rounders (equally engaging when watching videos and preparing assignments); collectors (primarily downloading video lectures but not necessarily watching them, presenting few assignments if any); and bystanders (signing up for the course but demonstrating extremely passive engagement). Other behavioural clusters like passive participants, active participants, community contributors (Koller, Ng, & Chen, 2013), dropouts, excellent students, gamblers or learners who played with the system, and social learners (Khalil, Kastl, & Ebner, 2016) demonstrate closely related features of learners' behaviour, although it is important to distinguish that they identify learners' need for socializing and networking. These learners tend to engage in course activities but they demonstrate specific interest in facilitating forum discussions or contributing and helping other learners by, e.g. ...
Article
Full-text available
Interfaces between students’ participation, number of submissions to discussion forums, attendance of online lectures and students’ performance on their assignments are significant for students’ success and achievements. Teachers’ activities become more significant when delivering blended or online courses and their role changes from knowledge deliverer to learning designer. Therefore, teachers are challenged to recognize new learning behaviour models and find new ways to engage and motivate learners. Universities offering study programs in blended or online way need to recognize learners’ behaviour, know how to analyse the data, make it “understandable” to teachers and learners, and learn how to adapt course curriculum based on this data. Results of the case study conducted at Vytautas Magnus University revealed that after logging in to Moodle learning platform, students tend to spend time checking forums or course assignments rather than browsing another course content. Moreover, a significant drop-out rate was noticed after the 4th click, when 24% of students tend to quit the session. The results of this research confirm the fact that online learners’ behaviour is changing rapidly, and teachers should be aware of that, understand preferred learning patterns and develop course content based on this data.
... Therefore, based on learners' engagement with the videos and assessments, the cluster of learners can be mapped onto a course engagement continuum. Fig. 1 maps the different clusters of MOOC learners identified in studies by Kizilcec, Piech, and Schneider (2013), Ferguson et al. (2015), and Khalil, Kastl, and Ebner (2016). ...
Article
Full-text available
Self-regulated learning (SRL) refers to how learners steer their own learning. Supporting SRL has been shown to enhance the use of SRL strategies and learning performance in computer-based learning environments. However, little is known about supporting SRL in Massive Open Online Courses (MOOCs). In this study, weekly SRL prompts were embedded as videos in a MOOC. We employed a sequential pattern mining algorithm, Sequential Pattern Discovery using Equivalence classes (cSPADE), on gathered log data to explore whether differences exist between learners who viewed the SRL-prompt videos and those who did not. Results showed that SRL-prompt viewers interacted with more course activities and completed these activities in a more similar sequential pattern than non SRL-prompt viewers. Also, SRL-prompt viewers tended to follow the course structure, which has been identified as a behavioral characteristic of students who scored higher on SRL (i.e., comprehensive learners) in previous research. Based on the results, implications for supporting SRL in MOOCs are discussed.
... Internet" MOOC, it would appear that all these measures did not really make a difference in truly "opening" the MOOC up to traditionally underrepresented groups. Instead of attracting a large and diverse crowd of users, we recruited a rather small and quite homogeneous group, which is very similar in its characteristics to those we have observed with other MOOCs on the iMooX platform(Neuböck, Kopp and Ebner 2015;Khalil, Kastl and Ebner 2016). The (academically) low-threshold but relevant topic, the accordance in language, and the publicity for the course failed to attract some of the targeted groups. ...
... The registered group was more active than the non-registered group in launching thematic posts. This result is consistent with the conclusion involving MOOC learners derived by Khalil, Kastl, and Ebner (2016), indicating that the voluntary external participants generally had a low posting frequency and certification rate compared with registered group. This phenomenon seems that a strong motivation to achieve good course performance had a positive effect on posting thematic content in forums. ...
Article
Full-text available
With the popularity of Small Private Online Courses (SPOCs) in higher education, a plentiful of discussion data has been increasingly generated in SPOC forums. With 752 undergraduates' discussion posts, this study aims to investigate students' engagement patterns within SPOC forums in terms of engagement behaviors and emotions. Firstly, we designed a behavioral code rule to identify posting-and content-level behaviors, and examined their association with course performance. Secondly, we built an emotion lexicon including positivity, negativity and confusion word sets, and adopted an emotion calculation approach to visualize emotional evolutionary trends and to examine emotional differences in registration types and course performance. The results show that, (1) the high-performing group was more active in the most engagement behaviors except for interactive postings. (2) The registered group delivered more threads and wrote richer vocabulary in post content. (3) Whether students were registered for a course or not did not have a significant effect on their emotional expressions, but the registered group exhibited more confusion in forum interactions at the end of the semester. (4) Positive emotion was prevailing for the entire population. Furthermore, compared with the low-achieving group, the high-performing group had higher emotion densities in three types of emotions.
... Many works proposed to cluster learners using different criterions. In [14] the authors proposed to cluster learners in MOOCs using the motivation criterion. In [15] the learners' clustering was established using a previously modeled engagement profile. ...
Conference Paper
Full-text available
iMooX.at is the Austrian MOOC platform founded in 2014. This platform offers free, openly licensed online courses for all, so called Massive Open Online Course (MOOCs). It aims to offer university education in an innovative and digital way.In this article, we will briefly look at the history of the platform and its main milestones till now. Finally, a few possible development steps will be pointed out and discussed.
Conference Paper
Full-text available
Massive Open Online Courses (MOOCs) have been tremendously spreading among Science, Technology, Engineering and Mathematics (STEM) academic disciplines. These MOOCs have served an agglomeration of various learner groups across the world. The leading MOOCs platform in Austria, the iMooX, offers such courses. This paper highlights authors’ experience of applying Learning Analytics to examine the participation of secondary school pupils in one of its courses called “Mechanics in everyday life”. We sighted different patterns and observations and on the contrary of the expected jubilant results of any educational MOOC, we will show, that pupils seemingly decided to consider it not as a real motivating learning route, but rather as an optional homework.
Conference Paper
Full-text available
Within the evolution of technology in education, Learning Analytics has reserved its position as a robust technological field that promises to empower instructors and learners in different educational fields. The 2014 horizon report (Johnson et al., 2014), expects it to be adopted by educational institutions in the near future. However, the processes and phases as well as constraints are still not deeply debated. In this research study, the authors talk about the essence, objectives and methodologies of Learning Analytics and propose a first prototype life cycle that describes its entire process. Furthermore, the authors raise substantial questions related to challenges such as security, policy and ethics issues that limit the beneficial appliances of Learning Analytics processes.
Article
Full-text available
MOOCs as granular systems: design patterns to foster participant activity MOOCs often suffer from high drop-out and low completion rates. At the beginning of the course, the audience is indeed " massive " ; thousands of people wait for the course to begin, but in the end only a low number of participants stay active and complete the course. This paper answers the research question " Is there a specific point during an xMOOC where learners decide to drop out of the course or to become lurkers? " by identifying MOOCs as a challenging learning setting with a " drop-out problem " and a decrease in participant activity after the fourth to fifth course week. These are the first results of a Learning Analytics view on participant activity within three Austrian MOOCs. This " drop-out point " led the paper to introduce a design pattern or strategy to overcome the " drop-out point " : " Think granular! " can be seen as an instructional design claim for MOOCs in order to keep participant activity and motivation high, and that results in three design patterns: four-week MOOCs, granular certificates and suspense peak narratives. 1. MOOCs: a challenging learning setting with a drop-out problem? The MOOC phenomenon was born in Canada in 2008 and has since then become a worldwide movement (Hay-Jew 2015, 614; Hollands & Tirthali 2014, 25f.; Jasnani 2013). MOOCs can be seen as an expression for a modern orientation towards learning as learning can no longer be seen as a formal act that depends only on universities, schools and other institutions within a formal education system. Learning has to be seen as a lifelong process that has become flexible and seamless, as Wong (2012) and Hay-Jew (2015) resume. It encompasses formal and informal learning and physical and digital (learning) worlds (Wong & Looi 2011; Wong 2012). MOOCs – in our short research study, mainly xMOOCs – are open (Rodriguez 2013) and conducted online, with only an internet connection and registration on an xMOOC platform. The American providers Coursera (www.coursera. org), edX (www.edx.org), the German platforms iversity (www. iversity.org) and MOOIN (www.mooin.oncampus.de) or the Austrian iMooX (www.imoox.at), for example, are necessary for attending courses from different fields. Therefore, the audience is very heterogeneous and cannot be predicted in advance, as it can be for traditional learning settings. It can nevertheless be stated that " the majority of MOOC participants are already well-educated with at least a B.A. degree " (Hollands & Tirthali 2014, 42). They have a certain experience within the learning or the educational context (Gaebel 2014, 25). There are almost no limitations regarding location, age, sex and education, to name a few variables. Thus, MOOC design has to respect this unpredictable heterogeneity, which results in a balancing act between multicity and unity regarding, for example, resources and prior knowledge or further information. As a consequence, MOOCs need to have a special instructional design (Jasnani 2013; Kopp & Lackner 2014) that focuses on different framework conditions. Jasnani (2013, 7) thus mentions a " lack of professional instructional design for MOOCs " which can be cited as one of the reasons for the low completion rates MOOCs suffer from. If we assume " an average 50,000 enrollments in MOOCs, with the typical completion rate of below 10%, approximately 7.5%, that amounts to 3,700 completions per 50,000 enrollments " (Ibid., 6) or even less: " Completion rates for courses offered by our interviewees ranged from around 3% to 15% of all enrollees. " (Hollands & Tirthali 2014, 42) Several investigations (Khalil & Ebner 2014) have already been conducted to identify reasons for
Conference Paper
Full-text available
Massive Open Online Courses became a worldwide phenomenon. Especially in Central Europe it is a subject of debates whether universities should invest more money or not. This research study likes to give first answers about typical MOOC participants based on data from different field studies of the Austrian MOOC-platform iMooX. It can be pointed out that the typical learner is a student or an adult learner, strongly interested in the course topic or just interested in learning with media and finally with self- contained learning competencies. The research work concludes that MOOCs broaden the educational field for universities and are a possibility to educate the public in a long run.
Article
Full-text available
Over the past few years, observers of higher education have speculated about dramatic changes that must occur to accommodate more learners at lower costs and to facilitate a shift away from the accumulation of knowledge to the acquisition of a variety of cognitive and non-cognitive skills. All scenarios feature a major role for technology and online learning. Massive open online courses(MOOCs) are the most recent candidates being pushed forward to fulfill these ambitious goals. To date, there has been little evidence collected that would allow an assessment of whether MOOCs do indeed provide a cost-effective mechanism for producing desirable educational outcomes at scale. It is not even clear that these are the goals of those institutions offering MOOCs. This report investigates the actual goals of institutions creating MOOCs or integrating them into their programs, and reviews the current evidence regarding whether and how these goals are being achieved, and at what cost.
Article
Full-text available
Massive Open Online Courses (MOOCs) und Open Educational Resources (OER) sind Schlagworte der heutigen Bildungswelt. In diesem Beitrag führen wir in beide ein und erklären ihre Abhängigkeit und ihren Zusammenhang. Anhand zweier Beispiele – „oncampus an der Fachhochschule Lübeck sowie „iMooX“ von zwei Grazer Universitäten – werden erste Erfahrungen mit MOOCs und OER aufgezeigt. In der abschließenden Diskussion sind sowohl die Herausforderungen als auch die bereits ersichtlichen Vorteile Teil der Betrachtung. Der Beitrag schließt mit der Erkenntnis, dass OER ein wichtiger Bestandteil des Bildungssystems von morgen sind.
Technical Report
Full-text available
This report sets out to help decision makers in higher education institutions gain a better understanding of the phenomenon of Massive Online Open Courses (MOOCs) and trends towards greater openness in higher education and to think about the implications for their institutions. The phenomena of MOOCs are described, placing them in the wider context of open education, online learning and the changes that are currently taking place in higher education at a time of globalisation of education and constrained budgets. The report is written from a UK higher education perspective, but is largely informed by the developments in MOOCs from the USA and Canada. A literature review was undertaken focussing on the extensive reporting of MOOCs through scholarly blogs, press releases as well as openly available reports and research papers. This identified current debates about new course provision, the impact of changes in funding and the implications for greater openness in higher education. The theory of disruptive innovation is used to help form the questions of policy and strategy that higher education institutions need to address.
Conference Paper
Full-text available
Many MOOCs initiatives continue to report high attrition rates among distance education students. This study investigates why students dropped out or failed their MOOCs. It also provides strategies that can be implemented to increase the retention rate as well as increasing overall student satisfaction. Through studying literature, accurate data analysis and personal observations, the most significant factors that cause high attrition rate of MOOCs are identified. The reasons found are lack of time, lack of learners’ motivation, feelings of isolation and the lack of interactivity in MOOCs, insufficient background and skills, and finally hidden costs. As a result, some strategies are identified to increase the online retention rate, and will allow more online students to graduate.
Conference Paper
Full-text available
As MOOCs grow in popularity, the relatively low completion rates of learners has been a central criticism. This focus on completion rates, however, reflects a monolithic view of disengagement that does not allow MOOC designers to target interventions or develop adaptive course features for particular subpopulations of learners. To address this, we present a simple, scalable, and informative classification method that identifies a small number of longitudinal engagement trajectories in MOOCs. Learners are classified based on their patterns of interaction with video lectures and assessments, the primary features of most MOOCs to date. In an analysis of three computer science MOOCs, the classifier consistently identifies four prototypical trajectories of engagement. The most notable of these is the learners who stay engaged through the course without taking assessments. These trajectories are also a useful framework for the comparison of learner engagement between different course structures or instructional approaches. We compare learners in each trajectory and course across demographics, forum participation, video access, and reports of overall experience. These results inform a discussion of future interventions, research, and design directions for MOOCs. Potential improvements to the classification mechanism are also discussed, including the introduction of more fine-grained analytics.
Chapter
Full-text available
This package provides most of the popular indices for cluster validation ready to use for the outputs produced by functions coming from the same package. It also proposes to user the best clustering scheme from the different results obtained by varying all combinations of number of clusters, distance measures, and clustering methods.