Conference PaperPDF Available

On Using Learning Analytics to Track the Activity of Interactive MOOC Videos

Authors:
Josef Wachtler at Graz University of Technology
Josef Wachtler
Mohammad Khalil at University of Bergen
Mohammad Khalil
Behnam Taraghi at Graz University of Technology
Behnam Taraghi
Martin Ebner at Graz University of Technology
Martin Ebner
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

It is widely known that interaction, as well as communication, are very important parts of successful online courses. These features are considered crucial because they help to improve students’ attention in a very significant way. In this publication, the authors present an innovative application, which adds different forms of interactivity to learning videos within MOOCs such as multiple-choice questions or the possibility to communicate with the teacher. Furthermore, Learning Analytics using exploratory examination and visualizations have been applied to unveil learners’ patterns and behaviors as well as investigate the effectiveness of the application. Based upon the quantitative and qualitative observations, our study determined common practices behind dropping out using videos indicator and suggested enhancements to increase the performance of the application as well as learners’ attention.
Figures - uploaded by Mohammad Khalil
Author content
All figure content in this area was uploaded by Mohammad Khalil
Content may be subject to copyright.
Content uploaded by Mohammad Khalil
Author content
All content in this area was uploaded by Mohammad Khalil on Apr 21, 2016
Content may be subject to copyright.
On Using Learning Analytics to Track the
Activity of Interactive MOOC Videos
Josef Wachtler, Mohammad Khalil, Behnam Taraghi, and Martin Ebner
Educational Technology
Graz University of Technology
M¨unzgrabenstraße 35A - 8010 Graz - Austria
josef.wachtler@tugraz.at
Abstract It is widely known that interaction, as well as communication,
are very important parts of successful online courses. These features are
considered crucial because they help to improve students’ attention in a
very significant way. In this publication, the authors present an innova-
tive application, which adds different forms of interactivity to learning
videos within MOOCs such as multiple-choice questions or the possibil-
ity to communicate with the teacher. Furthermore, Learning Analytics
using exploratory examination and visualizations have been applied to
unveil learners’ patterns and behaviors as well as investigate the effec-
tiveness of the application. Based upon the quantitative and qualitative
observations, our study determined common practices behind dropping
out using videos indicator and suggested enhancements to increase the
performance of the application as well as learners’ attention.
1 Introduction
It is a common knowledge that interaction, as well as the communication, are
very important influencing factors of students’ attention. This indicates that
different possibilities of interaction should be offered at a MOOC 1in all possible
directions. So it is vital to offer some communication channels like e-mail or a
discussion forum, and in addition it is suggested that a form of interaction with
the content of the course itself is available [9]. [1]
The attention is considered as the most crucial resource for human learning
[5]. Due to that, it is from high importance to understand and to analyze this
factor. The results of such an analysis should be used to further improve the
different methods of attention enhancing [6]. Moreover, learning analytics plays a
major factor into enhancing learning environments components such as the video
indicator of MOOCs and finally acts into reflecting and benchmarking the whole
learning process [7]. In this publication, the usage of a web-based information
system which provides the possibility to enrich the videos of a MOOC with
different forms of interactivity (see Section 3) is presented. This paper covers an
experiment on a MOOC named Making - Creative, digital creating with children2
1short for Massive Open Online Course
2http://imoox.at/wbtmaster/startseite en/maker.html (last accessed Jannuary
2016)
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes.
This volume is published and copyrighted by its editors.
SE@VBL 2016 workshop at LAK’16, April 26, 2016, Edinburgh, Scotland
8
and is attended by both, school-teachers as well as people who educate children
in non-school settings. It is scheduled in seven weeks with at least one video
per week. A detailed analysis of the activity of the attendees at the videos is
presented by Section 4.
Finally, this work aims to show how learning analytics could be done to
monitor the activity of the students within videos of a MOOC. In other words,
the research goal of this publication could be summarized to ”using an interactive
video platform to support students’ attention and to analyze their participation”.
2 Related work
In comparison to the approach shown by Section 3there are other services pro-
viding similar features.
First, there is the possibility to use the built-in features of Youtube3(e.g. text
annotations or polls) itself. However, the different methods of analysis are very
limited. A further tool is Zaption4which provides various forms of interactive
content for videos (e.g. multiple-choice questions) at planned positions as well
as a rich set of analysis possibilities. Unfortunately, it shows the position of the
interactions in the timeline of the video. This means that the users are able
to jump from interaction to interaction without really watching the video. In
comparison to that, a tool named EdTed5also offers the possibility to enrich a
video with questions. However, the questions are not bound to a position in the
video and furthermore, they could be accessed every time during the video.
The real-world pendant of interactive learning videos is known as ARS 6,
which enables the lecturer to present questions to students during the lecture in
a standard classroom situation [10] [4]. Based on that, it offers several possibil-
ities of analysis. It is well-known that an ARS has the power to enhance both,
students’ attention and participation [2]. This means that the addition of inter-
activity to learning videos tries to generate similar benefits as those generated
by an ARS.
3 Interactions in Learning Videos
To provide interactive learning videos a web-based information system called
LIVE 7first introduced by [3] is developed. It offers the possibility to embed
different forms of interaction in videos (e.g. multiple-choice questions). As indi-
cated above (see Section 1), the main purpose of these interactions is to support
the attention of the students. The functionalities of LIVE could be categorized
by the tasks of three different types of users.
3https://www.youtube.com/ (last accessed Jannuary 2016)
4http://www.zaption.com/ (last accessed Jannuary 2016)
5http://ed.ted.com/ (last accessed Jannuary 2016)
6short for Audience-Response-System
7short for LIVE Interaction in Virtual learning Environments
9
The first ones are normal users who could be seen as students. They are
only allowed to watch the videos and to participate to the interactions. Figure 1
shows a screenshot of a playing video which is currently paused and overlaid by
an interaction (1). To resume playing, it is required to respond to the interac-
tion which means that the displayed multiple-choice question has to be answered
in this example. Furthermore, it can be seen that there are some other control
elements on the right side of the videos (2). They could be used to invoke in-
teractions manually. For instance, it is possible to ask a question to the teacher.
[12]
Figure 1. A Screenshot of LIVE shows a video interrupted by a multiple-choice ques-
tion.
In comparison to that, the users of the second group are equipped with
teacher privileges. They are additionally able to embed interactions in the videos
as well as to view different forms of analysis. To add an interaction, the teacher
has to select its position within the video by using a preview of it or by entering
the position. With a dialog, the teacher can embed multiple-choice questions or
text-based questions in the video and furthermore, it is possible to add an image
to a question. [12]
The analysis consists of several parts. At first, there is a list of all students
who watched the video and for each student in this list it is shown how much of
the videos they watched. In addition, a chart shows the number of users (green)
and views (red) across the timeline of the video (see Figure 2). This chart could
be used to identify the most interesting part of the video. Furthermore, it is
possible to access a detailed analysis of each student. It shows the timeline of
the video and marks each watched part of it with a bar (see Figure 3). If such a
bar is hovered with the mouse-pointer, additional information is displayed. This
consists of the time of the joining and the leaving of this watched timespan in
relative as well as the absolute values. [12]
10
Figure 2. A Screenshot of the timeline analysis.
Figure 3. A Screenshot of the watched-parts analysis.
In comparison to these forms of analysis related to the watching of the stu-
dents, there is also a detailed statistic about the answers to the embedded ques-
tions. This means that for the multiple-choice questions, the answers of the
students as well as their correctness is displayed. For the text-based questions,
LIVE displays answers of the students and the teacher has to evaluate them
manually because text-based answers are impossible to check automatically. [12]
The third group of users are researchers. They are able to download different
forms of analysis as raw data. This means that they can select a video and obtain
the data as a spreadsheet (CSV 8).
Finally, the following list aims to give a summarizing overview of the features
of LIVE [3] [12]:
–only available for registered and authenticated users
–different methods of interaction
•automatically asked questions and captchas9
•asking questions to the lecturer by the learners
•asking text-based questions to the attendees live or at pre-defined posi-
tions
•multiple-choice questions at pre-defined positions
•reporting technical problems
–different possibilities of analysis [11]
•a detailed logging of the watched time-spans to point out at which time
a user watched which part of the video
8short for Comma-Separated Values
9short for Completely Automated Public Turing Test to Tell Computers and Humans
Apart
11
•a calculation of an attention level to measure the attention of the stu-
dents
–raw data download for researchers
4 Evaluation
This section presents a very detailed analysis of the videos of the MOOC as
well as of the multiple-choice questions. For that, the data provided by LIVE is
evaluated using visualizations and exploratory analysis.
First, the delay of response to the questions provided by LIVE in the MOOC
videos during the seven weeks is demonstrated by two figures. Figure 4visualizes
a box plot. The x-axis records MOOC videos during the period of the course,
while the y-axis shows students’ delay of response in seconds. This period was
limited to 60 seconds. Students are categorized to certified students, who finished
the course successfully and applied for a certificate, and non-certified students. In
this figure, we tried to study the difference in behavior between both categories.
In some of the weeks, certified students took more time to answer the questions
such as in week 4 and week 7. For instance, certified students median in week 4
was 15 seconds, while the median for the non-certified students was 13 seconds.
Furthermore, there was 3 seconds difference in the median between certified and
non-certified students in week 7. Additionally, the median in week 1 and week 5
are typically the same with an insignificant variation between the first and the
third quartiles.
Figure 4. A box plot showing the reaction delays to multiple-choice questions.
In comparison to that, Figure 5visualizes a violin plot. The x-axis indicates
students’ status. This visualization summarizes the students’ status and the
12
delay of response time to the multiple-choice questions in all of the MOOC
videos. The thickness of the blue violin shape is slightly wider than the red one
in the (8-13) seconds range, which indicates the more time needed to answer
the questions. In addition to that, the non-certified violin shape holds more
outliers attributes than the certified division. It is believed from the previous two
observations, that certified students took less time in answering the questions in
general. This case can be explained as the questions were easy to answer if the
student were paying enough attention to the video lectures.
Figure 5. A violin plot summarizes the reaction delays to multiple-choice questions.
Figure 6displays the timespan division in percentage and the timing of the
first multiple-choice question represented as a vertical dashed line. Using this
visualization, we can infer the relevance timing of the first question to describe
the drop rate during videos. The questions were programmed to pop up after 5%
of any MOOC video. Students may watch the first few seconds and make skips
or drop out after that [13], and this can be seen in the plot where students are
dropping in the early 15% of the videos. To grab the attention of the students
and maintain a wise attrition rate, the multiple-choice questions were intended
to be shown randomly in the high drop rate scope. Further, week 6 was tested
to check the postponed question effect on the retention rate. The data in the
figure also shows that students do not drop out a learning video in the range
between 20%-80%, unless they replay it on that period and spend time on a
particular segment to understand a complex content. The promising outcomes
are seen with a stable attrition rate in the last four weeks when students are
offered an interactive content during the video indoctrinate process.
In Figure 7, the data is displayed in order to trace the video drop ratio of
each second in every video. The x-axis displays the percentage of videos. The
13
Figure 6. Timespan division in percentage and the timing of the first multiple-choice
question.
Figure 7. Trace of the video drop ratio of each second in every video.
14
colored points specify the video name and the watchers count. While the black
shadowed points indicate number of views. For instance, it is obvious that the
data in the first three weeks shows more views per user which can be explained
as an initial interest of the first online course weeks. On the other hand, the
views nearly equaled the number of users from week 4 to the last week. Another
interesting observation is the slow drop rate during the videos in all of the weeks
despite the high drop in the last 2-3% of every video. A clarification of such
attitude is due to the closing trailer of every video which most students jump
over it.
Due to the independency of the examined MOOC, each video of this course
does not rely on the previous one. The activity of every video varies in every
week. For this reason, Figure 8shows activity of the total number of stop and
play actions in the MOOC videos. The blue points denote the certified students
while the orange ones denote the non-certified students. In fact, the first three
weeks reflect proper enthusiastic count of actions. We realized that there was a
distinct activity by the non-certified students in week 5. A reasonable clarifica-
tion is because of the interesting topic of that week which was about 3D-Printing.
However, their engagement becomes much less in the last two weeks, as this was
proven in other MOOCs’ videos analysis [8].
Figure 8. The activity of the total number of stops and plays in the MOOC videos.
15
5 Conclusion
Catching the attention of learners in online videos of MOOCs is an intriguing
argument across learning analytics discussions. With this publication, the usage
of an interactive video platform presenting videos of a MOOC is shown. It points
out the main functionalities of this platform as well as the participation and
the activity of the students. Additionally, we demonstrated an evaluation of
this system in order to examine its performance and describe the behavior of
students. Finally, the results show that the main concept behind latching on the
students’ attention becomes attainable through evaluating the questions’ content
and the interactions timing.
References
1. Carr-Chellman, A., Duchastel, P.: The ideal online course. British Journal of
Educational Technology 31(3), 229–241 (2000), http://dx.doi.org/10.1111/
1467-8535.00154
2. Ebner, M.: Introducing live microblogging: how single presentations can be en-
hanced by the mass. Journal of research in innovative teaching 2(1), 91–100 (2009)
3. Ebner, M., Wachtler, J., Holzinger, A.: Introducing an information system for
successful support of selective attention in online courses. In: Universal Access in
Human-Computer Interaction. Applications and Services for Quality of Life, pp.
153–162. Springer (2013)
4. Haintz, C., Pichler, K., Ebner, M.: Developing a web-based question-driven audi-
ence response system supporting byod. J. UCS 20(1), 39–56 (2014)
5. Heinze, H.J., Mangun, G.R., Burchert, W., Hinrichs, H., Scholz, M., M¨unte, T.F.,
G¨os, A., Scherg, M., Johannes, S., Hundeshagen, H., Gazzaniga, M.S., Hillyard,
S.A.: Combined spatial and temporal imaging of brain activity during visual se-
lective attention in humans. Nature 372, 543–546 (Dec 1994)
6. Helmerich, J., Scherer, J.: Interaktion zwischen lehrenden und lernenden in medien
unterst¨utzten veranstaltungen. In: Breitner, M.H., Bruns, B., Lehner, F. (eds.)
Neue Trends im E-Learning, pp. 197–210. Physica-Verlag HD (2007)
7. Khalil, M., Ebner, M.: Learning analytics: principles and constraints. In: Proceed-
ings of World Conference on Educational Multimedia, Hypermedia and Telecom-
munications. pp. 1326–1336 (2015)
8. Khalil, M., Ebner, M.: What can massive open online course (mooc) stakeholders
learn from learning analytics? Learning, Design, and Technology. An International
Compendium of Theory, Research, Practice, and Policy. Springer. Accepted, in
print. (2016)
9. Lackner, E., Ebner, M., Khalil, M.: Moocs as granular systems: design patterns to
foster participant activity. eLearning Papers 42, 28–37 (2015)
10. Tobin, B.: Audience response systems, stanford university school of medicine.
http://med.stanford.edu/irt/edtech/contacts/documents/2005-11_AAMC_
tobin_audience_response_systems.pdf (2005), [Online; accessed 2012-10-09]
11. Wachtler, J., Ebner, M.: Attention profiling algorithm for video-based lectures. In:
Learning and Collaboration Technologies. Designing and Developing Novel Learn-
ing Experiences, pp. 358–367. Springer (2014)
16
12. Wachtler, J., Ebner, M.: Support of video-based lectures with interactions-
implementation of a first prototype. In: World Conference on Educational Mul-
timedia, Hypermedia and Telecommunications. vol. 2014, pp. 582–591 (2014)
13. Wachtler, J., Ebner, M.: Impacts of interactions in learning-videos: A subjective
and objective analysis. In: EdMedia: World Conference on Educational Media and
Technology. vol. 2015, pp. 1642–1650 (2015)
17
... Interactive features [11, 15, 20, 33, 34, 41, 46, 47, 51, 53, 56, 62, 82, 84, 91, 100, 106, 110, 113, 114, 118-120, 125, 129, 130, 132, 142, 149-151, 154, 157, 162, 172, 175, 178, 188, 199, 208, 210, 217, 222, 234, 238, 240, 247-250, 259, 277] Quizzes embedded in the video, virtual reality and 360-degree characteristics, in-video annotations, feedback to the learner, games, hyperlinks, interface elements for navigation Production style [6,30,42,45,49,55,56,87,88,93,94,136,140,180,181,190,191,197,207,212,214,223,224,229,230,236] Khan-style, with a talking head, dialogue-style or monologue-style, slide-based, with animations, paper-based explanations, classroom-style videos, Instructional design and principles [5,58,60,75,95,101,135,140,141,169,185,212,226,241,246,256,258,270,271] Multimedia learning principles, guidelines for effective engaging, pedagogical characteristics of didactical mathematical knowledge, according to film theory, interaction theory, limited capacity theory Others [52,116,204,211,242] Music, sign language, inclusion of a summary, content design, visual complexity, content design: text layout, image position, slide size, slide design, text ration, image ration of all developed tools (see Figure 5). Also, please note that some articles proposed systems consisting of two or more tools [110,114,249,250,269]. Figure 6 shows the video characteristics that have been used in articles to develop VBL tools. It can be seen that text and visual features were the most used characteristics, mainly by recommender systems, information extraction and summarization methods, as well as navigational tools. ...
... Walk et al. [250] included quizzes in the video with the aim of enhancing active learning. Wachtler et al. [249] presented an application that can include several types of questions such as single-and multiple-choice questions, as well as questions to the instructor. Ma and Ma [159] developed an automatic question generation method using the video transcript and an external knowledge graph. ...
... Walk et al. [250] developed a tool that shows the audience retention for specific video segments, the number of times the segment was re-watched, and how much time learners spent on it. Wachtler et al. [249] presented a tool that provided an analysis about the video (e.g., number of views) and also about the student (e.g., extent to which the video was watched). Long et al. [156] proposed a framework to analyze patterns regarding video usage, for instance, usage of a single video, specific video segments, all videos in a course, etc. ...
A Closer Look into Recent Video-based Learning Research: A Comprehensive Review of Video Characteristics, Tools, Technologies, and Learning Effectiveness
Preprint
Full-text available
  • Jan 2023
People increasingly use videos on the Web as a source for learning. To support this way of learning, researchers and developers are continuously developing tools, proposing guidelines, analyzing data, and conducting experiments. However, it is still not clear what characteristics a video should have to be an effective learning medium. In this paper, we present a comprehensive review of 257 articles on video-based learning for the period from 2016 to 2021. One of the aims of the review is to identify the video characteristics that have been explored by previous work. Based on our analysis, we suggest a taxonomy which organizes the video characteristics and contextual aspects into eight categories: (1) audio features, (2) visual features, (3) textual features, (4) instructor behavior, (5) learners activities, (6) interactive features (quizzes, etc.), (7) production style, and (8) instructional design. Also, we identify four representative research directions: (1) proposals of tools to support video-based learning, (2) studies with controlled experiments, (3) data analysis studies, and (4) proposals of design guidelines for learning videos. We find that the most explored characteristics are textual features followed by visual features, learner activities, and interactive features. Text of transcripts, video frames, and images (figures and illustrations) are most frequently used by tools that support learning through videos. The learner activity is heavily explored through log files in data analysis studies, and interactive features have been frequently scrutinized in controlled experiments. We complement our review by contrasting research findings that investigate the impact of video characteristics on the learning effectiveness, report on tasks and technologies used to develop tools that support learning, and summarize trends of design guidelines to produce learning videos
View
... The application of interactive components to learning videos also provides different possibilities of learning analytics to both, the teachers and researchers [7]. This means that interactions could be used to evaluate the understanding of the students. ...
... It can be seen that the remarks about the connection between the distribution of the interactive components and the success rate of the students were gathered in the context of different studies. For that the possibilities of learning analytics provided by the application of the interactions to videos were used [7]. ...
Scheduling Interactions in Learning Videos A State Machine Based Algorithm
Article
Full-text available
  • Jul 2019
Based on the currently developing trend of so called Massive Open Online Courses it is obvious that learning videos are more in use nowadays. This is some kind of comeback because due to the maxim "TV is easy, book is hard" [1][2] videos were used rarely for teaching. A further reason for this rare usage is that it is widely known that a key factor for human learning is a mechanism called selective attention [3][4]. This suggests that managing this attention is from high importance. Such a management could be achieved by providing different forms of interaction and communication in all directions. It has been shown that interaction and communication is crucial for the learning process [6]. Because of these remarks this research study introduces an algorithm which schedules interactions in learning videos and live broadcastings. The algorithm is implemented by a web application and it is based on the concept of a state machine. Finally, the evaluation of the algorithm points out that it is generally working after the improvement of some drawbacks regarding the distribution of interactions in the video.
View
... It can be reasonably concluded that the experimental group's superior performance in terms of technology proficiency and digital literacy levels can be attributed to the various types of interaction present in the videos. The incorporation of interactive elements into videos can facilitate interaction between the learner and the content, transforming students from passive viewers to active participants (Ugur & Okur, 2016;Wachtler, Khalil, Taraghi, & Ebner, 2016;Zhang et al., 2006). This approach assists students in focusing their attention on course subjects and enhances their learning effectiveness by motivating them (Zhang et al., 2006). ...
Interactive Videos in Web-Based Education: Technology Proficiency and Digital Literacy Levels
Article
Full-text available
  • Jul 2024
The purpose of this study is to determine the effects of using interactive video in web-based courses on students' technology proficiency and digital literacy levels as well as the students' perspectives on this. For this purpose, the study was conducted with quasi-experimental patterns, one of the quantitative research approaches. The quantitative data collected with data collection tools were analyzed with the dependent and independent samples t-tests, and the qualitative data were analyzed using content analysis. The study's findings revealed that there was a significant difference between the technology proficiency and digital literacy pre-test and post-test scores of the students in the experimental group. Moreover, based on the study findings, it was seen that there was a significant difference favorable to the experimental group between the technology proficiency and digital literacy scores of the experimental and control groups. Study results showed that using interactive videos in web-based lessons can improve students' technology proficiency and digital literacy levels. This study will contribute to the literature in terms of the positive effects of using interactive video in web-based courses on students' technology proficiency and digital literacy levels as well as the students' perspectives on this.
View
... Previous studies have explored the use of video analytics to identify and explain learning patterns (Hu et al., 2020;Kim et al., 2014); at-risk students (Sinha et al., 2014); learning outcomes (Lang et al., 2020) and self-regulation strategies (Baker et al., 2021). Video analytics can also benefit the instructional design of courses (Wachtler et al., 2016). For example, Guo et al. (2014) identified a correlation between the student attention span and course videos length. ...
Video Analytics in Digital Learning Environments: Exploring Student Behaviour Across Different Learning Contexts
Article
Full-text available
  • Aug 2023
The use of videos in teaching has gained impetus in recent years, especially after the increased attention towards remote learning. Understanding students’ video-related behaviour through learning (and video) analytics can offer instructors significant potential to intervene and enhance course designs. Previous studies explored students’ video engagement to reveal learning patterns and identify at-risk students. However, the focus has been mostly placed on single contexts, and therefore, limited insights have been offered about the differences and commonalities between different learning settings. To that end, the current paper explored student video engagement in three disparate contexts. Following a case study research approach, we uncovered the commonalities and differences of video engagement in the context of SPOC, MOOC, and an undergraduate university course. The findings offer a deeper and more comprehensive understanding of students’ video-related engagement and shed light into several key aspects related to video analytics that should be considered during the design of video-based learning (e.g., learning objectives in relation to video type or context). Additionally, the three cases indicated the important role of the content type, the length, and the aim of the video on students’ engagement. Further implications of the work are also discussed in the paper.
View
... In this respect, timing refers to the intervention having been presented at the perceived correct time; whereas, content refers to the suitability of the intervention with the video content at that moment. Wachtler et al. (2016) emphasized the importance of evaluating timing and content of interventions in interactive videos. For this, two questions are presented in the survey (one for timing and one for content). ...
An Intervention Framework for Developing Interactive Video Lectures based on Video Clickstream Behavior: A Quasi- Experimental Evaluation
Article
  • Feb 2022
The purpose of this study is to develop an intervention framework based on video clickstream interactions for delivering superior user experience for video lectures. Apart from existing studies on data driven interventions, this study focuses on video clickstream interactions to identify timely interventions for creating interactive video lectures. First, a framework was developed through an exploratory experiment, in which 29 students' clickstream behaviors were tracked on an online platform and then individual interviews were held with 17 of the students and a subject-matter expert. The framework shows how click types are transformed into interactive elements with five question types (where, why, which, how, what). It includes click types, click reasons, interventions, actions, and interactive elements. Then, a quasi-experimental study was performed with 18 students to investigate the effect of the proposed framework on the students' satisfaction and engagement. The results showed that students' satisfaction significantly increased for interactive videos created using the proposed framework when motivation was controlled. In addition, students' frequency to go back to important points decreased significantly in interactive videos, whilst students' frequency to skip unimportant points increased significantly in interactive videos. In conclusion, the proposed framework can be used to transform linear videos to interactive videos.
View
... When information is aggregated, it may be to show the collective student knowledge to get a more complete picture of what the class knows as a whole (Slotta et al., 2013). Additionally, aggregated data can be used to see overall class patterns, such as average activity and actions when video watching (Wachtler, Khalil, Taraghi, & Ebner, 2016) or the behaviours that students engage in across a course (Jovanović, Gašević, Dawson, Pardo, & Mirriahi, 2017). Although these methods can provide useful information about student behaviour and actions over time, they cannot make predictions based on students' previous and current actions. ...
Real-Time Prediction of Students’ Activity Progress and Completion Rates
Article
Full-text available
  • Sep 2020
In classrooms, some transitions between activities impose (quasi-)synchronicity, meaning there is a need for learners to move between activities at the same time. To make real-time decisions about when to move to the next activity, teachers need to be able to balance the progress of their students as they work at different paces. In this paper, we present a set of estimators that can be used in real time to predict the progress and completion rates of students working on computer-supported activities that can be divided into sequential subtasks. With our estimators, we investigate what effect the average progress rate of the class, a given number of previous steps, or weighting the proportion of progress assigned to each subtask has on predictions of students’ progress. We find that accounting for the average class progress rate near the beginning of the activity can improve predictions over baseline. Additionally, weighted subtasks decrease prediction accuracy for activities where the behaviour of faster students diverges from the average behaviour of the class. This paper contributes to our ability to provide accurate student progress predictions and to understand the behaviour of students as they progress through the activity. These real-time predictions can enable teachers to optimize learning time in their classrooms.
View
A survey of visual analytics techniques for online education
Article
  • Aug 2022
Visual analytics techniques are widely utilized to facilitate the exploration of online educational data. To help researchers better understand the necessity and the efficiency of these techniques in online education, we systematically review related works of the past decade to provide a comprehensive view of the use of visualization in online education problems. We establish a taxonomy based on the analysis goal and classify the existing visual analytics techniques into four categories: learning behavior analysis, learning content analysis, analysis of interactions among students, and prediction and recommendation. The use of visual analytics techniques is summarized in each category to show their benefits in different analysis tasks. At last, we discuss the future research opportunities and challenges in the utilization of visual analytics techniques for online education.
View
Automatic Authentication of Students at an Interactive Learning-Video Platform
Conference Paper
Full-text available
  • Jul 2019
There are many reasons for the implementation of authentication on learning platforms. For instance, it is required for the teachers to identify individual students if the learning platform offers some kind of assessment. In addition, authentication is the base for a successful monitoring of the attendance of the students. Compulsory attendance is often applied because many positive effects have been reported by authors of several studies. To monitor the attendance at online videos interactive components can be applied. Another benefit of such interactive components is that they help retain the attention of the students. This study was carried out to examine how the students used a video learning platform which provides interactive components of the videos as part of a course offered at Graz University of Technology. Up until now, a major drawback of this platform has been that the students have been required to register manually on the platform. Now, students are able to use the platform without manual registration and authentication, because these steps happen automatically via the main learning platform that provides all of the course materials. Furthermore, the course design and the application of the interactive components, presenting multiple-choice questions, are evaluated. It is pointed out that the concept improves the performance of the students and equips the teacher with valuable feedback regarding the students' interests.
View
Increasing Learning Efficiency and Quality of Students´HomeworkStudents´Homework by Attendance Monitoring and Polls at Interactive Learning Videos
Conference Paper
Full-text available
  • Jun 2018
Due to the fact that students are confronted with a growing amount of texts, colours, figures and shapes and due to their ability to process only a limited number of such information simultaneously, it seems to be obvious that efforts should be made to increase the students' attention-levels. This is important because research results have indicated that selective attention is considered as the most valuable resource in the process of human learning. The application of interaction and communication to the process of learning is a useful strategy to direct the students' attention. It seems to be obvious that this is also true for learning videos. Therefore, this work contains a description of how a video platform with interactive components can be used to support the students and teacher. The video platform is explained and evaluated by analysing its usage in a large teaching course at an institution of higher education. The application of this strategy improved the students' performance and optimized the teacher's workload.
View
Learning Analytics in Massive Open Online Courses
Article
Full-text available
  • Feb 2018
Educational technology has obtained great importance over the last fifteen years. At present, the umbrella of educational technology incorporates multitudes of engaging online environments and fields. Learning analytics and Massive Open Online Courses (MOOCs) are two of the most relevant emerging topics in this domain. Since they are open to everyone at no cost, MOOCs excel in attracting numerous participants that can reach hundreds and hundreds of thousands. Experts from different disciplines have shown significant interest in MOOCs as the phenomenon has rapidly grown. In fact, MOOCs have been proven to scale education in disparate areas. Their benefits are crystallized in the improvement of educational outcomes, reduction of costs and accessibility expansion. Due to their unusual massiveness, the large datasets of MOOC platforms require advanced tools and methodologies for further examination. The key importance of learning analytics is reflected here. MOOCs offer diverse challenges and practices for learning analytics to tackle. In view of that, this thesis combines both fields in order to investigate further steps in the learning analytics capabilities in MOOCs. The primary research of this dissertation focuses on the integration of learning analytics in MOOCs, and thereafter looks into examining students' behavior on one side and bridging MOOC issues on the other side. The research was done on the Austrian iMooX xMOOC platform. We followed the prototyping and case studies research methodology to carry out the research questions of this dissertation. The main contributions incorporate designing a general learning analytics framework, learning analytics prototype, records of students' behavior in nearly every MOOC's variables (discussion forums, interactions in videos, self-assessment quizzes, login frequency), a cluster of student engagement...
View
Impacts of Interactions in Learning-Videos: A Subjective and Objective Analysis
Conference Paper
Full-text available
  • Jul 2015
It is clear that a system known as selective attention is the most crucial resource for human learning (Heinze et al. 1994). Due to this fact a web based information system is developed to support the attention of the watchers of a learning video. This is done by enriching the video with different forms of interactions. Among others there are interactions presenting multiple-choice questions at predefined positions in the video as well as randomly occurring interactions displaying general questions, which are not content related. To gain a basic plan which points out how to distribute the interactions over the video the usage of the information system at a lecture is analyzed. For that on the one hand some feedback of the users is provided and on the other hand an objective analysis of the results of the multiple-choice questions is done by evaluating their time of occurrence in the video.
View
Learning Analytics: Principles and Constraints
Conference Paper
Full-text available
  • Jun 2015
Within the evolution of technology in education, Learning Analytics has reserved its position as a robust technological field that promises to empower instructors and learners in different educational fields. The 2014 horizon report (Johnson et al., 2014), expects it to be adopted by educational institutions in the near future. However, the processes and phases as well as constraints are still not deeply debated. In this research study, the authors talk about the essence, objectives and methodologies of Learning Analytics and propose a first prototype life cycle that describes its entire process. Furthermore, the authors raise substantial questions related to challenges such as security, policy and ethics issues that limit the beneficial appliances of Learning Analytics processes.
View
MOOCs as granular systems: design patterns to foster participant activity
Article
Full-text available
  • Jun 2015
MOOCs as granular systems: design patterns to foster participant activity MOOCs often suffer from high drop-out and low completion rates. At the beginning of the course, the audience is indeed " massive " ; thousands of people wait for the course to begin, but in the end only a low number of participants stay active and complete the course. This paper answers the research question " Is there a specific point during an xMOOC where learners decide to drop out of the course or to become lurkers? " by identifying MOOCs as a challenging learning setting with a " drop-out problem " and a decrease in participant activity after the fourth to fifth course week. These are the first results of a Learning Analytics view on participant activity within three Austrian MOOCs. This " drop-out point " led the paper to introduce a design pattern or strategy to overcome the " drop-out point " : " Think granular! " can be seen as an instructional design claim for MOOCs in order to keep participant activity and motivation high, and that results in three design patterns: four-week MOOCs, granular certificates and suspense peak narratives. 1. MOOCs: a challenging learning setting with a drop-out problem? The MOOC phenomenon was born in Canada in 2008 and has since then become a worldwide movement (Hay-Jew 2015, 614; Hollands & Tirthali 2014, 25f.; Jasnani 2013). MOOCs can be seen as an expression for a modern orientation towards learning as learning can no longer be seen as a formal act that depends only on universities, schools and other institutions within a formal education system. Learning has to be seen as a lifelong process that has become flexible and seamless, as Wong (2012) and Hay-Jew (2015) resume. It encompasses formal and informal learning and physical and digital (learning) worlds (Wong & Looi 2011; Wong 2012). MOOCs – in our short research study, mainly xMOOCs – are open (Rodriguez 2013) and conducted online, with only an internet connection and registration on an xMOOC platform. The American providers Coursera (www.coursera. org), edX (www.edx.org), the German platforms iversity (www. iversity.org) and MOOIN (www.mooin.oncampus.de) or the Austrian iMooX (www.imoox.at), for example, are necessary for attending courses from different fields. Therefore, the audience is very heterogeneous and cannot be predicted in advance, as it can be for traditional learning settings. It can nevertheless be stated that " the majority of MOOC participants are already well-educated with at least a B.A. degree " (Hollands & Tirthali 2014, 42). They have a certain experience within the learning or the educational context (Gaebel 2014, 25). There are almost no limitations regarding location, age, sex and education, to name a few variables. Thus, MOOC design has to respect this unpredictable heterogeneity, which results in a balancing act between multicity and unity regarding, for example, resources and prior knowledge or further information. As a consequence, MOOCs need to have a special instructional design (Jasnani 2013; Kopp & Lackner 2014) that focuses on different framework conditions. Jasnani (2013, 7) thus mentions a " lack of professional instructional design for MOOCs " which can be cited as one of the reasons for the low completion rates MOOCs suffer from. If we assume " an average 50,000 enrollments in MOOCs, with the typical completion rate of below 10%, approximately 7.5%, that amounts to 3,700 completions per 50,000 enrollments " (Ibid., 6) or even less: " Completion rates for courses offered by our interviewees ranged from around 3% to 15% of all enrollees. " (Hollands & Tirthali 2014, 42) Several investigations (Khalil & Ebner 2014) have already been conducted to identify reasons for
View
Support of Video-Based lectures with Interactions - Implementation of a first prototype
Conference Paper
Full-text available
  • Jun 2014
In general videos have a more or less consuming character without any interaction possibilities. Due to this fact a web-based application is developed which offers different methods of communication and interaction to a certain learning-video. This should help attendees to avoid that they become tired and annoyed. The lecturer is able to use interactions to omit whether learners are able to understand the content of the video. For instance the developed web-application offers the possibility to add multiple-choice-questions at predefined positions. Furthermore there are many different kind of analysis as for example a detailed attention-profile. In this publication the implementation of a first prototype is described as well as a first field study which points out that learners? interactions and engagements increased arbitrarily.
View
Attention Profiling Algorithm for Video-Based Lectures
Conference Paper
Full-text available
  • Jun 2014
  • Lect Notes Comput Sci
Due to the fact that students' attention is the most crucial resource in a high-quality course it is from high importance to control and analyze it. This could be done by using the interaction and the communication because they are known as valuable influencing factors of the attention. In this publication we introduce a web-based information system which implements an attention-profiling algorithm for learningvideos as well as live-broadcastings of lectures. For that different methods of interaction are offered and analyzed. The evaluation points out that the attention profiling algorithm delivers realistic values.
View
Introducing an Information System for Successful Support of Selective Attention in Online Courses
Conference Paper
Full-text available
  • Jul 2013
  • Lect Notes Comput Sci
Human learning processes are strongly depending on attention of each single learner. Due to this fact any measurement helping to increase students' attention is from high importance. Till now there are some developments called Audience-Response-Systems only available for face-to-face education even for masses. In this publication we introduce a web-based information system which is also usable for online-systems. Students' attention will be conserved based on different interaction forms during the live stream of a lecture. The evaluation pointed out that the system helps to enlarge the attention of each single participant.
View
Developing a Web-Based Question-Driven Audience Response System Supporting BYOD
Article
Full-text available
  • Mar 2014
Question-driven Audience Response Systems (ARSs) are in the focus of research since the 1960s. Since then, the technology has changed and therefore systems have evolved too. This work is about conception and implementation of the web-based ARS RealFeedback which uses the principle of bring your own device (BYOD). A state-of-the-art analysis compares the features of existing web-based ARSs. The most important findings are used for the conception and the implementation of the system. Thinking-aloud tests, and the first usages during lectures confirm that the chosen requirements are very significant and valuable for lecturers
View
Introducing live microblogging: How single presentations can be enhanced by the mass
Article
Full-text available
  • Jan 2009
Web 2.0 technologies pervade our daily life as well as educational settings. A fairly new approach is communication through so called microblogging channels. Mobile devices with Internet access can be used to send short messages from a microblog. Combined with social network environments, applications such as Twitter, Jaiku, Pownce, and Plurk enrich our ways of communication. At Graz University of Technology (TU Graz) some research work has been done to investigate using microblogging tools to improve face-to-face lectures. The study described in this paper took place at a large international conference on e-learning, where a Twitter channel was established for discussion among the participants of the conference and those from outside. This stream was also used to “tweet” (post on Twitter) statements during the keynote presentations. By viewing the tweet-channel via an additional projector, the audience was able to follow the live-blogging session synchronously to the ongoing speech.
View
Interaktion zwischen Lehrenden und Lernenden in Medien unterstützten Veranstaltungen
Chapter
Full-text available
  • Jun 2007
Bereits seit längerer Zeit haben eLearning und Multimedia Einzug in die universitäre Lehre gehalten. Die Einsatzmöglichkeiten und -szenarien sind dabei vielfältig und reichen von der Verwendung von PowerPoint-Folien bis hin zu eLearning-Kursen, die komplett online bearbeitet werden. Die Lernarrangements müssen sich jedoch mit den folgenden Fragestellungen auseinandersetzen: Können die Lernenden den Ausführungen folgen? Ist das Vortragstempo angemessen? Reicht das Vorwissen der Studierenden aus? Haben die Teilnehmer die Ausführungen inhaltlich verstanden? Sind die Lerninhalte zielgruppengerecht aufbereitet? Passen die Lerninhalte zur Zielgruppe? Diese Fragestellungen können in der Regel im Rahmen der Interaktion zwischen Dozenten und Studierenden gelöst werden. Interaktion und Reaktion sind daher entscheidende Bestandteile der Wissensvermittlung. Allerdings zeigt die Erfahrung, dass beides aus unterschiedlichen Gründen in Lernarrangements nicht ausreichend oder gar nicht zu Stande kommt. Die folgenden Ausführungen behandeln im ersten Teil das Problemfeld Interaktion von Lehrenden und Lernenden in unterschiedlichen Veranstaltungsarrangements. Dazu werden drei typische Arten der Wissensvermittlung in den Vordergrund gestellt, mit denen am Lehrstuhl für BWL und Wirtschaftsinformatik der Universität Würzburg bereits hinreichend Erfahrung gesammelt wurde. Es handelt sich um (multimedial unterstützte) Präsenzveranstaltungen, Televorlesungen und Blended Learning Arrangements. Auf Basis der gewonnenen Erkenntnisse wurde am Lehrstuhl Prof. Thome ein Werkzeug entwickelt, das die auftretenden Schwierigkeiten hinsichtlich Feedback und Interaktion zumindest in Teilen löst und jetzt seit einem Jahr produktiv im Einsatz ist. Eine Vorstellung dieses Werkzeuges und ein Erfahrungsbericht sind Gegenstand des zweiten Teils.
View
View