Conference PaperPDF Available

Automatic Authentication of Students at an Interactive Learning-Video Platform

Conference Paper

Automatic Authentication of Students at an Interactive Learning-Video Platform

Abstract and Figures

There are many reasons for the implementation of authentication on learning platforms. For instance, it is required for the teachers to identify individual students if the learning platform offers some kind of assessment. In addition, authentication is the base for a successful monitoring of the attendance of the students. Compulsory attendance is often applied because many positive effects have been reported by authors of several studies. To monitor the attendance at online videos interactive components can be applied. Another benefit of such interactive components is that they help retain the attention of the students. This study was carried out to examine how the students used a video learning platform which provides interactive components of the videos as part of a course offered at Graz University of Technology. Up until now, a major drawback of this platform has been that the students have been required to register manually on the platform. Now, students are able to use the platform without manual registration and authentication, because these steps happen automatically via the main learning platform that provides all of the course materials. Furthermore, the course design and the application of the interactive components, presenting multiple-choice questions, are evaluated. It is pointed out that the concept improves the performance of the students and equips the teacher with valuable feedback regarding the students' interests.
Content may be subject to copyright.
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Automatic Authentication of Students at an Interactive
Learning-Video Platform
Josef Wachtler
Educational Technology
Graz University of Technology
Austria
josef.wachtler@tugraz.at
Marco Scherz
Wor king Grou p Sus tain able Cons truc tion
Institute of Tec hno logy an d Te stin g o f C onst ruc tion Mat eri als
Graz University of Technology
Austria
marco.scherz@tugraz.at
Martin Ebner
Educational Technology
Graz University of Technology
Austria
martin.ebner@tugraz.at
Abstract: There are many reasons for the implementation of authentication on learning platforms.
For instance, it is required for the teachers to identify individual students if the learning platform
offers some kind of assessment. In addition, authentication is the base for a successful monitoring
of the attendance of the students. Compulsory attendance is often applied because many positive
effects have been reported by authors of several studies. To monitor the attendance at online videos
interactive components can be applied. Another benefit of such interactive components is that they
help retain the attention of the students. This study was carried out to examine how the students
used a video learning platform which provides interactive components of the videos as part of a
course offered at Graz University of Technology. Up until now, a major drawback of this platform
has been that the students have been required to register manually on the platform. Now, students
are able to use the platform without manual registration and authentication, because these steps
happen automatically via the main learning platform that provides all of the course materials.
Furthermore, the course design and the application of the interactive components, presenting
multiple-choice questions, are evaluated. It is pointed out that the concept improves the
performance of the students and equips the teacher with valuable feedback regarding the students’
interests.
Introduction
Authentication on learning platforms is often required for different reasons. On the one hand, authentication is a
necessary base for monitoring the attendance of students if it is compulsory. On the other hand, such platforms could
be used for assessment and, for that reason, because of that it is required that the teacher knows which answer to the
questions of the assessment is from which student.
Compulsory attendance is common in many courses of high education because it is considered to be helpful for the
students if they attend the classes (Rodgers 2002). This assumption is supported by the findings of different studies.
For example, it is claimed by (Park & Kerr 1990) that the most effective strategy students could use to avoid a
negative grade is to attend as many of the in-class sessions as possible. This claim has also been supported by a
study of (Romer 1993). Complete attendance has been claimed to result in a second-best grade on average. In
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
contrast, students who attend only one-quarter of the in-class sessions are prone to obtain the third-best grade.
Similar findings have been reported by (Devadoss & Foltz 1996), which states that full attendance increases the
grade by a full degree in comparison with an attendance of 50% of the class sessions. In addition, it has been shown
that occasional attendance checks are helping the students to feel more strongly supported by the teacher (Bai &
Chang 1996).
A possible solution to apply compulsory attendance to online learning-videos is the application of interactive
components. The study by (Wachtler & Ebner 2017) showed that it is possible to reach the same accuracy of
monitoring attendance on online video learning platforms as in standard classroom settings. A further benefit of the
interactive components is that they are helpful to address the fact that students are confronted daily with many
pieces of text, figures, colours and shapes in steadily increasing amounts. It seems to be obvious that students are not
able to process these huge amounts of information. They can only handle limited amounts at once (Shiffrin &
Gardner 1972) and, because of that a large part is filtered out centrally (Moran & Desimone 1985). This mechanism
is called selective attention (Heinze et al. 1994), which is considered to be the most crucial resource for human
learning. It has been shown that managing this attention is important for the learning success (Ebner et al. 2013).
To achieve such a management of the attention of the students a tool is required. It has been shown that interaction
and communication play important roles in attention management. It is recommended that such interactive
components should be provided in all directions and forms. This means that possibilities to communicate in an
online course should not only be offered from the teacher to student and vice versa, but it should also be possible for
the students to interact with the content itself. In addition, it is important that there are methods of communication in
synchronous ways (e.g. online chats) and asynchronous ways like e-mail or forums. The interactive components can
also be used for assessment purposes (Wachtler et al. 2018) (Carr-Chellman & Duchastel 2000) (Ebner & Holzinger
2003).
A video platform that was first introduced by Ebner et al. (2013) and which is named LIVE (Live Interaction in
Virtual learning Environments) has been developed and is used at Graz University of Technology to monitor
attendance and support attention management. This platform has been used in lectures over the past three years. In
the first year, the main goal was to evaluate whether it was possible to monitor the students’ attendance (Wachtler &
Ebner 2017). In the second year, a focus was placed on evaluating the students’ performance during the interactions
offered, which were providing questions and polls (Wachtler et al. 2018). At these two usages the students were
required to register and to authenticate at the platform manually. For that, a username was provided for each student
by the teacher, which enabled the teacher to identify them easily. Because some students ignored the requirement to
use the provided username a way to link the platform directly to the learning platform of the university has been
developed to avoid this problem. This study presents this solution and evaluates its application at the course taught
in 2018. Furthermore, the didactic concept used during the second year was improved. Because of that the research
problem addressed by this study could be summarized to “evaluating the automatic authentication at a platform for
learning-videos in combination with interactive components”.
At first some related work is presented. After that, the video platform that provides the interactive components is
explained so readers can gain a better understanding of the context. This is followed by a detailed description of the
implementation of the process of automatic authentication. The outcome of the evaluation is presented after
explaining the course design. The results of the evaluation are then discussed. Finally, the main parts of the study are
summed up.
Related Work
A main goal of using authentication on online learning platforms is to monitor attendance. To evaluate the online
approach, it has to be compared with standard offline methods. A common way to monitor the attendance of the
students in a classroom or a lecture theatre is the usage of a signature list. Some problems are related to this
approach. First, a student could easily leave the lecture after signing the list or one student could sign the list on
behalf of many others. These problems are even more common in larger classes. To avoid these drawbacks, it is
required to place some personnel at the doors of the classroom to oversee the process of signing and leaving.
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Furthermore, it is possible to check the ID cards of the students. Another approach includes the usage of an ARS
(Audience Response System). In this scenario, the students will have to authenticate at the system and their answers
are used as a proof of their attendance. It seems to be obvious that this only works if the ARS is used regularly
during the lecture.
Some platforms presenting learning-videos are offering interactive components or even focusing on them in detail.
Such a platform was introduced by (Kovacs 2015). This platform segments the learning-video into parts that are
related to specific questions. This approach places the students in a position to watch the video driven by the
question. This means that they could decide, based on the question, if they want to watch the segment or not. The
evaluation of this platform pointed out that the performance of the students is significantly better at the interactive
videos in comparison to normal videos. Another platform for interactive learning-videos which presents embedded
questions was introduced by (Cummins et al. 2016). This platform was evaluated a course implementing a flipped
classroom. The performance of the students was reported to increase as the interactive components were used in
several ways. Because of that the author strongly recommends the application of such interactive questions.
Interactive Video Platform
As mentioned above, a web platform supporting learning-videos with interactive components implements the
automatic authentication. The platform is implemented using the programming language Python and the Django-
Web-Framework. LIVE provides the possibility to the teacher to embed different types of questions and other
interactive components in a video. In addition, many features of analysis are offered.
It seems to be obvious that a platform with the requirement to authenticate needs a management of the users.
Because of that four different groups of accounts are provided (Wachtler & Ebner 2014a) (Wachtler & Ebner 2017):
Ordinary students are only able to watch the videos, participate by answering the interactive questions and
analyse their own performance.
Users with teacher-privileges are additionally allowed to create video events with interactions. Moreover,
they can evaluate the performance of those who attend events they create.
Researchers have the clearance to download all the data generated by LIVE in the form of spreadsheets.
To manage the privileges of the users as well as to change the settings of the web platform, there are also
Administrators.
As indicated, the web platform enriches videos with different methods of interaction. As shown by (Fig. 1) a video
is shown to the students. It can be seen that the video is currently interrupted by an interaction. In this example, a
multiple-choice question is presented. The occurrence of such an interaction pauses the video and because of that the
students are forced to react to the question in order to continue watching. This means that they at least have to
decline to answer or to submit an answer in the best case. On the right hand side of the video some control elements
are located. At the top there is a button that allows students to leave the event. Below this button, a bar is presented
which indicates how attentive the watching student currently is. For that the colours of a traffic light are used. The
analysis of the attention of the students is provided by the so-called “attention profiling algorithm” presented below.
After this analysis some control elements are shown which could be used by the students to invoke interactions by
themselves.
While creating the video, the teacher has to select the interactions which should be offered. The following
interactions are offered by LIVE (Wachtler & Ebner 2014a) (Wachtler et al. 2016b) (Wachtler & Ebner 2017):
Simple Questions
o general questions which are not related to the content of the video
o random and automatic
o useful for bridging a longer phase during which no content-related questions are asked
Solve CAPTCHAs
o used for the same reasons as the simple questions above
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Figure 1: A video is interrupted by a multiple-choice question
Text-Based Questions
o the teacher can ask text-based questions to the students
o at live-broadcastings, she/he could enter the question into a textbox and send it to the students
instantly
o in videos, she/he must place such a question at a specific position in the video before releasing it
Multiple-Choice Questions
o real multiple-choice questions or true/false questions
o before deploying the video, the teacher can add questions of this type at pre-defined timepoints
throughout the video
As mentioned the students can trigger interactions by themselves. To provide such functionalities, the teacher has to
activate some of the following possibilities while creating the video (Wachtler & Ebner 2014a) (Wachtler et al.
2016b) (Wachtler & Ebner 2017):
Ask Teacher
o the attendees can ask a question to the teacher by entering it into a text box
o to answer the question, the teacher could use a specific dialog or send it by e-mail
Report a Technical Problem
o the students can report a technical problem to the teacher via a dialog
o this feature is mainly used during live-broadcastings to report problems with the video feed
Set Attention
o with the help of a slider, the watchers can indicate their current level of attention
In addition to these possibilities of interactivity which are available while the video runs there is a function to offer
some interactivity after watching. This means that the teacher is able to create polls which are shown to the students
when they leave the video. To ensure that the students have enough knowledge to participate in the polls the teacher
has to state how much of the video has to have been watched before the poll appears. Each poll consists of a
question and several possible answers. The students must choose only one answer. In addition, they are able to enter
some text to explain their choice.
Attention-Profiling Algorithm
To record the students’ behaviour and assess their attention levels while watching a video, a so-called “attention
profiling algorithm” is implemented (Wachtler & Ebner 2014b) by LIVE. The algorithm is split into two parts. The
first part of the algorithm is used to track the behaviour of the students while they watch the video. In contrast to that
the second part calculates an attention level which tries to indicate how attentive the students are.
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
As the student watches the video, their behaviour is recorded and accordingly tracked. To achieve this detailed
monitoring, the following values are recorded (Wachtler & Ebner 2014b):
Absolute time of joining (e.g. 2018-07-21 13:45:27)
Relative time of joining (e.g. 0 minutes 0 seconds)
Absolute time of leaving (e.g. 2018-07-21 13:48:49)
Relative time of leaving (e.g. 3 minutes 22 seconds)
To support these recorded values the second part of the “attention profiling algorithm” is used. It calculates a
number ranging from 0% which means that the student is completely absent to 100%. The latter value expresses that
the student in question is fully attentive. The calculation is mainly based on the reaction times to the interactions
shown to the students during the video. Consequently, the attention level decreases if the reaction time increases.
The combination of the two parts of the algorithm enables the teacher to fully monitor the attendance of the students.
This means that the recording of the watched timespans states what was watched and the attention level indicates
how serious the watching was.
Learning Analytics
LIVE provides many functionalities that can be used to analyse the performance of the students. This analysis
consists of two parts. The first one includes evaluations of the students’ behaviour while watching. For that, the
values recorded by the “attention profiling algorithm” described above are visualized. The second part is responsible
for analysing the performance of the students at the interactive components.
To obtain an initial overview of which parts of the video were watched, the teacher could use the timeline analysis
(Fig. 2), which provides a diagram as output. Along the x-axis it prints the timeline of the video and in contrast to
that the y-axis shows the number of students. The red line indicates how many times the video was viewed. This
information is supported by the green line which expresses the number of different users which are responsible for
the views. Below the diagram, the exact numbers are printed. For that, the vertical cross-hair has to be moved along
the x-axis.
Figure 2: A timeline showing the number of students watching a video.
In addition to this overview there is a list of all students who attended to the video. Data are given on how much of
the video each student watched. By referring to this list, the teacher to monitor the students’ attendance. When
clicking on a name of a student in this list a detailed analysis of the selected student is printed (Fig. 3). In this
analytical output, the timeline represents the video. A red bar represents each part of the video watched. By hovering
the bars with the mouse pointer, detailed information is shown. This includes the exact values of the watched
timespan and the calculated attention level.
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Figure 3: The visualisation of the recorded watched timespans
The second part of the offered analysis features consists of the evaluation of the interactions. Different lists are
available for each type of interaction. In general, each list prints the questions and the answers of the students in a
detailed way. It seems to be obvious in the case of multiple-choice questions the correctness of the answers could be
evaluated automatically. The text-based questions, however, have to be checked manually by the teacher.
Automatic Authentication
To implement the automatic authentication feature for LIVE a standard named LTI (Learning Tools Interoperability)
is used. This standard was developed by the IMS Global Learning Consortium in order to connect different learning
platforms. This means that one such platform could use a different platform which provides other functionalities.
There are three specific entities involved as shown by (Fig. 4). The first one is the student who uses a learning
platform (e.g. Moodle). This type of learning platform is called an LTI consumer and uses a third-party learning
tool. The student has to be registered and authenticated at the learning platform. In this study, the students were
required to use the learning platform to access all of the course materials, including interactive videos. In addition,
they were able to use the account details of the official student management system of the university. This means
that their names and e-mail addresses were available for the learning platform. The interactive videos were
presented to the students using LIVE, which is called a learning tool. This learning tool is the equivalent of the LTI
provider. As mentioned above, LIVE requires registered and authenticated users. For that, the learning platform has
to authenticate the student at the learning tool in an automatically. The automatic authentication sends the required
user data to the learning tool, ensure that the student ends up at the learning tool registered and authenticated. To
provide these functionalities the means of the LTI standard are used. This standard ensures that the process of
authenticating happens in a secure way.
Figure 4: A student is automatically authenticated at a learning tool through a learning platform.
To add the functionalities of LTI to LIVE it has been required to implement an LTI provider. For that, a library has
been developed which provides such a provider to a project based on the Django-Web-Framework. With this library
it is possible to integrate an LTI provider by simply defining some settings. The developed library was released on
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Github in compliance with the terms of the MIT licence as open source: https://github.com/wachjose88/django-lti-
provider-auth
When building applications offering an LTI provider it is required to test them. For that, an LTI consumer is needed.
Such a consumer should be able to run locally to test the application which runs locally too. A simple version was
developed and released, because no functional LTI consumer for local usage was available:
https://github.com/wachjose88/local-lti-consumer
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Course Design
LIVE has been used in courses offered at the Graz University of Technology for the past three years (Wachtler et al.
2018). The collected experiences of these years in the lecture course Building Materials Basics - Laboratory
Practicals showed that LIVE could be conveniently and practically used by students and teachers. Because of the
holistic analysis and evaluation of the application and of the course design, steady improvements are possible. The
focus in this article is the improvement of the registration system by the extension of the automatic authentication by
a mechanism called Learning Tools Interoperability (see above). Next to the technical extensions of LIVE, in the
most recent course design allowed students to earn extra points (bonus task) by submitting own-created multiple-
choice questions which will be probably embedded in further videos. In addition, the same course is offered a
second time in the year because of the huge number of interested students. Due to that the design of the course was
slightly adapted for the second session.
In general, the course design established by (Wachtler et al. 2018) was maintained. Therefore, the course was
divided into theoretical and practical parts again. The theoretical content was presented in individual lectures. The
theoretical part was subdivided into the following subject areas:
1. Aggregate
2. Binders 1
3. Binders 2
4. Fresh Concrete
5. Hardened Concrete 1
6. Hardened Concrete 2
7. Steel
8. Synthetic Materials
As indicated above the lecture was additionally offered in the winter terms, because of the huge number of students
who had registered in the preceding summer terms. In (Fig.5) the numbers of registered students are shown. It is
clearly visible that the number of students is not very high in the winter terms.
Figure 5: Number of registered students in summer and winter terms
Because of that the design of the course Building Materials Basics - Laboratory Practicals, which had initially been
conceptualized to meet the needs of a large number of registered students (summer term), was adapted to meet the
needs of a smaller number of registered students (winter term). These two concepts are illustrated by (Fig. 6).
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Figure 6: Course design of the lecture Building Materials Basics - Laboratory Practicals for a large number of
students (left) and adapted course design for a small number of students (right)
In the practical part of the course (course design for a large number of registered students), material experiments
were demonstrated to students in the Institute of Technology and Testing of Construction Materials laboratories.
After the teachers presented these experiments, the students had to reproduce the experiments and create videos of
them. The videos were presented to the students and enriched with interactive components using LIVE. More
information and advantages about the course design were presented and evaluated by (Wachtler et al. 2018). To earn
extra points, the students were asked to submit own-created questions, which are useful for the understanding of the
content. With these submitted questions the teacher should be enabled to gain a better understanding of the topics
which are of more interest. Based on that, the questions included in the videos of the following year could be
adapted accordingly.
In the adapted course design for a small number of students, the teachers held a guided tour of each laboratory for
each of the single subject areas, rather than demonstrating experiments with construction materials. Furthermore, no
videos had to be produced by the students, because of the small number of students. New multiple-choice questions
were included in the existing videos of the summer term and were used to evaluate the students via LIVE.
Evaluation
This section presents the results of the lecture for the 2018 summer term by evaluating the data generated by LIVE.
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Evaluation of Automatic Authentication
Prior to the 2018 summer term, the student registration procedure took place with usernames provided by the
teacher. This means that after the introductory lecture, the teachers created a username for each student. The
students were required to use the username provided to register at LIVE. In addition, the teacher told the students
about the importance to use the usernames provided in the introductory lecture because not using the provided
username will lead to a negative grade. The created usernames were issued to the students via the learning platform
together with a link to the registration form of LIVE where the students were required to register using the
usernames provided.
Because the teacher is required to identify the students to evaluate their performance, the usernames were created
using a schema that enabled such identification. The mentioned schema is illustrated by (Fig. 7). It can be seen that
the names of the students were used in combination with a running number.
Figure 7: Requirements for usernames
With the adapted registration system, it is no longer required to use such usernames because the students are logged
in at LIVE automatically via the learning platform. The names and e-mail addresses of the students from the official
student management system of the University are used at LIVE. In this way, the teacher is easily able to identify the
individual students.
In the previous year (summer term of 2017), 12 students could not be identified because these students did not use
the provided usernames. This corresponds to 8% (12 out of 150 registered students) more negative marks due to
their disregard of the registration requirements. In comparison to that in this summer term, there were no negative
grades due to unidentifiable students choosing an arbitrary username.
Evaluation of Multiple-Choice Questions
In each video four multiple-choice questions were placed presenting content related questions to the students in a
way that was similar to the previous years (Wachtler et al. 2018). Again, the answers of the students were evaluated.
These results of the multiple-choice questions are summarized by (Fig. 8).
The individual questions (4 questions per subject area) and the students’ answers are shown. The height of the bars
indicates the number of students who answered the questions. The green bars show the correct answers, and the
turquoise ones, the incorrect answers given by the students.
According to these results, question 3 (Q3) from the subject area Steel was the easiest one. This question was
answered correctly by 121 students and answered incorrectly by 2 students. The most difficult question was Q2 from
the subject area ‘Aggregate. This question was answered correctly by 97 students and answered incorrectly by 27
students.
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
After summing up the answer tendencies a statement about the individual subject areas can be made. The green bars
(Fig. 9) show the sums of all correct answers to the four questions within one subject area and the turquoise bars
show the wrong answers.
Figure 8: Evaluation of multiple-choice questions by individual question
Figure 9: Evaluation of multiple-choice questions by subject area
The results indicate that the questions in the subject area ‘Steelwere answered correctly most often (475 correct
answers and 17 wrong answers). The biggest problems occurred in the subject areas of ‘Aggregate’ (417 correct
answers and 79 wrong answers), Hardened Concrete 2’ (430 correct answers and 66 wrong answers) and Hardened
Concrete 1’ (439 correct answers and 61 wrong answers).
Evaluation of Submitted Multiple-Choice Questions (Bonus Task)
Out of 127 registered students, 121 students completed the course. This corresponds to a graduation rate of around
95%. Of these 121 students, 31 students performed the bonus task (creating a multiple-choice question for each
subject area). This corresponds to around 26%. These submitted questions were compared with the questions created
417$ 449$ 449$ 455$ 439$ 430$
475$ 462$
79$ 39$ 35$ 33$ 61$ 66$ 17$ 26$
0$
50$
100$
150$
200$
250$
300$
350$
400$
450$
500$
550$
Aggregate$ Binders$1$ Binders$2$ Fresh$Concrete$ Hardened$
Concrete$1$
Hardened$
Concrete$2$
Steel$ Synthetic$
Materials$
Answers$[n]$
Subject$Areas$
Correct$Answer$ Wrong$Answer$
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
by the teachers, with the aim of identifying the students' interests. Teachers could also identify which content
appeared to be particularly relevant to the students during the experiments.
An overview of the studentssubmitted multiple-choice questions is presented by (Fig. 10).
Figure 10: Submitted multiple-choice questions of students (bonus task) and evaluation
In the subject area ‘Aggregate, the 31 students suggested none of the questions more than once. In the subject area
Binders 1, questions concerning the Blaine fineness were addressed 5 times. General questions about the different
testing possibilities in the field of binders were asked second most in frequency in this subject area (3 questions). In
the subject area Binders 2, the most frequent questions were asked about the Bending Tensile Strength (4
mentions). Issues of Compressive Strength and Heat of Hydration came in second place (2 questions each). In the
subject area ‘Fresh Concrete, 7 questions on the subject of Slump Flow were proposed. Also, questions were asked
about concrete properties at high temperatures with a high frequency (6 questions). There were 5 proposed questions
in the subject area ‘Hardened Concrete 1on the subject of the test specimens. Questions on the topic of Concrete
Storage occurred in second place with 4 mentions. With 3 proposed questions, the subject of testing with
Rebounding Hammer placed in third place. In the subject area ‘Hardened Concrete 2 10 questions were proposed
on Frost Resistance Testing. With 7 proposed questions, the topic of Water Penetration placed second in this subject
area. The storage of hardened concrete, with 3 more entries, came in third place. In the subject area ‘Steel, 16
questions were proposed on the subject of steel failures. Of these, 6 questions on the topic of general steel failures
and 5 questions each on the Ductile Fracture and Brittle Fracture were asked. The Charpy Testing Method was the
most frequently addressed topic in the subject area Synthetic Materials’ with 15 entries. General questions about
synthetic materials failures came in second place.
Since no topics were addressed more than once in the subject area ‘Aggregate, no statement can be made here about
the response tendencies of the students to the questions created by the teachers. In the subject area Binders 1’, it can
be seen that question 1 and question 2 (see Fig. 8 – Binders 1Q1 and Q2) were better answered than the other two
questions. Question 1 (Q1) addressed the Blaine Fineness, and question 2 (Q2) covered the topic of the Vicat
Testing Method. InBinders 2, no topics overlapped between the students’ and teachers’ questions. In the subject
area Fresh Concrete, there is also a tendency to recognize that those questions about the most frequently mentioned
topics of the students were most often answered correctly by the students (see Fig. 8 Fresh Concrete’ Q1). No
statement can be made about the response tendencies in the subject area Hardened Concrete 1’, as the teacher's
questions were inconsistent with those of the students. In the subject areaHardened Concrete 2, however, a clear
31$ 31$ 31$ 31$ 31$ 31$ 31$ 31$31$
23$ 23$
18$ 19$
11$
15$ 16$
0$
5$ 4$
7$
5$
10$
6$
15$
0$
3$ 2$
6$
4$
7$
5$
0$0$ 0$
2$
0$
3$ 3$
5$
0$0$
2$
0$ 1$ 0$ 1$ 0$
2$
0$
5$
10$
15$
20$
25$
30$
35$
Aggregate$ Binders$1$ Binders$2$ Fresh$
Concrete$
Hardened$
Concrete$1$
Hardened$
Concrete$2$
Steel$ Synthetic$
Materials$
Number$of$questions$
Subject$areas$
Submitted$multiple-choice$questions$ Different$proposed$questions$
Multiple$proposed$questions$(1st)$ Multiple$proposed$questions$(2nd)$
Multiple$proposed$questions$(3rd)$ Same$proposed$questions$as$teachers$
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
agreement can be seen. The topic of Frost Resistance Testing, about which the students also mentioned the most
questions, was most frequently answered correctly. In the subject areaSteel, the teachers did not ask any questions
about the topic of steel failures. As a result, no tendencies could be analysed, since the students questions o were
not congruent with those of the teachers. In the subject area Synthetic Materials, the question that addressed the
Charpy Testing Method was most frequently answered correctly. Thus, the students also answered questions
correctly that had been frequently mentioned more often in this subject area.
Discussion
As illustrated by (Fig. 5), not as many students enrolled in the winter terms in the lecture course due to conflicts with
other lecture courses offered as part of the bachelor’s programme. Since it was not possible to provide a large
number of teachers for a small number of registered students during the winter terms, a suitable solution was found
by using the adapted course design (Fig. 6). This design allowed students to complete the course during the winter
term without any loss in the quality of instruction or minimizing the course contents. However, this process only
works if videos created during the previous summer term have the required quality.
As stated above, the usage of the improved authentication system of LIVE no students received a negative grade
because of formal errors. This is considered to be a huge improvement, because the students are able to focus mainly
on the course content. In addition, the teacher’s workload is reduced because she/he is no longer required to create a
list of usernames for the students.
The high number of correct answers to the multiple-choice questions can most likely explained by the fact that the
students had the opportunity to watch all videos several times. The goal of LIVE in this sense is not to allow an
evaluation of test questions to be made, but to help students deal with the course contents in detail. After speaking
with the students, we determined that the most common approach taken involved the following steps:
1. Watching the videos
2. Answering the multiple-choice questions
3. Re-reading the teaching materials, if the student was uncertain about the answer
4. Re-watching the videos
5. Answering the multiple-choice question with additional knowledge
The results of the analysis from LIVE show that the students achieved a better grade-point average than they had
before using LIVE. The students may have felt more comfortable with the course content, because the questions in
the videos encouraged the students to provide a correct answer. This statement is supported by the most common
approach to watch the videos according to the report of the students (see above).
By comparing the additional questions created by the students (bonus task) and the answers to the questions created
by the teachers, it is recognizable that, in particular, the topics that were frequently addressed in the students’
questions were answered more correctly than others. This indicates that the submitted questions were mainly related
to topics that were of high interest to the students. By providing a bonus task, the studentsareas of interest could be
identified. These findings allow teachers to more concretely adapt their lectures to the needs of students, but also
focus on important topics that were not addressed by the students.
Conclusions
The on-going development of a learning tool which supports videos with interactive components is presented. This
tool embeds different types of questions in videos in order to help the students remain attentive while watching the
video. For a better usage of the tool in combination with other learning platforms, automatic authentication through
such a third-party learning platform has been implemented by using the concepts of Learning Tools Interoperability.
This is considered to be an important feature because the learning tool requires registered and authenticated users.
To evaluate the performance of this automatic authentication the tool was used in a lecture at Graz University of
Technology on the basis of a setting that had been developed earlier (Wachtler et al. 2018). It was pointed out that
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
with the integration of the automatic authentication there were no negative grades due to unidentifiable students.
Furthermore, the administrative workload of the teacher was reduced.
The course design was adapted only in a minor way. Still, several embedded questions were answered by the
students with a high degree of accuracy. we also observed that the questions influenced the way the students learned,
because they tried to answer the questions correctly. The students were also asked to submit questions which helped
them understand the lecture content. These questions were used to identify topics of interest to the students.
The course is anchored in the curriculum of the Bachelor Degree Programme for the next few years, and thus the
course will continue to be planned in this form. For that, it is planned to adapt the questions and the focus of the
lecture according to the insights gathered at the results from the presented evaluations.
References
Bai, Y., & Chang, T. S. (2016). Effects of class size and attendance policy on university classroom interaction in Taiwan.
Innovations in Education and Teaching International, 53(3), 316-328.
Carr-Chellman, A., & Duchastel, P. (2000). The ideal online course. British Journal of Educational Technology, 31(3), 229-241.
Cummins, S., Beresford, A., & Rice, A. (2016). Investigating engagement with in-video quiz questions in a programming course.
IEEE Transactions on Learning Technologies, (1), 1-1.
Devadoss, S., & Foltz, J. (1996). Evaluation of factors influencing student class attendance and performance. American Journal
of Agricultural Economics, 78(3), 499-507.
Ebner, M., & Holzinger, A. (2003). Instructional Use of Engineering Visualization: Interaction Design in e-Learning for Civil
Engineering. Humancomputer interaction, theory and practice, 1, 926-930.
Ebner, M., Wachtler, J., & Holzinger, A. (2013). Introducing an information system for successful support of selective attention
in online courses. In Universal Access in Human-Computer Interaction. Applications and Services for Quality of Life (pp. 153-
162). Springer Berlin Heidelberg.
Ebner, M., Lackner, E., & Kopp, M. (2014, October). How to MOOC?-A pedagogical guideline for practitioners. In The
International Scientific Conference eLearning and Software for Education (Vol. 4, p. 215). " Carol I" National Defence
University.
Haintz, C., Pichler, K., & Ebner, M. (2014). Developing a Web-Based Question-Driven Audience Response System Supporting
BYOD. J. UCS, 20(1), 39-56.
Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs, H., Scholz, M., Münte, T. F., ... & Hillyard, S. A. (1994). Combined spatial
and temporal imaging of brain activity during visual selective attention in humans. Nature.
Helmerich, J., & Scherer, J. (2007). Interaktion zwischen lehrenden und lernenden in medien unterstützten veranstaltungen. In
Neue Trends im E-Learning (pp. 197-210). Physica-Verlag HD.
Khalil, H., & Ebner, M. (2013). Interaction Possibilities in MOOCsHow Do They Actually Happen. In International Conference
on Higher Education Development (pp. 1-24).
Kovacs, G. (2015, April). QuizCram: A Question-Driven Video Studying Interface. In Proceedings of the 33rd Annual ACM
Conference Extended Abstracts on Human Factors in Computing Systems (pp. 133-138). ACM.
Moran, J., & Desimone, R. (1985). Selective attention gates visual processing in the extrastriate cortex. Frontiers in cognitive
neuroscience, 229, 342-345.
Park, K. H., & Kerr, P. M. (1990). Determinants of academic performance: A multinomial logit approach. The Journal of
Economic Education, 21(2), 101-111.
Draft originally published in: Wachtler, J., Scherz, M. & Ebner, M. (2019). Automatic Authentication of Students at an Interac-
tive Learning-Video Platform. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp. 569-582). Amster-
dam, Netherlands: Association for the Advancement of Computing in Education (AACE). Retrieved July 1, 2019 from
https://www.learntechlib.org/primary/p/210069/.
Rodgers, J. R. (2002). Encouraging tutorial attendance at university did not improve performance. Australian Economic Papers,
41(3), 255-266.
Romer, D. (1993). Do students go to class? Should they?. The Journal of Economic Perspectives, 7(3), 167-174.
Shiffrin, R. M., & Gardner, G. T. (1972). Visual processing capacity and attentional control. Journal of experimental psychology,
93(1), 72.
Wachtler, J., & Ebner, M. (2014a, June). Support of Video-Based lectures with Interactions-Implementation of a first prototype.
In World Conference on Educational Multimedia, Hypermedia and Telecommunications (Vol. 2014, No. 1, pp. 582-591).
Wachtler, J., & Ebner, M. (2014b). Attention Profiling Algorithm for Video-Based Lectures. In Learning and Collaboration
Technologies. Designing and Developing Novel Learning Experiences (pp. 358-367). Springer International Publishing.
Wachtler, J., & Ebner, M. (2015, June). Impacts of interactions in learning-videos: A subjective and objective analysis. In
EdMedia: World Conference on Educational Media and Technology (Vol. 2015, No. 1, pp. 1611-1619).
Wachtler, J., Khalil, M., Taraghi, B., & Ebner, M. (2016a). On Using Learning Analytics to Track the Activity of Interactive
MOOC Videos. In Proceedings of the workshop on Smart Environments and Analytics in Video-Based Learning (SE@ VBL),
LAK2016.
Wachtler, J., Hubmann, M., Zöhrer, H., & Ebner, M. (2016b). An analysis of the use and effect of questions in interactive
learning-videos. Smart Learning Environments, 3(1), 13.
Wachtler, J., & Ebner, M. (2017, June). On Using Interactivity to Monitor the Attendance of Students at Learning-Videos. In
EdMedia: World Conference on Educational Media and Technology (pp. 356-366). Association for the Advancement of
Computing in Education (AACE).
Wachtler, J., Scherz, M., & Ebner, M. (2018, June). Increasing Learning Efficiency and Quality of Students´ Homework by
Attendance Monitoring and Polls at Interactive Learning Videos. In EdMedia+ Innovate Learning (pp. 1357-1367). Association
for the Advancement of Computing in Education (AACE).
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
Due to the fact that students are confronted with a growing amount of texts, colours, figures and shapes and due to their ability to process only a limited number of such information simultaneously, it seems to be obvious that efforts should be made to increase the students' attention-levels. This is important because research results have indicated that selective attention is considered as the most valuable resource in the process of human learning. The application of interaction and communication to the process of learning is a useful strategy to direct the students' attention. It seems to be obvious that this is also true for learning videos. Therefore, this work contains a description of how a video platform with interactive components can be used to support the students and teacher. The video platform is explained and evaluated by analysing its usage in a large teaching course at an institution of higher education. The application of this strategy improved the students' performance and optimized the teacher's workload.
Conference Paper
Full-text available
Many studies are claiming that compulsory attendance has its benefits for students because it is positively influencing their performance in consequence. This means that fully attending students are receiving a better grade than those who skip parts of the course. Whereas there are existing technologies to monitor the attendance of students in standard classroom situations the possibilities to provide such functionalities at online videos are limited. Because of that this work introduces an approach to assess the performance of students at videos. For that a web-platform which offers a detailed recording of the watched parts of videos is presented. The evaluation shows the performance of the students at a course consisting of videos with compulsory attendance and furthermore it points out that the approach for monitoring the attendance at online videos is basically working.
Article
Full-text available
This study focuses on the positioning of interactive questions within learning videos. It is attempted to show that the position of a question’s occurrence has an impact on the correctness rate of its answer and the learning success. As part of the study, the interactive learning videos in which the questions are placed are used as teaching materials with a class. The pupils have been working with the videos for around one month and some interesting results could be obtained. It is shown that questions which are asked too early in the videos are answered incorrectly more often than other questions. This manuscript also recommends an adequate positioning of the first question in learning videos. The new hypothesis that the length of intervals between popping up questions plays a minor role at rather short learning videos is constructed in this publication. Moreover, the positive impact on the long-term learning success of the participants of learning videos is determined.
Conference Paper
Full-text available
It is widely known that interaction, as well as communication, are very important parts of successful online courses. These features are considered crucial because they help to improve students’ attention in a very significant way. In this publication, the authors present an innovative application, which adds different forms of interactivity to learning videos within MOOCs such as multiple-choice questions or the possibility to communicate with the teacher. Furthermore, Learning Analytics using exploratory examination and visualizations have been applied to unveil learners’ patterns and behaviors as well as investigate the effectiveness of the application. Based upon the quantitative and qualitative observations, our study determined common practices behind dropping out using videos indicator and suggested enhancements to increase the performance of the application as well as learners’ attention.
Conference Paper
Full-text available
QuizCram is a question-focused format for navigating and reviewing lecture videos. QuizCram shows users a question to answer, with an associated video segment. Users navigate through the video segments by answering questions. We also allow users to review using a timeline of previously answered questions and videos. To encourage users to review questions, QuizCram keeps track of their question-answering and video-watching history and recommends users to review questions they have not fully mastered. QuizCram-format courses can be generated automatically from lectures with in-video quizzes, though the format is flexible enough to accommodate multiple questions per video segment. Our user study comparing QuizCram to in-video quizzes found that users practice answering and reviewing questions more when using QuizCram, and are better able to remember answers to questions they encountered.
Conference Paper
Full-text available
It is clear that a system known as selective attention is the most crucial resource for human learning (Heinze et al. 1994). Due to this fact a web based information system is developed to support the attention of the watchers of a learning video. This is done by enriching the video with different forms of interactions. Among others there are interactions presenting multiple-choice questions at predefined positions in the video as well as randomly occurring interactions displaying general questions, which are not content related. To gain a basic plan which points out how to distribute the interactions over the video the usage of the information system at a lecture is analyzed. For that on the one hand some feedback of the users is provided and on the other hand an objective analysis of the results of the multiple-choice questions is done by evaluating their time of occurrence in the video.
Article
Full-text available
In-video quizzes are common in many distance learning platforms, including those from Coursera and EdX. However the effectiveness of in-video quizzes has not previously been assessed. In this paper we describe the construction and instrumentation of an Interactive Video Lecture Platform to measure student engagement with in-video quizzes. We also investigate the use of in-video quizzes as an approach to mitigate the lack of feedback available to students and lecturers in videos and traditional lectures. Finally, we evaluate the effectiveness of augmenting video with the ability to answer and receive feedback to quiz questions embedded directly within the video. We observed that student engagement with in-video questions was consistently high (71-86%) across two cohorts (N1=81, N2=84) with a rate of 1 question per 8.7 minutes of video. We identified three broad levels of engagement with the quiz questions and four motivations, including challenge seeking and completionism, which explain some of the observed behaviour. The results from this investigation demonstrate that in-video quizzes were successful in creating an engaging and interactive mode of content delivery. We recommend that in-video quizzes be used to increase the interactivity of video content as well as supporting formative assessment within a flipped classroom environment.
Article
Full-text available
Using a qualitative response analysis, the authors estimated and compared key determinants of course grades with those derived with OLS.
Conference Paper
Full-text available
In general videos have a more or less consuming character without any interaction possibilities. Due to this fact a web-based application is developed which offers different methods of communication and interaction to a certain learning-video. This should help attendees to avoid that they become tired and annoyed. The lecturer is able to use interactions to omit whether learners are able to understand the content of the video. For instance the developed web-application offers the possibility to add multiple-choice-questions at predefined positions. Furthermore there are many different kind of analysis as for example a detailed attention-profile. In this publication the implementation of a first prototype is described as well as a first field study which points out that learners? interactions and engagements increased arbitrarily.
Article
Classroom interaction experience is one of the main parts of students’ learning lives. However, surprisingly little research has investigated students’ perceptions of classroom interaction with different attendance policies across different class sizes in the higher education system. To elucidate the effects of class size and attendance policy on classroom interaction, we administered a survey to 1304 undergraduates in Taiwan. The questionnaire consisted of six dimensions: classmate supportiveness, classmate pressure, teacher encouragement, teacher supportiveness, study preparedness and class participation. The results indicated that attendance policy and class size have an interaction effect only on the teacher trait. Students in a small class with a never-taking attendance policy feel more supported and encouraged by their teacher compared with those in larger classes with the same attendance policy. When class size is larger, students in classes employing an occasionally taken attendance policy feel more supported from their instructors than those in the never-check attendance policy class. Attendance policy doesn’t influence the student or the classmate trait, but class size does. Larger classes lower classmate supportiveness, student preparedness, and class participation compared with smaller classes.