ArticlePDF Available

Uncovering student learning profiles with a video annotation tool: Reflective learning with and without instructional norms

Article

Uncovering student learning profiles with a video annotation tool: Reflective learning with and without instructional norms

Abstract and Figures

This study explores the types of learning profiles that evolve from student use of video annotation software for reflective learning. The data traces from student use of the software were analysed across four undergraduate courses with differing instructional conditions. That is, the use of graded or non-graded self-reflective annotations. Using hierarchical cluster analysis, four profiles of students emerged: minimalists, task-oriented, disenchanted, and intensive users. Students enrolled in one of the courses where grading of the video annotation software was present, were exposed to either another graded course (annotations graded) or non-graded course (annotations not graded) in their following semester of study. Further analysis revealed that in the presence of external factors (i.e., grading), more students fell within the task-oriented and intensive clusters. However, when the external factor is removed, most students exhibited the disenchanted and minimalist learning behaviors. The findings provide insight into how students engage with the different features of a video annotation tool when there are graded or non-graded annotations and, most importantly, that having experience with one course where there are external factors influencing students’ use of the tool is not sufficient to sustain their learning behaviour in subsequent courses where the external factor is removed.
Content may be subject to copyright.
1
Uncovering student learning profiles with a
video annotation tool: Reflective learning with
and without instructional norms
Negin Mirriahi1, Daniyal Liaqat2, Shane Dawson3, and Dragan Gašević4
1University of New South Wales, Australia
2University of Toronto, Canada
3University of South Australia, Australia
4University of Edinburgh, United Kingdom
Introduction
Higher education institutions are increasingly adopting blended or digital learning
strategies to better meet the demands and expectations of prospective students
(Garrison & Kanuka, 2004; Gosper, Malfroy, & McKenzie, 2013; Graham,
Woodfield, & Harrison, 2013). At the same, there is a growing evidence base
demonstrating the impact of blended learning on student learning performance1.
In essence, blended learning approaches are seen to better promote academic
performance and higher order learning outcomes when compared to more
traditional and fully online modes of instruction (Al-Qahtani & Higgins, 2013;
Chen, Lambert, & Guidry, 2010; Torrisi-Steele & Drew, 2013). While blended
learning offers much potential to meet the challenges associated with a shifting
education landscape driven in part by changing student expectations, competing
demands on student time, as well as learning and teaching quality, there remain
questions regarding how students engage with such technologies to specifically
support their learning strategies and approaches (Lust, Elen, & Clarebout, 2013;
Lust, Vandewaetere, Ceulemans, Elen, & Clarebout, 2011). Although there are a
number of learning tools available for facilitating blended learning (e.g. blogs,

1For the purpose of the study presented in this paper, we have adopted Garrison and Vaughan's
(2008) definition of blended learning as the “integration of thoughtfully selected and
complementary face-to-face and online approaches and technologies” (p. 148).
2
wikis, and discussion forums), one particular technology that has gained in
momentum in blended learning settings is the use of video-based learning
techniques (Giannakos, Chorianopoulos, & Chrisochoides, 2014). While in some
blended learning models, students are asked to watch a lecture video prior to
coming to a face-to-face session, in other learning settings, students are asked to
watch video recordings of their own presentations or performances and provide
reflective comments on their perceived strengths and weaknesses. This study
focuses on the latter whereby students reflect on their video recorded
performance. A video annotation tool is used to facilitate students’ reflective
practice and promote self-regulated learning proficiency as they watch the video
recordings of the own performances. The primary aim of this study was to
investigate how students engage with the video recordings and video annotation
tool in a blended learning setting. The findings from the study have important
implications for future course design when integrating videos as an instructional
medium.
Students’ Engagement with Technology
At present, the learning management system (LMS) is one of the most commonly
adopted technologies for supporting course delivery in higher education today.
The LMS is essentially an aggregation of differing tools that support the provision
of content as well as student collaborations such as discussion forums or self-
reflection and assessment tools. Despite the vast number of learning technologies
accessible for the contemporary student cohort, not all students avail themselves
of these tools or engage them in a manner that effectively supports their learning
process (Lust et al., 2013, 2011; Yen & Lee, 2011). Drawing on the earlier work
of Winne (2006), Lust and colleagues (2013) pose two inter-connected reasons
why students’ choose not to engage with an educational technology – internal and
3
external conditions. Broadly speaking, the first relates to the low proficiency of
students’ self-regulated learning. Essentially, students’ self-regulated skills are
largely insufficient to enable them to identify when to engage with a particular
technology in order to support their learning (Lust et al., 2013). This premise was
well demonstrated in their study investigating student’s engagement with the
various tools available in a LMS in an undergraduate course.
Through cluster analyses, Lust and colleagues (2013, 2011) aimed to identify
patterns in students’ use of learning tools across two sequential phases of a single
course offered in a blended learning model. The course design consisted of series
of lectures that was complemented by the LMS and teacher support (instructor
and tutors). The first phase of the course focused on factual and comprehension of
concepts while the second phase related to the application of these learned
concepts. Lust et al. (2013, 2011) noted that based on the frequency and duration
of tool use and the types of tools adopted (e.g. lower order vs. higher order
learning) three clusters of students emerged in the first phase of the course
instruction. Namely: no-users, intensive users, and selective users. In the second
phase of the study a fourth cluster emerged, termed limited users. The authors
noted that the students in cluster one (the no-users) had a significantly lower level
of engagement with the learning tools than the students in cluster two (the
intensive users) and in cluster three (the selective users).
The intensive users were frequent users of multiple tools within the LMS,
particularly the practice quizzes. The selective users, in phase one, were more
exploratory in their engagement of the various tools opting primarily for basic
information and scaffolding type tools. However, for phase two, this particular
cluster strategically accessed the video lectures and attended face-to-face support
sessions showing signs of goal-orientation. Finally, the limited users, a cluster
4
emerging only in the second phase of the course, used the online learning tools
significantly less than both the intensive and selective users and focused most of
their effort in attending face-to-face lecturers and learning support sessions.
Temporal analyses over the duration of phases one and two revealed that most
students across all three clusters (identified in phase one) transitioned into the
limited users cluster during phase two. This suggests that the students perceived
their adopted tool use for phase one was no longer applicable given the altered
course conditions in phase two. This shift in tool selection and engagement shows
signs of student self-regulation and agency towards their learning (Winne, 2006).
While students are active agents in their learning process, the choices they make
are influenced by both internal and external conditions. The work of Lust and
colleagues (2013, 2011) illustrates that the internal conditions, such as
metacognitive awareness, motivation, and prior knowledge, affect student tool
choice and application of these tools for learning. These choices are based on an
individual’s past experience and capacity to relate to the external conditions
associated with the course such as level of academic guidance and support
alongside the driving instructional context (e.g. learning activities, formative and
summative assessment tasks).
The discussion of internal conditions (self-regulated learning) influencing tool
choice and engagement segues to the second factor proposed by Lust colleagues,
that is, the instructor norms (i.e., external conditions). Research in factors
influencing students’ use of a LMS has shown that instructor norms can impact
upon students’ decisions to engage or not engage with an educational technology
(McGill & Klobas, 2009). For instance, if students are aware that the instructor of
the course perceives that the use of a particular technology is beneficial for their
learning there is a corresponding increase in the use of that particular technology
5
among the student cohort. This notion resonates with Perkins (1985) who posited
that students do not always engage with the learning opportunities presented or
features of technology if they are not fully aware of the gains they will achieve
from it. Further, the role of external conditions or factors such as assessment and
feedback can also influence student learning (Hattie & Timperley, 2007) and in
particular, ongoing and timely feedback specific to students’ performance can
promote meta-cognitive awareness and encourage students (Gibbs & Simpson,
2004). While there is much literature on the influence of assessment design and
feedback in general, to date there has been limited research investigating the role
such external factors play on students’ self-regulation of learning or agency in
terms of the choice and use of video technologies incorporated to support their
study. In particular, the effect of the interplay between internal and external
conditions on students’ adoption of video technology across multiple courses
involving different instructional conditions and pedagogical approaches is limited.
Hence, this study aims to address this gap in the literature by conducting cluster
analysis on students’ use of a video annotation tool that was adopted across four
courses. The subsequent cluster analyses are based on the different instructional
conditions encountered in the various courses. In this case, the presence or
absence of external conditions namely, graded self-reflection annotations.
Video Annotation Software
The integration of video content into online and blended courses is rapidly
becoming the norm for higher education (Yousef, Chatti, & Schroeder, 2014) with
an increasing number of video annotation software seeking to make the simple
transmission of video and audio content a more collaborative and dynamic
process (Cross, Bayyapunedi, Ravindran, Cutrell, & Thies, 2014). For example,
Aubert, Prié, and Canellas (2014) discuss various uses of video annotation in e-
6
learning contexts, particularly massive open and online courses (MOOCs) that
provide opportunities for students to annotate lecture videos or their own
recordings (e.g. self-reflection) individually or collaboratively. The development
of, and research associated with, these web-based video annotation technologies
has been available over the past several years. For instance, one of the initial
video annotation tools, the Microsoft Research Annotation System (MRAS) was
designed to aid student engagement through the use of note-taking (time-stamped
annotations) while viewing video content. An early experimental study using
MRAS demonstrated that students had preference towards using the annotation
tool with video content over more traditional note-taking within a live lecture
context (Bargeron, Gupta, Grudin, & Sanocki, 1999). More recently, the Media
Annotation Tool (MAT) has the additional features of a structured annotation
learning cycle whereby students can annotate a form of media, see their peers’ and
teachers’ comments, and then provide final reflective notes (Colasante & Fenn,
2009). A pilot case study incorporating the use of MAT was undertaken with pre-
service teachers specializing in physical education who were requested to
complete the annotation learning cycle by viewing videos of their own teaching
scenarios and those of their peers’ in order to enhance critical reflection in a
collaborative manner (Colasante, 2011). Survey, interview, and observational data
revealed that the majority of students valued the peer and teacher feedback
features of MAT and that the media annotations are effective for enhancing
student learning. Similarly, Rich and Hannafin (2008) reported on a variety of
video annotation tools being used by pre-service teachers, in particular to reflect
on their teaching practice and refine their skills. Hence, while video annotation
software is not a novel technology and the above and similar studies (Bargeron et
al., 1999; Colasante, 2011; Magenheim, Reinhardt, Roth, Moi, & Engbring, 2010)
7
have shown that students perceive video annotation technologies to be valuable
for their learning and reflective practice, they have relied heavily on self-report
data rather than more automated logged data that can be derived from the
student’s actual use of the technology. While self-reported data can shed light on
how students’ perceive technologies (e.g., typically to understand perceived
usefulness and ease of use), the methodology is subject to social desirability bias
where students may provide a desired response rather than the most accurate
response (Beretvas, Meyers, & Leite, 2002; Gonyea, 2005). Furthermore, a
reliance on students’ recall of previous behaviour with a specific technology can
lead to the collection of inaccurate data about the activities taken by learners while
using the technology (Winne & Jamieson-Noel, 2002). Moreover, due to
individual differences (e.g., metacognitive skills and motivation), students tend to
approach their learning differently (Winne, 2013). However, the identification of
these strategies is questionable due to the abovementioned potential inaccuracies
associated with self-reported data.
In contrast, learning analytics and data mining techniques are applied to extract
users’ actual behavioral data with technologies. Hence, learning analytics and
educational data mining can provide more objective data of a student’s actual use
of technologies in lieu of the individuals recall of their activity (Greller &
Drachsler, 2012). However, the use of data from students’ interactions with videos
and associated tools (e.g. video annotation software) to analyze their use and
engagement with videos is still at an early stage with few studies leveraging such
data to understand students’ actual experiences (Giannakos, Chorianopoulos, &
Chrisochoides, 2015). For example, in a study investigating students’ note-taking
behavior while watching lecture videos, Mu (2010) analysed the logged data
captured through students interactions with the specific technology. This data was
8
used to analyse the length and frequency of students’ notes. A further example
using data mining techniques for analyzing student use of video can be found in
the work of Brooks, Epp, Logan, and Greer (2011). These researchers applied
various data mining methods to analyse objective data from students’ engagement
with lecture videos to discover patterns in students’ use of the recordings. Using
k-means clustering, Brooks et al. (2011) revealed five types of student
engagement with video lectures: minimal active learners who rarely access the
videos; high activity learners who watch a portion of each lecture video on a
weekly basis; deferred learners who began to access the videos towards the
second half of the semester; and two clusters of just-in-time learners who
accessed the videos either only a week prior to the midterm exam or the week of
the midterm exam. By extending the studies by Mu (2010) and Brooks et al.'s
(2011), this paper advances the research in video analytics to specifically explore
students’ use of video annotation software for reflective purposes in differing
instructional conditions (graded vs. non graded) to identify patterns in students’
learning behaviour.
Learning Technology Usage Profiles
Derived from the research noted above, it would appear that student engagement
with a technology can be classified around particular learning profiles. For
example, Lust et al. (2013) identified four clusters (profiles) of student use with
the LMS. Similarly, Brooks et al. (2011) also noted four clusters based on student
engagement with lecture videos. In a further study undertaken by Phillips, Maor,
Preston, and Cumming-Potvin (2012) also investigating student use of lecture
video recordings, multiple user profiles were defined based on patterns of student
engagement with the various recordings. In this case, the profiles ranged from the
low level access of non-users and random users through to frequently accessed
9
profiles such as high-achieving and conscientious users. Essentially, these studies
suggest there is potential for logged data derived from student interactions with
technologies, to provide a measure of self-regulated learning proficiency and the
impact of external conditions on learning behavior and tool adoption. To date,
there have been few studies that have aimed to investigate the interaction between
the usage profiles and the instructional conditions (external conditions) of a
course of study. This study aims to address these deficits by examining the
profiles of students based on the data available from their use of a video
annotation tool when exposed to differing instructional conditions. Moreover, the
study looks at the effects of usage profiles on academic achievement of the
students. This is an important issue to investigate since self-regulated learning
skills are recognized as important for academic achievement (Pintrich & de Groot,
1990).
Research Questions
1. What are the main learning profiles that emerge from the use of video
annotation software?
2. Do different instructional methods influence the development of the learning
profiles identified based on student engagement or use of video annotation
software?
3. What is the effect of the learning profiles that emerge from the use of video
annotation software on students’ academic achievement?
Method
Setting and Sample
A case study approach was deemed the most appropriate research design given
that the data were collected from a single disciplinary area (performing arts) in
10
one higher education institution in North America (Eisenhardt, 1989) and the
researchers lacked any control over the behaviours of the students and instead
investigated learning and engagement within the natural context of the course
(Yin, 2009). Following institutional ethics approval, secondary log data from all
courses that used a locally-hosted video annotation tool called the Collaborative
Lecture Annotation System (CLAS), in the 2012-2013 academic year were
extracted. At this time, all potentially identifiable information such as username or
student number were transformed using a randomly generated code to ensure
student and instructor privacy. The secondary data extracted from the log files
only contained information relating to student and instructor interactions with the
video annotation tool. Hence, the conditions for ethics approval required that the
teaching context and approach for each course was inferred from the secondary
data collected involving the adoption of CLAS. Initial observation of the data
revealed that students in four of the courses used the tool for self-reflection
purposes (e.g. students described their performance and noted goals for
improvement), and that of these four courses, two incorporated graded assessment
of the student annotations as observed in the feedback text provided by the
instructors. Hence, the research team concluded that for two of the courses, the
use of the video annotation tool was not graded and hence, any student use was
optional and supplemental to the course. However, for the remaining two courses,
the reflective annotation activity was a graded component of the course in which
the students received instructional feedback on their reflections, offering them
guidance on how to improve their subsequent reflections. The CLAS tool was
developed by the institution at which the study was conducted. The CLAS tool
was used predominantly for the first time during the particular academic year
11
(2012-2013) and the students in this study did not have any prior experience with
the tool.
The participating courses in the study were situated in the performing arts
discipline and consistently involved students’ self-reflective annotations or
comments on their own performance. The study was restricted to the analysis of
the use of CLAS within the four performing arts courses. This decision was
informed by the unique feature of the video annotation tool affording students
opportunity to make time-stamped and general annotations on their individual
performance recordings (detailed below). Time and date stamps of the recorded
data further showed that two of the courses were offered in the first semester of
the year and two other courses offered in the subsequent semester. The data also
revealed that a proportion of the student cohort was enrolled in one of the courses
in the first semester and then progressed to one of two courses offered in the
subsequent semester. The randomly generated IDs for the students also revealed
that for one of the courses (Course 1) all students posted annotations to the same
set of videos. However, for the other three courses, each student posted
annotations to an individual video. Hence, it can be inferred that for Course 1,
students could see each other’s annotations and were annotating a group
performance. Conversely, for the other three courses, the students annotated their
own performance only and were therefore unlikely to have shared their reflective
posts with their peers. Furthermore, since the research team did not have any
control as to how the video annotation tool was used by the students nor how it
was integrated into the curriculum, such as a graded component or non-graded,
the study had the characteristics of a natural experiment (Dunning, 2012). Figure
1 below shows the pedagogical context and the progression of courses.
Course 1 (N=31) Course 3 (N=28)
12
(Semester 1)
Ungraded activity, social
(Semester 2)
Graded activity, individual
Course 2 (N=40)
(Semester 1)
Graded activity, individual
Course 4 (N=20)
(Semester 2)
Ungraded activity, individual
Fig.1
Pedagogicalapproachandprogressionofstudentsbetweentheundergraduatecourses
includedinthestudy
Learning environment – video annotation tool
At the higher education institution where this study occurred, a locally-hosted
video annotation tool, CLAS, was available for enhancing students’ experience of
watching recorded lectures, performances, or presentations by posting time-
stamped annotations and general comments for reflective practice or self-study
while viewing the recordings to develop their metacognitive skills (Authors, 2012;
Authors 2013). Since the tool was locally hosted, students’ interactivity with the
tool, or ‘mouse-click’ trace data, was captured and stored in the database. CLAS
allowed instructors to provide access to videos to students only enrolled in their
course and to restrict students from directly downloading the videos. This feature
ensures that any student and instructor activity with a video via CLAS can be
recorded and stored. Various types of data were captured, such as the number of
times students made, edited, or deleted an annotation, the use of various types of
video playing functions (e.g., play, pause, forward, and rewind), and the time a
user made their first and latest annotation. These types of data, analysed
collectively, can be used to provide insight into students’ learning profiles when
using video annotation tools. Figure 2 illustrates the various features of the video
annotation tool.
13
Fig.2
AscreenshotoftheinterfaceofCLAS,thevideoannotationsoftwareusedinthestudy
Variables
Of the various clickstream data captured by the video annotation tool, 12
particular variables, derived from the trace data logged by the video annotation
tool, were selected to represent students’ interaction with the tool and the different
ways they can choose to engage with this particular technology. The analysis of
the engagement data provides further insight into student learning profiles. The
following five variables measure the students viewing patterns based on how they
interact with the video control buttons. Such viewing patterns can show whether
students choose to view videos non-stop or spend time rewinding or fast-forward
to reach particular points in the video as well as how much of the videos they
view.
Fast-forward: The total number of times forwarding each video.
Rewind: The total number of times rewinding each video.
Non-stop: The total number of times activating the play button for the
entire duration of a video without transitioning to another function.
14
Pause: Total number of times pausing a video.
Time watched: Total amount of time each video has played. This variable,
however, has the limitation of only capturing data based on students’
mouse-clicks with the play button. While students may ‘play’ a video, it is
not known for certain whether they actually viewed the video or were
engaging with something else while the video played in background.
The following six variables relate to students use of the annotation functions in
the video annotation tool. The main additional feature of using the video
annotation tool rather than viewing the videos in any other video streaming
player, is the capability the tool provides for making time-stamped annotations
while viewing the video that can be revisited later.
Annotations total: Total number of annotations students make in each
video.
Annotations edited: Total number of times students edit annotations in a
video. This measure shows whether students write an annotation and later
go back and make a change or leave their annotation as it is.
Annotations deleted: Total number of annotations students delete in each
video. This measure shows if students return to an annotation and select to
delete it.
Videos annotated: Total number of videos students make at least one
annotation on.
Earliest annotation added: The earliest date and time each student made
their first annotation from the time that the video was available to them.
This measure represents whether the students waited a long time before
making their first annotation (the main feature of the tool and requirement
15
for some of the students) or whether students made their first annotation
right away.
Latest annotation added: The length of time from when the video was first
available to the time the last annotation was made by the student. This
variable, along with the earliest annotation added, shows whether students
were engaged with the activity of annotating the video during a long
period of time (i.e. if they made their initial annotation soon after the video
was available and their last annotation at a considerably later time) or if
their initial and final annotations were close in time.
Transition graphs were constructed for each student in each course of the study.
These graphs were used to gain a holistic view into the learning strategies adopted
by the students. Transition graphs were created from a contingency matrix in
which rows and columns were all events logged by the video annotation tool. The
rows denoted the start and the columns the end nodes of the transition edges. To
create a transition edge from event A to event B, number one was written in the
matrix cell intersecting row A and column B. There was a sequential increase in
the number in that cell for any future appearance of the edge from event A to
event B. To capture the temporal nature of video and reflect on the differences in
temporal distribution of different events captured by trace data (Authors, 2014),
events were associated with temporal quartiles of videos they belonged too (e.g.,
create annotation in quartile one, or pause in quartile two).
Density of the transition graphs was the final and 12th variable that was calculated
for each student in each course of the study. This variable shows the extent to
which students clicked on subsequent different functions within the video
annotation tool or the number of transitions they accumulate. The overall network
density is measured by considering all possible transitions between features of the
16
video annotation tool based on the total possible transitions across all students
(Hadwin, Nesbit, Jamieson-Noel, Code, & Winne, 2007). The more functions a
student clicked on, and the greater the number of transitions they had, the larger
the network density and metacognitive monitoring. As posited by Hadwin et al.
(2007), greater graph density of students’ activity shows that they are
experimenting with different learning strategies, and thus, have a higher level of
metacognitive monitoring activity. In contrast, a lower graph density illustrates
that the student has already selected key strategies to aid their learning process.
Hence, as the network density declines, there is parallel assumption that the
students’ metacognitive monitoring and self-regulated learning also declines.
Figure 3 illustrates two network graphs. The network graph at the top (a) shows
an example of a student with many transitions in the tool and hence more density
while the network graph on the bottom (b) shows an example where there are
fewer transitions. The density measure represents whether students tended to click
on many different functions while engaging with the video annotation tool or had
decided to use a few key features.
17
a)
b)
Fig. 3
Examplesoftransitionsgraphsoftwostudentsenrolledincourse2(a)andcourse4(b)of
thestudy,respectively.
18
Finally, the students’ grades from the four courses were used to gauge the effect
of the learning profiles on academic achievement.
Data Analysis Method
This study applied Ward’s (1963) hierarchical cluster method as it can effectively
uncover the underlying data structure without human intervention or having to
interpret or rely on self-reported data (Alexander, Jetton, & Kulikowich, 1995).
Clickstream data of students’ use of the video annotation tool and the density of
the transition graphs, specifically the 12 variables noted in the previous section,
were used to identify clusters of student user behaviour. To account for different
scales, all data were standardized. The dendrogram in Figure 4 illustrates the
results of the hierarchical cluster analysis revealing four clusters. Each line at the
bottom of the diagram represents a student. Students merge with other students or
groups to form larger groups. The height of the merged line represents the
dissimilarity between groups. A four-cluster solution minimizes intergroup
dissimilarity and maximizes intragroup dissimilarity.
Fig. 4
Dendrogramillustratingresultsofthehierarchicalclusteranalysis
Cluster A Cluster B Cluster C Cluster D
Cluster A
Cluster B
Cluster C
Cluster D
19
Consistent with del Valle and Duffy (2009), to confirm whether four clusters is an
accurate number, each of the 12 variables were compared across the four clusters.
Levene’s test for homogeneity of variance was undertaken to determine if
variances in the different clusters are equal. In cases where the assumption of
homogeneity of variance is not violated, one way ANOVAs were conducted.
Where Levene’s test resulted in a significant difference from homogeneity, the
non-parametric Kruskal-Wallis ANOVA was applied. The ANOVAs and
Kruskal-Wallis ANOVA tests identified significant differences for all 12
variables. Subsequent post-hoc tests (Tukey HSD and Mann-Whitney U pair-wise
comparisons) with the Holm-Bonferroni adjustment (to control for Type 1 error
rate due to multiple comparisons) revealed the significant differences between
pairs of clusters. To account for the differences between the four courses, the
comparisons between the identified clusters were performed within each of the
four courses.
Results
The cluster analysis identified a four-cluster solution as the most optimal and
meaningful for interpretation as illustrated in the dendrogram (Figure 4). The
distances of any additional clusters were too close to individual cases and thus
would unnecessarily over-fit the data. For the purpose of interpreting the
differences between the clusters and due to the non-normal distribution of the
data, medians along with the 25th and 75th percentiles are shown in Table 1 and
followed by Figure 5 illustrates the differences between the clusters based on
centred mean values (i.e. z-scores) of the 12 video annotation software usage
variables.
20
Table 1
TheDescriptiveStatistics(mediansand25thand75thpercentiles;1stand3rdquartiles)ofthe12
UsageVariables
Cluster A (N=37) Cluster B (N=44) Cluster C (N=16) Cluster D (N=22)
Variable Me
d
Q1, Q3 Me
d
Q1, Q3 Me
d
Q1, Q3 Me
d
Q1, Q3
Videos
annotated
3.00 2.00, 3.00 4.00 4.00, 5.00 4.00 3.00, 4.75 4.00 4.00,
5.00
Total
annotations
11.00 8.00,
18.00
73.00 58.00,
95.00
53.50 36.00,
62.50
65.50 54.50,
95.25
Annotations
deleted
0.00 0.00, 1.00 2.00 1.00, 4.00 0.00 0.00, 1.00 22.00 6.00,
38.25
Annotations
edited
0.00 0.00, 2.00 3.50 0.00, 6.00 0.00 0.00, 2.00 36.50 0.04,
2.28
Time
watched
1535.0
0
1022.00,
2294.00
4387.50 3465.50,
5789.50
1018.
00
312.50,
3690.25
11201.5
0
8327.50,
13066.00
Pause 19.00 7.00,
37.00
97.00 70.25,
127.50
19.50 8.25,
37.75
180.50 106.75,
292.00
Non stop 0.00 0.00,1.00 2.00 1.00, 4.00 0.00 0.00, 1.75 6.50 3.00,
12.00
Rewind 12.00 2.00,
27.00
32.00 20.00, 54.50 7.00 0.75,
15.50
110.00 76.50,
189.75
Fast forward 3.00 0.00, 6.00 22.00 15.75, 35.00 7.50 0.75,
10.75
37.50 12.00,
53.00
Density 0.16 0.13, 0.19 0.25 0.22, 0.28 0.09 0.05, 0.15 0.31 0.27,
0.34
Earliest
annotation
4683.0
0
3171.00,
5530.00
3474.00 1994.50,
5230.50
8786.
00
6471.00,
10371.00
3989.50 2985.25,
6376.75
Latest
annotation
8603.0
0
5729.00,
8770.00
10645.0
0
8851.25,
19249.75
4111
4.00
27647.75,
58399.75
11139.5
0
9264.75,
19572.25
Fig. 5
Comparisonofthefourclustersbasedonthecenteredmeanvalues(i.e.,z‐scores)ofthe12
variablesusedinthestudy
21
The results of the ANOVA and Kruskal-Wallis ANOVA tests (as appropriate
given the results of the Levene’s tests) revealed significant differences for all 12
variables. While the ANOVA tests showed that there were significant differences
present between clusters on the 12 variables, pair-wise comparisons with the
Holm-Bonferroni adjustment revealed where these differences are present
amongst the pairs of clusters.
Table 2
PairwiseComparisonbetweenClustersbyusingTukeyHSDTestandMann–WhitneyU(MW)
asPosthocTestsofANOVAandKruskalWallisANOVA,Respectively
Variables Test statistics Clusters
A
(Minimalists)
and B (Task-
Focused)
Clusters
A (Minimalists)
and C
(Disenchanted)
Clusters
A
(Minimalists)
and D
(Intensive
Users)
Clusters
B (Task-
Focused) and C
(Disenchanted)
Clusters
B (Task-
Focused)
and D
(Intensive
Users)
Clusters
C
(Disenchanted)
and D
(Intensive
Users)
Videos
annotated
(Tukey)
F(3)=45.20,
p<.001
* * *
Annotations
Total (MW)
H(3)=78.61,
p<.001
* * *
Annotations
Deleted (MW)
H(3)=54.95,
p<.001
* * * *
Annotations
Edited (MW)
H(3)=55.38,
p<.001
* * * *
Time Watched
(MW)
H(3)=79.07,
p<.001
* * * * *
Pause (MW) H(3)=76.15,
p<.001
* * * * *
Non-Stop
(MW)
H(3)=42.28,
p<.001
* * * *
Rewind (MW) H(3)=64.48,
p<.001
* * * * *
Fast Forward
(MW)
H(3)=67.20,
p<.001
* * * *
Density
(Tukey)
F(3)=29.63,
p<.001
* * * * * *
Earliest
annotation
(Tukey)
F(3)=20.04,
p<.001
* * *
Latest
annotation
(MW)
H(3)=60.66,
p<.001
* * * * *
Grades (MW) H(3)=30.65,
p<.001
* * *
Note: * denotes significance at <0.05
22
Table 2 shows that for all six possible pairings of clusters, significant differences
were observed for the majority of the variables. However, for the density measure
in particular, there were significant differences for all six cluster pairings. The
significant differences amongst the pairs involved both variables related to
students’ viewing of the videos and their use of the annotation functions. Table 3
presents the results of the comparison of the students’ overall grades between the
clusters with the comparisons within the four individual courses.
Table 3
SummaryoftheComparisonofGradesbetweentheClusterswithComparisonswithintheFour
Courses
Cluster Course 1
N
, M (SD)
Course 2
N
, M (SD)
Course 3
N
, M (SD)
Course 4¥
N
, M (SD)
A (minimalists) 29, 93.41 (3.55) 2, 69.00 (4.24) - 2, 86.00 (0.00)
B (task-focused) - 21, 88.71 (5.13) 19, 86.05 (4.40) 4, 83.50 (3.70)
C (disenchanted) - 4, 79.00 (7.87) - 12, 82.25 (6.03)
D (intensive) - 13, 87.00 (10.31) 9, 90.11 (2.85) -
Total 29, 93.41 (3.55) 40, 86.20 (6.69) 28, 87.36 (4.36) 18, 82.94 (5.24)
Legend:N–numberofstudents,M–meanvalue,SD–standarddeviationvalue.Gradesfortwocases
inbothCourse1andCourse4weremissing(andthus,thedifferencebetweenthenumbersinthe
tableandFig.6.
Course1‐H(3)=9.22,p=.027.SignificantdifferencesbetweenClusterAandClusterB,between
ClusterAandClusterD,betweenClusterBandClusterC,andbetweenClusterCandClusterD.
Course3–H(1)=5.05,p=.024.
¥Course4–H(2)=1.34,p=.511.
In the following section we discuss the significant differences amongst the
clusters and how these findings can be interpreted to explain the four types of
observed learning profiles.
Interpretation and Discussion of Clusters –
Research Question 1
The results of post-hoc tests presented in Table 2 and the centred means illustrated
in Figure 5 help identify patterns in students’ engagement with various features of
the video annotation tool across all four courses in this study regardless of the
instructional design (graded and ungraded use of the video annotation tool). These
23
analyses were used to answer the first research question that investigated profiles
of users of a video annotation software for reflective learning. Specifically,
differentiating characteristics of each cluster are observed, interpreted, and labeled
below. Table 4 summarises the differences and interpretations between the
clusters.
Cluster A: Minimalists
Cluster A represents the second largest number of students (n = 37 or 32% of the
study sample). Compared with the students in the remaining three clusters, this
cluster had significantly fewer annotations and videos annotated overall. The
annotations, in particular, are designed to help students reflect on and self-regulate
their skills when viewing videos of their own performance. Hence, it is of
particular interest that this fairly large group of students made a very limited
number of annotations. Further, the students in this cluster had significantly lower
use of the video viewing features as well compared with clusters B and D.
However, this cluster had significantly higher density of the transition graphs than
those in cluster C. This may be due to social sharing whereby the students in this
particular cluster were predominantly in the course where annotations were based
on a group performance (research question #2 discussed in the next section). The
sharing of annotations may promote high levels of metacognitive monitoring
leading to higher density in transitions. This suggests that although the students in
this first cluster have minimal engagement with the video viewing functions and
overall fewer annotations, their transition from one feature to the next is more
extensive than the students in cluster C who may not have been sharing their
annotations with their peers and hence, had lower metacognitive monitoring. This
finding resonates with Hadwin et al. (2007) who posited that students who have
lower graph density have specific studying or learning behaviour while those who
24
have higher graph density have not yet confirmed their learning strategies and are
trying different tactics showing signs of greater self-regulated learning and
metacognitive monitoring. The higher levels of metacognitive monitoring are
important, as according to Azevedo, Moos, Greene, Winters, and Cromley (2008)
they associated with an increase of feeling of knowing, judgment of learning, and
monitoring of progress toward goals. However, due to the overall minimal
engagement, this first cluster of students are considered the minimalists, based
loosely on del Valle and Duffy’s (2009) third cluster of students who exhibited
limited engagement with online resources, infrequent logins, and minimum
commitment to their learning much like Lust et al.'s (2013) no-users and Brooks
et al.'s (2011) minimal active learners.
Cluster B: Task-Focused
Cluster B represents the majority of the students (n=44 or 37%) in the study. The
students in cluster B produced the highest amount of video annotations and
annotated the highest number of videos compared with all other clusters. In
particular, cluster B showed significantly more video annotations than in cluster A
and significantly more videos annotated than in cluster A. Similarly, the students
in cluster B had significantly higher engagement with the video viewing functions
than students in clusters A and C. However, the students in cluster B used the
video features significantly less than those in cluster D with the exception of fast-
forwarding. In particular, they viewed significantly less of the videos (time-
watched) and showed patterns of less non-stop viewing yet had the highest
amount of annotations. Hence, their behaviour can be interpreted as displaying a
task-focused approach whereby the students use the video annotation tool to the
extent they require in order to reflect on their performance and make the necessary
time-stamped and general annotations. Furthermore, the density of transitions
25
between functions is significantly higher for the students in this cluster compared
with clusters A and C illustrating a higher element of metacognitive monitoring as
they experimented with different learning strategies to achieve their outcomes
(Hadwin et al., 2007). Overall, due to the highest use of the key feature of the
video annotation tool, making annotations, and the greater engagement with the
video viewing functions and density, the group of students in this cluster are
classified as task-focused. The classification, task-focused, is based loosely on del
Valle and Duffy’s (2009) second cluster of students who were described to have a
get it done approach. The task-focused group also resonates with Cleave, Edelson,
and Beckwith’s (1993) cluster of dominators who were focused and goal-oriented,
and Lust et al.'s (2013) selective users that are described as strategically accessing
video lectures in a goal directed way.
Cluster C: Disenchanted
Cluster C represents the smallest number of students (n=16 or 13% of the study
sample) and shows a behavioural pattern of significantly less interaction with the
video viewing features than clusters B and D yet significantly more annotations in
total and videos annotated in total than cluster A, the minimalist cluster. While the
students in this cluster appear to be the first to annotate a video once it was
available, and subsequently the last to post a final annotation as well, there was
limited sustained effort compared to students in clusters B and D. Further, the
graph density of this cluster is significantly lower than that of all other clusters
contributing to the characterization of the lack of continuous or sustained effort in
using various features of the tool, and thus, lower level of metacognitive
monitoring. The lower graph density of this group of students illustrates that they
have likely identified their learning strategy and do not need to experiment how
they transition between different features of the tool (Hadwin et al., 2007). While
26
the students in cluster C clearly engaged with the video annotation tool more so
than the minimalist cluster and were first to try the annotation functionality than
students in other clusters, they revealed a pattern of surface engagement rather
than a deep engagement with the tool. Adapting Barab, Bowdish, and Lawless’
(1997) cluster description of disenchanted users as those who glanced at various
features, but did not explore most in depth, the term applies to cluster C in this
study as the students who tried most of the functionality but without any sustained
effort or depth. The emergence of this particular cluster verifies Brooks et al.'s
(2011) hypothesis that a disillusioned group of students would initially access a
video lecture tool and gradually decline their use due to a perception that the tool
does not effectively support their learning.
Cluster D: Intensive
Cluster D represents the second lowest number of students (n=22 or 18.5% of the
study sample) and is comprised of students exhibiting behaviours that can be
interpreted as putting in the most amount of effort or self-driven approach. The
students in this particular cluster revealed significantly higher use of all video
viewing features (except for fast-forwarding) than all other clusters. In particular,
they watched significantly more of the videos (time-watched) and engaged in
more non-stop viewing than all other clusters. Although they did not have the
highest amount of annotations or videos annotated than the task-oriented (cluster
B), they were close behind and had a significantly higher number of annotations
than the minimalists (cluster A). Due to their extensive use of all functions in the
video annotation tool, this fourth cluster is classified as the intensive students as
adapted from Lust et al.'s (2013) cluster of students who accessed many, if not all
available learning tools frequently and intensively. This fourth cluster also
exhibited significantly higher density measures than all other clusters as the
27
students transitioned from one feature to the next. This suggests that this cluster
had the highest level of metacognitive monitoring activity (Hadwin et al., 2007).
Likewise, this cluster is also similar to Barab et al.’s (1997) cyber cartographers
cluster as those who are goal-oriented, commit time to have deep engagement, and
demonstrate self-efficacy. Table 4 summarises the four user profiles of video
annotation for each cluster, as discussed above.
Table 4
SummaryoftheFourProfilesofUsersofVideoAnnotationSoftware
Cluster Profile
Minimalists
(Cluster A)
Minimal engagement overall with all features based on:
Significantly lower use of all video features compared with
clusters B and D
Significantly lower amount of annotations and videos
annotated than ALL other clusters
Significantly more density than cluster C only and
relatively low metacognitive monitoring activity
Task Focused
(Cluster B)
On-task, based on:
significantly higher amount of use of all video viewing
features compared with cluster A and C but significantly
less than cluster D (except fast forward)
highest amount of total annotations and videos annotated
than other clusters but significantly higher than cluster A
significantly higher density of transitions between features
compared with clusters A and C, but not much less than
cluster D.
Disenchanted
(Cluster C)
Tried all features, earliest to annotate a video but overall no
sustained engagement based on:
Significantly more videos annotated and total annotations
compared with cluster A only
No significant difference with cluster A (although slightly
more use) with respect to use of video view features nor
deleting or editing annotations
Significantly less use of video viewing features than
clusters B and D
Significantly earliest and latest annotation posting
Significantly lowest density of transitions between features
compared with ALL other clusters (i.e., low metacognitive
monitoring)
Intensive
(Cluster D)
High-effort, self-driven approach based on:
significantly higher amount of use of most video viewing
features (exception of fast forwarding) compared to ALL
clusters
significantly higher density of transitions compared to ALL
clusters (i.e., highest metacognitive monitoring)
significantly higher number of annotations than cluster A.
28
Discussion of Behavioural Patterns in Different
Courses – Research Questions 2 & 3
The four clusters emerging from the hierarchical cluster analysis in this study
reveal distinct types of interaction with the video annotation tool and, therefore,
different learning profiles. However, as noted earlier, there were variations in the
instructional design of the four courses. Two of the courses included graded
general comment annotations and two courses were designed with an optional and
ungraded use of the tool. Longitudinal analysis of the results revealed that some of
the students in the graded two courses continued on to a subsequent course where
the annotation activity and use was completely optional. An analysis of the
profiles described in Table 4 for the four clusters could lead to the hypothesis that
the minimalists or disenchanted users as mostly in courses where the use of the
tool is optional unlike the other two clusters who exhibited a far greater number of
annotations and videos annotated. A cross-tabulation of the four clusters against
the four courses helps identify the behavioural patterns most dominant in order to
answer the second research question that investigated whether different
instructional methods influence learner profiles based on their use of a video
annotation software. Figure 6 illustrates the cross-tabulation and reveals the
spread of students in each cluster across the four courses.
29
Fig. 6
Cross‐tabulationofclustersandcourses:clusterA–minimalists;clusterB–task‐focused;
clusterC–disenchanted;andclusterD–intensive.
As Figure 6 demonstrates, a large portion of the students in cluster A, the
minimalists, were enrolled in Course 1 where the use of the video annotation tool
was optional and ungraded with a small number enrolled in Course 4 where the
use of the video annotation tool was also optional. Similarly, the majority of
students associated with cluster C, the disenchanted, were also enrolled in Course
4. Hence, the pattern arising confirms the hypothesis that the majority of the
minimalists and disenchanted profiles were enrolled in courses where the use of
the video annotation tool was entirely voluntary. The only difference being that
the students in Course 4 included those who had previously taken Course 2 where
the use of the video annotation tool was required and general comment
annotations graded. However, since the students in Course 4 included students
who had previously taken a course where annotating and commenting on videos
was a required activity, there was a higher proportion of disenchanted students
(cluster C) in Course 4 than minimalists (cluster A). There are two significant
differences between these two clusters. The first concerns the greater number of
●●
31
221413
19 9
4412
Course 1
Course 2
Course 3
Course 4
Cluster A Cluster B Cluster C Cluster D
Cluster
Offering
30
annotations and videos annotated by the disenchanted students than the
minimalists. This may be due to the students in Course 4 becoming more
accustomed to annotating their videos or beginning to appreciate how the activity
enhances their learning and self-regulation skills due to their prior experience in
Course 2. This could also be a result of students in Course 1 being conscious that
their peers may view their annotations since they all annotated the same videos
unlike the students in the other courses and were, therefore, more carefully
monitoring their annotations leading to a fewer number. The second concerns the
higher graph density exhibited by the minimalists. This may be attributed to the
majority of minimalists in Course 1 where they did not have prior experience with
CLAS. Based on Hadwin et al.'s (2007) earlier work, minimalists may have been
trialing different learning strategies, although minimally, leading to a greater
graph density as they accessed different features.
While the required video annotation activities in Course 2 may have contributed
to more disenchanted students in the later course than minimalists, Figure 6
illustrates that only four task-focused students (Cluster B) and none of the
intensive students (Cluster D) were enrolled in the final course. The majority of
the task-focused students (Cluster B) were almost equally spread across courses 2
and 3 while all of the intensive students (Cluster D) were spread across these same
two courses. Hence, a distinct and expected pattern emerges where by the students
who engaged with the functions of the video annotation tool the most were
enrolled in the courses where the general annotations were graded and the activity
was a required component of the curriculum. In other words, they exhibited
behaviours consistent with staying on task, self-driven, and making the most use
of the features available within the video annotation tool. This is not surprising
since their use of the video annotation tool was graded in Courses 2 and 3.
31
However, the main conclusion from the cross-tabulation lies in the lack of
intensive students and very few task-focused students in Course 4 despite their
previous use of the video annotation tool in Course 2 when it was graded.
While the study did not explore the intentions beneath students’ engagement with
the video annotation tool or lack thereof, there are a number of possible reasons or
hypotheses for why students in Course 4 largely fall within Cluster C, the
disenchanted and, minimally, Cluster A, the minimalists. One possible
explanation may be that despite the students in Course 4 having some experience
with the use of graded video annotations, the intrinsic motivation and self-
regulated learning skills to identify when viewing and annotating videos of their
previous performances enhances their learning experience still requires further
development. While the students who fall in the disenchanted cluster may have
begun to develop these skills more so than those who fall in the minimalist cluster,
more external regulation and instructors acknowledging the usefulness of the tool,
instructional norms (McGill & Klobas, 2009) are required. In other words,
experience in one prior course with graded use of a video annotation system is not
sufficient for students to appreciate the full value of viewing and annotating one’s
own recorded performance. Instead, students require more scaffolding and
external motivating factors (e.g. assessed or graded activities) to encourage their
use of the educational technologies (Lust et al., 2011; Perkins, 1985).
As shown in Table 3, statistically significant differences in grades between the
clusters were revealed in Courses 2 and 3, in which the use of video annotation
software was a graded activity. Specifically, in Course 2, the grades of the
students in clusters A (minimalists) and C and (disenchanted) were significantly
lower than those of the students in clusters B (task-focused) and D (intensive). In
Course 3, the intensive students (cluster D) had significantly higher grades than
32
task-focused students (cluster B). These findings reinforce the importance of the
patterns of students’ technology engagement particularly in courses in which tasks
with the technology are graded. Moreover, the reasons linked to the differences in
self-regulated skills between the different clusters, as discussed above, are
probable explanations for the differences in the academic achievement as well.
Future research should also account for individual differences (e.g. motivation or
metacognitive awareness) – representative of internal conditions as per Winne’s
(2006) model of self-regulated learning – when investigating the effects of
learning profiles on academic achievement.
Implications for Practice
The findings in this study have several implications for pedagogical practice,
namely blended learning course design integrating video annotation technologies
for enhancing students’ reflective practice. Although there are studies on the use
of video annotation tools to aid students’ reflections on their own performance in
pre-service teacher education programs (Colasante, 2011; Magenheim et al.,
2010) and in medical education (Hulsman, Harmsen, & Fabriek, 2009), the
studies have relied heavily on self-reports rather than data collected from students
actual use of video annotation software. Further, these studies have not
specifically explored patterns in students’ learning profiles about learning
strategies they used when using a video annotation tool under different
instructional methods. Hence, the emerging use profiles of intensive and task-
oriented clusters of students appearing more in courses where annotations are
graded and, in particular, more disenchanted students in a course with ungraded
annotations despite having previously enrolled in a course with graded
annotations shows that students are slowly moving towards self-regulating their
33
learning. Since their level of use continues to be relatively limited, the study
shows that students will need more than one course where they are incentivized
through the use of assessment (e.g. grades associated with their annotations) to
engage with the video annotation tool to further develop their self-regulated skills
(Lust et al., 2013) and awareness of the learning opportunities it presents (Perkins,
1985). Hence, when developing a blended learning curriculum educators need to
consider that one course with graded use of a video annotation tool may not be
sufficient in developing and sustaining students’ appreciation of the reflective
exercise. Rather, educators should design a program of courses whereby the tool
is introduced with a set of extrinsic motivators (e.g., grades) linked via a series of
courses with gradual movement towards more optional use of the tool in order to
support and scaffold students’ understanding of how using a video annotation can
enhance their learning experience. This concept of incentivizing effort is well
noted in the literature related to intrinsic and extrinsic motivation of learners. For
instance, Cerasoli, Nicklin, and Ford (2014) undertook a meta-analysis of
motivation research to identify the relationship between extrinsic and intrinsic
motivation on student learning performance. The authors demonstrated that
extrinsic motivation or the use of incentives were significant predictors of the
quantity of performance for individual students. However, quality was best
predicted by student interest or intrinsic motivation. Similarly, in the present study
we suggest that the use of grades is applied as incentive to promote use and
activity of the video annotation tool while students develop sufficient self-
regulatory skills in the direct application of the tool to aid their learning. Research
in self-determination theory also well notes the use of incentives or extrinsic
motivation can be used to promote student intrinsic motivation (Koestner, Ryan,
Bernieri, & Holt, 1984; Ryan, Mims, & Koestner, 1983).
34
Implications for Research
While the study provides insight into the learning profiles of students when using
a video annotation tool, and the differences observed when the pedagogical
approach is more formative than summative, there are four primary limitations
that future research can address. First, as the research design was a case study
focusing on the use of a video annotation system in a single institution where the
courses that used the tool for self-reflection purposes were in the performing arts
discipline only, the findings cannot be widely generalized to other disciplines or
settings. Future studies exploring students’ learning profiles when using a video
annotation tool for reflective practice in other disciplines and institutions are
required to support or refute the findings in this study. Second, future studies
could include a triangulation of cluster analysis of use profiles and cross-
tabulation of students enrolled in courses where the video annotation activity is
graded vs ungraded with students’ performance (i.e. overall grades) in order to
better understand the characteristics of the clusters and whether various learning
profiles correlate with better overall course performance. Third, since this study
focused only on students’ trace data based on their interactions with the video
annotation tool, future research capturing students’ intentions either through
surveys or interviews behind the way they used the tool will help to explain the
observed learning profiles. Finally, experimental studies where students have
greater opportunities to engage with a video annotation tool when annotations are
graded prior to having the option to use it will help reveal the extent that extrinsic
motivation is required prior to students developing their own intrinsic motivation
and self-regulated approach to their reflective learning.
35
Acknowledgements
Tobeincludedafterreview.
References
Alexander,P.A.,Jetton,T.L.,&Kulikowich,J.M.(1995).Interrelationshipof
knowledge,interest,andrecall:Assessingamodelofdomainlearning.
JournalofEducationalPsychology,87(4),559.
Al‐Qahtani,A.A.Y.,&Higgins,S.E.(2013).Effectsoftraditional,blendedand
e‐learningonstudents’achievementinhighereducation:E‐Learning,
blendedandtraditionallearning.JournalofComputerAssisted
Learning,29(3),220–234.http://doi.org/10.1111/j.1365‐
2729.2012.00490.x
Aubert,O.,Prié,Y.,&Canellas,C.(2014).Leveragingvideoannotationsin
video‐basede‐learning.In7thInternationalConferenceonComputer
SupportedEducation.Barcelone,Spain.Retrievedfrom
http://arxiv.org/abs/1404.4607
Azevedo,R.,Moos,D.C.,Greene,J.A.,Winters,F.I.,&Cromley,J.G.(2008).
Whyisexternally‐facilitatedregulatedlearningmoreeffectivethan
self‐regulatedlearningwithhypermedia?EducationalTechnology
ResearchandDevelopment,56(1),45–72.
http://doi.org/10.1007/s11423‐007‐9067‐0
Barab,S.A.,Bowdish,B.E.,&Lawless,K.A.(1997).Hypermedianavigation:
Profilesofhypermediausers.EducationalTechnologyResearchand
Development,45(3),23–41.
Bargeron,D.,Gupta,A.,Grudin,J.,&Sanocki,E.(1999).Annotationsfor
streamingvideoontheWeb:systemdesignandusagestudies.
ComputerNetworks,31(11),1139–1153.
36
Beretvas,S.N.,Meyers,J.L.,&Leite,W.L.(2002).Areliabilitygeneralization
studyoftheMarlowe‐Crownesocialdesirabilityscale.Educationaland
PsychologicalMeasurement,62(4),570–589.
http://doi.org/10.1177/0013164402062004003
Bielawski,L.,&Metcalf,D.(2003).Blendedelearning:integratingknowledge,
performance,support,andonlinelearning(1sted.).Amherst,Mass.:
HRDPress.
Brooks,C.,Epp,C..,Logan,G.,&Greer,J.(2011).Thewho,what,when,and
whyoflecturecapture.InProceedingsoftheFirstInternational
ConferenceonLearningAnalyticsandKnowledge(pp.86–92).Banff,
Alberta,Canada:ACM.http://doi.org/10.1145/2090116.2090128
Cerasoli,C.P.,Nicklin,J.M.,&Ford,M.T.(2014).Intrinsicmotivationand
extrinsicincentivesjointlypredictperformance:A40‐yearmeta‐
analysis.PsychologicalBulletin,140(4),980.
Chen,P.‐S.D.,Lambert,A.D.,&Guidry,K.R.(2010).Engagingonlinelearners:
TheimpactofWeb‐basedlearningtechnologyoncollegestudent
engagement.Computers&Education,54(4),1222–1232.
http://doi.org/10.1016/j.compedu.2009.11.008
Cleave,J.B.,Edelson,D.,&Beckwith,R.(1993).Amatterofstyle:Ananalysis
ofstudentinteractionwithacomputer‐basedlearningenvironment.
AmericanEducationalResearchAssociation(AERA)AnnualMeeting,
Atlanta,GA.
Colasante,M.(2011).Usingvideoannotationtoreflectonandevaluate
physicaleducationpre‐serviceteachingpractice.AustralasianJournal
ofEducationalTechnology,27(1),66–88.
37
Colasante,M.,&Fenn,J.(2009).“mat”:ANewMediaAnnotationToolwithan
InteractiveLearningCycleforApplicationinTertiaryEducation.In
WorldConferenceonEducationalMultimedia,Hypermediaand
Telecommunications(Vol.2009,pp.3546–3551).Retrievedfrom
http://www.editlib.org/pv/31992
Cross,A.,Bayyapunedi,M.,Ravindran,D.,Cutrell,E.,&Thies,W.(2014).
VidWiki:enablingthecrowdtoimprovethelegibilityofonline
educationalvideos(pp.1167–1175).ACMPress.
http://doi.org/10.1145/2531602.2531670
delValle,R.,&Duffy,T.M.(2009).Onlinelearning:Learnercharacteristics
andtheirapproachestomanaginglearning.InstructionalScience,
37(2),129–149.http://doi.org/10.1007/s11251‐007‐9039‐0
Dunning,T.(2012).NaturalExperimentsintheSocialScienc‐es:ADesign
BasedApproach.CambridgeUniversitiyPress.
Eisenhardt,K.M.(1989).Buildingtheoriesfromcasestudyresearch.
AcademyofManagementReview,14(4),532–550.
Garrison,D.R.,&Kanuka,H.(2004).Blendedlearning:Uncoveringits
transformativepotentialinhighereducation.TheInternetandHigher
Education,7(2),95–105.
http://doi.org/10.1016/j.iheduc.2004.02.001
Garrison,D.R.,&Vaughan,N.(2008).Blendedlearninginhighereducation.
SanFrancisco:Jossey‐BassInc.Publishers.
Giannakos,M.N.,Chorianopoulos,K.,&Chrisochoides,N.(2014).Collecting
andmakingsenseofvideolearninganalytics.InFrontiersinEducation
38
Conference(FIE),2014IEEE(pp.1–7).IEEE.Retrievedfrom
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7044485
Giannakos,M.N.,Chorianopoulos,K.,&Chrisochoides,N.(2015).Making
senseofvideoanalytics:Lessonslearnedfromclickstream
interactions,attitudes,andlearningoutcomeinavideo‐assisted
course.TheInternationalReviewofResearchinOpenandDistributed
Learning,16(1).Retrievedfrom
http://www.irrodl.org/index.php/irrodl/article/view/1976
Gibbs,G.,&Simpson,C.(2004).Conditionsunderwhichassessmentsupports
students’learning.LearningandTeachinginHigherEducation,1(1),
3–31.
Gonyea,R.M.(2005).Self‐reporteddataininstitutionalresearch:Reviewand
recommendations.NewDirectionsforInstitutionalResearch,
2005(127),73–89.
Gosper,M.,Malfroy,J.,&McKenzie,J.(2013).Students’experiencesand
expectationsoftechnologies:AnAustralianstudydesignedtoinform
planninganddevelopmentdecisions.AustralasianJournalof
EducationalTechnology,29(2).Retrievedfrom
http://ascilite.org.au/ajet/submission/index.php/AJET/article/view/
127
Gosper,M.,Malfroy,J.,McKenzie,J.,&Rankine,J.(2011).Students’
engagementwithtechnologies:Implicationsforuniversitypractice.
ChangingDemands,ChangingDirections.ProceedingsAsciliteHobart
2011,504–508.
39
Graham,C.R.,Woodfield,W.,&Harrison,J.B.(2013).Aframeworkfor
institutionaladoptionandimplementationofblendedlearningin
highereducation.TheInternetandHigherEducation,18,4–14.
http://doi.org/10.1016/j.iheduc.2012.09.003
Greller,W.,&Drachsler,H.(2012).TranslatingLearningintoNumbers:A
GenericFrameworkforLearningAnalytics.EducationalTechnology&
Society,15(3),42–57.
Hadwin,A.F.,Nesbit,J.C.,Jamieson‐Noel,D.,Code,J.,&Winne,P.H.(2007).
Examiningtracedatatoexploreself‐regulatedlearning.Metacognition
andLearning,2(2‐3),107–124.http://doi.org/10.1007/s11409‐007‐
9016‐7
Hattie,J.,&Timperley,H.(2007).ThePowerofFeedback.Reviewof
EducationalResearch,77(1),81–112.
http://doi.org/10.3102/003465430298487
Hulsman,R.L.,Harmsen,A.B.,&Fabriek,M.(2009).Reflectiveteachingof
medicalcommunicationskillswithDiViDU:Assessingthelevelof
studentreflectiononrecordedconsultationswithsimulatedpatients.
PatientEducationandCounseling,74(2),142–149.
http://doi.org/10.1016/j.pec.2008.10.009
Koestner,R.,Ryan,R.M.,Bernieri,F.,&Holt,K.(1984).Settinglimitson
children’sbehavior:Thedifferentialeffectsofcontrollingvs.
informationalstylesonintrinsicmotivationandcreativity.Journalof
Personality,52(3),233–248.http://doi.org/10.1111/j.1467‐
6494.1984.tb00879.x
40
López‐Pérez,M.V.,Pérez‐López,M.C.,&Rodríguez‐Ariza,L.(2011).Blended
learninginhighereducation:Students’perceptionsandtheirrelation
tooutcomes.Computers&Education,56(3),818–826.
http://doi.org/10.1016/j.compedu.2010.10.023
Lust,G.,Elen,J.,&Clarebout,G.(2013).Regulationoftool‐usewithina
blendedcourse:Studentdifferencesandperformanceeffects.
Computers&Education,60(1),385–395.
http://doi.org/10.1016/j.compedu.2012.09.001
Lust,G.,Vandewaetere,M.,Ceulemans,E.,Elen,J.,&Clarebout,G.(2011).
Tool‐useinablendedundergraduatecourse:InSearchofuser
profiles.Computers&Education,57(3),2135–2144.
http://doi.org/10.1016/j.compedu.2011.05.010
Magenheim,J.,Reinhardt,W.,Roth,A.,Moi,M.,&Engbring,D.(2010).
IntegrationofaVideoAnnotationToolintoaCoactiveLearningand
WorkingEnvironment.InKeyCompetenciesintheKnowledgeSociety
(pp.257–268).Springer.Retrievedfrom
http://link.springer.com/chapter/10.1007/978‐3‐642‐15378‐5_25
McGill,T.J.,&Klobas,J.E.(2009).Atask–technologyfitviewoflearning
managementsystemimpact.Computers&Education,52(2),496–508.
http://doi.org/10.1016/j.compedu.2008.10.002
Mu,X.(2010).Towardseffectivevideoannotation:Anapproachto
automaticallylinknoteswithvideocontent.Computers&Education,
55(4),1752–1763.http://doi.org/10.1016/j.compedu.2010.07.021
41
Perkins,D.N.(1985).TheFingertipEffect:HowInformation‐Processing
TechnologyShapesThinking.EducationalResearcher,14(7),11–17.
http://doi.org/10.3102/0013189X014007011
Phillips,R.,Maor,D.,Preston,G.,&Cumming‐Potvin,W.(2012).Exploring
learninganalyticsasindicatorsofstudybehaviour.InProceedingsof
EdMedia:WorldConferenceonEducationalMediaandTechnology(pp.
2861–2867).Denver,Colorado,USA:AssociationfortheAdvancement
ofComputinginEducation.Retrievedfrom
http://researchrepository.murdoch.edu.au/10460/
Pintrich,P.R.,&deGroot,E.V.(1990).Motivationalandself‐regulated
learningcomponentsofclassroomacademicperformance.Journalof
EducationalPsychology,82(1),33–40.http://doi.org/10.1037/0022‐
0663.82.1.33
Rich,P.J.,&Hannafin,M.(2008).VideoAnnotationTools:Technologiesto
Scaffold,Structure,andTransformTeacherReflection.Journalof
TeacherEducation,60(1),52–67.
http://doi.org/10.1177/0022487108328486
Ryan,R.M.,Mims,V.,&Koestner,R.(1983).Relationofrewardcontingency
andinterpersonalcontexttointrinsicmotivation:Areviewandtest
usingcognitiveevaluationtheory.JournalofPersonalityandSocial
Psychology,45(4),736–750.http://doi.org/10.1037/0022‐
3514.45.4.736
Torrisi‐Steele,G.,&Drew,S.(2013).Theliteraturelandscapeofblended
learninginhighereducation:theneedforbetterunderstandingof
42
academicblendedpractice.InternationalJournalforAcademic
Development,1–13.http://doi.org/10.1080/1360144X.2013.786720
Ward,J.H.(1963).HierarchicalGroupingtoOptimizeanObjectiveFunction.
JournaloftheAmericanStatisticalAssociation,58(301),236–244.
http://doi.org/10.1080/01621459.1963.10500845
Winne,P.H.(2006).HowSoftwareTechnologiesCanImproveResearchon
LearningandBolsterSchoolReform.EducationalPsychologist,41(1),
5–17.http://doi.org/10.1207/s15326985ep4101_3
Winne,P.H.(2013).Learningstrategies,studyskills,andself‐regulated
learninginpostsecondaryeducation.InM.B.Paulsen(Ed.)Higher
Education:HandbookofTheoryandResearch(Vol.28,pp.377–403).
SpringerNetherlands.Retrievedfrom
http://link.springer.com/chapter/10.1007%2F978‐94‐007‐5836‐
0_8#
Winne,P.H.,&Jamieson‐Noel,D.(2002).Exploringstudents’calibrationof
selfreportsaboutstudytacticsandachievement.Contemporary
EducationalPsychology,27(4),551–572.
Yen,J.‐C.,&Lee,C.‐Y.(2011).Exploringproblemsolvingpatternsandtheir
impactonlearningachievementinablendedlearningenvironment.
Computers&Education,56(1),138–145.
http://doi.org/10.1016/j.compedu.2010.08.012
Yin,R.K.(2009).CaseStudyResearch:DesignandMethods(4thed.).
ThousandOaks,CA:SagePublications,Inc.
Yousef,A.M.F.,Chatti,M.A.,&Schroeder,U.(2014).Video‐basedlearning:A
criticalanalysisoftheresearchpublishedin2003‐2013andfuture
43
visions.IneLmL2014,TheSixthInternationalConferenceonMobile,
Hybrid,andOnlineLearning(pp.112–119).Retrievedfrom
http://www.thinkmind.org/index.php?view=article&articleid=elml_2
014_5_30_50050
... This technique has echoes with the probabilistic process mining techniques in that it provides a likelihood of movement from one process to the next, in terms of relative percentage frequencies; effectively transition probabilities. The Mirriahi et al. (2016) study used transition graphs, which reflect elements of ENA and PM, albeit in a cruder form. The Siadaty et al. (2016b) also used a similar, if more sophisticated, method to articulate, in a network space, the associations between intervention events and SRL activities. ...
... Other studies used less models in the consolidation process (e.g., Kinnebrew et al. (2014)), and the Heirweg et al. (2020) study does not even explicitly state the source models, consolidating as it does, from general SRL theory. The promise of consolidating SRL constructs, or mapping them onto other pedagogic models (e.g., Mirriahi et al. (2016)) is an intriguing one. We should, however, sound a note of caution; the appropriation of SRL strategies from different models to measure specific constructs is a choice that raises questions of construct validity. ...
Article
Full-text available
We present a systematic literature review of data-driven self-regulated learning (SRL) that emphasises the methodological importance of temporality and sequence, as opposed to conventional statistical analysis. Researchers seem unanimous in their view of the importance of SRL in modern online and blended educational settings; this is borne out by number of reviews of literature on the subject. There has, as yet, been no systematic treatment of SRL in the context of its conceptualisation as a phenomenon that unfolds in sequences over time. To address this limitation, this review explores the corpus of work (n = 53) in which SRL and its related dimensions are analysed through the lenses of temporality, sequence and order. The results show that, in the pursuit of validity and impact, key decisions need to be addressed in regard to theoretical grounding, data collection, and analytic methods. Based on these outcomes, we propose a framework of directives and questions to aid researchers who want to push forward the field. This framework comprises four sub-areas: i) methodological considerations, relating to data capture and analytic processes; ii) theoretical considerations, relating to the usage of models of self-regulated learning and their related dimensions; iii) validity considerations, relating to the robustness of the chosen analytic outcomes studied; and iv) temporal considerations, relating to the articulation of analytic outcomes in the context of temporality and sequence.
... Routinely collected digital data of student location, such as WiFi, have also been analyzed via network approaches to understand student collocation in face-to-face settings in relation to performance [41]. In addition to analysis of relationships between actors and objects in learning settings, network analysis in education has been applied to a vast variety of other markedly different research problems: identification of the structure of different types of navigation with video resources [42], curriculum patterns and pathways [43], similarities in structures of in-course resources [44], and structural patterns in content analysis of various learning related texts through a method called Epistemic Network Analysis (ENA) [35]. ...
Article
Full-text available
For over five decades, researchers have used network analysis to understand educational contexts, spanning diverse disciplines and thematic areas. The wealth of traditions and insights accumulated through these interdisciplinary efforts is a challenge to synthesize with a traditional systematic review. To overcome this difficulty in reviewing 1791 articles researching the intersection of networks and education, this study combined a scientometric approach with a more qualitative analysis of metadata, such as keywords and authors. Our analysis shows rapidly growing research that employs network analysis in educational contexts. This research output is produced by researchers in a small number of developed countries. The field has grown more recently, through the surge in the popularity of data-driven methods, the adoption of social media, and themes as teacher professional development and the now-declining MOOC research. Our analysis suggests that research combining networks and educational phenomena continues to lack an academic home, as well as remains dominated by descriptive network methods that depict phenomena such as interpersonal friendship or patterns of discourse-based collaboration.
... Routinely collected digital data of student location, such as WiFi, have also been analyzed via network approaches to understand student collocation in face-to-face settings in relation to performance [41]. In addition to analysis of relationships between actors and objects in learning settings, network analysis in education has been applied to a vast variety of other markedly different research problems: identification of the structure of different types of navigation with video resources [42], curriculum patterns and pathways [43], similarities in structures of in-course resources [44], and structural patterns in content analysis of various learning related texts through a method called Epistemic Network Analysis (ENA) [35]. ...
Preprint
Full-text available
For over five decades, researchers have used network analysis to understand educational contexts, spanning diverse disciplines and thematic areas. The wealth of traditions and insights accumulated through these interdisciplinary efforts is a challenge to synthesize with a traditional systematic review. To overcome this difficulty in reviewing 1791 articles researching the intersection of networks and education, this study combined a scientometric approach with a more qualitative analysis of metadata, such as keywords and authors. Our analysis shows rapidly growing research that employs network analysis in educational contexts. This research output is produced by researchers in a small number of developed countries. The field has grown more recently, through the surge in the popularity of data-driven methods, the adoption of social media, and themes as teacher professional development and the now-declining MOOC research. Our analysis suggests that research combining networks and educational phenomena continues to lack an academic home, as well as remains dominated by descriptive network methods that depict phenomena such as interpersonal friendship or patterns of discourse-based collaboration.
... Liu, Yang, Williams, and Wang (2019) introduce a note-taking platform which allows learners to add annotations on video transcripts and later reinterpret and synthesise their notes. Nonetheless, the effectiveness of annotations in VBL depends on students' learning strategies and motivation (Pardo et al., 2015;Mirriahi, Liaqat, Dawson, & Gašević, 2016), which emphasises the need to design adaptive interventions on video annotation. In recent studies, text and learning analytics are leveraged to characterise the learning process in VBL (Joksimović et al., 2019;Dodson et al., 2018a;Seo et al., 2021). ...
Article
Full-text available
Developing and maintaining constructive engagement is a crucial challenge in learning by watching videos. AVW-Space is an online video-based learning platform which enhances student engagement via note-taking and personalised support. Previous studies with AVW-Space show that students who write comments, especially high-quality comments, learn more. The goal of the study reported in this paper is to encourage students to write better-quality comments. After automating the assessment of comment quality using machine learning approaches, we developed Quality nudges which encourage students to write better comments. We conducted a study in a first-year engineering course to analyse the learning effects of the Quality nudges. The results show that Quality nudges enhanced constructive engagement and learning. The contribution of this research is in proposing methodology for increasing the quality of student comments in video-based learning.
... It would also be advisable to study the relationship with internal vs. external motivation variables (e.g. final grades) and their use in broader contexts such as a complete program and different grades (Mirriahi, Joksimović, Gašević, & Dawson, 2018;Mirriahi, Liaqat, Dawson, & Gašević, 2016). The very nature of the competence acquisition process demands it, because there is a profusion of technological innovations and multimedia messages in our social, family and professional life, and students need to consider these messages through a more critical and professional vision. ...
Article
Full-text available
University education requires students to be trained both at university and at external internship centres. Because of Covid-19, the availability of multimedia resources and examples of practical contexts has become vital. Multimedia annotation can help students reflect on the professional world, collaborating and interacting with colleagues online. This study aims to encourage collaborative practical thinking by using new video annotation technologies. 274 students participated in an experiment of task design focusing on the analysis of a technology-based, award-winning educational innovation project. With mixed research design, qualitative and quantitative data exported from the video annotation platform used was collected and analysed. The results show differences in the quality and quantity of the answers: in the tasks with broad Folksonomy they are more numerous but more dispersed in their analysis, and vice versa. The quality of the answers given with narrow Folksonomy is also higher in both texts and videos modes. Producing multimedia annotations is a practical way to encourage students to practise reflective reasoning about the professional reality.
... However, more recently, studies such as Zaier et al. (2020) and Cattaneo et al., 2020 continue to argue for video annotation's ability to facilitate peer evaluation, academic feedback, and self-evaluation. In behavior analysis, Mirriahi et al. (2016) investigated cluster diagrams of video annotation behavior and determined that viewers reflected one of four behavior types: minimalist, disenchanted, task-focused, or intensive group clusters. Pérez-Torregrosa et al. (2017) reviewed the literature on video annotation and noted that prior to 2017, only 19 studies had been conducted on video annotation in education; they provide an overview of the increase in studies of video annotation in learning interventions from 2006 to 2016. ...
Article
Full-text available
Collaborative Video Annotation (CVA) is a kludge where learners annotate video together, experiencing both the video and each other’s annotations in a dynamic discussion. Three scenes from small group CVA discussions were selected for analysis from 14 CVA discussions where 8–12 learners interacted via the annotation tool on top of a video. The twenty-second scenes were analyzed for semiotic meaning-making practices and this revealed a variety of participatory media literacy levels among these undergraduates. Topics of discussions were related but not identical, and the selected exemplars showed a range of attention to communicative features of the media. Discussions evolved in dramatically different ways due to the interplay of images, text, and learner choices. Results suggest that converged media require new literacies educators would be wise to explore and wiser still to educate our learners about.
Article
This paper outlines the feasibility of replacing a midterm assessment with a student-created digital media project. We examine the benefits of a peer review process for student-created pedagogy, the effectiveness of the student-generated digital media, and self-evaluations as a replacement for traditional forms of assessment. We conclude that this innovative teaching and assessment model is both effective and timely as more college courses are being taught digitally and/or asynchronously.
Article
Background Medical education all over the world is undergoing paradigm shift. Video recording of student's performance and self-annotation are emerging as valuable tools for self-directed learning among students. Study was conducted to find the effectiveness of video annotation tool in reflective learning. The learning outcome was to find whether the video annotation is helping in critical reflection and improving the perception of students on guideline compliance while learning the technique of epidural insertion. Methods In phase 1; following pretest, the students were made to observe three epidural insertions and perform one epidural insertion. In phase 2; following posttest, two faculty members analyzed the depth of reflection using the Reflection Rubric. Students perception was recorded using the Reflective practice survey. Results The average score of students after pretest was 76%. The posttest score was 84% (p value 0.003). In depth analysis using the reflection rubric we found 52.38% of the total reflections had a score of two, showing they were at introspection level only. 25.71% of reflections were having score of one, showing that they were just habitual answers. Only 21.9% of the total reflections had score of three; and none of them were critically reflecting. All the students (18/18) agreed that recording the session was meaningful. Conclusion The art of critical reflection is learnt by relentless effort. Yet it helps the students to reflect on the whole process introspecting and understanding what went wrong. Video annotation turns out to be a valuable tool in reflective learning
Article
In this paper, we propose a novel approach to the assessment of the emergent socio-cognitive roles learners adopted during peer interactions. The approach posits that different dimensions of peer interaction emerging from temporal-semantic discourse information and the structure of interactions can be used to diagnostically reveal the emergent roles of learners during peer interactions. As such, the combination of two established methodologies, Group Communication Analysis (GCA) that centers on temporal semantic properties of online discourse with Social Network Analysis (SNA) that reflects structural interpersonal patterns of online interactions are used to gain a deeper understanding of the emergent, socio-cognitive roles learners adopt during peer interactions at scale. The proposed approach is named socio-cognitive group communication and interpersonal position (SCIP) analysis and is defined as a combination of these two distinct and complementary analytic techniques. The proposed SCIP approach is examined on data produced during peer interaction in a massive open online course (MOOC) delivered via Coursera. Using SCIP analysis, learner activity is described through five roles: Lurkers, Followers, Socially Detached, Influential Actors and Hyper Posters. We conclude the paper with a detailed discussion of the theoretical, methodological, and practical implications for peer interaction research. The scalability of the methodology opens the door for future research efforts directed towards understanding and improving peer-interactions at scale.
Article
Full-text available
This paper provides an overview of the design process of a new online media annotation tool. This work-in-progress report will step through some design decisions as aided by reviewing learning theory and related experiences outlined in the literature; design principles from a user interface perspective; and user testing of the first design iteration. The first of a three stage development of the media annotation tool, ‘mat’ (MAT), is designed for learning from video, with later stages enabling other media (audio, image, other) plus assignment building with media inserts. User testing reinforced several design decisions plus initiated some change.
Conference Paper
Full-text available
The rapidly-developing 21st century world of work and knowledge calls for self-directed lifelong (SDL) learners. While higher education must embrace the types of pedagogies that foster SDL skills in graduates, the pace of change in education can be glacial. This paper describes a social annotation technology, the Collaborative Lecture Annotation System (CLAS), that can be used to leverage existing teaching and learning practices for acquisition of 21st Century SDL skills. CLAS was designed to build upon the artifacts of traditional didactic modes of teaching, create enriched opportunities for student engagement with peers and learning materials, and offer learners greater control and ownership of their individual learning strategies. Adoption of CLAS creates educational experiences that promote and foster SDL skills: motivation, self-management and self-monitoring. In addition, CLAS incorporates a suite of learning analytics for learners to evaluate their progress, and allow instructors to monitor the development of SDL skills and identify the need for learning support and guidance. CLAS stands as an example of a simple tool that can bridge the gap between traditional transmissive pedagogy and the creation of authentic and collaborative learning spaces. © 2012 Dawson, S., Macfadyen, L., Risko, E., Foulsham, T. & Kingstone, A.
Conference Paper
Full-text available
Video-Based Learning (VBL) has a long history in the educational design research. In the past decade, the interest in VBL has increased as a result of new forms of online education, such as flipped classrooms, and most prominently MOOCs. VBL has unique features that make it an effective Technology-Enhanced Learning (TEL) approach. This study critically analyzed the current research of VBL published in 2003-2013 to build a deep understanding on what are the educational benefits and effectiveness that VBL has on teaching and learning. 67 peer reviewed papers were selected in this review and categorized into four main dimensions, namely, effectiveness, teaching methods, design, and reflection. In the light of the discussion of current research in terms of these categories, we present the future vision and research opportunities of VBL that support self-organized and network learning
Article
Full-text available
This case study examined the integration of a media annotation tool (MAT) into the learning and assessment activities of an undergraduate teaching (physical education) course. The media form or artefact for annotation was video recordings demonstrating individual learners' teaching practice. The learners categorised (marked sections) and annotated their videos and received peer and teacher feedback within the tool. Their use of MAT was analysed to determine if this learning environment was effective in the context of the case, to critically reflect upon and evaluate pre-service teaching practice. The research site was RMIT University, Melbourne, and data was collected from the pilot users of MAT, a third year class and their teacher/key academic, using pre- and post-surveys and interactive process interviews (combined sessions of direct observation and semi-structured interviews). The data indicated MAT was effective for the main learning purpose of the case, but also identified some areas for further consideration.
Article
- This paper describes the process of inducting theory using case studies from specifying the research questions to reaching closure. Some features of the process, such as problem definition and construct validation, are similar to hypothesis-testing research. Others, such as within-case analysis and replication logic, are unique to the inductive, case-oriented process. Overall, the process described here is highly iterative and tightly linked to data. This research approach is especially appropriate in new topic areas. The resultant theory is often novel, testable, and empirically valid. Finally, framebreaking insights, the tests of good theory (e.g., parsimony, logical coherence), and convincing grounding in the evidence are the key criteria for evaluating this type of research.
Chapter
Interest in learning strategies and study tactics seems at all-time high. In this chapter, approaches to studying are conceptualized as self-regulated learning whereby learners survey tasks, generate goals and devise plans for studying, study and make on-the-spot adaptations, and may substantially renovate studying methods. Relative to this framework, the content of books describing learning strategies is examined, methods used to research learners’ approaches to studying are analyzed, and results of studies investigating the effects of learning strategies/study tactics are summarized. Shortcomings of research methodologies may account for overall modest and rather variable findings about the benefits of learning strategies/study tactics. Also, factors are identified that may hinder learners’ uptake and productive use of learning strategies/study tactics. A plan is proposed for improving and accelerating research on learning strategies. It involves learners as colleagues in personalized programs of research that, in the aggregate, advance learning science.
Article
This unique book is the first comprehensive guide to the discovery, analysis, and evaluation of natural experiments – an increasingly popular methodology in the social sciences. Thad Dunning provides an introduction to key issues in causal inference, including model specification, and emphasizes the importance of strong research design over complex statistical analysis. Surveying many examples of standard natural experiments, regression-discontinuity designs, and instrumental-variables designs, Dunning highlights both the strengths and potential weaknesses of these methods, aiding researchers in better harnessing the promise of natural experiments while avoiding the pitfalls. Dunning also demonstrates the contribution of qualitative methods to natural experiments and proposes new ways to integrate qualitative and quantitative techniques. Chapters complete with exercises, and appendices covering specialized topics such as cluster-randomized natural experiments, make this an ideal teaching tool as well as a valuable book for professional researchers.
Article
A correlational study examined relationships between motivational orientation, self-regulated learning, and classroom academic performance for 173 seventh graders from eight science and seven English classes. A self-report measure of student self-efficacy, intrinsic value, test anxiety, self-regulation, and use of learning strategies was administered, and performance data were obtained from work on classroom assignments. Self-efficacy and intrinsic value were positively related to cognitive engagement and performance. Regression analyses revealed that, depending on the outcome measure, self-regulation, self-efficacy, and test anxiety emerged as the best predictors of performance. Intrinsic value did not have a direct influence on performance but was strongly related to self-regulation and cognitive strategy use, regardless of prior achievement level. The implications of individual differences in motivational orientation for cognitive engagement and self-regulation in the classroom are discussed.