Content uploaded by Srecko Joksimovic
Author content
All content in this area was uploaded by Srecko Joksimovic on Mar 03, 2018
Content may be subject to copyright.
A preview of the PDF is not available
Linguistic Characteristics of Reflective States in Video Annotations under Different Instructional Conditions
Abstract and Figures
Video-based self-reflection and annotation is receiving increasing attention within the education literature. The importance of such technologies in education relate, in part, to the interactive nature and functionality these tools bring to aid learning engagement. In particular, these tools are well aligned with the need to promote and develop student meta-cognitive skills through the use of self-reflection activities. However, in the context of video-based learning environments, the nature of a students’ self-reflective process is not well understood. We attempt to address this gap in the literature in two main ways. First, we developed a coding instrument to assess the depth of a students’ self-reflection captured through the use of a video annotation tool. We then explored the linguistic and discourse properties of the student self-reflections using Coh-Metrix, a theoretically grounded computational linguistics facility. The adopted approach applies comprehensive analysis of language and discourse features associated with the specificity of students’ internal self-feedback, which is externalized as self-reflections in video annotations. The results suggest that levels of self-reflection have characteristically different linguistic properties, and these differences align with the underlying cognitive mechanisms associated with distinct reflective activities. The paper provides a detailed discussion of the findings in the context of the theoretical, methodological, and practical implications associated with video-based self-reflection and video annotation.
Figures - uploaded by Srecko Joksimovic
Author content
All figure content in this area was uploaded by Srecko Joksimovic
Content may be subject to copyright.
... Several models were developed to analyze the content of reflective writing, both in terms of the broad themes covered [6], [9] and the depth of reflection exhibited [5], [7], [10] by learners. While traditional approaches to assessing reflective writing have utilized manual content analysis, with the advances in natural language processing and computational content analytics, there have been few attempts to use automated approaches to assess student reflective writing [1], [3], [11]- [15]. These automated assessment studies have mainly evaluated the descriptive nature of the writing contents triggered by an experience, with a lesser focus on understanding the specificity of learners' reflections, primarily positioned in traditional classroom settings and professional degree programs within higher education. ...
... While the first two approaches are expert-driven and utilize patterns elicited by experts, the machine learning approaches are fully automated, detecting the patterns without any human supervision [12]. The studies adopting machine-learning approaches are based on natural language processing (NLP) methods, utilizing textual features derived from Coh-Metrix indices [1], [11], Linguistic Inquiry and Word Count variables [1], [35]- [38], or simple linguistic utterances such as N-grams [1], [12] to evaluate reflective writing into different categories. ...
... While there has been a considerable number of attempts within educational research to automate the process of evaluating self-reflection and reflective writing, most of these automated systems are based on iterative or hybrid iterativevertical models [1], [3], [11], [12] with a few exceptions [14], [15]. As a consequence, these models thereby cannot distinguish learners based on reflective depth. ...
Self-reflection and reflective writing have been pivotal for developing a deep understanding of concepts and fostering professional competency in learners. The confluence of the importance of reflective practices within the educational curriculum and the increased proliferation of technology have resulted in numerous studies of how to use automated approaches to analyze broad themes of reflection exhibited by learners in higher educational settings. However, there is a dearth of research in the context of automated analysis of reflective writing demonstrated by professionals within workplace learning. Building on a four-level reflective content analysis model, this article evaluates the depth of reflection exhibited by learners within a professional development MOOC using an automated assessment classifier. Our results identify the varying association of linguistic features across these different levels of reflection. The proposed approach can effectively support the deployment at scale of the labor-intensive evaluation of reflective writing by highly trained professionals.
... Todos ellos son nuevas oportunidades, técnicas y modelos para la formación de docentes que han surgido al albor del desarrollo tecnológico en la digitalización y computación del vídeo, pero que de algún modo sirven de soporte y recurso, y en algunos casos algo más que esto, al definir el sentido de los modelos pedagógicos con los que se asocian, como de las concepciones acerca de lo que se entiende que debe ser la formación de docentes. Tal es el caso del Modelo microteaching (Sezen-Barrie et al., 2014), el Modelo TPACK (Cabero, 2014;Leiva-Núñez et al., 2018) y otros modelos que preconizan el análisis sobre la práctica y la interpretación de significados a través de esta reflexión -en la acción y sobre la acción- (Schön, 1992), reflexión crítica y la autorregulación del aprendizaje (Kim Chau & Mei Po, 2021), el análisis del discurso narrativo (Joksimović et al., 2018), la investigación acción (Ardley & Johnson, 2018), las redes profesionales, y comunidades de prácticas -CoP-(Brown & Poortman, 2018) y de indagación -CoI- (Blau & Shamir-Inbal, 2021), siendo estas dos últimas las más generalizadas actualmente y más propias del uso de las metodologías y plataformas tecnológicas de AV, al dotarlas de un aprendizaje más activo, formativo, reflexivo, y con más retroalimentación sobre el recurso vídeo, tanto entre los estudiantes del mismo grupo de clase como dotando de significados y valores en la comunicación evaluativa entre los docentes y los estudiantes. ...
... All of them are new opportunities, techniques and models for teacher training that have emerged at the dawn of technological development in video digitization and computing, but that somehow serve as a support and resource, and in some cases something more than this, by defining the meaning of the pedagogical models with which they are associated, as of the conceptions about what it is understood that teacher training should be. Such is the case of the Microteaching Model (Sezen- Barrie et al., 2014), the TPACK Model (Cabero, 2014;Leiva-Núñez et al., 2018) and other models that advocate the analysis on practice and the interpretation of meanings through this reflection -in action and on action- (Schön, 1992), critical reflection and self-regulation of learning (Kim Chau & Mei Po, 2021), narrative discourse analysis (Joksimović et al., 2018), action research (Ardley & Johnson 2018), professional networks, and communities of practice -CoP-(Brown & Poortman, 2018) and communities of inquiry -CoI- (Blau & Shamir-Inbal, 2021), the latter two being the most widespread nowadays and more typical of the use of VA methodologies and technological platforms, by providing them with a more active, formative, reflective learning, and with more feedback on the video resource, both among students in the same class group and by providing meanings and values in the evaluative communication between teachers and students. ...
xiste una tradición en el uso de videos digitales en la formación del profesorado donde las anotaciones de vídeo son una tecnología emergente que ofrece un uso más activo a los vídeos disponibles en internet y los creados en las prácticas de aula. Sus posibilidades técnicas consisten en crear anotaciones en los vídeos de forma individual para el portafolios o compartida en grupos de clases y redes profesionales. Como tecnología en evolución ofrece nuevas posibilidades para la formación docente mediante etiquetado social e hipervínculos que pueden ser analizados. El presente estudio es una revisión de la literatura reciente desde 2018 a 2022 sobre el uso de las anotaciones multimedia en la formación docente mediante el protocolo PRISMA, seleccionando desde bases de datos (WOS, Scopus y ERIC) 244 referencias que tras aplicar los criterios de exclusión e inclusión se obtuvieron 25 referencias que aplican sólo en el tópico de la formación de docentes. Se ofrece una lista de las plataformas más empleadas, los métodos de investigación utilizados, el perfil de los participantes y los tópicos de aplicación. Es compartida la utilidad de las anotaciones de video en la formación de docentes especialmente para el análisis y reflexión sobre las prácticas.
... Many solutions have been proposed to increase video viewers' engagement and motivation levels, including the introduction of training sessions in self-directed learning strategies (Garc ıa Botero et al., 2019), the embedding of peer feedback sessions , the provision of extra assistance in instructional videos (Bonafini et al., 2017) and the embedment of students' designed multimedia learning resources and assessment tasks (Joksimovi c et al., 2019). One of the convenient ways is to provide additional scaffolds inside the videos, that is, the presentation of captions, especially for secondlanguage viewers who need to develop content knowledge and overcome language barriers simultaneously (Yang & Chang, 2014). ...
The benefits of using social annotations to promote collaborative learning experiences have been investigated in the context of digital reading and listening, while its implications in multimedia learning contexts remain unclear. To investigate the influence of social annotations on multimedia language learning, we invited 16 African participants who were learning Chinese as a second language through a MOOC platform in China. They were given full autonomy to study and to shift instructional videos between three modes: no visual aids, linguistic captions, and social annotations. Guided by self-determination theory, the qualitative data were collected through one-to-one interviews, while quantitative data, such as the duration of watching and the number of questions raised by participants, were obtained from observations. Statistical analysis suggests that MOOC learners demonstrated a higher level of engagement and motivation in the social annotation mode than in the other two modes. These statistical findings can be explained from the following three perspectives: content and emotion-related social annotations by other L1 viewers can assist in (1) checking their comprehension of the video by comparing their understanding with others; (2) adjusting learning goals at both the linguistic and content level and therefore using self-regulated learning strategies to a greater extent, especially in actively seeking assistance and other learning resources, and (3) relating the video content to the teaching curriculum/personal experience/learning goals, therefore creating a sense of belonging to a larger language learner community as language learners, interpreters and contributors to the video social annotations.
... In addition, the system may help to identify the cognitive depth of accumulated timeline-anchored comments through content analysis and adapt the layout accordingly. Recent researchers (Joksimović et al., 2019) have proposed coding instruments to identify the depth of self-reflection that is revealed in annotations to video lecture timelines. Such text-analytic methods can be adopted to design platforms that can adapt the layout to content depth automatically. ...
Many video sites or learning platforms allow real-time chatting or asynchronous commenting on specific time points during video lectures. Comments, as user-generated knowledge, facilitate social interaction but also affect cognitive learning. The visual layout of these comments can affect learners' attention and learning, but the effect has rarely been studied. This study compares two common layouts (embedded vs. separated) and considers the content depth of comments through a laboratory eye-tracking experiment involving 40 participants. The results suggest that, with both layouts, learners switched attention to the comments every 10 seconds and stayed focused for 1.3 seconds on average before returning attention to the video. With an embedded layout, learners switched attention more frequently to the comments and remembered more surface-level comments. With a separate layout presenting deep-level comments, learners searched for information faster and performed better on open-book quizzes. We outline the design implications of using timeline-anchored comments to promote online learning.
... The focus is on gaining insight into distinct learning skills and can take place before, during or after the learning activity. Joksimovic et al. (2019) show that metacognitive awareness can promote reflective states of consciousness. Their study builds on the assumption concerning how metacognitive knowledge shapes this awareness of own cognitive processes and how one understands, manages or regulates these processes in order to enhance learning. ...
p>La formación para la competencia argumentativa se produce con el distanciamiento del ciudadano frente a mensajes de los medios cuando emite juicios de valor. Esta competencia es básica en profesionales de educación, pero ¿cómo desarrollamos dicha competencia? Se utilizan herramientas y metodología de anotaciones multimedia sobre video. El diseño de investigación fue mixto utilizando análisis correlacional de medias, estadística textual y análisis cualitativo mediante etiquetado social con tres indicadores de una rúbrica de argumentación: justificación, evidencia y conclusiones. Participaron 39 estudiantes de postgrado, que produjeron 74 anotaciones sobre texto y video, y 191 evaluaciones entre pares y con heteroevaluación, sobre las anotaciones o juicios de valor emitidos. Los resultados indican similitud entre las anotaciones sobre el contenido analizado, pero no sobre la cantidad y calidad. Se concluye que los valores de las anotaciones de vídeo son superiores a las de texto, la aplicación de la rúbrica mejoró los resultados argumentativos.</p
In der Lehrerinnen- und Lehrerbildung hat der Einsatz videobasierter Methoden Tradition und gewinnt weiter an Zuspruch. Den Vorteilen dieser Form stehen jedoch auch Nachteile gegenüber: Die Flüchtigkeit der Videoform erschwert eine gehaltvolle gemeinsame Auseinandersetzung mit den Inhalten, besonders in asynchronen Online-Settings. Tools zur kollaborativen Annotation von Videos können hier einen Ausweg bieten. Der Markt hierfür ist jedoch disparat und unübersichtlich, sodass es schwierig ist, die richtige Anwendung zu finden. Dabei will der vorliegende Beitrag helfen, indem die maßgeblichen Anwendungen identifiziert und mit ihren Vor- und Nachteilen aufgeführt werden.
In der Lehrerinnen- und Lehrerbildung hat der Einsatz videobasierter Methoden Tradition und gewinnt weiter an Zuspruch. Den Vorteilen dieser Form stehen jedoch auch Nachteile gegenüber: Die Flüchtigkeit der Videoform erschwert eine gehaltvolle gemeinsame Auseinandersetzung mit den Inhalten, besonders in asynchronen Online-Settings. Tools zur kollaborativen Annotation von Videos können hier einen Ausweg bieten. Der Markt hierfür ist jedoch disparat und unübersichtlich, sodass es schwierig ist, die richtige Anwendung zu finden. Dabei will der vorliegende Beitrag helfen, indem die maßgeblichen Anwendungen identifiziert und mit ihren Vor- und Nachteilen aufgeführt werden.
This chapter reports on teaching an Auditing and Assurance course using team-based scenario learning (TBSL) for undergraduates. Introducing TBSL as a collaborative active learning tool encourages students to learn from one another and provides opportunities for communication, discussion, argument, and elaboration. The chapter utilises simulated scenarios in three different situations such as analysis of audit risk factors, material misstatements, and audit and non-audit services. It is followed by discussion forums, and videos are utilised to represent real-life audit situations. The combination of these various learning activities encourages diverse arguments being put forward by students in their discussions, and creates opportunities for critical thinking and to interact with the group members. In this regard, attention is given to student group formation and workshops on collaborative skills are conducted. The TBSL requires students to be confident to give peer feedback and engage in self-reflection on their learning journey. In line with the authentic learning interactions, the scenario-based assessment focuses on rewarding students on the development of essential skills and capabilities needed in auditing.
Video annotation (VA), a tool which allows commentaries to be synchronized with video content has recently received significant research attention in education. However, the application contexts of these studies are varied and fragmented. A review was therefore undertaken with the objectives to find out the extent to which the use of VA has been explored for different instructional purposes and summarize the potential affordances of VA in supporting student learning. Articles related to the use of VA in education context were searched from 2011 to 2020 (Nov). Of the final 32 eligible studies, it was found that VA tools were used predominantly to develop teaching practices, enhance learners’ conceptual understanding of video content and develop workplace skills as well as clinical practices. Five most dominant educational affordances of VA tools were summarized as follows: (1) facilitating learners’ reflection (2) facilitating feedback process (3) enhancing comprehension of video content (4) promoting students’ learning satisfaction and positive attitude and (5) convenience and ease. With the outstanding weight of research evidence gained on educational affordances offered by VA, it is convincing that advancing the use of VA in education can further expand the learning opportunities in 21st century classrooms. Keywords: Affordances, Education, Feedback, Learners’ reflection, Video annotation
Completion rates for massive open online classes (MOOCs) are notoriously low, but learner intent is an important factor. By studying students who drop out despite their intent to complete the MOOC, it may be possible to develop interventions to improve retention and learning outcomes. Previous research into predicting MOOC completion has focused on click-streams, demographics, and sentiment analysis. This study uses natural language processing (NLP) to examine if the language in the discussion forum of an educational data mining MOOC is predictive of successful class completion. The analysis is applied to a subsample of 320 students who completed at least one graded assignment and produced at least 50 words in discussion forums. The findings indicate that the language produced by students can predict with substantial accuracy (67.8 %) whether students complete the MOOC. This predictive power suggests that NLP can help us both to understand student retention in MOOCs and to develop automated signals of student success.
This paper presents the results of a natural experiment investigating the effects of instructional conditions and experience on the adoption and sustained use of a learning tool. The experiment was conducted with undergraduate students, enrolled into four performing art courses (N=77) at a research intensive university in Canada. The students used the video annotation software CLAS for course‐based self‐assessment on their performances. Although existing research offers insights into the factors predicting students' intentions of accepting a learning tool, much less is known about factors that affect actual adoption and sustained tool use. The study explored the use of CLAS amongst undergraduate students in four courses across two consecutive semesters. Trace data of students' tool use, graph‐based measures of metacognitive monitoring, and text cohesion of video annotations were used to estimate the volume of tool use and the quality of the learning strategy and learning products created. The results confirmed that scaffolding (e.g., graded activity with instructional feedback) is required to guide students' initial tool use, although scaffolding did not have an independent significant effect on the quantity of tool use. The findings demonstrated that the use of the tool is strongly influenced by the experience an individual student gains from scaffolded conditions. That is, the students sustained their use of the learning tool in future courses even when the tool use was not graded nor was instructional feedback provided. An important implication is that students' tool use is not solely driven by motivation – rather, it is shaped by instructional conditions and experience with the tool use.
The purpose of this article is to explore a model of learning that proposes that various learning strategies are powerful at certain stages in the learning cycle. The model describes three inputs and outcomes (skill, will and thrill), success criteria, three phases of learning (surface, deep and transfer) and an acquiring and consolidation phase within each of the surface and deep phases. A synthesis of 228 meta-analyses led to the identification of the most effective strategies. The results indicate that there is a subset of strategies that are effective, but this effectiveness depends on the phase of the model in which they are implemented. Further, it is best not to run separate sessions on learning strategies but to embed the various strategies within the content of the subject, to be clearer about developing both surface and deep learning, and promoting their associated optimal strategies and to teach the skills of transfer of learning. The article concludes with a discussion of questions raised by the model that need further research.
The rapidly-developing 21st century world of work and knowledge calls for self-directed lifelong (SDL) learners. While higher education must embrace the types of pedagogies that foster SDL skills in graduates, the pace of change in education can be glacial. This paper describes a social annotation technology, the Collaborative Lecture Annotation System (CLAS), that can be used to leverage existing teaching and learning practices for acquisition of 21st Century SDL skills. CLAS was designed to build upon the artifacts of traditional didactic modes of teaching, create enriched opportunities for student engagement with peers and learning materials, and offer learners greater control and ownership of their individual learning strategies. Adoption of CLAS creates educational experiences that promote and foster SDL skills: motivation, self-management and self-monitoring. In addition, CLAS incorporates a suite of learning analytics for learners to evaluate their progress, and allow instructors to monitor the development of SDL skills and identify the need for learning support and guidance. CLAS stands as an example of a simple tool that can bridge the gap between traditional transmissive pedagogy and the creation of authentic and collaborative learning spaces. © 2012 Dawson, S., Macfadyen, L., Risko, E., Foulsham, T. & Kingstone, A.
The 4th edition of the Handbook of Research on Educational Communications and Technology expands upon the previous 3 versions, providing a comprehensive update on research pertaining to new and emerging educational technologies. Chapters that are no longer pertinent have been eliminated in this edition, with most chapters being completely rewritten, expanded, and updated Additionally, new chapters pertaining to research methodologies in educational technology have been added due to expressed reader interest. Each chapter now contains an extensive literature review, documenting and explaining the most recent, outstanding research, including major findings and methodologies employed. The Handbook authors continue to be international leaders in their respective fields; the list is cross disciplinary by design and great effort was taken to invite authors outside of the traditional instructional design and technology community. © Springer Science+Business Media New York 2014. All rights reserved.
The focus of this chapter is to examine the psychological principles of metacognition and identify how second language (L2) educators can increase learners’ awareness of their metacognition. The chapter will present research findings to support the integration of metacognitive strategy awareness training within an L2 curriculum. Training learners to be more cognizant and reflective of how they engage in language learning facilitates the development of learner autonomy.
Learners self-regulate learning when they examine the learning environment and monitor learning processes to inform decisions about managing learning processes. Under the assumption that learners are agents with inherent capability to make and act on decisions, self-regulated learning (SRL) is ubiquitous. Decisions learners make about regulating learning, however, can promote or impede achievement and other valued outcomes. Research on SRL and its roles in education faces multiple challenges. Evidence suggests productive SRL can be fostered through evidenced-based designs for learning activities and environments, by promoting particular beliefs among teachers, and helping learners discover value in education.