No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Using Process Mining to Examine the Sustainability of Instructional Support: How Stable Are the Effects of Metacognitive Prompting on Self-Regulatory Behavior?
Abstract
The current study investigates the sustainability of metacognitive prompting on self-regulatory behavior using a Process Mining approach. Previous studies confirmed beneficial short-term effects of metacognitive prompts on the learning process and on learning outcomes. However, the question of how stable these effects are for similar tasks in the future so far remains unanswered. Also, the use of online trace methods and the emergence of new analytical approaches allow deeper insights into the sequential structure of learning behavior. Therefore, we examined long-term effects of instructional support on sub-processes of self-regulated learning using Process Mining. Think-aloud protocols from 69 university students were measured during two hypermedia learning sessions about Educational Psychology. Metacognitive prompts supported the experimental group (n = 35)only during the first session. Based on a process model generated by using the data of the first learning task, we analysed the sustainability of effects during the second learning session. Results showed significant differences between the experimental and control group regarding the frequency of metacognitive strategies, which remain stable over time. Additionally, the application of Process Mining indicated which sequences of learning events were transferred to the second session. Our findings demonstrate the benefits of evaluating instructional support using analysis techniques that take into account the sequential structure of learning processes. While the results provide initial evidence for sustainable long-term effects on self-regulatory behavior, they have to be replicated in future research.
... For example, Li et al. (2022) assessed medical students' self-regulation of learning and concluded that outperformers had a higher transition probability across the different SRL phases of SRL than their counterparts. Banner and Sonnenberg Sonnenberg & Bannert, 2019) presented graphs relating to students' self-regulated learning. They explained that examining SRL as a process can understand a high variation hidden in the organization of learning activities. ...
... They found that the SRL process pattern of the academically successful students was closer to what SRL models described and demonstrated deep processing activities (e.g., monitoring) in their learning. Sonnenberg and Bannert (2019) leveraged PM to test the long-term effects of metacognitive prompts on SRL behaviors. They checked the degree of conformance between the first-and the second learning session and showed that metacognitive prompts had long-term effects, evidenced by the sustained SRL process in two learning sessions. ...
... As such, researchers know reporters' thoughts and observe their actions, which increases the validity of think-aloud data. The think-aloud technique has been employed to capture SRL events before, during, and after a complex task Sonnenberg & Bannert, 2019). And the think-aloud data can indicate specific rather than general regulatory events (Greene & Azevedo, 2009). ...
Learning analytics dashboards (LADs) are often used to display real-time data indicating student learning trajectories and outcomes. Successful use of LADs requires teachers to orient their dashboard reviews with clear goals, apply appropriate strategies to interpret visualized information on LADs and monitor and evaluate their interpretations to meet goals. This process is known as self-regulated learning (SRL). Critical as it is, little research investigates teachers’ SRL in LAD usage. The present study addressed the gap by examining teachers’ SRL and sought to understand how teachers’ SRL relates to their use of LADs. To this end, a case study was designed in which ten participants were invited to use a LAD for asynchronous online problem-based learning. Think-aloud techniques and process mining methods were applied. The findings show that teachers were cognitive regulation in the early stage of LAD usage and became more metacognitive regulated later. The comparison of SRL between the good and the weak regulators indicates that the good self-regulators enacted more monitoring and evaluation events. Thus their regulator pattern was more non-linear. The qualitative analysis of think-aloud protocols reveals that teachers with good SRL are more likely to use the LAD to diagnose issues in student learning and collaboration. The study highlights the importance of SRL for teachers’ success in using LAD for data-driven instructions. The study also reinforces the importance of fostering teachers’ SRL, which accounts for teachers’ professional success in the digital era.
... However, little attention is paid to how the behavior patterns of HA students affect the academic performance of LA students' in OSDL. Majority of existing studies use think-aloud data to analyze the behavior patterns of the learning process (Heirweg et al., 2020;Sonnenberg & Bannert, 2019;Toh & Kirschner, 2020). Such data are likely to interfere with students' learning though. ...
... In the online learning context, every behavior of a student is due to purposeful self-direction. Majority of current research uses self-reporting tools, such as think-aloud protocols (Heirweg et al., 2020;Sonnenberg & Bannert, 2019) to determine online students' behavior patterns. Think-aloud protocols are useful when mapping individuals' OSDL activities during actual learning and provide atomic perspectives into the strategies adopted. ...
... When designing alternative differentiation paths, teachers can provide LA students with HA student behavior patterns for OSDL reference. This study also provides fresh methodological perspectives for evaluating OSDL by utilizing PM techniques to determine behavior patterns from log data (Sonnenberg & Bannert, 2019). ...
One of the recognized ways to enhance teaching and learning is having insights into the behavior patterns of students. Studies that explore behavior patterns in online self-directed learning (OSDL) are scant though. In addition, the focus is lacking on how high-achieving (HA) students’ behavior patterns affect the academic performance of low-achieving (LA) students. To fill these research gaps, this study investigates (1) how the behavior patterns in OSDL vary between HA and LA students and (2) how HA students’ behavior patterns affect LA students’ academic performance. We used three perspectives of learning achievement, engagement, and cognitive load to examine academic performance. By utilizing process mining, we reviewed the log data of 71 college students on the Moodle platform and designed a pretest–posttest test without a control group. Results show obvious variances in the behavior patterns between HA and LA students. In particular, HA students performed more OSDL behaviors; their behavior patterns were more in line with self-directed logic. By contrast, LA students exhibited unmethodical behavior patterns; they were unable to process course content in depth. An instructional intervention was created with HA students’ behavior patterns as basis. The engagement of LA students increased, and their cognitive load was reduced after the instructional intervention. However, their learning achievement did not increase substantially. The interview results were consistent with the quantitative data. These findings indicate that the behavior patterns of HA students can shed light on how to guide the OSDL of LA students. This study also provides fresh methodological perspectives for assessing OSDL.
... In SRL, learners are expected to engage in goal setting, planning, monitoring and evaluating the enactment of plans, and reflecting on outcomes (Winne & Hadwin, 1998). By regulating, students seek an in-depth understanding of complex topics and therefore gain increased performance (Sonnenberg & Bannert, 2019). Given the complexity of SRL, researchers have invested efforts to investigate salient features of SRL that may predict student learning outcomes. ...
... Winne and Perry (2000) proposed that SRL measurement should be online, focusing on the actual regulatory activities of learners while learning is in the process. SRL activities identified by the event measures are considered more accurate and objective than those by self-reported measures (Sonnenberg & Bannert, 2019). However, there are also concerns about whether the data collected by a single tool (e.g., traces) are sufficient to identify the salient feature of SRL (Azevedo & Gašević, 2019). ...
... Learners verbally express their simultaneous thoughts while performing a learning task (Ericsson & Simon, 1980). Such a technique is unobtrusive and does not change the sequence and content of thoughts, which is valuable for capturing more implicit activities, such as monitoring activities (Sonnenberg & Bannert, 2019). Despite advantages, relying on sing data for SRL measurement might cause issues. ...
Teachers’ self-regulated learning (SRL) plays a crucial role in developing technological pedagogical content knowledge (TPACK), a complex professional skill. It is crucial to identify teachers’ SRL activities that may lead to favorable TPACK. Previous studies have focused on the analysis of individual data sources from self-reported surveys or log files, which are insufficient to capture all SRL activities in the TPACK context. While multimodal learning analytics (MMLA) has the potential to improve SRL measurement, it remains unknown how multimodal data collected from different sources can be combined to identify salient features of SRL activities and examine how TPACK outcomes can be predicted by SRL activities identified from multimodal data. This study combined multimodal data from computer logs and think-aloud data to analyze teachers’ SRL activities in designing a technology-integrated lesson. We identified the salient features of SRL from the combined data and explored how identified SRL activities might predict TPACK outcomes reflected in teacher-generated lesson plans. The results of random forest regression analysis show that three SRL activities from the logs and two from the think-aloud data formed the best combination that explained a significant proportion of variances in TPACK performance. The impact of MMLA in SRL measurement and the implication of this study are discussed.
... For example, presenting prompts at spaced intervals during a 20-minute learning session improved metacognitive activities, as opposed to presenting prompts just once before learning (Thillmann et al., 2009). Similar results were reported that distributing prompts throughout the learning session aided students' node selections for each online navigation step (e.g., Sonnenberg & Bannert, 2019). Thus, it was anticipated that presenting prompts throughout the learning episode would have stronger effects compared with presenting prompts once. ...
... With this in mind, researchers suggested an in-depth analysis of process data (Azevedo & Gaševi c, 2019;Bannert et al., 2015). Thinkaloud protocols have been used to ascertain different types of SRL activities (e.g., Sonnenberg & Bannert, 2019) and self-explanation creates a mental state conducive for the calibration between judgement of learning and actual performance (Chi et al., 1994). Another fine-grained approach is trace technology which could capture online data and assess how students respond to prompts in real time (Hadwin et al., 2007). ...
... the key aspects of the task or the specific features of a problem. For example, directed prompts were effective in activating planning strategies by asking students to set concrete sub-goals (Lehmann et al., 2014), or stimulating reflection by asking students to choose from a list of reasons for navigation decisions (Sonnenberg & Bannert, 2019). The rationale for using specific prompts is that they provide explicit guidance and increase the saliency of task-relevant features, which helps learners to allocate their cognitive resources in an optimal way. ...
Background
It has been assumed that prompting students to plan, monitor and evaluate their learning process could stimulate strategy use and thereby improve learning outcomes.
Objectives
This study aimed to examine the effects of metacognitive prompts on students' self‐regulated learning (SRL) and learning outcomes in the context of computer‐based learning environments (CBLEs).
Methods
To achieve this, the current study took a meta‐analytic approach to critically evaluate evidence for the effectiveness of metacognitive prompts and identify potential moderators of the effects.
Results and conclusions
With random‐effects models, the results showed that metacognitive prompts significantly enhanced SRL activities (g = 0.50, 95% confidence interval [0.37, 0.63]) and learning outcomes (g = 0.40, 95% confidence interval [0.31, 0.49]) relative to the control conditions. Furthermore, moderator analyses revealed that the effects varied as a function of three prompts features: feedback, specificity and adaptability.
Implications
Developing task‐specific, individual‐adaptive prompts and feedback should be a design principle in CBLEs, such that the prompt effect could be retained, sustainably enhanced and transferred to novel situations.
... Computer-supported collaborative learning (CSCL) "refers to situations in which computer technology plays a significant role in shaping the collaboration" (Goodyear et al., 2014, p. 440). Accounting for tracking student learning over time has grown as an area of interest for CSCL researchers, many of whom draw on Reimann (2009) research on process mining (e.g., Thompson et al., 2014;Malmberg et al., 2015;Sonnenberg and Bannert, 2018). In many cases, CSCL research addresses the challenge of accounting for individual and group-level learning (Cress and Hesse, 2013;Stahl et al., 2014). ...
... DTMC statistical models have been applied in many fields to model complex real-world processes, such as estimating the time of travel on highways (e.g., Yeon et al., 2008) or inferring the magnitude of financing costs (Hennessy and Whited, 2007). In the learning sciences, DTMC has been used to model patterns of decision-making (e.g., Reimann, 2009), problem-solving ; selfregulation (e.g., Malmberg et al., 2015;Sonnenberg and Bannert, 2018) and idea generation . ...
Introduction
Researchers in the learning sciences have been considering methods of analysing and representing group-level temporal data, particularly discourse analysis, in Computed Supported Collaborative Learning for many years.
Methods
This paper compares two methods used to analyse and represent connections in discourse, Discrete Time Markov Chains and Epistemic Network Analysis. We illustrate both methods by comparing group-level discourse using the same coded dataset of 15 high school students who engaged in group work. The groups were based on the tools they used namely the computer, iPad, or Interactive Whiteboard group. The aim here is not to advocate for a particular method but to investigate each method’s affordances.
Results
The results indicate that both methods are relevant in evaluating the code connection within each group. In both cases, the techniques have supported the analysis of cognitive connections by representing frequent co-occurrences of concepts in a given segment of discourse.
Discussion
As the affordances of both methods vary, practitioners may consider both to gain insight into what each technique can allow them to conclude about the group dynamics and collaborative learning processes to close the loop for learners.
... Approaches for micro-level analyses are necessary to obtain insights about SRL from such moment-to-moment data [4,6,7]. Researchers have explored SRL using multiple approaches, such as knowledge engineering [8], sequential pattern mining [9], lag-sequential analysis [10], statistical discourse analysis [11], and process mining [12,13]. The current study employs an emerging method, epistemic network analysis (ENA) [14], for the in-depth analyses of SRL processes. ...
... In contrast, process mining depicts a holistic SRL process, where actions have directional connections [12]. This technique has been applied [13] to compare processes between groups who did/did not receive metacognitive prompts when studying topics in educational psychology. However, process mining does not allow a global statistical test for the difference between groups and different weighting for individuals [19]. ...
The micro-level analyses of how students’ self-regulated learning (SRL) behaviors unfold over time provides a valuable framework for understanding their learning processes as they interact with computer-based learning environments. In this paper, we use log trace data to investigate how students self-regulate their learning in the Betty’s Brain environment, where they engage in three categories of open-ended problem-solving actions: information seeking, solution construction and solution assessment. We use Epistemic Network Analysis (ENA) to provide us with an overall understanding of the co-occurrences between action types both within and between the three action categories. Comparisons of epistemic networks generated for two groups of students, those with low and high performance, provided us with insights into their self-regulated behaviors.
... Process mining uses algorithms to analyze and identify process models (i.e., the dominant process flows) from event data (i.e., recorded logged data or coded events from verbal or behaviour protocols). It takes into account all events to generate a comprehensive process model, facilitating the analysis of the relative sequence and arrangement of these events (Sonnenberg & Bannert, 2019). As such, this method is particularly relevant in the context of regulated learning to capture the temporal and sequential nature of the specific interactions for deliberation that draw out the control processes of regulation. ...
Socially shared regulation in learning (SSRL) contributes to successful collaborative learning (CL). Empirical research into SSRL has received considerable attention recently, with increasingly available multimodal data, advanced learning analytics (LA), and artificial intelligence (AI) providing promising research avenues. Yet, integrating these with traditional datasets remains a challenge in SSRL research due to the misalignment between theoretical constructs, methodological assumptions, and data structure. To address this challenge and expand our understanding of the nature of SSRL, the present research adopted a human–AI collaboration approach in a three-layer analysis to examine group interactions in response to cognitive and emotional regulation triggering events. Two-level theoretical lenses — macro-level (regulatory aspects) and micro-level (deliberative interactions) — were used to analyze 2,125 utterances from video-recorded tasks of ten groups of three Finnish secondary students (N=30). Results showed two types of deliberation patterns for SSRL, namely 1) the Plan and Implementation Approach (PIA) associated with adaptive patterns, and 2) the Trials and Failure Approach (TFA) associated with maladaptive patterns. Our findings revealed that groups often fail to recognize, or are ill-equipped to respond to, emerging regulatory needs. These findings advance SSRL theories and research methodologies by utilizing AI-enhanced LA to offer new insights into group dynamics and regulatory strategies.
... In response to this, numerous studies attempted to foster and support students' SRL skills in online learning environments through a variety of approaches. These methods include open learner models (Bull et al., 2014;Ferreira da Rocha et al., 2023;Guerra et al., 2016Guerra et al., , 2018Kay et al., 2022;Law et al., 2017;Sun et al., 2023;Tacoma et al., 2018;Winne, 2021), dashboards (Alphen & Bakker, 2016;Hsiao et al., 2016;Mejia et al., 2017;Muldner et al., 2015), interventions (Cicchinelli et al., 2018;Jansen et al., 2020;Müller & Seufert, 2018;Zarei Hajiabadi et al., 2023), metacognitive prompts (Engelmann et al., 2021;Pieger & Bannert, 2018;Sonnenberg & Bannert, 2019), and others. For systematic literature reviews of SRL-supporting tools, see Alvarez et al. (2022), Araka et al. (2020), Edisherashvili et al. (2022), Heikkinen et al. (2023), Hooshyar et al. (2020), and Matcha et al. (2020). ...
Background
Self-regulated learning (SRL) strategies can be domain specific. However, it remains unclear whether this specificity extends to different subtopics within a single subject domain. In this study, we collected data from 210 college students engaged in a computer-based learning environment to examine the heterogeneous manifestations of learning behaviors across four distinct subtopics in introductory statistics. Further, we explore how the time spent engaging in metacognitive strategies correlated with learning gain in those subtopics.
Results
By employing two different analytical approaches that combine data-driven learning analytics (i.e., sequential pattern mining in this case), and theory-informed methods (i.e., coherence analysis), we discovered significant variability in the frequency of learning patterns that are potentially associated with SRL-relevant strategies across four subtopics. In a subtopic related to calculations, engagement in coherent quizzes (i.e., a type of metacognitive strategy) was found to be significantly less related to learning gains compared to other subtopics. Additionally, we found that students with different levels of prior knowledge and learning gains demonstrated varying degrees of engagement in learning patterns in an SRL context.
Conclusion
The findings imply that the use—and the effectiveness—of learning patterns that are potentially associated with SRL-relevant strategies varies not only across contexts and domains, but even across different subtopics within a single subject. This underscores the importance of personalized, context-aware SRL training interventions in computer-based learning environments, which could significantly enhance learning outcomes by addressing the heterogeneous relationships between SRL activities and outcomes. Further, we suggest theoretical implications of subtopic-specific heterogeneity within the context of various SRL models. Understanding SRL heterogeneity enhances these theories, offering more nuanced insights into learners’ metacognitive strategies across different subtopics.
... With regard to research question (2), the results indicated that students who received MS support significantly outperformed the non-scaffolding group on selfregulatory efficacy. This result is in accord with previous research indicating that MS has a positive effect on helping students trigger a series of metacognitive thinking and behaviors, which enables them to regulate their own learning process and leads them to positively evaluate their self-regulatory efficacy (Peters & Kitsantas, 2010;Sonnenberg & Bannert, 2019;Valencia-Vallejo et al., 2019). In mobile technology-enhanced classrooms, MS can help students manage the complexities of selfregulation by supporting their ability to adjust learning goals and strategies within a dynamic digital context. ...
Supporting students with effective and sustainable self-regulated learning in mobile technology-enhanced learning environments has become increasingly crucial. This study investigated the effects of metacognitive scaffolding (MS) and inner speech on students’ metacognition and self-regulatory efficacy in mobile technology-enhanced interactive classrooms. A total of 161 eighth-grade students participated in this experiment, and four different conditions (social-and-individual MS, social MS, individual MS, and non-MS support) were randomly assigned to the four groups. The research findings suggested that while groups with MS support significantly promoted students’ metacognition and self-regulatory efficacy compared with the non-scaffolding group, there were no significant differences observed among the three experimental groups. Inner speech could moderate the effect of MS. For students with high-level inner speech, the groups that received MS support outperformed the non-scaffolding group; the group receiving both social and individual MS did not show a synergistic effect; and no significant differences were found between the social MS group and the individual MS group. Notably, for students with low-level inner speech, the social MS group showed a significantly higher level of metacognition than the individual MS group; however, across all groups, differences in self-regulatory efficacy were not significant. The study offers important insights regarding the MS implications in mobile technology-enriched learning environment.
... Information processing theory also illustrates the significant role of metacognition in learners' use of self-regulated learning strategies (Winne & Hadwin, 1998). Specifically, individuals with higher levels of metacognition in learning activities may possess more knowledge about learning and learning strategies and excel at controlling their learning processes, employing more cognitive strategies to evaluate and monitor new understandings of the learning content (Sonnenberg & Bannert, 2019). ...
Based on the questionnaire investigation approach and the investigation of 2989 college students from five colleges in Henan, Jiangsu, and Shanxi as research samples, this study explored the chain mediating roles of learning strategy and learning behavior in the relationship between metacognition and learning engagement. Results indicate that: (1) there is a positive correlation between metacognition and college students' learning engagement; (2) both learning strategy and learning behavior mediate the relationship between metacognition and learning engagement; (3) learning strategy and learning behavior are chain mediation of the relationship between metacognition and college students' learning engagement. This suggests that metacognition is a key factor influencing college students' learning engagement, and this influence functions through the chain mediation of earning strategy and learning behavior.
... Les permite tomar el control de su propio proceso de aprendizaje, identificar obstáculos, aplicar estrategias efectivas y, en última instancia, lograr un aprendizaje más profundo y duradero. La metacognición no solo influye en la calidad del aprendizaje individual, sino que también impacta la forma en que los educadores diseñan y facilitan experiencias de aprendizaje efectivas [23]. ...
La metacognición, la metadidáctica, la metaafectividad y la metatecnología son conceptos fundamentales en el campo de la educación y el aprendizaje. Este artículo se propone llevar a cabo una reflexión sobre estos conceptos y su impacto en los ambientes de aprendizaje. A través de esta reflexión, el artículo busca proporcionar una perspectiva analítica, interpretativa y crítica que pueda enriquecer la toma de decisiones en el diseño de ambientes de aprendizaje efectivos y que pueda informar futuras prácticas educativas. Al comprender a fondo estos conceptos y sus interconexiones, los educadores pueden tomar decisiones más informadas sobre cómo estructurar el aprendizaje y apoyar el desarrollo de habilidades cognitivas y emocionales en los estudiantes. Esto, a su vez, puede llevar a la mejora de las prácticas educativas y a un entorno de aprendizaje más enriquecedor y efectivo en el que los estudiantes puedan prosperar.
... Second, an increasing amount of recent LA research has moved beyond analyzing the frequencies of isolated events, such as page access or assessment attempts, to identifying and visualizing patterns in sequences of related learning events that are more indicative of student learning strategies (Huber & Bannert, 2023;Taub et al., 2018Taub et al., , 2022Wortha et al., 2019). Most of those studies utilize advanced analysis techniques such as sequential pattern mining, clustering, and process mining and interpret the impact of learning design on the learning process through the framework of self-regulated learning (SRL; Saint et al., 2020Saint et al., , 2021Sonnenberg & Bannert, 2019;Taub et al., 2022). ...
The current study measures the extent to which students’ self-regulated learning tactics and learning outcomes change as the result of a deliberate, data-driven improvement in the learning design of mastery-based online learning modules. In the original design, students were required to attempt the assessment once before being allowed to access the learning material. The improved design gave students the choice to skip the first attempt and access the learning material directly. Student learning tactics were measured using a multi-level clustering and process mining algorithm, and a quasi-experiment design was implemented to remove or reduce differences in extraneous factors, including content being covered, time of implementation, and naturally occurring fluctuations in student learning tactics. The analysis suggests that most students who chose to skip the first attempt were effectively self-regulating their learning and were thus successful in learning from the instructional materials. Students who would have failed the first attempt were much more likely to skip it than those who would have passed the first attempt. The new design also resulted in a small improvement in learning outcome and median learning time. The study demonstrates the creation of a closed loop between learning design and learning analytics: first, using learning analytics to inform improvements to the learning design, then assessing the effectiveness and impact of the improvements.
... The design of the trace parser is based on Bannert's framework of SRL in hypermedia learning context, which categories SRL processes into cognition, metacognition, and emotion [7,9]. Extensive empirical studies were conducted with this framework to collect learning trace data and validate the measurement of SRL processes [24,[42][43][44]. The trace parser consists of an action library -which converts raw trace data to learning actions -and a process library -which labels learning actions into SRL processes (a summary of the trace parser is presented in Figure 3; a complete table of action and process libraries can be found in the Appendix at this link. ...
A sophisticated grasp of self-regulated learning (SRL) skills has become essential for learners in computer-based learning environment (CBLE). One aspect of SRL is the plan-making process, which, although emphasized in many SRL theoretical frameworks, has attracted little research attention. Few studies have investigated the extent to which learners complied with their planned strategies, and whether making a strategic plan is associated with actual strategy use. Limited studies have examined the role of prior knowledge in predicting planned and actual strategy use. In this study, we developed a CBLE to collect trace data, which were analyzed to investigate learners' plan-making process and its association with planned and actual strategy use. Analysis of prior knowledge and trace data of 202 participants indicated that 1) learners tended to adopt strategies that significantly deviated from their planned strategies, 2) the level of prior knowledge was associated with planned strategies, and 3) neither the act of plan-making nor prior knowledge predicted actual strategy use. These insights bear implications for educators and educational technologists to recognise the dynamic nature of strategy adoption and to devise approaches that inspire students to continually revise and adjust their plans, thereby strengthening SRL.
... Much of the literature reviewing the role of metacognitive monitoring and advanced learning technologies have focused on intelligent tutoring systems, implying that metacognitive processes are often partially responsible for both short-term and long-term learning outcomes (Azevedo & Wiedbusch, 2023;Sonnenberg & Bannert, 2019). However, as Azevedo (2020) highlights in a review of the current issues, challenges, and opportunities for the study of metacognition, there is a paucity of research examining metacognition within VR or other immersive learning environments. ...
This study examines the embodied ways in which learners monitor their cognition while learning about exponential functions in an immersive virtual reality (VR) based game, Pandemic by Prisms of Reality. Traditionally, metacognitive monitoring has been assessed through behavioural traces and verbalised instances. When learning in VR, learners are fully immersed in the learning environment, actively manipulating it based on affordances designed to support learning, offering insights into the relationship between physical interaction and metacognition. The study collected multimodal data from 15 participants, including think‐aloud audio, bird's‐eye view video recordings and physiological data. Metacognitive monitoring was analysed through qualitative coding of the think‐aloud protocol, while movement was measured via optical flow analysis and cognitive load was assessed through heart rate variability analysis. The results revealed embodied metacognition by aligning the data to identify learners' physical states alongside their verbalised metacognition. The findings demonstrated a temporal interplay among cognitive load, metacognitive monitoring, and motion during VR‐based learning. Specifically, cognitive load, indicated by the low‐ and high‐frequency heart rate variability index, predicted instances of metacognitive monitoring, and monitoring predicted learners' motion while interacting with the VR environment. This study further provides future directions in understanding self‐regulated learning processes during VR learning by utilizing multimodal data to inform real‐time adaptive personalised support within these environments.
Practitioner notes
What is already known about this topic Immersive virtual reality (VR) environments have the potential to offer personalised support based on users' individual needs and characteristics.
Self‐regulated learning (SRL) involves learners monitoring their progress and strategically regulating their learning when needed.
Multimodal data captured during VR learning, such as birds‐eye‐view video, screen recordings, physiological changes and verbalisations, can provide insights into learners' SRL processes and support needs.
What this paper adds Provides insights into the embodied aspects of learners' metacognitive monitoring during learning in an immersive VR environment.
Demonstrates how SRL processes can be captured via the collection and analysis of multimodal data, including think‐aloud audio, bird's‐eye view video recordings and physiological data, to capture metacognitive monitoring and movement during VR‐based learning.
Contributes to the understanding of the interplay between cognitive load, metacognitive monitoring, and motion in immersive VR learning.
Implications for practice and/or policy Researchers and practitioners can use the causal relationships identified in this study to identify instances of SRL in an immersive VR setting.
Educational technology developers can consider the integration of online measures, such as cognitive load and physiological arousal, into adaptive VR environments to enable real‐time personalised support for learners based on their self‐regulatory needs.
... Research suggests that SRL scaffolding is positively associated with improvements in academic performance and learning processes. Specifically, SRL scaffolding has been examined from various perspectives, such as the persistence of scaffolding effects Sonnenberg and Bannert, 2019), effectiveness of technological scaffolds (Milikić et al., 2018;Lahza et al., 2022), the utility of scaffold training (Bannert, 2009), impact on group activities and group performance (Molenaar et al., 2011), influence of demographic factors (Pieger and Bannert, 2018), and association with different goal orientations (Duffy and Azevedo, 2015). Moreover, the effects of scaffolding have been examined in diverse contexts, including educational settings (Azevedo et al., 2004;Bannert, 2009;Sonnenberg and Bannert, 2016) and workplaces (Siadaty et al., 2016a,b). ...
Self-regulated learning (SRL) is the ability to regulate cognitive, metacognitive, motivational, and emotional states while learning and is posited to be a strong predictor of academic success. It is therefore important to provide learners with effective instructions to promote more meaningful and effective SRL processes. One way to implement SRL instructions is through providing real-time SRL scaffolding while learners engage with a task. However, previous studies have tended to focus on fixed scaffolding rather than adaptive scaffolding that is tailored to student actions. Studies that have investigated adaptive scaffolding have not adequately distinguished between the effects of adaptive and fixed scaffolding compared to a control condition. Moreover, previous studies have tended to investigate the effects of scaffolding at the task level rather than shorter time segments—obscuring the impact of individual scaffolds on SRL processes. To address these gaps, we (a) collected trace data about student activities while working on a multi-source writing task and (b) analyzed these data using a cutting-edge learning analytic technique— ordered network analysis (ONA)—to model, visualize, and explain how learners' SRL processes changed in relation to the scaffolds. At the task level, our results suggest that learners who received adaptive scaffolding have significantly different patterns of SRL processes compared to the fixed scaffolding and control conditions. While not significantly different, our results at the task segment level suggest that adaptive scaffolding is associated with earlier engagement in SRL processes. At both the task level and task segment level, those who received adaptive scaffolding, compared to the other conditions, exhibited more task-guided learning processes such as referring to task instructions and rubrics in relation to their reading and writing. This study not only deepens our understanding of the effects of scaffolding at different levels of analysis but also demonstrates the use of a contemporary learning analytic technique for evaluating the effects of different kinds of scaffolding on learners' SRL processes.
... A successful LA Intervention design requires a sound pedagogical approach along with appropriate LA cycles to provide feedback for both cognitive and behavioral processes (Sedrakyan et al., 2020). In course instruction, intervention is known to be an ongoing process and should be developed iteratively in order to find the most effective way to enhance student's learning (Sonnenberg & Bannert, 2019). This paper firstly introduces a model of instructional intervention series, depicted in Fig. 2, based on the LA cycle model, and then proposes i-Ntervene which is an integrated platform that reinforces effective intervention cycles for instructor-led programming courses, as illustrated in Fig. 3. ...
Apart from good instructional design and delivery, effective intervention is another key to strengthen student academic performance. However, intervention has been recognized as a great challenge. Most instructors struggle to identify at-risk students, determine a proper intervention approach, trace and evaluate whether the intervention works. This process requires extensive effort and commitment, which is impractical especially for large classes with few instructors. This paper proposes a platform, namely i-Ntervene, that integrates Learning Management System (LMS) automatic code grader, and learning analytics features which can empower systematic learning intervention for large programming classes. The platform supports instructor-pace courses on both Virtual Learning Environment (VLE) and traditional classroom setting. The platform iteratively assesses student engagement levels through learning activity gaps. It also analyzes subject understanding from programming question practices to identify at-risk students and suggests aspects of intervention based on their lagging in these areas. Students’ post-intervention data are traced and evaluated quantitatively to determine effective intervention approaches. This evaluation method aligns with the evidence-based research design. The developed i-Ntervene prototype was tested on a Java programming course with 253 first-year university students during the Covid-19 pandemic in VLE. The result was satisfactory, as the instructors were able to perform and evaluate 12 interventions throughout a semester. For this experimental course, the platform revealed that the approach of sending extrinsic motivation emails had more impact in promoting learning behavior compared to other types of messages. It also showed that providing tutorial sessions was not an effective approach to improving students’ subject understanding in complex algorithmic topics. i-Ntervene allows instructors to flexibly trial potential interventions to discover the optimal approach for their course settings which should boost student’s learning outcomes in long term.
... Whether prompts lead to good learning outcomes in multiple similar learning situations is still under debate (long-term effects). Some studies have found that prompts can improve learning performance on topics from the domain of educational psychology after three weeks (Bannert et al. 2015;Christoph and Maria 2019), but these results have been challenged by other studies (Breitwieser et al. 2022;Engelmann et al. 2021). For example, Engelmann et al. (2021) did not observe a long-term effect for meta-cognitive prompts in a second learning session after three weeks. ...
The hypermedia environment is among the most prevalent contemporary self-regulated learning (SRL) environments; however, methods for improving the effectiveness of students’ multi-session SRL in such environments remain under discussion. In this study, two experiments were conducted to explore whether and how prompts and feedback benefit performance during multi-session SRL in a hypermedia learning environment. A total of 76 senior students participated in Experiment 1, which used a mixed 2 (prompting condition: prompt, no prompt) × 2 (feedback condition: feedback, no feedback) × 2 (learning session: Session 1 and Session 2) design to explore the effects of prompting and feedback on the multi-session learning process in a hypermedia environment. The results indicated that, in learning Session 1, performance in the prompt condition was significantly better than in the unprompted condition, with or without feedback; in learning Session 2, participants in the prompt condition with feedback performed significantly better than those in the other three conditions. Students in the group with a prompt and feedback had the most accurate meta-comprehension absolute accuracy in both learning sessions. Experiment 2 recruited 94 secondary school students to further explore whether the combination of prompts and different types of feedback led to different learning outcomes according to the division of feedback timing. A mixed 2 (prompt condition: prompt, no prompt) × 3 (feedback condition: delayed feedback, immediate feedback, no feedback) × 2 (learning session: Session 1 and Session 2) design was used. The results indicated that, in learning Session 1, the prompt condition outperformed the unprompted condition with or without feedback; in learning Session 2, students with prompted delayed feedback outperformed the other five conditions. We also found that although there was no significant difference in meta-comprehension monitoring accuracy between delayed and immediate feedback, both groups performed significantly better than those in the no feedback condition. These results suggest that the combination of prompts and feedback in hypermedia environments facilitates student performance better than prompts or feedback alone; this improvement may be related to the correction of poor internal student feedback.
... In general, most research using AIED systems has focused on cognitive strategies and implemented experimental designs that foster learning across domains and topics, measuring learning gains by using pretests-posttests based on experimental comparisons of different scaffolding methods, human and computerized tutor interventions, and so forth (e.g., Aleven & Koedinger, 2002;Chi et al., 2001;Taub & Azevedo, 2020). This research has implied metacognitive processes may have been wholly or partially responsible for both short-term and long-term learning gains (e.g., Sonnenberg & Bannert, 2019). In comparison, the majority of metacognition research has focused on self-report measures embedded in AIED systems and, in some cases, made inferences about metacognitive judgments, processes and evaluations without collecting process data while students used AIED systems. ...
... Based on this, recent studies have explored SRL behavior in the learning process. More and more studies regard SRL as a real-time process event in learning and problem solving [14]. This is partly due to the progress of the digital learning environment, which can record learners' behavior at the fine-grained level. ...
Self-regulated learning is one of the important skills to achieve learning goals and is also the key factor to ensure the quality of online learning. With the rapid development of intelligent education and information technology, online learning behavior has become a new trend in the development of education modernization. Behavior data of online learning platforms are an important carrier to reflect the learners’ initiative to plan, monitor, and regulate their learning process. Self-regulated learning (SRL) is one of the important skills to achieve learning goals and is an essential means to ensure the quality of online learning. However, there are still great challenges in studying the types and sequential patterns of learners’ self-regulated learning behaviors in online environments. In addition, for higher education, the defects of the traditional education mode are increasingly prominent, and self-regulated learning (SRL) has become an inevitable trend. Based on Zimmerman’s self-regulation theory model, this paper first classifies learning groups using the hierarchical clustering method. Then, lag sequence analysis is used to explore the most significant differences in SRL behavior and its sequence patterns among different learning groups. Finally, the differences in academic achievement among different groups are discussed. The results are as follows: (1) The group with more average behavior frequency tends to solve online tasks actively, presenting a “cognitive oriented” sequential pattern, and this group has the best performance; (2) the group with more active behavior frequency tends to improve in the process of trial and error, showing a “reflective oriented” sequence pattern, and this group has better performance; (3) the group with the lowest behavior frequency tends to passively complete the learning task, showing a “negative regulated” sequence pattern, and this group has poor performance. From the aspects of stage and outcome of self-regulated learning, the behavior sequence and learning performance of online learning behavior mode are compared, and the learning path and learning performance of different learning modes are fully analyzed, which can provide reference for the improvement of online learning platform and teachers’ teaching intervention.
... This has prompted recommendations that technology-based learning environments be designed-for example, with metacognitive tools-to support SRL (Sonnenberg & Bannert, 2015). Indeed, a growing literature spanning, education, psychology, and computer science has advocated for more process-oriented approaches to understanding self-regulatory behaviors (Cerezo et al., 2019;Sonnenberg & Bannert, 2019). ...
The present paper builds on the literature that emphasizes the importance of self-regulation for academic learning or self-regulated learning (SRL). SRL research has traditionally focused on count measures of SRL processing events, however, another important measure of SRL is being recognized: time-on-task. The current study captures the influence of time spent on learning performance. We study time-on-task—from the perspective of self-regulated learning—in the context of clinical reasoning in an intelligent tutoring system. Specifically, we examine the link between the time spent in the three phases of SRL (forethought, performance, and self-reflection) and confidence in diagnosis and diagnosis correctness. Our analyses revealed non-significant links between the time spent in the three phases of SRL and diagnosis correctness. On the other hand, significant associations were found between the time spent in the three phases of SRL and confidence in diagnosis (confidence in diagnosis was: positively associated with time spent in forethought phase; negatively associated with time spent in performance phase; and, positively associated with time spent in self-reflection phase). In addition, confidence in diagnosis was positively linked to diagnosis correctness. Considering learning time offers an alternative perspective on regulation of learning and problem-solving performance. We conclude by offering implications of our findings and recommendations for further research.
... The need to measure SRL as a process, via so-called online measures, has been expressed several times (Molenaar & Järvelä, 2014;Winne & Perry, 2000;Zimmerman, 2008). Some innovative instruments such as think-aloud protocols (Sonnenberg & Bannert, 2019), log files (Bernacki, 2018), data mining (Lajoie et al., 2021), or electrodermal activity (Malmberg et al., 2019) have been developed in recent years. For example, Molenaar et al. (2021) have depicted student learning progress through moment-bymoment learning curves, thus providing deeper insights into when students need additional learning support. ...
Self-regulated learning (SRL) provides the foundation for building sustainable knowledge and is therefore important for schools, classrooms, and lifelong learning in general. Especially in vocational education and training, the concept of SRL remains fundamental as it relates to preparing future employees. However, further research is needed on how vocational students situationally regulate their learning process and the extent to which this may be related to a dispositional change in their SRL. In this study, we analyzed longitudinal questionnaire data from 159 students who attended either SRL-conducive or regular vocational classes. We refer to Perry and colleagues' (2018) framework of an SRL-conducive learning environment, which focuses on (meta)cognitive, motivational, and emotional aspects of learning. Using multilevel analysis, we found differences in the development of (meta)cognitive components of learning, whereas no clear differences could be identified for motivational and emotional components. The results support the assumption that process analyses can be used to draw a more differentiated picture of SRL in vocational schools. Moreover, indirect approaches to promoting SRL should be designed to include all SRL-relevant aspects.
... An interesting avenue for further research is making the algorithm applicable for explanatory research and hypothesis testing. An initial promising direction is conformance checking; specifically, the opportunity to compare two models from different samples to verify similarities and deviations (see Van der Aalst et al., 2012, Sonnenberg & Bannert, 2018. ...
This paper presents a fine-grained process analysis of 22 students in a classroom-based learning setting. The students engaged (and failed) in problem-solving attempts prior to instruction (i.e., the Productive-Failure approach). We used the HeuristicsMiner algorithm to analyze the data of a quasi-experimental study. The applied algorithm allowed us to investigate temporally structured think-aloud data, to outline productive and unproductive problem-solving strategies. Our analyses and findings demonstrated that HeuristicsMiner enables researchers to effectively mine problem-solving processes and sequences, even for smaller sample sizes, which cannot be done with traditional code-and-count strategies. The limitations of the algorithm, as well as further implications for educational research and practice, are also discussed.
... Different designs of LA interventions have different impacts on learning. For example, the LA interventions by (Siadaty et al., 2016b) and (Sonnenberg & Bannert, 2019) were aimed at supporting the subprocess of task analysis in SRL. The former study found their LA intervention had a large impact on learning, whereas the latter found it had a medium impact. ...
During the past years scholars have shown an increasing interest in supporting students' self-regulated learning (SRL). Learning analytics (LA) can be applied in various ways to identify a learner’s current state of self-regulation and support SRL processes. It is important to examine how LA has been used to identify the need for support in different phases of SRL cycle, which channels are used to mediate the intervention and how efficient and impactful the intervention is. This will help the learners to achieve the anticipated learning outcomes. The systematic literature review followed PRISMA 2020 statement to examine studies that applied LA interventions to enhance SRL. The search terms used for this research identified 753 papers in May 2021. Of these, 56 studies included the elements of LA, SRL, and intervention. The reviewed studies contained various LA interventions aimed at supporting SRL, but only 46% of them revealed a positive impact of an intervention on learning. Furthermore, only four studies reported positive effects for SRL and covered all three SRL phases (planning, performance, and reflection). Based on the findings of this literature review, the key recommendation is for all phases of SRL to be considered when planning interventions to support learning. In addition, more comparative research on this topic is needed to identify the most effective interventions and to provide further evidence on the effectiveness of interventions supporting SRL.
... This paper presents an overview of over 10 years of research on SRL with MetaTutor, a hypermedia-based ITS designed to scaffold college students' SRL while they learn about the human circulatory system. MetaTutor's architecture and instructional features are designed based on Winne (2018; model of SRL, empirical evidence on human (Azevedo et al., 2008;Chi, 2021) and computerized tutoring (Nye et al., 2014;du Boulay and Luckin, 2016;Lester, 2016, 2018;Graesser, 2020), AI in educational systems for metacognition and SRL (Aleven and Koedinger, 2002;Azevedo and Aleven, 2013;Biswas et al., 2016;Azevedo and Wiedbusch, in press), Mayer and Fiorella (in press) principles of multimedia learning, and extensive research on SRL, ITSs, serious games, simulations, and open-ended hypermedia from our team and other researchers (Bannert et al., 2014;Biswas et al., 2018;Schunk and Greene, 2018;Sonnenberg and Bannert, 2019;Lajoie, 2021). ...
Self-regulated learning (SRL) is critical for learning across tasks, domains, and contexts. Despite its importance, research shows that not all learners are equally skilled at accurately and dynamically monitoring and regulating their self-regulatory processes. Therefore, learning technologies, such as intelligent tutoring systems (ITSs), have been designed to measure and foster SRL. This paper presents an overview of over 10 years of research on SRL with MetaTutor, a hypermedia-based ITS designed to scaffold college students’ SRL while they learn about the human circulatory system. MetaTutor’s architecture and instructional features are designed based on models of SRL, empirical evidence on human and computerized tutoring principles of multimedia learning, Artificial Intelligence (AI) in educational systems for metacognition and SRL, and research on SRL from our team and that of other researchers. We present MetaTutor followed by a synthesis of key research findings on the effectiveness of various versions of the system (e.g., adaptive scaffolding vs. no scaffolding of self-regulatory behavior) on learning outcomes. First, we focus on findings from self-reports, learning outcomes, and multimodal data (e.g., log files, eye tracking, facial expressions of emotion, screen recordings) and their contributions to our understanding of SRL with an ITS. Second, we elaborate on the role of embedded pedagogical agents (PAs) as external regulators designed to scaffold learners’ cognitive and metacognitive SRL strategy use. Third, we highlight and elaborate on the contributions of multimodal data in measuring and understanding the role of cognitive, affective, metacognitive, and motivational (CAMM) processes. Additionally, we unpack some of the challenges these data pose for designing real-time instructional interventions that scaffold SRL. Fourth, we present existing theoretical, methodological, and analytical challenges and briefly discuss lessons learned and open challenges.
... Model discovery and conformance checking were combined to analyze the temporal order of spontaneous individual regulation activities and how findings can be applied to identify process patterns in SRL events. Hypermedia learning sessions were used in [67], applying PM to detect long-term effects of metacognitive prompting on SRL in a follow-up learning task without instructional support. Combining the Interactive Digital Narratives (IDN) approach under Prolific, a scientific crowdsourcing platform and PM, Estupiñán and Szilas [68] examined the understanding of user engagement, particularly, spotting when and what happened when user engagement dropped. ...
Process Mining (PM) emerged from business process management but has recently been applied to educational data and has been found to facilitate the understanding of the educational process. Educational Process Mining (EPM) bridges the gap between process analysis and data analysis, based on the techniques of model discovery, conformance checking and extension of existing process models. We present a systematic review of the recent and current status of research in the EPM domain, focusing on application domains, techniques, tools and models, to highlight the use of EPM in comprehending and improving educational processes.
... The resulting process model is a generalized representation of the sequences that characterize the data (Reimann, Frerejean & Thompson, 2009). Process mining has been increasingly used in SRL research, as it allows researchers to investigate regulatory patterns (Bannert, Reimann & Sonnenberg, 2014;Paans et al., 2019;Reimann et al., 2009;Reimann et al., 2014;Schoor & Bannert, 2012;Sonnenberg & Bannert, 2018). To mine the video data, a log was created, which included the coded events with their timestamps. ...
This dissertation explores adaptive regulation in collaborative learning by using novel technologies for data collection and analysis. Adaptation is a key feature of regulated learning, because through adaptation learners change their ways of thinking and learning when faced with challenges. However, because of methodological limitations, studying adaptation has been challenging. This dissertation focuses on exploring the differences between high- and low-challenge sessions in terms of phases of regulation, how metacognitive monitoring triggers adaptation, and how sequences of adaptive regulation and maladaptive behavior emerge throughout a learning session.
To capture adaptation in group learning situations, two data sets were collected in authentic learning situations. The first data collection took place during a mathematics didactics course organized for teacher education students; the second data collection was conducted during an advanced physics course for high-school students. The data collected included log, video, and heart rate data. Process-oriented methods were used to combine qualitative analysis of the video data and group-level analysis of changes in heart rate values.
The results indicate that in high-challenge sessions learners return to planning throughout the session, which can be interpreted as a sign of adaptation. The results also show that monitoring acts as a trigger for adaptation and report how group-level small-scale adaptation can be evidenced by considering the phase, target, and valence of shared monitoring events. Physiological state transitions defined from the heart rate data have potential to reveal information about whether the group is on track in the learning process.
The findings provide insight into how adaptation happens in an authentic collaborative learning setting. Methodologically, the study provides an innovative solution for capturing adaptation using multimodal data and novel analytical methods. At the theoretical level, the dissertation contributes to the field with details of the relationship between metacognitive monitoring and small-scale adaptation. For pedagogical practice, this study signals a need for adaptive support during the learning process: collaborative groups take very different routes to success in terms of when and how they adapt their learning process.
... One of the common approaches often used in SRL measurement settings is concurrent think-aloud protocols [12]. A recent study by [40] integrated coded think-aloud and trace data and applied process mining to understand students' self-regulatory behaviors. A similar approach, such as collecting students' realtime self-reported behaviors in the LMS, can provide a detailed trace of the learning process. ...
Research has emphasized that self-regulated learning (SRL) is critically important for learning. However, students have different capabilities of regulating their learning processes and individual needs.
To help students improve their SRL capabilities, we need to identify
students’ current behaviors. Specifically, we applied instructional
design to create visible and meaningful markers of student learning
at different points in time in LMS logs. We adopted knowledge engineering to develop a framework of proximal indicators representing
SRL phases and evaluated them in a quasi-experiment in two different learning activities. A comparison of two sources of collected
students’ SRL data, self-reported and trace data, revealed a relatively high agreement between our classifications (weighted kappa,
𝜅 = .74 and 𝜅 = .68). However, our indicators did not always discriminate adjacent SRL phases, particularly for enactment and adapting
phases, compared with students’ real-time self-reported behaviors.
Our behavioral indicators also were comparably successful at classifying SRL phases for different self-regulatory engagement levels.
This study demonstrated how the triangulation of various sources
of students’ self-regulatory data could help to unravel the complex
nature of metacognitive processes.
... Future research should aim to use other observational measures in addition to frequency, including but not limited to duration of the online interaction and the time-stamped trace data of sequences of the online interaction (Mirriahi et al., 2018;Winne et al., 2017). Through applying advanced process-mining methods to analysing these types of digital trace data, the more dynamic nature of students' online learning can be revealed (Jovanović et al., 2017;Sonnenberg & Bannert, 2019). Notwithstanding these limitations, the results offer some valuable insights into why the quality of student experiences of learning in blended course designs vary, which may be useful in improving student learning. ...
This study combined the methods from student approaches to learning and learning analytics research by using both self-reported and observational measures to examine the student learning experience. It investigated the extent to which reported approaches and perceptions and observed online interactions are related to each other and how they contribute to variation in academic performance in a blended course design. Correlation analyses showed significant pairwise associations between approaches and frequency of the online interaction. A cluster analysis identified two groupings of students with different reported learning orientations. Based on the reported learning orientations, one-way ANOVAs showed that students with understanding orientation reported deep approaches to and positive perceptions of learning. The students with understanding orientation also interacted more frequently with the online learning tasks and had higher marks than those with reproducing orientation, who reported surface approaches and negative perceptions. Regression analyses found that adding the observational measures increased 36% of the variance in the academic performance in comparison with using self-reported measures alone (6%). The findings suggest using the combined methods to explain students’ academic performance in blended course designs not only triangulates the results but also strengthens the acuity of the analysis. © 2020. Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BYNC- ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. All Rights Reserved.
... The common learning indicators used in the MMLA literature are behavior, attention, cognition [89], metacognition [90], emotion [91], collaboration, interaction [47], engagement, and learning performance. Some of them can be further classified. ...
Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, this paper systematically surveys 346 articles on MMLA published during the past three years. For this purpose, we first present a conceptual model for reviewing these articles from three dimensions: data types, learning indicators, and data fusion. Based on this model, we then answer the following questions: 1. What types of data and learning indicators are used in MMLA, together with their relationships; and 2. What are the classifications of the data fusion methods in MMLA. Finally, we point out the key stages in data fusion and the future research direction in MMLA. Our main findings from this review are (a) The data in MMLA are classified into digital data, physical data, physiological data, psychometric data, and environment data; (b) The learning indicators are behavior, cognition, emotion, collaboration, and engagement; (c) The relationships between multimodal data and learning indicators are one-to-one, one-to-any, and many-to-one. The complex relationships between multimodal data and learning indicators are the key for data fusion; (d) The main data fusion methods in MMLA are many-to-one, many-to-many and multiple validations among multimodal data; and (e) Multimodal data fusion can be characterized by the multimodality of data, multi-dimension of indicators, and diversity of methods.
Understanding processes in self-regulated learning (SRL) and tailoring appropriate instructional support to help students become more productive self-regulated learners has been on the agenda of SRL researchers for decades. New data modalities and analytic methods are becoming increasingly available to augment existing methodologies, enhance SRL measurement, test theoretical assumptions about SRL and inform future instructional support. Though promising, this research direction is yet to be fully explored. To learn more about how multimodal learner data and analytic methods can be used to improve research and support for SRL, we invited for a conversation Professors Maria Bannert, Inge Molenaar, and Phil Winne, three prominent scholars who have been extensively researching SRL over the past few decades. The conversation included two parts (1) Studying SRL via Analytics and (2) Supporting SRL via Analytics. The discussion identified several major areas for future research, including integrating multiple data channels in a meaningful way to improve theoretical understanding of SRL, and supporting learners by offering them options on what to do next, rather than by saying that they missed an opportunity to engage in a particular SRL process. Following the polyphonic research methodology, the lead authors and the interviewed SRL scholars co-authored this chapter. A podcast of the conversation is available at https://spotifyanchor-web.app.link/e/NwvHdDh3MMb.
Researchers have indicated the importance of engaging learners in self-regulated learning (SRL) states when situated in game-based learning contexts; however, it remains a challenge for both educational and educational technology researchers to effectively integrate both. To this end, this study investigated how SRL strategies are interwoven with game-based learning in higher education by searching the web of science database to systematically review the papers published between 2009 and 2020 in academic journals. The encoded dimensions ranged from the primary research purpose to research issues, including application domains, research methods, duration of the studies, SRL strategies, game types, and game genres. It was found that since 2015, the research purposes have become increasingly diverse, with skills acquisition in game-based learning being regarded as the most important goal, followed by knowledge acquisition and behavior change. Such games took goal orientation, peer learning, and regulating as the main SRL strategies, which exerted a positive effect on learning performance, self-efficacy/confidence, attitudes/effort, satisfaction/interest, and learning behavior. Meanwhile, these SRL strategies were well embedded into problem-solving, simulation, multi-type, and RPG game types against the setting of the real-life-related storyline as the main game genre. Since previous studies lacked the systematic application of all SRL strategies within a game-based learning environment, they could not uncover the dynamic and cyclic processes of SRL in game-based learning environments. Hence, this study proposed corresponding suggestions for future research issues as a reference for researchers, teachers, and decision-makers.
The study investigates the effects of using immediate feedback as a teaching strategy. Additionally, the impacts of immediate feedback and gender on students' classroom learning outcomes were examined. The study was designed as a quasi-experimental, pretest-posttest, experimental, and control group. The sample consisted of 225 junior secondary school level 2 (JSS 2) students. Students were chosen from two intact coeducational classes and split into experimental and control groups. All relevant data was collected using a study tool called the Science, Technology, English Language & Mathematics Achievement Test-Questionnaire (STEMAT-Q), which was developed, validated, and used. Data collected were analysed using the mean, standard deviation, Student's t-test, and analysis of covariance (ANCOVA). The study's findings demonstrate that an immediate feedback technique significantly affects the learning outcomes of students. However, for treated male and female students, the interaction effects of the immediate feedback technique and gender on classroom learning outcomes were not significant. Immediate feedback is particularly successful at addressing student confusion, correcting errors, identifying learning gaps, bridging gender differences in student learning outcomes, and inspiring students to learn well. Based on the above findings, the researcher recommends the provision of immediate feedback for students during the learning process or class discussion or activities to enhance their learning skills and help them retain key concepts, ideas, and principles.
Individuals who engage in self-regulated learning (SRL) tend to perform better in complex learning tasks. However, learners’ ability to self-regulate can vary. To understand and support learners’ SRL, collecting information about their engagement in specific learning processes in the context of learning tasks is necessary. However, SRL is sufficiently complex that it is not directly observable. Capturing the SRL processes that occur during learning, as students interact with elements of tasks hosted on virtual learning technologies (e.g., learning management systems; LMS), is possible because learners’ actions generate observable events that these technologies log. However, discerning how these events reflect SRL processes poses several major theoretical, methodical, and analytical challenges. To address these challenges, we present two projects to illustrate how researchers validated inferences about SRL processes. We demonstrate how observational indicators drawn from multiple channels of event data must be (a) collected from the technologies’ log files and the record of learners’ self-reports of their learning process, (b) instrumented to describe learner, event, and context, and (c) integrated and temporally aligned. Afterward, we how researchers can hypothesize about the SRL processes digital events reflect and test inferences using secondary channels of explanatory data provided by learners during the tasks.KeywordsSelf-regulated learningMultimodal dataMeasurement validation
Casadevante et al. (Curr Psychol 42: 4272–4285, 2023) used an objective test and found that regulation of response speed was related to better performance in a category learning task. The present study aims at analysing whether the relation between regulation of response speed and learning exists in an associative learning task. We developed ad hoc the Treasure Forest, an objective test consisting of a computerized associative learning task. We conducted a first study with 86 university students to assess the relation between spontaneous response speed and learning. Results showed that participants who acted slowly learned more than their mates who acted faster (t (83) = 8.898, p < .001, η2 = .672). Moreover, some students who began the task acting too fast to learn decreased their response speed by the second half of the task and simultaneously their learning index improved (t (11) = 2.325, p < .05, d = .721). Hence, self-regulating the response speed was linked to associative learning. We conducted a second study to analyse the influence of an external speed regulation on learning. The intervention group (N = 99) was prevented from clicking more than one click per second while the control group (N = 85) acted without restrictions. The intervention group achieved a higher learning index than the control group, who acted faster (t (160) = 4.828, p < .001, η2 =.117). Hence, regulating response speed promoted associative learning. We concluded that regulating response speed promoted associative learning, and we hypothesized that training self-regulation of response speed may improve learning and academic performance. Besides, we highlight the utility of employing objective test for analysing self-regulation.
The empirical study investigates what log files and process mining can contribute to promoting successful learning. We want to show how monitoring and evaluation of learning processes can be implemented in the educational life by analyzing log files and navigation behavior. Thus, we questioned to what extent log file analyses and process mining can predict learning outcomes. This work aims to provide support for learners and instructors regarding efficient learning with computer-based learning environments (CBLEs). We evaluated log file and questionnaire data from students (N = 58) who used a CBLE for two weeks. Results show a significant learning increase after studying with the CBLE with a very high effect size (p < .001, g = 1.71). A cluster analysis revealed two groups with significantly different learning outcomes accompanied by different navigation patterns. The time spent on learning-relevant pages and the interactivity with a CBLE are meaningful indicators for Recall and Transfer performance. Our results show that navigation behaviors indicate both beneficial and detrimental learning processes. Moreover, we could demonstrate that navigation behaviors impact the learning outcome. We present an easy-to-use approach for learners as well as instructors to promote successful learning by tracking the duration spent in a CBLE and the interactivity.
We have been working to understand when, how, and what makes regulation in collaborative learning functional aiming to understand the process of collaboration so that we could better inform learners and teachers in practice. Multimodal learning data collection and learning process analytics have guided our work. Seamless and accurate integration of multimodal learning data for measuring regulation in collaborative learning can be seen as a future direction of multimodal learning analytics (MMLA). Collaborative groups can be considered complex systems. The cognitive, emotional, motivational, and behavioural states of the group and its’ members are related to each other and in constant flux. Therefore, research has shown that regulation is a crucial process for making the maladaptive process of collaborative learning more adaptive. Regulation in collaborative learning involves groups taking metacognitive control of the task together through negotiated, iterative fine-tuning of internal and external conditions as needed. Metacognitive monitoring is always an internal mental process, but it can be externalized via visible interactions with other group members in collaborative situations. When aiming to support the adaptive collaborative learning process, we highlight two aspects: metacognitive awareness and participation in cognitive and socio-emotional interaction. In this chapter we present our recent empirical progress in these two aspects and discuss how learning process data and MMLA can be used to unravel regulation in collaborative learning and practical implications for collaborative learning.KeywordsSelf-regulated learningSSRLMetacognitionCollaborative learningMultimodal methods
Self-regulated learning theory is central to computer supported collaborative learning (CSCL) and depends on learner autonomy to create socially shared learning, and yet function within the restraints and goals of a specific class pedagogy. By integrating the rich theoretical CSCL literature with an inductively derived theory of role-playing game practice, we develop an insightful foundation for designing, implementing, and measuring the effectiveness of low-cost scripts. This takes the form of a prompt (mere exposure prompt) that nudges learners toward a pedagogical goal while maintaining freedom of learner creativity and minimizing instructor intrusion. We assert learner engagement can be viewed through the lens of role-playing’s emphasis on aligning players’ creative agendas with game design to create a shared imagined space. Through behavioral trace data and social network analysis, we measure behaviors that differ between test/control groups, receiving the prompt, and comparing a fully online versus blended course delivery over a semester of group-based simulated business negations following role-playing game design principles. Fully online test group members accurately recall the prompt’s messages while exhibiting behaviors congruent with the pedagogical script. Learners in the blended mode recall the prompt, but their behavior is unchanged. This suggests socially shared regulation of learning in the classroom context conforms to established classroom norms, overlooking the script prompt. Learners in the fully online mode, in contrast, initiate fewer social interactions, but search out opportunities across many players, thereby demonstrating the effect of the script prompt message.
Analyzing the event logs extracted from the process-aware information systems provide critical insights into improving the organizational processes. This case study reports the essential findings and lessons from a process mining project run in analyzing the postdoctoral research grant process in Iran's National Elites Foundation (INEF). Different deductions are reached by exploring the process participants' activities in the INEF web portal containing (i) the organizational inefficiencies exposed through the process mining techniques, where the most time-consuming activities are detected and suggested to the domain experts, and (ii) the decision tree technique applied in determining how the successful applicants are scored. The extracted rules indicate an 18% application admission with a final score of more than 403. This article is contributed to interpreting the behavioral patterns in INEF and determining who among the applicants has a higher chance of receiving the grant, supporting the policymakers and managers to assign rational budgeting and adopting human resource strategies.
In recent years, unobtrusive measures of self-regulated learning (SRL) processes based on log data recorded by digital learning environments have attracted increasing attention. However, researchers have also recognised that simple navigational log data or time spent on pages are often not fine-grained enough to study complex SRL processes. Recent advances in data-capturing technologies enabled researchers to go beyond simple navigational logs to measure SRL processes with multi-channel data. What multi-channel data can reveal about SRL processes, and to what extent can the addition of peripheral and eye-tracking data with navigational log data change and improve the measurement of SRL are key questions that require further investigation. Hence, we conducted a study and collected learning trace data generated by 25 university students in a laboratory setting, that aimed to address this problem by enhancing navigational log data with peripheral and eye-tracking data. We developed a trace-based measurement protocol of SRL, which interpreted raw trace data from multi-channel data into SRL processes. Specifically, the study compared the frequency and duration of SRL processes detected, how much duration and times of occurrences of the detected SRL processes were affected or refined. We also used a process mining technique to analyses how temporal sequencing of the detected SRL processes changed by enriching navigational log data with peripheral and eye-tracking data. The results revealed that by adding new data channels, we improved the capture of learning actions and detected SRL processes while enhancing the granularity of the measurement. In comparison to the use of navigational logs only, the completeness of temporal sequencing relationships between SRL processes with multi-channel data improved. In addition, we concluded that eye-tracking data is valuable for measuring and extracting SRL processes, and it should receive more attention in the future.
Background
Learners in Massive Open Online Courses (MOOCs) are presented with great autonomy over their learning process. Learners must engage in self‐regulated learning (SRL) to handle this autonomy. It is assumed that learners' SRL, through monitoring and control, influences learners' behaviour within the MOOC environment (e.g., watching videos). The exact relationship between SRL and learner behaviour has however not been investigated.
Objectives
We explored whether differences in SRL are related to differences in learner behaviour in a MOOC. As insight in this relationship could improve our understanding of the influence of SRL on behaviour, could help explain the variety in online learner behaviour, and could be useful for the development of successful SRL support for learners.
Methods
MOOC learners were grouped based on their self‐reported SRL. Next, we used process mining to create process models of learners' activities. These process models were compared between groups of learners.
Results and conclusions
Four clusters emerged: average regulators, help seekers, self‐regulators, and weak regulators. Learners in all clusters closely followed the designed course structure. However, the process models also showed differences which could be linked to differences in the SRL scores between clusters.
Takeaways
The study shows that SRL may explain part of the variability in online learner behaviour. Implications for the design of SRL interventions include the necessity to integrate support for weak regulators in the course structure.
Success in online and blended courses requires engaging in self-regulated learning (SRL), especially for challenging STEM disciplines, such as physics. This involves students planning how they will navigate course assignments and activities, setting goals for completion, monitoring their progress and content understanding, and reflecting on how they completed each assignment. Based on Winne & Hadwin’s COPES model, SRL is a series of events that temporally unfold during learning, impacted by changing internal and external factors, such as goal orientation and content difficulty. Thus, as goal orientation and content difficulty change throughout a course, so might students’ use of SRL processes. This paper studies how students’ SRL behavior and achievement goal orientation change over time in a large (N = 250) college introductory level physics course taught online. Students’ achievement goal orientation was measured by repeated administration of the achievement goals questionnaire-revised (AGQ-R). Students’ SRL behavior was measured by analyzing their clickstream event traces interacting with online learning modules via a combination of trace clustering and process mining. Event traces were first divided into groups similar in nature using agglomerative clustering, with similarity between traces determined based on a set of derived characteristics most reflective of students’ SRL processes. We then generated causal nets for each cluster of traces via process mining and interpreted the underlying behavior and strategy of each causal net according to the COPES SRL framework. We then measured the frequency at which students adopted each causal net and assessed whether the adoption of different causal nets was associated with responses to the AGQ-R. By repeating the analysis for three sets of online learning modules assigned at the beginning, middle, and end of the semester, we examined how the frequency of each causal net changed over time, and how the change correlated with changes to the AGQ-R responses. Results have implications for measuring the temporal nature of SRL during online learning, as well as the factors impacting the use of SRL processes in an online physics course. Results also provide guidance for developing online instructional materials that foster effective SRL for students with different motivational profiles.
In online learning, students’ learning behavior might change as the course progresses. How students adjust learning behaviors aligned with course requirements reflects their self-regulated learning strategies. Analyzing students’ learning patterns can help instructors understand how the course design or activities shape students’ learning behaviors, including their learning beliefs and motivation, and facilitate teaching decision makings accordingly. This study aims to propose a scientific analytic method to understand students’ self-regulated learning (SRL) patterns. The whole process includes the following four steps: (1) encoding behavioral patterns; (2) detecting turning points and chunking behavioral patterns; (3) grouping similar patterns; and (4) interpreting results. A case study with 4604 K-12 students from 476 courses was conducted to validate the proposed method. Five successful patterns, three at-risk patterns, and three average patterns were identified. The case study indicated that successful students showed at least one of the following characteristics: (1) Balanced, (2) Proactive and Balanced, and (3) Balanced with one highly engaged behavior. The at-risk students showed the following characteristics: (1) Oscillatory and (2) Low Engaged. Patterns which led to successful or at-risk conditions are compared and connected with corresponding SRL strategies. Practical and research implications are discussed in the article as well.
Guided inquiry learning is an effective method for learning about scientific concepts.
The present study investigated the effects of combining video modeling (VM)
examples and metacognitive prompts on university students’ (N = 127) scientific
reasoning and self-regulation during inquiry learning. We compared the effects of
watching VM examples combined with prompts (VMP) to watching VM examples
only, and to unguided inquiry (control) in a training and a transfer task. Dependent
variables were scientific reasoning ability, hypothesis and argumentation quality,
and scientific reasoning and self-regulation processes. Participants in the VMP and
VM conditions had higher hypothesis and argumentation quality in the training task
and higher hypothesis quality in the transfer task compared to the control group.
There was no added benefit of the prompts. Screen captures and think aloud protocols
during the two tasks served to obtain insights into students’ scientific reasoning
and self-regulation processes. Epistemic network analysis (ENA) and process mining
were used to model the co-occurrence and sequences of these processes. The
ENA identified stronger co-occurrences between scientific reasoning and self-regulation
processes in the two VM conditions compared to the control condition. Process
mining revealed that in the VM conditions these processes occurred in unique
sequences and that self-regulation processes had many self-loops. Our findings show
that video modeling examples are a promising instructional method for supporting
inquiry learning on both the process and the learning outcomes level.
Objetivo: Comparar o efeito de dois métodos de ensino (prompts instrucionais e estudo autodirigido) na competência diagnóstica de arritmias supraventriculares em estudantes de medicina. Métodos: Estudo experimental com três fases: avaliação inicial (fase 1), treinamento (fase 2) e avaliação tardia (fase 3). A amostra foi composta por 24 estudantes, que na fase 1 forneceram os diagnósticos dos ritmos cardíacos de 11 traçados de Eletrocardiograma (ECG). Na fase 2, a amostra foi dividida em dois grupos. O Grupo 1 adotou o estudo autodirigido, e o Grupo 2 prompts instrucionais. Na fase 3 (após uma semana), toda a amostra forneceu os diagnósticos dos ritmos cardíacos de 11 novos traçados de ECG. Resultados: Após a fase 2, os alunos melhoraram os escores diagnósticos em relação à fase 1, independentemente o método de ensino utilizado. Na fase 3, a média das notas do Grupo 2 superou o Grupo 1 (p = 0,034). Conclusão: O uso de prompts instrucionais no ensino de ritmos cardíacos, foi capaz de melhorar a acurácia diagnóstica dos estudantes para as arritmias estudadas.
Blended course designs have been increasingly adopted in medical education. However, research on the relations between the key aspects of students' learning experience and their learning outcomes often only measure students' self-reported experience, neglecting what they actually do in learning. This study combined both self-reported and observational measures of students' learning experiences and examined the relations between the two sets of measures and their contributions to learning outcomes. Australian medical students were asked to report their approaches to, and perceptions of, learning. Their frequency and duration of their interactions with both online formative and summative tasks were observed and recorded. Correlation analyses showed that the learning outcomes was positively related to deep approaches to using online technologies and duration of interactions with online summative tasks. The hierarchical multiple regression analysis found that the self-reported approaches and duration of interactions jointed explained the learning outcomes, accounting for 6% of the variance. The study demonstrated the complementary nature of using both self-reported and observational measures of students' learning experiences to explain the learning outcomes in blended course designs.
Process Mining is a new kind of Business Analytics and has emerged as a powerful family of Process Science techniques for analysing and improving business processes. Although Process Mining has managerial benefits, such as better decision making, the scientific literature has investigated it mainly from a computer science standpoint and appears to have overlooked various possible applications. We reviewed management-orientated literature on Process Mining and Business Management to assess the state of the art and to pave the way for further research. We built a seven-dimension framework to develop and guide the review. We selected and analysed 145 papers and identified eleven research gaps sorted into four categories. Our findings were formalised in a structured research agenda suggesting twenty-five research questions. We believe that these questions may stimulate the application of Process Mining in promising, albeit little explored, business contexts and in mostly unaddressed managerial areas.
This study applies a time-driven approach to model self-regulated learning (SRL) on the basis of elapsed time metrics in the context of open-ended learning environments (OELEs), specifically, network-based tutors. In doing so, we examine how students allocated attentional resources to distinct phases of SRL as a measure of depth of information processing. Student teachers (N=68) were assigned to two different versions of nBrowser: a static version where the network did not converge on the basis of student interactions and a dynamic version where the network was continually updated by the system. Students designed a lesson plan and completed pre- and post-test self-report measures of knowledge gains. In both the experimental conditions, the results show four distinct SRL profiles that are relatively consistent and can be detected on the basis of behavioral patterns logged by the system across behaviors, namely, planning, requesting hints, studying examples, and monitoring. Although students who allocated more attentional resources to studying examples performed more poorly, their efforts to engage in planning, requesting hints, and monitoring were found to predict knowledge gains and design skills. Furthermore, students assigned to the dynamic version of the system outperformed those assigned to the static version in pedagogical knowledge gains.
Self-regulated learning (SRL) has a predictable and instrumental effect on learning complicated knowledge. This study investigates the role of SRL in acquiring technological pedagogical content knowledge (TPACK), an important aspect of teachers’ effective technology use. The present study identified several regulatory procedural patterns used by teachers in the context of their TPACK achievements. A computer-based context, nBrowser, was used to facilitate teachers lesson planning around technology usage. Teachers log file data were analysed using process mining approaches. Findings indicate that high TPACK performers are more likely to perform self-regulative activities (e.g., monitoring) in developing TPACK compared to the low performers. Higher TPACK performers are more goal-oriented, demonstrate more monitoring and are more iterative in using all SRL processes in contrast to low performers who only partially regulate their problem solving. Such findings support previous research. This study adopts a novel approach for understanding the relations between SRL and TPACK. It offers opportunities to examine how teachers enact SRL as they move from the beginning to later stages of designing lessons and provides insights to researchers who study SRL in TPACK domains. Furthermore, the findings can assist educational designers in developing interventions for promoting TPACK development by concentrating on teachers’ SRL abilities.
Self-regulated learning (SRL) theorists propose that learners' motivations and cognitive and metacognitive processes interact dynamically during learning, yet researchers typically measure motivational constructs as stable factors. In this study, self-efficacy was assessed frequently to observe its variability during learning and how learners' efficacy related to their problem-solving performance and behavior. Students responded to self-efficacy prompts after every fourth problem of an algebra unit completed in an intelligent tutoring system. The software logged students' problem-solving behaviors and performance. The results of stability and change, path, and correlational analyses indicate that learners' feelings of efficacy varied reliably over the learning task. Their prior performance (i.e., accuracy) predicted subsequent self-efficacy judgments, but this relationship diminished over time as judgments were decreasingly informed by accuracy and increasingly informed by fluency. Controlling for prior achievement and self-efficacy, increases in efficacy during one problem-solving period predicted help-seeking behavior, performance, and learning in the next period. Findings suggest that self-efficacy varies during learning, that students consider multiple aspects of performance to inform their efficacy judgments, and that changes in efficacy influence self-regulated learning processes and outcomes.
We discuss the articles of this special issue with reference to an important yet previously only implicit dimension of study quality: alignment across the theoretical and methodological decisions that collectively define an approach to self-regulated learning. Integrating and extending work by leaders in the field, we propose a framework for evaluating alignment in the way self-regulated learning research is both conducted and reported. Within this framework, the special issue articles provide a springboard for discussing methodological promises and pitfalls of increasingly sophisticated research on the dynamic, contingent, and contextualized features of self-regulated learning.
IntroductionThe conceptualization of self-regulated learning (SRL) has been profoundly articulated in recent special issues focused on the similarities and distinctions between SRL and related phenomena like self-regulation, metacognition, and motivation as well as the development of new environments and methods that improve SRL research (Alexander 2008; Azevedo 2005a, 2005b, 2007; Azevedo and Hadwin 2005; Greene and Azevedo 2010; Gress and Hadwin 2010; Järvelä and Hadwin 2013; Molenaar and Järvelä 2014; Perry 2002; Winne and Baker 2013). These special issues have examined how the affordances of (usually computer-based) learning environments can provide the means to observe phenomena described in SRL models and to empirically test theoretical assumptions (Azevedo 2005b; Azevedo and Hadwin 2005; Efklides 2011; Winne and Hadwin 2008; Winne 2010, 2011; Zimmerman and Schunk 2008, 2011). In this special issue, we tighten the focus of this ongoing discussion by focusing on three key features ...
While the effect of scaffolding on learning has received much attention, less is known about its effect on students’ strategy use, especially in transfer activities. This study focuses on students’ adaptive behaviours as a function of given scaffolding and when transitioning from a scaffolded to an unstructured activity. We study this in the context of a complex physics simulation in which students choose between 124 different actions. We evaluate (i) how the scaffolding affects students’ building and testing behaviours, (ii) whether these behaviours transfer to an unstructured activity, and (iii) the relationship between the adapted behaviours and learning. A repeated-measures MANOVA suggests that students adapt their learning behaviours according to the demands and affordances of the task and the environment, and that these strategies transfer from a scaffolded to an unstructured activity. No significant relationships were found between these patterns and learning.
This meta-analysis examined research on the effects of self-regulated learning scaffolds on academic performance in computer-based learning environments from 2004 to 2015. A total of 29 articles met inclusion criteria and were included in the final analysis with a total sample size of 2648 students. Moderator analyses were performed using a random effects model that focused on the three main areas of scaffold characteristics (including the mechanism, functions, delivery forms, mode, and number of scaffolds; how to promote self-regulated learning by scaffolds); demographics of the selected studies (including sample groups, sample size, learning domain, research settings, and types of computer-based learning environments); and research methodological features (including research methods, types of research design, types of organization for treatment, and duration of treatment). Findings revealed that self-regulated learning scaffolds in computer-based learning environments generally produced a significantly positive effect on academic performance (ES = 0.438). It is also suggested that both domain-general and domain-specific scaffolds can support the entire process of self-regulated learning since they demonstrated substantial effects on academic performance. Different impacts of various studies and their methodological features are presented and discussed.
New methods for gathering and analyzing data about events that comprise self-regulated learning (SRL) support discoveries about patterns among events and tests of hypotheses about roles patterns play in learning. Five such methodologies are discussed in the context of four key questions that shape investigations into patterns in SRL. A framework for this review is provided by a model that structures SRL in terms of: conditions of a task, operations, products generated by operations, evaluations of work and standards used in evaluations (COPES; Winne in Journal of Educational Psychology, 89, 397–410, 1997). Four recommendations are made for future work on SRL as patterned activity: prune models of SRL with experimental tests, explicitly include goals in data, ensure learners have options for SRL by training them in tactics and strategies, and provide learners access to accurate displays about the events and patterns that comprise SRL.
Referring to current research on self-regulated learning, we analyse individual regulation in terms of a set of specific sequences of regulatory activities. Successful students perform regulatory activities such as analysing, planning, monitoring and evaluating cognitive and motivational aspects during learning not only with a higher frequency than less successful learners, but also in a different order—or so we hypothesize. Whereas most research has concentrated on frequency analysis, so far, little is known about how students’ regulatory activities unfold over time. Thus, the aim of our approach is to also analyse the temporal order of spontaneous individual regulation activities. In this paper, we demonstrate how various methods developed in process mining research can be applied to identify process patterns in self-regulated learning events as captured in verbal protocols. We also show how theoretical SRL process models can be tested with process mining methods. Thinking aloud data from a study with 38 participants learning in a self-regulated manner from a hypermedia are used to illustrate the methodological points.