ArticlePDF Available

Improving Measurements of Self-Regulated Learning

DOI: 10.1080/00461520.2010.517150
Authors:
Philip H Winne at Simon Fraser University
Philip H Winne
Download full-text PDFDownload full-text PDF
Read full-text
Download citation

Abstract and Figures

Articles in this special issue present recent advances in using state-of-the-art software systems that gather data with which to examine and measure features of learning and particularly self-regulated learning (SRL). Despite important advances, there remain challenges. I examine key features of SRL and how they are measured using common tools. I advance the case that traces of cognition and metacognition offer critical information about SRL that other state-of-the-art measurements cannot.
Figures - uploaded by Philip H Winne
Author content
All figure content in this area was uploaded by Philip H Winne
Content may be subject to copyright.
Winne Improving Measurements of Self-Regulated Learning 20
10.pdf
Content uploaded by Philip H Winne
Author content
All content in this area was uploaded by Philip H Winne
Content may be subject to copyright.
Winne Improving Measurements of Self-Regulated Learning 20
10.pdf
Content uploaded by Philip H Winne
Author content
All content in this area was uploaded by Philip H Winne
Content may be subject to copyright.
A preview of the PDF is not available
... Despite the key role of achievement goals in guiding learners' plans and actions, few studies have evaluated the limits of selfreported survey responses as representations of goals within dynamic learning contexts. Many studies have criticized limits of self-reported survey responses in capturing dynamic constructs relevant to SRL [2,18,52,54,61,63]. The field of learning analytics is also increasingly aligning behavioral trace data with theoretical constructs as a complement to surveys. ...
... Another inherent drawback of surveys is that they require learners' careful attention to and monitoring of recalled information to generate accurate descriptions of their goals or motivation. Winne [54] pointed out that researchers do not know how learners selectively sample experience forming a basis for what they report in surveys. Survey respondents apply a somewhat mysterious computational process to integrate multiple recalled experiences into one answer to a survey item. ...
... Zhou and Winne [61] also did not probe the causes of discrepancies between survey responses and trace data. The stronger correlation between trace data and posttest achievement, as theory predicts [54,56], does not guarantee that trace data can be validly interpreted as measuring motivation. Learning analytics researchers should be cautious in advancing interpretations and forming analytics relating to learners' achievement goals. ...
View
... self-report surveys to identify students with low SRL skills (Broadbent & Poon, 2015;Winne, 2010), while acknowledging limitations such as inaccuracy, subjectivity (Brown, 2017;Miller, 2016) and bias (Ganda & Boruchovitch, 2018). ...
... Click-stream data provides detailed, frequent, and unobtrusive records of users' click behavior in online environments such as logging into the learning platform, watching video lectures, browsing course materials, examining resources, and submitting assignments. Researchers can analyze these behaviors from the perspective of individual cognition, i.e., making sense of the learning experience and metacognitive acts, i.e., an awareness and understanding of one's own thought processes and therefore provide promising opportunities for tracing and measuring self-regulated learning (Winne, 2010). ...
... By linking trace and automated log data of participant activity, Winne and Hadwin first advocated the use of clickstream data to assess how self-regulated learning scaffolding impacts performance in computer-supported learning environments (Winne et al., 2010;Winne & Hadwin, 2013). Trace data from earlier studies (Winne, 1982;Winne & Perry, 2000) reveal insights for studying both temporal and sequential analysis of self-regulated learning (Malmberg et al., 2013;Panadero et al., 2015b;Winne et al. 2011). ...
The Effect of Self-Regulated Learning in Online Professional Training
Article
Full-text available
  • Jan 2023
With the rapid expansion of mobile, blended, and seamless learning, researchers claim two factors, lack of self-discipline and poor time management, adversely impact learning performance. In online educational environments, reduced social interactions and low engagement levels generate high dropout rates. Self-regulated learning (SRL), the individual ability to check progress toward a goal and manage learning behavior, appears critical to adult online learning success. Clickstream data can observe, record, and evaluate patterns of users' real-time learning behavior in an online learning environment. Linking clickstream data with performance outcomes allows researchers to assess online learning behaviors and academic performance. The guiding research question was: Are students who apply SLR strategies more likely to demonstrate mastery of knowledge and skills in a self-directed e-learning context? Clickstream data and performance measures were analyzed to explore whether task and cognitive conditions influence how SLR strategies are applied in online training.
View
... The articles written by Panadero and Kizilcec (2017) ranked second and third in terms of the intensity of outbursts. They mostly began the study of the interaction between Motivation, Metacognition, Emotion, and other factors, indicating that the research on self-regulated learning has begun to mature and deepen in recent years, and they began to [74][75][76][77][78]. Their research on self-regulated learning showed diversification in terms of theoretical perspectives and applications to teaching practice, which improved the theoretical model of self-regulated learning, indicating that the research of scholars in this period began to turn to applications in teaching practice and theoretical knowledge construction. ...
... They mostly began the study of the interaction between Motivation, Metacognition, Emotion, and other factors, indicating that the research on self-regulated learning has begun to mature and deepen in recent years, and they began to pay attention the improvement of learning in technological environments. [74][75][76][77][78]. Their research on self-regulated learning showed diversification in terms of theoretical perspectives and applications to teaching practice, which improved the theoretical model of self-regulated learning, indicating that the research of scholars in this period began to turn to applications in teaching practice and theoretical knowledge construction. ...
Visualization Analysis of Global Self-Regulated Learning Status, Hotspots, and Future Trends Based on Knowledge Graph
Article
Full-text available
  • Feb 2023
Self-regulated learning (SRL) has been an important topic in the field of global educational psychology research since the last century, and its emergence is related to researchers’ reflections on several educational reforms. To better study the research history and developmental trend of SRL, in this work, the Web of Science core collection database was used as a sample source, “self-regulated learning” was searched as the theme, and 1218 SSCI documents were collected from 30 September 1986, to 2022. We used CiteSpace software to visualize and analyze the number of publications, countries, institutions, researchers, keywords, highly cited literature, authors’ co-citations, keyword clustering, and timeline in the field of self-regulated learning research, and to draw related maps. It was found that the articles related to self-regulated learning were first published in the American Journal of Educational Research in 1986, and that self-regulated learning-related research has received increasing attention in recent decades, wherein research on self-regulated learning is roughly divided into three periods: the budding period from 1986 to 2002, the flat development period from 2003 to 2009, and the rapid development period from 2010 to 2022. The number of papers published in the United States, China, Australia, and Germany is relatively high, and the number of papers published in Spain is low compared with that in the United States. During this period, the University of North Carolina in the United States and McGill University in Canada were the institutions with the most publications; Azevedo Roger and Lajoie Susanne P were the most-published scholars in the field of self-regulated learning research; the journal publication with the highest impact factor was Computers Education; and the primary research interests in self-regulated learning mainly focused on Performance, Strategy, Students, Achievement, Motivation, and Metacognition. Furthermore, the most-cited study related to SRL research was Formative assessment and self-regulated learning: a model and seven principles of good feedback practice.
View
... The passage of time concerns when, how often, or how long learning events of interest occur. A limitation of this temporal property is that it omits events before and after the events of interest, i.e., the contextual information (Winne, 2010). By contrast, the order in time addresses this limitation by focusing on the relative arrangement of learning events, for example, a sequence of events indicating that a learner reads relevant material after viewing the results of a quiz. ...
... For example, SRL events may be operationalized at three levels (Winne, 2010): the occurrence level, the contingency level, and the patterned contingency level. In this operationalization, the occurrence level considers the features of individual events, e.g., the frequency of taking a quiz. ...
Sequential pattern mining in educational data: The application context, potential, strengths, and limitations
Preprint
Full-text available
  • Feb 2023
Increasingly, researchers have suggested the benefits of temporal analysis to improve our understanding of the learning process. Sequential pattern mining (SPM), as a pattern recognition technique, has the potential to reveal the temporal aspects of learning and can be a valuable tool in educational data science. However, its potential is not well understood and exploited. This chapter addresses this gap by reviewing work that utilizes sequential pattern mining in educational contexts. We identify that SPM is suitable for mining learning behaviors, analyzing and enriching educational theories, evaluating the efficacy of instructional interventions, generating features for prediction models, and building educational recommender systems. SPM can contribute to these purposes by discovering similarities and differences in learners' activities and revealing the temporal change in learning behaviors. As a sequential analysis method, SPM can reveal unique insights about learning processes and be powerful for self-regulated learning research. It is more flexible in capturing the relative arrangement of learning events than the other sequential analysis methods. Future research may improve its utility in educational data science by developing tools for counting pattern occurrences as well as identifying and removing unreliable patterns. Future work needs to establish a systematic guideline for data preprocessing, parameter setting, and interpreting sequential patterns.
View
... Students experience a variety of discrete affective states during learning and affective and self-regulatory components are highly related to engagement and learning in computer-based learning environments (e.g., Baker et al., 2010;D'Mello, 2013). Although evaluating and responding to these factors poses a number of methodological challenges (e.g., Greene & Azevedo, 2010;Winne, 2010), technology and convergent research around online education provide new opportunities to advance the measure, study, and support of SEL and SRL. ...
View
View
Towards a fuller picture: Triangulation and integration of the measurement of self‐regulated learning based on trace and think aloud data
Article
Full-text available
Background Many learners struggle to productively self‐regulate their learning. To support the learners' self‐regulated learning (SRL) and boost their achievement, it is essential to understand the cognitive and metacognitive processes that underlie SRL. To measure these processes, contemporary SRL researchers have largely utilized think aloud or trace data, however, not without challenges. Objectives In this paper, we present the findings of a study that investigated how concurrent analysis and integration of think aloud and trace data could advance the measurement of SRL and assist in better understanding the mechanisms of SRL processes, especially those details that remain obscured by observing each data channel individually. Methods We concurrently collected think aloud and trace data generated by 44 university students in a laboratory setting and analysed those data relative to the same timeline. Results We found that the two data channels could be interchangeably used to measure SRL processes for only 17.18% of all the time segments identified in a learning task. Moreover, SRL processes for around 45% of all the time segments could be detected via either trace data or think aloud data. For another 27.17% of all the time segments, different SRL processes were detected in both data channels. Conclusions Our results largely suggest that the two data collection methods can be used to complement each other in measuring SRL. In particular, we found that think aloud and trace data could provide different perspectives on SRL. The integration of the two methods further allowed us to reveal a more complex and more comprehensive temporal associations among SRL processes compared to using a single data collection method. In future research, the integrated measurement of SRL can be used to improve the detection of SRL processes and provide a fuller picture of SRL.
View
View
Promoting student engagement and preparation in flipped learning’s pre-class activities – A systematic review
Conference Paper
  • Dec 2020
Flipped learning is an instructional model with core features of pre-class learning followed by in- class practice of knowledge through active and collaborative learning. However, non-compliance with pre-class activities is a commonly cited challenge in flipped learning, where a lack of student engagement leads to insufficient pre-class preparation, diminishing flipped learning’s intended benefits significantly as the students will then not participate well in the in-class activities. It lends consideration to the potential obstacles and solutions in increasing students’ engagement in pre-class learning, paving the way to pre-class learning compliance to achieve the full potential of flipped learning in improving students’ learning outcomes. This study uses systematic literature review with qualitative content analysis to identify three main areas, namely technological, pedagogical, and student perceptions, to instil optimal conditions to promote pre-class engagement and preparation.
View
Model Penilaian Portofolio untuk Meningkatkan Kemandirian Peserta Didik: Sebuah Kajian Literatur
Article
  • Jul 2021
Currently the world is preparing for the era of society 5.0, in this era the education sector encourages the development of a more personal way of learning, so that students' learning independence and unique approaches to learning are prioritized. Learning independence plays an important role for students, because students are required to be able to take responsibility for making decisions related to their learning process and have the ability to carry out the decisions they make. In fact, the learning independence of students tends to be low, this is due to the learning objectives and non-specific learning methods that lead to increasing learning independence. Teachers must have the will and skills to change the way they relate to students, starting from paying attention to students' initial independence, teaching materials used, learning, scenarios or syntax and learning assessment. The main focus of this study is on the use of the portfolio assessment model as an effort to increase the learning independence of students in learning economics. The results of the study concluded that the use of the portfolio assessment model can increase the learning independence of students. So that the portfolio assessment model needs to be developed to increase the learning independence of students in learning economics
View
View
View
View
View
Modeling academic achievement by self-reported versus traced goal orientation
Article
We examined achievement goals measured by self-reports and by traces (behavioral indicators) gathered as undergraduates used software tools to study a multimedia-formatted article. Traces were operationalized by tags participants applied to selections of text and hyperlinks they clicked in the article. Tags and hyperlinks were titled to represent achievement goal orientations. Self-reported goal orientations did not correlate with goals traced as actions. In separate regression models, traces of goal orientations were stronger predictors of achievement than self reports. We suggest future research include traces in studies of achievement goals because traces reflect proximal events that comprise learning activities that can supplement static orientations that are operationally defined to be indifferent to the dynamics of learning activities.
View
View
View
View
View
How Software Technologies Can Improve Research on Learning and Bolster School Reform
Article
  • Mar 2006
Realizing the promise of software technologies in education requires thinking differently about how software simultaneously can serve research and contribute to learning. This article examines 3 axioms underlying contemporary educational psychology: Learners construct knowledge, learners are agents, and data include lots of randomness. By drawing out corollaries of these axioms, this research uncovers significant challenges researchers face in using classical forms of experimental research to build a basis for school reform and for testing school reforms using randomized field trials. This article describes a software system, gStudy, that is designed to address these challenges by gathering finer grained data that better support theorizing about the processes of learning and self-regulated learning. This research illustrates how this can be realized and suggests 10 ways that using software like gStudy can help pull up research by its bootstraps and bolster searches for what works.
View