No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Finding and fixing errors in worked examples: Can this foster learning outcomes?
Abstract
Learning from worked examples is an effective learning method in well-structured domains. Can its effectiveness be further enhanced when errors are included? This was tested by determining whether a combination of correct and incorrect solutions in worked examples enhances learning outcomes in comparison to correct solutions only, and whether a mixture of correct and incorrect solutions is more effective when the errors are highlighted. In addition, the effectiveness of fostering self-explanations was assessed. In Experiment 1, the participants learned to solve probability problems under six conditions that constituted a 2×3-factorial design (Factor 1: correct and incorrect solutions with highlighting the errors vs. correct and incorrect solutions without highlighting the errors vs. correct solutions only; Factor 2: prompting written self-explanations vs. no prompts). An aptitude-treatment interaction was found: providing correct and incorrect solutions fostered far transfer performance if learners had favourable prior knowledge; if learners had poor prior knowledge correct solutions only were more favourable. Experiment 2 replicated this interaction effect. Thus, a mixture of correct and incorrect solutions in worked examples enhanced learning outcomes only for “good” learners. In addition, Experiment 2 showed that confronting learners with incorrect solutions changed the quality of their self-explanations: on the one hand, new types of effective self-explanations could be observed, but on the other hand the amount of the very important principle-based self-explanations was substantially reduced. A possible measure to prevent this negative side effect of incorrect solutions is discussed.
... For example, self-explanations by students turn out to be important [16]. The request for selfexplanations (and also finding or fixing errors [17,18]) in our approach is an important difference from the simple presentation of solved problems in lectures. ...
... Refs. [1,3,11,16,17,[27][28][29][30][31][32][33][34][35][36]) relevant for the concept and our conclusions for worked examples in theoretical physics can be found in the supplementary material bellow this paper. ...
... The task to correct their own mistakes in mid-term exams can help students to increase their performance in the final exam [18]. However there is only a positive effect in finding and explaining errors in worked examples for learners with adequate prior knowledge [17], so implementing errors in worked examples too early can overwhelm weaker learners with little prior knowledge. To prohibit such an overchallenge, weaker learners need additional support by explicitly marking errors [17] or by expert explanations and feedback why certain steps in the solution are correct or incorrect [36]. ...
Studying worked examples has been shown by extensive research to be an effective method for learning to solve well-structured problems in physics and mathematics. The effectiveness of learning with worked examples has been demonstrated and documented in many research projects. In this work, we propose a new four-step approach for teaching with worked examples that includes writing explanations and finding and correcting errors. This teaching method can even be implemented in courses in which homework performance constitutes part of the grading system. This four-step approach is illustrated in the context of Lagrangian mechanics, which is ideal for the application of worked examples due to its universal approach to solve problems.
... Tracing how a dysfunctional procedure differs from a functional one and understanding why a procedure is dysfunctional could help learners to correctly update schemata of functional procedures and to create schemata of dysfunctional procedures (i.e., negative knowledge; Oser et al., 2012). When learners thoroughly elaborate the features of dysfunctional procedures, they are more likely to address their own knowledge gaps (e.g., Große and Renkl, 2007;Durkin and Rittle-Johnson, 2012; Barbieri and Booth, 2020). ...
... It remains unclear what form of instruction on dysfunctional procedures as a complement to instruction on functional procedures is appropriate to promote error awareness and the development of an EIRS in pre-service teachers. On the one hand, specific exemplification of how not to do something using dysfunctional (or erroneous) examples might encourage learners to identify, comprehend, explain, and/or remedy own dysfunctional procedures by referring to underlying concepts, principles, or strategies (e.g., Große and Renkl, 2007;Durkin and Rittle-Johnson, 2012;Barbieri and Booth, 2020). On the other hand, students might not benefit from exemplifications, if their own solution approach does not resemble the dysfunctional procedures presented; specific exemplifications of other students' dysfunctional procedures that have not been used by the learners themselves might even distract them from becoming aware of the correctness of their own approach (Loibl and Leuders, 2019). ...
... The specific form of instruction on dysfunctional procedures does not seem to be relevant for the internalization of the EIRS, but particularly the fact that dysfunctional strategies of solving a problem situation are presented at all. Apparently, underlining typical dysfunctional solution approaches of other pre-service teachers leads attention to the principles to be learned (Große and Renkl, 2007;Oser et al., 2012). Surprisingly, when it came to applying the EIRS to Frontiers in Education 14 frontiersin.org ...
Pre-service teachers face difficulties when dealing with problem situations in the classroom if their evidence-informed reasoning script (EIRS) is not adequately developed. An EIRS might be promoted by demonstrating how to implement evidence-informed reasoning after a problem-solving activity on an authentic case. However, it is unclear what form of instruction is appropriate to promote pre-service teachers in the development of an EIRS. The present 2×3-factorial experimental intervention study investigated how different forms of instruction on functional procedures (example-free vs. example-based) and on dysfunctional procedures (without vs. example-free vs. example-based) affect the development of an EIRS. N = 384 pre-service teachers worked on a written case vignette of a problem situation in a problem-solving phase, in which the crucial steps of the EIRS were prompted externally. In the subsequent instruction phase, the participants compared their own solution with an example-free or example-based instruction on functional procedures, which was either supplemented by an example-free or example-based instruction on typical dysfunctional procedures or not at all. The participants’ learning success (declarative EIRS; near and far transfer problem-solving performance) and error awareness were assessed. The results revealed that the example-based instruction on functional procedures led to a higher learning success than the example-free instruction. Both forms of instruction on dysfunctional procedures improved learning success compared to learning without one. During learning, error awareness was higher for learners who worked with an example-free instruction on dysfunctional procedures. In order to promote the development of an EIRS in pre-service teachers, it is promising to provide instruction after problem-solving that presents a functional example of evidence-informed reasoning for the given problem and that also points out typical dysfunctional approaches to solving the problem. The results highlight the importance of selecting appropriate scaffolds in case-based learning approaches that aim to develop cognitive schemata. The mechanisms that explain when and why instructions on dysfunctional procedures work need to be further explored.
... As delineated in Wong and Lim's (2019b) Prevention-Permission-Promotion (3P) framework, errors can be approached in different ways to optimize learning in various educational contexts, such as observing, allowing, inducing, or even guiding errors when not avoiding them entirely. For instance, errors are observed when studying others' incorrect solutions in worked examples or case studies (e.g., Adams et al., 2014;Barbieri and Booth, 2020;Booth et al., 2013;Durkin and Rittle-Johnson, 2012;Große and Renkl, 2007;Heemsoth and Heinze, 2014;Joung et al., 2006), whereas errors are passively allowed to occur naturalistically during exploration in discovery learning (e.g., Alfieri et al., 2011;Bruner, 1961;Kirschner et al., 2006;Klahr and Nigam, 2004). Alternatively, in errorful generation (e.g., Kang et al., 2011;Kornell et al., 2009;Potts and Shanks, 2014), productive failure (e.g., Kapur, 2008Kapur, , 2016Kapur and Bielaczyc, 2012), and error management training (e.g., Keith and Frese, 2008), errors are actively induced through purposefully adding challenge to the task or withholding instruction, and encouraging learners to guess or produce the correct answers. ...
... One might then question whether the derring effect stems from simply exposing learners to incorrect responses, particularly when their deliberate errors are juxtaposed with the actual concepts during error correction. For instance, relative to studying only correct solutions, some studies have found that comparing a mixture of correct and incorrect solutions produces better transfer in domains such as experimental design concept learning (Corral and Carpenter, 2020), mathematical problem-solving (Große and Renkl, 2007;Loibl and Leuders, 2019), computer programming (Beege et al., 2021), and behavior modeling training (Baldwin, 1992;Taylor et al., 2005). Presumably, comparing correct versus incorrect responses may highlight the diagnostic features that define the to-be-learned concept, and in turn foster richer mental models and deeper learning that facilitate transfer. ...
... Here, the advantage of deliberate erring was established among students who reported relatively low prior knowledge of the learning material. In contrast, when observing errors in worked examples, learners with low prior knowledge have been found to benefit more from studying correct solutions only, whereas learners with high prior knowledge benefit from studying a mixture of correct and incorrect solutions (Große and Renkl, 2007). To reconcile these findings and inform predictions about when errorful learning is more or less likely to be helpful, future work could examine the nature of interactions between learner characteristics and the learning task. ...
Unlabelled:
Transfer of learning is a fundamental goal of education but is challenging to achieve, especially where far transfer to remote contexts is at stake. How can we improve learners' flexible application of knowledge to distant domains? In a counterintuitive phenomenon termed the derring effect, deliberately committing and correcting errors in low-stakes contexts enhances learning more than avoiding errors. Whereas this benefit has been demonstrated with tests in domains similar to those in the initial learning task, the present set of three experiments (N = 120) investigated whether deliberate erring boosts far transfer of conceptual knowledge to dissimilar domains. Undergraduates studied scientific expository texts either by generating conceptually correct responses or by deliberately generating conceptually erroneous responses then correcting them. Deliberate erring improved not only retention (Experiment 1), but also far transfer on inferential test questions that required applying the learned concepts to remote knowledge domains (e.g., from biology/vaccines to geography/forest management techniques; Experiment 2). This advantage held even over a control that further involved spotting and correcting the same errors that one's peers had deliberately made (Experiment 3). Yet, learners failed to predict or recognize the benefits of deliberate erring even after the test. Altogether, these results suggest that the derring effect is specific to generating incorrect, but not correct, elaborations. Neither does mere exposure to others' errors nor juxtaposing these errors with the correct responses suffice. Rather, guiding learners to personally commit and correct deliberate errors is vital for enhancing generalization and far transfer of learning to distant knowledge domains.
Supplementary information:
The online version contains supplementary material available at 10.1007/s10648-023-09739-z.
... More guided methods include for example the presentation of explicit instructional text or correct worked examples, which provide structure and direct the learner in the process Loibl et al., 2017). Less guided methods, on the other hand, include for example embedding problem-solving tasks or erroneous worked examples, which have more of an explorative character and ask learners, at least partly, to develop solutions by themselves (Große and Renkl, 2007;Kapur, 2008). Various studies have already confirmed the conduciveness of each of these teaching methods for deeper learning processes (e.g., Sweller and Cooper, 1985;Kalyuga, 2007;Booth et al., 2013). ...
... Traditional research on the worked example effect has mainly focused on completely correct examples (e.g., Paas, 1992;Sweller et al., 1998). Over time, however, a branch of research has emerged that investigates the learning effectiveness of erroneous worked examples (e.g., Joung et al., 2006;Große and Renkl, 2007 (Bransford and Schwartz, 1999;VanLehn, 1999;Siegler, 2002;Joung et al., 2006;Barbieri and Booth, 2016). One widely cited explanation is offered by cascade theory (VanLehn, 1999) which describes the learning or problem-solving process as a continuous sequence of impasses, reflections, and repairments. ...
... In the best case, the learner finds the correct operator and can free himself or herself from the impasse. EE may be more likely to provoke such impasse situations than CE (Große and Renkl, 2007;Heemsoth and Heinze, 2014). The assignment to comprehend why a correctly applied problem-solving operator is correct probably does not stimulate as much reflection on the valid rules and limits of this operator as the assignment to comprehend why an incorrectly applied operator is incorrect (Siegler and Chen, 2008). ...
Several studies highlight the importance of the order of different instructional methods when designing learning environments. Correct but also erroneous worked examples are frequently used methods to foster students’ learning performance, especially in problem-solving. However, so far no study examined how the order of these example types affects learning. While the expertise reversal effect would suggest presenting correct examples first, the productive failure approach hypothesizes the reversed order to be learning-facilitating. In addition, congruency of subsequent exemplified problems was tested as a moderator of the effect of order on learning. For example, with arithmetic tasks, congruent problems target exactly the same calculation while incongruent problems refer to different calculations. Following cascade theory, a model of cognitive skill acquisition, presenting correct examples first should be more effective when the subsequent exemplified problems are different. To test the (conflicting) hypotheses, 83 university students were assigned to one of the four conditions in a 2 (correct vs. erroneous example first) × 2 (same vs. different exemplified problems) between-subject design. Learners navigated through a slideshow on the topic of Vedic mathematics consisting of explicit instruction, worked examples differing in terms of the experimental condition, and transfer problems. Although no main or interaction effects were found regarding students’ learning performance, mediational analysis offered support for the expertise reversal effect, as it indicated that there is a significant indirect effect of order via mental load on learning. Presenting correct examples first and erroneous examples second resulted in a lower mental load, which in turn was associated with better learning performance. In contrast, presenting erroneous examples first and correct examples second resulted in a more accurate self-assessment of learning performance. These findings offer first insights into the question of how the presentation order of different example types impacts learning and provide practical recommendations for the design of educational media. Results are discussed in light of the ongoing debate regarding the question if less guided instructional methods should precede or succeed more guided methods.
... Before the pre-experimental phase began, the classroom teacher read a script that explained how to engage with the software. During the pre-experimental phase, the participants reported their grade level (9 to 12) and age, and took a prior knowledge probability pretest consisting of eight questions (four relevant order and four irrelevant order probability problems) that were adapted with permission from the lead authors of Große and Renkl (2007) and Foster et al. (2018). A maximum of 16 minutes was allowed to complete the pretest problems, which were presented in random order. ...
... To investigate the dependent variable of problem-solving performance, 8 probability problems (4 relevant order and 4 irrelevant order) from Große and Renkl (2007), also adapted for use by Foster et al. (2018), were used with the permission of Große, Foster, and Dunlosky. The problems were presented in random order for all participants. ...
... The homogeneity of regression slopes assumption means that there should be no significant interaction between the covariate and the independent variable. This interaction effect was included in the model and checked for significance to determine whether the fourth assumption held.Instruments PretestTo investigate prior knowledge, 8 probability problems (4 relevant order, 4 irrelevant order) fromGroße and Renkl (2007), also adapted for use byFoster et al. (2018), were used with the permission of Große, Foster, and Dunlosky. The problems were presented on Gorilla Experiment Builder(Anwyl-Irvine et al., 2018) in random order in the problem-solving format. ...
Previous research has shown mixed results of allowing learners to control their access to guidance in the form of worked examples during instruction. This study aimed to improve the instructional design knowledge base through a randomized controlled trial that tested the effects of three types of control on final test performance and subjective ratings of cognitive load. One hundred seventy-two 9th-12th grade students from an independent school took a pretest for prior knowledge and were randomly sorted into one of three instructional treatments: Instruction that allowed students to choose between studying worked examples and solving equivalent problems, an identical instruction to the first group with the difference being the provision of information for how to manage the instruction based on principles of example-based learning, and instruction with a fixed sequence of alternating worked examples and problems. Cognitive load was measured with a 9-point mental effort scale after each trial of the learning phase, and a posttest of problem-solving performance was administered 13 days later. An exploratory analysis of the control decisions of the two learner-controlled groups suggested that learners often violated established principles of example-based learning, and that providing information prompts for how to manage the instruction improved some of the selection behavior. The results of the hypothesis testing showed a significant effect of prior knowledge on performance and cognitive load, but non-significant differences in cognitive load and posttest performance between the groups. The fact that both null hypotheses of the research could not be rejected prevented conclusions about the efficacy of allowing learners to control their use of worked examples during problem solving. However, the results did provide evidence of the effectiveness of supporting learners with helpful information when they are expected to control guidance during instruction. Recommendations for future research were provided.
... Compared with problem solving, worked examples can reduce the cognitive load of learners and can enable students to learn principles and acquire problem solving schema successfully (Bokosmaty et al., 2015). Research has shown the benefits of worked examples for many fields such as mathematics and physics (Große & Renkl, 2007;van Gog et al., 2011). Problem solving makes learners have to keep many aspects in mind such as the current problem state, the goal state, differences between these states, etc. ...
... It requires a lot of cognitive resources. In contrast, when studying worked examples, the learners are freed from performance demands and they can concentrate on gaining understanding (Große & Renkl, 2007;van Gog et al., 2011). ...
... The items in the experiment were mainly from Foster et al. (2018) and Große and Renkl (2007). Other items were added according to the purpose of the current study. ...
If individuals are equipped with a strategy repertoire, how do they select strategies? Worked examples result in better performance than problem-solving; therefore, this study investigates which strategy students will choose to use in math. Three experiments examined if (1) material difficulty, (2) strategy knowledge, and (3) knowledge intervention affected strategy selection. The results showed that (1) self-regulated learners chose to study worked examples or solve problems based on the difficulty of material. Learners preferred worked examples for difficult material but problem-solving for simple material. Learners who studied material of different difficulty showed different selection trends. (2) Strategy knowledge affected strategy selection. Learners with strategy knowledge chose worked examples more than those without strategy knowledge, who themselves chose problem-solving, even after a problem was solved incorrectly. (3) Participants who experienced strategy knowledge intervention made more reasonable strategy selection. Learners without strategy knowledge significantly increased the selection of worked examples post-intervention, and were more likely to study worked examples after a problem was solved incorrectly.
... In addition to presenting real-world classroom situations, it is often recommended to provide pre-service teachers with examples illustrating how to do something by the means of a teacher modeling "best practice" (Oonk et al., 2004;Rosaen et al., 2004;Sonmez and Hakverdi-Can, 2012 (Oser et al., 2012). However, examples of how not to do something can also be beneficial for learning, as learners are encouraged to identify, comprehend, explain, and/or fix dysfunctional procedures by referring to underlying rules or principles (Große and Renkl, 2007;Durkin and Rittle-Johnson, 2012;Booth et al., 2013;Barbieri and Booth, 2020). By performing elaboration processes, learners acquire knowledge of what to do and what not to do in certain situations (negative knowledge; Oser et al., 2012). ...
... Some studies from different domains than teacher education indicated that a simultaneous presentation of written examples is more beneficial to learning and transfer than a sequential presentation of written examples (e.g., Gentner et al., 2003;Star and Rittle-Johnson, 2009 Considering the genuine logic of learning from errors, which is (1) noticing that an error was committed, (2) reconstructing the error by comparing it to one functional solution and (3) deriving error avoidance strategies (Oser and Spychiger, 2005;Oser et al., 2012;Tulis et al., 2016), it seems favorable to present the dysfunctional example first. The phenomenon of negative stimuli attracting more attention than positive ones (negativity bias; positive-negative asymmetry; Ohira et al., 1998) suggests that presenting a dysfunctional example before a functional example is more likely to guide learners' attention to target incidents than the other way around (Große and Renkl, 2007;Booth et al., 2013;Barbieri and Booth, 2020). ...
... Furthermore, negative stimuli attract more attention than positive ones (negativity bias; positive-negative asymmetry; Ohira et al., 1998). The dysfunctional example seems to have guided learners' attention to target incidents (Große and Renkl, 2007;Booth et al., 2013;Barbieri and Booth, 2020) and stimulated them to find corresponding incidents in the subsequent functional example, even without being requested by the second prompt. In contrast to this, the learners who analyzed the functional video example first were initially presented to classroom interactions that were conformed to the instructional text presented before. ...
Everyday teaching requires teachers to deal with a variety of pedagogical issues, such as classroom disruptions. Against the background of on-going calls for an evidence-informed practice, teachers should ground their pedagogical decisions not only on subjective theories or experience-based knowledge but also on educational theories and empirical findings. However, research suggests that pre- and in-service teachers rather refer to experiential knowledge than to educational knowledge when addressing practical, pedagogical issues. One reason for the infrequent use of educational knowledge is that acquired knowledge has remained inert and cannot be applied to complex situations in practice. Therefore, implementing learning with contrastive (i.e., functional and dysfunctional) video examples in teacher education seems promising to promote pre-service teachers’ acquisition of educational knowledge. The 2×2-intervention study (N = 220) investigated the effects of the video sequence (dysfunctional-functional/functional-dysfunctional) and of video analysis prompts (with/without) on learning outcomes (concept knowledge, application knowledge) and on learning processes (written video analyses). Results revealed that the sequence dysfunctional-functional led to higher application knowledge in the post-test. There was no sequencing effect on concept knowledge. Prompted groups showed higher concept knowledge and application knowledge in the post-test. Furthermore, both experimental factors affected learning processes, which resulted in higher learning outcomes. In conclusion, learning with contrastive video examples in teacher education seems to be more effective if the video examples are presented in the sequence dysfunctional-functional and if instructional prompts guide the video analysis. The results substantiate the relevance of instructional guidance in learning with video examples and broaden the scope of validity of the concept of learning from errors.
... Worked examples are a form of direct instruction in which learners are presented with a step-by-step solution to a problem [44]. A controversial adaptation of worked examples is erroneous examples, which deliberately replace some of the correct steps in the solution with incorrect steps. ...
... A controversial adaptation of worked examples is erroneous examples, which deliberately replace some of the correct steps in the solution with incorrect steps. Though some intuitively believe that intentionally introducing students to incorrect steps hinders learning, researchers argue that it introduces a desirable difficulty in which students must self-explain why steps are incorrect and reconcile incorrect and correct conceptions [2,3,44,102]. ...
... Similarly, studying erroneous examples causes more confusion and frustration than correct examples [2,91]. While examining learner differences, Große & Renkl [44] found that erroneous examples were more effective for students with higher prior knowledge while correct examples were more effective for students with lower prior knowledge. As a result, erroneous examples might be best suited for more advanced learners who can treat finding and recovering from errors as a puzzle rather than a reflection of their aptitude. ...
... In a variety of studies, it became evident that showing errors in worked examples had benefits for learning (e.g., Booth et al., 2013;Durkin & Rittle-Johnson, 2012;Gadgil et al., 2012;Große & Renkl, 2007;McLaren et al., 2015;Tsovaltzi et al., 2013;Wagner et al., 2018). Showing hypothetical errors fostered reflection processes and made learners aware of their errors. ...
... When discussing learning from worked examples, prior knowledge of the learners has to be considered as well. Learners with low prior knowledge did not benefit from erroneous worked examples in contrast to learners with high prior knowledge (Große & Renkl, 2007). Other research outlined that novices and advanced students improved their learning scores significantly when learning with erroneous examples but students with high prior knowledge showed better performance on problem-solving (Chen et al., 2016). ...
... Thus, the following hypothesis was formulated: Große and Renkl (2004) pointed out that the learning content should be presented correctly before showing erroneous worked examples because learners have to acquire knowledge first to gain enough expertise to detect errors. Thus, learners should intellectually gain the ability to work with the embedded error (Oser & Spychiger, 2005) and should explicitly be instructed to detect, explain, and correct errors (Große & Renkl, 2007;Heitzmann et al., 2018). In consequence, errors can only unfold their potential beneficial effects when learners detect and explain these errors correctly. ...
Learning from erroneous worked examples could enhance learning in contrast to problem-solving tasks. The type of error was hypothesized to be a moderator and accuracy of error detection and correction a mediator of this effect. This study examines the influence of simple syntactic (the structure of the code) and complex semantic (the logic or content of the code) errors in a programming scenario. Overall, 128 students were assigned to a two (syntactic errors: yes vs. no) × two (semantic errors: yes vs. no) factorial between-subjects design. Students’ accuracy in error detection and correction, learning performance, mental load, and mental effort were measured. Results showed that learners receiving syntactic errors detected and corrected errors with higher accuracy which leads to higher learning performance. Semantic errors did not influence learning-related variables since semantic errors were too difficult for novice learners to detect and fix. The postulated moderation and mediation could be supported.
... However, incorrect worked examples may be inappropriate for learners who are inexperienced in problem solving or in a specific topic. They tend not to recognize the errors presented in a solution because of the mistakes they frequently make (Grobe & Renkl, 2007). The existing evidence then offers a trade-off worth exploring in this context between the level of activity engagement and students' problem-solving experience: do students of statics need to self-explain complete worked examples because they do not have much experience with the topic? ...
... Likewise, significant results on gain as a function of example condition suggest that changes in conceptual understanding tend to evidence better results when students attempt to self-explain incomplete worked examples. Existing literature suggests that incorrect worked examples may be counterproductive for novice learners (Barbieri et al., 2021;Grobe & Renkl, 2007). Students who reach an impasse during problem solving are more likely to learn from this process (VanLehn et al., 2003). ...
Background: The complexity and diversity of problems and concepts in different engineering subjects represent a great challenge for students. Traditional approaches to teaching statics are ineffective in helping some students overcome the learning barriers that underlie learning statics and developing problem-solving skills.
Purpose: This article explores how self-explanation activities may support student learning in statics. Specifically, this study examines the characteristics of student self-explanations of worked examples and their relationship with students´ conceptual change.
Design/Method: The study population included 147 undergraduate
engineering students enrolled in a statics course. The students wrote their selfexplanations at each step of an incomplete or incorrect worked example in the context of static equilibrium. Students0 self-explanations were qualitatively analyzed using content analysis to identify the approaches used. We used
descriptive and inferential statistics to identify differences in students0 conceptual understanding of statics, based on their approach to self-explanation.
Results: We identified four self-explaining approaches: restricted explanations, elemental explanations, inferential explanations, and strategic explanations. After completing the activity, students who self-explained incomplete worked examples showed better results in the quality of their explanations and conceptual change than students in the incorrect worked example condition.
Conclusions: The findings suggest a relationship between the type of worked example, students' approaches to self-explaining, and their conceptual change and problem-solving skills in statics. To increase the quality of the students' explanations and to improve their conceptual understanding, additional prompts
or initial training in self-explaining may be required within the workedexamples context.
... The relationship between prior knowledge and students' switch decisions during self-regulated learning is unknown. Thus, we measured students' prior knowledge with self-assessments (Braithwaite and Goldstone 2015;Große and Renkl 2007) and objective tests (Braithwaite and Goldstone 2012;Große and Renkl 2007;Rau et al. 2010) prior to completing two phases of a self-regulated learning task. Phase 1 occurred prior to covering the target content in the course and Phase 2 occurred after covering this content, which allowed us to investigate how changes in prior knowledge affect switch decisions. ...
... The relationship between prior knowledge and students' switch decisions during self-regulated learning is unknown. Thus, we measured students' prior knowledge with self-assessments (Braithwaite and Goldstone 2015;Große and Renkl 2007) and objective tests (Braithwaite and Goldstone 2012;Große and Renkl 2007;Rau et al. 2010) prior to completing two phases of a self-regulated learning task. Phase 1 occurred prior to covering the target content in the course and Phase 2 occurred after covering this content, which allowed us to investigate how changes in prior knowledge affect switch decisions. ...
Learning complex concepts is necessary for student success, but it is often challenging. Learning such concepts can be influenced by students’ study order choices during learning to switch to a new category (interleaved study order) or stay within the same category (blocked study order). Students often prefer stay decisions during learning and make relatively few switch decisions; however, an open question is whether students’ switch decisions are related to their level of prior knowledge in the domain and the learning strategy they use (retrieval practice versus study). To examine these relationships, we recruited undergraduate students from an introductory geology course. Prior to the course modules on rock classification, students self-rated their knowledge, took a prior knowledge test, classified rock exemplars by completing study or retrieval practice trials, and made study order choices. Students then completed assignments and attended lectures in their geology course on igneous, sedimentary, and metamorphic rocks. Next, students self-rated their knowledge, took a new prior knowledge test, completed study or retrieval practice trials, made study order decisions, took final classification tests, and self-reported beliefs about study order choices. Even though students’ knowledge increased after course modules on rock identification, and most students believed that domain knowledge impacts study decisions, prior knowledge did not predict students’ switch decisions. In contrast, students who completed retrieval practice trials made substantially more switch decisions (i.e., interleaved study) than did students who completed study trials.
... Even incorrect examples can be beneficial in learning to avoid errors (Durkin & Rittle-Johnson, 2012;Siegler & Chen, 2008). Based on evidence that explicit description of the error within an example facilitates learning (Große & Renkl, 2007), our intervention presented a causal theory error made in an example and explained why the specific causal inference was not warranted given the evidence (Stark et al., 2011). ...
... The intervention was designed following recommendations from example-based instruction (Shafto et al., 2014;Siegler & Chen, 2008;Van Gog & Rummel, 2010). Worked examples may be especially important in learning to reason about new problems (Renkl et al., 1998), along with explicitly stating the causal theory error and explanation within the example (Große & Renkl, 2007;Stark et al., 2011). Our intervention also emphasized recognizing causal theory errors in others' reasoning, which may be easier to learn than attempting to correct one's own errors. ...
When reasoning about science studies, people often make causal theory errors by inferring or accepting a causal claim based on correlational evidence. While humans naturally think in terms of causal relationships, reasoning about science findings requires understanding how evidence supports—or fails to support—a causal claim. This study investigated college students’ thinking about causal claims presented in brief media reports describing behavioral science findings. How do science students reason about causal claims from correlational evidence? And can their reasoning be improved through instruction clarifying the nature of causal theory error? We examined these questions through a series of written reasoning exercises given to advanced college students over three weeks within a psychology methods course. In a pretest session, students critiqued study quality and support for a causal claim from a brief media report suggesting an association between two variables. Then, they created diagrams depicting possible alternative causal theories. At the beginning of the second session, an instructional intervention introduced students to an extended example of a causal theory error through guided questions about possible alternative causes. Then, they completed the same two tasks with new science reports immediately and again 1 week later. The results show students’ reasoning included fewer causal theory errors after the intervention, and this improvement was maintained a week later. Our findings suggest that interventions aimed at addressing reasoning about causal claims in correlational studies are needed even for advanced science students, and that training on considering alternative causal theories may be successful in reducing casual theory error.
... Students are encouraged to go beyond noticing surface characteristics and to think deeply about how erroneous steps differ from correct ones and why a solution step is incorrect (Durkin & Rittle-Johnson, 2012). This might help them to correctly update schemas of correct concepts and strategies and, moreover, to create schemas for erroneous strategies (Durkin & Rittle-Johnson, 2012;Große & Renkl, 2007;Siegler, 2002;Van den Broek & Kendeou, 2008;VanLehn, 1999), reducing the probability of recurring erroneous solutions in the future (Siegler, 2002). ...
... However, erroneous examples are typically presented separately from correct examples, requiring learners to use mental resources to recall the gist of the no longer visible correct solutions (e.g., Große & Renkl, 2007;Stark et al., 2011). Splitting attention across time increases the likelihood that mental resources will be expended on activities extraneous to learning, which subsequently may hamper learning (i.e., temporal contiguity effect: e.g., Ginns, 2006). ...
There is a need for effective methods to teach critical thinking (CT). One instructional method that seems promising is comparing correct and erroneous worked examples (i.e., contrasting examples). The aim of the present study, therefore, was to investigate the effect of contrasting examples on learning and transfer of CT-skills, focusing on avoiding biased reasoning. Students ( N = 170) received instructions on CT and avoiding biases in reasoning tasks, followed by: (1) contrasting examples, (2) correct examples, (3) erroneous examples, or (4) practice problems. Performance was measured on a pretest, immediate posttest, 3-week delayed posttest, and 9-month delayed posttest. Our results revealed that participants’ reasoning task performance improved from pretest to immediate posttest, and even further after a delay (i.e., they learned to avoid biased reasoning). Surprisingly, there were no differences in learning gains or transfer performance between the four conditions. Our findings raise questions about the preconditions of contrasting examples effects. Moreover, how transfer of CT-skills can be fostered remains an important issue for future research.
... Furthermore, erroneous examples in assessments provide students with opportunities to reflect upon and demonstrate their own understanding of the material in the hopes of not repeating those errors in the future (Quibble 2004, Yerushalmi and Polingher 2006, Koehler 2020. Große and Renkl (2007) note that, whereas students are able to explain in their own words why erroneous statements are incorrect, they are less likely to describe the underlying principles that would correct the error; requiring students to also fix the mistake closes the loop in this learning cycle. It is true that identifying and correcting error-filled examples can lead to more "confrustion" (confusion and frustration) compared with simply solving incomplete examples (Richey et al. 2019), but student learning in the long run does not suffer. ...
We introduce a novel type of assessment that allows for efficient grading of higher order thinking skills. In this assessment, a student reviews and corrects a technical memo that has errors in its formulation or process. To overcome the grading challenges imposed by essay-type responses in large undergraduate courses, we provide a Visual Basic for Applications Excel tool for instructors, ensuring efficient grading of student submissions. We report our experience using this type of assessment in a multisection introductory business analytics course over several years and present survey-based evidence indicating that students perceive it to be clear and beneficial for learning.
Supplemental Material: Data is available at https://www.informs.org/Publications/Subscribe/Access-Restricted-Materials
... • Providing additional support when critiquing flawed examples (Tsovaltzi et al., 2010) • For example, highlighting of intuitive ideas (Durkin & Rittle-Johnson, 2012;Große & Renkl, 2007), or use of annotator function to critique (Gerard & Linn, 2022) Using critique in class helps students to develop integrated knowledge of science topics if adequate support is provided and they have to think and cannot just copy ...
Revising science explanations allows learners to develop a coherent understanding of scientific phenomena by integrating different ideas, a key learning outcome according to the Knowledge Integration (KI) Framework (Linn & Eylon, 2011). We explore the design of critique activities and analyze how peer critique helps 1305 middle school students to revise science explanations. We compare three alternative designs that differ in the characteristics of the peer ideas the student is asked to critique. Specifically, students critiqued a peer’s explanation about thermodynamics that describes observations (intuitive); combines intuitive and normative ideas (partial); or connects a claim with scientific evidence (linked). Students wrote and revised their explanation and critiqued their peer’s ideas in an online learning environment with random assignment to the critique conditions. Initial and revised explanations were scored using a KI rubric that rewarded linking claims and evidence. Overall, students improved from initial to revised explanation. Students in the linked condition received the highest KI scores. Additionally, in all conditions, students who started with a higher KI score had an advantage over students who started with a lower KI score. This difference was strongest in the intuitive condition. Qualitative analyses indicated that many students copied ideas from the example, regardless of their correctness. This might explain why students in the linked condition performed better in the revision than those in the partial or intuitive conditions. We suggest that a more effective critique activity is to compare two explanations and offer more guidance for students (e.g., evaluation criteria or reflection prompts).
... In relation to the four guidelines presented by Rittle-Johnsson et al. (2017), this type of SEP is intended to direct attention to important aspects of the content. The second type of SEP was designed as solved examples (see Große & Renkl, 2007) which students are asked to evaluate in terms of whether they are properly executed. In this variant of SEP, the solution process is either clearly stated and possible to follow, or containing a statement to be discussed (see Figure 1 and Figure 3). ...
This study concerns mediating activities in student discussions during collaborative work with self-explanation prompts (SEPs). While the aim of most other tasks, from the students' perspective, can be perceived as finding the correct answer, discussions supported by SEPs require a different approach, because students must engage in mathematical discussions, and explain their insights into the mathematics at hand. In this study, we explore activities that are fostered by SEPs. The analysis of the activities taking place, reveal five mediating activities to promote in teaching, but also potential hiders for the intended discussion to occur.
... Another possible explanation for the lack of difference could be that, although processing the emotional claim information might have come (partly) at the expense of clinical information, as found in previous studies [55][56][57], it did not affect diagnostic accuracy scores in our participants, because advanced learners such as our participants might be able to handle more cognitive load since they have already mastered the basics. A similar theory is confirmed by the studies of Große and Renkl [58,59], which show that especially learners with solid knowledge profit from incorrect examples. In addition, several studies show that enrichment of case vignettes with atypical disease presentations, rare conditions or complex contexts is especially beneficial for advanced students [42,[60][61][62]. ...
Background
Using malpractice claims cases as vignettes is a promising approach for improving clinical reasoning education (CRE), as malpractice claims can provide a variety of content- and context-rich examples. However, the effect on learning of adding information about a malpractice claim, which may evoke a deeper emotional response, is not yet clear. This study examined whether knowing that a diagnostic error resulted in a malpractice claim affects diagnostic accuracy and self-reported confidence in the diagnosis of future cases. Moreover, suitability of using erroneous cases with and without a malpractice claim for CRE, as judged by participants, was evaluated.
Methods
In the first session of this two-phased, within-subjects experiment, 81 first-year residents of general practice (GP) were exposed to both erroneous cases with (M) and erroneous cases without (NM) malpractice claim information, derived from a malpractice claims database. Participants rated suitability of the cases for CRE on a five-point Likert scale. In the second session, one week later, participants solved four different cases with the same diagnoses. Diagnostic accuracy was measured with three questions, scored on a 0–1 scale: (1) What is your next step? (2) What is your differential diagnosis? (3) What is your most probable diagnosis and what is your level of certainty on this? Both subjective suitability and diagnostic accuracy scores were compared between the versions (M and NM) using repeated measures ANOVA.
Results
There were no differences in diagnostic accuracy parameters (M vs. NM next step: 0.79 vs. 0.77, p = 0.505; differential diagnosis 0.68 vs. 0.75, p = 0.072; most probable diagnosis 0.52 vs. 0.57, p = 0.216) and self-reported confidence (53.7% vs. 55.8% p = 0.390) of diagnoses previously seen with or without malpractice claim information. Subjective suitability- and complexity scores for the two versions were similar (suitability: 3.68 vs. 3.84, p = 0.568; complexity 3.71 vs. 3.88, p = 0.218) and significantly increased for higher education levels for both versions.
Conclusion
The similar diagnostic accuracy rates between cases studied with or without malpractice claim information suggests both versions are equally effective for CRE in GP training. Residents judged both case versions to be similarly suitable for CRE; both were considered more suitable for advanced than for novice learners.
... Research about learning with errors shows that such activities are most beneficial for advanced students and only with an explicit intervention [29,30]. Upon asking at the end of the survey, some students stated that they found this small exercise to be very instructive; some even wished for more exercises of this kind. ...
This study aimed at evaluating how students perceive the linguistic quality and scientific accuracy of ChatGPT responses to physics comprehension questions. A total of 102 first- and second-year physics students were confronted with three questions of progressing difficulty from introductory mechanics (rolling motion, waves, and fluid dynamics). Each question was presented with four different responses. All responses were attributed to ChatGPT, but in reality, one sample solution was created by the researchers. All ChatGPT responses obtained in this study were wrong, imprecise, incomplete, or misleading. We found little differences in the perceived linguistic quality between ChatGPT responses and the sample solution. However, the students rated the overall scientific accuracy of the responses significantly differently, with the sample solution being rated best for the questions of low and medium difficulty. The discrepancy between the sample solution and the ChatGPT responses increased with the level of self-assessed knowledge of the question content. For the question of highest difficulty (fluid dynamics) that was unknown to most students, a ChatGPT response was rated just as good as the sample solution. Thus, this study provides data on the students’ perception of ChatGPT responses and the factors influencing their perception. The results highlight the need for careful evaluation of ChatGPT responses both by instructors and students, particularly regarding scientific accuracy. Therefore, future research could explore the potential of similar “spot the bot” activities in physics education to foster students’ critical thinking skills.
... However, Patchan et al. (2013) showed that writers with high general verbal ability also received similar feedback from low-ability peers, whereas low-ability writers of initial solutions received more valuable feedback from high-ability peers. Therefore, as a leverage point, it might be recommendable to assign initial solutions of more competent learners to less competent learners and initial solutions of less competent learners to more competent learners (comparable to positive vs. erroneous worked examples; Große & Renkl, 2007;Tsovaltzi et al., 2012). ...
Advancements in artificial intelligence are rapidly increasing. The new‐generation large language models, such as ChatGPT and GPT‐4, bear the potential to transform educational approaches, such as peer‐feedback. To investigate peer‐feedback at the intersection of natural language processing (NLP) and educational research, this paper suggests a cross‐disciplinary framework that aims to facilitate the development of NLP‐based adaptive measures for supporting peer‐feedback processes in digital learning environments. To conceptualize this process, we introduce a peer‐feedback process model, which describes learners' activities and textual products. Further, we introduce a terminological and procedural scheme that facilitates systematically deriving measures to foster the peer‐feedback process and how NLP may enhance the adaptivity of such learning support. Building on prior research on education and NLP, we apply this scheme to all learner activities of the peer‐feedback process model to exemplify a range of NLP‐based adaptive support measures. We also discuss the current challenges and suggest directions for future cross‐disciplinary research on the effectiveness and other dimensions of NLP‐based adaptive support for peer‐feedback. Building on our suggested framework, future research and collaborations at the intersection of education and NLP can innovate peer‐feedback in digital learning environments.
Practitioner notes
What is already known about this topic There is considerable research in educational science on peer‐feedback processes.
Natural language processing facilitates the analysis of students' textual data.
There is a lack of systematic orientation regarding which NLP techniques can be applied to which data to effectively support the peer‐feedback process.
What this paper adds A comprehensive overview model that describes the relevant activities and products in the peer‐feedback process.
A terminological and procedural scheme for designing NLP‐based adaptive support measures.
An application of this scheme to the peer‐feedback process results in exemplifying the use cases of how NLP may be employed to support each learner activity during peer‐feedback.
Implications for practice and/or policy To boost the effectiveness of their peer‐feedback scenarios, instructors and instructional designers should identify relevant leverage points, corresponding support measures, adaptation targets and automation goals based on theory and empirical findings.
Management and IT departments of higher education institutions should strive to provide digital tools based on modern NLP models and integrate them into the respective learning management systems; those tools should help in translating the automation goals requested by their instructors into prediction targets, take relevant data as input and allow for evaluating the predictions.
... At the same time, the study also shows the potential of ChatGPT as an educational tool, since false ChatGPT responses can be used to educate students, e.g., in an activity like "spot the bot" as suggested by [7]. Research about learning with errors shows that such activities are most beneficial for advanced students and only with an explicit intervention [28,29]. Upon asking at the end of the survey, some students stated that they found this small exercise to be very instructive; some even wished for more exercises of this kind. ...
This study aimed at evaluating how students perceive the linguistic quality and scientific accuracy of ChatGPT responses to physics comprehension questions. A total of 102 first- and second-year physics students were confronted with three questions of progressing affordance from introductory mechanics (rolling motion, waves, and fluid dynamics). Each question was presented with four different responses. All responses were attributed to ChatGPT, but in reality one sample solution was created by the researchers. All ChatGPT responses were wrong, imprecise, incomplete, or misleading. We found little differences in the perceived linguistic quality between ChatGPT responses and the sample solution. However, the students rated the overall scientific accuracy of the responses significantly differently, with the sample solution being rated best for the questions of low and medium affordance. The discrepancy between the sample solution and the ChatGPT responses increased with the level of self-assessed knowledge of the question content. For the question of highest affordance (fluid dynamics) that was unknown to most students, a ChatGPT response was rated just as good as the sample solution. Thus, this study provides data on the students' perception of ChatGPT responses and the factors influencing their perception. The results highlight the need for careful evaluation of ChatGPT responses both by instructors and students, particularly regarding scientific accuracy. Therefore, future research could explore the potential of similar "spot the bot"-activities in physics education to foster students' critical thinking skills.
... Other research has identified an inability of guided instructions to remediate errors emerging from student misconceptions among previously learned skills [22]; this suggests that deeply ingrained misconceptions may be more difficult to rectify over time. Other works [10,15,21] explored the use of error analysis methods by studying students' ability to identify and explain exhibited errors. These studies have explored presenting erroneous examples to students by asking them to detect and explain the error in the examples. ...
Prior work analyzing tutoring sessions provided evidence that highly effective tutors, through their interaction with students and their experience, can perceptively recognize incorrect processes or “bugs” when students incorrectly answer problems. Researchers have studied these tutoring interactions examining instructional approaches to address incorrect processes and observed that the format of the feedback can influence learning outcomes. In this work, we recognize the incorrect answers caused by these buggy processes as Common Wrong Answers (CWAs). We examine the ability of teachers and instructional designers to identify CWAs proactively. As teachers and instructional designers deeply understand the common approaches and mistakes students make when solving mathematical problems, we examine the feasibility of proactively identifying CWAs and generating Common Wrong Answer Feedback (CWAFs) as a formative feedback intervention for addressing student learning needs. As such, we analyze CWAFs in three sets of analyses. We first report on the accuracy of the CWAs predicted by the teachers and instructional designers on the problems across two activities. We then measure the effectiveness of the CWAFs using an intent-to-treat analysis. Finally, we explore the existence of personalization effects of the CWAFs for the students working on the two mathematics activities.
... Incorrect worked examples are often used in combination with correct examples. For instance, Große and Renkl (2007) studied the effects of a combination of correct and incorrect examples compared to correct examples only. They found that students with high prior knowledge benefitted more from a combination of correct and incorrect examples, whereas those with low prior knowledge benefitted from the correct examples only materials. ...
The current meta-analysis quantifies the average effect of worked examples on mathematics performance from elementary grades to postsecondary settings and to assess what moderates this effect. Though thousands of worked examples studies have been conducted to date, a corresponding meta-analysis has yet to be published. Exclusionary coding was conducted on 8033 abstracts from published and grey literature to yield a sample of high quality experimental and quasi-experimental work. This search yielded 43 articles reporting on 55 studies and 181 effect sizes. Using robust variance estimation (RVE) to account for clustered effect sizes, the average effect size of worked examples on mathematics performance outcomes was medium with g = 0.48 and p = 0.01. Moderators assessed included example type (correct vs. incorrect examples alone or in combination with correct examples), pairing with self-explanation prompts, and timing of administration (i.e., practice vs. skill acquisition). The inclusion of self-explanation prompts significantly moderated the effect of examples yielding a negative effect in comparison to worked examples conditions that did not include self-explanation prompts. Worked examples studies that used correct examples alone yielded larger effect sizes than those that used incorrect examples alone or correct examples in combination with incorrect examples. The worked examples effect yields a medium effect on mathematics outcomes whether used for practice or initial skill acquisition. Correct examples are particularly beneficial for learning overall, and pairing examples with self-explanation prompts may not be a fruitful design modification. Theoretical and practical implications are discussed.
... Correctly worked examples are an effective method for initial acquisitions of procedural knowledge (Adams et al., 2014). However, Große and Renkl (2007), in their study involving university students, suggested that introducing errors in the learning process can encourage students to reflect on what they know and help them create clear and more complete explanations of the solutions. In the example below, students are presented with an erroneous example, and asked to spot and explain the errors. ...
Boaler’s taxonomy can guide the design of mathematics questions so that they c reinforce growth mindset beliefs. Utilising Boaler's taxonomy in addition to the we established Bloom’s taxonomy to guide question setting may increase the possibi promoting growth mindset. Multiple directions for future research are described.
... Other factors may also be at play. For example, their training provided a simple feedback of correct or wrong response but not whether the chosen answer was higher or lower than the actual number, a type of feedback that is believed to improve the training effect (e.g., Fitzsimmons et al., 2021;Große & Renkl, 2007;Laski & Siegler, 2007). ...
Many cross-sectional studies have shown that number line estimation is associated with other mathematical skills, but there has been limited longitudinal research. To systematically examine such associations longitudinally at the earliest stage of mathematical learning, the current study tested 40 Chinese preschoolers (mean age = 4.97 years, SD = 0.18) and followed them up 8 months later. For both waves of data collection, children were administered six tasks: number line estimation, dot counting, comparison of two dot arrays, comparison of triple dot arrays, symbolic number comparison, and simple addition. Results of two-wave cross-lagged panel analysis showed that (a) dot counting and non-symbolic numerical comparison at Time 1 had significant longitudinal associations with number line estimation at Time 2, (b) number line estimation had bidirectional associations with symbolic number comparison, and (c) number line estimation at Time 1 had a marginally significant longitudinal association with simple addition at Time 2. These results extend the small but accumulating literature on the longitudinal relations between number line estimation and other mathematical skills and specify the important role of number line estimation in the early development of mathematical skills.
... If CS is merely provided, the students could make little progress in applying the underlying math principles and solution methods (Pillai et al., 2020). Moreover, based on expert reverse effect (Sweller, 2016) and the example-based learning theory (Große & Renkl, 2007), PK could be an essential moderator between the feedback type and results of LSE but present evidence is limited. ...
Teacher feedback may promote students learning from self-made errors (LSE), but it still needs to be refined. This study examined the effect of feedback type on the result of LSE. Three simple and five medium difficult items were identified for the LSE process. One hundred and thirty-five secondary school students received right / wrong (RW) plus 1) none, 2) knowledge prompts (KP), 3) correct solutions (CS), or 4) KP and CS, respectively, after the pretest. Students who received all the RW, KP, and CS got higher change scores on the self-reported corrected items than those who only received the RW. Moreover, the "better feedback" differed for students with different levels of prior knowledge. Potential explanations and practical implications are discussed.
... The importance of prior knowledge was shown in several studies implementing example-based learning (c.f., Stark, 1999). Große and Renkl (2007) showed that prior knowledge is also important for learning with erroneous examples because it supports effective self-explanation processes resulting in higher learning outcomes. Prior knowledge also supports the learners' understanding of what is wrong in a given situation and why it is wrong and therefore the potential of learning from errors can fully unfold (Siegler, 2002). ...
Acquiring scientific argumentation competence is a major goal of higher education study
programs, but it is common for students to use erroneous arguments often. In erroneous
arguments, scientific evidence is not adequately used to support the claim. From a conceptual change perspective, students have misconceptions of the normative use of scientific evidence for supporting claims. Thus, correcting these misconceptions is advised. These misconceptions can be overcome by comparing erroneous and correct arguments. An
instructional approach for such a comparison is learning from advocatory errors. By learning
from advocatory errors, students acquire knowledge about argumentation errors and how to
avoid them by analyzing argumentation errors made by relevant others. Learning from
advocatory errors has been shown to effectively foster students’ argumentation competence. However, previous studies have shown that learning from advocatory errors needs instructional support. In this contribution, we compare how two forms of instructional support, elaboration prompts versus testing prompts, perform. In a first condition, instructional support in form of testing prompts was given. In a second condition, elaboration prompts were given. These conditions were compared to an intervention without instructional support. The results indicate that both forms of instructional support foster argumentation competence by learning from advocatory errors. However, there were no differences in the outcome between these two forms of instructional support. Additionally, the results show no gain in argumentation competence in the intervention without instructional support. Again, the result shows that instructional support is necessary when fostering argumentation competence by learning from advocatory errors. (The research presented is published and available at https://www.frontiersin.org/articles/10.3389/feduc.2020.00126/full)
... Simply exposing students to incorrect examples may not be enough to improve learning, as students may not understand what makes the error wrong [38]. Thus, it may be necessary for students to have sufficient scaffolds when learning from erroneous examples, especially if they do not have prior in-depth knowledge [39,40]. Second, future research should include a control group that uses worked-out examples and elaboration training to compare the learning effects of the two examples (i.e., worked-out example vs. erroneous example), refute the deficiencies of the erroneous example, and further explore the advantages of the erroneous example in the field of learning. ...
Background
Constructivism theory has suggested that constructing students’ own meaning is essential to successful learning. The erroneous example can easily trigger learners’ confusion and metacognition, which may “force” students to process the learning material and construct meaning deeply. However, some learners exhibit a low level of elaboration activity and spend little time on each example. Providing instructional scaffolding and elaboration training may be an efficient method for addressing this issue. The current study conducted a randomized controlled trial to examine the effectiveness of erroneous example elaboration training on learning outcomes and the mediating effects of metacognitive load for Chinese students in medical statistics during the COVID-19 pandemic.
Methods
Ninety-one third-year undergraduate medical students were randomly assigned to the training group (n = 47) and the control group (n = 44). Prerequisite course performance and learning motivation were collected as covariates. The mid-term exam and final exam were viewed as posttest and delayed-test to make sure the robustness of the training effect. The metacognitive load was measured as a mediating variable to explain the relationship between the training and academic performance.
Results
The training significantly improved both posttest and delayed-test performance compared with no training (Fposttest = 26.65, p < 0.001, Partial η² = 0.23; Fdelayed test = 38.03, p < 0.001, Partial η² = 0.30). The variation trend in metacognitive load in the two groups was significantly different (F = 2.24, p < 0.05, partial η² = 0.20), but metacognitive load could not explain the positive association between the treatment and academic performance (β = − 0.06, se = 0.24, 95% CI − 0.57 to 0.43).
Conclusions
Erroneous example learning and metacognitive demonstrations are effective for academic performance in the domain of medical statistics, but their underlying mechanism merits further study.
... They found that participants who observed the correct-only model performed better on the subsequent behavioral task than participants who observed the correct and incorrect model. Similarly, in a study that used both correct and incorrect worked examples to teach mathematics, Grobe & Renkl (2007) showed that students with more prior knowledge learned better from both correct and incorrect examples, whereas students with low prior knowledge benefitted only from correct examples. Most of the students in our sample had low prior knowledge for statistics and R programming, thus, it is possible they would have benefited more from a correct model than a model that demonstrated correct and incorrect responses (coping model). ...
... They found that participants who observed the correct-only model performed better on the subsequent behavioral task than participants who observed the correct and incorrect model. Similarly, in a study that used both correct and incorrect worked examples to teach mathematics, Grobe & Renkl (2007) showed that students with more prior knowledge learned better from both correct and incorrect examples, whereas students with low prior knowledge benefitted only from correct examples. Most of the students in our sample had low prior knowledge for statistics and R programming, thus, it is possible they would have benefited more from a correct model than a model that demonstrated correct and incorrect responses (coping model). ...
In this study, we examined the effect of video-based feedback designed to highlight a peer engaging in effective thinking processes on self-efficacy beliefs and learning outcomes (performance on a delayed quiz). Students in an introductory statistics course participated in an online learning activity where they received feedback in one of three randomly assigned conditions: a video of a peer demonstrating the process of arriving at a correct answer (mastery condition), a peer making mistakes then self-correcting those errors before arriving at a correct answer (coping condition), or a screenshot of a peer’s correct worked example (as a control). Results indicated that students who watched the mastery videos, but not the coping videos, rated their self-efficacy higher and scored higher on a class quiz taken more than a day after the feedback intervention than students who viewed a worked example. However, students in the two video conditions did not significantly differ in terms of either self-efficacy and quiz performance. The results of this study, although modest in scope, illustrate how the design of feedback could lead to noticeable differences in student learning.
... For example, Barbieri and Booth (2020) demonstrated that exposure to errors improves algebraic equation solving. As the study of Große and Renkl (2007) showed an advantage of learning with erroneous worked examples only for learners with a high level of prior knowledge, the two fictitious students in our worked examples ultimately solve the problem adequately. ...
This study examined whether learning with heuristic worked examples can improve students’ competency in solving reality-based tasks in mathematics (mathematical modeling competency). We randomly assigned 134 students in Grade 5 and 180 students in Grade 7 to one of three conditions: control condition (students worked on reality-based tasks), worked example condition (students studied worked examples representing a realistic process of problem-solving by fictitious students negotiating solutions to the tasks), and prompted worked example condition (students additionally received self-explanation prompts). In all three conditions, the students worked on the tasks individually and independently for 45 min. Dependent measures were mathematical modeling competency (number of adequate solution steps and strategies) and modeling-specific strategy knowledge. Results showed that although strategy knowledge could be improved through the intervention for fifth and seventh graders, modeling competency was improved only for seventh graders. The prompting of self-explanations had no additional effect for either fifth or seventh graders.
... The concept 'learning from errors' or LFE has been successfully applied in mathematics [Brodie 2014], physics [Große and Renkl 2007] and CS (especially in programming) [Shah et al. 2017]. A recent study by Metcalfe [2017] showed that errorful learning followed by constructive feedback is a vehicle to achieving purposeful learning. ...
Structured Query Language (SQL) remains the standard language used in Relational Database Management Systems (RDBMSs) and has found applications in healthcare (patient registries), businesses (inventories, trend analysis), military, education, etc. Although SQL statements are English-like, the process of writing SQL queries is often problematic for nontechnical end-users in the industry. Similarly, formulating and comprehending written queries can be confusing, especially for undergraduate students. One of the pivotal reasons given for these difficulties lies with the simple syntax of SQL, which is often misleading and hard to understand. An ideal solution is to present these two audiences: undergraduate students and nontechnical end-users with learning and practice tools. These tools are mostly electronic and can be used to aid their understanding, as well as enable them to write correct SQL queries. This work proposes a new approach aimed at understanding and writing correct SQL queries using principles from Formal Language and Automata Theory. We present algorithms based on: regular expressions for the recognition of simple query constructs, context-free grammars for the recognition of nested queries, and a jumping finite automaton for the synthesis of SQL queries from natural language descriptions. As proof of concept, these algorithms were further implemented into interactive software tools aimed at improving SQL comprehension. Evaluation of these tools showed that the majority of participants agreed that the tools were intuitive and aided their understanding of SQL queries. These tools should, therefore, find applications in aiding SQL comprehension at higher learning institutions and assist in the writing of correct queries in data-centered industries.
... Correctly worked examples are an effective method for initial acquisitions of procedural knowledge (Adams et al., 2014). However, Große and Renkl (2007), in their study involving university students, suggested that introducing errors in the learning process can encourage students to reflect on what they know and help them create clear and more complete explanations of the solutions. In the example below, students are presented with an erroneous example, and asked to spot and explain the errors. ...
CONTEXT Dropout from engineering studies at tertiary level remains a persistent global problem. The social psychology theory of mindset explains how behaviour necessary for successful engagement with challenging academic content can be derailed by beliefs about intelligence as fixed-at-birth rather than growth mindset beliefs that intelligence can always be further developed. Given the complexity of research involving humans and the early stage of mindset research in tertiary settings, it is not surprising that the results of a recent systematic literature review on growth mindset interventions in engineering education did not identify a leading intervention. However, the review suggested that growth mindset interventions should address the broader education context and not only individual students. PURPOSE OR GOAL Of all subjects, mathematics is one where fixed mindset beliefs are more frequently seen in the general population. High performing students may be at risk from the negative effects of a fixed mindset when they encounter new challenges at university. This research explores the potential of creating growth or fixed mindsets through the words used in mathematics questions. Examples from mathematics assessment tasks will be analysed to see how they align with mindset principles described in a taxonomy by Boaler (2015). APPROACH OR METHODOLOGY/METHODS A modified version of the Delphi Technique was used to reach consensus on the applicability of Boaler's taxonomy to undergraduate mathematics courses. Questions from past assessments from first-year mathematics courses were compiled, based on their potential to match the categories in Boaler's taxonomy. In six meetings over three months, all three authors discussed and classified the selected questions into the categories from Boaler's taxonomy. Where questions did not fit, modifications were brainstormed to see if modified questions could align with one or more categories from the taxonomy. ACTUAL OR ANTICIPATED OUTCOMES Examples matching all categories of Boaler's taxonomy are presented and contrasted with non-examples on the same mathematics topics. CONCLUSIONS/RECOMMENDATIONS/SUMMARY Boaler's taxonomy can guide the design of mathematics questions so that they can also reinforce growth mindset beliefs. Utilising Boaler's taxonomy in addition to the well-established Bloom's taxonomy to guide question setting may increase the possibility of promoting growth mindset. Multiple directions for future research are described.
... However, the relation between prior knowledge and the use of worked examples varies by the worked example format. For example, when worked examples contain errors, only students with sufficient prior knowledge benefit from them (Große & Renkl, 2007 (Sweller, 2020;Sweller et al., 2019). For instance, students who viewed on-screen text that duplicated an audio explanation of how lightning forms scored lower on retention and knowledge transfer compared to their counterparts who did not view extra on-screen text, likely due to cognitive overload (Mayer et al., 2001). ...
Worked examples are effective learning tools for algebraic equation‐solving. However, they are typically presented in a static concise format which only displays the major derivation steps in one static image. The current work explores how worked examples that vary in their extensiveness (i.e., detail) and degree of dynamic presentation (i.e., static vs. sequential line‐by‐line vs. dynamic format that demonstrates the problem‐solving process) impact learning. We conducted an online experiment in which 230 algebra students completed a pretest, studied worked examples in one of six presentation conditions, and completed a posttest. We found that overall, students improved from pretest to posttest after viewing the worked examples; we did not find significant differences on posttest performance between worked example presentations. These results have implications for the design of worked examples in online tutoring systems as well as for cognitive load theory and perceptual learning theory in the design of worked examples. This article is protected by copyright. All rights reserved.
... Another open question pertains to the extent that the efficacy of deliberate erring is moderated by student characteristics such as one's level of expertise or prior knowledge. For instance, whereas learners with less domain knowledge have been found to benefit from studying only correct solutions via worked examples, these appear to be redundant for learners with greater domain knowledge, who benefit more from studying both incorrect and correct solutions (Große & Renkl, 2007). In view that students in our study benefited from deliberate erring even when they possessed relatively low self-reported prior knowledge of the studied material, the derring effect may well be amplified for students with high prior knowledge who may, in turn, be better equipped to relate their existing knowledge to the studied material when formulating plausible incorrect responses. ...
Our civilization recognizes that errors can be valuable learning opportunities, but for decades, they have widely been avoided or, at best, allowed to occur as serendipitous accidents. The present research tested whether greater learning success could paradoxically be achieved through making errors by intentional design, relative to traditional errorless learning methods. We show that deliberately committing and correcting errors even when one knows the correct answers enhances learning—a counterintuitive phenomenon that we termed the derring effect. Across two experiments (N = 160), learners engaged in open-book study of scientific expository texts and were then tested on their retention and higher-order application of the material to analyze a novel news event. Deliberate error commission and correction during study produced not only better recall performance, but also superior knowledge application compared to two errorless study techniques that are popular among students and educational researchers: copying with underlining, and elaborative studying with concept mapping (Experiment 1). These learning benefits persisted even over generating alternative conceptually correct answers, revealing that the derring effect is not merely attributable to generation or elaboration alone, but is unique to producing incorrect responses (Experiment 2). Yet, learners were largely unaware of these advantages even after experiencing them. Our results suggest that avoiding errors in learning may not always be most optimal. Rather, deliberate erring is a powerful strategy to enhance meaningful learning. We discuss implications for educational practice in redesigning conventional approaches to errors: To err is human; to deliberately err is divine.
... Accordingly, one potential avenue for future work is to examine the extent that deliberately committing and correcting procedural errors enables learners to solve problems more effectively and efficiently, while improving their ability to apply learned procedures to solve novel problems. Notably, in the domain of probability problem-solving, learning with worked examples containing incorrect solution steps rather than only correct ones has been found to enhance far transfer to novel test problems with dissimilar surface and structural features, particularly for learners with high prior knowledge (Große & Renkl, 2007). It will thus be worthwhile to consider how deliberate erring can be integrated with such methods of instruction, in conjunction with examining potential interactions with learner characteristics. ...
How can we strategically and systematically learn from our errors? Over their long history, errors have traditionally been prevented entirely or, at best, permitted to occur spontaneously. Across three experiments, we tested and found evidence for a counterintuitive phenomenon that we termed the derring effect—deliberately committing errors even when one already knows the correct answers produces superior learning than avoiding them, particularly when one's errors are corrected. Learners engaged in an educationally relevant task of learning scientific term-definition concepts via open-book study by deliberately generating conceptually incorrect definitions with or without correction, or copying and underlining them (Experiment 1). On a cued recall test, deliberate erring outperformed errorless copying, with error correction yielding an additional benefit. This advantage of deliberate erring persisted over actively generating alternative conceptually correct answers (Experiment 2), which in turn surpassed copying. Even when errorless generation was given a further boost to involve a higher degree of elaboration by prompting learners to generate a specific real-world example that illustrated or applied each concept, deliberately committing and correcting errors still produced better learning (Experiment 3). Altogether, the derring effect is neither fully attributable to a generation nor an elaboration benefit, but stems at least in part from enhanced target processing specific to having first deliberately produced incorrect responses. Notwithstanding deliberate erring's prowess, learners were largely oblivious to its benefits, misjudging the strategy as less effective. Both theoretical and educational implications of positioning errors as active, systematic, and intentional events in learning are discussed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
... Examples with errors (i.e., incorrect worked examples) support students to use more self-explanations when interpreting the concept, identifying errors, and correcting bugs (Zhi et al., 2018). Zhi and colleagues (2018) found that analyzing incorrect examples can effectively support older students' learning about loops; however, analyzing a mixture of correct and incorrect worked examples can be helpful for students who have sufficient prior knowledge (Große & Renkl, 2007). ...
We conducted an in-depth analysis to better understand the role of playing versus analyzing worked examples when learning programming commands. Our findings, focused on two pairs of third graders, demonstrated that students did not use complex programming commands when only playing; whereas, when supported through analysis of worked examples, they did use subroutines. Both pairs started to identify repeating patterns in their code once they had a worked example where a subroutine was used to repeat a set of commands. Yet, having time to play before analyzing worked examples in programming contexts may provide subtle benefits, especially for more abstract commands such as subroutines. Unlike prior studies, our findings suggest that looping concepts can be taught within the concept of subroutines, even with young students.
In this study, effects of asking participants to make different types of considerations when solving a realistic word problem were investigated. A two-factorial experiment with the factors “addressing realistic considerations” (with vs. without) and “addressing mathematical operations” (with vs. without) was conducted. It was assumed that reality-based considerations would lead to reality-based problem-solving strategies, thus fostering real-life solutions, while considering mathematical operations would lead to problem-solving strategies usually promoted in school, which were expected to impair realistic solutions. Analyses are based on N = 165 participants. The results showed that being asked to make reality-based considerations did not significantly affect realistic solutions ( F (1, 161) = 2.43, p = 0.121, η p ² = 0.015), while being asked to consider appropriate mathematical operations significantly impaired realistic solutions ( F (1, 161) = 8.54, p = 0.004, η p ² = 0.050). These findings suggest that inducing typical school problem-solving strategies may be detrimental when it comes to solving mathematical problems in a realistic way.
El artículo tiene como objetivo determinar el rendimiento académico y errores en la resolución de tipos de problemas de aplicación de la función cuadrática, de estudiantes de enseñanza media de la Región de Los Lagos y de la Región de Los Ríos en Chile. El enfoque es cualitativo descriptivo con estudio de casos. Fueron elaboradas y aplicadas una prueba de matemática con problemas de respuesta abierta y un cuestionario de opinión. A través de los resultados, se evidencia el mayor rendimiento académico en los problemas rutinarios de contexto puramente matemático y de contexto fantasista, pero con dificultad en la resolución de problemas no rutinarios. Además, los errores con origen en las actitudes afectivas y emocionales asociados a bloqueos al iniciar la resolución, olvido a la hora de plantear la función cuadrática, prevalecen por sobre los errores con origen en un obstáculo y errores con origen en la ausencia del sentido.
Productive Failure is an instructional approach in which students engage in problem‐solving attempts prior to instruction. The Productive Failure approach has been found to be more supportive for conceptual knowledge acquisition during subsequent instruction compared to a direct‐instruction condition in which students receive instruction first followed by problem solving. Failing at problem‐solving attempts prior to instruction is considered to prepare students to better understand the concepts underlying the problem. However, it is an open question whether students need to fail themselves to be effectively prepared for subsequent instruction. We therefore compared the effectiveness of two alternative preparatory activities: Students either engaged in problem‐solving attempts themselves or were shown problem‐solving attempts of another student. This experimental variation allows to show the necessity of own failure for the effectiveness of the PF approach. The results indicated that the two conditions were equally effective at preparing students for subsequent instruction. This article is protected by copyright. All rights reserved.
This article reviews literature on worked examples in the context of programming activities. We focus on two types of examples, namely code-tracing and code-generation examples, because there is sufficient research on these to warrant a review. We synthesize key results according to themes that emerged from the review. This synthesis aims to provide practical guidance for educators and shed light on future research opportunities. While there is established work in some areas (e.g., dynamic code-tracing examples in the form of program visualization tools, utility of subgoals in code-generation examples, incomplete examples in the form of Parsons puzzles), there are also gaps. Thus, the article concludes with directions for future work on examples in computer science education.
Troubleshooting activities require students to diagnose teacher-crafted erroneous examples by detecting and explaining the conceptual errors driving them. In a previous study, the author tested whether diagnosing erroneous examples and then scoring them using a rubric that contained the related worked examples, a step-by-step strategy to solve a problem, would help 8th graders studying simple electric circuits detect and learn from the conceptual errors in the erroneous examples. The findings showed that this graded activity was more effective than the traditional activity with the same worked examples but without rubrics. To determine whether older students with greater metacognitive skills still need the support provided by the graded activity, five 10th grade advanced physics classes completed a pretest/intervention/posttest after finishing a unit on geometric optics. Students in each class were randomly assigned to the graded (69 students) or the traditional activity (73 students). Students assigned to the graded activity detected and learned more from the conceptual errors in the erroneous examples than students administered the traditional activity. Thus, the findings of the current and the previous studies suggest the greater effectiveness of the graded activity as compared to the traditional activity, regardless of the students’ level of metacognitive awareness or topic.
Introduction
Exemplars are an effective strategy for initial cognitive and psychomotor skills acquisition and promoting students’ self-regulation. Yet, there is limited evidence about students’ use and perception of exemplars in dental education. Therefore, this study aims to develop a scale to measure students’ perception of exemplars in an Australian Dental School.
Materials and methods
Phase I assessed the relevant literature and students’ responses to an interview. Four themes were identified for developing a 19 items questionnaire rated by an expert panel. Phase II piloted the questionnaire amongst a group of 30 students. Phase III analysed psychometric and qualitative open-ended questions data obtained from a large group of students.
Results
The principal axis factoring resulted in one single factor that explained over 62% of the variance and had an alpha of .88. The number of questions were reduced from 19 to 6 items with loadings of 0.72 or above for each item. Students reported exemplars are beneficial for their learning and to visually guide them to understand procedures and their intended outcomes. This prompts them to think and mentally prepare before performing. Students further reported using exemplars provided by their courses almost as often as exemplars they search in social media.
Conclusion
Well developed exemplars could play a key role in enhancing learning and could assist teaching. Understanding students’ use and perception of exemplars will help educators to develop and evaluate appealing exemplars. Therefore, this study developed a new instrument to measure students’ perception of exemplars, verified its reliability and validity.
Bu araştırmanın amacı, hata temelli aktivitelerin 7. Sınıf veri analizi alt öğrenme alanına ait çizgi grafiği, ortalama, medyan ve mod kavramlarını öğrenmeye etkisini belirlemektir. Araştırma hatalı örneklerle istatistik kavramlarının öğretilmesine dayalı yarı deneysel bir çalışma modelidir. Çalışmanın katılımcıları 2018-2019 öğretim yılında 7. sınıfta öğrenim gören rastgele seçilmiş 70 öğrenciden oluşmaktadır. Öğrenim sürecinde hatalı çözülmüş örnekler içeren sekiz çalışma kağıdı kullanılmış, başarı testi hazırlanarak ön- test ve son- test olarak uygulanmıştır. Elde edilen veriler SPSS paket programı kullanılarak analiz edilmiştir. Etkinliklerin etkisini incelemek amacıyla t-testi sonuçlarına yer verilmiştir. Analiz sonuçları, hata temelli aktivitelerin çizgi grafiği, ortalama, medyan ve mod kavramlarının öğrenilmesine olumlu etki yaptığını göstermiştir.
The present study examines the effectiveness of incorporating worked examples with prompts for self-explanation into a middle school math textbook. Algebra 1 students (N = 75) completed an equation-solving unit with textbooks either containing the original practice problems or in which a portion of those problems were converted into a combination of correct, incorrect, and incomplete examples. Students completed pre- and posttest measures of algebraic feature knowledge, equation-solving skills, and error anticipation.
Example-based textbook assignments increased students’ equation-solving skills and their ability to anticipate errors one might make when solving problems. Differences in students’ anticipation of various types of errors are also examined. Error anticipation, a particular form of negative knowledge, is a potentially important skill that relates to algebraic feature knowledge and equation-solving skills.
This study examined the effects of activating prior knowledge and refutation text structure on students’ comprehension of counterintuitive science material. Developmental studies students, who also were incompetent readers, either read a passage that referred directly to their known misconceptions about a science topic and then refuted those misconceptions, or they read a passage that only described the topic. Text structure was crossed with three levels of prior knowledge activation (augmented activation, activation only, and no activation). We found statistically significant differences that favored augmented activation over activation only and no activation over activation only. Additionally, the students favored refutation text over nonrefutation text. Implications are drawn for future research and developmental studies instruction.
Cognitive load theory (CLT) originated in the 1980s and underwent substantial development and expansion in the 1990s by researchers from around the globe. As the articles in this special issue demonstrate, it is a major theory providing a framework for investigations into cognitive processes and instructional design. By simultaneously considering the structure of information and the cognitive architecture that allows learners to process that information, cognitive load theorists have been able to generate a unique variety of new and sometimes counterintuitive instructional designs and procedures.
Microdevelopment is the process of change in abilities, knowledge and understanding during short time-spans. This book presents a new process-orientated view of development and learning based on recent innovations in psychology research. Instead of characterising abilities at different ages, researchers investigate processes of development and learning that evolve through time and explain what enables progress in them. Four themes are highlighted: variability, mechanisms that create transitions to higher levels of knowledge, interrelations between changes in the short-term scale of microdevelopment and the crucial effect of context. Learning and development are analysed in and out of school, in the individual's activities and through social interaction, in relation to simple and complex problems and in everyday behaviour and novel tasks. With contributions from the foremost researchers in the field Microdevelopment will be essential reading for all interested in cognitive and developmental science.
Notes that previous research has shown that when an experimenter or a tutor prompts students to self-explain orally, generating such self-explanations is effective for learning. If self-explanations are readily produced by prompting, then it would be trivial to implement an automated prompting system using a computer interface. In an attempt to replicate previous research using a human prompter with spoken self-explanations, two experiments were designed using a computer prompter with typed self-explanations. The first experiment ( N=20) tested the effectiveness of spontaneously typed self-explaining while using a computer interface without prompting. The results show that the amount of self-explaining was low given the amount observed in past research. Typing seems to have caused the students to paraphrase the materials instead. The second experiment( N=20) tested the effectiveness of an automatic computer prompter, as compared to a human prompter using the same interface. Automatic prompting was just as effective as human prompting, and prompting did increase the amount of typed self-explanations and learning. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
In Experiment 1, students who studied example word problems that were grouped by
t test, correlation, and chi-square were more likely to sort subsequent problems on the basis of structure and less likely to sort on the basis of surface characteristics than students who received no examples. In Experiment 2, this pattern was strongest when students studied structure-emphasizing rather than surface-emphasizing examples. In Experiment 3, students who studied and practiced 4 structure-emphasizing worked-out examples of
t test and correlation problems were more likely to apply the appropriate statistical test correctly to subsequently presented statistics word problems than students who had studied surface emphasizing examples. This pattern was strong for lower but not for higher ability students. Implications of a schema construction theory are discussed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
The video survey is a promising new approach for studying classrooms and teaching across cul-tures. Drawing from experience in working with two cross-cultural video surveys, the Third In-ternational Mathematics and Science Study (TIMSS) and its follow-up study (TIMSS-R), this article presents some of the challenges of studying classrooms across cultures and some of the ways that the video survey can deal with these challenges. The article begins by identifying some of the issues in cross-cultural research and classroom surveys that led to the creation of the video survey. Examples from the TIMSS and the TIMSS-R video studies are then used to illus-trate some benefits and limitations of video surveys and to share some of the lessons that were learned about studying classrooms across cultures. STIGLER, GALLIMORE, HIEBERT Studying classrooms and teaching across cultures is fraught with challenges, yet ripe with opportunity. First, this article considers the opportunities afforded by cross-cultural studies of teaching. Next, we lay out what we see as some major meth-odological challenges posed by such research and outline a new method that we are developing to address some of these challenges. This new method, which we call the video survey, evolved over the past 5 years in our work on the Third Interna-tional Mathematics and Science Study (TIMSS) and its fol-low-up study (TIMSS-R; the R stands for repeat). Using examples from these studies (the TIMSS-R is still in progress), we identify some advantages and disadvantages of video sur-veys and share some lessons we have learned about studying classrooms across cultures.
Traditionally, Cognitive Load Theory (CLT) has focused on instructional methods to decrease extraneous cognitive load so that available cognitive resources can be fully devoted to learning. This article strengthens the cognitive base of CLT by linking cognitive processes to the processes used by biological evolution. The article discusses recent developments in CLT related to the current view in instructional design that real-life tasks should be the driving force for complex learning. First, the complexity, or intrinsic cognitive load, of such tasks is often high so that new methods are needed to manage cognitive load. Second, complex learning is a lengthy process requiring learners motivational states and levels of expertise development to be taken into account. Third, this perspective requires more advanced methods to measure expertise and cognitive load so that instruction can be flexibly adapted to individual learners needs. Experimental studies are reviewed to illustrate these recent developments. Guidelines for future research are provided.
In order to facilitate the transition fromlearning from worked examples in earlier stagesof skill acquisition to problem solving inlater stages, it is effective to successivelyfade out worked solution steps – in comparisonto the traditional method of employingexample-problem pairs that is frequently usedin cognitive-load research. In the presentstudies, the learning processes and mechanismsthat occur when learning in a computer-basedlearning environment containing faded workedsolution steps were examined across twoexperiments. The first experiment showed thatthe position of the faded steps did notinfluence learning outcomes; instead,individuals learned most about those principlesthat were faded. This suggested that specificself-explanation activities are triggered byfaded steps. The second experiment investigatedthis hypothesis directly by collecting andanalyzing thinking-aloud protocols generated bythe learners during their interaction with thelearning environment. No effect on productivelearning events including self-explanationscould be found. It could, however, be shownthat fading is associated with fewerunproductive learning events and, thereby,lends itself to better learning outcomes.
The representation of physics problems in relation to the organization of physics knowledge is investigated in experts and novices. Four experiments examine (a) the existence of problem categories as a basis for representation; (b) differences in the categories used by experts and novices; (c) differences in the knowledge associated with the categories; and (d) features in the problems that contribute to problem categorization and representation. Results from sorting tasks and protocols reveal that experts and novices begin their problem representations with specifiably different problem categories, and completion of the representations depends on the knowledge associated with the categories. For, the experts initially abstract physics principles to approach and solve a problem representation, whereas novices base their representation and approaches on the problem's literal features.
Abstract
Cognitive load theory has been designed to provide guidelines intended to assist in the presentation of information in a manner that encourages learner activities that optimize intellectual performance. The theory assumes a limited capacity working memory that includes partially independent subcomponents to deal with auditory/verbal material and visual/2- or 3-dimensional information as well as an effectively unlimited long-term memory, holding schemas that vary in their degree of automation. These structures and functions of human cognitive architecture have been used to design a variety of novel instructional procedures based on the assumption that working memory load should be reduced and schema construction encouraged. This paper reviews the theory and the instructional designs generated by it.
When new information is presented to learners, it must be processed in a severely limited working memory. Learning reduces working memory limitations by enabling the use of schemas, stored in long-term memory, to process information more efficiently. Several instructional techniques have been designed to facilitate schema construction and automation by reducing working memory load. Recently, however, strong evidence has emerged that the effectiveness of these techniques depends very much on levels of learner expertise. Instructional techniques that are highly effective with inexperienced learners can lose their effectiveness and even have negative consequences when used with more experienced learners. We call this phenomenon the expertise reversal effect. In this article, we review the empirical literature on the interaction between instructional techniques and levels of learner experience that led to the identification of the expertise reversal effect.
Complex learning aims at the integration of knowledge, skills, and attitudes; the coordination of qualitatively different constituent skills; and the transfer of what is learned to daily life or work settings. Recent instructional theories stress authentic learning tasks as the driving force for learning; but due to the complexity of those tasks, learning may be hampered by the limited processing capacity of the human mind. In this article we present a framework for scaffolding practice and just-in-time information presentation, aiming to control cognitive load effectively. We briefly describe a design model for complex learning consistent with cognitive load theory. Theoretical and practical implications of the presented framework are discussed.
Individuelle und schulische Bedingungsfaktoren für Argumentationen und Beweise im Mathematikunterricht / K. Reiss ... - In: Bildungsqualität von Schule / hrsg. von Manfred Prenzel ... - Weinheim u.a. : Beltz, 2002. - S. 51-64 S. - (Zeitschrift für Pädagogik : Beiheft ; 45)
This chapter first provides an overview of the various stages of cognitive skill acquisition in order to situate learning from worked-out examples in the course of learning a new skill. It then defines more precisely the meaning of learning from worked-out examples, provides reasons for its effectiveness, and discusses the relevance of self-explanations and instructional explanations as factors moderating the effectiveness of studying examples. It discusses the problem of structuring the transition from studying examples to problem solving. It describes a preliminary instructional model of example-based skill acquisition that currently guides our research. Finally, it outlines a set of research questions intended to guide future research, such as adapting the proposed fading procedure to individual learners. © 2007 by Frank E. Ritter, Josef Nerb, Erno Lehtinen, and Timothy M. O'Shea. All rights reserved.
Various strategies such as self-explanation (Chi, 2000), collaborative problem solving (Ellis, Klahr, & Siegler, 1993), scaffolding (Vygotsky, 1978), reciprocal teaching (Brown & Palinscar, 1989), and learning from worked-out examples (Mwangi & Sweller, 1998), have been used successfully to facilitate learning and understanding. Psychologists are particularly interested in the cognitive processes underlying and affected by these methods, the varying effectiveness of each across different domains, and the mechanisms that are associated with the learning that results from the utilization of each. Although these techniques are different in form, each one encourages the student to engage in learning during which knowledge is actively processed, and mental models and schema are constructed and reconstructed. The goal of this study was to extend our knowledge of the mechanisms by which students acquire knowledge and the strategies that could be used to facilitate these processes. The effects of feedback and self-explanation have been examined under various conditions, and within various domains (Chi, de Leeuw, Chiu, & LaVancher, 1994; Mwangi & Sweller, 1998; Tudge, Winterhoff, & Hogan, 1996). Because both have shown to have advantageous effects under many circumstances, they were used together in this study of algebra problem solving. To extend prior research, both the self-explanation of correct and incorrect solutions was elicited and compared to the condition in which only the correct answer was self-explained. It was hypothesized that students who received feedback and were asked to explain both correct and incorrect solutions would demonstrate the most improvement in solving algebra word problems. Method Participants included 80 college students (60 females, mean age = 19.73 years, SD = 2.05), including 50 Caucasians, 12 African Americans, 10 Hispanics, 6 Asians, and 2 "Others". An algebra pretest consisting of 14 multiple-choice compare word problems (4 simple-direct, 5 simple-indirect, and 5 complex) was used to assess algebra problem-solving abilities. Participants then participated in a directed practice session during which they were randomly assigned to one of four experimental conditions (No feedback/"Explain own" (Control), Ambiguous feedback/"Explain own and alternative", Feedback/Explain correct, and Feedback/Explain correct and incorrect"). Students were asked to provide algebraic equations for each of 10 problems, and to explain why they thought these equations were correct (or incorrect). Finally, an algebra post-test, identical in form to the pretest, was administered.
Zusammenfassung: Die Effektivität des Lernens aus Lösungsbeispielen im Vergleich zum Lernen durch Problemlösen wird derzeit mit der Cognitive Load-Theorie erklärt: Beim Lernen aus Lösungsbeispielen wird das Arbeitsgedächtnis weniger belastet, daher bleibt mehr Raum für Lern- und Verstehensprozesse. Um diese These direkt experimentell zu überprüfen, wurde das dual task-Paradigma eingesetzt. 80 Studierende der Geistes- und Sozialwissenschaften wurden auf die Zellen eines 2 × 2-faktoriellen Designs (Faktor 1: Lernen aus Lösungsbeispielen vs. Lernen durch Problemlösen; Faktor 2: mit vs. ohne Zweitaufgabe) verteilt. Der Lernstoff war Wahrscheinlichkeitsrechnung. Der Lernerfolg wurde mit einem Nachtest erfasst. Das Befundmuster hinsichtlich des Lernerfolgs und der Reaktionszeiten auf eine Zweitaufgabe in den vier experimentellen Gruppen stimmte mit den aus der Cognitive-Load-Theorie abgeleiteten Hypothesen überein.
An instructional design theory for concept teaching is presented. The theory is based on direct empirical validation from a programmatic line of instructional systems research. Concept learning is viewed as a two-phase process: (a) formation of conceptual knowledge and (b) development of procedural knowledge. Two fundamental components of the proposed theoretical model are content structure variables and instructional design variables. A rational combination of these components, based on a content analysis that takes into account the learning model, provides the means for the selection of one of four basic instructional design strategies. Research studies that contribute to the model are reviewed, and the model is described with reference to instructional methods and cognitive processes.
The knowledge required to solve algebra manipulation problems and procedures designed to hasten knowledge acquisition were studied in a series of five experiments. It was hypothesized that, as occurs in other domains, algebra problem-solving skill requires a large number of schemas and that schema acquisition is retarded by conventional problem-solving search techniques. Experiment 1, using Year 9, Year 11, and university mathematics students, found that the more experienced students had a better cognitive representation of algebraic equations than less experienced students as measured by their ability to (a) recall equations, and (b) distinguish between perceptually similar equations on the basis of solution mode. Experiments 2 through 5 studied the use of worked examples as a means of facilitating the acquisition of knowledge needed for effective problem solving. It was found that not only did worked examples, as expected, require considerably less time to process than conventional problems, but that subsequent problems similar to the initial ones also were solved more rapidly. Furthermore, decreased solution time was accompanied by a decrease in the number of mathematical errors. Both of these findings were specific to problems identical in structure to the initial ones. It was concluded that for novice problem solvers, general algebra rules are reflected in only a limited number of schemas. Abstraction of general rules from schemas may occur only with considerable practice and exposure to a wider range of schemas.
We investigated whether the findings from worked-out example research on the effects of self-explanation prompts and on instructional explanations can be generalized to other example types – in this case: solved example problems. Whereas worked-out examples consist of a problem formulation, solution steps, and the final solution, solved example problems merely provide the problem formulation and the solution. We employed a first module of a computer-based environment for student teachers presenting solved example problems from which they learned how to select and to design worked-out examples for high school students. The participants (47 student teachers for German low- and medium-track schools and 33 student teachers for high-track schools) were randomly assigned to the cells of our 2 × 2 design (with vs. without self-explanation prompts; with vs. without instructional explanations). The effects of the different program versions on objective and subjective learning outcomes, on the perceived helpfulness of the program, and on the learning time were analyzed. As learning process data, the written and spoken self-explanations were assessed. The following main results were obtained: Particularly self-explanation prompts had favorable effects on learning outcomes, whereas instructional explanations can reduce the student teacher’s self-explanation activities and thereby the learning outcomes. Whereas “objectively” the most favorable learning outcomes were obtained when only self-explanation prompts were employed, the student teachers perceived their learning outcome best when only instructional explanations were provided. It can be concluded that the status of self-explanation prompts and of instructional explanations is comparable irrespective of the example type presented for learning.
The present study investigated the effects of text structure in the acquisition of the concept of energy and the overcoming of specific preconceptions associated with it. Cypriot sixth-grade students read either a simple expository text that presented factual information or a refutation text that also explicitly addressed two common preconceptions and proceeded to refute them. Both texts were used as adjuncts to the standard science instruction that is typically provided in the Cypriot elementary school. Students who read the refutation text outperformed students who read the expository text and students who received standard instruction only. In contrast, the influence of the expository text was negligible and generally comparable to that of standard instruction. The implications of these results for the instructional use of refutational text structures were discussed.
Already at a very young age, children experience the wide applicability and intrinsic simplicity of linear/proportional relations. In primary and secondary school mathematics education, moreover, extensive attention is paid to this type of relations. In the long run, students develop the misbelief that each relation can be quantified as proportional, called the “illusion of linearity”.The best-known misconception originating from such a “synthetic model of linearity” is that if a geometrical figure enlarges k times, its area and/or volume become k times larger too. This article reports and discusses a teaching experiment aimed at remedying this misconception in 8th graders. Ten experimental lessons were developed in order to obtain a conceptual change in these students. The learning results were tested by means of a pretest–post-test–retention test design with an experimental and control group.The problem-solving behaviour of control group students did not change. In the experimental group, the intervention was successful: students’ automatic use of proportional strategies for solving non-proportional geometry problems drastically decreased. Never theless, some students continued to reason proportionally for all types of problems, while others suddenly started to apply non-proportional strategies to proportional problems too. The linearity illusion was broken in most students, but this did not always result in a deep conceptual understanding of proportional and non-proportional situations and relations.
The present paper analyzes the self-generated explanations (from talk-aloud protocols) that “Good” and “Poor” students produce while studying worked-out examples of mechanics problems, and their subsequent reliance on examples during problem solving. We find that “Good” students learn with understanding: They generate many explanations which refine and expand the conditions for the action parts of the example solutions, and relate these actions to principles in the text. These self-explanations are guided by accurate monitoring of their own understanding and misunderstanding. Such learning results in example-independent knowledge and in a better understanding of the principles presented in the text. “Poor” students do not generate sufficient self-explanations, monitor their learning inaccurately, and subsequently rely heavily on examples. We then discuss the role of self-explanations in facilitating problem solving, as well as the adequacy of current AI models of explanation-based learning to account for these psychological findings.
Worked examples are an effective instructional means to teach complex problem-solving skills. It has been argued that worked examples decrease extraneous load, enabling more Working Memory (WM) resources to be directed to activities that facilitate learning and transfer performance. Hence, cognitive load research has started to shift its focus towards finding instructional techniques that impose a germane cognitive load by stimulating the allocation of WM resources to such activities. This special issue provides an overview of recent experimental research on ways to further optimise the design and delivery of worked examples in order to foster learning and transfer.
Acquiring a complex cognitive skill often involves learning principles of the task domain in the midst of solving problems or studying examples. Cascade is a model of such learning. It includes both rule-based reasoning and several kinds of analogical, case-based reasoning. Task domain principles are represented as rules, and Cascade learns new rules at rule-learning events, which are initiated by an impasse and utilize multiple kinds of reasoning. In this article, I evaluate Cascade's model of rule-learning events by analyzing ones gleaned from protocols of physics students serving problems and studying examples. As expected, Cascade's model is overly simple, but it appears feasible to extend it to cover all the observed learning events. The data themselves were surprising in that there are few learning events relative to the number that could have occurred, and those that did occur often involved forms of reasoning that are considerably shallower than expected. The data suggest ways that instruction can be improved to increase both the quantity and depth of learning events.
Worked examples are instructional devices that provide an expert's problem solution for a learner to study. Worked-examples research is a cognitive-experimental program that has relevance to classroom instruction and the broader educational research community. A frame- work for organizing the findings of this research is proposed, leading to instructional design principles. For instance, one instructional design principle suggests that effective examples have highly integrated components. They employ multiple modalities in presentation and emphasize conceptual structure by labeling or segmenting. At the lesson level, effective instruction employs multiple examples for each conceptual problem type, varies example formats within problem type, and employs surface features to signal deep structure. Also, examples should be presented in close proximity to matched practice problems. More- over, learners can be encouraged through direct training or by the structure of the worked example to actively self:explain examples. Worked examples are associated with early stages of skill development, but the design principles are relevant to constructivist research and teaching.
A framework is presented for distinguishing between two types of mental representations formed while reading a text: The textbase is a representation built in the process of comprehension, and a situation model is built to represent the situation described in the text. Two studies are reported that explore the relative contribution of each type of representation and their interaction during problem solving. In the first, grade school children solved easy and hard arithmetic word problems of three types: change, combine, and compare. When asked to recall, reconstruction of the problems occurred and was related to solution performance. Children tended to recall problems already solved on the basis of the situation model used in solutions and not by reproducing the original textbase. In the second study, college students formed mental maps while reading two types of texts describing the layout of a town: The survey text described the town in geographical terms, and the route version presented the same information as a series of instructions for driving through the town. A dichotomy was seen between remembering the text and learning from it; the former was dependent on text coherence, and the latter depended on the formation of a situation model. Implications for instruction are discussed in terms of clarifying goals for the use of texts and distinguishing between instruction aimed toward recall and instruction aimed toward learning.
The goal of this study was to Investigate individual differences in learning from worked-out examples with respect to the quality of self-explanations. Restrictions of former studies (e.g., lacking control of time-on-task) were avoided and additional research questions (e.g., reliability and dimensionality of self-explanation characteristics) were addressed. An investigation with 36 university freshmen students of education working in individual sessions was conducted. The domain was probability calculation. Prior knowledge and the quality of self-explanations (protocols of the individuals' thinking aloud) were assessed as predictors of learning. A post-test was employed to measure the learning gains as the dependent variable. The following main results were obtained. Most self-explanation characteristics could be regarded as relatively stable person characteristics. The individual differences in the quality of self-explanations were, however, found to be multidimensional. Most Important, even when controlling for time-on-task (quantitative aspect), learning gains could be substantially predicted by qualitative differences of self-explanation characteristics. In particular, successful learners tended to employ more prlnciplebased explanations, more explication of operator-goal combinations, and more anticipative reasoning. In addition, there were two types of effective learners, labeled anticipative reasoners and principle-based explainers.
Are multiple examples necessary for schema induction
- K Scheiter
- P Gerjets
- J Schuh
Scheiter, K., Gerjets, P., & Schuh, J. (2003). Are multiple examples necessary for schema induction? In F. Schmalhofer, R. Young, & G. Katz
(Eds.), Proceedings of EuroCogSci 03. The European Cognitive Science Conference 2003 (pp. 283e288). Mahwah, NJ: Erlbaum.
TIMSS e Impulse für Schule und Unterricht. Forschungsbefunde, Reforminitiativen, Praxisberichte und Video-Dokumente [TIMSS e impulses for school and instruction. Research findings, reform initiatives, reports from the field, and video documents]
- E Klieme
- J Baumert
Klieme, E., & Baumert, J. (Eds.). (2001). TIMSS e Impulse für Schule und Unterricht. Forschungsbefunde, Reforminitiativen, Praxisberichte und
Video-Dokumente [TIMSS e impulses for school and instruction. Research findings, reform initiatives, reports from the field, and video
documents]. Bonn: Bundesministerium für Bildung und Forschung.
New perspectives on conceptual change
- W Schnotz
- S Vosniadou
- Carretero
Schnotz, W., Vosniadou, S., & Carretero, M. (Eds.). (1999). New perspectives on conceptual change. Oxford: Elsevier.
Learning and Instruction 17 (2007) 612e634 Author's personal copy
- S Große
S. Große, A. Renkl / Learning and Instruction 17 (2007) 612e634
Author's personal copy
Self-explanations: how students study and use examples in learning to solve problems
- Chi
The expertise reversal effect
- Kalyuga