Korbinian Moeller’s research while affiliated with University of Tübingen and other places

In increasing numbers of classrooms worldwide, students use digital learning software. However, we know little about the trajectories of usage and the performance within such digital learning software over the academic year. This study analyzed real-world longitudinal data from a mathematics learning software used in classrooms in Germany and the Netherlands (~16,000 students who worked on >23 million problems). We evaluated students' usage and performance trajectories across an academic year by examining the percentage of students using the software, worked-through problems, active days and weeks, as well as performance. Our results indicate a decline in both usage and performance over the course of the academic year, with overall lower usage in Germany than in the Netherlands. Our findings highlight the need for further research into the factors maintaining or increasing the usage of and performance in classroom-integrated digital learning software over extended periods.

In increasing numbers of classrooms worldwide, students use digital learning software. However, we know little about the trajectories of usage and the performance within such digital learning software over the academic year. This study analyzed real-world longitudinal data from a mathematics learning software used in classrooms in Germany and the Netherlands (16,000 students who worked on >23 million problems). We evaluated students' usage and performance trajectories across an academic year by examining the percentage of students using the software, worked-through problems, active days and weeks, as well as performance. Our results indicate a decline in both usage and performance over the course of the academic year, with overall lower usage in Germany than in the Netherlands. Our findings highlight the need for further research into the factors maintaining or increasing the usage of and performance in classroom-integrated digital learning software over extended periods.

Research on fostering learning about percentages within intelligent tutoring systems (ITSs) is limited. Additionally, there is a lack of data‐driven approaches for improving the design of ITS to facilitate learning about percentages. To address these gaps, we first investigated whether students' understanding of basic mathematical skills (eg, arithmetic, measurement units and geometry) and fractions within an ITS predicts their understanding of percentages. We then applied a psychological network analysis to evaluate interdependencies within the data on 44 subtopics of basic mathematical concepts, fractions and percentages. We leveraged a large‐scale dataset consisting of 2798 students using the ITS bettermarks and working on approximately 4.1 million mathematical problems. We found that advanced arithmetic, measurement units, geometry and fraction understanding significantly predicted percentage understanding. Closer inspection indicated that percentage understanding was best predicted by problems sharing similar features, such as fraction word problems and fraction/natural number multiplication/division problems. Our findings suggest that practitioners and software developers may consider revising specific subtopics which share features with percentage problems for students struggling with percentages. More broadly, our study demonstrates how evaluating interdependencies between subtopics covered within an ITS as a data‐driven approach can provide practical insights for improving the design of ITSs. Practitioner notes What is already known about this topic Longitudinal studies showed that basic mathematical skills predict fraction understanding. There is only limited evidence on whether similar predictions can be observed for percentage understanding—in general and within intelligent tutoring systems. Process data from such intelligent tutoring systems can be leveraged to pursue both educational research questions and optimizing digital learning software. Problems involving percentages typically are word problems requiring multiplications and/or divisions. What this paper adds Similar to the case of fractions, students' performance on advanced arithmetic, measurement units and geometry significantly predicted performance with percentages. Students' performance with fractions also predicted performance with percentages significantly. A psychological network analysis was applied to evaluate specific interdependencies between a range of subtopics (eg, Multiplying and dividing fractions, Adding and subtracting fractions and Calculating with percentages). Fraction word problems and fraction problems involving multiplication/division turned out to be the best predictors of understanding percentages. Implications for practice and/or policy When facing difficulties with percentages, revision of previous mathematical concepts sharing similar features (eg, fraction word problems, fraction/natural number multiplication/division problems) may be advised. Software developers may consider implementing such data‐driven revision recommendations for students facing difficulties within intelligent tutor systems. Psychological network analysis can be utilized as a learning analytics method for easy‐to‐access visualizations illustrating relationships between a large range of different subtopics.

The development and distribution of digital learning software, such as intelligent tutoring systems (ITSs), have evolved into a billion-dollar industry, impacting a vast number of students worldwide. While ITSs have significant potential to effectively transform learning and support teachers, the extent to which ITSs rely on teacher support to maintain students’ engagement with the ITS is less well understood. This study evaluates exceptionally rich data from an ITS for learning mathematics used in Germany and the Netherlands (~139,000,000 problems; n ~ 194,000 students) between 2016 and 2023. We compared whether students’ engagement varied based on how they used the ITS: with teacher support (i.e., teachers assigned mathematics problems to students) versus without teacher support (i.e., students self-assigned problems). Our results indicated that students in Germany and the Netherlands consistently (i) dropped out later, (ii) were active for significantly more weeks, and (iii) worked through more mathematics problems each week with teacher support as compared to without teacher support. We observed these results robustly across academic school years within the examined period. Overall, our study challenges the usefulness of digital learning software as stand-alone tools and emphasizes the need for embedding such software in real-life learning scenarios involving teachers.

Background: Students who learn with adaptive intelligent tutoring systems (ITS) are often provided with additional learning materials to close their knowledge gaps when the system recognizes difficulties in a topic. Students may be nudged with notification badges to close their knowledge gaps. Aim: This study evaluated the effectiveness of implementing a small red button as a notification badge to close students’ knowledge gaps. Additionally, the association between closing knowledge gaps and students’ overall performance was investigated while controlling for relevant factors (e.g., students’ usage behavior, adaptive system features, and teachers’ assignment types). Samples: We considered 105,081 students (in 6,668 classes working on ~25 million problems) who used an ITS for learning mathematics. Methods: We employed hierarchical linear regression analysis to evaluate whether students were more likely to close their knowledge gaps in a period in which notification badges were implemented in the ITS, as compared to a period without notification badges. Moreover, we used psychological network analysis to evaluate whether closing knowledge gaps was associated with students’ performance, relative to other relevant factors.Results: Students closed 28% of their knowledge gaps when they were nudged with a notification badge but only closed 5% when they were not nudged. Closing knowledge gaps predicted better student performance and was a more important predictor than students’ usage behavior, other adaptive system features, and teachers’ assignment types. Conclusions: Our study highlights the relevance of adaptive nudges within ITS for closing knowledge gaps and thus improving learning.

Background: COVID-19-related school closures globally disrupted students' education, leading to declines in academic performance. However, some studies also reported performance increases within intelligent tutoring systems (ITSs). Yet, the latter evidence stems exclusively from so-called “Western” countries (e.g., Germany). Hence, whether these findings generalize to other parts of the world remains unclear. Aim: We sought to evaluate whether performance trajectories within ITSs for learning mathematics during COVID-19-related school closures were similar for students from Germany and from Uruguay—a country and cultural context that has not yet received systematic research attention in this line of research.Samples: We analyzed two longitudinal ITS datasets from Uruguay (1,357 students; 38,673 problem sets) and Germany (8,751 students; 328,948 problem sets).Methods: We applied hierarchical linear regressions to evaluate how COVID-19-related school closures affected students' performance in mathematics in both countries.Results: We observed significant performance increases during periods of COVID-19-related school closures relative to students’ performance during the same period in previous years for both samples. These performance increases were associated with increases in relative usage, with students who used the ITS relatively more during COVID-19-related school closures than before also performing better. Conclusions: Our results indicate that previous results of increased academic performance within ITS for learning mathematics during periods of COVID-19-related school closures also pertain to students from Uruguay and highlight the need to diversify research samples. Our findings corroborate the notion that ITSs are a promising and effective tool for distance learning in different regions of the world.

The adoption of intelligent tutoring systems (ITSs) worldwide has led to a considerable accumulation of process data as students interact with different learning topics within these systems. Typically, these learning topics are structured within ITSs (e.g., the fraction topic includes subtopics such as a fraction number line subtopic). However, there is a lack of methods that offer quick, data-driven insights into the content structure of ITSs, particularly through easily accessible visualizations. Here, we applied psychological network analysis to process data (230,241 students; 5,365,932 problem sets) from an ITS for learning mathematics to explore performance interdependencies between 40 different subtopics. We argue that the visualization of these content interdependencies allows for a quick empirical evaluation of the validity of the existing structuring of the respective learning content. These insights allow for deriving recommendations concerning potential changes in the ITS structure and are thus highly valuable for ITS developers. Our results are also relevant for researchers as the interdependencies illustrated through psychological network analysis can contribute towards a better understanding of the interplay between mathematical skills. Together, our results indicate that psychological network analysis represents a valuable data-driven method to evaluate and optimize ITSs.