Steven Ritter's research while affiliated with Carnegie Learning and other places
What is this page?
This page lists the scientific contributions of an author, who either does not have a ResearchGate profile, or has not yet added these contributions to their profile.
It was automatically created by ResearchGate to create a record of this author's body of work. We create such pages to advance our goal of creating and maintaining the most comprehensive scientific repository possible. In doing so, we process publicly available (personal) data relating to the author as a member of the scientific community.
If you're a ResearchGate member, you can follow this page to keep up with this author's work.
If you are this author, and you don't want us to display this page anymore, please let us know.
It was automatically created by ResearchGate to create a record of this author's body of work. We create such pages to advance our goal of creating and maintaining the most comprehensive scientific repository possible. In doing so, we process publicly available (personal) data relating to the author as a member of the scientific community.
If you're a ResearchGate member, you can follow this page to keep up with this author's work.
If you are this author, and you don't want us to display this page anymore, please let us know.
Publications (33)
Automated, data‐driven decision making is increasingly common in a variety of application domains. In educational software, for example, machine learning has been applied to tasks like selecting the next exercise for students to complete. Machine learning methods, however, are not always equally effective for all groups of students. Current approac...
Transitioning from in-person to remote instruction has forced teachers to navigate unexpected constraints while providing meaningful learning experiences for their students. This transition has drastically changed how teachers orchestrate learning for their students. To explore these unique orchestration challenges, we used needs finding and valida...
Building on recent work related to measuring situational, in-the-moment motivation and the stability of motivation profiles, this study explores the nature of situational motivation profiles constructed with measurements of achievement goals during middle and high school students’ algebra-focused intelligent tutoring system (ITS) learning during an...
We present a brief case study of a multi-year learning engineering effort to iteratively redesign the problem-solving experience of students using the “Solving Quadratic Equations” workspace in Carnegie Learning’s MATHia intelligent tutoring system. We consider two design changes, one involving additional scaffolds for the problem-solving task and...
Extensive literature in artificial intelligence in education focuses on developing automated methods for detecting cases in which students struggle to master content while working with educational software. Such cases have often been called “wheel-spinning,” “unproductive persistence,” or “unproductive struggle.” We argue that most existing efforts...
This paper describes a new, open source tool for A/B testing in educational software called UpGrade. We motivate UpGrade's approach, describe development goals and UpGrade's software architecture, and provide a brief overview of working within UpGrade to define and monitor experiments. We conclude with some avenues for future research and developme...
If we wish to embed assessment for accountability within instruction, we need to better understand the relative contribution of different types of learner data to statistical models that predict scores and discrete achievement levels on assessments used for accountability purposes. The present work scales up and extends predictive models of math te...
If we wish to embed assessment for accountability within instruction, we need to better understand the relative contribution of different types of learner data to statistical models that predict scores on assessments used for accountability purposes. The present work scales up and extends predictive models of math test scores from existing literatu...
An Architecture for Plug-in Tutor Agents (Ritter and Koedinger 1996) proposed a software architecture designed around the idea that tutors could be built as plug-ins for existing software applications. Looking back on the paper now, we can see that certain assumptions about the future of software architecture did not come to be, making the particul...
Generalizability of models of student learning is a highly desirable feature. As new students interact with educational systems, highly predictive models, tuned to increasing amounts of data from previous learners, presumably allow such systems to provide a more individualized, optimal learning path, give better feedback, and provide a more effecti...
Learners often think math is unrelated to their own interests. Instructional software has the potential to provide personalized instruction that responds to individuals' interests. Carnegie Learning's MATHia™ software for middle school mathematics asks learners to specify domains of their interest (e.g., sports & fitness, arts & music), as well as...
Most work on learning curves for ITSs has focused on the knowledge components (or skills) included in the curves, aggregated across students. But an aggregate learning curve need not have the same form as subsets of its underlying data, so learning curves for subpopulations of students may take different forms. We show that disaggregating a skill’s...
One function of a student model in tutoring systems is to select future tasks that will best meet student needs. If the inference procedure that updates the model is inaccurate, the system may select non-optimal tasks for enhancing students' learning. Poor selection may arise when the model assumes multiple knowledge components are required for a s...
We describe a methodology for simulating student behavior to predict the effects of skill-learning parameter changes on system
behavior. Validation against data collected after the changes were made shows that accurate predictions can be made despite
a different cohort of students. Furthermore, deviations from the predictions may help explain unexp...
Carnegie Learning’s Cognitive Tutors for mathematics have been the subject of a wide variety of research [3,4] and are the
most widely deployed Intelligent Tutoring Systems. Currently, over 560,000 students and 2,700 schools in all 50 United States
are using them. Many ITS researchers understand these tutors from printed works but have not had the...
Geometric proof has long been a topic of study within Intelligent Tutoring Systems [3,4]. Proof is interesting because it
supports a variety of solutions and strategies. However, implementation is challenging and such tutors have not been widely
deployed.
Mixing worked-out examples with problem solving has been shown to be an effective blend of educational activities [1]. Given
the positive impact on learning, some Intelligent Tutoring Systems attempt to incorporate worked-out examples into their learning
environments. The approach taken by Carnegie Learning’s Cognitive Tutor, called Interactive Exa...
Intelligent Tutoring Systems (ITSs) that employ a model-tracing methodology have consistently shown their effectiveness. However, what evidently makes these tutors effective, the cognitive model embedded within them, has traditionally been difficult to create, requiring great expertise and time, both of which come at a cost. Furthermore, an interfa...
In Cognitive Tutors, student skill is represented by estimates of student knowledge on various knowledge components. The estimate for each knowledge component is based on a four-parameter model developed by Corbett and Anderson [Nb]. In this paper, we investigate the nature of the parameter space defined by these four parameters by modeling data fr...
For 25 years, we have been working to build cognitive models of mathematics, which have become a basis for middle- and high-school curricula. We discuss the theoretical background of this approach and evidence that the resulting curricula are more effective than other approaches to instruction. We also discuss how embedding a well specified theory...
The main goal of the work presented here is to allow for the broader dissemination of intelligent tutoring technology. To accomplish this goal, we have two clear objectives. First, we want to allow different types of people to author model-tracing intelligent tutoring systems (ITSs) than can now do so. Second, we want to enable an author to create...
Efforts to improve performance in mathematics have put pressure on educational evaluators to improve the rigor of their evaluation designs. This paper reports the results of a rigorous evaluation of the Cognitive Tutor Algebra I curriculum, which is substantially based on an intelligent tutoring system. We emphasize the importance of presenting det...
Middle school mathematics teachers are often forced to choose between assisting students' development and assessing students' abilities because of limited classroom time available. To help teachers make better use of their time, we are integrating assistance and assessment by utilizing a web-based system ("Assistment") that will offer instruction t...
Our efforts to commercialize Cognitive Tutors have led us to a runtime representation that is significantly different from
the production system representation used in the Tutor Development Kit. This paper describes our new representation, which
we call the Tutor Runtime Environment (TRE).
The movement towards high stakes testing promises to encourage rigor and accountability in middle school mathematics, but there is a danger that a too- narrow focus on testing will take time and attention away from mathematics instruction. The fundamental dilemma that teachers face in trying to use assessment to guide instruction (i.e., to figure o...
Citations
... The model might exploit such information during training -e.g., to achieve representation invariance -or during inference, in order to perform group-speci c calibration or thresholding. Such a model inherits the well-documented problems that arise from sorting humans into categories based predominantly on sociocultural constructs [4,30], as opposed to physiological di erences relevant to the prediction task at hand. As an alternative, consider now the case of a model that has learned an implicit representation of group membership. ...
... Scientific findings can then generally contribute to the theory used to inform subsequent educational software designs. Further extensions of this work can be found at upgradeplatform.org, which supports UpGrade, an open-source platform for A/B testing in educational software (Ritter et al, 2020). ...
... In some cases, these two types of dashboard are being integrated. For example MATHia LiveLab predicts if a student will fail to reach mastery and provides these predictions to teachers (Fancsali et al., 2020). ...
... Online English teaching assistance system, using decision tree Frontiers in Education 03 frontiersin.org algorithms and neural network models, was also implemented, which improved the efficiency of teaching (Fancsali et al., 2018;Zheng et al., 2019;Sun et al., 2021). However, the hidden layer(s) likes a black box, as the common factors inside cannot be directly observed. ...
... This could consequently have an impact on their performance and graduation prospects. The same goes for students who graduated from high school with a low Grade-Point Average (GPA), who may be more likely than those with high GPAs to perform poorly in college [5]. Earlier LA models depended on preset parameters to deliver a single set of predictions within a given time frame. ...
... The Shipborne Over-and Underwater integrated Mobile Mapping System (SiOUMMS) described in this paper integrates a multi-beam echo sounder, laser scanner, GNSS, and Inertial Measurement Unit (IMU) device onboard and fixedly connects the sensors to ensure that their spatial relationship remains unchanged, which realizes their physical integration and provides the basis for seamless integration of over-and under-water point clouds. To meet the practical demands of various application scenarios and support a diversity of sensors, plug-in (Grapenthin et al. 2014;Ritter et al. 2016) and multi-process network technologies are used to integrate data acquisition and cooperative control with multi-sensors and to ensure high cohesion and low coupling in the system. This enhances the performability, testability, maintainability, and load balancing of the system, as is highlighted in this paper and can provide a good reference framework for the integration of similar systems. ...
... Correct attempts after an initial hint are now credited (i.e., skill mastery estimates increase) like immediate correct attempts, and correct attempts after mid-level hints now leave the skill mastery estimate unchanged. In addition, MA-THia's BKT parameter estimates for each skill (used to determine the models' responsiveness to correct and incorrect answers) are now frequently set based on data-driven estimation techniques [32,39] as opposed to mostly being set according to expert judgment in earlier Cognitive Tutor versions. ...
... Quality math education, particularly at a young age, is of crucial importance for students growing into an increasingly technology-driven world. Intelligent tutoring systems (ITSs) have demonstrated their effectiveness in improving math learning outcomes [1,10,24]. Many ITSs have a component that provides automated feedback for students while they solve math questions. ...
... Later, with the rise of classic machine learning methods such as logistic regression models [117], another direction followed by KT is to use parametric factor analysis approaches which trace a student's knowledge states and perform the answer prediction based on modeling a variety of factors [16,17,86], including: (1) aspects about students such as prior knowledge, learning capacity, or learning rate; (2) aspects about learning materials such as familiarity, number of previous practices, or difficulty; (3) aspects about a learning environment itself such as the nature of the learning channel (paper-or computer-based) and the temporal context of the practice time (within an examination period or a regular study period). In addition to these, psychological studies about the learning behavior [75] and forgetting behavior [52] of students have also suggested additional factors, such as the time lapse between a student's different interactions and the number of times on practicing learning materials, to be considered when tracing knowledge states. It is worth noting that this direction of KT is still active [35,111] and is considered as an alternative to the recent state-of-the-art approaches based on deep learning. ...
Reference: Knowledge Tracing: A Survey
... Log data from math tutors have also been used to predict student scores on end-of-year state accountability exams, resulting in better prediction than paper-pencil benchmark tests and standardized tests [4,18,21]. These models become better still when supplemented with data on student strategies [36]. ...