Kenneth R. Koedinger

Carnegie Mellon University | CMU · Human-Computer Interaction Institute

Leveraging a scientific infrastructure for exploring how students learn, we have developed cognitive and statistical models of skill acquisition and used them to understand fundamental similarities and differences across learners. Our primary question was why do some students learn faster than others? Or do they? We model data from student performa...

Personalized Learning2 (PL2) is a mentor professional development platform designed to improve efficiency and workplace training through scenario-based instruction and personalized support. Combining research-driven mentor training with artificial intelligence-powered (AI-powered) software, PL2 connects mentors, often under-trained tutors, to perso...

Background Skill integration is vital in students' mastery development and is especially prominent in developing code tracing skills which are foundational to programming, an increasingly important area in the current STEM education. However, instructional design to support skill integration in learning technologies has been limited. Objectives Th...

Conversational agents (CAs) provide opportunities for improving the interaction in evaluation surveys. To investigate if and how a user-centered conversational evaluation tool impacts users' response quality and their experience, we build EVA - a novel conversational course evaluation tool for educational scenarios. In a field experiment with 128 s...

In utility-value interventions, students learn about and reflect on the potential usefulness of academic content. These interventions have proven effective at enhancing student motivation and performance in in-person math and science classes. Thus, the present case study describes the design of a utility-value intervention to be used in the context...

Gaming the system, a behavior in which learners exploit a system's properties to make progress while avoiding learning, has frequently been shown to be associated with lower learning. However, when we applied a previously validated gaming detector across conditions in experiments with an algebra tutor, the detected gaming was not associated with le...

Advances in Natural Language Processing offer techniques to de- tect the empathy level in texts. To test if individual feedback on certain students’ empathy level in their peer review writing pro- cess will help them to write more empathic reviews, we developed ELEA, an adaptive writing support system that provides students with feedback on the cog...

Background Museum exhibits encourage exploration with physical materials typically with minimal signage or guidance. Ideally children get interactive support as they explore, but it is not always feasible to have knowledgeable staff regularly present. Technology-based interactive support can provide guidance to help learners achieve scientific unde...

This report showcases a new type of online homework system that provides students with a free-form interface and dynamic feedback. The ORCCA Tutor (Open-Response Chemistry Cognitive Assistance Tutor) is a production rules-based online tutoring system utilizing the Cognitive Tutoring Authoring Tools (CTAT) developed by Carnegie Mellon University. In...

Recent advances in machine learning have made it possible to train artificially intelligent agents that perform with super-human accuracy on a great diversity of complex tasks. However, the process of training these capabilities often necessitates millions of annotated examples -- far more than humans typically need in order to achieve a passing le...

Simulations of human learning have shown potential for supporting ITS authoring and testing, in addition to other use cases. To date, simulated learner technologies have often failed to robustly achieve perfect performance with considerable training. In this work we identify an impediment to producing perfect asymptotic learning performance in simu...

Computer tutor data indicate that more learning opportunities yield greater achievement, but also confirm there are gaps in the number and quality of opportunities marginalized students receive that technology alone does not address. Personalized learning with mentors can close this gap in opportunities but is expensive to implement. We introduce a...

jats:p>Practice is essential for learning. However, for many interpersonal skills, there often are not enough opportunities and venues for novices to repeatedly practice. Role-playing simulations offer a promising framework to advance practice-based professional training for complex communication skills, in fields such as teaching. In this work, we...

Analytics of student learning data are increasingly important for continuous redesign and improvement of tutoring systems and courses. There is still a lack of general guidance on converting analytics into better system design, and on combining multiple methods to maximally improve a tutor. We present a multi-method approach to data-driven redesign...

While Artificial Intelligence in Education (AIED) research has at its core the desire to support student learning, experience from other AI domains suggest that such ethical intentions are not by themselves sufficient. There is also the need to consider explicitly issues such as fairness, accountability, transparency, bias, autonomy, agency, and in...

Intelligent tutoring systems are effective for improving students’ learning outcomes (Pane et al. 2013; Koedinger and Anderson, International Journal of Artificial Intelligence in Education, 8 , 1–14, 1997; Bowen et al. Journal of Policy Analysis and Management, 1 , 94–111 2013). However, constructing tutoring systems that are pedagogically effecti...

Design-loop adaptivity, which involves data-driven redesign of an instructional system based on student learning data, has shown promise in improving student learning. We present a general, systematic approach that combines new and existing data mining and instructional design methods to redesign intelligent tutors. Our approach is driven by the ma...

Simulated learners represent computational theories of human learning that can be used to evaluate educational technologies, provide practice opportunities for teachers, and advance our theoretical understanding of human learning. A key challenge in working with simulated learners is evaluating the accuracy of the simulation compared to the behavio...

Adaptive math software supports students’ learning by targeting specific math knowledge components. However, widespread use of adaptive math software in classrooms has not led to the expected changes in student achievement, particularly for racially minoritized students and students situated in poverty. While research has shown the power of human m...

Along with substantial consensus around the power of active learning, comes some lack of precision in what its essential ingredients are. New educational technologies offer vehicles for systematically exploring benefits of alternative techniques for supporting active learning. We introduce a new genre of Intelligent Science Station technology that...

Compared to other helping professions, teacher training typically lacks sufficient opportunities for novices to practice new skills. When teachers learn, they listen to people talk about teaching, or talk about teaching themselves, but they very rarely do the work of teaching. Games and simulations offer a promising framework to advance practice-ba...

Using data to understand learning and improve education has great promise. However, the promise will not be achieved simply by AI and Machine Learning researchers developing innovative models that more accurately predict labeled data. As AI advances, modeling techniques and the models they produce are getting increasingly complex, often involving t...

Research studies show that teachers increase the success of education technologies in rural settings by supporting students via technology support, domain-relevant explanations, enforcing discipline, and maintaining student engagement. However, a teacher's presence hinders student collaboration in some cultural contexts, and some students may not h...

Tablet-based educational technologies provide a supplement to traditional classroom-based early literacy education, especially in regions with limited schooling resources. Prior work has probed how children generally interact with and learn from these technologies, however, there is limited research on student engagement with applications that util...

In schools and colleges around the world, open-ended home-work assignments are commonly used. However, such assignments require substantial instructor effort for grading, and tend not to support opportunities for repeated practice. We propose UpGrade, a novel learnersourcing approach that generates scalable learning opportunities using prior studen...

Determining the impact of belief bias on everyday reasoning is critical for understanding how our beliefs can influence how we judge arguments. We examined the impact of belief bias on the user’s ability to identify logical fallacies in political arguments. We found that participants had more difficulty identifying logical fallacies in arguments th...

Massive Open Online Courses (MOOCs) often incorporate lecture-based learning along with lecture notes, textbooks, and videos to students. Moreover, MOOCs also incorporate practice activities and quizzes. Student learning in MOOCs can be tracked and improved using state-of-the-art student modeling. Currently, this means employing conventional studen...

This demo will showcase Tigris-an online workflow tool developed as part of the LearnSphere project. LearnSphere is a community data infrastructure to support learning improvement online, and brings together a number of data repositories including DataShop (Stamper et al., 2010) and DiscourseDB (Rosé & Ferschke, 2016). Instruction is a data-rich ac...

Selective sustained attention, or the ability to allocate perceptual and mental resources to a single object or event, is an important cognitive ability widely assumed to be required for learning. Assessing young children's selective sustained attention is challenging due to the limited number of sensitive and developmentally appropriate performanc...

Researchers have developed cognitive systems capable of human-level performance at complex tasks, but constructing these systems required substantial time and expertise. To address this challenge, a new line of research has begun to coalesce around the concept of cognitive systems that users can teach rather than program. A key goal of this researc...

Understanding how learning transfers from one task to another is a critical topic in learning science. In this paper, we investigate the impact of the scope and granularity of learning transfer by comparing three models across multiple data sets. Prior work demonstrated the value of component models of learning transfer that group items into knowle...

Understanding how learning transfers from one task to another is a critical topic in learning science. In this paper, we investigate the impact of the scope and granularity of learning transfer by com- paring three models across multiple data sets. Prior work demon- strated the value of component models of learning transfer that group items into kn...

A cognitive model of human learning provides information about skills a learner must acquire to perform accurately in a task domain. Cognitive models of learning are not only of scientific interest, but are also valuable in adaptive online tutoring systems. A more accurate model yields more effective tutoring through better instructional decisions....

A cognitive model of human learning provides information about skills a learner must acquire to perform accurately in a task domain. Cognitive models of learning are not only of scientific interest, but are also valuable in adaptive online tutoring systems. A more accurate model yields more effective tutoring through better instructional decisions....

The proliferation of fake news has underscored the importance of critical thinking in the civic education curriculum. Despite this recognized importance, systems designed to foster these kinds of critical thinking skills are largely absent from the educational technology space. In this work, we utilize an instructional factors analysis in conjuncti...

Smartphone- and tablet-based learning systems are often posited as solutions for closing early literacy gaps between rural and urban regions in emerging economies. These systems are often developed based on experiences with students in urban contexts, limiting their success rates with children from rural areas who have had little to no prior exposu...

CS Education makes heavy use of online educational tools like IDEs, Learning Management Systems, eTextbooks, interactive programming environments, and other smart content. Instructors and students would benefit from greater interoperability between tools. CS Ed researchers increasingly make use of the large collections of data generated by click st...

English Language Learners (ELLs) are a substantial portion of the students who enroll in MOOCs. In order to fulfill the promise of MOOCs – i.e., making higher education accessible to everyone with an internet connection – appropriate interventions should be offered to students who struggle with the language of course content. Through the analysis o...

In this age of fake news and alternative facts, the need for a citizenry capable of critical thinking has never been greater. While teaching critical thinking skills in the classroom remains an enduring challenge, research on an ill-defined domain like critical thinking in the educational technology space is even more scarce. We propose a difficult...

Many game designers aim to optimize difficulty to make games that are "not too hard, not too easy." However, recent experiments have shown that even moderate difficulty can reduce player engagement. The present work investigates other design factors that may account for the purported benefits of difficulty, such as choice, novelty and suspense. The...

The current study introduces a model for measuring student diligence using online behaviors during intelligent tutoring system use. This model is validated using a full academic year dataset to test its predictive validity against long-term academic outcomes including end-of-year grades and total work completed by the end of the year. The model is...

Making MOOCs accessible to English Language Learners (ELLs) requires that students understand the language of instruction, and that instructional strategies address their unique learning challenges. Through the analysis of clickstream log data gathered from two MOOC courses deployed on Coursera, Introduction to Psychology and Statistical Thermodyna...

Presents DIBBs project results, identifies and recognizes achievements, discusses current challenges (technical financial, and social), and discusses future challenges and models to address them, with the goal of informing a future vision of data cyberinfrastructure and the science and engineering disciplines it enables.

This workshop will explore community based repositories for educational data and analytic tools that are used to connect researchers and reduce the barriers to data sharing. Leading innovators in the field, as well as attendees, will identify and report on bottlenecks that remain toward our goal of a unified repository. We will discuss these as wel...

This paper proposes grounded feedback as a way to provide implicit verification when students are working with a novel representation. In grounded feedback, students’ responses are in the target, to-be-learned representation, and those responses are reflected in a more-accessible linked representation that is intrinsic to the domain. By examining t...

Can experimenting with three-dimensional (3D) physical objects in mixed-reality environments produce better learning and enjoyment than flat-screen two-dimensional (2D) interaction? We explored this question with EarthShake: a mixed-reality game bridging physical and virtual worlds via depth-camera sensing, designed to help children learn basic phy...

The recent surge in interest in using educational data mining on student written programs has led to discoveries about which compiler errors students encounter while they are learning how to program. However, less attention has been paid to the actual code that students produce. In this paper, we investigate programming data by using learning curve...

With the growing popularity of MOOCs and sharp trend of digitalizing education, there is a huge amount of free digital educational material on the web along with the activity logs of large number of participating students. However, this data is largely unstructured and there is hardly any information about the relationship between material from dif...