Vincent Aleven's research while affiliated with Carnegie Mellon University and other places

Publications (284)

Conference Paper
Full-text available
One pedagogical technique that promotes conceptual understanding in mathematics learners is self-explanation integrated with worked examples (e.g., Rittle-Johnson et al., 2017). In this work, we implemented self-explanations with worked examples (correct and erroneous) in a software-based Intelligent Tutoring System (ITS) for learning algebra. We d...
Conference Paper
Full-text available
Integrating visual representations in an interactive learning activity effectively scaffolds performance and learning. However, it is unclear whether and how sustaining or interleaving visual scaffolding helps learners solve problems efficiently and learn from problem solving. We conducted a classroom study with 63 middle-school students in which w...
Article
Recent work has explored how complementary strengths of humans and artificial intelligence (AI) systems might be productively combined. However, successful forms of human–AI partnership have rarely been demonstrated in real‐world settings. We present the iterative design and evaluation of Lumilo, smart glasses that help teachers help their students...
Article
This report showcases a new type of online homework system that provides students with a free-form interface and dynamic feedback. The ORCCA Tutor (Open-Response Chemistry Cognitive Assistance Tutor) is a production rules-based online tutoring system utilizing the Cognitive Tutoring Authoring Tools (CTAT) developed by Carnegie Mellon University. In...
Chapter
Transitioning from in-person to remote instruction has forced teachers to navigate unexpected constraints while providing meaningful learning experiences for their students. This transition has drastically changed how teachers orchestrate learning for their students. To explore these unique orchestration challenges, we used needs finding and valida...
Chapter
Orchestration tools may support K-12 teachers in facilitating student learning, especially when designed to address classroom stakeholders’ needs. Our previous work revealed a need for human-AI shared control when dynamically pairing students for collaborative learning in the classroom, but offered limited guidance on the role each agent should tak...
Chapter
Despite the drastic change to school environments due to the COVID-19 pandemic, it is still important that educational technology researchers conduct school-based research to understand the impact of technology in an authentic learning context, even remotely. However, the transition to remote research has made it challenging for researchers to coll...
Conference Paper
Full-text available
Despite the prevalence of conducting classroom studies using educational technology, it is underexplored what practical benefits classroom studies with educational technology offer to teachers and students, and what concerns or challenges they perceive. Our interviews found that teachers view study participation as a meaningful learning opportunity...
Conference Paper
Full-text available
Prior research shows that self-explanation promotes understanding by helping learners connect new knowledge with prior knowledge. However, despite ample evidence supporting the effectiveness of self-explanation, an instructional design challenge emerges in how best to scaffold self-explanation. In particular, it is an open challenge to design self-...
Conference Paper
Full-text available
Research has shown that tape diagrams are beneficial for algebra learning. However, it is unclear whether certain visual features of tape diagrams have implications for learning. We investigated, with undergraduate students and math teachers, whether tape diagrams with different visual features (color, presence of outer lines, and position of the c...
Preprint
Full-text available
Although visual representations are generally beneficial for learners, past research also suggests that often only a subset of learners benefits from visual representations. In this work, we designed and evaluated anticipatory diagrammatic self- explanation, a novel form of instructional scaffolding in which visual representations are used to guide...
Conference Paper
Full-text available
Analytics of student learning data are increasingly important for continuous redesign and improvement of tutoring systems and courses. There is still a lack of general guidance on converting analytics into better system design, and on combining multiple methods to maximally improve a tutor. We present a multi-method approach to data-driven redesign...
Preprint
Full-text available
Recent work has explored how complementary strengths of humans and artificial intelligence (AI) systems might be productively combined. However, successful forms of human-AI partnership have rarely been demonstrated in real-world settings. We present the iterative design and evaluation of Lumilo, smart glasses that help teachers help their students...
Article
Full-text available
When students are working collaboratively and communicating verbally in a technology‐enhanced environment, the system cannot track what collaboration is happening outside of the technology, making it difficult to fully assess the collaboration of the students and adapt accordingly. In this article, we propose using gaze measures as a proxy for cogn...
Article
Full-text available
Educational technologies are often developed such that students work on specific social levels (e.g., individual, small group, whole class) at specific times. However, in the reality of the classroom, learning activities are not so cleanly divided, with transitions occurring between social levels for students at different times. To support these so...
Chapter
Dynamically transitioning between individual and collaborative learning activities during a class session (i.e., in an un-planned way, as-the-need-arises) has advantages for students, but existing orchestration tools are not designed to support such transitions. This work reports findings from a technology probe study that explored alternative desi...
Article
The analysis of multiple data streams is a long‐standing practice within educational research. Both multimodal data analysis and temporal analysis have been applied successfully, but in the area of collaborative learning, very few studies have investigated specific advantages of multiple modalities versus a single modality, especially combined with...
Chapter
Extensive literature in artificial intelligence in education focuses on developing automated methods for detecting cases in which students struggle to master content while working with educational software. Such cases have often been called “wheel-spinning,” “unproductive persistence,” or “unproductive struggle.” We argue that most existing efforts...
Chapter
Full-text available
Design-loop adaptivity, which involves data-driven redesign of an instructional system based on student learning data, has shown promise in improving student learning. We present a general, systematic approach that combines new and existing data mining and instructional design methods to redesign intelligent tutors. Our approach is driven by the ma...
Conference Paper
Full-text available
Many studies have shown that visual representations can enhance student understanding of STEM concepts. However, prior research suggests that visual representations alone are not necessarily effective across a broad range of students. To address this problem, we created a novel, scaffolded form of diagrammatic self-explanation in which students exp...
Conference Paper
Full-text available
When designing an instructional tool and using it in pedagogical activities, it is essential that designers and users understand what pedagogical affordances and constraints the tool provides to support its successful integration into targeted pedagogical activities. Toward this end, we developed Pedagogical Affordance Analysis (PAA). PAA involves...
Chapter
Educational AI (AIEd) systems are increasingly designed and evaluated with an awareness of the hybrid nature of adaptivity in real-world educational settings. In practice, beyond being a property of AIEd systems alone, adaptivity is often jointly enacted by AI systems and human facilitators (e.g., teachers or peers). Despite much recent research ac...
Article
Since the 1960s, researchers have been trying to optimize the sequencing of instructional activities using the tools of reinforcement learning (RL) and sequential decision making under uncertainty. Many researchers have realized that reinforcement learning provides a natural framework for optimal instructional sequencing given a particular model of...
Article
Research on Computer-Supported Collaborative Learning (CSCL) has provided significant insights into why collaborative learning is effective and how we can effectively provide support for it. Building on this knowledge, we can investigate when collaboration is beneficial to support learning. Specifically, collaborative and individual learning are of...
Article
Involving stakeholders throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. However, given the complexity of learning analytics (LA) systems, it can be challenging to meaningfully involve non-technical stakeholders throughout their design and development. This article reports...
Conference Paper
Full-text available
There are strong potential benefits to be had by integrating intelligent tutoring systems (ITSs) with each other, but few instances of successful integration are known in the literature. Given the central role that student models play in ITS architectures, and given that different ITS platforms tend to have their own student models, a key challenge...
Chapter
As artificial intelligence (AI) increasingly enters K-12 classrooms, what do teachers and students see as the roles of human versus AI instruction, and how might educational AI (AIED) systems best be designed to support these complementary roles? We explore these questions through participatory design and needs validation studies with K-12 teachers...
Article
A successful instructional method is to engage learners with exploratory problem-solving before providing explanations of the canonical solutions and foundational concepts. A key question is whether and what type of guidance will lead learners to explore more productively and how this guidance will affect subsequent learning and transfer. We invest...
Conference Paper
Full-text available
Instruction that adapts to individual learner characteristics is often more effective than instruction that treats all learners as the same. A practical approach to making MOOCs adapt to learners may be by integrating frameworks for intelligent tutoring systems (ITSs). Using the Learning Tools Interoperability standard (LTI), we integrated two inte...
Chapter
ITS authoring tools make creating intelligent tutoring systems more cost effective, but few authoring tools make it easy to flexibly incorporate an open-ended range of student modeling methods and learning analytics tools. To support a cumulative science of student modeling and enhance the impact of real-world tutoring systems, it is critical to ex...
Chapter
When used in K-12 classrooms, intelligent tutoring systems (ITSs) can be highly effective in helping students learn. However, they might be even more effective if designed to work together with human teachers, to amplify their abilities and leverage their complementary strengths. In the present work, we designed a wearable, real-time teacher awaren...
Chapter
Computer programming is becoming important in almost every profession. However, programming is still difficult for students to learn. In this work, we focus on helping students acquire strong conceptual and procedural knowledge of programing. We propose to create a new Intelligent Tutoring System (ITS) that will support students in two types of con...
Conference Paper
When used in classrooms, personalized learning software allows students to work at their own pace, while freeing up the teacher to spend more time working one-on-one with students. Yet such personalized classrooms also pose unique challenges for teachers, who are tasked with monitoring classes working on divergent activities, and prioritizing help-...
Conference Paper
Full-text available
This paper aims to link student facing Learning Analytics Dashboards (LADs) to the corpus of research on Open Learner Models (OLMs), as both have similar goals. We conducted a systematic review of literature on OLMs and compared the results with a previously conducted review of LADs for learners in terms of (i) data use and modelling, (ii) key publ...
Conference Paper
Accurately modeling individual students' knowledge growth is important in many applications of learning analytics. A key step is to decompose the knowledge targeted in the instruction into detailed knowledge components (KCs). We search for an accurate KC model for basic equation solving skills, using data from an intelligent tutoring system (ITS),...
Conference Paper
Intelligent Tutoring Systems (ITSs) help students learn but often are not designed to support teachers and their practices. A dashboard with analytics about students’ learning processes might help in this regard. However, little research has investigated how dashboards influence teacher practices in the classroom and whether they can help improve s...
Conference Paper
Even though Intelligent Tutoring Systems (ITS) have been shown to help students learn, little research has investigated how a dashboard could help teachers help their students. In this paper, we explore how a dashboard prototype designed for an ITS affects teachers’ knowledge about their students, their classroom lesson plans and class sessions. We...
Conference Paper
A focus in recent AIED research is to create adaptive support for learners in inquiry learning environments. However, only few examples of such support have been demonstrated. Our work focuses on Invention activities, inquiry activities in which students generate representations that explain data presented as contrasting cases. To help teachers imp...
Conference Paper
Full-text available
Social interdependence is a key concept in CSCL research. However, investigations of students' positive and negative interdependence during collaborative activities have often relied on self-report, rather than dialogue analysis. Bringing together politeness and social interdependence theory, we assessed dialogue indicators of positive and negative...
Article
Full-text available
Prior research shows that representational competencies that enable students to use graphical representations to reason and solve tasks is key to learning in many science, technology, engineering, and mathematics domains. We focus on two types of representational competencies: (1) sense making of connections by verbally explaining how different rep...
Article
Educational games and intelligent tutoring systems (ITS) both support learning by doing, although often in different ways. The current classroom experiment compared a popular commercial game for equation solving, DragonBox and a research-based ITS, Lynnette with respect to desirable educational outcomes. The 190 participating 7th and 8th grade stud...
Conference Paper
The gold standard for identifying more effective pedagogical approaches is to perform an experiment. Unfortunately, frequently a hypothesized alternate way of teaching does not yield an improved effect. Given the expense and logistics of each experiment, and the enormous space of potential ways to improve teaching, it would be highly preferable if...
Chapter
In working towards unraveling the mechanisms of productive collaborative learning, dual eye tracking is a potentially helpful methodology. Dual eye tracking is a method where eye-tracking data from people working on a task are analyzed jointly, for example to extract measures of joint visual attention. We explore how eye gaze relates to effective c...
Article
Full-text available
How might we combine intelligent tutors and educational games to increase learning and interest? Specifically, how might we effectively balance assistance and penalties in educational games? To investigate this question, we created two versions of an educational game for learning policy argumentation called Policy World. The game (only) version pro...
Conference Paper
Classroom experiments that evaluate the effectiveness of educational technologies do not typically examine the effects of classroom contextual variables (e.g., out-of-software help-giving and external distractions). Yet these variables may influence students' instructional outcomes. In this paper, we introduce the Spatial Classroom Log Explorer (SP...
Conference Paper
Intelligent tutoring systems (ITSs) are commonly designed to enhance student learning. However, they are not typically designed to meet the needs of teachers who use them in their classrooms. ITSs generate a wealth of analytics about student learning and behavior, opening a rich design space for real-time teacher support tools such as dashboards. W...
Article
Full-text available
Open Learner Models (OLMs) have great potential to support students’ Self-Regulated Learning (SRL) in Intelligent Tutoring Systems (ITSs). Yet few classroom experiments have been conducted to empirically evaluate whether and how an OLM can enhance students’ domain level learning outcomes through the scaffolding of SRL processes in an ITS. In two cl...
Article
Within educational data mining, many statistical models capture the learning of students working individually. However, not much work has been done to extend these statistical models of individual learning to a collaborative setting, despite the effectiveness of collaborative learning activities. We extend a widely used model (the additive factors...
Chapter
This chapter describes MOOC on educational data mining (EDM)/learning analytics, Big Data in education (referred to later as BDEMOOC in some cases). It also describes BDEMOOC's goals, its design and pedagogy, its content, and the research it afforded. Big Data in education was offered in its first version on the Coursera platform as one of the inau...
Conference Paper
Although learning with Intelligent Tutoring Systems (ITS) has been well studied, little research has investigated what role teachers can play, if empowered with data. Many ITSs provide student performance reports, but they may not be designed to serve teachers’ needs well, which is important for a well-designed dashboard. We investigated what stude...
Conference Paper
Making effective problem selection decisions is a challenging Self-Regulated Learning skill. Students need to learn effective problem-selection strategies but also develop the motivation to use them. A mastery-approach orientation is generally associated with positive problem selection behaviors such as willingness to work on new materials. We cond...
Conference Paper
Intelligent tutoring systems (ITS) and MOOCs tend to have complementary pedagogical approaches, but their combination is rarely (if ever) seen. A key obstacle may be technical integration. We present a generalizable case study of extending ITS authoring technology to make tutors easily embeddable into a variety of MOOC/e-learning platforms and run...
Conference Paper
For cross-pollination between ITS authoring tools, it may be useful to know the prevalence of tutoring behaviors crafted with these tools. As a case study, we analyze the problem units of Mathtutor, a web-based intelligent tutor for middle-school mathematics built, as an example-tracing tutor, with the Cognitive Tutor Authoring Tools (CTAT). We foc...
Article
Full-text available
The literature on concept formation has demonstrated that humans are capable of learning concepts incrementally, with a variety of attribute types, and in both supervised and unsupervised settings. Many models of concept formation focus on a subset of these characteristics, but none account for all of them. In this paper, we present TRESTLE, an inc...
Conference Paper
Learning-by-doing in MOOCs may be enhanced by embedding intelligent tutoring systems (ITSs). ITSs support learning-by-doing by guiding learners through complex practice problems while adapting to differences among learners. We extended the Cognitive Tutor Authoring Tools (CTAT), a widely-used non-programmer tool kit for building intelligent tutors,...
Article
In this position paper we contrast a Dystopian view of the future of adaptive collaborative learning support (ACLS) with a Utopian scenario that – due to better-designed technology, grounded in research – avoids the pitfalls of the Dystopian version and paints a positive picture of the practice of computer-supported collaborative learning 25 years...
Article
Full-text available
In 2009, we reported on a new Intelligent Tutoring Systems (ITS) technology, example-tracing tutors, that can be built without programming using the Cognitive Tutor Authoring Tools (CTAT). Creating example-tracing tutors was shown to be 4–8 times as cost-effective as estimates for ITS development from the literature. Since 2009, CTAT and its associ...
Article
Help seeking is an important process in self-regulated learning (SRL). It may influence learning with intelligent tutoring systems (ITSs), because many ITSs provide help, often at the student’s request. The Help Tutor was a tutor agent that gave in-context, real-time feedback on students’ help-seeking behavior, as they were learning with an ITS. Ke...
Article
Prior research shows that multiple representations can enhance learning, provided that students make connections among them. We hypothesized that support for connection making is most effective in enhancing learning of domain knowledge if it helps students both in making sense of these connections and in becoming perceptually fluent in making conne...
Article
Full-text available
Our 1997 article in IJAIED reported on a study that showed that a new algebra curriculum with an embedded intelligent tutoring system (the Algebra Cognitive Tutor) dramatically enhanced high-school students’ learning. The main motivation for the study was to demonstrate that intelligent tutors that have cognitive science research embedded in them c...
Conference Paper
Having insights into players' learning has important implications for design in an educational game. Empirical learning curve analysis is an approach from intelligent tutoring systems literature for measuring student learning within a system in terms of the skills involved. The approach can be used to evaluate how well different hypothesized models...
Conference Paper
A key challenge in ITS research and development is to support tutoring at scale, for example by embedding tutors in MOOCs. An obstacle to at-scale deployment is that ITS architectures tend to be complex, not easily deployed in browsers without significant server-side processing, and not easily embedded in a learning management system (LMS). We pres...
Conference Paper
Full-text available
Developing models of the knowledge and skills being exercised in a task is an important component of the design of any instructional environment. Developing these models is a labor intensive process. When working in exploratory and open-ended environments (EOLEs) the difficulty of building a knowledge model is amplified by the amount of freedom aff...
Conference Paper
To be able to provide better support for collaborative learning in Intelligent Tutoring Systems, it is important to understand how collaboration patterns change. Prior work has looked at the interdependencies between utterances and the change of dialogue over time, but it has not addressed how dialogue changes during a lesson, an analysis that allo...
Conference Paper
Making effective problem selection decisions is an important yet challenging self-regulated learning (SRL) skill. Although efforts have been made to scaffold students’ problem selection in intelligent tutoring systems (ITS), little work has tried to support students’ learning of the transferable problem selection skill that can be applied when the...
Conference Paper
We describe a study of naturalistic teacher guidance for an exploratory learning activity called Invention. Our study illustrates a specific pedagogical style, whereby the teacher offers little feedback and few explanations, but largely poses questions, to help students identify and remedy their own errors. These findings have informed the design o...