Arthur C. Graesser

• University of Memphis

Large Language Model (LLM) agents are increasingly used in AI-supported education. Effective inter-agent communication boosts collaborative problem-solving efficiency and lowers the cost of deploying LLM-driven educational systems. However, few studies have systematically examined how different communication strategies affect on agents' problem-sol...

Artificial intelligence (AI) holds significant potential for enhancing student learning. This reflection critically examines the promises and limitations of AI for cognitive learning processes and outcomes, drawing on empirical evidence and theoretical insights from research on AI-enhanced education and digital learning technologies. We critically...

Transversal skills describe a broad spectrum of skills that are considered to be essential for thriving in today’s society and tackling the challenges of the twenty-first century. Therefore, a high demand is placed on educators to teach these skills to their students. Unfortunately, the conceptualization of transversal skills remains vague with dif...

Learner performance data collected by Intelligent Tutoring Systems (ITSs), such as responses to questions, is essential for modeling and predicting learners' knowledge states. However, missing responses due to skips or incomplete attempts create data sparsity, challenging accurate assessment and personalized instruction. To address this, we propose...

The advancement of multi-agent workflows leveraging Large Language Models (LLMs) for tackling complex tasks, such as mathematical problem-solving, has garnered significant interest among researchers. However, the communication strategies within these workflows remain underexplored, hindering comprehensive performance comparisons. This gap is partic...

Learning performance data, such as correct or incorrect answers and problem-solving attempts in Intelligent Tutoring Systems (ITSs), facilitate the assessment of knowledge mastery and the delivery of effective instructions. However, these data tend to be highly sparse (80% $\sim$ 90% missing observations) in most real-world applications. This data...

Complex problem solving (CPS) is a key competence in educational contexts with strong conceptual links to students' overall intelligence. However, the mechanisms underlying successful CPS are not fully understood. Therefore, this study investigated several factors presumed to be relevant to CPS success using log file data to code each individual st...

This paper assesses the ability of semantic text models to assess student responses to electronics questions compared with that of expert human judges. Recent interest in text similarity has led to a proliferation of models that can potentially be used for assessing student responses. However, it is unclear whether these models perform as well as e...

This research investigates the ability of semantic text models to assess student responses during tutoring compared with expert human judges. Recent interest in text similarity has led to a proliferation of models that can potentially be used for assessing student responses; however, whether these models perform as well as traditional distributiona...

We argue in this paper that there is currently no adequate theoretical framework or model that spans the twelve odd year trajectory from non‐reader to proficient reader, nor addresses fine‐grain skill acquisition, mastery and integration. The target construct itself, reading proficiency, as often operationalized as an endpoint of formal secondary s...

A large percentage of adults throughout the world have low reading skills. Computer technologies can potentially help these adults improve their literacy in addition to instructors at literacy centers. AutoTutor was designed to teach comprehension strategies by implementing conversational ‘trialogues’ in which two computer agents (tutor and peer) h...

A common goal of Intelligent Tutoring Systems (ITS) is to provide learning environments that adapt to the varying abilities and characteristics of users. This type of adaptivity is possible only if the ITS has information that characterizes the learning behaviors of its users and can adjust its pedagogy accordingly. This study investigated an intel...

Adult learners with low literacy skills compose a highly heterogeneous population in terms of demographic variables, educational backgrounds, knowledge and skills in reading, self-efficacy, motivation etc. They also face various difficulties in consistently attending offline literacy programs, such as unstable worktime, transportation difficulties,...

This paper describes a new automated disengagement tracking system (DTS) that detects learners’ maladaptive behaviors, e.g. mind-wandering and impetuous responding, in an intelligent tutoring system (ITS), called AutoTutor. AutoTutor is a conversation-based intelligent tutoring system designed to help adult literacy learners improve their reading c...

The recognized importance of computational thinking has helped to propel the rapid development of related educational efforts and programs over the past decade. Given the multi-faceted nature of computational thinking, which goes beyond programming and computer science, however, approaches and practices for developing students’ computational thinki...

Computational thinking is widely recognized as important, not only to those interested in computer science and mathematics but also to every student in the twenty-first century. However, the concept of computational thinking is arguably complex; the term itself can easily lead to direct connection with “computing” or “computer” in a restricted sens...

Intelligent Tutoring Systems (ITS) are computer learning environments that offer personalized instruction to help learners master knowledge and skills. ITSs are superior to typical computer-based training because their intelligent algorithms can adapt to the needs of the individual learner at a fine-grained level. However, most ITSs do not take adv...

Critical Thinking in Psychology - edited by Robert J. Sternberg January 2020

Design and design thinking are vital to creativity and innovation, and have become increasingly important in the current movement of developing and implementing integrated STEM education. In this editorial, we build on existing research on design and design thinking, and discuss how students’ learning and design thinking can be developed through de...

Design Recommendations for Intelligent Tutoring Systems (ITSs) explores the impact of intelligent tutoring system design on education and training. Specifically, this volume examines "Self-Improving Systems". The "Design Recommendations" book series examines tools and methods to reduce the time and skill required to develop Intelligent Tutoring Sys...

Coding is a process of assigning meaning to a given piece of evidence. Evidence may be found in a variety of data types, including documents, research interviews, posts from social media, conversations from learning platforms, or any source of data that may provide insights for the questions under qualitative study. In this study, we focus on text...

No abstract available for this critical reflection. Please request full text.

No abstract available for this editorial. Please request full text.

Relatedness between user input and an ideal response is a salient feature required for proper functioning of an Intelligent Tutoring System (ITS) using natural language processing. Improper assessment of text input causes maladaptation in ITSs. Meta-assessment of user responses in ITSs can improve instruction efficacy and user satisfaction. Therefo...

The importance of collaboration is growing in an era where knowledge-based tasks are increasingly accomplished by teams of people with complementary roles and expertise, as opposed to individuals doing isolated work. Moreover, the nature of collaboration is shifting to a more sophisticated skillset that includes accomplishing tasks through mediated...

This article introduces three distinctive features of a conversation-based intelligent tutoring system called AutoTutor. AutoTutor was designed to teach low literacy adult learners comprehension strategies across different levels of discourse processing. In AutoTutor, three-way conversations take place between two computers agents (a teacher agent...

In this paper, we consider a minimalistic and behavioristic view of AIS to enable a standardizable mapping of both the behavior of the system and of the learner. In this model, the learners interact with the learning resources in a given learning environment following preset steps of learning processes. From this foundation, we make several subsequ...

Standardization of unstructured information such as freely generated verbal responses in adaptive instructional systems poses many challenges. For instance, free responses have no clear delimitation of knowledge components, i.e., basic learning units (BLU), and therefore identifying the BLUs in such responses automatically is a challenge. We will r...

This paper describes the concept of a hybrid tutor as a type of adaptive instructional system (AIS). A hybrid tutor is a confederation of several digital learning resources and human interactions so that the right resource is available to the learner at the right time. We discuss a method for combining several existing educational technologies into...

Conversational Intelligent Tutoring Systems (ITSs) are expensive to develop. While simple online courseware could be easily authored by teachers, the authoring of conversational ITSs usually involves a team of experts with different expertise, including domain experts, linguists, instruction designers, programmers, artists, computer scientists, etc...

One out of six adults in the United States possesses low literacy skills. Many advocates believe that technology can pave the way for these adults to gain the skills that they desire. This article describes an adaptive intelligent tutoring system called AutoTutor that is designed to teach adults comprehension strategies across different levels of d...

The rapidly evolving and global field of STEM education has placed ever-increasing calls for interdisciplinary research and the development of new and deeper scholarship in and for STEM education. In this editorial, we focus on the topic of thinking, first with a brief overview of related studies and conceptions in the past. We then problematize a...

This chapter describes the testing of the computer-human interface of Virtual Internship Authorware (VIA), an authoring tool for creating web-based virtual internships. The authors describe several benchmark tasks that would be performed by authors who create lessons on the subject matter of land science. Performance on each task was measured by ta...

This special issue presents evaluations of four intelligent tutoring systems. These systems were funded under the Office of Naval Research’s STEM Grand Challenge for intelligent tutoring systems. The systems each represent aspects of how ITS can address STEM education or how aspects of multiple systems can be integrated to support STEM education. T...

Assessment and Learning in Knowledge Spaces (ALEKS) is one of the widely used online intelligent tutoring systems (ITS) in the USA, but it has rarely been included in meta-analyses of ITS efficacy to help students learn. We conducted a meta-analysis to assess the effectiveness of ALEKS on learning. A total of 15 empirical studies were conducted bet...

Collaborative problem solving (CPS) has been receiving increasing international attention because much of the complex work in the modern world is performed by teams. However, systematic education and training on CPS is lacking for those entering and participating in the workforce. In 2015, the Programme for International Student Assessment (PISA),...

An intelligent agent can play a significant role in interactive learning, assessment, and teamwork (Bay-lor, 2011; Johnson, Phillips, & Chase, 2009; Chou, Chan, & Lin, 2003; Johnson & Lester, 2016; Kumar, Ai, Beuth, & Rosé, 2010; Moreno, Mayer, Spires & Lester., 2001; Schroeder, Adesope, & Gilbert, 2013). Like Barrón-Estrade, Zatarain-Cabada, Orama...

Rus, V., Gautam, D., Bowman, D., Graesser, A. C., & Shaffer, D. W. (2017, January). Markov analysis of students’ professional skills in virtual internships. In Proceedings of the 30th Florida Artificial Intelligence Research Society Conference.

Reading comprehension is often assessed by having students read passages and administering a test that assesses their understanding of the text. Shorter assessments may fail to give a full picture of comprehension ability while more thorough ones can be time consuming and costly. This study used data from a conversational intelligent tutoring syste...

Technological advancements have facilitated the implementation of the core components of intelligent tutoring systems (ITS) that adapt to individual students. Some of these ITSs attempt to simulate the discourse and pedagogical strategies of human tutors. Natural language processing tools and agent-based software are increasingly working alongside...

ElectronixTutor is a new Intelligent Tutoring System for electronics that integrates multiple intelligent learning resources, including AutoTutor, Dragoon, LearnForm, ASSISTments, and BEETLE-II, as well as Point & Query hotspots on diagrams and numerous text documents on the subject of electronics. ElectronixTutor's student model contains a set of...

Knowledge decays across breaks in instruction. Learners lack the metacognition to self-assess their knowledge decay and effectively self-direct review, as well as lacking interactive exercises appropriate to their individual knowledge level. Adaptive learning systems offer the potential to mitigate these issues, by providing open learner models to...

This work is a step towards full automation of auto-mentoring processes in multi-player online environments such as virtual internships. We focus on automatically identifying speaker's intentions, i.e. the speech acts of chat utterances, in such virtual internships. Particularly, we explore several machine learning methods to categorize speech acts...

This paper describes a novel automated disengagement tracing system (DTS) that detects mind wandering in students using AutoTutor, an Intelligent Tutoring System (ITS) with conversational agents. DTS is based on an un-supervised learning method and thus does not rely on any self-reports of disengagement. We analyzed the reading time and response ac...

Background The Office of Naval Research (ONR) organized a STEM Challenge initiative to explore how intelligent tutoring systems (ITSs) can be developed in a reasonable amount of time to help students learn STEM topics. This competitive initiative sponsored four teams that separately developed systems that covered topics in mathematics, electronics,...

Video-based self-reflection and annotation is receiving increasing attention within the education literature. The importance of such technologies in education relate, in part, to the interactive nature and functionality these tools bring to aid learning engagement. In particular, these tools are well aligned with the need to promote and develop stu...

In this paper, we conduct a Markov analysis of learners' professional skill development based on their conversations in virtual internships, an emerging category of learning systems characterized by the epistemic frame theory. This theory claims that professionals develop epistemic frames, or the network of skills, knowledge, identity, values, and...

Vector based word representation models are typically developed from very large corpora with the hope that the representations are reliable and have wide coverage, i.e. they cover, ideally, all words. However, we often encounter words in real world applications that are not available in a single vector-based model. In this paper, we present a novel...

Virtual internships are online simulations of professional practice where students play the role of interns at a fictional company. During virtual internships, participants complete activities and then submit write-ups in the form of short answers, digital notebook entries. Prior work used classifiers trained on participant data to automatically as...

In this study we developed and evaluated a crowdsourcing-based latent semantic analysis (LSA) approach to computerized summary scoring (CSS). LSA is a frequently used mathematical component in CSS, where LSA similarity represents the extent to which the to-be-graded target summary is similar to a model summary or a set of exemplar summaries. Resear...

This chapter comments on the contributions in this edited volume and identifies some challenges for future research on serious games. The contributors used rigorous experimental methods to systematically assess the impact of many components of serious games on learning and motivation. The games are serious because there is alignment with relevant i...

This chapter describes some attempts to promote deep learning (as opposed to shallow learning) through conversational pedagogical agents. Learning environments with agents have been developed to serve as substitutes for humans who range in expertise from novices to experts. For example, AutoTutor helps students learn by holding a dialogue in natura...

Despite a surge of empirical work on student participation in online learning environments, the causal links between the learning-related factors and processes with the desired learning outcomes remain unexplored. This study presents a systematic literature review of approaches to model learning in Massive Open Online Courses offering an analysis o...

The Generalized Intelligent Framework for Tutoring (GIFT) is a research prototype with three general goals associated with its functions and components: 1) lower the skills and time required to author Intelligent Tutoring Systems (ITSs) in a variety of task domains; 2) provide effective adaptive instruction tailored to the needs of each individual...

This book is the fifth in a planned series of books that examine key topics (e.g., learner modeling, instruc-tional strategies, authoring, domain modeling, assessment, impact on learning, team tutoring, machine learning, and potential standards) in intelligent tutoring system (ITS) design through the lens of the Generalized Intelligent Framework fo...

Prior research has shown that students learn from Intelligent Tutoring Systems (ITS). However, students’ attention may drift or become disengaged with the task over extended amounts of instruction. To remedy this problem, researchers have examined the impact of game-like features (e.g., a narrative) in digital learning environments on motivation an...

This article describes conversation-based assessments with computer agents that interact with humans through chat, talking heads, or embodied animated avatars. Some of these agents perform actions, interact with multimedia, hold conversations with humans in natural language, and adaptively respond to a person’s actions, verbal contributions, and em...

Background: The Office of Naval Research (ONR) organized a STEM challenge initiative to explore how intelligent tutoring systems (ITS) can be developed in a reasonable amount of time to help students learn STEM topics. This competitive initiative sponsored four teams that separately developed systems that covered topics in mathematics, electronics,...

In the United States, more than 15 % of adults are struggling with low literacy skills. Adults who struggle with reading are a heterogeneous group with an extremely varied set of skills and experiences. Difficult circumstances often dictate their ability to attend classes regularly thus leading to problems in their ability to engage with and retain...

The focus of ElectronixTutor is to build an intelligent tutoring system technology for Navy-relevant applications in training. The goal is to have an ITS for Apprentice Technician Training (ATT) courses in electronics for naval trainees who have completed boot camp and are in the process of A-school training under the Navy Educational Training Comm...

The Center for the Study of Adult Literacy (CSAL) seeks to improve our understanding of ways to advance the reading skills of adult learners. Our web-based instructional tutor uses trialogues in the AutoTutor framework to deliver lessons in reading comprehension. We have found a way to manipulate proven comprehension strategies to fit the daily tas...

In this paper, we applied the crowdsourcing approach to develop an automated popularity summary scoring, called wild summaries. In contrast, the golden standard summaries generated by one or more experts are called expert summaries. The innovation of our study is to compute LSA (Latent Semantic Analysis) similarities between target summary and wild...

AutoTutor helps students learn by holding a conversation in natural language. AutoTutor is adaptive to the learners’ actions, verbal contributions, and in some systems their emotions. Many of AutoTutor’s conversation patterns simulate human tutoring, but other patterns implement ideal pedagogies that open the door to computer tutors eclipsing human...

This chapter explores the prospects of integrating games with intelligent tutoring systems (ITSs). The hope is that there can be learning environments that optimize both motivation through games and deep learning through ITS technologies. Deep learning refers to the acquisition of knowledge, skills, strategies, and reasoning processes at the higher...

We examined the coherence of trauma memories in a trauma-exposed community sample of 30 adults with and 30 without posttraumatic stress disorder. The groups had similar categories of traumas and were matched on multiple factors that could affect the coherence of memories. We compared the transcribed oral trauma memories of participants with their m...

Advances in computational linguistics and discourse science have made it possible to analyze conversation on multiple levels of language and discourse. Conversation patterns have been identified when humans collaboratively solve problems in tutorial dialogue and in small groups. Naturalistic patterns of collaboration have also been compared with th...

Engagement during reading can be measured by the amount of time readers invest in the reading process. It is hypothesized that disengagement is marked by a decrease in time investment as compared with the demands made on the reader by the text. In this study, self-paced reading times for screens of text were predicted by a text complexity score cal...

This paper investigates how the peer agent’s learning competency affects English learners’ reading, engagement, system self-efficacy, and attitudes toward the peer agent in a trialogue-based intelligent tutoring system (ITS). Participants learned a summarizing reading strategy in the compare-contrast text structure in the ITS. Results detected the...

We present the design of a novel conversational intelligent tutoring system, called DeepTutor. DeepTutor is based on cognitive theories of learning, the framework of Learning Progressions proposed by the science education research community, and deep natural language and dialogue processing techniques and principles. The focus of the paper is on th...

We manipulated three types of short feedback (emotional, epistemic, and neutral) in an intelligent tutoring system designed to help struggling adult readers improve reading comprehension strategies. Although participants self-reported a preference for emotional feedback, there were no differences in individual motivation or usefulness ratings betwe...

This paper reports initial results of an evaluation for an ITS that follows service-oriented principles to integrate natural language tutoring into an existing adaptive learning system for mathematics. Self-explanation tutoring dialogs were used to talk students through step-by-step worked solutions to Algebra problems. These worked solutions prese...

Integrating work from psychology, computational linguistics, and political science, we explore the relationship between political party and linguistic intergroup bias. We investigated linguistic patterns in the discourse of 91 Democratic and 96 Republican senate members. Senate member speeches (N = 229,526) delivered between 1989-2006 were analyzed...

Inferencing is defined as 'the act of deriving logical conclusions from premises known or assumed to be true', and it is one of the most important processes necessary for successful comprehension during reading. This volume features contributions by distinguished researchers in cognitive psychology, educational psychology, and neuroscience on topic...

The goal of this article is to preserve and distribute the information presented at the LASI (2014) workshop on Coh-­Metrix , a theoretically grounded , computational linguistics facility that analyzes texts on multiple levels of language and discourse. The workshop focused on the utility of Coh-­‐Metrix in discourse theory and educational practice...

Recent research has shown that natural disasters present political problems for societies, as these events make both citizens and leaders vulnerable. Autocratic leaders use language strategically following natural disasters to maximize their time in office. We introduce a new data set derived from using computational linguistic programs (LIWC and C...

Connections established between learners via interactions are seen as fundamental for connectivist pedagogy. Connections can also be viewed as learning outcomes, i.e. learners' social capital accumulated through distributed learning environments. We applied linear mixed effects modeling to investigate whether the social capital accumulation interpr...

Recent studies have used Coh-Metrix, an automated text analyzer, to assess differences in language characteristics across different genres and academic disciplines (Graesser, McNamara, & Kulikowich, 2011; McNamara, Graesser, McCarthy, & Cai, 2014). Coh-Metrix analyzes text on many constructs at different levels, including Word Concreteness (vs. abs...

Formality has long been of interest in the study of discourse, with periodic discussions of the best measure of formality and the relationship between formality and text categories. In this research, we explored what features predict formality as humans perceive the construct. We categorized a corpus consisting of 1158 discourse samples published i...