Danielle McNamara

Danielle McNamara
Arizona State University | ASU · Department of Psychology

PhD Experimental Psychology, MA Clinical Psychology, BA Linguistics

About

466
Publications
192,459
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
18,951
Citations
Introduction
Danielle S. McNamara, Ph.D., is a Professor of Psychology in the Psychology Department and Senior Scientist in the Institute for the Science of Teaching and Learning at Arizona State University. She is an international expert in the fields of cognitive and learning sciences, comprehension, literacy, writing, natural language processing, and intelligent tutoring systems. Much of her research employs computational linguistics—such as Natural Language Processing (NLP) techniques—as a means of analyzing discourse. Such tools allow for quick, efficient, and reliable analyses of large corpuses of text, which is particularly relevant and valuable when analyzing big data. Over the past few years, Dr. McNamara has been involved in the development and testing of multiple NLP tools that have been used in various research projects involving essay writing, reading comprehension, second language learning, and creativity. Additionally, she has applied these tools to intelligent tutoring systems, the Writing Pal and iSTART, in order to assess students’ written responses and provide automated feedback. iSTART, provides instruction and adaptive practice on reading comprehension strategies, and the Writing Pal provides instruction and practice to use writing strategies for persuasive writing. These systems are appropriate and effective with middle and high school students, and are now being adapted for adult literacy learners. She continues to explore how NLP tools can be applied to other learning environments such as computer supported collaborative learning environments and massive open online courses (MOOCs). Visit www.soletlab.com for more information.

Publications

Publications (466)
Article
Full-text available
The goal of this study was to assess the relationships between computational approaches to analyzing constructed responses made during reading and individual differences in the foundational skills of reading in college readers. We also explored if these relationships were consistent across texts and samples collected at different institutions and t...
Article
Few studies have explored how general skills in both reading and writing influence performance on integrated, source-based writing. The goal of the present study was to consider the relative contributions of reading and writing ability on multiple-document integrative reading and writing tasks. Students in the U.S. (n = 94) completed two tasks in w...
Article
The present study examined the extent to which adaptive feedback and just-in-time writing strategy instruction improved the quality of high school students’ persuasive essays in the context of the Writing Pal (W-Pal). W-Pal is a technology-based writing tool that integrates automated writing evaluation into an intelligent tutoring system. Students...
Article
The benefits of writing strategy feedback are well established. This study examined the extent to which adding spelling and grammar checkers support writing and revision in comparison to providing writing strategy feedback alone. High school students (n = 119) wrote and revised six persuasive essays in Writing Pal, an automated writing evaluation a...
Article
Full-text available
Age of acquisition (AoA) is a measure of word complexity which refers to the age at which a word is typically learned. AoA measures have shown strong correlations with reading comprehension, lexical decision times, and writing quality. AoA scores based on both adult and child data have limitations that allow for error in measurement, and increase t...
Article
Little quantitative research has explored which clinician skills and behaviors facilitate communication. Mutual understanding is especially challenging when patients have limited health literacy (HL). Two strategies hypothesized to improve communication include matching the complexity of language to patients’ HL (“universal tailoring”); or always u...
Article
Full-text available
Learning to paraphrase supports both writing ability and reading comprehension, particularly for less skilled learners. As such, educational tools that integrate automated evaluations of paraphrases can be used to provide timely feedback to enhance learner paraphrasing skills more efficiently and effectively. Paraphrase identification is a popular...
Article
Full-text available
Successful text comprehension requires readers to engage in a number of coherence-building processes. This study examined how analyzing the cohesion of students’ constructed responses can be used to evaluate these coherence-building processes and the extent to which they vary across readers’ individual differences and across types of texts. We pose...
Article
Full-text available
Synthesis writing is widely taught across domains and serves as an important means of assessing writing ability, text comprehension, and content learning. Synthesis writing differs from other types of writing in terms of both cognitive and task demands because it requires writers to integrate information across source materials. However, little is...
Chapter
The ability to automatically assess the quality of paraphrases can be very useful for facilitating literacy skills and providing timely feedback to learners. Our aim is twofold: a) to automatically evaluate the quality of paraphrases across four dimensions: lexical similarity, syntactic similarity, semantic similarity and paraphrase quality, and b)...
Chapter
Text summarization is an effective reading comprehension strategy. However, summary evaluation is complex and must account for various factors including the summary and the reference text. This study examines a corpus of approximately 3,000 summaries based on 87 reference texts, with each summary being manually scored on a 4-point Likert scale. Mac...
Chapter
iSTART is a game-based intelligent tutoring system (ITS) designed to improve students’ reading skills by providing training on reading comprehension strategies. Game-based practice in iSTART follows two main approaches: generative practice and identification practice. Generative practice games ask students to author self-explanations using one or m...
Chapter
The ability to objectively quantify the complexity of a text can be a useful indicator of how likely learners of a given level will comprehend it. Before creating more complex models of assessing text difficulty, the basic building block of a text consists of words and, inherently, its overall difficulty is greatly influenced by the complexity of u...
Article
We compared high school students' performance in a traditional comprehension assessment requiring them to identify key information and draw inferences from single texts, and a scenario-based assessment (SBA) requiring them to integrate, evaluate and apply information across multiple sources. Both assessments focused on a non-academic topic. Perform...
Chapter
Dialogism is a philosophical theory centered on the idea that life involves a dialogue among multiple voices in a continuous exchange and interaction. Considering human language, different ideas or points of view take the form of voices, which spread throughout any discourse and influence it. From a computational point of view, voices can be operat...
Chapter
Reading comprehension is key to knowledge acquisition and to reinforcing memory for previous information. While reading, a mental representation is constructed in the reader’s mind. The mental model comprises the words in the text, the relations between the words, and inferences linking to concepts in prior knowledge. The automated model of compreh...
Conference Paper
Automated Writing Evaluation systems have been developed to help students improve their writing skills through the automated delivery of both summative and formative feedback. These systems have demonstrated strong potential in a variety of educational contexts; however, they remain limited in their personalization and scope. The purpose of the cur...
Article
Full-text available
Literacy skills are critical for future success, yet over 60% of high school seniors lack proficient reading skills according to standardized tests. The focus on high stakes, standardized test performance may lead educators to “teach-to-the-test” rather than supporting transferable comprehension strategies that students need. StairStepper can fill...
Article
The COVID-19 pandemic has changed the entire world, while the impact and usage of online learning environments has greatly increased. This paper presents a new version of the ReaderBench framework, grounded in Cohesion Network Analysis, which can be used to evaluate the online activity of students as a plug-in feature to Moodle. A Recurrent Neural...
Article
Prior knowledge is one of the strongest contributors to comprehension, but there is little specificity about different aspects of prior knowledge and how they impact comprehension. This article introduces the Multidimensional Knowledge in Text Comprehension framework, which conceptualizes prior knowledge along four intersecting dimensions: amount,...
Chapter
This chapter describes development efforts that build upon the Interactive Strategy Trainer for Active Reading and Thinking-2 (iSTART-2), an intelligent tutoring system that provides self-explanation strategy instruction to improve reading comprehension. The chapter reflects on considerations of the unique needs of adult literacy learners, and outl...
Article
Limited health literacy (HL) partially mediates health disparities. Measurement constraints, including lack of validity assessment across racial/ethnic groups and administration challenges, have undermined the field and impeded scaling of HL interventions. We employed computational linguistics to develop an automated and novel HL measure, analyzing...
Article
Objective In the National Library of Medicine funded ECLIPPSE Project (Employing Computational Linguistics to Improve Patient-Provider Secure Emails exchange), we attempted to create novel, valid, and scalable measures of both patients’ health literacy (HL) and physicians’ linguistic complexity by employing natural language processing (NLP) techniq...
Article
The substantial expansion of secure messaging (SM) via the patient portal in the last decade suggests that it is becoming a standard of care, but few have examined SM use longitudinally. We examined SM patterns among a diverse cohort of patients with diabetes (N = 19 921) and the providers they exchanged messages with within a large, integrated hea...
Article
Full-text available
iSTART-E is a web-based intelligent tutor developed for Spanish-speaking students to improve their reading comprehension through self-explanation strategy training. This study examined the effects of a blended comprehension strategy intervention on students' reading comprehension skill. Chilean high school students (n = 22) completed nine iSTART-E...
Article
Background: Low literacy skills impact important aspects of communication, including health-related information exchanges. Unsuccessful communication on the part of physician or patient contributes to lower quality of care, is associated with poorer chronic disease control, jeopardizes patient safety and can lead to unfavorable healthcare utilizati...
Article
Full-text available
Objective: To develop novel, scalable, and valid literacy profiles for identifying limited health literacy patients by harnessing natural language processing. Data source: With respect to the linguistic content, we analyzed 283 216 secure messages sent by 6941 diabetes patients to physicians within an integrated system's electronic portal. Socio...
Article
Full-text available
Computer-based learning environments (CBLEs) provide unprecedented opportunities for personalized learning at scale. One such system, iSTART (Interactive Strategy Training for Active Reading and Thinking) is an adaptive, game-based tutoring system for reading comprehension. This paper describes how efforts to increase personalized learning have imp...
Chapter
Full-text available
Reading comprehension requires readers to connect ideas within and across texts to produce a coherent mental representation. One important factor in that complex process regards the cohesion of the document(s). Here, we tackle the challenge of providing researchers and practitioners with a tool to visualize text cohesion both within (intra) and bet...
Chapter
Full-text available
Theories of discourse argue that comprehension depends on the coherence of the learner’s mental representation. Our aim is to create a reliable automated representation to estimate readers’ level of comprehension based on different productions, namely self-explanations and answers to open-ended questions. Previous work relied on Cohesion Network An...
Chapter
Full-text available
A key writing skill is the capability to clearly convey desired meaning using available linguistic knowledge. Consequently, writers must select from a large array of idioms, vocabulary terms that are semantically equivalent, and discourse features that simultaneously reflect content and allow readers to grasp meaning. In many cases, a simplified ve...
Article
For decades, educators have relied on readability metrics that tend to oversimplify dimensions of text difficulty. This study examines the potential of applying advanced artificial intelligence methods to the educational problem of assessing text difficulty. The combination of hierarchical machine learning and natural language processing (NLP) is l...
Chapter
Full-text available
Online collaborative learning environments open new research opportunities, for example, the analysis of learning outcomes, the identification of learning patterns, the prediction of students’ behaviors, and the modeling and visualization of social relations and trends among students. Moodle is an online educational platform which supports both stu...
Article
Patients with diabetes and limited health literacy (HL) may have suboptimal communication exchange with their health care providers and be at elevated risk of adverse health outcomes. These difficulties are generally attributed to patients’ reduced ability to both communicate and understand health-related ideas as well as physicians’ lack of skill...
Article
Full-text available
Literacy is a critically important and contemporary issue for educators, scientists, and politicians. Efforts to overcome the challenges associated with illiteracy, and the subsequent development of literate societies, are closely related to those of poverty reduction and sustainable human development. In this paper, the authors examine literacy fr...
Article
Full-text available
In this inter-disciplinary study, researchers from three departments analyzed a corpus of journal abstracts written by American, British, or Japanese scientists. The purpose of our study was to shed light on differences between the writing of Japanese scientists and the writing of their native English speaking colleagues. The analysis was conducted...
Chapter
Intelligent Tutoring Systems (ITSs) focus on promoting knowledge acquisition, while providing relevant feedback during students’ practice. Self-explanation practice is an effective method used to help students understand complex texts by leveraging comprehension. Our aim is to introduce a deep learning neural model for automatically scoring student...
Article
Background Little is known about patients who have caregiver proxies communicate with healthcare providers via portal secure messaging (SM). Since proxy portal use is often informal (e.g., sharing patient accounts), novel methods are needed to estimate the prevalence of proxy-authored SMs. Objective (1) Develop an algorithm to identify proxy-autho...
Chapter
Full-text available
Summarization is an effective strategy to promote and enhance learning and deep comprehension of texts. However, summarization is seldom implemented by teachers in classrooms because the manual evaluation of students’ summaries requires time and effort. This problem has led to the development of automated models of summarization quality. However, t...
Chapter
Full-text available
Theories of discourse comprehension assume that understanding is a process of making connections between new information (e.g., in a text) and prior knowledge, and that the quality of comprehension is a function of the coherence of the mental representation. When readers are exposed to multiple sources of information, they must make connections bot...
Chapter
This study investigated the effect of incorporating spelling and grammar checking tools within an automated writing tutoring system, Writing Pal. High school students (n = 119) wrote and revised six persuasive essays. After initial drafts, all students received formative feedback about writing strategies. Half of the participants were also given ac...
Conference Paper
Full-text available
This research paper describes our tool for loading and analyzing Piazza data. Piazza is one of the most popular tools for communication in classes. It makes the question answering among students and between students and teaching staff easier and more organized. In addition, analyzing Piazza data can provide us with more insight on how the students...
Article
Full-text available
The assessment of argumentative writing generally includes analyses of the specific linguistic and rhetorical features contained in the individual essays produced by students. However, researchers have recently proposed that an individual’s ability to flexibly adapt the linguistic properties of their writing may more accurately capture their profic...
Article
Full-text available
This study explores whether linguistic features can predict second language writing proficiency in the Michigan English Language Assessment Battery (MELAB) writing tasks. Advanced computational tools were used to automatically assess linguistic features related to lexical sophistication, syntactic complexity, cohesion, and text structure of writing...
Conference Paper
Adapting tasks to the individual has been shown to improve learning, and improve learners' experiences in the learning process. This study investigated how adaptive task selection affected learners' experiences in iSTART, an intelligent tutoring system for improving reading comprehension. Participants (n = 59) engaged with iSTART for 7 hours across...
Conference Paper
Full-text available
This study examines the extent to which instructions to self-explain vs. other-explain a text lead readers to produce different forms of explanations. Natural language processing was used to examine the content and characteristics of the explanations produced as a function of instruction condition. Undergraduate students (n = 146) typed either self...
Article
Full-text available
Limited health literacy is a barrier to optimal healthcare delivery and outcomes. Current measures requiring patients to self-report limitations are time-consuming and may be considered intrusive by some. This makes widespread classification of patient health literacy challenging. The objective of this study was to develop and validate "literacy pr...
Data
Survey questions coding and definitions. (PDF)
Article
Full-text available
We explored relations between reading comprehension performance and self-reported components of metacognition in middle-school children. Students' self-reported metacognitive strategies in planning and evaluation accounted for significant variance in reading comprehension performance on questions involving inferences. In Study 2, middle school stud...
Preprint
Full-text available
Limited health literacy can be a barrier to healthcare delivery, but widespread classification of patient health literacy is challenging. We applied natural language processing and machine learning on a large sample of 283,216 secure messages sent from 6,941 patients to their clinicians for this study to develop and validate literacy profiles as in...
Article
This study evaluates high school students’ perceptions of automated writing feedback, and the influence of these perceptions on revising, as a function of varying modes of computer-based writing instruction. Findings indicate that students’ perceptions of automated feedback accuracy, ease of use, relevance, and understandability were favorable. Imm...
Chapter
Full-text available
This study assesses the extent to which machine learning techniques can be used to predict question quality. An algorithm based on textual complexity indices was previously developed to assess question quality to provide feedback on questions generated by students within iSTART (an intelligent tutoring system that teaches reading strategies). In th...
Chapter
Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a sco...
Chapter
Full-text available
This study examines math success within a blended undergraduate course using a Cohesion Network Analysis (CNA) approach while controlling for individual differences and click-stream variables that may also predict math success. Linear models indicated that math success was related to days spent on the forum and by students who more regularly posted...
Chapter
Massive Open Online Courses (MOOCs) have become an important platform for teaching and learning because of their ability to deliver educational accessibility across time and distance. Online learning environments have also provided new research opportunities to examine learning success at a large scale. One data tool that has been proven effective...
Chapter
Interactive Strategy Training for Active Reading and Thinking en Español, or iSTART-E, is a new intelligent tutoring system (ITS) that provides reading comprehension strategy training for Spanish speakers. This paper reports on studies evaluating the efficacy of iSTART-E in real-world classrooms in two different Spanish-speaking countries. In Study...
Conference Paper
While hierarchical machine learning approaches have been used to classify texts into different content areas, this approach has, to our knowledge, not been used in the automated assessment of text difficulty. This study compared the accuracy of four classification machine learning approaches (flat, one-vs-one, one-vs-all, and hierarchical) using na...
Article
Background knowledge is a strong predictor of reading comprehension, yet little is known about how different types of background knowledge affect comprehension. The study investigated the impacts of both domain and topic-specific background knowledge on students’ ability to comprehend and learn from science texts. High school students (n = 3,650) c...
Article
A critical challenge for computer-based writing instruction is providing appropriate and adaptive practice. The current study examined three modes of computer-based writing practice with the goal of identifying those with the greatest learning and motivational value. High school students learned about writing strategies by studying lessons within t...
Conference Paper
Self-explanations are commonly used to assess on-line reading comprehension processes. However, traditional methods of analysis ignore important temporal variations in these explanations. This study investigated how dynamical systems theory could be used to reveal linguistic patterns that are predictive of self-explanation quality. High school stud...