Jiayi Zhang

• University of Pennsylvania

Detection of carelessness in digital learning platforms has relied on the contextual slip model, which leverages conditional probability and Bayesian Knowledge Tracing (BKT) to identify careless errors, where students make mistakes despite having the knowledge. However, this model cannot effectively assess carelessness in questions tagged with mult...

This study explores the potential of the large language model GPT-4 as an automated tool for qualitative data analysis by educational researchers, exploring which techniques are most successful for different types of constructs. Specifically, we assess three different prompt engineering strategies-Zero-shot, Few-shot, and Few-shot with contextual i...

Detection of carelessness in digital learning platforms has relied on the contextual slip model, which leverages conditional probability and Bayesian Knowledge Tracing (BKT) to identify careless errors, where students make mistakes despite having the knowledge. However, this model cannot effectively assess carelessness in questions tagged with mult...

This paper examines the advantages and limitations of conducting automated coding of virtual tutoring session transcripts using the GPT-4 Turbo model via the OpenAI API. We compare three coding methods: (1) zero-shot, which relies solely on construct definitions; (2) few-shot, which includes annotated examples; and (3) coding with context, which pr...

Self-regulated learning (SRL) is essential for learning across various contexts and domains. While technology-based learning environments can support SRL, comparisons of SRL processes across learning platforms and domains are scarce. As most prior research has investigated SRL patterns across learner performance levels, methods are lacking to inves...

Digital learning games have been increasingly adopted in classrooms to facilitate learning and to promote learning outcomes. Contrary to common beliefs, many digital learning games can be more effective for female students than male students in terms of learning and affective outcomes. However, the in-game learning mechanisms that explain these dif...

In qualitative data analysis, codebooks offer a systematic framework for establishing shared interpretations of themes and patterns. While the utility of codebooks is well-established in educational research, the manual process of developing and refining codes that emerge bottom-up from data presents a challenge in terms of time, effort, and potent...

Programming courses can be challenging for first year university students, especially for those without prior coding experience. Students initially struggle with code syntax, but as more advanced topics are introduced across a semester, the difficulty in learning to program shifts to learning computational thinking (e.g., debugging strategies). Thi...

Manuscript available at: https://doi.org/10.1007/978-3-031-64315-6_2 Ethical issues matter for artificial intelligence in education (AIED). Simultaneously, there is a gap between fundamental ethical critiques of AIED research goals and research practices doing ethical good. This article discusses the divide between AIED ethics (i.e., critical soci...

Think-aloud protocols are a common method to study self-regulated learning (SRL) during learning by problem-solving. Previous studies have manually transcribed and coded students' verbalizations, labeling the presence or absence of SRL strategies and then examined these SRL codes in relation to learning. However, the coding process is difficult to...

When students procrastinate on programming assignments, it can hinder the quality of their code and negatively impact their grades. In contrast, when students actively delay working on assignments to prepare to code (e.g., reading or seeking help), it can be an effective self-regulated learning (SRL) strategy beneficial to programming performance....

This study uses Knowledge Engineering (KE) to develop an automated model of problem-solving strategy invention (PSSI) behavior (defined as inventing a new strategy for solving a math problem, outside of system-offered default strategies). The PSSI model identified the students inventing new strategies, and examined the relationship between PSSI beh...

This study uses Knowledge Engineering (KE) to develop an automated model of problem-solving strategy invention (PSSI) behavior (defined as inventing a new strategy for solving a math problem, outside of system-offered default strategies). The PSSI model identified the students inventing new strategies, and examined the relationship between PSSI beh...

Pedagogical agents offer significant promise for engaging students in learning. In this paper, we investigate students’ conversational interactions with a pedagogical agent in a game-based learning environment for middle school science education. We utilize word embeddings of student-agent conversations along with features distilled from students’...

Peer review, a commonly-used pedagogy in contemporary education has been found to positively influence student learning, benefitting both feedback provider and recipient. However, the quality of the feedback may vary, and lower-quality feedback (e.g., lacking specificity), is less likely to be implemented by the recipient, leading to suboptimal out...

We present detectors that automatically identify when students game the system, a maladaptive learning strategy where students attempt to succeed by exploiting properties of a learning environment. In contrast to previous detectors that detected this behavior within students' interaction with learning activities, we detect within students' text-bas...

The effects of educational anxiety have been observed across multiple disciplines; anxiety negatively influences cognition, self-regulation, performance, and educational outcomes. However, there has been limited research on anxiety within the context of interactive learning environments. In the current research, we expand this by assessing whether...

Understanding indicators in self-regulated learning (SRL) that affect mathematical success using quantitative techniques such as epistemic networks hold potential for providing effective scaffolds that draw directly from the learner’s perspective. Tied to learning success, SRL provides a range of frameworks for identifying students' affective, cogn...

Self-regulated learning (SRL) is a critical component of mathematics problem-solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model (Winne, 2017), prop...

Cohen's Kappa has been used in interrater reliability calculation for decades, often for small samples. Recently, QE researchers have argued that Kappa cannot validly be used without much larger samples based on very conservative assumptions: treating all degrees of error as equally problematic and conducting an analysis analogous to statistical po...

Considerable amount of research in educational data mining has focused on developing efficient algorithms for Knowledge Tracing (KT). However, in practice, many real-world learning systems used at scale struggle to implement KT capabilities, especially if they weren't originally designed for it. One key challenge is to accurately label existing ite...

Self-regulated learning (SRL) is a critical component of mathematics problem solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model, proposes that five...

In past work, time management interventions involving prompts, alerts, and planning tools have successfully nudged students in online courses, leading to higher engagement and improved performance. However, few studies have investigated the effectiveness of these interventions over time, understanding if the effectiveness maintains or changes based...

The effort to learn and the regulation of learning are key to successful learning. Voluntary practice has been shown to improve learning and is associated with having generally good self-regulated learning. At the same time, procrastination often slows the learning process and is associated with less than ideal regulation of learning. In this paper...

Previous studies on the accuracy of knowledge tracing models have typically considered the performance of all student actions. However, this practice ignores the difference between students' initial and later attempts on the same skill. To be effective for uses such as mastery learning, a knowledge tracing model should be able to infer student know...

Digital learning games are thought to support learning by increasing enjoyment and promoting deeper engagement with the content, but few studies have empirically tested hypothesized pathways between digital learning games and learning outcomes. Decimal Point, a digital learning game that teaches decimal operations and concepts to middle school stud...

Digital learning games are thought to support learning by increasing enjoyment and promoting deeper engagement with the content, but few studies have empirically tested hypothesized pathways between digital learning games and learning outcomes. Decimal Point, a digital learning game that teaches decimal operations and concepts to middle school stud...