Carolin Hahnel

• Professor
• Ruhr University Bochum

Background Learning analytics dashboards (LAD) have been developed as feedback tools to help students self‐regulate their learning (SRL) by using the large amounts of data generated by online learning platforms. Despite extensive research on LAD design, there remains a gap in understanding how learners make sense of information visualised on LADs a...

Online evaluation skills such as assessing the credibility and relevance of Internet sources are crucial for students' self-regulated learning on the Internet, yet many struggle to identify reliable information online. While AI-based chatbots have made progress in teaching various skills, their application in improving online evaluation skills rema...

Learning in asynchronous online settings (AOSs) is challenging for university students. However, the construct of learning engagement (LE) represents a possible lever to identify and reduce challenges while learning online, especially, in AOSs. Learning analytics provides a fruitful framework to analyze students' learning processes and LE via trace...

Learning in asynchronous online settings (AOSs) is challenging for university students. However, the construct of learning engagement (LE) represents a possible lever to identify and reduce challenges while learning online, especially, in AOSs. Learning analytics provides a fruitful framework to analyze students' learning processes and LE via trace...

Learning Analytics Dashboards (LAD) have been developed as feedback tools to help students self-regulate their learning (SRL), using the large amounts of data generated by online learning platforms. Despite extensive research on LAD design, there remains a gap in understanding how learners make sense of information visualised on LADs and how they s...

With the ongoing digitalization of home automation and appliances, users often face situations in which they lack the information required to reach an intended goal when using an unfamiliar device. To solve such technical problems, users need to acquire and apply operational knowledge in a self-regulated way. Our study assesses skills in technical...

Das Leseverständnis multipler Texte, das über das Leseverständnis einzelner Texte hinausgeht, ist unerlässlich, um Informationen aus mehreren textlichen Quellen sammeln, bewerten und integrieren zu können. Multiple Texte begegnen Leserinnen und Lesern etwa bei einer Internetrecherche oder beim selbstständigen Erarbeiten eines Themas, wie es von Stu...

The efficiency of cognitive component skills is typically assessed with speeded performance tests. Interpreting only effective ability or effective speed as efficiency may be challenging because of the within-person dependency between both variables (speed-ability tradeoff, SAT). The present study measures efficiency as effective ability conditiona...

In Germany, fifteen-year-olds achieved an average of 480 points in reading literacy. This means that reading literacy in 2022 does not differ significantly from the OECD average. Compared to the German averages from the 2018 and 2012 PISA studies, the average reading literacy score in Germany in 2022 has fallen significantly. The group of particula...

Zusammenfassung: Der Beitrag präsentiert die Entwicklung einer Kurzskala zur Erfassung technikbezogener Einstellungen sowie Selbstwahrnehmungen im Kontext der Nutzung von Alltagstechnik. Zur Validierung der Interpretation der resultierenden Testwerte wurden die interne Struktur der Skala, Beziehungen zu Drittvariablen sowie die Annahme geprüft, das...

Following an extended perspective of evidence-centered design, this study provides a methodological exemplar of the theory-based construction of process indicators from log data. We investigated decision-making processes in web search as the target construct, assuming that individuals follow a heuristic search (focusing on search results vs. websit...

Multiple document comprehension (MDC) is an essential skill for university students, making it beneficial to improve it. We assume that the value assigned to MDC is important to foster MDC since it can be a predictor for behavioral choices, such as working with self-study material. Using self-study material is typical for university learning, but i...

Contributions in the Special Issue The special issue assembles papers centring around log data analysis, natural language processing, and machine learning used to advance educational assessment. They demonstrate how semi‐ and unstructured data such as log and text data can, despite their challenging nature, be handled appropriately to benefit educa...

The cover image is based on the Editorial Artificial Intelligence on the Advance to Enhance Educational Assessment: Scientific Clickbait or Genuine Gamechanger? by Fabian Zehner and Carolin Hahnel, https://doi.org/10.1111/jcal.12810 . image

Background. A priori assumptions about specific behavior in test items can be used to process log data in a rule-based fashion to identify the behavior of interest. In this study, we demonstrate such a top-down approach and created a process indicator to represent what type of information processing (flimsy, breadth-first, satisficing, sampling, la...

Mit IGLU 2021 hat in Deutschland ein Wechsel hin zur digitalen Messung der Lesekompetenz der Schülerinnen und Schüler stattgefunden. Dieser Wechsel wurde von einer papierbasierten Brücken-Studie begleitet, die es erlaubt, mögliche Veränderungen aufgrund des Moduswechsels abzuschätzen und zu berücksichtigen. Auf Basis der Analysen der nationalen Dat...

. Reading is fundamental for higher education and scientific research, and it is primarily organized by academic libraries. Academic reading at the same time is characterized by specific requirements and benefits, which are addressed by available reading options, practices, and media. Although socialized literacies, routinized reading practices and...

Background Computer‐based assessment allows for the monitoring of reader behaviour. The identification of patterns in this behaviour can provide insights that may be useful in informing educational interventions. Objectives Our study aims to explore what different patterns of reading activity exist, and investigates their interpretation and consis...

Multiple document comprehension (MDC) refers to the ability to integrate information from multiple sources into a coherent representation, which requires specific cognitive processes. Assuming that epistemic beliefs are domain-related, this study investigates exploratively how epistemic beliefs in the domains of science and history affect the maste...

International large-scale assessments such as PISA or PIAAC have started to provide public or scientific use files for log data; that is, events, event-related attributes and timestamps of test-takers’ interactions with the assessment system. Log data and the process indicators derived from it can be used for many purposes. However, the intended us...

The NAEP EDM Competition required participants to predict efficient test-taking behavior based on log data. This paper describes our top-down approach for engineering features by means of psychometric modeling, aiming at machine learning for the predictive classification task. For feature engineering, we employed, among others, the Log-Normal Respo...

The study investigates automated and controlled cognitive processes that occur when university students read multiple documents (MDs). We examined data of 401 students dealing with two MD sets in a digital environment. Performance was assessed through several comprehension questions. Recorded log data gave indications about students’ time allocatio...

Recent research suggests that readers' subjective task understanding influences reading processes and outcomes. Therefore, the present study's aim was to investigate whether the task demands that readers retrospectively report relate to multiple document comprehension strategies and outcome. A total of 310 university students completed three units...

Efficiency in reading component skills is crucial for reading comprehension, as efficient subprocesses do not extensively consume limited cognitive resources, making them available for comprehension processes. Cognitive efficiency is typically measured with speeded tests of relatively easy items. Observed responses and response times indicate the l...

As Internet sources provide information of varying quality, it is an indispensable prerequisite skill to evaluate the relevance and credibility of online information. Based on the assumption that competent individuals can use different properties of information to assess its relevance and credibility, we developed the EVON (evaluation of online inf...

The digital revolution has made a multitude of text documents from highly diverse perspectives on almost any topic easily available. Accordingly, the ability to integrate and evaluate information from different sources, known as multiple document comprehension, has become increasingly important. Because multiple document comprehension requires the...

The OECD Programme for the International Assessment of Adult Competencies (PIAAC) was the first computer-based large-scale assessment to provide anonymised log file data from the cognitive assessment together with extensive online documentation and a data analysis support tool. The goal of the chapter is to familiarise researchers with how to acces...

The 2nd Annual WPI-UMASS-UPENN EDM Data Mining Challenge required contestants to predict efficient test-taking based on log data. In this paper, we describe our theory-driven and psychometric modeling approach. For feature engineering, we employed the Log-Normal Response Time Model for estimating latent person speed, and the Generalized Partial Cre...

Das Verständnis multipler Dokumente (Multiple Document Comprehension, MDC) wird als Fähigkeit verstanden, aus verschiedenen Informationsquellen eine integrierte Repräsentation eines inhaltlichen Gegenstandsbereichs zu konstruieren. Als solche ist sie sowohl für die erfolgreiche Bewältigung eines Studiums als auch für gesellschaftliche Partizipation...

Educational largescale assessments risk their temporal comparability when shifting from paperto computerbased assessment. A recent study showed how text responses have altered alongside PISA’s mode change, indicating mode effects. Uncertainty remained, however, because it compared students from 2012 and 2015. We aimed at reproducing the findings in...

Multiple document comprehension is the ability to construct an integrated representation of a specific topic based on several sources. It is an important competence for university students; however, there has been so far no established instrument to assess multiple document comprehension in a standardized way. Therefore, we developed a test coverin...

Die mittlere Lesekompetenz der Fünfzehnjährigen in Deutschland liegt bei der PISA-Studie 2018 signifikant über dem OECD-Durchschnitt. Der Mittelwert der Lesekompetenz der Fünfzehnjährigen in Deutschland unterscheidet sich 2018 nicht signifikant von den deutschen Durchschnittswerten bei den PISA-Studien 2009 und 2015, ist aber deutlich höher als bei...

The study investigates the cognitive load of students working on tasks that require the comprehension of multiple documents (Mul-tiple Document Comprehension, MDC). In a sample of 310 students, perceived task difficulty (PD) and mental effort (ME) were examined in terms of task characteristics, individual characteristics, and students' processing be...

For many years, reading comprehension in the Programme for International Student Assessment (PISA) was measured via paper‐based assessment (PBA). In the 2015 cycle, computer‐based assessment (CBA) was introduced, raising the question of whether central equivalence criteria required for a valid interpretation of the results are fulfilled. As an exte...

Background With digital technologies, competence assessments can provide process data, such as mouse clicks with corresponding timestamps, as additional information about the skills and strategies of test takers. However, in order to use variables generated from process data sensibly for educational purposes, their interpretation needs to be valida...

In this paper, we developed a method to extract item-level response times from log data that are available in computer-based assessments (CBA) and paper-based assessments (PBA) with digital pens. Based on response times that were extracted using only time differences between responses, we used the bivariate generalized linear IRT model framework (B...

A critical evaluation of results to find useful information is essential when doing a web search. In this study, we investigated the evaluation skills of secondary school students, based on their behavior in selecting links from a search engine result page (SERP). To clarify the role of reading when evaluating online information, we assessed studen...

Receiving and using web-based information has become part of everyday life, but the non-linear presentation of information can make considerable demands on cognitive resources, affecting text comprehension. This study examined whether memory updating predicts students' comprehension of digital hypertext over and above skills in reading linearly str...

Introducción. La imputación múltiple (IM) es uno de los métodos más recomendados para sustituir valores perdidos en datos de investigación. Este artículo se dedica al manejo de los valores perdidos en MES, analizando dos bases de datos de Psicología Educativa y a recomendaciones para investigadores orientados a las aplicaciones.Método. Presentamos...

Die Studie untersucht Zusammenhänge zwischen dem Leseverständnis und basalen Prozessen des Leseverstehens auf Wort- und Satzebene sowie des Arbeitsgedächtnisses 15-jähriger Jugendlicher. Es wurde den Fragen nachgegangen, ob Unterschiede in der Effizienz der betrachteten Teilkomponenten zum einen die Lesekompetenz selbst, zum anderen Veränderungen i...

Introduction. Multiple imputation (MI) is one of the most highly recommended methods for replacing missing values in research data. The scope of this paper is to demonstrate missing data handling in SEM by analyzing two modified data examples from educational psychology, and to give practical recommendations for applied researchers. Method. We prov...

Reading and understanding digital text that is organized in a non-linear hypertext format can be challenging for students as it requires a more self-directed selection of text pieces compared to reading linear texts. This study aims at investigating how individual differences in students' skills in comprehending digital text can be explained by the...