Thorben Jansen

Leibniz Institute for Science and Mathematics Education | IPN · Ed. Research and Ed. Psychology

Background: Teachers' assessments of student writing proficiency are essential for adaptive instruction and providing feedback. However, studies have revealed a lack of objectivity in text quality ratings of the same text in different contexts, although texts of identical quality should receive identical teacher judgments. Aims: Our two studies-one...

People’s subjective beliefs about themselves affect what people think and, consequently, what they do. Positive self-beliefs are important for many life outcomes, from academic success to well-being, especially during K–12 education as a crucial developmental period. Many empirical studies and meta-analyses have examined correlates of self-beliefs....

Background: Students need support when working on complex tasks such as writing. Reward-learning theories posit that verbal rewards, like feedback-depending on its design-can support students' extrinsic and intrinsic motivation. So far, empirical studies have focused on using praise feedback to foster extrinsic motivation. However, researchers have...

Background: Successful feedback should provide learning goals, evaluate current performance and indicate improvement strategies. Furthermore, feedback can only positively affect student performance if students actively engage with it. Thus, it is necessary to consider the feedback reception process in addition to the feedback information itself. Ai...

Recent investigations in automated essay scoring research imply that hybrid models, which combine feature engineering and the powerful tools of deep neural networks (DNNs), reach state-of-the-art performance. However, most of these findings are from holistic scoring tasks. In the present study, we use a total of four prompts from two different corp...

Through this editorial, we will provide guidance to readers and potential authors of systematic literature reviews with or without meta-analysis that aim to submit this type of research to Learning and Individual Differences. We also aim at providing guidance on a more general level beyond this specific journal and to point out relevant considerati...

Argumentative writing upper secondary level education analytic analysis language quality content structure A B S T R A C T Argumentative writing in English as a foreign language (EFL) is an important skill in upper secondary education in Germany and Switzerland. This article provides insights into students' EFL writing skills in the aspects of lang...

Feedback can play a pivotal role in learning but is only effective when students use it. However, the use of feedback varies greatly from student to student and these differences must be explained. Hence, the construct receptivity to instructional feedback (RIF) was introduced to assess students’ attitudes towards and engagement with human-provided...

Feedback is crucial for learning complex tasks like writing; yet its creation is time-consuming , often leading to students receiving insufficient feedback. Generative artificial intelligence, particularly Large Language Models (LLMs) like ChatGPT 3.5-Turbo, has been discussed as a solution for providing more feedback. However, there needs to be mo...

In this paper, we present the DARIUS (Digital Argumentation Instruction for Science) corpus for argumentationquality on 4589 essays written by 1839 German secondary school students. The corpus is annotated according toa fine-grained annotation scheme, ranging from a broader perspective like content zones, to more granular featureslike argumentation...

Recent investigations in automated essay scoring research imply that hybrid models, which combine feature engineering and the powerful tools of deep neural networks (DNNs), reach state-of-the-art performance. However, most of these findings are from holistic scoring tasks. In the present study, we use a total of four prompts from two different corp...

Background: Revising is central in the process of writing and for acquiring writing skills. Setting revision goals is challenging especially for novice writers: students do not revise frequently, especially in homework conditions that demand high levels of self-regulation. Empirical studies of goal-setting interventions providing a detailed list of...

The current article investigated the role that the school context plays in teachers’ well-being in terms of emotional exhaustion and job satisfaction. First, we analyzed between-school variation in well-being and indicators of social interactions. Second, we examined the association between these indicators and well-being on the teacher and school...

Teachers' assessment of students' performance on complex tasks, such as writing, is important both for their teaching and for stu-dents' learning. Teachers must be able and motivated to assess texts correctly. According to theoretical assumptions, feedback can help promote the diagnostic competencies required to assess texts correctly, but, up unti...

Simulated diagnostic situations function as learning environments and research tools to deepen the understanding of diagnostic processes. In this conceptual paper, we present a framework that fulfils two purposes. First, we categorize simulations of diagnostic situations using six descriptive components: physical environment, social embeddedness, d...

Rubrics are defined as coherent sets of criteria for students’ work that include detailed descriptions of quality for a specific learning task. They are used in different subjects to guide student learning and assess its results. This study demonstrates the design and development of an assessment rubric for writing emails in English as a foreign la...

Students’ academic achievement is a central predictor of a long list of important educational outcomes, such as access to higher education and socioeconomic success. Prior studies have extensively focused on identifying variables that are related to academic achievement and an important variable in this context appears to be students’ personality....

While many methods for automatically scoring student writings have been proposed, few studies have inquired whether such scores constitute effective feedback improving learners’ writing quality. In this paper, we use an EFL email dataset annotated according to five analytic assessment criteria to train a classifier for each criterion, reaching huma...

The central purpose of our study is to analyze whether teacher experience and content knowledge relate to teacher judgments on student report texts. Ninety-one experienced teachers, including 27 specializing in the subject of the German language, and 136 teacher students, including 37 majoring in German Studies, participated in this study. Particip...

School exams often measure students' higher-order-thinking-skills (HOTS) subsequent to students' lower-order-thinking-skills (LOTS). Teachers should assess students' HOT separately from students’ LOT. Three experimental studies (N = 36/41/40) investigated whether assessing LOTS-related performances can distort subsequent judgments of HOTS-related p...

Academic motivation is an essential predictor of school success in K 12 education. Accordingly, many meta-analyses have examined variables associated with academic motivation. However, a central question remains unanswered: What is the relative strength of the relations of both student variables (achievement, socioemotional variables, and backgroun...

Background To teach evolution efficiently teachers must be able to diagnose their students’ ideas and understanding of the phylogeny of organisms. This encompasses different facets of content-specific professional knowledge, that is, knowledge about core ideas and theories, as well as knowledge about respective misconceptions. However, as findings...

When judging subject-specific aspects of students’ texts, teachers should assess various characteristics, e.g., spelling and content, independently of one another since these characteristics are indicators of different skills. Independent judgments enable teachers to adapt their classroom instruction according to students’ skills. It is still uncle...

Zusammenfassung. Argumentatives Schreiben ist eine bedeutsame Kompetenz in der Fremdsprache Englisch. Entsprechende Schreibaufgaben sind Teil von Schulabschlussprüfungen in der Sekundarstufe II und von Zugangstests für Hochschulen (z.B. TOEFL ® ). Trotz ihrer Bedeutsamkeit wurden diese komplexen Schreibleistungen bisher im Kontext großer Schulleist...

Teacher judgment accuracy (TJA) is a central part of diagnostic competence. Research has revealed heterogeneous results regarding teacher characteristics as determinants of TJA. In the present experimental study N=36 experienced and N=47 student teachers holistically and analytically assessed four student essays. We compared the judgments of teache...

Das Projekt eRUBRIC untersucht die Wirksamkeit rubricbasierten Feedbacks beim englischen Schreiben in einer digitalen Lernumgebung. Bei der geplanten Studie handelt es sich um ein computerbasiertes Experiment, in dessen Rahmen Schülerinnen und Schüler der achten Klasse verschiedene Feedbackinterventionen zum Schreiben formaler E-Mails erhalten. Die...

The assessment of text quality is a transdisciplinary issue concerning the research areas of educational assessment, language technology, and classroom instruction. Text length has been found to strongly influence human judgment of text quality. The question of whether text length is a construct-relevant aspect of writing competence or a source of...

Assessing student writing is a complex task and experimental studies have shown that textual determinants, such as spelling or vocabulary, affect teachers’ judgments of other analytic criteria in student essays. Among these elements, organisation has not been extensively explored. This experimental study examines how organisational quality influenc...

The authors discovered a number of minor inaccuracies on pp. 58–59 of this article. However, they do not affect any of the main claims regarding the influence of lexical features on text assessment made in that study. In this note, we provide a corrected proof of the inaccuracies (marked in bold). The authors would like to apologize for any inconve...

Numerous studies have examined the relationship between lexical features of students’ compositions and judgements of text quality. However, the degree to which teachers’ judgements are influenced by the quality of vocabulary in students’ essays with regard to their assessment of other textual characteristics is relatively unexplored. This experimen...

This experimental study examined the influence of vocabulary and spelling on teachers’ perceptions of English as a second language (ESL) essays. Sixty-nine pre-service teachers from Switzerland and Germany rated four upper-intermediate ESL argumentative essays and provided written comments on strengths and weaknesses of the student compositions. In...