Claus H. Carstensen

University of Bamberg · Institut für Psychologie

Disengaged responding poses a severe threat to the validity of educational large-scale assessments, because item responses from unmotivated test-takers do not reflect their actual ability. Existing identification approaches rely primarily on item response times, which bears the risk of misclassifying fast engaged or slow disengaged responses. Proce...

This study aims to investigate how test scores from PIAAC (Programme for the International Assessment of Adult Competencies) can be interpreted, by comparing the PIAAC competencies literacy and numeracy to reasoning and perceptual speed. Dimensionality analyses supported, that the PIAAC competencies can be separated into a common factor overlapping...

Background After elementary school, students in Germany are separated into different school tracks (i.e., school types) with the aim of creating homogeneous student groups in secondary school. Consequently, the development of students’ reading achievement diverges across school types. Findings on this achievement gap have been criticized as dependi...

This study applied a three-step latent class analysis (LCA) approach to explore latent classes of learning strategy use and their moderation effects on the relationships between motivation and mathematics performance. The data of 15-year-old students from five East Asian educational systems related to Chinese culture in the Programme of Internation...

In large-scale educational assessments, interviewers should ensure standardized settings for all participants. However, in practice many interviewers do not strictly adhere to standardized field protocols. Therefore, systematic interviewer effects for the measurement of mathematical competence were examined in a representative sample of N = 5,139 G...

In the context of item response theory (IRT), linking the scales of two measurement points is a prerequisite to examine a change in competence over time. In educational large-scale assessments, non-identical test forms sharing a number of anchor-items are frequently scaled and linked using two− or three-parametric item response models. However, if...

Parents play a pivotal role in introducing their children to science, but little is known about the nature of an early science-related home learning environment. This study examines different aspects of the home learning environment and their associations with children's science knowledge. Mediation analyses of a sample of 257 five-year-old prescho...

Land use change, natural resource use and climate change are challenging Sustainable Development issues (SDGs 13-15). Fostering the competencies to deal with such issues is one core task for current educational endeavors. Among these competencies, decision-making competencies are central. In detail, we investigate how learners evaluate alternative...

Das Nationale Bildungspanel (NEPS) untersucht seit 2009 Bildungsverläufe in Deutschland. In sechs Startkohorten werden bildungsrelevante Kompetenzen und Prädiktoren in regelmäßigen Abständen vom Neugeborenen- bis zum Erwachsenenalter erfasst. Im Rahmen der Schulkohorten (Startkohorte 3 im 5. Jahrgang; Startkohorte 4 im 9. Jahrgang) wurde eine Serie...

Assessment literacy is a crucial aspect of teachers’ professional knowledge and relevant to fostering students’ learning. Concerning experimentation, teachers have to be able to assess student achievement when students form hypotheses, design experiments, and analyze data. Therefore, teachers need to be familiar with criteria for experimentation as...

The National Educational Panel Study (NEPS) provides data on the development of various competence domains across the life span. Because research questions using these competences typically pertain to latent relationships between constructs, this paper gives an overview of the concept of plausible values and how to estimate unbiased effects that ac...

This study examined learning strategy use in mathematics among East Asian students in East Asian educational systems. By employing latent class analysis on the Programme for International Student Assessment (PISA) 2012 data, we found four classes of learning strategy types, namely memorization with metacognitive strategies (17.49%), metacognitive s...

Measuring growth in an item response theory framework requires aligning two tests on a common scale known as longitudinal linking. So far, no consensus exists regarding the appropriate method for the linking of longitudinal data scaled according to the Rasch model in large-scale assessments. Therefore, an empirical study was conducted within the Ge...

Bestimmt erinnern Sie sich noch an Situationen in der eigenen Schulzeit, in denen Lernergebnisse beurteilt wurden. Die Mathearbeit wird benotet zurückgegeben und Emily ist voller Erwartung. Die mündliche Note in Geschichte, die Moritz bekommen hat, war ungerecht; in der Pause wird darüber heiß diskutiert. Die Lehrerin verschiebt den geplanten Test...

The selection and measurement of competencies, reflecting educational effects in a lifelong learning perspective, represents a major challenge for the German National Educational Panel Study (NEPS). Data on the development of competencies serves as a central point of reference for all other parts of the study. These competencies have to be relevant...

Dieser Beitrag zeigt die Analysepotentiale des Nationalen Bildungspanels für die Bildungsarmutsforschung anhand der Erwachsenenkohorte auf. Wir analysieren, ob die absolute Zertifikatsarmut über die Kohorten abgenommen hat und ob es soziale Herkunftsunterschiede in der Wahrscheinlichkeit, zertifikatsarm zu sein, gibt. Des Weiteren untersuchen wir V...

Response styles can influence item responses in addition to a respondent’s latent trait level. A common concern is that comparisons between individuals based on sum scores may be rendered invalid by response style effects. This paper investigates a multidimensional approach to modeling traits and response styles simultaneously. Models incorporating...

In Germany, the respondents who had participated in the 2012 survey of the Programme for the International Assessment of Adult Competencies (PIAAC) were re-approached for the panel study PIAAC-L. PIAAC-L aims at investigating the longitudinal effects of skill outcomes over the life course and the development of the key skills assessed in PIAAC. Mor...

Competence data from low-stakes educational large-scale assessment studies allow for evaluating relationships between competencies and other variables. The impact of item-level nonresponse has not been investigated with regard to statistics that determine the size of these relationships (e.g., correlations, regression coefficients). Classical appro...

The investigation of learning processes by assessing students’ experience along with objective characteristics within a classroom context has a long tradition in empirical learning process research (e.g. Sembill, 1984 et passim; Wild & Krapp, 1996). However, most of the existing studies confine themselves to psychological variables that seem to be...

This chapter introduces the basics of Granger causation. It describes multidimensional longitudinal item response models and the investigation of Granger causation with multidimensional longitudinal item response models. The chapter provides an empirical example on the topic of science knowledge of children. It addresses potential limitations of th...

In order to precisely assess the cognitive achievement and abilities of students, different types of items are often used in competence tests. In the National Educational Panel Study (NEPS), test instruments also consist of items with different response formats, mainly simple multiple choice (MC) items in which one answer out of four is correct and...

The National Educational Panel Study (NEPS) provides data on the development of competencies across the whole life span. Plausible values as a measure of individual competence are provided by explicitly including background variables that capture individual characteristics in the corresponding Item Response Theory model. Despite tremendous efforts...

Within the German National Educational Panel Study (NEPS), N = 578 students in Grade 5 and N = 1,186 students in Grade 9 with special educational needs in the area of learning (SEN-L) took part in feasibility studies examining how to include students with special needs in large-scale assessments like the NEPS (Heydrich, Weinert, Nusser, Artelt, & C...

The National Educational Panel Study (NEPS) aims at investigating the development of competencies across the whole life span and developing tests for assessing different competence domains. In order to compare competencies across different measurement occasions and examine competence development over time the different measurements must be placed o...

Assessing competencies of students with special educational needs in learning (SEN-L) poses a challenge for large-scale assessments (LSAs). For students with SEN-L, the available competence tests may fail to yield test scores of high psychometric quality, which are—at the same time—measurement invariant to test scores of general education students....

When competence tests are administered, subjects frequently omit items. These missing responses pose a threat to correctly estimating the proficiency level. Newer model-based approaches aim to take nonignorable missing data processes into account by incorporating a latent missing propensity into the measurement model. Two assumptions are typically...

Dieser Artikel betrachtet die messinvariante Erfassung bildungsrelevanter Konstrukte mit Hilfe schriftlicher Befragungen bei Schülerinnen und Schülern an Förderschulen und Hauptschulen in der 5. Jahrgangsstufe. Um optimale Administrationsbedingungen für Schülerinnen und Schüler mit sonderpädagogischem Förderbedarf Lernen zu erforschen, wurde ein ex...

The National Educational Panel Study (NEPS) aims at investigating the development of competencies across the whole lifespan. Competencies are assessed via tests and competence scores are estimated based on models of Item Response Theory (IRT). IRT allows a comparison of test scores—and, thus, the investigation of change across time and differences...

When questionnaire data with an ordered polytomous response format are analyzed in the framework of item response theory using the partial credit model or the generalized partial credit model, reversed thresholds may occur. This led to the discussion of whether reversed thresholds violate model assumptions and indicate disordering of the response c...

The National Educational Panel Study (NEPS) provides data on the development of competencies across the whole life span. Plausible values as a measure of individual competence are provided by explicitly including background variables that capture individual characteristics in the corresponding Item Response Theory models. Despite tremendous efforts...

An important pre-requisite of trend analyses in large scale educational assessments is the measure-ment invariance of the testing instruments across cycles. This paper investigates the measurement invariance of the PISA 2000 and PISA 2009 reading instruments using Item Response Theory models. Links between the PISA 2000 and PISA 2009 instruments we...

Since the publication of the PISA 2006 study results the question of reporting trends over the PISA cycles has received a lot of interest. This chapter discusses the possibilities and limitations of trend analyses based on data from this international comparative study and using complex test designs. The chapter succeeds trend analyses which were c...

The eight articles of this Special Issue describe the assessment of competencies within the German National Education Panel Study (NEPS) for major educational- stage-comprehensive assessment domains of the NEPS. The Editorial outlines the overall framework for the selection and conceptualization of the assessment of competencies against the backgro...

The National Educational Panel Study (NEPS) as a newly set up large-scale assessment study in Germany has accepted the challenge of including students with special educational needs (SEN) into its conceptual design. Particularly, students with SEN in the area of learning (SEN-L) are oversampled within the NEPS. Their educational biographies and rel...

The occurrence of differential item functioning (DIF) for gender indicates that an instrument may not be functioning equivalently for men and women. Aside from DIF effects, item responses in personality questionnaires can also be influenced by response styles. This study analyzes the German NEO-PI-R regarding its differential item functioning for m...

The consistency of extreme response style (ERS) and non-extreme response style (NERS) across the latent variables assessed in an instrument is investigated. Analyses were conducted on several PISA 2006 attitude scales and the German NEO-PI-R. First, a mixed partial credit model (PCM) and a constrained mixed PCM were compared regarding model fit. If...

Standard unidimensional Rasch models assume that persons with the same ability parameters are comparable. That is, the same interpretation applies to persons with identical ability estimates as egards the underlying mental processes triggered by the test. However, research in cognitive psychology shows that persons at the same trait level may emplo...

Natural sciences are an educational topic in kindergartens. At the same time, little is known about how to promote and facilitate the learning of natural sciences for children in kindergarten. This contribution reports on a quasi-experimental study with 245 children, which compares the learning situations ‘experiments’ with ‘discussions on the rele...

Frühkindlichen Bildungsprozessen wird seit einiger Zeit große Bedeutung beigemessen. Dabei wird auch das naturwissenschaftliche Lernen in den Blick genommen. Für das Vorschulalter liegen bislang jedoch kaum Untersuchungen zur Naturwissenschaftlichen Kompetenz und ihrer Förderung vor. Im vorliegenden Beitrag wird die Entwicklung und Validierung eine...

The selection and measurement of competencies, reflecting educational effects in a lifelong learning perspective, represents a major challenge for the German National Educational Panel Study. Data on the development of competencies serves as a central point of reference for all other parts of the Panel Study. These competencies have to be relevant...

The aim of the research presented here is the use of extensions of longitudinal item response theory (IRT) models in the analysis and comparison of group-specific growth in large-scale assessments of educational outcomes. A general discrete latent variable model was used to specify and compare two types of multidimensional item-response-theory (MIR...

This paper focuses interest in science as one of the attitudinal aspects of scientific literacy. Large‐scale data from the Programme for International Student Assessment (PISA) 2006 are analysed in order to describe student interest more precisely. So far the analyses have provided a general indicator of interest, aggregated over all contexts and c...

Introduction The Berkeley Evaluation and Assessment Research (BEAR) Center has for the last several years been involved in the development of an assessment system, which we call the BEAR Assessment System. The system consists of four principles, each associated with a practical “building block” [Wilson 2005] as well as an activity that helps integr...

The study analyzes the effects of grade repetition on science and mathematics achievement and on self-concept in mathematics using longitudinal data from a representative sample of 9th graders in Germany. Same-age comparisons were applied between three groups: (a) the retained students, (b) a matched group of promoted students, and (c) the entire g...

A general diagnostic model was used to specify and compare two multidimensional item-response-theory (MIRT) models for longitudinal data: (a) a model that handles repeated measurements as multiple, correlated variables over time (Andersen, 1985) and (b) a model that assumes one common variable over time and additional orthogonal variables that quan...

Correct responses to the unitized items of PISA 2006 rely to differing extents on the contextual stimulus supplied. This difference is referred to in this study as the degree of contextualization. A selection of science items from PISA 2006 has been assigned to two categories, not by competencies as in the framework for the PISA survey, but by the...

In this paper, we use data from the German PISA 2003 sample to study the effects of central exit examinations on student performance, student attitudes, and teacher behavior. Unlike earlier studies we use (i) a value-added measure to pin down the effect of central exit exams on learning in the last year before the exam and (ii) separate test scores...

Die Veröffentlichungen von Ergebnissen der letzten PISA-Erhebung geben Anlass, Möglichkeiten und Grenzen von Trendanalysen in internationalen Vergleichsstudien mit komplexen Testdesigns zu untersuchen. Der Beitrag schließt an Trendanalysen an, die in Deutschland bei PISA 2006 durchgeführt wurden. In einem ersten Schritt werden Voraussetzungen für T...

This study analyses the effects which repeating a class has on ninth grade students’ development of mathematical competency. The following research questions were addressed: How many students repeat grades in the different types of schools? How do students who repeat a grade differ from those who do not in their performance and background character...

The publication of the results of the last PISA Study prompts an examination of the possibilities and limitations of trend analyses of international comparative studies with complex test designs. This paper follows up on trend analyses which were carried Out in Germany for PISA 2006. In a first step, the requirements for trend analyses of studies s...

In large-scale educational assessments, such as the Programme for International Student Assessment (PISA) and the Trends in Mathematics and Science Study (TIMSS), a primary concern is with the estimation of the population-level characteristics of a number of latent variables and the relationships between latent variables and other variables. Typica...

The present volume is a collection of chapters on research and development work on extensions of the Rasch model (RM; Rasch, 1960) that have focused on relaxing some fundamental constraints of the original RM, while preserving many of the unique features of the model. More specifically, the volume presents extensions of the RM in which certain homo...

Rasch’s measurements model has been generalized in many different ways (see Chapter 1, this volume). One direction of generalizing the Rasch model is to consider not only two factors of the response probability, i.e., persons and items, but an additional third factor. This means to extend the two-dimensional data matrix to a three-dimensional data...

The following values have no corresponding Zotero field: Author Address: Educational Testing Service, Princeton, NJ, US Research Notes: 20070626 von Davier, A. A., Carstensen, C. H., & von Davier, M. (2006). Linking Competencies in Educational Settings and Measuring Growth (No. RR-06-12). Princeton, NJ: ETS. Der Report ist eine meist sehr knapp geh...

In this chapter we illustrate the use of item response models to analyze data result-ing from the measurement of competencies. These item response models deal with data from an instrument such as a test, an inventory, an attitude scale, or a question-naire – we will call this "test data." The main purpose of using item response models is the measur...

The following values have no corresponding Zotero field: ID - 218