Johannes Hartig

Johannes Hartig
DIPF - Leibniz Institute for Research and Information in Education · Educational Quality and Evaluation

Prof. Dr.

About

165
Publications
89,688
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
4,014
Citations
Citations since 2016
64 Research Items
2592 Citations
20162017201820192020202120220100200300400
20162017201820192020202120220100200300400
20162017201820192020202120220100200300400
20162017201820192020202120220100200300400
Introduction
Johannes Hartig currently works at the Educational Quality and Evaluation, German Institute for International Educational Research (DIPF). Johannes does research in Educational Psychology, Psychometrics and Differential Psychology.
Additional affiliations
September 2008 - January 2010
Universität Erfurt
Position
  • Professor (Full)

Publications

Publications (165)
Article
Full-text available
When scaling psychological tests with methods of item response theory it is necessary to investigate to what extent the responses correspond to the model predictions. In addition to the statistical evaluation of item misfit, the question arises as to its practical significance. Although item removal is undesirable for several reasons, its practical...
Article
Testing item fit is central in item response theory (IRT) modeling, since a good fit is necessary to draw valid inferences from estimated model parameters. Infit and outfit fit statistics, widespread indices for detecting deviations from the Rasch model, are affected by data factors, such as sample size. Consequently, the traditional use of fixed i...
Article
Full-text available
The article focuses on estimating effects in nonrandomized studies with two outcome measurement occasions and one predictor variable. Given such a design, the analysis approach can be to include the measurement at the previous time point as a predictor in the regression model (ANCOVA), or to predict the change-score of the outcome variable (CHANGE)...
Presentation
Full-text available
Educational injustices are an important issue in educational discourses. One of these is correlations between social difference categories and learning gains, which have been shown to exist by many studies. Our school system is therefore unjust. However, is the whole school system equally unjust? There are few and mixed research findings. The aim o...
Article
Learning using the Internet has become a vital factor for academic success in higher education. Students increasingly rely on the Internet as their main information source. However, related research is still an emerging and highly fragmented field. Therefore, this study aims to provide a comprehensive and integrative review of research literature o...
Article
This paper aims to provide evidence regarding the interpretation that test scores of a newly developed test instrument reflect learning progress in competence in sustainability management. As competence in sustainability management is conceptualised as mainly acquired through academic opportunities to learn (OTLs), students in courses with relevant...
Article
Full-text available
This paper presents the development of an instrument for the assessment of system competence in the field of Education for Sustainable Development (ESD). Based on an already existing, more complex model of system competence for the school subject geography, we have developed a test that refers to central themes and principles of ESD using exclusive...
Article
This study aimed at examining the suitability of questionnaire instruments commonly used in large-scale assessments for measuring non-cognitive school effectiveness criteria. It focused on questions of reliability and validity for capturing changes in students within schools across time and the instruments’ sensitivity to school effects. The aim wa...
Article
Full-text available
Upon a drastic decline of the giraffe population in the wild, conservation efforts and therefore the role of zoos have become more important than ever. With their unique opportunities, zoos provide excellent conditions to study animal behavior, expanding the knowledge about the giraffe's behavior repertoire and their ability to adapt. This study th...
Article
Full-text available
When scaling data using item response theory, valid statements based on the measurement model are only permissible if the model fits the data. Most item fit statistics used to assess the fit between observed item responses and the item responses predicted by the measurement model show significant weaknesses, such as the dependence of fit statistics...
Chapter
In diesem Kapitel sollen die Auswirkungen von Mehrsprachigkeit bei standardisiert durchgeführten Leistungserfassungen mithilfe von Testaufgaben betrachtet werden, wie sie beispielsweise in der Individualdiagnostik, z. B. zur Klärung eines individuellen Förderbedarfs, oder bei Bildungsstudien wie PISA (Programme for International Student Assessment)...
Chapter
Im vorliegenden Kapitel geht es um die Anwendung von IRT-Modellen im Rahmen der empirischen Bildungsforschung. Bei großen Schulleistungsstudien werden spezifische Vorteile der IRT genutzt, um beispielsweise das Matrix-Sampling von Testaufgaben, die Erstellung paralleler Testformen und die Entwicklung computerisierter adaptiver Tests zu ermöglichen....
Chapter
Das Gütekriterium der Validität ist ein zentrales Qualitätskriterium. Es ist den Gütekriterien Objektivität und Reliabilität übergeordnet. Validität bezieht sich darauf, inwieweit Interpretationen von Testwerten und beabsichtigte Verwendungen von Tests gerechtfertigt sind. Das Verständnis von Validität hat sich in den letzten Jahrzehnten deutlich w...
Chapter
In diesem Kapitel werden verschiedene Möglichkeiten beschrieben, wie ein Testergebnis bzw. Testwert deskriptivstatistisch interpretiert werden kann. Bei der normorientierten Interpretation wird der Testwert in einen Normwert transformiert, der einen Vergleich mit den Testwerten anderer Personen einer Bezugsgruppe (den „Testnormen“) erlaubt. Die Tes...
Article
The interaction of internal clocks and environmental conditions determines the daily behavioural rhythm of an animal. Due to the strong influence of light, these circadian rhythms oscillate in mammals over a cycle length of about 24 hours, equivalent to the daily light-dark cycle. The relation between activity and rest within this cycle is species-...
Chapter
International large-scale assessments are very successful. One key factor of this success is their rigorous methodological and psychometric basis. Because education systems worldwide are subject to rapid changes, international large-scale assessments need to evolve as well. We describe five current methodological challenges that should be addressed...
Chapter
This article focuses on the reliability of measurements and the validity of their interpretations in the context of social sciences, in which researchers aim to measure theoretical constructs that cannot be directly observed. In order to measure an unobservable construct, researchers need to decide how to operationalize the construct, that is, defi...
Chapter
Die PIAAC Studie und die LEO Studie setzen sich mit der Erfassung der Lese- und Schreibkompetenz auseinander, setzen aber unterschiedliche Konstrukte und Assessments ein. Der Umstand, dass gleich zwei Studien literale Kompetenzen erfassen, führt daher zu Fragen bezüglich der Vergleichbarkeit der Studien: (a) Wie stehen die beiden Konstrukte in LEO...
Article
The manuscript focuses on effects in nonrandomized studies with two outcome measurement occasions and one explanatory variable, and in which groups already differ at the pretest. Such study designs are often encountered in educational and instructional research. Two prominent approaches to estimate effects are (1) covariance analytical approaches a...
Chapter
Only limited evidence exists to explain competence in sustainability management. In sustainability research, especially value-related constructs are emphasized to predict sustainable behaviour. In addition, theoretical-conceptual considerations toward competence and some empirical analyses of competence measurement highlight the potential of affect...
Article
Testing whether items fit the assumptions of an item response theory model is an important step in evaluating a test. In the literature, numerous item fit statistics exist, many of which show severe limitations. The current study investigates the root mean squared deviation (RMSD) item fit statistic, which is used for evaluating item fit in various...
Article
Full-text available
Abstract Background Item position effects have been a common concern in large-scale assessments as changing the order of items in booklets may have an undesired effect on test performance. If every test taker would be affected by the effect in the very same way, comparisons between groups of individuals would still be valid. However, research has s...
Article
There is a large base of empirical evidence that points to the importance of school leadership with regard to functional schools and student achievement. Based on this importance, a change of principal can be a critical event for a school. Routines might be altered or given up completely, social relationships can be disrupted, and a school may lose...
Presentation
Full-text available
Many studies on school effectiveness use achievement in a single domain as a relevant effectiveness criterion. However, this practice has been criticized and additional criteria were proposed, among them a positive reading attitude, a positive self-concept or classroom participation. This leads to the essential educational research question about t...
Article
The paper discusses an examination of the dimensions of a competence model for sustainability management. A central assumption is that the dimensions of the competence model differ according to knowledge representation (i. e., declarative vs. schematic and strategic knowledge) and content area (i. e., business administration and sustainability from...
Article
In 2005, the Austrian government committed all schools to develop school policies supporting the learning of their students. One purpose of the paper is to seek evidence for the effectiveness of this governmental action. A second purpose is to suggest a special kind of multilevel model which can be applied to longitudinal data at the school level w...
Article
Full-text available
Link: https://rdcu.be/bBHpL Advances in computer technology and artificial intelligence create opportunities for developing adaptive language learning technologies which are sensitive to individual learner characteristics. This paper focuses on one form of adaptivity in which the difficulty of learning content is dynamically adjusted to the learner...
Article
Full-text available
The Swiss teacher education and training system offers a practically and academically oriented path for aspiring commercial vocational education and training (VET) teachers. Although teachers’ content knowledge (CK) and pedagogical content knowledge (PCK) are considered crucial for teaching quality and students’ achievement, little is known about S...
Article
Full-text available
Instructional sensitivity is the psychometric capacity of tests or single items of capturing effects of classroom instruction. Yet, current item sensitivity measures’ relationship to (a) actual instruction and (b) overall test sensitivity is rather unclear. The present study aims at closing these gaps by investigating test and item sensitivity to t...
Presentation
Schuleffektivität kann als die Zielerreichung einer Schule definiert werden (Scheerens, 2013). Allerdings lässt diese Definition offen, welche Ziele Schulen erreichen sollen. Viele Schuleffektivitätsstudien verwenden ausschließlich Schulleistungen als relevante Effektivitätskriterien. Diese Praxis wurde jedoch stark kritisiert und es wurden zusätzl...
Article
Full-text available
The IRT models most commonly employed to estimate within-item multidimensionality are compensatory and suggest that some dimensions (e.g., traits or abilities) can make up for a lack in others. However, many assessment frameworks in educational large-scale assessments suggest partially compensatory relations among dimensions. In two Monte-Carlo sim...
Presentation
Effects of classroom composition on the development of students' learning motivation Students have different prior achievement levels. A common way to handle this heterogeneity is the practice of ability grouping. Those who are in favor of homogeneous classrooms, argue teachers could provide more effective teaching for students with similar ability...
Article
Full-text available
Purpose Collaborative care is effective in improving symptoms of patients with depression. The aims of this study were to characterize symptom trajectories in patients with major depression during one year of collaborative care and to explore associations between baseline characteristics and symptom trajectories. Methods We conducted a cluster-ran...
Data
Frequency of HCA contacts per patient. (TIF)
Data
Trajectories of depressive symptoms (PHQ-9) over 12 intervention months for the best fitting two-class model. All 280 patients for whom monitoring data was available were included in the analyses. (TIF)
Presentation
Many studies on school effectiveness use achievement as a relevant effectiveness criterion. However, this practice has been criticized and additional criteria were proposed, among them reading attitudes. In order to use reading attitudes as an effectiveness criterion in a longitudinal study, it is necessary to investigate whether measures of readin...
Article
Testergebnisse von Schülerinnen und Schülern dienen regelmäßig als ein zentrales Kriterium für die Beurteilung der Effektivität von Schule und Unterricht. Gültige Rückschlüsse über Schule und Unterricht setzen voraus, dass die eingesetzten Testinstrumente mögliche Effekte des Unterrichts auffangen können, also instruktionssensitiv sind. Jedoch wird...
Data
The supporting information shows on of the 17 test problems we used for validating the system competence model.
Article
Full-text available
Questionnaires for the assessment of attitudes and other psychological traits are crucial in educational and psychological research, and item response theory (IRT) has become a viable tool for scaling such data. Many international large-scale assessments aim at comparing these constructs across countries, and the invariance of measures across count...
Article
Full-text available
Valid inferences on teaching drawn from students’ test scores require that tests are sensitive to the instruction students received in class. Accordingly, measures of the test items’ instructional sensitivity provide empirical support for validity claims about inferences on instruction. In the present study, we first introduce the concepts of absol...
Article
Full-text available
In repeated cross-sections of organizations, different individuals are sampled from the same set of organizations at each time point of measurement. As a result, common longitudinal data analysis methods (e.g., latent growth curve models) cannot be applied in the usual way. In this contribution, a multilevel structural equation modeling approach to...
Chapter
The project “Modeling competencies with multidimensional item-response-theory models” examined different psychometric models for student performance in English as a foreign language. On the basis of the results of re-analyses of data from completed large scale assessments, a new test of reading and listening comprehension was constructed. The items...
Article
Testing item fit is an important step when calibrating and analyzing item response theory (IRT)-based tests, as model fit is a necessary prerequisite for drawing valid inferences from estimated parameters. In the literature, numerous item fit statistics exist, sometimes resulting in contradictory conclusions regarding which items should be excluded...
Article
System competence in geography education. A theoretically based and empirically verified competence structure model. Pursuant to the German educational standards in geography the system concept is considered a central basic concept. Being a bridging subject, geography faces special challenges with regard to the imparting of a related system compet...
Article
Full-text available
Valid and reliable standardized assessment of nursing competencies is needed to monitor the quality of vocational education and training (VET) in nursing and evaluate learning outcomes for care work trainees with increasingly heterogeneous learning backgrounds. To date, however, the modeling of professional competencies has not yet evolved into pro...
Article
Full-text available
In der pädagogisch-psychologischen Selbstkonzeptliteratur findet sich vielfach die Forderung nach optimistischen Fähigkeitsselbstkonzepten. Längsschnittstudien, die dezidiert den Auswirkungen von Selbstüber- und -unterschätzungen auf die Leistungsentwicklung nachgehen, existieren bislang jedoch kaum. In der vorliegenden Studie wurde mittels Residua...
Article
Ziel der Studie: Zur Identifikation eines familiären oder hereditären Darmkrebsrisikos wurde vom „Netzwerk gegen Darmkrebs e. V.“ ein 4-Item-Fragebogen entwickelt. Ziel dieser Studie war die Überprüfung der Validität beim Einsatz bei 40- bis 54-jährigen Personen im hausärztlichen Setting. Methodik: In vier hausärztlichen Praxen des „Forschungsnetzw...
Article
Full-text available
Background The use of comprehensive instruments for guideline appraisal is time-consuming and requires highly qualified personnel. Since practicing physicians are generally busy, the rapid-assessment Mini-Checklist (MiChe) tool was developed to help them evaluate the quality and utility of guidelines quickly. The aim of this study was to validate t...
Chapter
Full-text available
Die Bildungsstandards im Fach Geographie für den Mittleren Schul-abschluss definieren das Systemkonzept als Hauptbasiskonzept des Faches (DGfG 2012, 10). Damit dieses Konzept die ihm zugesprochene Schlüsselfunktion der Optimierung und Weiterentwick-lung geographischer Bildungsprozesse übernehmen kann, ist dessen theoretische und empirische Fundieru...
Article
The use of Information and Communication Technology (ICT) is of immense importance in today’s digital knowledge society. As a basis for private and vocational participation in society, ICT literacy has been widely discussed in recent decades. Although motivational and metacognitive facets play an important role in developing ICT literacy and compet...
Article
The study reported here investigates the validity of judgments made when aligning tests to the Common European Framework of Reference (CEFR). Listening tests operationalizing pre-defined difficulty-determining characteristics were to be aligned to CEFR levels. We employed a modified version of the item-descriptor-matching-method. Ten judges stated...
Article
Placement and screening tests serve important functions, not only with regard to placing learners at appropriate levels of language courses but also with a view to maximizing the effectiveness of administering test batteries. We examined two widely reported formats suitable for these purposes, the discrete decontextualized Yes/No vocabulary test an...
Article
The care of older people was described as involving substantial emotion-related affordances. Scholars in vocational training and nursing disagree whether emotion-related skills could be conceptualized and assessed as a professional competence. Studies on emotion work and empathy regularly neglect the multidimensionality of these phenomena and their...
Article
Full-text available
According to the German Educational Standards in Geography, system competence is a fundamental geographic concept. Hence, in the competence area of subject-specific knowledge, system competence is a key category of geographic thinking. In other words, geography lessons are to contribute considerably to a deepened understanding of complex human-envi...
Article
Full-text available
Unter Verwendung von Mehrebenen-Regressionsmodellen werden in diesem Beitrag Zusammenhänge zwischen von Altenpflegeschülerinnen und -schülern wahrgenommenen betrieblichen Ausbildungsbedingungen und der Testleistung in einem psychometrischen Test berufsfachlicher Handlungskompetenz im unmittelbar bewohnerbezogenen Bereich der Pflege älterer Menschen...
Article
Full-text available
Wie lässt sich Systemdenken messen? Darstellung eines empirisch validierten Kompetenzmodells zur Erfassung geographischer Systemkompetenz Die deutschen Bildungsstandards im Fach Geographie weisen das Systemkonzept als zentrales Basiskonzept aus. Im Kompetenzbereich Fachwissen wird Systemkompetenz damit zur Schlüsselkategorie geographischen Denke...
Article
Full-text available
Students’ performance in assessments is commonly attributed to more or less effective teaching. This implies that students’ responses are significantly affected by instruction. However, the assumption that outcome measures indeed are instructionally sensitive is scarcely investigated empirically. In the present study, we propose a longitudinal mult...