Rob Meijer

Rob Meijer
University of Groningen | RUG · Psychometrics and Statistics Research Program

About

189
Publications
48,140
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
4,878
Citations
Introduction
Skills and Expertise

Publications

Publications (189)
Preprint
The accessibility to electronic devices and the novel statistical methodologies available have allowed researchers to comprehend psychological processes at the individual level. However, there are still great challenges to overcome as, in many cases, collected data are more complex than the available models are able to handle. For example, most met...
Article
Soccer coaches and scouts typically assess in-game soccer performance to predict players’ future performance. However, there is hardly any research on the reliability and predictive validity of coaches’ and scouts’ performance assessments, or on strategies they can use to optimize their predictions. In the current study, we examined whether robust...
Preprint
Although mechanical combination results in more valid judgments and decisions than holistic combination, existing publications suggest that mechanical combination is rarely used in practice. Yet, these publications are either descriptions of anecdotal experiences or outdated surveys. Therefore, in several Western countries, we conducted two surveys...
Article
Full-text available
Purpose In Mokken scaling, the Crit index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions. Methods We co...
Article
Full-text available
A robust finding in psychological research is that combining information with a mechanical rule results in more valid predictions than combining information holistically in the mind. Nevertheless, information is typically combined holistically in practice , resulting in suboptimal predictions and decisions. Earlier research showed that decision mak...
Preprint
Full-text available
A robust finding in psychological research is that combining information with a mechanical rule results in more valid predictions than combining information holistically in the mind. Nevertheless, information is typically combined holistically in practice, resulting in suboptimal predictions and decisions. Earlier research showed that decision make...
Experiment Findings
Full-text available
Deze bijlage bevat (1) een beschrijving van de inhoud van de enquete, (2) de itemanalyse en betrouwbaarheidsschattingen en (3) de resultaten van een factoranalyse.
Article
Full-text available
The trait-state-occasion model (TSO) is a popular model within the latent state-trait theory (LST). The TSO allows distinguishing the trait and the state components of the psychological constructs measured in longitudinal data, while also taking into account the carry-over effects between consecutive measurements. In the present study, we extend a...
Article
Full-text available
Robust scientific evidence shows that human performance predictions are more valid when information is combined mechanically (with a decision rule) rather than holistically (in the decision-maker’s mind). Yet, information is often combined holistically in practice. One reason is that decision makers lack knowledge of evidence-based decision making....
Article
Full-text available
Traditionally, researchers have used time series and multilevel models to analyze intensive longitudinal data. However, these models do not directly address traits and states which conceptualize the stability and variability implicit in longitudinal research, and they do not explicitly take into account measurement error. An alternative to overcome...
Article
Full-text available
Scouts of soccer clubs are often the first to identify talented players. However, there is a lack of research on how these scouts assess and predict overall soccer performance. Therefore, we conducted a large-scaled study to examine the process of talent identification among 125 soccer scouts. Through an online self-report questionnaire, scouts wer...
Article
Full-text available
In personnel- and educational selection, a substantial gap exists between research and practice, since evidence-based assessment instruments and decision-making procedures are underutilized. We provide an overview of studies that investigated interventions to encourage the use of evidence-based assessment methods, or factors related to their use. T...
Article
Background In first episode psychosis (FEP) baseline negative symptoms (BNS) and relapse both predict less favorable functional outcome. Relapse-prevention is one of the most important goals of treatment. Apart from discontinuation of antipsychotics, natural causes of relapse are unexplained. We hypothesized that BNS, apart from predicting worse fu...
Preprint
Full-text available
Traditionally, researchers have used time series and multilevel models to analyze intensive longitudinal data. However, these models do not directly address traits and states which conceptualize the stability and variability implicit in longitudinal research, and they do not explicitly take into account measurement error. An alternative to overcome...
Article
Full-text available
Resilience is a key construct to understand when athletes continue to perform optimally, or when they break down. Although there is consensus that resilience can be conceptualized as a dynamic process, it remains an open question whether studying such a process on a group level adequately represents the individuals within a given sample. As a first...
Article
Full-text available
Predicting performance in soccer games has been a major focus within talent identification and development. Past research has mainly used performance levels, such as elite vs. non-elite players, as the performance to predict (i.e. the criterion). Moreover, these studies have mainly focused on isolated performance attributes as predictors of soccer...
Article
Full-text available
Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention for the effect of violations of commonly used rules of thumb. In this study, the...
Article
Full-text available
In the past decades, much research has examined the negative effects of stressors on the performance of athletes. However, according to evolutionary biology, organisms may exhibit growth under stress, a phenomenon called antifragility. For both coaches and their athletes, a key question is how to design training conditions to help athletes develop...
Preprint
Full-text available
In this study we examined the consequences of ignoring violations of assumptions un-derlying the use of sum scores in assessing attention problems (AP), and if psychomet-rically more refined models improve predictions of relevant outcomes in adulthood. Tracking Adolescents’ Individual Lives data were used. AP symptom properties were examined using...
Preprint
Full-text available
In this chapter, the practical consequences of violations of unidimensionality on selection decisions in the framework of unidimensional item response theory (IRT) models are investigated based on simulated data. The factors manipulated include the severity of violations, the proportion of misfitting items, and test length. The outcomes that were c...
Preprint
Full-text available
PurposeIn Mokken scaling, the Crit index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions. Methods We cond...
Article
Full-text available
Talent identification research in soccer comprises the prediction of elite soccer performance. While many studies in this field have aimed to empirically relate performance characteristics to subsequent soccer success, a critical evaluation of the methodology of these studies has mostly been absent in the literature. In this position paper, we disc...
Article
To investigate the effect of innovations in the teaching–learning environment, researchers often compare study results from different cohorts across years. However, variance in scores can be attributed to both random fluctuation and systematic changes due to the innovation, complicating cohort comparisons. In the present study, we illustrate how us...
Article
Full-text available
Objectives: In this study, we examined the consequences of ignoring violations of assumptions underlying the use of sum scores in assessing attention problems (AP) and if psychometrically more refined models improve predictions of relevant outcomes in adulthood. Methods: Tracking Adolescents' Individual Lives data were used. AP symptom propertie...
Article
Full-text available
A longstanding concern about admissions to higher education is the underprediction of female academic performance by admission test scores. One explanation for these findings is selection system bias, that is, not all relevant KSAOs that are related to academic performance and gender are included in the prediction model. One solution to this proble...
Preprint
Full-text available
Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention devoted to the effect of violations of commonly used rules of thumb. In this stud...
Article
Full-text available
Background The Four-Dimensional Symptom Questionnaire (4DSQ) is a self-report questionnaire designed to measure distress, depression, anxiety, and somatization. Prior to computing scale scores from the item scores, the three highest response alternatives (‘Regularly’, ‘Often’, and ‘Very often or constantly present’) are usually collapsed into one c...
Preprint
The aim of this study was twofold: First, we investigated whether scores on an admission test lead to similar predictions in future study success when administered in a proctored- and an unproctored setting. Second, we explored how Bayesian modeling can be of help to interpret admission-testing data. Results showed that the mode of administration o...
Technical Report
An evaluation of the national selection procedure for Dutch judges in training, commissioned by the Council for the Judiciary [Raad voor de Rechtspraak]
Article
Full-text available
Learners differ in their learning aptitude. Modern computerized fact-learning systems take these individual differences into account by adapting repetition schedules to the learner's characteristics. Adaptation is based on monitoring responses during learning and using these responses to inform the model's decisions about when to introduce and repe...
Article
Full-text available
In our target article, we proposed the application of the dynamical systems approach to studying how the dynamic process of resilience unfolds over time. Sparked by the commentaries by Bryan and colleagues, Galli and Pagano, and Kiefer and colleagues, we aim to provide clarifications of the dynamical systems approach as well as possible extensions...
Article
Objectives The Patient‐Reported Outcomes Measurement Information System (PROMIS) Health Organization has compiled and calibrated item banks for various domains in the United States, and these item banks have been translated into Dutch language. Methods The item banks for Anxiety and Depression have been administered in two samples, one drawn from...
Article
Full-text available
We investigated the validity of curriculum-sampling tests for admission to higher education in two studies. Curriculum-sampling tests mimic representative parts of an academic program to predict future academic achievement. In the first study, we investigated the predictive validity of a curriculum-sampling test for first year academic achievement...
Data
Descriptive statistics for predictor variables in Study 1. (PDF)
Data
Observed correlations between predictors and first year academic outcomes per cohort. (PDF)
Data
Descriptive statistics for criterion variables in Study 1. (PDF)
Data
Construct saturation multiple regression results based on uncorrected co S1. (PDF)
Data
Predictive- and incremental validity of curriculum-sampling test scores over high school GPA, based on data of applicants for whom high school GPA data were available. (PDF)
Data
Descriptive statistics for the variables in Study 2. (PDF)
Data
Incremental validity of specific skills tests over the curriculum-sampling test based on observed correlations. (PDF)
Article
Full-text available
The selection of athletes has been a central topic in sports sciences for decades. Yet, little consideration has been given to the theoretical underpinnings and predictive validity of the procedures. In this paper, we evaluate current selection procedures in sports given what we know from the selection psychology literature. We contrast the popular...
Article
Full-text available
Background: Adequate recognition of mental health problems is a prerequisite for successful treatment. Although most people tend to consult their general practitioner (GP) when they first experience mental health problems, GPs are not very well equipped to screen for various forms of psychopathology to help them determine clients' need for treatme...
Article
Full-text available
[This corrects the article on p. 305 in vol. 8, PMID: 28326049.].
Chapter
This chapter presents an overview of unidimensional item response theory (IRT) modeling. Unidimensional IRT models have become important tools to evaluate the quality of psychological and educational measurement instruments. The chapter distinguishes parametric and nonparametric IRT models, and IRT models for dichotomous and polytomous item scores....
Article
Full-text available
On the road to excellence, it is essential to develop resilience, that is, to be able to positively adapt within the context of significant adversity. Researchers tend to agree that resilience is a complex process with a multitude of underlying variables. To stimulate research on the process of resilience, we propose the dynamical system approach t...
Chapter
Higher Education Admissions Practices - edited by María Elena Oliveri January 2020
Article
Full-text available
The flipped classroom is becoming more popular as a means to support student learning in higher education by requiring students to prepare before lectures and actively engaging students during lectures. While some research has been conducted into student performance in the flipped classroom, students’ study behaviour throughout a flipped course has...
Article
Full-text available
Background The Four-Dimensional Symptom Questionnaire (4DSQ) (Huisarts Wetenschap 39: 538–47, 1996) is a self-report questionnaire developed in the Netherlands to distinguish non-specific general distress from depression, anxiety, and somatization. This questionnaire is often used in different populations and settings and there is a paper-and-penci...
Article
Background Depressive patients can present with complex and different symptom patterns in clinical care. Of these, some may report patterns that are inconsistent with typical patterns of depressive symptoms. This study aimed to evaluate the validity of person-fit statistics to identify inconsistent symptom reports and to assess the clinical usefuln...
Article
There is an increasing interest in the use of broadened criteria for admission to higher education, often assessed through noncognitive instruments. We argue that there are several reasons why, despite some significant progress, the use of noncognitive predictors to select students is problematic in high-stakes educational selection and why the inc...
Article
Full-text available
In this article, the practical consequences of violations of unidimensionality on selection decisions in the framework of unidimensional item response theory (IRT) models are investigated based on simulated data. The factors manipulated include the severity of violations, the proportion of misfitting items, and test length. The outcomes that were c...
Article
Full-text available
Assessment in higher education is challenging because teachers face more students, with less contact time as compared to primary and secondary education. Therefore, teachers and management are often interested in efficient ways of giving students diagnostic feedback and providing information on the basis of subscores is one method that is often use...
Article
Applicant perceptions of methods used in admission procedures to higher education were investigated using organizational justice theory. Applicants to a psychology study program completed a questionnaire about several admission methods. General favorability, ratings on justice dimensions, relationships between general favorability and these dimensi...
Article
Non-cognitive constructs such as personality traits and behavioral tendencies show predictive validity for academic performance and incremental validity over and above cognitive constructs. Therefore, non-cognitive predictors are increasingly used in admission procedures for higher education, typically measured using-self-report instruments. It is...
Article
Person misfit on a self-report measure refers to a response pattern that is unlikely given a theoretical measurement model. Person misfit may reflect low quality self-report data, for example due to random responding or misunderstanding of items. However, recent research in the context of psychopathology suggests that person misfit may reflect atyp...
Article
Full-text available
Checking the validity of test scores is important in both educational and psychological measurement. Person-fit analysis provides several statistics that help practitioners assessing whether individual item score vectors conform to a prespecified item response theory model or, alternatively, to a group of test takers. Software enabling easy access...
Article
Full-text available
Background: In search of empirical classifications of depression and anxiety, most subtyping studies focus solely on symptoms and do so within a single disorder. This study aimed to identify and validate cross-diagnostic subtypes by simultaneously considering symptoms of depression and anxiety, and disability measures. Method: A large cohort of...
Article
Full-text available
In this article, we discuss the practical usefulness of selecting future medical students on the basis of increasingly popular non-academic tests (eg multiple mini-interviews, situational judgment tests) in addition to academic tests. Non-academic tests assess skills such as ethical decision making, communication and collaboration skills, or traits...
Article
Computerized adaptive tests (CATs) for positive and negative psychotic experiences were developed and tested in N = 5705 help‐seeking, non‐psychotic young individuals. Instead of presenting all items, CATs choose a varying number of different items during test administration depending on respondents' previous answers, reducing the average number of...
Article
Full-text available
We studied the validity of two methods for predicting academic performance and student-program fit that were proximal to important study criteria. Applicants to an undergraduate psychology program participated in a selection procedure containing a trial-studying test based on a work sample approach, and specific skills tests in English and math. Te...
Data
Dataset Psychology Applicants. Uncommon values were merged to categories to assure anonymity of the participants. (SAV)
Article
High data quality is an important prerequisite for sound empirical research. Meade and Craig (2012) and Huang, Curran, Keeney, Poposki, and DeShon (2012) discussed methods to detect unmotivated or careless respondents in large web-based questionnaires. We first discuss these methods and present multi-test extensions of person-fit statistics as alte...