Daniel McNeish

Arizona State University | ASU · Department of Psychology

Multilevel models are an increasingly popular method to analyze data that originate from a clustered or hierarchical structure. To effectively utilize multilevel models, one must have an adequately large number of clusters; otherwise, some model parameters will be estimated with bias. The goals for this paper are to (1) raise awareness of the probl...

In psychology and the behavioral sciences generally, the use of the hierarchical linear model (HLM) and its extensions for discrete outcomes are popular methods for modeling clustered data. HLM and its discrete outcome extensions, however, are certainly not the only methods available to model clustered data. Although other methods exist and are wid...

Empirical studies in psychology commonly report Cronbach's alpha as a measure of internal consistency reliability despite the fact that many methodological studies have shown that Cronbach's alpha is riddled with problems stemming from unrealistic assumptions. In many circumstances, violating these assumptions yields estimates of reliability that a...

Clustered data are common in many fields. Some prominent examples of clustering are employees clustered within supervisors, students within classrooms, and clients within therapists. Many methods exist that explicitly consider the dependency introduced by a clustered data structure, but the multitude of available options has resulted in rigid disci...

Technological advances have led to an increase in intensive longitudinal data and the statistical literature on modeling such data is rapidly expanding, as are software capabilities. Common methods in this area are related to time-series analysis, a framework that historically has received little exposure in psychology. There is a scarcity of psych...

Intensive longitudinal designs are increasingly popular for assessing moment-to-moment changes in mood, affect, and interpersonal or health behavior. Compliance in these studies is never perfect given the high frequency of data collection, so missing data are unavoidable. Nonetheless, there is relatively little existing research on missing data wit...

Scale scores in psychology studies are commonly accompanied by a reliability coefficient like alpha. Coefficient alpha is an index that summarizes reliability across the entire score distribution, implying equal precision for all scores. However, an underappreciated fact is that reliability can be conditional such that scores in certain parts of th...

Introduction Smoking, obesity, and insufficient physical activity are modifiable health risk behaviors. Self-regulation is one fundamental behavior change mechanism often incorporated within digital therapeutics as it varies momentarily across time and contexts and may play a causal role in improving these health behaviors. However, the role of mom...

The standardized root mean squared residual (SRMR) is commonly reported to evaluate approximate fit of latent variable models. As traditionally defined, SRMR summarizes the discrepancy between observed covariance elements and implied covariance elements. However, current applications of latent variable models often include additional features like...

Recent reviews report that about 80% of empirical factor analyses are applied to Likert-type responses and that it is exceedingly common to treat Likert-type item responses as continuous. However, traditional model fit index cutoffs like the root-mean-square error of approximation ≤ .06 or comparative fit index ≥ .95 were derived to have 90+% sensi...

Factor analysis is commonly used in behavioral sciences to measure latent constructs, and researchers routinely consider approximate fit indices to ensure adequate model fit and to provide important validity evidence. Due to a lack of generalizable fit index cutoffs, methodologists suggest simulation-based methods to create customized cutoffs that...

Research in education and behavioral sciences often involves the use of latent variable models that are related to indicators, as well as related to covariates or outcomes. Such models are subject to interpretational confounding, which occurs when fitting the model with covariates or outcomes alters the results for the measurement model. This has r...

This paper reflects on some practical implications of the excellent treatment of sum scoring and classical test theory (CTT) by Sijtsma et al. (Psychometrika 89(1):84–117, 2024). I have no major disagreements about the content they present and found it to be an informative clarification of the properties and possible extensions of CTT. In this pape...

Background The Evidence-Based Practice Attitude Scale (EBPAS) is a widely used measurement tool to assess mental health providers’ attitudes toward adopting research-based interventions. To date, this scale has yet to be used or validated among mental health professionals in Latin America. This study investigated the factor structure, psychometric...

Scale validation is vital to psychological research because it ensures that scores from measurement scales represent the intended construct. Fit indices are commonly used to provide quantitative evidence that a proposed factor structure is plausible. However, there is a mismatch between guidelines for evaluating fit of the factor models and the dat...

Psychological data are often clustered within organizational units, which violates the independence assumption in standard regression models. Clustered errors, multilevel models, and fixed-effects models all address this issue, but in different ways. Disciplinary preferences for approaching clustered data are strong, which can restrict questions re...

There is high-level interest in diversifying workforces, which has led organizations—including the U.S. Armed Forces—to reevaluate recruiting and selection practices. The U.S. Coast Guard (USCG) has encountered particular difficulties in diversifying its workforce, and it relies mainly on the Armed Services Vocational Aptitude Battery (ASVAB) for a...

Purpose: To examine patterns in adolescent and young adult tobacco use, comparing Latinx foreign-born children and children of foreign-born parents (i.e., children of immigrants(COI)) to Latinx US-born children of US-born parents (i.e., children of nonimmigrants,(CONI)) and to CONI White youth who grew up in small and rural towns. Methods: Data...

A recent review found that 11% of published factor models are hierarchical with second-order factors. However, dedicated recommendations for evaluating hierarchical model fit have yet to emerge. Traditional benchmarks like RMSEA<0.06 or CFI>0.95 are often consulted, but they were never intended to generalize to hierarchical models. Through simulati...

Intensive longitudinal designs are increasingly popular, as are dynamic structural equation models (DSEM) to accommodate unique features of these designs. Many helpful resources on DSEM exist, though they focus on continuous outcomes while categorical outcomes are omitted, briefly mentioned, or considered as a straightforward extension. This viewpo...

To evaluate the fit of a confirmatory factor analysis model, researchers often rely on fit indices such as SRMR, RMSEA, and CFI. These indices are frequently compared to benchmark values of .08, .06, and .96, respectively, established by Hu and Bentler (Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus n...

Factor analysis is often used to model scales created to measure latent constructs, and internal structure validity evidence is commonly assessed with indices like RMSEA, and CFI. These indices are essentially effect size measures and definitive benchmarks regarding which values connote reasonable fit have been elusive. Simulations from the 1990s s...

Objectives: This uncontrolled pilot study examined the feasibility, acceptability, and preliminary HIV and psychological health effects of iTHRIVE 365, a multicomponent intervention designed by and for Black same gender loving men (SGLM) to promote: health knowledge and motivation, Black SGLM social support, affirming healthcare, and housing and o...

Much of the existing longitudinal mediation literature focuses on panel data where relatively few repeated measures are collected over a relatively broad timespan. However, technological advances in data collection (e.g., smartphones, wearables) have led to a proliferation of short duration, densely collected longitudinal data in behavioral researc...

Commentary in Widaman and Revelle (2022) argued that sum scoring is justified as long as unidimensionality holds because sum score reliability is defined. My response begins with a review of the literature supporting the perspective we adopted in the original article. I then conduct simulation studies to assess the psychometric properties of sum sc...

Educational researchers continue to debate the relative contribution of individual and environmental factors to learning. Concomitant with the proliferation of longitudinal educational testing following students and schools over time, recent research has shown that nonlinear mixed effect models can be parameterized to directly estimate quantities m...

Many educators assume that choice in writing leads to better writing outcomes; however, there are few studies to support this belief. In the present study, we examined the effects of choice and preference on writing quality with college students. The students wrote two argumentative essays on controversial topics in special education. For the topic...

Many behavioral researchers are interested in measuring constructs such as mood, affect, or cognition—all of which cannot be observed directly. Instead, researchers administer items from surveys, tests, or scales to indirectly measure aspects of the construct. Psychometrics is a branch of statistics dedicated to determining whether scores created f...

The degree to which test scores can support justified and fair decisions about demographically diverse participants has been an important aspect of educational and psychological testing for millennia. In the last 30 years, this aspect of measurement has come to be known as consequential validity, and it has sparked scholarly debate as to how respon...

Background: The aim of this study was to identify distinct trajectories of BMI growth from 2 to 7.5 years and examine their associations with markers of cardiometabolic risk at age 7.5 years among a sample of low-income Mexican American children. Methods: This longitudinal cohort study recruited 322 mother-child dyads to participate prenatally a...

Dynamic fit index cutoffs are a recently proposed combination of methodological developments and software used to determine if a factor analysis model provides reasonable fit to the data to which it is applied. The associated Shiny Application and R package attempt to provide users with accessible tools to derive customized fit index cutoffs withou...

Though the Actor-Partner Interdependence Model (APIM) has been extended to accommodate longitudinal data in the multilevel modeling (MLM) and structural equation modeling (SEM) frameworks, intensive longitudinal dyadic data with many (20+) timepoints provide technical challenges for researchers that neither framework fully addresses. We provide an...

Growth mixture models (GMMs) are a popular method to identify latent classes of growth trajectories. One shortcoming of GMMs is nonconvergence, which often leads researchers to apply covariance equality constraints to simplify estimation, though this may be a dubious assumption. Alternative model specifications have been proposed to reduce nonconve...

Assessing whether a multiple-item scale can be represented with a one-factor model is a frequent interest in behavioral research. Often, this is done in a factor analysis framework with approximate fit indices like RMSEA, CFI, or SRMR. These fit indices are continuous measures, so values indicating acceptable fit are up to interpretation. Cutoffs s...

Objective: The relationship between smoking and adolescents' peer relationships is complex, with studies showing increased risk of smoking for adolescents of both very high and very low social position. A key question is whether the impact of social position on smoking depends on an adolescent's level of coping motives (i.e., their desire to use s...

The current paper is motivated by longitudinal progress tests given to medical students in the United Kingdom, which are used to assess students' applied medical knowledge during their learning programme. The main analytic interest is the maximum competency each student achieves on the assessment and the point in the programme at which each student...

Use of Bayesian methods has proliferated in recent years as technological and software developments have made Bayesian methods more approachable for researchers working with empirical data. Connected with the increased usage of Bayesian methods in empirical studies is a corresponding increase in recommendations and best practices for Bayesian metho...

Model fit assessment is a central component of evaluating confirmatory factor analysis models and the validity of psychological assessments. Fit indices remain popular and researchers often judge fit with fixed cutoffs derived by Hu and Bentler (1999). Despite their overwhelming popularity, methodological studies have cautioned against fixed cutoff...

Researchers have noted a nonlinear association between reading instruction dosage (i.e., hours of instruction) and reading outcomes for Grade K–3 students with reading difficulties (K–3 SWRD). In this article, we propose a nonlinear meta-analysis as a method to identify both the maximum effect size and optimal dosage of reading interventions for K–...

Individual differences in the timing of developmental processes are often of interest in longitudinal studies, yet common statistical approaches to modeling change cannot directly estimate the timing of when change occurs. The time-to-criterion framework was recently developed to incorporate the timing of a prespecified criterion value; however, th...

Context-appropriate infant physiological functioning may support emotion regulation and mother-infant emotion coregulation. Among a sample of 210 low-income Mexican-origin mothers and their 24-week-old infants, dynamic structural equation modeling (DSEM) was used to examine whether within-infant vagal functioning accounted for between-dyad differen...

Growth mixture models (GMMs) are applied to intervention studies with repeated measures to explore heterogeneity in the intervention effect. However, traditional GMMs are known to be difficult to estimate, especially at sample sizes common in single-center interventions. Common strategies to coerce GMMs to converge involve post hoc adjustments to t...

A large body of literature suggests that parent-child separation predicts child maladjustment. However, further advancement in methodology is needed to account for heterogeneity in types of separation. Additionally, given a lack of research examining different types of separation as predictors of offspring substance use, further research into this...

Technological advances have increased the prevalence of intensive longitudinal data as well as statistical techniques appropriate for these data, such as dynamic structural equation modeling (DSEM). Intensive longitudinal designs often investigate constructs related to affect or mood and do so with multiple item scales. However, applications of int...

Growth mixture models (GMMs) are a popular method to identify latent classes of growth trajectories. One shortcoming of GMMs is nonconvergence, which often leads researchers to apply covariance equality constraints to simplify estimation. This approach is criticized because it introduces a dubious homoskedasticity assumption across classes. Alterna...

Growth mixture models (GMMs) are a popular method to uncover heterogeneity in growth trajectories. Harnessing the power of GMMs in applications is difficult given the prevalence of nonconvergence when fitting GMMs to empirical data. GMMs are rooted in the random effect tradition and nonconvergence often leads researchers to modify their intended mo...

Research conducted in US schools shows Summer learning loss in test scores. If this Summer loss is not incorporated into models of student ability growth, assumptions will be violated because Fall scores will be overestimated and Spring scores will be underestimated, which can be particularly problematic when evaluating teacher or school effectiven...

Deciding which random effects to retain is a central decision in mixed effect models. Recent recommendations advise a maximal structure whereby all theoretically relevant random effects are retained. Nonetheless, including many random effects often leads to nonpositive definiteness. A typical remedy is to simplify the random effect structure by rem...

Objective: To characterize the heterogeneity in response to lifestyle intervention among Latino adolescents with obesity. Research design & methods: We conducted secondary data analysis of 90 Latino adolescents (age 15.4±0.9 y, female 56.7%) with obesity (BMI% 98.1±1.5%) that were enrolled in a 3-month lifestyle intervention and were followed fo...

Model fit assessment is a central component of evaluating confirmatory factor analysis models. Fit indices like RMSEA, SRMR, and CFI remain popular and researchers often judge fit based on suggestions from Hu and Bentler (1999), who derived cutoffs that distinguish between fit index distributions of true and misspecified models. However, methodolog...

A common way to form scores from multiple-item scales is to sum responses of all items. Though sum scoring is often contrasted with factor analysis as a competing method, we review how factor analysis and sum scoring both fall under the larger umbrella of latent variable models, with sum scoring being a constrained version of a factor analysis. Des...

Standard multilevel models focus on variables that predict the mean while the within-group variability is largely treated as a nuisance. Recent work has shown the advantage of including predictors for both the mean (the location submodel) and the variability (the scale submodel) within a single model. Constrained versions of the model can be fit in...

Standard multilevel models focus on variables that predict the mean while the within-group variability is largely treated as a nuisance. Recent work has shown the advantage of including predictors for both the mean (the location submodel) and the variability (the scale submodel) within a single model. Constrained versions of the model can be fit in...

Scholars have lamented that current methods of assessing student performance do not align with contemporary views of learning as situated within students, contexts, and time. Here, we introduce and describe one theoretical-psychometric paradigm—termed dynamic measurement—designed to provide a valid representation of the way students respond to scho...

Given inconsistent findings emerging in the literature between motherhood and emotional well-being, it is important to employ cutting-edge methods to evaluate mothers' dynamic emotional experiences. As anticipated by theory, attachment anxiety and avoidance may uniquely predict fluctuations in mothers' positive emotion, which may be yoked in partic...

Change over time is frequently nonlinear, which can present unique statistical challenges. Generally, different approaches for nonlinear growth engage in a tradeoff between interpretable parameters, expedient estimation, or how specific the model must be about the nature of the nonlinearity. Latent basis models are one method that can circumvent tr...

Changes to educational policies have proliferated testing data to include multiple-administration assessments that repeatedly measure student performance over time. Psychometric models—extended for this type of data—estimate quantities typically associated with assessments that are given once, such as ability at a specific time point. This article...

Perhaps more than at any other time in history, the development of mathematical skill is critical for the long-term success of students. Unfortunately, on average, U.S. students lag behind their peers in other developed countries on mathematics outcomes, and within the United States, an entrenched mathematics achievement gap exists between students...

Psychometric models for longitudinal test scores typically estimate quantities associated with single-administration tests, like ability at each time-point. However, models for longitudinal tests have not considered opportunities to estimate new quantities that are unavailable from single-administration tests. Specifically, we discuss dynamic measu...

Growth mixture models (GMMs) are prevalent for modeling unknown population heterogeneity via distinct latent classes. However, GMMs are riddled with convergence issues, often requiring researchers to atheoretically alter the model with cross-class constraints simply to obtain convergence. We discuss how within-class random effects in GMMs exacerbat...

Time-varying covariates (TVCs) are a common component of growth models. Though mixed effect models (MEMs) and latent curve models (LCMs) are often seen as interchangeable, LCMs are generally more flexible for accommodating TVCs. Specifically, the standard MEM constrains the effect of TVCs across time-points whereas the typical LCM specification can...

The current study examined whether social status and social integration, two related but distinct indicators of an adolescent's standing within a peer network, mediate the association between risky symptoms (depressive symptoms and deviant behavior) and substance use across adolescence. The sample of 6,776 adolescents participated in up to seven wa...

The present study investigated the identification of end of first grade (n = 125) and end of third grade (n = 77) reading comprehension difficulties using beginning of first grade decoding-related and language-related predictors. Reading comprehension was defined using a composite of three standardized reading comprehension measures. Students at or...

In small sample contexts, Bayesian estimation is often suggested as a viable alternative to frequentist estimation, such as maximum likelihood estimation. Our systematic literature review is the first study aggregating information from numerous simulation studies to present an overview of the performance of Bayesian and frequentist estimation for s...

Dynamic measurement modeling (DMM) is a psychometric paradigm that uses longitudinal data to estimate individual students' growth in measured skills over the course of an educational program (i.e., growth scores). DMM represents a more formal way of assessing learning progress across the health professional education continuum. In this report, the...

Effect partitioning is almost exclusively performed with multilevel models (MLMs) – so much so that some have considered the two to be synonymous. MLMs are able to provide estimates with desirable statistical properties when data come from a hierarchical structure; but the random effects included in MLMs are not always integral to the analysis. As...

Advances in data collection have made intensive longitudinal data easier to collect, unlocking potential for methodological innovations to model such data. Dynamic structural equation modeling (DSEM) is one such methodology but recent studies have suggested that its small N performance is poor. This is problematic because small N data are omniprese...

Dynamic measurement modeling (DMM) has been shown to improve the consequential validity of longitudinal mathematics assessment in the Early Childhood Longitudinal Study-Kindergarten (ECLS-K) database. Here, the authors demonstrate the capability of DMM to similarly improve the consequential validity of ECLS-K reading assessment through the estimati...

The goal of the present study was to compare children's word learning through print text, video, and electronic text in the context of a cross-age peer-learning program implemented in linguistically diverse kindergarten and fourth grade classrooms that included English Learners (ELs) and their non-EL peers. Children were assessed at pre- and post-t...

Recent methodological studies have investigated the properties of multilevel models with small samples. Previous work has primarily focused on continuous outcomes and little attention has been paid to count outcomes. The estimation of count outcome models can be difficult because the likelihood has no closed-form solution, meaning that approximatio...

This study employed a mixed method, longitudinal design to examine adolescent beliefs about their competence, control, and social belongingness (emotional support from teachers and peers) across the transition from middle school (8th grade) to high school (9th grade). Qualitative data based on open-ended questions suggested that students (N = 93; 1...

Dynamic Measurement Modeling (DMM) is a recent framework for measuring developing constructs whose manifestation occurs after an assessment is administered (e.g., learning capacity). Empirical studies have suggested that DMM may improve consequential validity of test scores because DMM learning capacity estimates were shown to be much less related...

Debate continues about whether the likelihood ratio test (TML) or goodness-of-fit indices are most appropriate for assessing data-model fit in structural equation models. Though potential advantages and disadvantages of these methods with large samples are often discussed, shortcomings concomitant with smaller samples are not. This paper aims to (1...

Dynamic measurement modeling (DMM) has been shown to improve the consequential validity of longitudinal mathematics assessment in the Early Childhood Longitudinal Study-Kindergarten (ECLS-K) database. Here, the authors demonstrate the capability of DMM to similarly improve the consequential validity of ECLS-K reading assessment through the estimati...

Objective: The current study examined whether an adolescent's standing within a school-bounded social network moderated the association between depressive symptoms and substance use across adolescence as a function of developmental and demographic factors (gender, parental education, and race/ethnicity). Method: The sample of 6,776 adolescents p...

Lance, Beck, Fan, and Carter (2016) recently advanced six new fit indices and associated cut-off values for assessing data-model fit in the structural portion of traditional latent variable path models. The authors appropriately argued that, while most researchers’ theoretical interest rests with the latent structure, they still rely on indices of...

We examined the mediating role of students’ interpersonal and academically related social goals in linking students’ perceptions of teacher and peer personal and academic emotional supports to their classroom behavior (prosocial, social responsibility) in a sample of young adolescents (n = 3,092) from 7 schools from the mid-Atlantic, Midwest, and S...

External misspecification, the omission of key variables from a structural model, can fundamentally alter the inferences one makes without such variables present. This article presents two strategies for dealing with omitted variables, the first a fixed parameter approach incorporating the omitted variable into the model as a phantom variable where...

In psychology, mixed effects models and latent curve models are both widely used to explore growth over time. Despite the widespread popularity, there remains some confusion regarding the overlap of these different approaches. Recent articles have shown that the two modeling frameworks are mathematically equivalent in many cases which is often inte...

Small samples are common in growth models due to financial and logistical difficulties of following people longitudinally. For similar reasons, longitudinal studies often contain missing data. Though full information maximum likelihood (FIML) is popular to accommodate missing data, the limited number of studies in this area have found that FIML ten...

Single-timepoint educational measurement practices are capable of assessing student ability at the time of testing but are not designed to be informative of student capacity for developing in any particular academic domain, despite commonly being used in such a manner. For this reason, such measurement practice systematically underestimates the pot...