Robert A Cribbie

York University · Department of Psychology (Quantitative Methods)

p>Measurement Invariance (MI) is often concluded from a nonsignificant chi-square difference test. Researchers have also proposed using change in goodness-of-fit indices (ΔGOFs) instead. Both of these commonly used methods for testing MI have important limitations. To combat these issues, To combat these issues, it was proposed using an equivalence...

p>Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide...

p>Measurement Invariance (MI) is often concluded from a nonsignificant chi-square difference test. Researchers have also proposed using change in goodness-of-fit indices (ΔGOFs) instead. Both of these commonly used methods for testing MI have important limitations. To combat these issues, To combat these issues, it was proposed using an equivalence...

p>Statistics play an important role in psychology, but statistics modules are notoriously unpopular amongst psychology students. We examined attitudes toward statistics and attitudes toward the statistical software package R in both undergraduate and postgraduate students across the duration of a statistics module. Participants’ responses were anal...

p>Statistics play an important role in psychology, but statistics modules are notoriously unpopular amongst psychology students. We examined attitudes toward statistics and attitudes toward the statistical software package R in both undergraduate and postgraduate students across the duration of a statistics module. Participants’ responses were anal...

p> Quantitative methods (QM) dominate empirical research in psychology. Unfortunately most researchers in psychology receive inadequate training in QM. This creates a challenge for researchers who require advanced statistical methods to appropriately analyze their data. Many of the recent concerns about research quality, replicability, and report...

p>Culpepper and Aguinis (2011) highlighted the benefit of using the errors-in-variables (EIV) method to control for measurement error and obtain unbiased regression estimates. The current study investigated the EIV method and compared it to change scores and analysis of covariance (ANCOVA) in a two-group pretest-posttest design. Results indicated t...

p>Culpepper and Aguinis (2011) highlighted the benefit of using the errors-in-variables (EIV) method to control for measurement error and obtain unbiased regression estimates. The current study investigated the EIV method and compared it to change scores and analysis of covariance (ANCOVA) in a two-group pretest-posttest design. Results indicated t...

p>Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide...

p> Quantitative methods (QM) dominate empirical research in psychology. Unfortunately most researchers in psychology receive inadequate training in QM. This creates a challenge for researchers who require advanced statistical methods to appropriately analyze their data. Many of the recent concerns about research quality, replicability, and report...

A popular measure of model fit in structural equation modeling (SEM) is the standardized root mean squared residual (SRMR) fit index. Equivalence testing has been used to evaluate model fit in structural equation modeling (SEM) but has yet to be applied to SRMR. Accordingly, the present study proposed equivalence-testing based fit tests for the SRM...

A commentary on the target paper “Replication value as a function of citation impact and sample size” (https://osf.io/preprints/metaarxiv/knjea) entitled “Statistical (non)Significance ≠ (un)Successful Replication: The Importance of the Smallest Effect Size of Interest”. In this commentary, we argue that the smallest effect size of interest (or min...

Bayesian statistics has gained substantial popularity in the social sciences, particularly in psychology. Despite its growing prominence in the psychological literature, many researchers remain unacquainted with Bayesian methods. This tutorial addresses the needs of curious applied psychology researchers and introduces Bayesian analysis as an acces...

It has been suggested that equivalence testing (otherwise known as negligible effect testing) should be used to evaluate model fit within structural equation modelling (SEM). In this study, we propose novel variations of equivalence tests based on the popular root mean squared error of approximation and comparative fit index fit indices. Using Mont...

This large, international dataset contains survey responses from N = 12,570 students from 100 universities in 35 countries, collected in 21 languages. We measured anxieties (statistics, mathematics, test, trait, social interaction, performance, creativity, intolerance of uncertainty, and fear of negative evaluation), self-efficacy, persistence, and...

Self-critical perfectionism and anxiety sensitivity are potential vulnerability factors for increased distress following performance failure. We hypothesized that participants who fail a statistics quiz will have lower state self-esteem, lower positive affect, and greater negative affect at posttest than those who get a good grade, after controllin...

Equivalence testing (ET) is a framework to determine if an effect is small enough to be considered meaningless, wherein meaningless is expressed as an equivalence interval (EI). Although traditional effect sizes (ESs) are important accompaniments to ET, these measures exclude information about the EI. Incorporating the EI is valuable for quantifyin...

Psychology is currently experiencing a replication crisis, wherein many attempts to replicate past studies have failed. Pre-registration is a valuable resource to enhance the replicability of research and maintain scientific rigour. However, the incorporation of information regarding effect size magnitude, namely the minimum meaningful effect size...

This large, international dataset contains survey responses from N = 12,570 students from 100 universities in 35 countries, collected in 21 languages. We measured anxieties (statistics, mathematics, test, trait, social interaction, performance, creativity, intolerance of uncertainty, and fear of negative evaluation), self-efficacy, persistence, and...

Self-critical perfectionism and anxiety sensitivity are potential vulnerability factors for increased psychological distress following performance failure. Statistics courses are a prevalent source of distress for many students. We hypothesized that participants who fail a statistics quiz will have lower state self-esteem, lower positive affect, an...

Behavioral science researchers are often interested in whether there is negligible interaction among continuous predictors of an outcome variable. For example, a researcher might be interested in demonstrating that the effect of perfectionism on depression is very consistent across age. In this case, the researcher is interested in assessing whethe...

Reporting and interpreting effect sizes (ESs) has been recommended by all major bodies within the field of psychology. In this systematic review, we investigated the reporting of ESs in six social-personality psychology journals from 2018, given that this area has been at the center of psychology’s replication crisis. Our results highlight that alt...

An effect size (ES) provides valuable information regarding the magnitude of effects, with the interpretation of magnitude being the most important. Interpreting ES magnitude requires combining information from the numerical ES value and the context of the research. However, many researchers adopt popular benchmarks such as those proposed by Cohen....

Les variables suppressives accroissent l’efficacité prédictive d’un ou plusieurs « prédicteurs » en supprimant la variance non pertinente. Bien que la suppression statistique (SS) soit utile sur les plans théorique et statistique, aucune recherche ne s’est penchée sur la fréquence ou l’interprétation de la SS dans les ouvrages psychologiques. Deux...

Background: Many youth with neurodevelopmental disorders (NDDs) experience mental health problems such as anxiety, depression or anger, and these are often associated with impairments of cognition and emotion regulation. The mechanisms that may be linking cognitive difficulties, emotion regulation and mental health are not known. Aims: The curre...

Behavioral science researchers are often interested in whether there is negligible interaction among continuous predictors of an outcome variable. For example, a researcher might be interested in demonstrating that the effect of perfectionism on depression is very consistent across age. In this case, the researcher is interested in assessing whethe...

The flipped classroom (FC) inverts the traditional classroom by having students participate in passive aspects of learning at home and active aspects of learning in class with the guide of an instructor. The introductory statistics course for nonmath majors may be especially suited to the FC model given its unique challenges as a required course fo...

An effect size (ES) provides valuable information regarding the magnitude of effects, with the interpretation of magnitude being the most important. Interpreting ES magnitude requires combining information from the numerical ES value and the context of the research. However, many researchers adopt popular benchmarks such as those proposed by Cohen....

Reporting and interpreting effect sizes has been recommended by all major bodies within the field of Psychology. In this systematic review, we investigated the reporting of ES in six Social-Personality Psychology journals from 2018, given that this area has been at the center of Psychology’s replication crisis. Our results highlight that although E...

Measurement Invariance (MI) is often concluded from a nonsignificant chi-square difference test. Researchers have also proposed using change in goodness of fit indices (∆GOFs) instead. Both of these commonly used methods for testing MI have important limitations. To combat these issues, Yuan and Chan (2016) proposed using an equivalence test (EQ) t...

Background The objective of this study was to evaluate the psychometric properties of the 17‐item Tampa Scale for Kinesiophobia (TSK) in youth. Methods Participants were 264 children and adolescents (58.7% female, M age=14.1 years, SD age=2.51) scheduled for major surgery who were assessed before surgery, while in hospital postoperatively, and at...

This paper presents a side-by-side consideration of multiplicity control procedures and replication as solutions to the problem of multiplicity. Several independent theoretical arguments are presented which demonstrate that replication serves several important functions, and that multiplicity control procedures have a number of serious flaws. Subse...

This paper presents a side-by-side consideration of multiplicity control procedures and replication as solutions to the problem of multiplicity. Several independent theoretical arguments are presented which demonstrate that replication serves several important functions, and that multiplicity control procedures have a number of serious flaws. Subse...

Comparing the means of independent groups is a concern when the assumptions of normality and variance homogeneity are violated. Robust means modeling (RMM) was proposed as an alternative to ANOVA-type procedures when the assumptions of normality and variance homogeneity are violated. The purpose of this study is to compare the Type I error and powe...

Lay abstract: Children with autism commonly experience difficulty controlling their emotions. Although existing treatments are successful in teaching critical emotion regulation skills, not all children improve. It is important to identify the factors that influence treatment response to be able to reach more children. This study aimed to identify...

Measurement Invariance (MI) is often concluded from a nonsignificant chi-square difference test. Researchers have also proposed using change in goodness-of-fit indices (ΔGOFs) instead. Both of these commonly used methods for testing MI have important limitations. To combat these issues, To combat these issues, it was proposed using an equivalence t...

Researchers are commonly interested in comparing the means of independent groups when distributions are nonnormal and variances are unequal. Robust means modeling (RMM) has been proposed as an alternative to ANOVA-type procedures when the assumptions of normality and variance homogeneity are violated. This paper extends work comparing the Type I er...

Statistical suppression occurs when adjusting for some third variable enhances or substantially modifies the association between an initial predictor and an outcome. Although many methodologists have discussed this phenomenon, very little work has examined suppression in longitudinal regression models such as the pretest-posttest design. This resea...

Statistics play an important role in psychology, but statistics modules are notoriously unpopular amongst psychology students. We examined attitudes toward statistics and attitudes toward the statistical software package R in both undergraduate and postgraduate students across the duration of a statistics module. Participants’ responses were analys...

Background: Cardiac rehabilitation (CR) is insufficiently available; even less so in low-resource settings. Health care administrators (HAs) are responsible for ensuring CR programs are offered and resourced. This study compared HA CR attitudes in North and South America, the contextual factors associated with these attitudes, and developed a scal...

Statistics plays an extremely important role within the discipline of psychology, but statistics courses are notoriously unpopular amongst psychology students. We examined attitudes toward statistics and attitudes toward the statistical software package R in both undergraduate and graduate students in psychology across the duration of a statistics...

Suppressors are third variables that increase the predictive power of one or more predictors by suppressing their irrelevant variance when included in a regression model. Although theoretically and statistically useful, no research has addressed the frequency or interpretation of statistical suppression (SS) in the psychological literature. Two stu...

Measurement Invariance (MI) is often concluded from a nonsignificant chi-square difference test. Researchers have also proposed using change in goodness-of-fit indices ( Δ GOFs) instead. Both of these commonly used methods for testing MI have important limitations. To combat these issues, To combat these issues, it was proposed using an equivalenc...

The over-reliance on the null hypothesis significance testing framework and its accompanying tools has recently been challenged. An example of such a tool is statistical power analysis, which is used to determine how many participants are required to detect a minimally meaningful effect size in the population at a given level of power and Type I er...

Two controversial topics related to the teaching of statistics to psychology students are (a) when to introduce statistical software and (b) which statistical software package to use. The current research looked at the use of statistical software in statistics classes from every university with a psychology program in Canada. Researchers collected...

This paper contrasts and compares multiplicity control procedures and replication in the control of familywise error and the maximization of statistical power. Results of a simulation study are used to show that replication provides similar familywise error control and power, and is logically superior to, multiplicity control procedures.

Two controversial topics related to the teaching of statistics to psychology students are (a) when to introduce statistical software and (b) which statistical software package to use. The current research looked at the use of statistical software in statistics classes from every university with a psychology program in Canada. Researchers collected...

Researchers often need to consider the practical significance of a relationship. For example, interpreting the magnitude of an effect size or establishing bounds in equivalence testing requires knowledge of the meaningfulness of a relationship. However, there has been little research exploring the degree of relationship among variables (e.g., corre...

Objective: Alexithymia is a personality trait that reflects deficits in the cognitive processing and regulation of emotions (Taylor & Bagby, 2013). It has been closely linked to childhood trauma and reported by individuals presenting with other trauma-related conditions, such as posttraumatic stress disorder (PTSD), dissociation, and interpersonal...

When one wishes to show no meaningful differences among group means, equivalence tests should be used, as a nonsignificant test of mean difference does not provide evidence supporting equivalence. This research proposes two modified stepwise procedures for controlling the familywise Type I error rate, based on the Bonferroni-type correction of k²/4...

Researchers are often interested in exploring predictors of change, and commonly use a regression based model or a gain score analysis to compare degree of change across groups. Methodologists have cautioned against the use of the regression based model when there are non-random group differences at baseline because this model inappropriately corre...

Researchers are often interested in demonstrating that variables are unrelated. However, declar-ing a lack of relationship (e.g., no mean difference or no correlation) through nonrejection of the traditional null hypothesis (e.g., H_0: \mu _1- \mu _2 = 0; H_0: \rho = 0) is inappropriate. The two one-sided tests (TOST) method for testing mean equiva...

Background: The Cambridge Neuropsychological Test Automated Battery (CANTAB) is a computerized tool used to measure cognitive function in diverse populations and is sensitive for assessing developmental changes in children. Although CANTAB has been used in several countries, its applicability in a Mexican child population is unknown. This study exa...

Researchers often need to consider the practical significance of a relationship. For example, interpreting the magnitude of an effect size or establishing bounds in equivalence testing requires knowledge of the meaningfulness of a relationship. However, there has been little research exploring the degree of relationship among variables (e.g., corre...

Determining a lack of association among two or more categorical variables is frequently necessary in psychological designs such as comparative outcome analyses, assessments of group equivalence at a baseline level, and therapy outcome evaluations. Despite this, the literature rarely offers information about, or technical recommendations concerning,...

Objective: To track psychosocial well-being over 2 years following cardiac rehabilitation (CR) initiation, and its' association with heart-health behaviors. Methods: Patients from 3 CR programs were approached at their first visit, and consenters completed a survey. Participants were emailed surveys again 6 months, 1 and 2 years later. Depressiv...

Alternatives for positively skewed and heteroscedastic data include the Yuen-Welch (YW) test, data transformations, and the generalized linear model (GzLM). Because the GzLM is rarely considered in psychology compared to the other two, we compared these strategies conceptually and empirically. The YW test generally has satisfactory power, but its t...

This paper, based on two companion studies, presents an in-depth analysis of preschooler coping with vaccination pain. Study 1 employed an autoregressive cross-lagged path model to investigate the dynamic and reciprocal relationships between young children's coping responses (how they cope with pain and distress) and coping outcomes (pain behaviors...

Researchers in psychology are frequently confronted with the issue of analysing multiple relationships simultaneously. For example, this could involve multiple outcome variables or multiple predictors in a regression framework. Current recommendations typically steer researchers toward familywise or falsediscovery rate Type I error control to limit...

Some researchers in psychology have ordinarily relied on traditional linear models when assessing the relationship between predictor(s) and a continuous outcome, even when the assumptions of the traditional model (e.g., normality, homoscedasticity) are not satisfied. Of those who abandon the traditional linear model, some opt for robust versions of...

Valid use of the traditional independent samples ANOVA procedure requires that the population variances are equal. Previous research has investigated whether variance homogeneity tests, such as Levene's test, are satisfactory as gatekeepers for identifying when to use or not to use the ANOVA procedure. This research focuses on a novel homogeneity o...

Researchers are often interested in establishing equivalence of population variances. Traditional difference-based procedures are appropriate to answer questions about differences in some statistic (e.g., variances, etc.). However, if a researcher is interested in evaluating the equivalence of population variances, it is more appropriate to use a p...

Culpepper and Aguinis (2011) highlighted the benefit of using the errors-in-variables (EIV) method to control for measurement error and obtain unbiased regression estimates. The current study investigated the EIV method and compared it to change scores and analysis of covariance (ANCOVA) in a two-group pretest-posttest design. Results indicated tha...

Quantitative methods (QM) dominate empirical research in psychology. Unfortunately most researchers in psychology receive inadequate training in QM. This creates a challenge for researchers who require advanced statistical methods to appropriately analyze their data. Many of the recent concerns about research quality, replicability, and reporting p...

Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide st...

Background Mental disorders are a leading cause of disability and early mortality. The objective of this study was to describe and compare psychosocial indicators and mental health service use among ethnoculturally-diverse Ontarians. Methods This is a cross-sectional analysis of the Ontario Health Study pilot investigation. Residents were mailed a...

Although random responding is prevalent and increases Type II errors, most psychologists avoid trying to identify it because the means to do so are extremely limited. We propose the inter-item standard deviation (ISD), a statistical index of response variance, is suited for this task. We hypothesized that random responders produce large ISDs becaus...

Therapeutic alliance has been considered an important factor in child psychotherapy and is consistently associated with positive outcomes. Nevertheless, research on alliance in the context of child trauma therapy is very scarce. This study examined the relationships between child therapeutic alliance and psychopathology in an empirically supported...

In this article we argue that there are 2 dominant underlying themes in discussions of strategies for dealing with diversity-similarity and difference. When we are dealing with social groups, a number of basic psychological processes, as well as popular media and research-based narratives, make it easier to highlight difference rather than similari...

State boredom – the experience of boredom in the moment – is related to a number of psychosocial issues. Until the recent creation of the Multidimensional State Boredom Scale (MSBS), research was constrained by the lack of a comprehensive, validated measure. However, the MSBS could benefit from further evaluation. To more thoroughly validate the MS...

Measures of clinical significance offer important information about psychological interventions that cannot be garnered from tests of the statistical significance of the change from pretest to posttest. For example, post-intervention comparisons to a nonclinical group often offer valuable information about the practical value of the change that occ...

Equivalence tests are an alternative to traditional difference-based tests for demonstrating a lack of association between two variables. While there are several recent studies investigating equivalence tests for comparing means, little research has been conducted on equivalence methods for evaluating the equivalence or similarity of two correlatio...

This investigation introduces a novel tool for identifying conscientious responders (CRs) and random responders (RRs) in psychological inventory data. The Conscientious Responders Scale (CRS) is a five-item validity measure that uses instructional items to identify responders. Because each item instructs responders exactly how to answer that partic...

Objective: Clinical significance determines whether an intervention makes a real difference in the everyday life of a client. One of the most recommended approaches for conducting group-level analyses of clinical significance is to evaluate whether the treated clinical group is equivalent to a normal comparison group (normative comparisons). The p...

Objective: This study assessed a mediational model in which negative automatic thoughts and anxiety sensitivity were hypothesized to mediate the relationship between perfectionism cognitions and depressive and anxiety symptoms. Participants: Participants were undergraduate students from an urban Canadian university. The data were collected from...

In behavioral science research there is often the need to determine if an outcome variable differs, or is equivalent, across groups. Significance tests are the most prevalently applied data analysis method for this type of question. The purpose of this study was to examine how statistical tests for equivalence and difference have been applied to co...

We investigated the effectiveness treatment for substance abusing youth at an innovative residential program in Canada, the Pine River Institute (PRI). We explored (a) whether pretreatment problems were predicted by individual differences; (b) what predicted treatment progression; (c) the amount of improvement pre- to posttreatment; and (d) what pr...

The purpose of this study was to evaluate a modified test of equivalence for conducting normative comparisons when distribution shapes are non-normal and variances are unequal. A Monte Carlo study was used to compare the empirical Type I error rates and power of the proposed Schuirmann-Yuen test of equivalence, which utilizes trimmed means, with th...

Proponents of what has been termed the Gender Similarities Hypothesis (GSH) have typically relied on meta-analyses as well as the generation of nonsignificant tests of mean differences to support their argument that the genders are more similar than they are different. In the present article, we argue that alternative statistical methodologies, suc...

When structural equation modeling (SEM) analyses are conducted, significance tests for all important model relationships (parameters including factor loadings, covariances, etc.) are typically conducted at a specified nominal Type I error rate (α). Despite the fact that many significance tests are often conducted in SEM, rarely is multiplicity cont...