Herbert Hoijtink

• Professor at Utrecht University

In this paper a method is proposed to determine whether the result from an original study is corroborated in a replication study. The paper is illustrated using two replication studies and the corresponding original studies from the Reproducibility Project: Psychology by the Open Science Collaboration. This method emphasizes the need to determine w...

Task-related functional MRI (fMRI) studies need to be properly powered with an adequate sample size to reliably detect effects of interest. But for most fMRI studies, it is not straightforward to determine a proper sample size using power calculations based on published effect sizes. Here, we present an alternative approach of sample size estimatio...

People suffering from dysphoria retrieve autobiographical memories distorted in content and affect, which may contribute to the aetiology and maintenance of depression. However, key memory difficulties in dysphoria remain elusive because theories disagree how memories of different valence are altered. Here, we assessed the psychophysiological expre...

Researchers can express their expectations with respect to the group means in an ANOVA model through equality and order constrained hypotheses. This paper introduces the R package SSDbain, which can be used to calculate the sample size required to evaluate (informative) hypotheses using the Approximate Adjusted Fractional Bayes Factor (AAFBF) for o...

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian e...

In the current study, we introduce the prior predictive p-value as a method to test replication of an analysis of variance (ANOVA). The prior predictive p-value is based on the prior predictive distribution. If we use the original study to compose the prior distribution, then the prior predictive distribution contains datasets that are expected giv...

The Akaike information criterion for model selection presupposes that the parameter space is not subject to order restrictions or inequality constraints. Anraku (1999) proposed a modified version of this criterion, called the order-restricted information criterion, for model selection in the one-way analysis of variance model when the population me...

The Akaike information criterion for model selection presupposes that the parameter space is not subject to order restrictions or inequality constraints.Anraku (1999) proposed a modified version of this criterion, called the order-restricted information criterion, for model selection in the one-way analysis of variance model when the population mea...

In the social and behavioral sciences, it is often not interesting to evaluate the null hypothesis by means of a p-value. Researchers are often more interested in quantifying the evidence in the data (as opposed to using p-values) with respect to their own expectationsrepresented by equality and/or inequality constrained hypotheses (as opposed to t...

The Bayes factor is increasingly used for the evaluation of hypotheses. These may betraditional hypotheses specified using equality constraints among the parameters of thestatistical model of interest or informative hypotheses specified using equality andinequality constraints. So far no attention has been given to the computation of Bayesfactors f...

There have been considerable methodological developments of Bayes factors for hypothesis testing in the social and behavioral sciences, and related fields. This development is due to the flexibility of the Bayes factor for testing multiple hypotheses simultaneously, the ability to test complex hypotheses involving equality as well as order constrai...

In the social and behavioral sciences, it is often not interesting to evaluate the null hypothesis by means of a p-value. Researchers are often more interested in quantifying the evidence in the data (as opposed to using p-values) with respect to their own expectations represented by equality and/or inequality constrained hypotheses (as opposed to...

Being exposed to narrative fiction may provide us with practice in dealing with social interactions and thereby enhance our ability to engage in mentalizing (understanding other people’s mental states). The current study uses a confirmatory Bayesian approach to assess the relationship between mentalizing and both the self-reported frequency of expo...

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian e...

This paper presents a novel Bayesian variable selection approach that accounts for the sign of the regression coefficients based on multivariate one-sided tests. We propose a truncated g prior to specify a prior distribution of coefficients with anticipated signs in a given model. Informative priors for the direction of the effects can be incorpora...

When two independent means μ1 and μ2 are compared, H0 : μ1 = μ2, H1 : μ1≠μ2, and H2 : μ1 > μ2 are the hypotheses of interest. This paper introduces the R package SSDbain, which can be used to determine the sample size needed to evaluate these hypotheses using the approximate adjusted fractional Bayes factor (AAFBF) implemented in the R package bain...

This Teacher’s Corner paper introduces Bayesian evaluation of informative hypotheses for structural equation models, using the free open-source R packages bain, for Bayesian informative hypothesis testing, and lavaan, a widely used SEM package. The introduction provides a brief non-technical explanation of informative hypotheses, the statistical un...

There has been a tremendous methodological development of Bayes factors for hypothesis testing in the social and behavioral sciences, and related fields. This development is due to the flexibility of the Bayes factor for testing multiple hypotheses simultaneously, the ability to test complex hypotheses involving equality as well as order constraint...

Researchers can express expectations regarding the ordering of group means in simple order constrained hypotheses, for example $H_i: \mu_1>\mu_2>\mu_3$, $H_c: \text{ not } H_i$, and $H_{i'}:\mu_3>\mu_2>\mu_1$. They can compare these hypotheses by means of a Bayes factor, the relative evidence for two hypotheses. The required sample size for a hypot...

The way humans perceive the outcomes of their actions is strongly colored by their expectations. These expectations can develop over different timescales and are not always complementary. The present work examines how long-term (structural) expectations – developed over a lifetime - and short-term (contextual) expectations jointly affect perception...

This paper presents a new statistical method and accompanying software for the evaluation of order constrained hypotheses in structural equation models (SEM). The method is based on a large sample approximation of the Bayes factor using a prior with a data-based correlational structure. An efficient algorithm is written into an R package to ensure...

Learning about hypothesis evaluation using the Bayes factor could enhance psychological research. The Bayes factor quantifies the support in the data for two competing hypotheses. These may be the traditional null and alternative hypotheses, but these may also be informative hypotheses like m1 > m2 > m3 and (m1 − m2) > (m2 − m3) where m1, m2, and m...

Learning about hypothesis evaluation using the Bayes factor could enhance psychologicalresearch. In contrast to null-hypothesis significance testing: it renders the evidence in favorof each of the hypotheses under consideration (it can be used to quantify support for thenull-hypothesis) instead of a dichotomous reject/do-not-reject decision; it can...

In the current study, we explain how replication of an analysis of variance can be tested with the prior predictive p-value. That is, we test to what degree the new data deviates from data that can predicted based on the original results, considering relevant features of the original study. The central role of claims by the original study is one of...

The software package Bain can be used for the evaluation of informative hypotheses with respect to the parameters of a wide range of statistical models. For pairs of hypotheses the support in the data is quantified using the approximate adjusted fractional Bayes factor (BF). Currently, the data have to come from one population or have to consist of...

The Bayes factor is increasingly used for the evaluation of hypotheses. These may be traditional hypotheses specified using equality constraints among the parameters of the statistical model of interest or informative hypotheses specified using equality and inequality constraints. Thus far, no attention has been given to the computation of Bayes fa...

Analyses are mostly executed at the population level, whereas in many applications the interest is on the individual level instead of the population level. In this paper, multiple N = 1 experiments are considered, where participants perform multiple trials with a dichotomous outcome in various conditions. Expectations with respect to the performanc...

Informative hypotheses are increasingly being used in psychological sciences because they adequately capture researchers’ theories and expectations. In the Bayesian framework, the evaluation of informative hypotheses often makes use of default Bayes factors such as the fractional Bayes factor. This paper approximates and adjusts the fractional Baye...

Research has shown that independent groups often differ not only in their means, but also in their variances. Comparing and testing variances is therefore of crucial importance to understand the effect of a grouping variable on an outcome variable. Researchers may have specific expectations concerning the relations between the variances of multiple...

Muth?n and Asparouhov (2012) propose to evaluate model fit in structural equation models based on approximate (using small variance priors) instead of exact equality of (combinations of) parameters to zero. This is an important development that adequately addresses Cohen's (1994) The Earth is Round (p < .05), which stresses that point null-hypothes...

The purpose of the current study was to apply and evaluate a procedure to elicit expert judgments about correlations, and to update this information with empirical data. The result is a face-to-face group elicitation procedure with as its central element a trial roulette question that elicits experts' judgments expressed as distributions. During th...

The assumption of latent monotonicity in item response theory models for dichotomous data cannot be evaluated directly, but observable consequences such as manifest monotonicity facilitate the assessment of latent monotonicity in real data. Standard methods for evaluating manifest monotonicity typically produce a test statistic that is geared towar...

This paper investigates the classical type I and type II error probabilities of default Bayes factors for a Bayesian t test. Default Bayes factors quantify the relative evidence between the null hypothesis and the unrestricted alternative hypothesis without needing to specify prior distributions for the unknown parameters based on one's prior belie...

An N=1 study may consist of a sequence of (placebo) treatments followed by multiple measurements of an outcome variable. An example is the study by Vrinten, et al. who evaluate the effect of intravenous immunoglobulin G on inflammation in hereditary neuropathy with liability to pressure palsy. In this study there was a sequence of seven (placebo) t...

An N=1 study may consist of a sequence of (placebo) treatments followed by multiple measurements of an outcome variable. An example is the study by Vrinten, et al. who evaluate the effect of intravenous immunoglobulin G on inflammation in hereditary neuropathy with liability to pressure palsy. In this study there was a sequence of seven (placebo) t...

Hereditary neuropathy with liability to pressure palsy (HNPP; tomaculous neuropathy) is a rare autosomal dominant disorder caused by a loss of function of the peripheral myelin protein 22 gene ( PMP22 ; OMIM #601097) for which no curative treatment exists. Symptoms consist of recurrent painless episodes of focal sensory loss and muscle weakness, wh...

Meta-analysis and Bayesian informative prior distributions are used for updating knowledge about treatment effects with new data. When the available data come from slightly different study populations, or from slightly different trials as compared with the new data, the researcher has to specify study weights to control the influence of the histori...

Bayesian evaluation of inequality constrained hypotheses enables researchers to investigate their expectations with respect to the structure among model parameters. This article proposes an approximate Bayes procedure that can be used for the selection of the best of a set of inequality constrained hypotheses based on the Bayes factor in a very gen...

There exist diverse approaches that can be used for cognitive diagnostic assessment, such as mastery testing, constrained latent class analysis, rule space methodology, diagnostic cognitive modeling, and person-fit analysis. Each of these approaches can be used within 1 of the 4 psychometric perspectives on diagnostic testing discussed by Borsboom...

Four different patterns of biased ratings of facial expressions of emotions have been found in socially anxious participants: higher negative ratings of (1) negative, (2) neutral, and (3) positive facial expressions than nonanxious controls. As a fourth pattern, some studies have found no group differences in ratings of facial expressions of emotio...

The generalized order-restricted information criterion (GORIC) is a generalization of the Akaike information criterion such that it can evaluate hypotheses that take on specific, but widely applicable, forms (namely, closed convex cones) for multivariate normal linear models. It can examine the traditional hypotheses H0: β1,1 = = βt,k and Hu: β1,1,...

Executive functions (EF) are closely related to math performance. Little is known, however, about the role of EF in numerical magnitude skills (NS), although these skills are widely acknowledged to be important precursors of math learning. The current study focuses on the different roles of updating, shifting, and inhibition in NS.EF and NS were as...

Half in jest we use a story about a black bear to illustrate that there are some discrepancies between the formal use of the p-value and the way it is often used in practice. We argue that more can be learned from data by evaluating informative hypotheses, than by testing the traditional null hypothesis. All criticisms of classical null hypothesis...

Researchers in the behavioural and social sciences often have expectations that can be expressed in the form of inequality constraints among the parameters of a structural equation model resulting in an informative hypothesis. The question they would like an answer to is "Is the Hypothesis Correct" or "Is the hypothesis incorrect?". We demonstrate...

The generalized order-restricted information criterion (goric) is a model selection criterion which can, up to now, solely be applied to the analysis of variance models and, so far, only evaluate restrictions of the form Rθ≤0Rθ≤0, where θθ is a vector of k group means and R a cm×kcm×k matrix. In this paper, we generalize the goric in two ways: (i)...

Unlabelled: Ratings of parents that have participated in a parent training for child externalizing behaviour problems might be biased (e.g., they may report symptom reduction to reward their own endeavours for attending the training). The potential for bias in parent ratings was investigated in a secondary analysis of an effectiveness study of a p...

In many types of statistical modeling, inequality constraints are imposed between the parameters of interest. As we will show in this paper, the DIC (i.e., posterior Deviance Information Criterium as proposed as a Bayesian model selection tool by Spiegelhalter, Best, Carlin, & Van Der Linde, 2002) fails when comparing inequality constrained hypothe...

This paper presents a refinement of the Bayesian Information Criterion (BIC). While the original BIC selects models on the basis of com-plexity and fit, the so-called prior-adapted BIC allows us to choose among statistical models that differ on three scores: fit, complexity, and model size. The prior-adapted BIC can therefore accommodate comparison...

In the present article we illustrate a Bayesian method of evaluating informative hypotheses for regression models. Our main aim is to make this method accessible to psychological researchers without a mathematical or Bayesian background. The use of informative hypotheses is illustrated using two datasets from psychological research. In addition, we...

The effect of an independent variable on a dependent variable is often evaluated with hypothesis testing. Sometimes, multiple studies are available that test the same hypothesis. In such studies, the dependent variable and the main predictors might differ, while they do measure the same theoretical concepts. In this article, we present a Bayesian u...

In many types of statistical modeling, inequality constraints are imposed between the parameters of interest. As we will show in this paper, the DIC (i.e., posterior Deviance Information Criterium as proposed as a Bayesian model selection tool by Spiegelhalter, Best, Carlin, & Van Der Linde, 2002) fails when comparing inequality constrained hypothe...

This article combines the best of both objective and subjective Bayesian inference in specifying priors for inequality and equality constrained analysis of variance models. Objectivity can be found in the use of training data to specify a prior distribution, subjectivity can be found in restrictions on the prior to formulate models. The aim of this...

This paper discusses a Fortran 90 program referred to as BIEMS (Bayesian inequality and equality constrained model selection) that can be used for calculating Bayes factors of multivariate normal linear models with equality and/or inequality constraints between the model parameters versus a model containing no constraints, which is referred to as t...

Researchers in the behavioral and social sciences often have one informative hypothesis with respect to the state of affairs in the population from which they sampled their data. The question they would like an answer to is ‘‘Is the Hypothesis Correct’’ or ‘‘Is it Not.’’ Classical statistics has not yet provided an approach with which this question...

An important application of multiple regression is predictor selection. When there are no missing values in the data, information criteria can be used to select predictors. For example, one could apply the small-sample-size corrected version of the Akaike information criterion (AIC), the (AICC). In this article, we discuss how information criteri...

Most researchers have specific expectations concerning their research questions. These may be derived from theory, empirical evidence, or both. Yet despite these expectations, most investigators still use null hypothesis testing to evaluate their data, that is, when analysing their data they ignore the expectations they have. In the present article...

In veel psychologische artikelen wordt klassieke nulhypothesetoetsing (NHT) gebruikt om onderzoeksvragen te beantwoorden. De resultaten kunnen echter onbevredigend zijn. Rechtstreeks de verwachtingen evalueren zou beter zijn, maar is niet mogelijk met NHT. We laten zien wat de nadelen zijn van NHT en hoe het beter kan, namelijk met Bayesiaanse mode...

Historical studies provide a valuable source of information for the motivation and design of later trials. Bayesian techniques offer possibilities for the quantitative inclusion of prior knowledge within the analysis of current trial data. Combining information from previous studies into an informative prior distribution is, however, a delicate cas...

The Akaike information criterion for model selection presupposes that the parameter space is not subject to order restrictions or inequality constraints. Anraku (1999) proposed a modified version of this criterion, called the order-restricted information criterion, for model selection in the one-way analysis of variance model when the population me...

Résumé Ce travail présente un facteur Bayésien pour la comparaison entre une hypothèse d'intérêt définie par des inégalités et son complément d'une part, l'hypothèse non contrainte d'autre part. Des ensembles équivalents d'hypothèses forment la base d'une quantification de la complexité d'une hypothèse d'intérêt. Nous montrons qu'une distribution a...

This mini-review illustrates that testing the traditional null hypothesis is not always the appropriate strategy. Half in jest, we discuss Aristotle's scientific investigations into the shape of the earth in the context of evaluating the traditional null hypothesis. We conclude that Aristotle was actually interested in evaluating informative hypoth...

In social science research, hypotheses about group means are commonly tested using analysis of variance. While deemed to be formulated as specifically as possible to test social science theory, they are often defined in general terms. In this article we use two studies to explore the current practice concerning group mean hypotheses. The first stud...

Most researchers have specific expectations concerning their research questions. These may be derived from theory, empirical evidence, or both. Yet despite these expectations, most investigators still use null hypothesis testing to evaluate their data, that is, when analysing their data they ignore the expectations they have. In the present article...

Researchers often have expectations about the research outcomes in regard to inequality constraints between, e.g., group means. Consider the example of researchers who investigated the effects of inducing a negative emotional state in aggressive boys. It was expected that highly aggressive boys would, on average, score higher on aggressive response...

In this article, a Bayesian model selection approach is introduced that can select the best of a set of inequality and equality constrained hypotheses for contingency tables. The hypotheses are presented in terms of cell probabilities allowing researchers to test (in)equality constrained hypotheses in a format that is directly related to the data....

Researchers often have expectations that can be expressed in the form of inequality constraints among the parameters of a structural equation model. It is currently not possible to test these so-called informative hypotheses in structural equation modeling software. We offer a solution to this problem using Mplus. The hypotheses are evaluated using...

Market segmentation is the process in marketing of grouping customers into smaller subgroups, according to a certain segmentation basis. Market segmentation is only practically useful if the effectiveness and profitability of marketing activities are influenced substantially by discerning separate homogeneous groups of customers. Using six criteria...

There are different confirmatory techniques to compare means, like hypothesis testing and (Bayesian) model selection. However, there is no software package in which these techniques are available. A Fortran 90 program is written, which enables researchers to apply these techniques to their data. Besides traditional hypotheses, like H-0 : mu(1) = mu...

In objective Bayesian model selection, a well-known problem is that standard non-informative prior distributions cannot be used to obtain a sensible outcome of the Bayes factor because these priors are improper. The use of a small part of the data, i.e., a training sample, to obtain a proper posterior prior distribution has become a popular method...

This article discusses comparisons of means using exploratory and confirmatory approaches. Three methods are discussed: hypothesis testing, model selection based on information criteria, and Bayesian model selection. Throughout the article, an example is used to illustrate and evaluate the two approaches and the three methods. We demonstrate that c...

This paper presents a Bayesian model based clustering approach for dichotomous item responses that deals with issues often encountered in model based clustering like missing data, large data sets and within cluster dependencies. The approach proposed will be illustrated using an example concerning Brand Strategy Research.

When analyzing repeated measurements data, researchers often have expectations about the relations between the measurement means. The expectations can often be formalized using equality and inequality constraints between (i) the measurement means over time, (ii) the measurement means between groups, (iii) the means adjusted for time-invariant covar...

Finding genes that are preferentially expressed in a particular tissue or condition is a problem that cannot be solved by standard statistical testing procedures. A relatively unknown procedure that can be used is the intersection-union test (IUT). However, two disadvantages of the IUT are that it is conservative and it conveys only the information...

The long-term effectiveness of parent training for children with externalizing behaviour problems under routine care within the German health care system is unclear. We report the 1-year follow-up results of the parent training component of the Prevention Program for Externalizing Problem Behaviour (PEP) for 270 children aged 3-10 years with extern...

This paper shows how an European mail order company uses data fusion in order to improve sales. To select the best data fusion algorithm, two traditional data fusion methods, that are, polytomeous logistic regression and nearest neighbor algorithms, are compared with two model based clustering approaches. Finally, it is shown how internal and exter...

Item response theory modeling was applied to the data of 1,321 bereaved individuals who completed the Dutch version of the Inventory of Complicated Grief--Revised (ICG-R)--a 29-item self-report measure of complicated grief (CG). The authors aimed to examine the information that each of the ICG-R items contributes to the measurement of overall CG se...

Hypothesis testing when the null hypothesis belongs to the univariate or multivariate normal linear model is discussed. More specifically it is shown how data can be replicated from the null distribution conditional on the sufficient statistics for the parameters of the null hypothesis at hand. This distribution will be called the similar null dist...

Datafusie, of het combineren van meerdere databestanden, is geen nieuw concept. Echter, dankzij verhoogde interesse in gedifferentieerde direct marketing strategieën, wordt datafusie steeds populairder in marketing. Dit artikel laat zien hoe marketinginformatie kan worden gefuseerd aan een klantendatabase. Gebruikmakend van een marketingtoepassing...

This chapter will provide an introduction to Bayesian data analysis. Using an analysis of covariance model as the point of departure , Bayesian parameter estimation (based on the Gibbs sampler), Bayesian hypothesis testing (using posterior predictive inference), and Bayesian model selection (via the Bayes factor) will be introduced. The chapter wil...

Data fusion, or combining multiple data sets in one data set, is not a new concept. However, due to the increasing desire of differentiated direct marketing strategies, it is getting more popular in marketing. This paper shows how marketing information can be fused to a company’s customer database. Using a real marketing application, two traditiona...

Exposure to premonitory sensations and response prevention of tics (ER) has been shown to be a promising new treatment for Tourette's syndrome (TS). The present study tested the hypothesis that habituation to unpleasant premonitory sensations associated with the tic is an underlying mechanism of change in ER. Patients rated the severity of sensatio...

This chapter deals with inequality constrained latent class analysis. As will be exemplified, researchers often have competing theories that can be translated into inequality constrained latent class models. After this translation it is rather straightforward to evaluate these theories.

Null hypothesis significance testing (NHST) is one of the main research tools in social and behavioral research. It requires the specification of a null hypothesis, an alternative hypothesis, and data in order to test the null hypothesis. The main result of a NHST is a p-value [3].

In Chapters 2, 3, and 4 inequality constrained analysis of variance was introduced and illustrated. This chapter contains an evaluation of inequality constrained analysis of variance. Section 5.2 contains an evaluation from the perspective of psychologists on the use of inequality constrained analysis of variance. The questions raised will be discu...