Robbie C. M. van Aert’s research while affiliated with Tilburg University and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (85)


Bayesian Evidence Synthesis: Safely and Efficiently Combining Statistical Evidence in Meta-Analyses
  • Preprint

October 2024

·

6 Reads

Joris Mulder

·

Robbie Cornelis Maria van Aert

Bayesian evidence synthesis refers to the process of combining statistical evidence between hypotheses from multiple studies. The evidence is quantified by the Bayes factor. Depending on the underlying model assumptions and research question, different methods can be used for a Bayesian evidence synthesis. The current paper gives an overview of possible models that can be used for this purpose which includes a common effect model, a random effects model, two hybrid effects models, and a fixed effects model. Furthermore, the respective synthesized Bayes factors under each modeling framework are given depending on the implied hypotheses that are tested. We also provide recommendations for prior distributions in Bayesian evidence synthesis for the popular effect size measures, including standardized mean difference, log odds ratio, and Pearson correlation. Additionally, the concept of safe evidence synthesis is described which is particularlyuseful in cumulative/sequential meta-analyses. With this overview and the recommendations for prior specification, researchers can make the best choice to synthesize the evidence from multiple studies. Empirical applications on statistical learning of people with a language impairmentand the incidence of seroma when exercising after breast cancer are used for illustrative purposes. Certain (new) synthesis methods are now also available in the R package BFpack.


Social capital and economic growth: A meta‐analysis

August 2024

·

13 Reads

Journal of Economic Surveys

This research provides a comprehensive, quantitative synthesis of the empirical literature on social capital and economic growth. It assesses 957 estimates from 83 studies. While our preferred estimate of the overall mean effect is close to zero and statistically insignificant, the main finding of our analysis is the substantial degree of effect heterogeneity. We find that social capital effects can range from large negative to large positive, suggesting that its impact on economic growth varies substantially depending on the context. However, our investigation was unable to trace the sources of this heterogeneity to any observable data, estimation, and study characteristics. This suggests that other factors, not included in our study, are responsible. Our analysis did uncover significant publication bias, indicating that the estimates of social capital's effects in the empirical literature are overstated. A further finding from our analysis is that we found no evidence that different types of social capital have different effects on economic growth.


Affective Responses to Acute Exercise: A Meta-Analysis of the Potential Beneficial Effects of a Single Bout of Exercise on General Mood, Anxiety, and Depressive Symptoms

May 2024

·

157 Reads

·

1 Citation

Psychosomatic Medicine

·

Robbie C M van Aert

·

Kiersten Donovan

·

[...]

·

Willem J Kop

Objective Acute exercise elicits various biobehavioral and psychological responses, but results are mixed with regard to the magnitude of exercise-induced affective reactions. This meta-analysis examines the magnitude of general mood state, anxiety, and depressive symptom responses to acute exercise while exploring exercise protocol characteristics and background health behaviors that may play a role in the affective response. Methods A total of 2,770 articles were identified from a MEDLINE/PubMed search and an additional 133 articles from reviews of reference sections. Studies had to have measured general mood before the acute exercise bout and within 30 minutes after exercise completion. Effect sizes were estimated using Hedges’ g , with larger values indicating improvement in the outcome measure. Results A total of 103 studies were included presenting data from 4,671 participants. General mood state improved from pre-exercise to post-exercise ( g = 0.336, 95%CI = 0.234,0.439). Anxiety ( g = 0.497, 95%CI = 0.263,0.730) and depressive symptoms ( g = 0.407, 95%CI = 0.249,0.564) also improved with exercise. There was substantial and statistically significant heterogeneity in each of these meta-analyses. This heterogeneity was not explained by differences in participants’ health status. Meta-regression analyses with potential moderators (intensity of exercise, mode of exercise, usual physical activity level, or weight status of participants) also did not reduce the heterogeneity. Conclusion This meta-analysis shows significantly improved general mood, decreased anxiety, and lower depressive symptoms in response to an acute bout of exercise. There was substantial heterogeneity in the magnitude of the effect sizes, indicating that additional research is needed to identify determinants of a positive affective response to acute exercise.


A Meta-Analytic Review of the Associations of Personality, Intelligence, and Physical Size With Social Status
  • Literature Review
  • Full-text available

February 2024

·

126 Reads

·

5 Citations

Psychological Bulletin

Theories have proposed diverse reasons for why individual differences such as personality traits lead to social status attainment in face-to-face groups. We integrated these different theoretical standpoints into a model with four paths from individual differences to status: a dominance, a competence, a virtue, and a micropolitics path. To investigate these paths, we meta-analyzed over 100 years of research on bivariate associations of personality traits, cognitive abilities, and physical size with the attainment of status-related outcomes in face-to-face groups (1,064 effects from 276 samples including 56,153 participants). The status-related outcome variables were admiring respect, social influence, popularity (i.e., being liked by others), leadership emergence, and a mixture of outcome variables. The meta-analytic correlations we found were largely in line with the micropolitics path, tentatively in line with the competence and virtue paths, and only partly in line with the dominance path. These findings suggest that status attainment depends not only on the competence and virtue of an individual but also on how individuals can enhance their apparent competence or virtue by behaving assertively, by being extraverted, or through self-monitoring. We also investigated how the relations between individual differences and status-related outcomes were moderated by kind of status-related outcome, nature of the group task, culture (collectivism/individualism), and length of acquaintance. The moderation analysis yielded mixed and inconclusive results. The review ends with directions for research, such as the need to separately assess and study the different status-related outcomes.

Download

Multistep estimators of the between‐study covariance matrix under the multivariate random‐effects model for meta‐analysis

December 2023

·

15 Reads

·

1 Citation

Statistics in Medicine

A wide variety of methods are available to estimate the between‐study variance under the univariate random‐effects model for meta‐analysis. Some, but not all, of these estimators have been extended so that they can be used in the multivariate setting. We begin by extending the univariate generalised method of moments, which immediately provides a wider class of multivariate methods than was previously available. However, our main proposal is to use this new type of estimator to derive multivariate multistep estimators of the between‐study covariance matrix. We then use the connection between the univariate multistep and Paule–Mandel estimators to motivate taking the limit, where the number of steps tends toward infinity. We illustrate our methodology using two contrasting examples and investigate its properties in a simulation study. We conclude that the proposed methodology is a fully viable alternative to existing estimation methods, is well suited to sensitivity analyses that explore the use of alternative estimators, and should be used instead of the existing DerSimonian and Laird‐type moments based estimator in application areas where data are expected to be heterogeneous. However, multistep estimators do not seem to outperform the existing estimators when the data are more homogeneous. Advantages of the new multivariate multistep estimator include its semi‐parametric nature and that it is computationally feasible in high dimensions. Our proposed estimation methods are also applicable for multivariate random‐effects meta‐regression, where study‐level covariates are included in the model.


Fig. 1. PRISMA flow diagram. The diagram depicts the information flow through the different phases of our systematic review, including the number of records identified, included and excluded, and the exclusion reasons.
Fig. 2. Number of audit studies across geographic regions and across time. World map visualizing the number of field audits included in our sample across different countries and territories (center) and across year of data collection (bottom left).
Fig. 3. Time trend of gender preferences for non-female-typed jobs. The results are based on a multilevel meta-regression model of male-typed and genderbalanced jobs combined, including gender inequality, study design complexity, and author gender ratio as control variables. Odds ratios above 1 indicate a greater preference for male applicants and odds ratios below 1 indicate greater preference for female applicants. The size of the circles is proportional to the number of applications represented by the respective data point.
Overview of publication bias statistics. The table reports the parameter estimates, 95% confidence intervals, and test statistics of three publication bias metrics: PET- PEESE, three-parameter selection model (3PSM), and p-uniform*. Empty cells with -indicate that this result was not reported by the particular method.
On the trajectory of discrimination: A meta-analysis and forecasting survey capturing 44 years of field experiments on gender and hiring decisions

November 2023

·

1,144 Reads

·

37 Citations

Organizational Behavior and Human Decision Processes

A preregistered meta-analysis, including 244 effect sizes from 85 field audits and 361,645 individual job applications, tested for gender bias in hiring practices in female-stereotypical and gender-balanced as well as male-stereotypical jobs from 1976 to 2020. A “red team” of independent experts was recruited to increase the rigor and robustness of our meta-analytic approach. A forecasting survey further examined whether laypeople (n = 499 nationally representative adults) and scientists (n = 312) could predict the results. Forecasters correctly anticipated reductions in discrimination against female candidates over time. However, both scientists and laypeople overestimated the continuation of bias against female candidates. Instead, selection bias in favor of male over female candidates was eliminated and, if anything, slightly reversed in sign starting in 2009 for mixed-gender and male-stereotypical jobs in our sample. Forecasters further failed to anticipate that discrimination against male candidates for stereotypically female jobs would remain stable across the decades.


Figure 1 A Strategy for Assessing, Understanding, and Improving Replicability in Research on Psychological Interventions
Info Box for Replication Studies in Clinical Psychology
Research Into Evidence-Based Psychological Interventions Needs a Stronger Focus on Replicability

September 2023

·

93 Reads

·

3 Citations

Clinical Psychology in Europe

Background It is a precondition for evidence-based practice that research is replicable in a wide variety of clinical settings. Current standards for identifying evidence-based psychological interventions and making recommendations for clinical practice in clinical guidelines include criteria that are relevant for replicability, but a better understanding as well refined definitions of replicability are needed enabling empirical research on this topic. Recent advances on this issue were made in the wider field of psychology and in other disciplines, which offers the opportunity to define and potentially increase replicability also in research on psychological interventions. Method This article proposes a research strategy for assessing, understanding, and improving replicability in research on psychological interventions. Results/Conclusion First, we establish a replication taxonomy ranging from direct to conceptual replication adapted to the field of research on clinical interventions, propose study characteristics that increase the trustworthiness of results, and define statistical criteria for successful replication with respect to the quantitative outcomes of the original and replication studies. Second, we propose how to establish such standards for future research, i.e., in order to design future replication studies for psychological interventions as well as to apply them when investigating which factors are causing the (non-)replicability of findings in the current literature.


Overview of the stratified sampling and matching procedure.
A Gaussian kernel density plot of included preprints over time per preprint category (COVID-19 versus non-COVID-19 preprint).
A Gaussian kernel density plot of the percentage statistics within a preprint that was inconsistent per preprint category (COVID-19 versus non-COVID-19 preprint).
The top panel shows the number of extracted statistics per type of statistic per preprint category (COVID-19 versus non-COVID-19 preprint). The results based on COVID-19 preprints are indicated by purple bars and those of non-COVID-19 preprints by yellow bars. The bottom panel shows the percentage of extracted statistics that was internally inconsistent per preprint category. Error bars indicate standard errors of the percentages.
Comparing the prevalence of statistical reporting inconsistencies in COVID-19 preprints and matched controls: a registered report

August 2023

·

79 Reads

·

4 Citations

The COVID-19 outbreak has led to an exponential increase of publications and preprints about the virus, its causes, consequences, and possible cures. COVID-19 research has been conducted under high time pressure and has been subject to financial and societal interests. Doing research under such pressure may influence the scrutiny with which researchers perform and write up their studies. Either researchers become more diligent, because of the high-stakes nature of the research, or the time pressure may lead to cutting corners and lower quality output. In this study, we conducted a natural experiment to compare the prevalence of incorrectly reported statistics in a stratified random sample of COVID-19 preprints and a matched sample of non-COVID-19 preprints. Our results show that the overall prevalence of incorrectly reported statistics is 9–10%, but frequentist as well as Bayesian hypothesis tests show no difference in the number of statistical inconsistencies between COVID-19 and non-COVID-19 preprints. In conclusion, the literature suggests that COVID-19 research may on average have more methodological problems than non-COVID-19 research, but our results show that there is no difference in the statistical reporting quality.


Fig. 1 Overestimation of true effect size (Pearson correlation coefficient) caused by outcome reporting bias for varying values of the correlations (r ) among the outcomes and the number of outcomes. These results were based on a sample size of 40 and true effect size equal to ρ = 0. R code of this figure is available at https://osf.io/umnaq/
Fig. 2 Effect size estimation of the random-effects model (RE) and random-effects meta-regression model with as moderator the variance (Var.) and standard deviation (St. dev.) of the observed outcomes' effect sizes, an estimate of the population variance (Pop. var.) and standard deviation (Pop. st. dev.), variance of the difference scores (Var. dif.) and standard deviation of the difference scores (St. dev. dif.), and the
Fig. 3 Effect size estimation of the random-effects model (RE) and random-effects meta-regression model with as moderator the variance (Var.) and standard deviation (St. dev.) of the observed outcomes' effect sizes, an estimate of the population variance (Pop. var.) and standard deviation (Pop. st. dev.), variance of the difference scores (Var. dif.) and standard deviation of the difference scores (St. dev. dif.), and the
Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach

July 2023

·

163 Reads

·

4 Citations

Behavior Research Methods

Outcome reporting bias (ORB) refers to the biasing effect caused by researchers selectively reporting outcomes within a study based on their statistical significance. ORB leads to inflated effect size estimates in meta-analysis if only the outcome with the largest effect size is reported due to ORB. We propose a new method (CORB) to correct for ORB that includes an estimate of the variability of the outcomes' effect size as a moderator in a meta-regression model. An estimate of the variability of the outcomes' effect size can be computed by assuming a correlation among the outcomes. Results of a Monte-Carlo simulation study showed that the effect size in meta-analyses may be severely overestimated without correcting for ORB. Estimates of CORB are close to the true effect size when overestimation caused by ORB is the largest. Applying the method to a meta-analysis on the effect of playing violent video games on aggression showed that the effect size estimate decreased when correcting for ORB. We recommend to routinely apply methods to correct for ORB in any meta-analysis. We provide annotated R code and functions to help researchers apply the CORB method.


Meta‐analyzing partial correlation coefficients using Fisher's z transformation

July 2023

·

27 Reads

·

20 Citations

Research Synthesis Methods

The partial correlation coefficient (PCC) is used to quantify the linear relationship between two variables while taking into account/controlling for other variables. Researchers frequently synthesize PCCs in a meta-analysis, but two of the assumptions of the common equal-effect and random-effects meta-analysis model are by definition violated. First, the sampling variance of the PCC cannot assumed to be known, because the sampling variance is a function of the PCC. Second, the sampling distribution of each primary study's PCC is not normal since PCCs are bounded between -1 and 1. I advocate applying the Fisher's z transformation analogous to applying Fisher's z transformation for Pearson correlation coefficients, because the Fisher's z transformed PCC is independent of the sampling variance and its sampling distribution more closely follows a normal distribution. Reproducing a simulation study by Stanley and Doucouliagos and adding meta-analyses based on Fisher's z transformed PCCs shows that the meta-analysis based on Fisher's z transformed PCCs had lower bias and root mean square error than meta-analyzing PCCs. Hence, meta-analyzing Fisher's z transformed PCCs is a viable alternative to meta-analyzing PCCs, and I recommend to accompany any meta-analysis based on PCCs with one using Fisher's z transformed PCCs to assess the robustness of the results.


Citations (40)


... Formulating hypotheses regarding the most effective profiles for attaining status is challenging due to potential complex interactions among these routes. While virtue, competence, and dominance positively predict status in variable-centered analyses (Bai et al., 2020;Grosz et al., 2024), a profile combining high levels of these routes might not be the only, or the most, effective one for status attainment. For instance, if high virtue ceases to enhance status in the presence of high competence (indicating a negative interaction), then moderate or low virtue might suffice as part of an effective profile. ...

Reference:

How Virtue, Competence, and Dominance Conjointly Shape Status Attainment at Work: Integrating Person-Centered and Variable-Centered Approaches
A Meta-Analytic Review of the Associations of Personality, Intelligence, and Physical Size With Social Status

Psychological Bulletin

... Gender differences also exist in work-life balance (Rashmi & Kataria, 2022). One reason for gender inequality may be the uneven division of family and community roles between men and women (Ren & Caudle, 2020;Schaerer et al., 2023;Thun, 2020). Many countries are still structured around gender roles and social norms in regard to employment relations, resulting in the disproportionate subordination of women since men are often regarded as breadwinners, while women are typically assumed to take care of the house (see review by Rashmi & Kataria, 2022;Schaerer et al., 2023). ...

On the trajectory of discrimination: A meta-analysis and forecasting survey capturing 44 years of field experiments on gender and hiring decisions

Organizational Behavior and Human Decision Processes

... (Clinical) psychological research faces various major challenges. These include underspecified theories (Burghardt & Bodansky, 2021), low measurement accuracy, the lack of robustness of research results, and duplicate research projects (Niemeyer et al., 2023;Tackett, Brandes & Reardon, 2019;. These challenges are especially relevant because of the resource-intensive nature of data collection in clinical research and the high social relevance of rapid improvements. ...

Research Into Evidence-Based Psychological Interventions Needs a Stronger Focus on Replicability

Clinical Psychology in Europe

... COVID-related articles were more likely to suffer from selection biases in randomized control trials, lack of representativeness in cohort studies, comparability issues in case-control studies, etc. [11,12]). However, other analyses suggest that neither the prevalence of statistical reporting errors nor the level of scrutiny in peer reviews differed systematically between COVID-related and non-COVID articles [13,14]. ...

Comparing the prevalence of statistical reporting inconsistencies in COVID-19 preprints and matched controls: a registered report

... Moreover, many authors may refrain from submitting studies with statistically insignificant and/or inconsistent (i.e., wrong-signed) effects for publication, anticipating rejection by journals (Egger & Smith, 1998). This issue, known as "outcome reporting bias", has garnered considerable attention in recent years (Hutton & Williamson, 2000;Hahn et al., 2002;Chan et al., 2004a,b;Williamson & Gamble, 2005;Dwan et al., 2008;Smyth et al., 2011;Pigott et al., 2013;Copas et al., 2013;Jennions et al., 2013;Stanley et al., 2017;Andrews & Kasy, 2019;Bom & Rachinger, 2019;Marks-Anglin & Chen, 2020;Page et al., 2022;van Aert & Wicherts, 2024). Selective reporting can take several forms, including the selective reporting of certain outcomes from a given study based on the nature and direction of the results, as well as the analytical manipulation of unfavorable results to make them statistically significant or theoretically interesting. ...

Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach

Behavior Research Methods

... Next, the Fisher Z-r test to find the r-confidence interval and the difference between the correlations. The Fisher Z test is useful for testing the difference in correlation between two variables being compared [57,58]. In this study, the Fisher Z-r test was conducted to investigate significant differences in the correlation pattern between emotional reactions and motivation between students with high, medium, and low mathematical comprehension. ...

Meta‐analyzing partial correlation coefficients using Fisher's z transformation
  • Citing Article
  • July 2023

Research Synthesis Methods

... Further, one could argue that the results of our analyses depend upon many parameters chosen by the researcher (see multiverse debate 113,114), which questions the replicability of the findings. We propose that the methodological choices made in our study set a new standard for future research. ...

Meta-Analyzing the Multiverse: A Peek Under the Hood of Selective Reporting

Psychological Methods

... In the above research, the KDE statistical method mainly focused on analyzing the independent impact of individual environmental factors on the siting of the Great Wall, without fully considering the spatial interactions between these factors. Partial correlation analysis can effectively reveal the independent correlation between two specific related variables by excluding the influence of other factors, under the context of a comprehensive analysis of various potential influencing factors [35]. Based on this characteristic, we introduced partial correlation analysis to explore how different terrain factors combine in specific ways to jointly influence the siting decisions of various Great Wall heritage elements. ...

A critical reflection on computing the sampling variance of the partial correlation coefficient
  • Citing Article
  • March 2023

Research Synthesis Methods

... Hence, a wealth of studies addressing the same research question is considered valuable in reaching robust generalizations. Meta-analysis allows for reaching consistent results through the effect size unit (Assen, et al., 2023;Bayraktar, 2021). ...

The Meta-Plot

Zeitschrift für Psychologie

... The estimated prevalence of cognitive impairment (CI) in adult WHO 2007 grades 1-3 glioma patients ranges between 27% and 83%. 6 This wide range can be attributed to diverse study methodologies, including different cognitive assessments and varying definitions of CI. The cognitive domains most frequently affected include executive functioning, psychomotor speed, attention, and memory. ...

Cognitive outcomes after multimodal treatment in adult glioma patients: A meta-analysis

Neuro-Oncology