Rachel Heyard

Recent large-scale replication projects (RPs) have estimated concerningly low reproducibility rates. Further, they all reported substantial degrees of shrinkage of effect size, where the replica tion effect size was found to be, on average, much smaller than the original effect size. Within these RPs, the included original-replication study-pairs c...

To investigate whether the use of positive, self-promotional language in grant proposals is associated with the gender of the applicant, we used a cross-sectional analysis on 8,150 grant proposals submitted to the Swiss National Science Foundation in life sciences disciplines across three different funding schemes, targeting applicants at different...

Background: The standard regulatory approach to assess replication success is the two-trials rule, requiring both the original and the replication study to be significant with effect estimates in the same direction. The sceptical p-value was recently presented as an alternative method for the statistical assessment of the replicability of study res...

Background The quality of COVID-19 preprints should be considered with great care, as their contents can influence public policy. Surprisingly little has been done to calibrate the public’s evaluation of preprints and their contents. The PRECHECK project aimed to generate a tool to teach and guide scientifically literate non-experts to critically e...

In several large-scale replication projects, statistically non-significant results in both the original and the replication study have been interpreted as a ‘replication success.’ Here, we discuss the logical problems with this approach: Non-significance in both studies does not ensure that the studies provide evidence for the absence of an effect...

To investigate whether the use of positive, self-promotional language in grant proposals is associated with the gender of the applicant, we used a cross-sectional analysis on 8,150 grant proposals submitted to the Swiss National Science Foundation in life sciences disciplines across three different funding schemes, targeting applicants at different...

In several large-scale replication projects, statistically non-significant results in both the original and the replication study have been interpreted as a “replication success”. Here we discuss the logical problems with this approach: Non-significance in both studies does not ensure that the studies provide evidence for the absence of an effect a...

Objective To explore how design emulation and population differences relate to variation in results between randomised controlled trials (RCT) and non-randomised real world evidence (RWE) studies, based on the RCT-DUPLICATE initiative (Randomised, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of E...

Recent large-scale replication projects (RPs) have estimated alarmingly low replicability rates. Within these RPs, the original-replication study pairs can vary substantially with respect to aspects of study design, outcome measures, and descriptive features of both the original and replication study population and study team. When broader claims a...

Recent large-scale replication projects (RPs) have estimated concerningly low reproducibility rates. Further, they all reported substantial degrees of shrinkage of effect size, where the replica tion effect size was found to be, on average, much smaller than the original effect size. Within these RPs, the included original-replication study-pairs c...

Background The quality of COVID-19 preprints should be considered with great care, as their contents can influence public policy. Surprisingly little has been done to calibrate the public’s evaluation of preprints and their contents. The PRECHECK project aimed to generate a tool to teach and guide scientifically literate non-experts to critically e...

In several large-scale replication projects, statistically non-significant results in both the original and the replication study have been interpreted as a “replication success”. Here we discuss the logical problems with this approach: Non-significance in both studies does not ensure that the studies provide evidence for the absence of an effect a...

Objectives: While randomized controlled trials (RCTs) are considered a standard for evidence on the efficacy of medical treatments, non-randomized real-world evidence (RWE) studies using data from health insurance claims or electronic health records can provide important complementary evidence. The use of RWE to inform decision-making has been ques...

In several large-scale replication projects, statistically non-significant results in both the original and the replication study have been interpreted as a "replication success". Here we discuss the logical problems with this approach: Non-significance in both studies does not ensure that the studies provide evidence for the absence of an effect a...

Background: The quality of COVID-19 preprints should be considered with great care, as we have seen that their contents can influence public policy. Efforts to improve preprint quality have mostly focused on introducing quick peer review, but surprisingly little has been done to calibrate the public’s evaluation of preprints and their contents. The...

Background : The quality of COVID-19 preprints should be considered with great care, as their contents can influence public policy. Efforts to improve preprint quality have mostly focused on introducing quick peer review, but surprisingly little has been done to calibrate the public’s evaluation of preprints and their contents. The PRECHECK project...

Increasing evidence suggests that the reproducibility and replicability of scientific findings is threatened by researchers employing questionable research practices (QRP) in order to achieve publishable, positive and significant results. Numerous metrics have been developed to determine replication success but it has not yet been established how w...

Funding agencies rely on peer review and expert panels to select the research deserving funding. Peer review has limitations, including bias against risky proposals or interdisciplinary research. The inter-rater reliability between reviewers and panels is low, particularly for proposals near the funding line. Funding agencies are also increasingly...

Sharing data and code as part of a research publication is crucial for ensuring the computational reproducibility of scientific work. But sharing should be done at the article submission stage, not after publication as it is now, say Rachel Heyard and Leonhard Held. Statisticians and data scientists have the skills and tools to make this change and...

The question of whether and to what extent research funding enables researchers to be more productive is a crucial one. In their recent work, Mariethoz et al. (Scientometrics, 2021. 10.1007/s11192-020-03.855-1 ) claim that there is no significant relationship between project-based research funding and bibliometric productivity measures and conclude...

This study investigates the effect of competitive project funding on researchers’ publication outputs. Using detailed information on applicants at the Swiss National Science Foundation and their proposal evaluations, we employ a case-control design that accounts for individual heterogeneity of researchers and selection into treatment (e.g. funding)...

Objectives: To trial a simplified, time and cost-saving method for remote evaluation of fellowship applications and compare this with existing panel review processes by analysing concordance between funding decisions, and the use of a lottery-based decision method for proposals of similar quality. Design: The study involved 134 junior fellowship...

Funding agencies rely on peer review and expert panels to select the research deserving funding. Peer review has limitations, including bias against risky proposals or interdisciplinary research. The inter-rater reliability between reviewers and panels is low, particularly for proposals near the funding line. Funding agencies are increasingly ackno...

Objectives To test a simplified evaluation of fellowship proposals by analyzing the agreement of funding decisions with the official evaluation, and to examine the use of a lottery-based decision for proposals of similar quality. Design The study involved 134 junior fellowship proposals (Postdoc.Mobility). The official method used two panel review...

This study investigates the impact of competitive project-funding on researchers' publication outputs. Using detailed information on applicants at the Swiss National Science Foundation (SNSF) and their proposals' evaluation, we employ a case-control design that accounts for individual heterogeneity of researchers and selection into treatment (e.g....

Objectives To examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency. Setting Swiss National Science Foundation (SNSF). Design Cross-sectional analysis of peer review reports submitted from 2009 to 2016 using linear mixed effects regression mode...

Clinical prediction models play a key role in risk stratification, therapy assignment and many other fields of medical decision making. Before they can enter clinical practice, their usefulness has to be demonstrated using systematic validation. Methods to assess their predictive performance have been proposed for continuous, binary, and time‐to‐ev...

Background The Swiss National Science Foundation (SNSF) supports fundamental and use-inspired research in all disciplines. Peer reviewers assess the proposals submitted to the SNSF. We examined whether the gender of applicants and reviewers and other factors influenced the summary scores awarded. Methods We analysed 38,250 reports on 12,294 grant a...

Purpose: To find ways to reduce the rate of over-triage without drastically increasing the rate of under-triage, we applied a current guideline and identified relevant pre-hospital triage predictors that indicate the need for immediate evaluation and treatment of severely injured patients in the resuscitation area. Methods: Data for adult trauma...

The development of clinical prediction models requires the selection of suitable predictor variables. Techniques to perform objective Bayesian variable selection in the linear model are well developed and have been extended to the generalized linear model setting as well as to the Cox proportional hazards model. Here, we consider discrete time‐to‐e...

There is now a large literature on optimal predictive model selection. Bayesian methodology based on the g-prior has been developed for the linear model where the median probability model (MPM) has certain optimality features. However, it is unclear if these properties also hold in the generalised linear model (GLM) framework, frequently used in cl...