Citation bias favoring statistically significant studies was present in medical research

CRC & Division of Clinical Epidemiology, Department of Health and Community Medicine, University of Geneva and University Hospitals of Geneva, Rue Gabrielle Perret-Gentil 6, 1211 GENEVE 14, Switzerland. Electronic address: .
Journal of clinical epidemiology (Impact Factor: 5.48). 03/2013; 66(3):296-301. DOI: 10.1016/j.jclinepi.2012.09.015
Source: PubMed

ABSTRACT Statistically significant studies may be cited more than negative studies on the same topic. We aimed to assess here whether such citation bias is present across the medical literature.
We conducted a cohort study of the association between statistical significance and citations. We selected all therapeutic intervention studies included in meta-analyses published between January and March 2010 in the Cochrane database, and retrieved citation counts of all individual studies using ISI Web of Knowledge. The association between the statistical significance of each study and the number of citations it received between 2008 and 2010 was assessed in mixed Poisson models.
We identified 89 research questions addressed in 458 eligible articles. Significant studies were cited twice as often as nonsignificant studies (multiplicative effect of significance: 2.14, 95% confidence interval: 1.38-3.33). This association was partly because of the higher impact factor of journals where significant studies are published (adjusted multiplicative effect of significance: 1.14, 95% confidence interval: 0.87-1.51).
A citation bias favoring significant results occurs in medical research. As a consequence, treatments may seem more effective to the readers of medical literature than they really are.

  • [Show abstract] [Hide abstract]
    ABSTRACT: Background-The preferential citation of studies with the highest success rates could exaggerate perceived effectiveness, particularly for treatments with widely varying published success rates such as radiofrequency catheter ablation for atrial fibrillation. Methods and Results-We systematically identified observational studies and clinical trials of radiofrequency catheter ablation of atrial fibrillation between 1990 and 2012. Generalized Poisson regression was used to estimate association between study success rate and total citation count, adjusting for sample size, journal impact factor, time since publication, study design, and whether first or last author was a consensus-defined pre-eminent expert. We identified 174 articles meeting our inclusion criteria (36 289 subjects). After adjustment only for time since publication, a 10-point increase above the mean in pooled reported success rates was associated with a 17.8% increase in citation count at 5 years postpublication (95% confidence interval, 7.1-28.4%; P<0.001). After additional adjustment for impact factor, sample size, randomized trial design, and pre-eminent expert authorship, the association remained significant (18.6% increase in citation count; 95% confidence interval, 7.6-29.6%; P<0.0001). In this full model, time since publication, impact factor, and pre-eminent expert authorship were significant covariates, whereas randomized control trial design and study sample size were not. Conclusions-Among studies of radiofrequency catheter ablation of atrial fibrillation, high success rate was independently associated with citation count, which may indicate citation bias. To readers of the literature, radiofrequency catheter ablation of atrial fibrillation could be perceived to be more effective than the data supports. These findings may have implications for a wide variety of novel cardiovascular therapies.
    Circulation Cardiovascular Quality and Outcomes 09/2014; 7(5). DOI:10.1161/CIRCOUTCOMES.114.000912 · 5.66 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: To examine whether the number of early online accesses to medical research articles predicts citations in the scientific literature over time. Cohort study of research articles published in the BMJ between January and June 1999. The number of online assesses within 1 week of publication was examined in relation to citation counts in 1999-2004, 2004-2009, and 2009-2014. The 148 included articles were accessed on average 691 times up publication, and each was cited on average 33 times in 1999-2004, 32 times in 2004-2009, and 26 times in 2009-2014. The logarithm of accesses predicted the logarithm of citations for all three subsequent periods, but the association weakened over time (correlation with citations in 1999-2004: 0.54, 2004-2009: 0.49, 2009-2014: 0.39; all P < 0.001). In addition to online accesses, the presence of an abstract also predicted more citations for all periods. Early interest in a medical research article, reflected by online accesses within a week of publication, predicts citations up to 15 years later. This strengthens the validity of online usage as a measure of the scientific merit of publications. Copyright © 2015 Elsevier Inc. All rights reserved.
  • 07/2014; 3(4):317-320. DOI:10.2217/cer.14.31


Available from
May 22, 2014