Characteristics Associated with Citation Rate of the Medical Literature

Division of Population Health Sciences, Hospital for Sick Children, Toronto, Ontario, Canada.
PLoS ONE (Impact Factor: 3.23). 02/2007; 2(5):e403. DOI: 10.1371/journal.pone.0000403
Source: PubMed


The citation rate for articles is viewed as a measure of their importance and impact; however, little is known about what features of articles are associated with higher citation rate.
We conducted a cohort study of all original articles, regardless of study methodology, published in the Lancet, JAMA, and New England Journal of Medicine, from October 1, 1999 to March 31, 2000. We identified 328 articles. Two blinded, independent reviewers extracted, in duplicate, nine variables from each article, which were analyzed in both univariable and multivariable linear least-squares regression models for their association with the annual rate of citations received by the article since publication. A two-way interaction between industry funding and an industry-favoring result was tested and found to be significant (p = 0.02). In our adjusted analysis, the presence of industry funding and an industry-favoring result was associated with an increase in annual citation rate of 25.7 (95% confidence interval, 8.5 to 42.8) compared to the absence of both industry funding and industry-favoring results. Higher annual rates of citation were also associated with articles dealing with cardiovascular medicine (13.3 more; 95% confidence interval, 3.9 to 22.3) and oncology (12.6 more; 95% confidence interval, 1.2 to 24.0), articles with group authorship (11.1 more; 95% confidence interval, 2.7 to 19.5), larger sample size and journal of publication.
Large trials, with group authorship, industry-funded, with industry-favoring results, in oncology or cardiology were associated with greater subsequent citations.

Download full-text


Available from: Jason W Busse,
  • Source
    • "Callaham et al. [16] use decision trees to predict citation counts of 204 publications from emergency medicine specialty meeting. They use features like impact factor of journal, research design, number of subjects, rated subjectively for scientific quality, newsworthiness etc. Kulkarni et al. [11] use linear regression and achieve an R 2 of 0.2 for the prediction of citation count for five year ahead window using 328 medical articles. They use features like journal name, month of publication, study design, clinical category of the article etc. Brody et al. [3] use information after the publication to forecast citation count. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The impact and significance of a scientific publication is measured mostly by the number of citations it accumulates over the years. Early prediction of the citation profile of research articles is a significant as well as challenging problem. In this paper, we argue that features gathered from the citation contexts of the research papers can be very relevant for citation prediction. Analyzing a massive dataset of nearly 1.5 million computer science articles and more than 26 million citation contexts, we show that average countX (number of times a paper is cited within the same article) and average citeWords (number of words within the citation context) discriminate between various citation ranges as well as citation categories. We use these features in a stratified learning framework for future citation prediction. Experimental results show that the proposed model significantly outperforms the existing citation prediction models by a margin of 8-10% on an average under various experimental settings. Specifically, the features derived from the citation context help in predicting long-term citation behavior.
    24th ACM Conference on Information and Knowledge Management (CIKM), Melbourne, Australia; 10/2015
  • Source
    • "These limitations may be amplified in lower impact journals as many readers place greater credence and emphasis on articles published in journals with higher impact factor (IF) [10]. However, it is accepted that citation rates may be violated and inflated for a variety of reasons [11] [12], What is new? "
    [Show abstract] [Hide abstract]
    ABSTRACT: To compare the methodological quality of systematic reviews (SRs) published in high- and low-impact factor (IF) Core Clinical Journals. In addition, we aimed to record the implementation of aspects of reporting, including Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram, reasons for study exclusion, and use of recommendations for interventions such as Grading of Recommendations Assessment, Development and Evaluation (GRADE). We searched PubMed for systematic reviews published in Core Clinical Journals between July 1 and December 31, 2012. We evaluated the methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR) tool. Over the 6-month period, 327 interventional systematic reviews were identified with a mean AMSTAR score of 63.3% (standard deviation, 17.1%), when converted to a percentage scale. We identified deficiencies in relation to a number of quality criteria including delineation of excluded studies and assessment of publication bias. We found that SRs published in higher impact journals were undertaken more rigorously with higher percentage AMSTAR scores (per IF unit: β = 0.68%; 95% confidence interval: 0.32, 1.04; P < 0.001), a discrepancy likely to be particularly relevant when differences in IF are large. Methodological quality of SRs appears to be better in higher impact journals. The overall quality of SRs published in many Core Clinical Journals remains suboptimal.
    Journal of clinical epidemiology 04/2014; 67(7). DOI:10.1016/j.jclinepi.2014.01.002 · 3.42 Impact Factor
  • Source
    • "Kulkarni and colleagues examined features of articles associated with higher citation rates in original articles, regardless of study methodology, published in three general medicine journals with high impact factors [22]. They extracted data on nine variables from three hundred twenty-eight articles and analysed them for their association with the annual rate of citations per article five years after publication. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Systematic reviews are important for informing clinical practice and health policy. The aim of this study was to examine the bibliometrics of systematic reviews and to determine the amount of variance in citations predicted by the journal impact factor (JIF) alone and combined with several other characteristics. We conducted a bibliometric analysis of 1,261 systematic reviews published in 2008 and the citations to them in the Scopus database from 2008 to June 2012. Potential predictors of the citation impact of the reviews were examined using descriptive, univariate and multiple regression analysis. The mean number of citations per review over four years was 26.5 (SD +/-29.9) or 6.6 citations per review per year. The mean JIF of the journals in which the reviews were published was 4.3 (SD +/-4.2). We found that 17% of the reviews accounted for 50% of the total citations and 1.6% of the reviews were not cited. The number of authors was correlated with the number of citations (r = 0.215, P < 0.001). Higher numbers of citations were associated with the following characteristics: first author from the United States (36.5 citations), an ICD-10 chapter heading of Neoplasms (31.8 citations), type of intervention classified as Investigation, Diagnostics or Screening (34.7 citations) and having an international collaboration (32.1 citations). The JIF alone explained more than half of the variation in citations (R2 = 0.59) in univariate analysis. Adjusting for both JIF and type of intervention increased the R2 value to 0.81. Fourteen percent of reviews published in the top quartile of JIFs (>=5.16) received citations in the bottom quartile (eight or fewer), whereas 9% of reviews published in the lowest JIF quartile (<=2.06) received citations in the top quartile (34 or more). Six percent of reviews in journals with no JIF were also in the first quartile of citations. The JIF predicted over half of the variation in citations to the systematic reviews. However, the distribution of citations was markedly skewed. Some reviews in journals with low JIFs were well-cited and others in higher JIF journals received relatively few citations; hence the JIF did not accurately represent the number of citations to individual systematic reviews.
    Systematic Reviews 09/2013; 2(1):74. DOI:10.1186/2046-4053-2-74
Show more