Steven Kpaka, Nathalie Krou-Danho, Safiatou Lingani, Aboua
Lucien, Bondo Monga, Delphine Chia N’guessan, Déborah
N’Guessan-Yao, Jean Louis N’Jampo, Patricia Patindé, Albert
Seri Sekou, Odette Tossou, Leo Weakland, Christiane Wondji
Gozo.At the US Centers for Disease Control and Prevention,we
thank Eliane Dogoré for help with coding; David Fluker,
Eleanor McLellan, and Robert Strotman for software support;
and Thomas Peterman, Marc Bulterys, Richard Jenkins,
RJ Simonds, and Monica Nolan for commenting on earlier ver-
sions of the manuscript. We also thank two BMJ reviewers for
their useful comments and suggestions. This study was
presented in part at the XIIth International Conference on
AIDS and STDs in Africa, Ouagadougou, Burkina Faso, Decem-
ber 9-13, 2001 [abstract 10BT2-6].
Contributors: See bmj.com.
Funding: The research project was funded by the Centers for
Disease Control and Prevention (CDC). All of the authors were
employed directly or indirectly by the CDC during the design,
collection and analysis of data, and write-up of findings.
Competing interests: None declared.
Ethical approval: Ethics committee of Côte d’Ivoire’s Ministry of
Public Health and the Institutional Review Board of the US
Centers for Disease Control and Prevention.
1 Cartoux M, Meda N, Van de Perre P, Newell ML, de Vincenzi I, Dabis F.
Acceptability of voluntary HIV testing by pregnant women in developing
countries: an international survey. AIDS 1998;12:2489-93.
Kiarie J, Nduati R, Koigi K, Musia J, John G. HIV-1 testing in pregnancy:
acceptability and correlates of return for test results. AIDS 2000;14:1468-
Ladner J, Leroy V, Msellati P, Nyirazliraje M, De Clercq A, Van de Perre P,
et al. A cohort study of factors associated with failure to return for HIV
post-test counseling in pregnant women: Kigali, Rwanda, 1992-1993.
Ekouevi DK, Rouet F, Becquet R, Inwoley A, Viho I, Tonwe-Gold B, et al.
Immune status and uptake of antiretroviral interventions to prevent
mother-to-child transmission of HIV-1 in Africa. J Acquir Immune Defic
Malonza IM, Richardson BA, Kreiss JK, Bwayo JJ, Stewart GC. The effect
of rapid HIV-1 testing on uptake of perinatal HIV-1 interventions: a ran-
domized clinical trial. AIDS 2003;17:113-8.
Wiktor SZ, Ekpini E, Karon JM, Nkengasong J, Maurice C, Severin ST, et
al. Short-course oral zidovudine for prevention of mother-to-child trans-
mission of HIV-1 in Abidjan, Côte d’Ivoire: a randomized trial. Lancet
Ezoua J, Sassan-Morokro M, Ekra A, Sidibé K, Maurice C, Nolan M, et al.
Trends in HIV prevalence among pregnant women attending urban
antenatal clinics in Côte d’Ivoire, 1997-2000. Abstract presented at the
XIIth International Conference on AIDS and STD in Africa. Ouagadougou,
Burkina Faso, 9-13 December 2001. [Abstract 12PT5-416.]
Nebié Y, Meda N, Leroy V, Mandelbrot L, Yaro S, Sombié I, et al. Sexual
and reproductive life of women informed of their HIV seropositivity: a
prospective cohort study in Burkina Faso. J Acquir Immune Defic Syndr
Mckenna Sl, Muyinda GK, Roth D, Mwali M, Ng’andu N, Myrick A, et al.
Rapid HIV testing and counseling for voluntary testing centers in Africa.
10 PainterTM.Voluntary counseling
high-leverage intervention for HIV/AIDS prevention in sub-Saharan
Africa. Soc Sci Med 2001;53:1397-411.
11 Allen SA, Karita E, N’Gandu N, Tichacek A. The evolution of voluntary
testing and counseling as an HIV prevention strategy. In: Gibney L,
DiClemente RJ, Vermund SH,eds. Preventing HIV in developing countries:
12 Farquhar C, Mbori Ngacha D, Bosire R, Nduati R, Kreiss J, John G. Preva-
lence and correlates of partner notification regarding HIV-1 in an ante-
natal setting in Nairobi, Kenya. Abstract presented at the XIII
International AIDS Conference, Durban, South Africa, 9-15 July 2000.
(Accepted 6 July 2004)
NewYork: Plenum Press,
Relation between online “hit counts” and subsequent
citations: prospective study of research papers in the BMJ
Thomas V Perneger
Evaluation of published medical research remains a
challenge. Two classic yardsticks are the citation count
(the number of times a given paper is cited by
others)1 2and the impact factor of the journal that
published the paper (which reflects the average
number of citations per article).2 3However, the
citation count can be assessed only several years after
publication, and the impact factor is not paper specific
and is thus virtually meaningless in assessing any
obtained rapidly and is paper specific, is the “hit
count” (the number of times a paper is accessed
online). Whether this count predicts citations is
unknown. I examined this issue prospectively in a
cohort of papers published in the BMJ.
measure, whichcan be
Methods and results
The study used articles published in volume 318 of the
BMJ (1999) in sections titled Papers, General Practice,
and Information in Practice. The hit counts (full text
articles, HTML version) for the main body of each arti-
cle within a week of publication were provided by a
BMJ staff member because the “hit parade” posted on
the journal website was found to be unreliable for
1999. I obtained the number of citations on 24 May
2004 from the ISI Web of Science, an internet service
to which the local medical library has a subscription.1
I also recorded for each paper the study design and the
number of pages.
Nine papers were excluded because they did not
report research (but reported discussions of, for exam-
ple, NHS management and statistics methods). The
remaining 153 papers comprised 29 randomised trials,
Logarithm of internet hits
Logarithm of citations + 1
Relation between citations and internet hits for 153 papers in volume
318 of the BMJ (1999)
Institute of Social
Thomas V Perneger
professor of health
BMJ VOLUME 3294 SEPTEMBER 2004 bmj.com
11 systematic reviews, 41 prospective studies, 8
case-control studies, 41 cross sectional surveys, 6 quali-
tative studies, and 17 other designs (such as economic
analyses or case reports).
The average hit count for the papers in the first
week after publication was 685 (SD 410;25th,50th,and
75th centiles 437, 578, and 795 respectively; range 175
to 3181); the average number of citations in the five
years after publication was 32.5 (SD 37.5; 25th, 50th,
and 75th centiles 9.5, 22, and 42.5 respectively; range 0
to 291). Only one paper was never cited. The hit count
was associated with the number of subsequent citations
(Pearson correlation coefficient: 0.50, P < 0.001). The
result was similar for logarithms of the counts (r = 0.54,
P < 0.001) (figure). For every 100 additional hits, 4.4
additional citations (95% confidence interval 3.1 to 5.7)
accrued over the five years.
The average hit count for randomised trials or
systematic reviews was 832, for prospective or case-
control studies was 747, and for cross sectional,
qualitative, and other studies was 545 hits (P=0.001).
Longer papers attracted more hits than short papers (an
extra 54.4 hits per page, P=0.004), but this association
became non-significant after adjustment for study design.
Citations were predicted by paper length (an extra
9.3 citations per page, P < 0.001) and study design
(randomised trials and systematic reviews yielded 46.0
citations, prospective and case-control studies 38.9
citations, and other designs 19.3 citations (P = 0.001).
When the hit count was included as predictor,however,
the effect of study design became non-significant; only
page length (an extra 7.3 citations per page, P < 0.001)
and the hit counts (an extra 3.7 citations per 100 hits,
P < 0.001) remained as independent predictors. These
variables explained 33% of variance in citation counts.
Papers that attracted the most hits on the BMJ website
in the first week after publication were subsequently
cited more often than less frequently accessed papers.
Thus early hit counts capture at least to some extent
the qualities that eventually lead to citation in the
My hypothesis is that “scientific value” explains the
association between hits and citations. Online readers
judge the scientific value of an article from the title
and the abstract, and if this assessment is favourable,
they access the full paper. The paper’s scientific value
also leads to citation by other researchers.4This
hypothesis is supported by the greater frequency of
both hits and citations for papers that used the
most scientifically rigorous study designs, such as
The number of early hits is a potentially useful
measure of the scientific value of published medical
research papers. Publication of hit counts by online
journals should be encouraged.
Daniel Berhane from the BMJ provided valid hit counts for the
Contributor: TVP is the sole contributor.
Competing interests: TVP is the editor of the International Jour-
nal for Quality in Health Care.
Ethical approval: Not required.
Adam D. The counting house. Nature 2002;415:726-9.
Walter G, Bloch S, Hunt G, Fisher K. Counting on citations: a flawed way
to measure quality. Med J Aust 2003;178:280-1.
Seglen PO. Why the impact factor of journals should not be used for
evaluating research. BMJ 1997;314:498-502.
Lee KP, Schotland M, Bacchetti P, Bero LA. Association of journal quality
indicators with methodological quality of clinical research articles. JAMA
(Accepted 10 July 2004)
What is already known on this topic
The value of a research study is traditionally
assessed through citation counts or by the impact
factor of the journal that published the study
Citation counts can be obtained only years after
publication, and the impact factor is not paper
What this study adds
For a cohort of papers published in the BMJ in
1999, the hit count on the website in the week
after online publication predicted the number of
citations in subsequent years; the hit count is a
potentially useful measure of the scientific value of
a research paper
Corrections and clarifications
Dr Foster’s case notes:How often are adverse events
reported in English hospital statistics?
This article by Paul Aylin and colleagues contains
errors that escaped the notice of both the BMJ and
the authors during the editorial process. The first
sentence of the opening box should read: “It has
been suggested that an estimated 850 000 medical
errors occur in NHS hospitals every year resulting
in 40 000 deaths.” In the “basic figures” section, the
first sentence should read: “On average 2.2% of all
episodes (about 275 000 [not 27 500] per year)
included a code for an adverse event.” And we
initially posted an incomplete version of table A on
bmj.com. It has now been updated to provide full
ICD-10 codes. We apologise for our lapses.
Operative vaginal delivery and neonatal and infant
adverse outcomes:population based retrospective analysis
We have been alerted to some errors in this paper
by Kitaw Demissie and colleagues (3 July, pp 24-6).
In the Participants section of the abstract and the
opening sentence of the Results section, the
published numbers for singleton live births were
wrong. The correct figures are 11 463 823 (instead
of 11 639 388) for the United States and 374 873
(instead of 556 597) for New Jersey. The authors
state that these revisions do not affect any of the
results in the table or the conclusion of the paper.
Some referencing errors crept into this Clinical
Review by Carol Chen-Scarabelli and Tiziano M
Scarabelli (7 August, pp 336-41). In the table
summarising clinical trials, Takata et al should be
reference w3 (not w2). In the “extra: further
information” on tilt testing protocols on bmj.com,
all references to w2 should in fact be to w6.
BMJ VOLUME 3294 SEPTEMBER 2004 bmj.com