ArticlePDF AvailableLiterature Review

The answer is 17 years, what is the question: Understanding time lags in translational research

Authors:

Abstract and Figures

This study aimed to review the literature describing and quantifying time lags in the health research translation process. Papers were included in the review if they quantified time lags in the development of health interventions. The study identified 23 papers. Few were comparable as different studies use different measures, of different things, at different time points. We concluded that the current state of knowledge of time lags is of limited use to those responsible for R&D and knowledge transfer who face difficulties in knowing what they should or can do to reduce time lags. This effectively 'blindfolds' investment decisions and risks wasting effort. The study concludes that understanding lags first requires agreeing models, definitions and measures, which can be applied in practice. A second task would be to develop a process by which to gather these data.
Content may be subject to copyright.
510
The answer is 17 years, what is the
question: understanding time lags
in translational research
Zoë Slote Morris
1
Steven Wooding
2
Jonathan Grant
2
1
Institute of Public Health, University of Cambridge, Cambridge CB2 0SR, UK
2
RAND Europe, Cambridge CB4 1YG, UK
Correspondenc e to: Jonathan Grant. Email: jgrant@rand.org
Summary
This study aimed to review the literature describing and quantifying time
lags in the health research translation process. Papers were included in the
review if they quantified time lags in the development of health
interventions. The study identified 23 papers. Few were comparable as
different studies use different measures, of different things, at different
time points. We concluded that the current state of knowledge of time lags
is of limited use to those responsible for R&D and knowledge transfer who
face difficulties in knowing what they should or can do to reduce time lags.
This effectively blindfolds investment decisions and risks wasting effort.
The study concludes that understanding lags first requires agreeing
models, definitions and m easures, which can be applied in practice. A
second task would be to develop a process by which to gather these data.
Introduction
Timely realization of the benefits of expensive
medical research is an international concern
attracting considerable policy effort around ‘trans-
lation’.
1,2
Policy interventions to improve trans-
lation respond to a vast empirical literature on
the difficulties of getting research across research
phases and into practice.
3–11
Both literature and policy tend to assume that
speedy translation of research into practice is a
good thing. Delays are seen as a waste of scarce
resources and a sacrifice of potential patient
benefit.
12
Although some lag will be necessary to
ensure the safety and efficacy of new interventions
or advances, in essence we should aim to optimize
lags. One recent study (of which JG and SW were
co-authors) estimating the economic benefit of car-
diovascular disease (CVD) research in the UK
between 1975 and 2005, found an internal rate of
return (IRR) of CVD research of 39%.
13
In other
words, a £1.00 investment in public/charitable
CVD research produced a stream of benefits
equivalent to earning £0.39 per year in perpetuity.
Of this, 9% was attrib utable to the benefit from
health improvements, which is the focus of this
paper. (The remaining 30% arise from ‘spillovers’
benefiting the wider economy.) This level of
benefit was calculated using an estimated lag of
17 years. Varying the lag time from 10 to 25
years produ ced rates of return of 13% and 6%,
respectively, illustrating that shortening the lag
between bench and bedside improves the
overall benefit of cardiovascular research. What
is notable is that all the above calculations
depended upon an estimated time lag; estimated
because, despite longstanding concerns about
them,
14
time lags in health research are little
understood.
It is frequently stated that it takes an average of
17 years for research evidence to reach clinical
practice.
1,3,15
Balas and Bohen,
16
Grant
17
and
Wratschko
18
all estimated a time lag of 17 years
measuring different points of the process. Such
convergence around an ‘average’ time lag of 17
years hides complexities that are relevant to
DECLARATIONS
Competing interests
None declared
Funding
This is an
independent paper
funded by the Policy
Research
Programme in the
Department of
Health. The views
expressed are not
necessarily those of
the Department
Ethical approval
Not applicable
Guarantor
JG
Contributorship
ZSM designed,
conducted and
analysed the
literature review,
and drafted and
revised the paper;
JG initiated the
project, drafted and
revised the paper,
and has led a
number of studies
cited that attempted
to measure lags; SW
revised the paper
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
REVIEW
policy and practice which would benefit from
greater understanding.
13
Despite longstanding concerns about delays in
getting research into practice, the literature on
time lags seems surprisingly under-developed. To
help address this gap, this paper aims to synthesize
existing knowledge and to offer a conceptual model
that can be used to standardize measurement and
thus help to quantify lags in future. This would
allow efforts to reduce lags to be focused on areas
of particular concern or value, or on areas where
interventions might be expected to have best
effect. It would also provide the potential for eval-
uating the cost-effectiveness of translation interven-
tions if their impact on lags can be measured. The
aim was to overlay empirical lag data onto the con-
ceptual model of translational research to provide
an overview of estimated time lags and where
they occur. The first part of the paper explores con-
ceptual models of the translation pipeline in order
to provide context. The second part of the paper
presents a review of the literature on time lags to
present current estimates and issues. This leads to
a discussion on the current state of understanding
about time lags and considers the implications for
future practice and policy.
Methods
For the first part of the study we identified litera-
ture that described conceptual models of trans-
lation. Our search was not intended to be
exhaustive, but included key policy documents
and searches of Google Scholar, Web of Science,
PubMed and EBSCO. Key words used to retrieve
relevant studies included ‘valley of death’,
‘bench to bedside’, ‘translational res earch’ and
‘commercialisation’. In general, ‘grey’ literature
was not included in the search, but the HERG
study
19
was included because of the authors’
involvement in it. The models in the literature
found by these methods were summarized into a
simple conceptual model.
For the second part of the study we r eviewed the
literature on time lags in health research. We used
the same methods and literature as for the first
part but included additional search terms such as
‘time lag’ or ‘time-lag’, ‘delays’, ‘time factors’
(PubMed MESH term) and ‘publication bias’. We
found a formal search yielded few relevant papers
so combined a number of approaches to increase
our confidence that relevant papers had been ident-
ified. We undertook bac kw ard and forwar d citation
tracking to identify related work and used searches
within targeted journals e.g. Scientometrics and
Journal of Translational Medicine.Toanalysethelag
data, w e used a data e xtraction templatewith the fol-
lowing fields: start and end dates for measurement
period, range, mean, median, dates used, topic,
country ofstudy. In addition, the start point and end-
point of the time lag measured in each study were
mapped onto specific stages in the conceptual
model developed in the first part of the st udy.
Findings
Conceptualising translational research
Understanding time lags requires a conceptual
model of how research in science is converted to
patient benefit so that the durations of activities
and waits can be measured. This process of con-
version of basic science to patient benefit is often
called ‘translation’.
1,2,18,20 22
Woolf has argued
that ‘translation research means different things
to different people’
23
and this is reflected in the
various models and definitions found in the litera-
ture. However, as translational research also
‘seems important to almost everyone’
23
there
would seem to be benefit in trying to unify
models and definitions.
We have attempted to synthesize these models
to identify key features of the translation process
and to offer a tentative unified model. This was
intended to help stakeholders agree a model
which could be used to support future data gath-
ering and better guide policy-making. We recog-
nized that drug development, public health,
devices and broader aspects of healthcare practice
will vary in nature. The translation process is sum-
marized briefly in Figure 1. Clearly this model can
be critiqued for being linear and we acknowledge
the considerable literature that challenges this
notion and accept that research translation is a
messy, iterative and complex process (see Balaconi
et al. for a good review of the liner models cri-
tiques and their partial rebuttal
24
). At the same
time, we would argue that for the purposes of
understanding and conceptualising time lags the
model is appropriate in showing common steps
found in the literature.
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Understanding time lags in translational research and why it matters
511
Acknowledgements
This paper derives
from work
undertaken by RAND
Europe within Centre
for Policy Research
in Science and
Medicine (PRISM),
funded by the UK
Department of
Health. The authors
thank NIHR for their
support
‘Translational research’ is typically separated
into two phases of research. Type 1 translation,
also somewhat confusingly called ‘bench to
bedside’, refers to the conversion of knowledge
from basic science research into a potential clinical
product for testing on human subjects. Type 2
translation, ‘research into practice’, tends to refer
to the process of converting promising inter-
ventions in clinical research into healthcare prac-
tice (thus is closer to the notion of the
‘bedside’).
2,20,21,25,26
Each phase of translational
research is associated with a set of research activi-
ties which contribute to lags.
27
These include pro-
cesses around grant awards, ethical approvals,
publication, phase I, II, III trials, approvals for
drugs, post-marketing testing, guideline prep-
aration and so forth. Some of these activities are
repeated in different phases grants and publi-
cations most particularly. Each activity involves a
lag, either because the effort required for carrying
out the task or as a result of non-value adding
waits. The activities are used as ‘markers’ in
studies of lags.
Conceptual models typically include ‘transla-
tional gaps’, which describe the movement from
one phase of research to another. Each of these is
also associated with delays, although precisely
what and where these gaps are, and how long
they are, is again not consistent in the literature.
Policy measures to expedite the translation
process typically focus on these gaps.
Estimating time lags in the translation
process
Table 1 shows a summary of estimates derived
from empirical studies of lags.
Figure 2 shows these time-lag estimates by
research phase. Some additional ‘averaging’ has
been necessary to provide single figures where
ranges only were used in the original paper. The
source data are presented in A ppendix A (see
http://jrsm.rsmjournals.com/content/104/12/510/
suppl/DC1).
Issues with measurement and estimation
As is shown in Table 1, studies of time lags in trans-
lation of research to practice often measure different
points in the process. For example, Decullier et al.
28
and Stern and Simes
29
measure time between
ethical approval and date of first publication;
Grant et al.
30
and HERG et al.
13
look at publication
to guideline; DiMasi calculated the length of time
within and between phases in US drug develop-
ment to calculate the costs associated with the
phases.
31
Sternitzke looked at commercialization of
pharmaceutical innovations from ‘chemical syn-
thesis’ to FDA approval.
32
Ioannidis attempted to
estimate the time lag between date of trial regis-
tration and several milestones to publication.
33
Grant et al., Mansfield and Comroe and Dripps
work backwards from practice to publication.
17,34 36
Not surprisingly given they are measuring
different lags, Figure 2 helps show that data are
generally sparse and estimates vary.
37
Some
studies report longer lags for publication to guide-
line
17,38
than others do for development
to commercialization.
18,32,35
Table 1 also points to
two substantive gaps in knowledge: the time lag
involved in and between discovery and develop-
ment (T1), and the time lag between publication
to practice. Only one study has ‘implementation’
into practice as its endpoint.
Measurement and reporting is often poor. For
example, Decullier et al. report ‘mean’ lags,
28
and
Dwan, in reporting Decullier et al., in their
review refer to ‘median’ lags.
36
Neither reports
distributions. Ranges or even interquartile
ranges as large as 221 years
38
are seldom
reported. Furthermore, where it was possible,
further investigation of the average revealed
wide variation; variation which is not highlighted
or discussed in the papers. For example, Hopewell
et al. in their review of publication bias conclude
that clinical trials with null or negative results
‘on average’ took ‘just over a year longer to be pub-
lished than those with positive results’.
39
This
average is associated with a range of six to eight
years for studies with a negative or null result,
Figure 1
A conceptual model of the journey of health
(biomedical) research from research into benefit,
as derived from the literature
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Journal of the Royal Society of Medicine
512
Table 1
Summary of studies of time lags in health research
Author Context Start of
time lag
End of time lag Time lag (years)
Dates Country Notes
Lower
range
Median Mean Higher
Range
Antman
(1992)
38
Treatment for
myocardial
infarction
Publication of
clinical trial
Guideline/
recommendation
6 13 19661992 US
Altman (1994)
46
Statistical techniques First
publication
Highly cited 4 6
Balas and
Bohen
(2000)
16
Various Original
research
Implementation 17 19681997 International Calculated from
adding a
number of
studies
together
Cockburn and
Henderson
(1996)
53
Drugs Date of
enabling
scientific
research
Date to market 11 28 67 Narrative
histories of
drug
discoveries,
19701995
US
Comanor and
Scherer
(1969)
55
Drugs Patent New entities 3 3 US
Comroe and
Dripps
(1976)
36
Top ten clinical
advances in
cardiovascular and
pulmonary
medicine and
surgery ECG
Publication Clinical advances 306 Key advances
since 1945
US
Contopoulos-
loannidis
(2008)
35
Publication
(First
description)
First specific use 0 221 High citations in
19902004
International Worked
backwards
from highly
cited (over
1000 citations
on WoS) to
the first
description;
interquartile
range
Contopoulos-
loannidis
(2008)
35
Publication
(First
description)
Highly cited
publication
14 24 24 44 High citations in
19902004
International Worked
backwards
from highly
cited (over
1000 citations
on WoS) to
the first
description;
interquartile
range
(Continued )
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Understanding time lags in translational research and why it matters
513
Table 1
Continued
Author Context Start of
time lag
End of time lag Time lag (years)
Dates Country Notes
Lower
range
Median Mean Higher
Range
Contopoulos-
loannidis
(2008)
35
Publication
(First
description)
First human use 0 28 High citations in
19902004
International Worked
backwards
from highly
cited (over
1000 citations
on WoS) to
the first
description;
interquartile
range
Decullier et al.
(2005)
26
Various Ethics
approval
Date for first
publication
Ethical approval
given in 1994;
study
conducted in
2000
France Does not report
for all papers,
but only by
direction of
results; does
not report
ranges
DiMasi (1991)
56
Not mentioned Clinical
testing
Submission to FDA 6.3 US drugs
DiMasi (1991)
56
Not mentioned Clinical
testing
Marketing approval 8.2 US drugs
DiMasi (2003)
29
R&D expenditure
from 1980 1999
Clinical
testing
Submission to FDA 6 19801999 US drugs
DiMasi (2003)
29
R&D expenditure
from 1980 1999
Clinical
testing
Marketing approval 7.5 19801999 US drugs
Grant et al.
(2000)
28
Various Publication Guideline 0 8 49 19881995 UK guideline Range
estimated
from Figure 1
Grant et al.
(2003)
17
Neonatal care Publication Most recent paper 13 17 21 19951999 UK Estimated from
graph
Harris et al.
(2010)
40
Cancer drugs Abstract Publication 0.4 0.75 1.6 20052007 UK Results changed
for abstract to
full
publications
in 3 out of 3
cases
HERG et al.
(2008)
13
CVD Publication Guideline 9 13
14 19752005 UK guideline Range varied by
topic; assume
a three year
lag in
publication;
and used the
same study
period
HERG et al.
(2008)
13
Mental health Publication Guideline 6 9 11 19752005 UK guideline Range varied by
topic; assume
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Journal of the Royal Society of Medicine
514
a three year
lag in
publication;
and used the
same study
period
Ioannidis
(1998)
31
AIDS Date of trial
registration
Publication 3.9 5.5 7 Studies
conducted
between 1986
and 1996
US Uses
interquartile
range
Ioannidis
(1998)
31
AIDS Date of trial
registration
Date of completion of
study
2 2.6 3.8 Studies
conducted
between 1986
and 1996
US Uses
interquartile
range
Ioannidis
(1998)
31
AIDS Completion of
study
First submission 0.7 1.4 2.3 Studies
conducted
between 1986
and 1996
US Uses
interquartile
range
Ioannidis
(1998)
31
AIDS First
submission
Publication 0.6 0.8 1.4 Studies
conducted
between 1986
and 1996
US Uses
interquartile
range;
negative
studies suffer
a substantial
time lag. With
some
expectations,
most of this
lag is
generated
after a trial
has been
completed.
(p. 284)
Mansfield
(1991)
33
Manufacturing
products, including
drugs
Academic
research
Commercialization 7 19751985 US Cites Gellman
who
calculated a
lag of 7.2 year
between
(19531973)
Misakian and
Biro (1998)
39
Passive smoking Funding
began
Date of first
publication
describing health
effects
3(+); 57
(); 3
(incon)
Studies started
between 1981
and 1995;
study
conducted
1995
US study of
funding bodies
Does not report
for all papers,
but only by
direction of
results; noted
that tobacco-
affiliated
organizations
did not
respond to
requests to
take part in
the study
despite
(Continued)
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Understanding time lags in translational research and why it matters
515
Table 1
Continued
Author Context Start of
time lag
End of time lag Time lag (years)
Dates Country Notes
Lower
range
Median Mean Higher
Range
several
requests
Pulido et al.
(1994)
47
Papers published in
Medicina Clı
´nica
Submission of
paper
Publication 0.81 0.86 0.92 Looked at 12
articles in
5-year cycles,
from 1962
1992; data for
1982
Spanish journal
articles
Study is in
Spanish; only
seems to
report data
from two
cycles (1982
and 1992)
Pulido et al.
(1994)
47
Papers published in
Medicina Clı
´nica
Submission of
paper
Publication 0.32 0.81 0.56 Same study as
above but,
data for 1992
Spanish journal
articles
Study is in
Spanish; only
seems to
report data
from two
cycles (1982
and 1992)
Stern and
Simes (1997)
8
Quantitative studies
submitted to Royal
Prince Albert
Hospital Ethics
Committee
Ethical
approval
Date of first
publication
3.9 (+);
6.9
(– or
inconc)
5.7 (+);
(– or
inconc)
Ethical approval
given in 1979
1981; study
conducted in
1992
Royal Prince
Alfred Hospital
Ethics
Committee
Applicants,
Australia
Does not report
for all papers,
but only by
direction of
results
Stern and
Simes (1997)
8
Trials submitted to
Royal Prince Albert
Hospital Ethics
Committee
Ethical
approval
Date of first
publication trial
data
3.7 (+);
7.0
(– or
inconc)
5.7 (+);
(– or
inconc)
Ethical approval
given in 1979
1981; study
conducted in
1992
Royal Prince
Alfred Hospital
Ethics
Committee
Applicants,
Australia
Does not report
for all papers,
but only by
direction of
results
Sternitzke
(2010)
30
Pharmacuetal
products; drugs
approved by FDA
Chemical
synthesis
FDA approval 11.5 US drugs Sternitzkes
estimates
derive from a
literature
review
Wang-Gilam
(2010)
25
Cancer trials Trial
application
Enrolment 0.3 0.44 20012008 US; two centres
Wratschko
(2009)
18
General pharma Drug
discovery
Commercialization 10 12 17 US book Derived from LR
Green (2005)
The difference between this value and the 17 years cited in the introduction is that for this study the authors also took into account estimates between
time of funding and publication and other studies (which are reviewed in this paper)
HERG = Health Economics Research Group at Brunel University
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Journal of the Royal Society of Medicine
516
compared with four to five years for those with
positive results. Comparing the slowest negative
publication with the fastest positive publication
makes a potential difference of four years half
of the maximu m lag.
Some studies aggregate data from earlier
studies without critical reflection or recognition
of this.
16,25
For example, Balas and Bohen calculate
an average of 17 years from original research to
practice formed from adding together a number
of single studies of different ph ases including
one that estimates a lag of 613 years.
16
Account-
ing for this changes their estimate of the time lag
between journal submission to use in practice
from between 17 years to 23 years.
Not surprisingly, studies a lso show variatio n
in time lags by domain
38
and even intervention
within a single domain. For example, examining
research relating to advances in neonatal care,
Grant et al. traced research papers back through
four ‘generations’ of publication. They found
‘the overall ti me between generations 1 to 4
ranges from 13 years (for artificial surfactant) to
21 years (for parenteral nutrition). The other
three adva nces took 17 years to develop
through four generations of citations’. Atman
et al.’s study of treatm ent for myocardial infarc-
tion yielded similar results: it took six years for
a review of evidence supporting the use of
thrombolytic drugs to result in a standard rec-
ommendation, whereas prophylactic lidocaine
was used widely in practice for 25 years base d
on no evidence of effectiveness.
40
Content also appears to influence time lags. A
common theme found in the literature concerns
publication bias, and their implications for
judging effectiveness.
28,29,33,37 39,41 47
Altman
looked at citations of new statistical techniques
applied to health and found that it took 46
years for a paper to receive 25 citations if the tech-
nique was new. An ‘expository article could
achieve 500 citations over the same period.
48
Contopoulos-Ioannidis found different publi-
cation trajectories for different types of
invention.
38
Studies also show that time lags are not stable
over time. For example, Pulido noted a difference
of 0.9 years or 0.3 years from acceptance to publi-
cation in 1992 and 1982, respectively.
49
DiMasi
reported a slight shortening of the approval
Figure 2
Chart showing the approximate range and average time lag reported in studies of time lags in health
research. NB HERG is the Health Economics Research Group at Brunel University
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Understanding time lags in translational research and why it matters
517
process between 1991 and 2003.
31
Tsuiji and
Tsutani reported reduced lags in the drug
approval process in Japan following a change in
policy to try and expedite it.
50
Single papers raise issues that are not generally
discussed but do seem relevant to measuring time
lags from publications in particular. These issues
include ‘generations’ of research
19
and overlaps
in research pub lications.
19,33
For example,
Ducullier, of the 649 studies they included, five
years later 59% had published research findings
but most (84%) had more than one paper from
the same study.
28
Discussion
This paper aimed to synthesize existing knowl-
edge to offer a conceptual model that can be
used to standardize measurement and thus help
to quantify lags in future. The strengths of the
study are that, to our knowledge, this provides
the first attempt to review lags comprehensively,
both in terms of using multiple approaches to
find studies, but also in attempting to quantify
time lags along the translation continuum. The
review exposed a number of weaknesses in the lit-
erature and gaps in knowledge, which are not
often discussed. Despite our attempts to be com-
prehensive, however, we are aware that studies
of time lags in health research are widely distribu-
ted and not easily identified using formal litera-
ture searches and we may have failed to capture
relevant studies. We struggled to find research
quantifying lags in basic research and the first
translation gap in particular.
Our aim to understand lags has been limited by
the weaknesses of existing data. Limitations of the
literature examined include the use of proxy
measures. Much of the literature on lags focuses
on dissemination and publication in peer-review
journals in particular as these are the most mea-
sureable. If there are significant lags in, say, the
grant or ethics process, this is less likely to be
reflected in current total lag estimations. More-
over, the variation in choice of proxy measures
means that studies are almost never measuring
the same thing, making valid aggregation and
generalization difficult.
There is a clear trend in the literature to seek a
single answer to a single question through the
calculation of an average. The variation found in
the literature suggests that this is not possible (or
even desirable), and variation matters. Moreover,
many of the published ‘averages’ are derived
from adding an empirically derived mean duration
for one section of journey from one point in time, in
one topic, and adding it to other parts without
reflection. Thus any poor estimates are transferred
forward into later analysis, and also hide a com-
plexity which is highly relevant to research policy.
There also appears to be a mismatch between
conceptual models of the translation process,
and the measuring of lags. For example, the gap
between guideline publication and translation
into actual practice is often ignored, suggesting
an under-estimation of the time lags in some
cases. On the other hand, interventions may
come into use before guidelines outlining them
have been published suggesting an over-
estimation of time lags in other cases.
Using different endpoints, different domains
and different approaches, Balas and Bohen
16
and
Grant et al.
30
both estimate the time lag in health
research being 17 years. Wratschko also suggested
17 years as the highest limit for the time taken
from drug discovery to commercialization.
18
It is
surprising that 17 is the answer to several related
but differing questions. Is this coincide nce or
not? One possible reason for the convergence is
the difficulty of measuring longer lags because
of limitations of citation indexes, other records
and recollections which provides a ceiling to
such estimates and leads to a convergence of
average lags.
While not able to adequately quantify time lags
in health research, this study provides lessons for
future research policy and practice. Concerns
about lags are not new
14
but are unresolved.
Based on the review, and our own work on
lags,
13,17,19,30,51
we would argue that an essential
step to being able to quantify time lags, and
thereby make improvements, requires stake-
holders to agree definitions, key stages and
measures. It also perhaps requires stakeholders to
develop a more nuanced understanding of when
time lags are good or bad, linked to policy choices
around ethics and governance for example,
52
or
reflect workforce issues.
52,53
Indeed, a recent paper
by Trochmin et al.
54
proposes a ‘process maker
model’ whereby they identify a set of operational
and measureable markers along a generalized
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Journal of the Royal Society of Medicine
518
pathway like that illustrated in Figure 1. It seems to
us that this provides an excellent framework to
support future data gathering and analysis and
thus provide a more informed base from which to
develop policy to address time lags.
Currently much of the complexity, and there-
fore the potential for improvement, are hidden in
this preference for ‘averages’. No attention is
given to understanding distributions and vari-
ations. This effectively ‘blindfolds’ investment
decisions and risks wasting efforts to reduce
lags. As noted in the introduction, some lags are
necessary to ensure the safety and efficacy of
implementing new research into practice. The dis-
cussion in the literature fails to consider what is
necessary or desirable, tending to assume that all
lags are unwelcome. A key question for policy is
to identify which lags are beneficial and which
are unnecessary, but to answer this question it is
necessary to have an accurate and comparable
estimate of the lags.
Conclusion
Translating scientific discoveries into patient
benefit more quickly is a policy priority of many
health research systems. Despite their policy sal-
ience, little is known about time lags and how
they should be managed. This lack of knowledge
puts those responsible for enabling translational
research at a disadvantage. An ambitious reason
for being able to accurately measure lags is that
it would be possible to look at their distribution
to identify research that is both slow and fast in
its translation. Further investigation of the charac-
teristics of research at both ends of a distribution
could help identify actionable policy interventions
that could speed up the translation process, where
appropriate, and thus increase the return on
research investment.
References
1 Westfall J, Mold J, Fagnan L. Practice-based research
“Blue Highways” on the NIH roadmap. JAMA
2007;297:4036
2 Cooksey D. A review of UK health research funding: Norwich:
HMSO, 2006
3 Trochim W. Translation Won’t Happen Without
Dissemination and Implementation: Some Measurement
and Evaluation Issues. 3rd Annual Conference on the Science
of Dissemination and Implementation. Bethesda, MD: 2010
4 Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou
O. Diffusion of Innovations in Service Organizations:
Systematic Review and Recommendations. Milbank Q
2004;82:581629
5 Rogers E. Diffusion of innovations. 4th edn. New York, NY:
The Free Press, 1995
6 Packer C, Simpson S, Stevens A. International diffusion of
new health technologies: A ten-country analysis of six
health technologies. Int J Technol Assess Health Care
2006;22:41928
7 Davis D, Evans M, Jadad A, et al. The case for knowledge
translation: shortening the journey from evidence to effect.
BMJ 2003;327:33 5
8 Lang ES, Wyer PC, Haynes RB. Knowledge translation:
Closing the evidence-to-practice gap. Ann Emerg Med
2007;49:35563
9 Graham ID, Logan J, Harrison MB, et al. Lost in knowledge
translation: Time for a map? J Contin Educ Health Prof
2006;26:1324
10 Graham I, Tetroe J, Group KTR. Some theoretical
underpinnings of knowledge translation. Acad Emerg Med
2007;14:93641
11 Baumbusch J, Kirkham S, Khan K, et al. Pursuing common
agendas: a collaborative model for knowledge translation
between research and practice in clinical settings. Res Nurs
Health 2008;31:130 40
12 Ward V, House A, Hamer S. Developing a framework
for transferring knowledge into action: a thematic
analysis of the literature. J Health Serv Res Policy
2009;14:15664
13 Health Economics Research Group, Office of Health
Economics, RAND Europe. Medical Research: Whatsit
Worth? Estimating the Economic Benefits from Medical Research
in the UK. London: UK Evaluation Forum, 2008
14 Yokote G, Utterback R. Time lapses in information
dissemination: research laboratory to physician’s office.
Bull Med Libr Assoc 1974;62:251
15 Green L, Ottoson J, Garcı
´
a C, Hiatt R. Diffusion theory and
knowledge dissemination, utilization, and integration in
public health. Annu Rev Public Health 2009;30 :151 74
16 Balas E, Boren S. Managing Clinical Knowledge for Health
Care Improvement. In: van Bemmel JH, McCray AT, eds.
Yearbook of Medical Informatics. Stuttgart: Schattauer
Verlagsgesellschaft mbH, 2000:65 70
17 Grant J, Green L, Mason B. Basic research and health: a
reassessment of the scientific basis for the support of
biomedical science. Res Eval 2003;12:21724
18 Wratschko K. Empirical Setting: The pharmaceutical
industry. Strategic Orientation and Alliance Portfolio
Configuration. New York, NY: Springer, 2009
19 Grant J, Green L, Mason B. From bedside to bench: Comroe
and Dripps revisited. HERG Research Report No 30. Uxbridge:
The Health Economics Research Group, Brunel
University, 2003
20 Khoury M, Gwinn M, Yoon P, Dowling N, Moore C,
Bradley L. The continuum of translation research
in genomic medicine: how can we accelerate the
appropriate integration of human genome discoveries
into health care and disease prevention? Genet Med
2007;9:665
21 Khoury MJ, Gwinn M, Ioannidis JP. The emergence of
translational epidemiology: from scientific discovery
to population health impact. Am J Epidemiol
2010;172:51724
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Understanding time lags in translational research and why it matters
519
22 Dougherty D, Conway P. The “3T’s” road map to transform
US health care: the “how” of high-quality care. JAMA
2008;299:231921
23 Woolf S. The meaning of translational research and why it
matters. JAMA 2008;299:21113
24 Balconi M, Brusoni S, Orsenigo L. In defence of the linear
model: An essay. Res Policy 2010;39:113
25 Mason C, Manzotti E. The Translation Cycle: round and
round in cycles is the only way forward for regenerative
medicine. Regen Med 2010;5:1535
26 Reis S, McDonald M, Byers S. Crossing the Research Valleys
of Death: The University of Pittsburgh Approach. Clin Trans
Sci 2008;1:9 10
27 Wang-Gillam A, Williams K, Novello S, Gao F, Scagliotti G,
Govindan R. Time to activate lung cancer clinical trials and
patient enrollment: A representative comparison study
between two academic centers across the Atlantic. J Clin
Oncol 2010;28:3803 7
28 Decullier E, Lheritier V, Chapuis F. Fate of biomedical
research protocols and publication bias in France:
retrospective cohort stud y. BMJ 2005;331:19
29 Stern J, Simes R. Publication bias: evidence of delayed
publication in a cohort study of clinical research projects.
BMJ 1997;315:640 5
30 Grant J, Cottrell R, Cluzeau F, Fawcett G. Evaluating
“payback” on biomedical research from papers cited in
clinical guidelines: applied bibliometric study. BMJ
2000;320:110711
31 DiMasi J, Hansen R, Grabowski H. The price of innovation:
new estimates of drug development costs. J Health Econ
2003;22:15185
32 Sternitzke C. Knowledge sources, patent protection, and
commercialization of pharmaceutical innovations. Res
Policy 2010;39:81021
33 Ioannidis J. Effect of the statistical significance of results on
the time to completion and publication of randomized
efficacy trials. JAMA 1998;279:281
34 van Luijn J, Stolk P, Gribnau F, Leufkens H. Gap in
publication of comparative information on new medicines.
Br J Clin Pharmacol 2008;65:716 22
35 Mansfield E. Academic research and industrial innovation.
Res Policy 1991;20:112
36 Cockburn I, Henderson R. Publicprivate interaction in
pharmaceutical research. PNAS 1996;93:12725
37 Dwan K, Altman D, Arnaiz J, et al. Systematic review of the
empirical evidence of study publication bias and outcome
reporting bias. PLoS One 2008;3:3081
38 Contopoulos-Ioannidis D, Alexiou G, Gouvias T, Ioannidis
J. Life cycle of translational research for medical
interventions. Science 2008;321:12989
39 Hopewell S, Clarke M, Stewart L, Tierney J. Time to
publication for results of clinical trials. Cochrane Database
Syst Rev 2007;(2):MR000011
40 Antman E, Lau J, Kupelnick B, Mosteller F, Chalmers T. A
comparison of results of meta-analyses of randomized
control trials and recommendations of clinical experts:
treatments for myocardial infarction. JAMA 1992;268:2408
41 Misakian A, Bero L. Publication bias and research on
passive smoking: comparison of published and
unpublished studies. JAMA 1998;280:2503
42 Harris P, Takeda A, Loveman E, Hartwell D. Time to full
publication of studies of anticancer drugs for breast cancer,
and the potential for publication bias. Int J Technol Assess
Health Care 2010;26:110 16
43 Takeda A, Loveman E, Harris P, Hartwell D, Welch K.
Time to full publication of studies of anti-cancer medicines
for breast cancer and the potential for publication bias: a
short systematic review. Health Technol Assess 2008;12:168
44 Easterbrook P, Gopalan R, Berlin J, Matthews D. Publication
bias in clinical research. Lancet 1991;337:867 72
45 Dickersin K, Chan S, Chalmersx T, Sacks H, Smith Jr H.
Publication bias and clinical trials. Control Clin Trials
1987;8:34353
46 Dickersin K. The existence of publication bias and risk
factors for its occurrence. JAMA 1990;263 :1385 9
47 Grol R, Grimshaw J. From best evidence to best practice:
effective implementation of change in patients’ care. Lancet
2003;362:122530
48 Altman D, Goodman S. Transfer of technology from
statistical journals to the biomedical literature: past trends
and future predictions. JAMA 1994;272:12932
49 Pulido M, Gonza
´
lez J, Sanz F. Original articles published in
Medicina Clı
´
nica during 30 years (1962 1992): number of
authors, interval between acceptance and publication and
bibliographic references. Med Clin 1994;103:7705
50 Tsuji K, Tsutani K. Follow the leader. Nature 2008;453:8512
51 Grant J, Lewison G, May R. Government funding of
research and development. Science 1997;278
:87880
52 Newby D, Webb D. Translational research: a priority for
health and wealth. Heart 2010;96:815 16
53 Holgate S. The future of lung research in the UK. Thorax
2007;62:102832
54 Trochim W, Kane C, Graham M, Pincus H. Evaluating
translation research. A process marker model. Clinical and
Translation Science 2011;4:153 62
55 Comroe J, Dripps R. Scientific basis for the support of
biomedical science. Science 1976;192:10511
56 DiMasi J, Hansen R, Grabowski H, Lasagna L. Cost of
innovation in the pharmaceutical industry. J Health Econ
1991;10:10742
J R Soc Med 2011: 104: 510 520. DOI 10.1258/jrsm.2011.110180
Journal of the Royal Society of Medicine
520
... Research awards which had been influential to policy and/or the [public] health care sector were targeted because this allowed an exploration of proven facilitators and barriers to the potential impacts being realised. Award selection based on completed research was most appropriate given the time-lags that often exist between completing the research and realised influence [21,22]. ...
Article
Full-text available
Research funded by the National Institute for Health Research Public Health Research Programme is being undertaken in a complex system which brings opportunities and challenges for researchers to maximise the impact of their research. This study seeks to better understand the facilitators, challenges and barriers to research impact and knowledge mobilisation from the perspective of UK public health researchers. A qualitative study using semi-structured interviews, informed by the Payback Framework, with public health researchers who held a research award with the National Institute for Health Research Public Health Research programme up to March 2017 was conducted. Following a thematic analysis, three strongly interlinked themes were extracted from the data and three key factors were highlighted as important for facilitating knowledge mobilisation and impact in UK public health research: (1) Public health researcher’s perception of the purpose of the research (2) Approaches to undertaking Knowledge mobilisation activities (3) The complex nature of public health research in the wider research context. These have been reflected onto the Payback framework. Public health researchers can maximise the likelihood for impact by being aware of the context in which they are undertaking research, using different methods, and employing several strategies to take advantage of opportunities. There is a need to support researchers with knowledge mobilisation activities and for funders to identify their expectations of the impact resulting from research. Our findings have relevance to public health researchers and funders interested in increasing the benefit that research brings to society.
... Implementation Science Communications (2022) 3:68 the multitude of actors, components, and processes involved before implementing IV iron in the health system. On average, integrating evidence-based interventions at scale takes 17 years and is likely even longer in lower-resourced settings [20][21][22]. Implementation research, which seeks to understand what, why, and how interventions work in real-world settings, plays a key role in designing appropriate implementation strategies that bridge the research-to-practice gap [23]. First, understanding the characteristics, behaviours, and needs of the target population and adapting the intervention accordingly to the socio-cultural and institutional contexts are keys to facilitating end-users access and buy-in from healthcare providers [24]. ...
Article
Full-text available
Background Antenatal iron supplementation is critical to maternal and child health; however, access and adherence to oral iron are inconsistent in many low- and middle-income countries (LMICs). Modern intravenous (IV) iron products have become available in high-income clinical settings and provide an opportunity to deliver high doses of iron in a single-short infusion during pregnancy. However, there is limited knowledge of the drivers and barriers for such an intervention to be effectively delivered and upscaled in LMICs. In this study protocol, we describe the implementation research programme to support an IV iron intervention in Malawi for pregnant women with moderate and severe anaemia. Methods The implementation research programme has three phases, each guided by implementation science conceptual frameworks. In Phase 1, we will conduct formative research (context assessment of the health system with key informant interviews) to determine how IV iron can be effectively introduced into routine antenatal care. We will use the findings to co-develop potential strategies with end-users and healthcare providers to improve intervention implementation. In Phase 2, we will disseminate the implementation strategies to support the uptake and delivery of the intervention in the study settings. In Phase 3, the intervention will be implemented, and we will conduct formative evaluation (interviews with end-users, healthcare providers, and analysis of health services data) to investigate the feasibility and acceptability of the intervention and strategies. We will also identify processes and contextual factors that facilitate or impede the delivery and uptake of IV iron. Discussion In LMICs, modern IV iron products present a novel opportunity to rapidly cure moderate and severe anaemia in pregnancy, thereby improving maternal and child health outcomes. This implementation research programme will provide guidance and recommendations on how best an IV iron intervention for pregnant women with anaemia can be implemented in an LMIC setting like Malawi. We will develop locally relevant and culturally appropriate implementation strategies by engaging with key stakeholders (pregnant women, healthcare providers, and policymakers) and identifying factors likely to facilitate successful implementation. The findings of this research can guide the implementation of an IV iron intervention in Malawi and other LMICs.
The value of including consumers’ and carers’ views at the early stages of study design is increasingly being recognised as essential to improving the relevance and quality of research. One method of achieving this is by actively seeking and regularly updating consumer and carer priorities for mental health research. The current study presents priorities for mental health research collected from two virtual World Cafés with consumers and carers (n = 4, n = 7) held in 2021. Over 200 priorities were identified (13 themes, 64 subthemes), which were then compared with two combined data collection activities from 2013 (face-to-face forum; n = 25), and 2017 (online survey; n = 70). There appears to be some evolution in consumer and carer priorities over time. A key difference was that in the previous studies, mental health service issues were at the individual service delivery level, whereas in the current study, a broader focus was on mental health systems of care and issues around service funding, accessibility, and equity of access. It is possible these changes may also have resulted from key differences between the studies, including the methods, setting, and participants. Overall, similar to our previous studies no clear priorities were identified; however, a significant number of important research topics were identified by consumers and carers, providing a rich agenda from which to improve the management of mental health.
Article
Translational science has been introduced as the nexus among the scientific and the clinical field, which allows researchers to provide and demonstrate that the evidence-based research can connect the gaps present between basic and clinical levels. This type of research has played a major role in the field of cardiovascular diseases, where the main objective has been to identify and transfer potential treatments identified at preclinical stages into clinical practice. This transfer has been enhanced by the intromission of digital health solutions into both basic research and clinical scenarios. This review aimed to identify and summarize the most important translational advances in the last years in the cardiovascular field together with the potential challenges that still remain in basic research, clinical scenarios, and regulatory agencies.
Article
Background: Given available effective biomedical and behavioral prevention and treatment interventions, HIV-related implementation research (IR) is expanding. The rapid generation and dissemination of IR to inform guidelines and practice has the potential to optimize the impact of the Ending the Epidemic Initiative and the HIV pandemic response more broadly. Methods: We leveraged a prior mapping review of NIH-funded awards in HIV and IR from January 2013 to March 2018 and identified all publications linked to those grants in NIH RePORTER through January 1, 2021 (n = 1509). Deduplication and screening of nonoriginal research reduced the count to 1032 articles, of which 952 were eligible and included in this review. Publication volume and timing were summarized; Kaplan-Meier plots estimated time to publication. Results: Among the 215 NIH-funded IR-related awards, 127 of 215 (59%) published original research directly related to the grant, averaging 2.0 articles (SD: 3.3) per award, largely in the early IR phases. Many articles (521 of 952, 55%) attributed to grants did not report grant-related data. Time from article submission to publication averaged 205 days (SD: 107). The median time-to-first publication from funding start was 4 years. Data dissemination velocity varied by award type, trending toward faster publication in recent years. Delays in data velocity included (1) time from funding to enrollment, (2) enrollment length, and (3) time from data collection completion to publication. Conclusion: Research publication was high overall, and time-to-publication is accelerating; however, over 40% of grants have yet to publish findings from grant-related data. Addressing bottlenecks in the production and dissemination of HIV-related IR would reinforce its programmatic and policy relevance in the HIV response.
Article
Introduction: Ending the HIV Epidemic initiatives provide a unique opportunity to use implementation scientific methods to guide implementation of evidence-based practices and evaluate their effectiveness in real-word settings to improve HIV inequities. This report demonstrates our use of Implementation Mapping (IM) to engage participating county health departments, AIDS services organizations, and community-based organizations in the development of a data dashboard to track the PrEP care continuum for the state of Alabama, an Ending the HIV Epidemic hotspot. Methods: Our project is guided by an overarching Implementation Research Logic Model and by the tenets of IM, a 5-step approach to support researchers and community partners in the systematic selection, development, and/or tailoring of implementation strategies to increase program adoption, implementation, and sustainability. Results: Step 1, the needs assessment, established baseline data elements for a PrEP care continuum for participating community-based organizations, AIDS services organizations, and the state health department as well as investigated their desire for data visualization and willingness to share data to inform initiatives to improve PrEP access. Step 2 identified adaptability, relative advantage, and complexity as determinants of intervention adoption. Based on findings from steps 1 and 2, the investigators and community partners determined to move forward with development of a data dashboard. Step 3 identified the following implementation strategies to support a dashboard, including development of educational materials, synchronous and asynchronous training, technical assistance, and improved record systems. Discussion: Using IM supports community-engaged researchers in designing strategies to end the epidemic that are context-specific and more impactful in real-word settings.
Article
Full-text available
Background: The literature reports long time lags between the several processes involved in the translation of drug research and development into clinical application. To expedite these processes, translational research has emerged as a process that can be applied to reduce the lag between scientific discoveries and their practical application. Thus, the objective of this study was to estimate the time lag in translational research of biological drugs for the treatment of rheumatoid arthritis included in the Brazilian Unified Health System [Sistema Único de Saúde (SUS)]. Methods: A descriptive retrospective study was conducted based on secondary data loaded by SUS users in public sources and systems to estimate the time lag between the publication of phase I clinical trial results to drug use in clinical settings. The dates of translational research activities were identified from markers and steps. Structured searches were conducted in the literature and reports from the National Commission for the Incorporation of Technologies in the SUS (Conitec) as well as from health authorities, and analyzed. Results: Between 2012 and 2019, SUS included five biological agents for the treatment of rheumatoid arthritis. The mean time lag from clinical development to use of these agents was 11.13 years (range, 8.57 to 12.90 years). The mean time lag for the stages of translational research were 5.30 (T1—basic research to clinical research), 5.08 (T2—clinical research to research synthesis), and 0.75 (T3—research synthesis to evidence-based practice) years. A shorter time lag was observed in the Brazilian case when it was possible to compare with other studies. Conclusions: The estimated time lag of biological drugs used in the treatment of rheumatoid arthritis was determined based on the translational research steps model adapted to the Brazilian context. Brazil has instituted legal frameworks that set deadlines for sanitary registration, health technology assessment (HTA), and the availability of drugs in the SUS, thus, allowing for a reduced stage T2 time lag. Nevertheless, improvements are still required in stages T1 and T2, especially in publishing the results of clinical trials.
Article
Full-text available
Background: The literature reports long time lags between the several processes involved in the translation of drug research and development into clinical application. To expedite these processes, translational research has emerged as a process that can be applied to reduce the lag between scientific discoveries and their practical application. Thus, the objective of this study was to estimate the time lag in translational research of biological drugs for the treatment of rheumatoid arthritis included in the Brazilian Unified Health System [Sistema Único de Saúde (SUS)]. Methods: A descriptive retrospective study was conducted based on secondary data loaded by SUS users in public sources and systems to estimate the time lag between the publication of phase I clinical trial results to drug use in clinical settings. The dates of translational research activities were identified from markers and steps. Structured searches were conducted in the literature and reports from the National Commission for the Incorporation of Technologies in the SUS (Conitec) as well as from health authorities, and analyzed. Results: Between 2012 and 2019, SUS included five biological agents for the treatment of rheumatoid arthritis. The mean time lag from clinical development to use of these agents was 11.13 years (range, 8.57 to 12.90 years). The mean time lag for the stages of translational research were 5.30 (T1—basic research to clinical research), 5.08 (T2—clinical research to research synthesis), and 0.75 (T3—research synthesis to evidence-based practice) years. A shorter time lag was observed in the Brazilian case when it was possible to compare with other studies. Conclusions: The estimated time lag of biological drugs used in the treatment of rheumatoid arthritis was determined based on the translational research steps model adapted to the Brazilian context. Brazil has instituted legal frameworks that set deadlines for sanitary registration, health technology assessment (HTA), and the availability of drugs in the SUS, thus, allowing for a reduced stage T2 time lag. Nevertheless, improvements are still required in stages T1 and T2, especially in publishing the results of clinical trials.
Article
Full-text available
Objectives: The objective of this study was to examine and explain the differential international diffusion of six health innovations. Methods: A retrospective diffusion study was undertaken of sildenafil, cyclooxygenase-II (COX 11) inhibitors, beta interferon, verteporfin, deep brain stimulators, and drug-eluting coronary stents in ten countries-Australia, Canada, Denmark, France, The Netherlands, Norway, Spain, Sweden, Switzerland, and the United Kingdom. We plotted diffusion curves of daily defined doses per quarter, vials or implants per million population, and examined the association between diffusion and five key variables. Results: Canada, Switzerland, and Sweden are generally high users of new technologies; Spain, Denmark, and particularly the United Kingdom are low users. Almost all countries experienced rapid adoption of sildenafil with diffusion to a similar level; there was variable adoption and diffusion of COX 11 inhibitors, verteporfin, and interferon beta; drug-eluting stents penetrated the market in a similar way in all but one country; and two countries had very different adoption patterns for deep brain stimulators. Above average health spending and the presence of health technology assessment (HTA) or other guidance reports are consistently associated with increased diffusion. Early warning activity and a national coverage decision being taken are more likely to be associated with a reduced diffusion. Conclusions: The significant differences in diffusion between different countries are not consistent with a neat evidence-based world. The tools available to policy makers to control diffusion (early warning systems, HTA, and a fourth hurdle) play some part in influencing diffusion but need close scrutiny of how successfully they operate.
Chapter
The pharmaceutical industry offers an ideal setting for my research for several reasons that will be elaborated on in greater detail in chapter 5.2 when the sample is defined. The following section contains an overview of the pharmaceutical industry as well as a discussion of the peculiarities of the pharmaceutical value chain and the special role of alliances, mergers and acquisitions for value creation in this vibrant business context.
Article
Objectives: To develop a methodology for evaluating the impact of research on health care, and to characterise the papers cited on clinical guidelines. Design: The bibliographic details of the papers cited in 15 clinical guidelines, developed in and for the United Kingdom, were collated and analysed with applied bibliometric techniques. Results: The median age of papers cited in clinical guidelines was eight years; most papers were published by authors living in either the United States (36%) or the United Kingdom (25%)—this is two and a half times more than expected as about 10% of all biomedical outputs are published in the United Kingdom; and clinical guidelines do not cite basic research papers. Conclusion: Analysis of the evidence base of clinical guidelines may be one way of tracking the flow of knowledge from the laboratory to the clinic. Moreover, such analysis provides a useful, clinically relevant method for evaluating research outcomes and different strategies in research and development.
Article
Objective. —To examine the temporal relationship between accumulating data from randomized control trials of treatments for myocardial infarction and the recommendations of clinical experts writing review articles and textbook chapters.Data Sources. —(1) MEDLINE search from 1966 to present; search terms used were myocardial infarction, clinical trials, multicenter studies, double-blind method, meta-analysis, and the text word "random:"; (2) references from pertinent articles and books; and (3) all editions of English-language general medical texts and manuals and review articles on treatment of myocardial infarction.Study Selection. —Randomized control trials of therapies for reducing the risk of total mortality in myocardial infarction (acute and secondary prevention). Review articles and textbook chapters dealing with the general clinical management of patients with myocardial infarction.Data Extraction. —Two authors read the material and recorded the results; disagreements were resolved by conference.Data Synthesis. —We used the technique of cumulative meta-analysis (performing a new meta-analysis when the results of a new clinical trial are published) and compared the results with the recommendations of the experts for various treatments for myocardial infarction. Discrepancies were detected between the meta-analytic patterns of effectiveness in the randomized trials and the recommendations of reviewers. Review articles often failed to mention important advances or exhibited delays in recommending effective preventive measures. In some cases, treatments that have no effect on mortality or are potentially harmful continued to be recommended by several clinical experts.Conclusions. —Finding and analyzing all therapeutic trials in a given field has become such a difficult and specialized task that the clinical experts called on to summarize the evidence in a timely fashion need access to better databases and new statistical techniques to assist them in this important task.(JAMA. 1992;268:240-248)
Article
The bibliographic details of research papers underpinning five clinical advances in neonatal intensive care were collated and analysed using applied bibliometric techniques. After a time-lag of about 17 years, between 2% and 21% of research underpinning clinical advances in NIC could be described as basic. This observation is at odds with Comroe and Dripps' 1976 finding.