METHODS OF SYSTEMATIC REVIEWS AND META-ANALYSIS
Preferred Reporting Items for Systematic Reviews
and Meta-Analyses: The PRISMA Statement
*, Alessandro Liberati
, Jennifer Tetzlaff
, Douglas G. Altman
The PRISMA Group
Ottawa Methods Centre, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
Department of Epidemiology and Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
`di Modena e Reggio Emilia, Modena, Italy
Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri, Milan, Italy
Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom
Accepted 22 June 2009
Systematic reviews and meta-analyses have become in-
creasingly important in health care. Clinicians read them
to keep up to date with their ﬁeld [1,2], and they are often
used as a starting point for developing clinical practice
guidelines. Granting agencies may require a systematic
review to ensure there is justiﬁcation for further research
, and some health care journals are moving in this
direction . As with all research, the value of a system-
atic review depends on what was done, what was found,
and the clarity of reporting. As with other publications,
the reporting quality of systematic reviews varies, limiting
readers’ ability to assess the strengths and weaknesses of
Several early studies evaluated the quality of review
reports. In 1987, Mulrow examined 50 review articles pub-
lished in four leading medical journals in 1985 and 1986
and found that none met all eight explicit scientiﬁc criteria,
such as a quality assessment of included studies .In
1987, Sacks and colleagues  evaluated the adequacy of
reporting of 83 meta-analyses on 23 characteristics in six
domains. Reporting was generally poor; between one and
14 characteristics were adequately reported (mean 57.7;
standard deviation 52.7). A 1996 update of this study
found little improvement .
In 1996, to address the suboptimal reporting of meta-
analyses, an international group developed a guidance
called the QUOROM Statement (QUality OfReporting
OfMeta-analyses), which focused on the reporting of
meta-analyses of randomized controlled trials . In this
article, we summarize a revision of these guidelines,
renamed PRISMA (Preferred Reporting Items for System-
atic reviews and Meta-Analyses), which have been updated
to address several conceptual and practical advances in the
science of systematic reviews (Box 1).
The terminology used to describe a systematic review
and meta-analysis has evolved over time. One reason for
changing the name from QUOROM to PRISMA was
the desire to encompass both systematic reviews and
meta-analyses. We have adopted the deﬁnitions used by
the Cochrane Collaboration . A systematic review is a re-
view of a clearly formulated question that uses systematic
and explicit methods to identify, select, and critically ap-
praise relevant research, and to collect and analyze data
from the studies that are included in the review. Statistical
Provenance: Not commissioned; externally peer reviewed. In order to
encourage dissemination of the PRISMA Statement, this article is freely
accessible on the PLoS Medicine Web site (http://medicine.plosjournals.org/)
and will be also publishedin the Annals of Internal Medicine,BMJ,Journal of
Clinical Epidemiology,andOpen Medicine. The authors jointly hold the
copyright of this article. For details on further use, see the PRISMAWeb site
Funding: PRISMA was funded by the Canadian Institutes of Health
Research; Universita di Modena e Reggio Emilia, Italy; Cancer Research
UK; Clinical Evidence BMJ Knowledge; the Cochrane Collaboration; and
GlaxoSmithKline, Canada. AL is funded, in part, through grants of the Ital-
ian Ministry of University (COFIN - PRIN 2002 prot. 2002061749 and
COFIN - PRIN 2006 prot. 2006062298). DGA is funded by Cancer Re-
search UK. DM is funded by a University of Ottawa Research Chair. None
of the sponsors had any involvement in the planning, execution, or write-up
of the PRISMA documents. Additionally, no funder played a role in draft-
ing the manuscript.
Membership of the PRISMA Group is provided in the Acknowledgments.
* Corresponding author. Dr. David Moher, University of Ottawa,
Department of Epidemiology and Community Medicine, 501 Smyth Road,
The Ottawa Hospital, General Camp., Ottwa, Ontario, Canada K1H 8L6
E-mail address:email@example.com (D. Moher).
0895-4356/09/$ esee front matter Ó2009 The Authors. Published by Elsevier Inc. All rights reserved.
Journal of Clinical Epidemiology 62 (2009) 1006e1012
methods (meta-analysis) may or may not be used to ana-
lyze and summarize the results of the included studies.
Meta-analysis refers to the use of statistical techniques in
a systematic review to integrate the results of included
3. Developing the PRISMA Statement
A three-day meeting was held in Ottawa, Canada, in
June 2005 with 29 participants, including review authors,
methodologists, clinicians, medical editors, and a consumer.
The objective of the Ottawa meeting was to revise and ex-
pand the QUOROM checklist and ﬂow diagram, as needed.
The executive committee completed the following tasks,
prior to the meeting: a systematic review of studies exam-
ining the quality of reporting of systematic reviews, and
a comprehensive literature search to identify methodologi-
cal and other articles that might inform the meeting, espe-
cially in relation to modifying checklist items. An
international survey of review authors, consumers, and
groups commissioning or using systematic reviews and
meta-analyses was completed, including the International
Network of Agencies for Health Technology Assessment
(INAHTA) and the Guidelines International Network
(GIN). The survey aimed to ascertain views of QUOROM,
including the merits of the existing checklist items. The
results of these activities were presented during the meeting
and are summarized on the PRISMA Web site (http://
Only items deemed essential were retained or added to
the checklist. Some additional items are nevertheless desir-
able, and review authors should include these, if relevant
. For example, it is useful to indicate whether the sys-
tematic review is an update  of a previous review,
and to describe any changes in procedures from those
described in the original protocol.
Shortly after the meeting a draft of the PRISMA check-
list was circulated to the group, including those invited to
Box 1 Conceptual Issues in the Evolution from
QUOROM to PRISMA
Completing a Systematic Review Is an Itera-
The conduct of a systematic review depends heavily on
the scope and quality of included studies: thus system-
atic reviewers may need to modify their original review
protocol during its conduct. Any systematic review re-
porting guideline should recommend that such changes
can be reported and explained without suggesting that
they are inappropriate. The PRISMA Statement (Items
5, 11, 16, and 23) acknowledges this iterative process.
Aside from Cochrane reviews, all of which should have
a protocol, only about 10% of systematic reviewers
report working from a protocol . Without a protocol
that is publicly accessible, it is difﬁcult to judge
between appropriate and inappropriate modiﬁcations.
Conduct and Reporting Research Are Distinct
This distinction is, however, less straightforward for
systematic reviews than for assessments of the
reporting of an individual study, because the reporting
and conduct of systematic reviews are, by nature,
closely intertwined. For example, the failure of a sys-
tematic review to report the assessment of the risk of
bias in included studies may be seen as a marker of
poor conduct, given the importance of this activity
in the systematic review process .
Study-Level Versus Outcome-Level Assess-
ment of Risk of Bias
For studies included in a systematic review, a thorough
assessment of the risk of bias requires both a ‘‘study-
level’’ assessment (e.g., adequacy of allocation con-
cealment) and, for some features, a newer approach
called ‘‘outcome-level’’ assessment. An outcome-
level assessment involves evaluating the reliability
and validity of the data for each important outcome
by determining the methods used to assess them in
each individual study . The quality of evidence
may differ across outcomes, even within a study, such
as between a primary efﬁcacy outcome, which is
likely to be very carefully and systematically mea-
sured, and the assessment of serious harms ,
which may rely on spontaneous reports by investiga-
tors. This information should be reported to allow an
explicit assessment of the extent to which an estimate
of effect is correct .
Continued. Importance of Reporting Biases
Different types of reporting biases may hamper the
conduct and interpretation of systematic reviews.
Selective reporting of complete studies (e.g., publica-
tion bias)  as well as the more recently empirically
demonstrated ‘‘outcome reporting bias’’ within indi-
vidual studies [40,41] should be considered by authors
when conducting a systematic review and reporting its
results. Though the implications of these biases on the
conduct and reporting of systematic reviews them-
selves are unclear, some previous research has identi-
ﬁed that selective outcome reporting may occur also in
the context of systematic reviews .
1007D. Moher et al. / Journal of Clinical Epidemiology 62 (2009) 1006e1012
Checklist of items to include when reporting a systematic review or meta-analysis.
Section/Topic # Checklist Item Reported on Page #
Title 1 Identify the report as a systematic review, meta-analysis, or both.
Structured summary 2 Provide a structured summary including, as applicable: background; objectives;
data sources; study eligibility criteria, participants, and interventions;
study appraisal and synthesis methods; results; limitations; conclusions and
implications of key ﬁndings; systematic review registration number.
Rationale 3 Describe the rationale for the review in the context of what is already known.
Objectives 4 Provide an explicit statement of questions being addressed with reference to
participants, interventions, comparisons, outcomes, and study design (PICOS).
5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address),
and, if available, provide registration information including registration number.
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report
characteristics (e.g., years considered, language, publication status) used
as criteria for eligibility, giving rationale.
Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with
study authors to identify additional studies) in the search and date last searched.
Search 8 Present full electronic search strategy for at least one database, including any limits
used, such that it could be repeated.
Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in
systematic review, and, if applicable, included in the meta-analysis).
Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently,
in duplicate) and any processes for obtaining and conﬁrming data from investigators.
Data items 11 List and deﬁne all variables for which data were sought (e.g., PICOS, funding sources)
and any assumptions and simpliﬁcations made.
Risk of bias in
12 Describe methods used for assessing risk of bias of individual studies
(including speciﬁcation of whether this was done at the study or outcome level), and how
this information is to be used in any data synthesis.
Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means).
Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done,
including measures of consistency (e.g., I
) for each meta-analysis.
Risk of bias
15 Specify any assessment of risk of bias that may affect the cumulative evidence
(e.g., publication bias, selective reporting within studies).
Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-
regression), if done, indicating which were pre-speciﬁed.
Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review,
with reasons for exclusions at each stage, ideally with a ﬂow diagram.
Study characteristics 18 For each study, present characteristics for which data were extracted
(e.g., study size, PICOS, follow-up period) and provide the citations.
Risk of bias
19 Present data on risk of bias of each study and, if available, any outcome-level assessment
(see Item 12).
20 For all outcomes considered (beneﬁts or harms), present, for each study: (a) simple summary
data for each intervention group and (b) effect estimates and conﬁdence intervals, ideally
with a forest plot.
Synthesis of results 21 Present results of each meta-analysis done, including conﬁdence intervals
and measures of consistency.
Risk of bias across studies 22 Present results of any assessment of risk of bias across studies (see Item 15).
Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses,
meta-regression [see Item 16]).
24 Summarize the main ﬁndings including the strength of evidence for each main
outcome; consider their relevance to key groups (e.g., health care providers, users,
and policy makers).
Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review level
(e.g., incomplete retrieval of identiﬁed research, reporting bias).
Conclusions 26 Provide a general interpretation of the results in the context of other evidence,
and implications for future research.
Funding 27 Describe sources of funding for the systematic review and other support
(e.g., supply of data); role of funders for the systematic review.
1008 D. Moher et al. / Journal of Clinical Epidemiology 62 (2009) 1006e1012
the meeting but unable to attend. A disposition ﬁle was cre-
ated containing comments and revisions from each respon-
dent, and the checklist was subsequently revised 11 times.
The group approved the checklist, ﬂow diagram, and this
Although no direct evidence was found to support
retaining or adding some items, evidence from other
domains was believed to be relevant. For example, Item
5 asks authors to provide registration information about
the systematic review, including a registration number,
if available. Although systematic review registration is
not yet widely available [12,13], the participating journals
of the International Committee of Medical Journal Editors
(ICMJE)  now require all clinical trials to be regis-
tered in an effort to increase transparency and account-
ability . Those aspects are also likely to beneﬁt
systematic reviewers, possibly reducing the risk of an
excessive number of reviews addressing the same ques-
tion [16,17] and providing greater transparency when
updating systematic reviews.
4. The PRISMA Statement
The PRISMA Statement consists of a 27-item checklist
(Table 1; see also Text S1 for a downloadable Word tem-
plate for researchers to re-use) and a four-phase ﬂow
diagram (Figure 1; see also Figure S1 for a downloadable
Word template for researchers to re-use). The aim of the
PRISMA Statement is to help authors improve the reporting
of systematic reviews and meta-analyses. We have focused
on randomized trials, but PRISMA can also be used as a ba-
sis for reporting systematic reviews of other types of
research, particularly evaluations of interventions. PRISMA
may also be useful for critical appraisal of published
systematic reviews. However, the PRISMA checklist is
not a quality assessment instrument to gauge the quality
of a systematic review.
5. From QUOROM to PRISMA
The new PRISMA checklist differs in several respects
from the QUOROM checklist, and the substantive speciﬁc
changes are highlighted in Table 2. Generally, the PRISMA
checklist ‘‘decouples’’ several items present in the QUOR-
OM checklist and, where applicable, several checklist items
are linked to improve consistency across the systematic
The ﬂow diagram has also been modiﬁed. Before includ-
ing studies and providing reasons for excluding others, the
review team must ﬁrst search the literature. This search re-
sults in records. Once these records have been screened and
eligibility criteria applied, a smaller number of articles will
# of records identified
# of additional records
identified through other
# of records after duplicates removed
# of records screened # of records
# of full-text articles
assessed for eligibility
# of studies included
in qualitative synthesis
# of full-text
# of studies included
Fig. 1. Flow of information through the different phases of a systematic review.
1009D. Moher et al. / Journal of Clinical Epidemiology 62 (2009) 1006e1012
remain. The number of included articles might be smaller
(or larger) than the number of studies, because articles
may report on multiple studies and results from a particular
study may be published in several articles. To capture this
information, the PRISMA ﬂow diagram now requests infor-
mation on these phases of the review process.
The PRISMA Statement should replace the QUOROM
Statement for those journals that have endorsed QUOROM.
We hope that other journals will support PRISMA; they
can do so by registering on the PRISMA Web site. To
underscore to authors, and others, the importance of transpar-
ent reporting of systematic reviews, we encourage supporting
journals to reference the PRISMA Statement and include the
PRISMA Web address in their Instructions to Authors. We
also invite editorial organizations to consider endorsing
PRISMA and encourage authors to adhere to its principles.
7. The PRISMA Explanation and Elaboration Paper
In addition to the PRISMA Statement, a supporting
Explanation and Elaboration document has been produced
 following the style used for other reporting guidelines
[19e21]. The process of completing this document
included developing a large database of exemplars to high-
light how best to report each checklist item, and identifying
a comprehensive evidence base to support the inclusion of
each checklist item. The Explanation and Elaboration
document was completed after several face to face meet-
ings and numerous iterations among several meeting partic-
ipants, after which it was shared with the whole group for
additional revisions and ﬁnal approval. Finally, the group
formed a dissemination subcommittee to help disseminate
and implement PRISMA.
The quality of reporting of systematic reviews is still not
optimal [22e27]. In a recent review of 300 systematic
reviews, few authors reported assessing possible publication
bias , even though there is overwhelming evidence both
for its existence  and its impact on the results of system-
atic reviews . Even when the possibility of publication
bias is assessed, there is no guarantee that systematic re-
viewers have assessed or interpreted it appropriately . Al-
though the absence of reporting such an assessment does not
necessarily indicate that it was not done, reporting an assess-
ment of possible publication bias is likely to be a marker of
the thoroughness of the conduct of the systematic review.
Several approaches have been developed to conduct sys-
tematic reviews on a broader array of questions. For exam-
ple, systematic reviews are now conducted to investigate
cost-effectiveness , diagnostic  or prognostic ques-
tions , genetic associations , and policy making
. The general concepts and topics covered by PRISMA
are all relevant to any systematic review, not just those
whose objective is to summarize the beneﬁts and harms
Substantive speciﬁc changes between the QUOROM checklist and the PRISMA checklist (a tick indicates the presence of the topic in QUOROM or
Section/Topic Item QUOROM PRISMA Comment
Abstract OOQUOROM and PRISMA ask authors to report an abstract. However,
PRISMA is not speciﬁc about format.
Introduction Objective OThis new item (4) addresses the explicit question the review
addresses using the PICO reporting system (which describes the participants,
interventions, comparisons, and outcome(s) of the systematic review),
together with the speciﬁcation of the type of study design (PICOS); the item
is linked to Items 6, 11, and 18 of the checklist.
Methods Protocol OThis new item (5) asks authors to report whether the review has
a protocol and if so how it can be accessed.
Methods Search OOAlthough reporting the search is present in both QUOROM and
PRISMA checklists, PRISMA asks authors to provide a full description
of at least one electronic search strategy (Item 8). Without such information
it is impossible to repeat the authors’ search.
Methods Assessment of risk of
bias in included studies
OORenamed from ‘‘quality assessment’’ in QUOROM. This item (12)
is linked with reporting this information in the results (Item 19).
The new concept of ‘‘outcome-level’’ assessment has been introduced.
Methods Assessment of risk
of bias across studies
OThis new item (15) asks authors to describe any assessments of
risk of bias in the review, such as selective reporting within the included studies.
This item is linked with reporting this information in the results (Item 22).
Discussion OOAlthough both QUOROM and PRISMA checklists address the discussion
section, PRISMA devotes three items (24e26) to the discussion. In PRISMA
the main types of limitations are explicitly stated and their discussion required.
Funding OThis new item (27) asks authors to provide information on any sources of funding
for the systematic review.
1010 D. Moher et al. / Journal of Clinical Epidemiology 62 (2009) 1006e1012
of a health care intervention. However, some modiﬁcations
of the checklist items or ﬂow diagram will be necessary in
particular circumstances. For example, assessing the risk of
bias is a key concept, but the items used to assess this in
a diagnostic review are likely to focus on issues such as
the spectrum of patients and the veriﬁcation of disease
status, which differ from reviews of interventions. The ﬂow
diagram will also need adjustments when reporting
individual patient data meta-analysis .
We have developed an explanatory document  to
increase the usefulness of PRISMA. For each checklist
item, this document contains an example of good reporting,
a rationale for its inclusion, and supporting evidence,
including references, whenever possible. We believe this
document will also serve as a useful resource for those
teaching systematic review methodology. We encourage
journals to include reference to the explanatory document
in their Instructions to Authors.
Like any evidence-based endeavor, PRISMA is a living
document. To this end we invite readers to comment on the
revised version, particularly the new checklist and ﬂow
diagram, through the PRISMA Web site. We will use such
information to inform PRISMA’s continued development.
The following people contributed to the PRISMA State-
ment: Doug Altman, DSc, Centre for Statistics in Medicine
(Oxford, UK); Gerd Antes, PhD, University Hospital Frei-
burg (Freiburg, Germany); David Atkins, MD, MPH,
Health Services Research and Development Service, Vet-
erans Health Administration (Washington, D. C., US); Vir-
ginia Barbour, MRCP, DPhil, PLoS Medicine (Cambridge,
UK); Nick Barrowman, PhD, Children’s Hospital of East-
ern Ontario (Ottawa, Canada); Jesse A. Berlin, ScD, John-
son & Johnson Pharmaceutical Research and Development
(Titusville, New Jersey, US); Jocalyn Clark, PhD, PLoS
Medicine (at the time of writing, BMJ, London, UK); Mike
Clarke, PhD, UK Cochrane Centre (Oxford, UK) and
School of Nursing and Midwifery, Trinity College (Dublin,
Ireland); Deborah Cook, MD, Departments of Medicine,
Clinical Epidemiology and Biostatistics, McMaster Univer-
sity (Hamilton, Canada); Roberto D’Amico, PhD, Universi-
`di Modena e Reggio Emilia (Modena, Italy) and Centro
Cochrane Italiano, Istituto Ricerche Farmacologiche Mario
Negri (Milan, Italy); Jonathan J. Deeks, PhD, University of
Birmingham (Birmingham, UK); P. J. Devereaux, MD,
PhD, Departments of Medicine, Clinical Epidemiology
and Biostatistics, McMaster University (Hamilton, Cana-
da); Kay Dickersin, PhD, Johns Hopkins Bloomberg
School of Public Health (Baltimore, Maryland, US); Mat-
thias Egger, MD, Department of Social and Preventive
Medicine, University of Bern (Bern, Switzerland); Edzard
Ernst, MD, PhD, FRCP, FRCP(Edin), Peninsula Medical
School (Exeter, UK); Peter C. Gøtzsche, MD, MSc, The
Nordic Cochrane Centre (Copenhagen, Denmark); Jeremy
Grimshaw, MBChB, PhD, FRCFP, Ottawa Hospital Re-
search Institute (Ottawa, Canada); Gordon Guyatt, MD,
Departments of Medicine, Clinical Epidemiology and Bio-
statistics, McMaster University (Hamilton, Canada); Julian
Higgins, PhD, MRC Biostatistics Unit (Cambridge, UK);
John P. A. Ioannidis, MD, University of Ioannina Campus
(Ioannina, Greece); Jos Kleijnen, MD, PhD, Kleijnen Sys-
tematic Reviews Ltd (York, UK) and School for Public
Health and Primary Care (CAPHRI), University of Maas-
tricht (Maastricht, Netherlands); Tom Lang, MA, Tom
Lang Communications and Training (Davis, California,
US); Alessandro Liberati, MD, Universita
`di Modena e Re-
ggio Emilia (Modena, Italy) and Centro Cochrane Italiano,
Istituto Ricerche Farmacologiche Mario Negri (Milan,
Italy); Nicola Magrini, MD, NHS Centre for the Evaluation
of the Effectiveness of Health Care eCeVEAS (Modena,
Italy); David McNamee, PhD, The Lancet (London, UK);
Lorenzo Moja, MD, MSc, Centro Cochrane Italiano, Istitu-
to Ricerche Farmacologiche Mario Negri (Milan, Italy);
David Moher, PhD, Ottawa Methods Centre, Ottawa Hospi-
tal Research Institute (Ottawa, Canada); Cynthia Mulrow,
MD, MSc, Annals of Internal Medicine (Philadelphia,
Pennsylvania, US); Maryann Napoli, Center for Medical
Consumers (New York, New York, US); Andy Oxman,
MD, Norwegian Health Services Research Centre (Oslo,
Norway); Ba’ Pham, MMath, Toronto Health Economics
and Technology Assessment Collaborative (Toronto, Cana-
da) (at the time of the ﬁrst meeting of the group,
GlaxoSmithKline Canada, Mississauga, Canada); Drum-
mond Rennie, MD, FRCP, FACP, University of California
San Francisco (San Francisco, California, US); Margaret
Sampson, MLIS, Children’s Hospital of Eastern Ontario
(Ottawa, Canada); Kenneth F. Schulz, PhD, MBA, Family
Health International (Durham, North Carolina, US); Paul
G. Shekelle, MD, PhD, Southern California Evidence
Based Practice Center (Santa Monica, California, US); Jen-
nifer Tetzlaff, BSc, Ottawa Methods Centre, Ottawa Hospital
Research Institute (Ottawa, Canada); David Tovey, FRCGP,
The Cochrane Library, Cochrane Collaboration (Oxford,
UK) (at the time of the ﬁrst meeting of the group, BMJ,Lon-
don, UK); Peter Tugwell, MD, MSc, FRCPC, Institute of
Population Health, University of Ottawa (Ottawa, Canada).
Figure S1 Flow of information through the different
phases of a systematic review (downloadable template doc-
ument for researchers to re-use).
Text S1 Checklist of items to include when reporting
a systematic review or meta-analysis (downloadable tem-
plate document for researchers to re-use).
Supporting data associated with this article can be
found, in the online version, at doi: 10.1016/
1011D. Moher et al. / Journal of Clinical Epidemiology 62 (2009) 1006e1012
 Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical lit-
erature. VI. How to use an overview. Evidence-Based Medicine
Working Group. JAMA 1994;272:1367e71.
 Swingler GH, Volmink J, Ioannidis JP. Number of published system-
atic reviews and global burden of disease: Database analysis. BMJ
 Canadian Institutes of Health Research (2006) Randomized con-
trolled trials registration/application checklist (12/2006). Available:
http://www.cihr-irsc.gc.ca/e/documents/rct_reg_e.pdf. Accessed 19
 Young C, Horton R. Putting clinical trials into context. Lancet
 Mulrow CD. The medical review article: State of the science. Ann
Intern Med 1987;106:485e8.
 Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC.
Meta-analysis of randomized controlled trials. New Engl J Med
 Sacks HS, Reitman D, Pagano D, Kupelnick B. Meta-analysis: An
update. Mt Sinai J Med 1996;63:216e24.
 Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, et al. for the
QUOROM group. Improving the quality of reporting of meta-analy-
sis of randomized controlled trials: The QUOROM statement. Lancet
 Green S, Higgins J, editors (2005) Glossary. Cochrane handbook for
systematic reviews of interventions 4.2.5. The Cochrane Collabora-
tion. Available: http://www.cochrane.org/resources/glossary.htm. Ac-
cessed 19 May 2009.
 Strech D, Tilburt J. Value judgments in the analysis and synthesis of
evidence. J Clin Epidemiol 2008;61:521e4.
 Moher D, Tsertsvadze A. Systematic reviews: When is an update an
update? Lancet 2006;367:881e3.
 University of York (2009) Centre for Reviews and Dissemination.
Available: http://www.york.ac.uk/inst/crd/. Accessed 19 May 2009.
 The Joanna Briggs Institute (2008) Protocols & work in progress.
prot.php. Accessed 19 May 2009.
 De Angelis C, Drazan JM, Frizelle FA, Haug C, Hoey J, et al. for the
International Committee Medical Journal Editors. Clinical trial regis-
tration: A statement from the International Committee of Medical
Journal Editors. CMAJ 2004;171:606e7.
 Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, et al. Selec-
tive serotonin reuptake inhibitors in childhood depression: Systematic re-
view of published versus unpublished data. Lancet 2004;363:1341e5.
 Bagshaw SM, McAlister FA, Manns BJ, Ghali WA. Acetylcysteine in
the prevention of contrast-induced nephropathy:A case study of the pit-
falls in the evolution of evidence. Arch Intern Med 2006;166:161e6.
 Biondi-Zoccai GG, Lotrionte M, Abbate A, Testa L, Remigi E, et al.
Compliance with QUOROM and quality of reporting of overlapping
meta-analyses on the role of acetylcysteine in the prevention of con-
trast associated nephropathy: Case study. BMJ 2006;332:202e9.
 Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche P, et al. the
PRISMA Group. The PRISMA statement for reporting systematic re-
views and meta-analyses of studies that evaluate health care interven-
tions: Explanation and elaboration. PLoS Med 2009;6:e1000100. doi:
 Altman DG, Schulz KR, Moher D, Egger M, Davidoff F, et al. for the
CONSORT group. The revised CONSORT statement for reporting
randomized trials: Explanation and elaboration. Ann Intern Med
 Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, et al.
for the STARD group. Towards complete and accurate reporting of
studies of diagnostic accuracy: The STARD explanation and elabora-
tion. Ann Intern Med 2003;138:W1eW12.
 Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC,
Mulrow CD, et al. for the STROBE initiative. Strengthening the
Reporting of Observational Studies in Epidemiology (STROBE): Ex-
planation and elaboration. Ann Intern Med 2007;147:W163e94.
 Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemi-
ology and reporting characteristics of systematic reviews. PLoS Med
 Bhandari M, Morrow F, Kulkarni AV, Tornetta P. Meta-analyses in
orthopaedic surgery: A systematic review of their methodologies.
JBone Joint Surg Am 83-A 2001;15e24.
 Kelly KD, Travers A, Dorgan M, Slater L, Rowe BH. Evaluating the
quality of systematic reviews in the emergency medicine literature.
Ann Emerg Med 2001;38:518e26.
 Richards D. The quality of systematic reviews in dentistry. Evid
Based Dent 2004;5:17.
 Choi PT, Halpern SH, Malik N, Jadad AR, Tramer MR, et al. Exam-
ining the evidence in anesthesia literature: A critical appraisal of sys-
tematic reviews. Anesth Analg 2001;92:700e9.
 Delaney A, Bagshaw SM, Ferland A, Manns B, Laupland KB.
A systematic evaluation of the quality of meta-analyses in the critical
care literature. Crit Care 2005;9:R575e82.
 Dickersin K. Publication bias: Recognizing the problem, understand-
ing its origins and scope, and preventing harm. In: Rothstein HR,
Sutton AJ, Borenstein M, editors. Publication bias in meta-analy-
sisdPrevention, assessment and adjustments. Chichester (UK): John
Wiley & Sons; 2005. p. 11e33.
 Sutton AJ. Evidence concerning the consequences of publication and
related biases. In: Rothstein HR, Sutton AJ, Borenstein M, editors.
Publication bias in meta-analysisdPrevention, assessment and ad-
justments. Chichester (UK): John Wiley & Sons; 2005. p. 175e92.
 Lau J, Ioannidis JP, Terrin N, Schmid CH, Olkin I. The case of the
misleading funnel plot. BMJ 2006;333:597e600.
 Ladabaum U, Chopra CL, Huang G, Scheiman JM, Chernew ME, et al.
Aspirin as an adjunct to screening for prevention of sporadic colorectal
cancer: A cost-effectiveness analysis. Ann Intern Med 2001;135:769 e81.
 Deeks JJ. Systematic reviews in health care: Systematic reviews of
evaluations of diagnostic and screening tests. BMJ 2001;323:157e62.
 Altman DG. Systematic reviews of evaluations of prognostic vari-
ables. BMJ 2001;323:224e8.
 Ioannidis JP, Ntzani EE, Trikalinos TA, Contopoulos-Ioannidis DG. Rep-
lication validity of genetic association studies. Nat Genet 2001;29:306e9.
 Lavis J, Davies H, Oxman A, Denis J, Golden-Biddle K, et al. Towards
systematic reviews that inform health care management and policy-
making. J Health Serv Res Policy 2005;10:35e48.
 Stewart LA, Clarke MJ. Practical methodology of meta-analyses
(overviews) using updated individual patient data. Cochrane Working
Group. Stat Med 1995;14:2057e79.
 Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, et al. Assessment of
methodological quality of primary studies by systematic reviews: Re-
sults of the metaquality cross sectional study. BMJ 2005;330:1053e5.
 Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, et al. for the
GRADE Working Group. GRADE: An emerging consensus on rating
quality of evidence and strength of recommendations. BMJ 2008;336:
 Schunemann HJ, Jaeschke R, Cook DJ, Bria WF, El-Solh AA, et al.
for the ATS Documents Development and Implementation Commit-
tee. An ofﬁcial ATS statement: Grading the quality of evidence and
strength of recommendations in ATS guidelines and recommenda-
tions. Am J Respir Crit Care Med 2006;174:605e14.
 Chan AW, Hrobjartsson A, Haahr MT, Gøtzsche PC, Altman DG.
Empirical evidence for selective reporting of outcomes in random-
ized trials: Comparison of protocols to published articles. JAMA
 Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting
bias in randomized trials funded by the Canadian Institutes of Health
Research. CMAJ 2004;171:735e40.
 Silagy CA, Middleton P, Hopewell S. Publishing protocols of system-
atic reviews: Comparing what was done to what was planned. JAMA
1012 D. Moher et al. / Journal of Clinical Epidemiology 62 (2009) 1006e1012