Systematic reviews need systematic searches

Ottawa Health Research Institute/Institute of Population Health University of Ottawa Ottawa K1N 6N5 Canada.
Journal of the Medical Library Association JMLA (Impact Factor: 0.99). 02/2005; 93(1):74-80.
Source: PubMed


This paper will provide a description of the methods, skills, and knowledge of expert searchers working on systematic review teams.
Systematic reviews and meta-analyses are very important to health care practitioners, who need to keep abreast of the medical literature and make informed decisions. Searching is a critical part of conducting these systematic reviews, as errors made in the search process potentially result in a biased or otherwise incomplete evidence base for the review. Searches for systematic reviews need to be constructed to maximize recall and deal effectively with a number of potentially biasing factors. Librarians who conduct the searches for systematic reviews must be experts.
Expert searchers need to understand the specifics about data structure and functions of bibliographic and specialized databases, as well as the technical and methodological issues of searching. Search methodology must be based on research about retrieval practices, and it is vital that expert searchers keep informed about, advocate for, and, moreover, conduct research in information retrieval. Expert searchers are an important part of the systematic review team, crucial throughout the review process-from the development of the proposal and research question to publication.

Download full-text


Available from: Jessie Mcgowan, Sep 30, 2015
23 Reads
  • Source
    • "Successful search strategy design involves knowledge of databases, indexing and database structures. Hence, successful search strategies typically involve experienced information specialists (McGowan and Sampson, 2005). Many academic libraries (such as those affiliated with veterinary colleges) have librarians with specialized training in systematic review search strategies. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This article is the fourth of six articles addressing systematic reviews in animal agriculture and veterinary medicine. Previous articles in the series have introduced systematic reviews, discussed study designs and hierarchies of evidence, and provided details on conducting randomized controlled trials, a common design for use in systematic reviews. This article describes development of a review protocol and the first two steps in a systematic review: formulating a review question, and searching the literature for relevant research. The emphasis is on systematic reviews of questions related to interventions. The review protocol is developed prior to conducting the review and specifies the plan for the conduct of the review, identifies the roles and responsibilities of the review team and provides structured definitions related to the review question. For intervention questions, the review question should be defined by the PICO components: population, intervention, comparison and outcome(s). The literature search is designed to identify all potentially relevant original research that may address the question. Search terms related to some or all of the PICO components are entered into literature databases, and searches for unpublished literature also are conducted. All steps of the literature search are documented to provide transparent reporting of the process.
    Zoonoses and Public Health 06/2014; 61 Suppl S1:28-38. DOI:10.1111/zph.12125 · 2.37 Impact Factor
  • Source
    • "In addition, the type of searching that is required to support systematic reviews (SRs) is more complex and time-consuming than searching for simple clinical queries. The demands placed on a searcher for the SR are much higher than for other searches because of the specific requirements of the SR [7] which is integral to science and must therefore be performed systematically, and made repeatable, verifiable and accountable [8,9]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The usefulness of Google Scholar (GS) as a bibliographic database for biomedical systematic review (SR) searching is a subject of current interest and debate in research circles. Recent research has suggested GS might even be used alone in SR searching. This assertion is challenged here by testing whether GS can locate all studies included in 21 previously published SRs. Second, it examines the recall of GS, taking into account the maximum number of items that can be viewed, and tests whether more complete searches created by an information specialist will improve recall compared to the searches used in the 21 published SRs. The authors identified 21 biomedical SRs that had used GS and PubMed as information sources and reported their use of identical, reproducible search strategies in both databases. These search strategies were rerun in GS and PubMed, and analyzed as to their coverage and recall. Efforts were made to improve searches that underperformed in each database. GS' overall coverage was higher than PubMed (98% versus 91%) and overall recall is higher in GS: 80% of the references included in the 21 SRs were returned by the original searches in GS versus 68% in PubMed. Only 72% of the included references could be used as they were listed among the first 1,000 hits (the maximum number shown). Practical precision (the number of included references retrieved in the first 1,000, divided by 1,000) was on average 1.9%, which is only slightly lower than in other published SRs. Improving searches with the lowest recall resulted in an increase in recall from 48% to 66% in GS and, in PubMed, from 60% to 85% . Although its coverage and precision are acceptable, GS, because of its incomplete recall, should not be used as a single source in SR searching. A specialized, curated medical database such as PubMed provides experienced searchers with tools and functionality that help improve recall, and numerous options in order to optimize precision. Searches for SRs should be performed by experienced searchers creating searches that maximize recall for as many databases as deemed necessary by the search expert.
    Systematic Reviews 12/2013; 2(1):115. DOI:10.1186/2046-4053-2-115
  • Source
    • "We find McGowans et al. statement “Systematic reviews need systematic researchers” [49] transferable from systematic reviews to any type of scientific work. Hence, the inappropriateness of Google Scholar to support any systematic and structured literature retrieval process is in the core of our critique. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Recent research indicates a high recall in Google Scholar searches for systematic reviews. These reports raised high expectations of Google Scholar as a unified and easy to use search interface. However, studies on the coverage of Google Scholar rarely used the search interface in a realistic approach but instead merely checked for the existence of gold standard references. In addition, the severe limitations of the Google Search interface must be taken into consideration when comparing with professional literature retrieval tools.The objectives of this work are to measure the relative recall and precision of searches with Google Scholar under conditions which are derived from structured search procedures conventional in scientific literature retrieval; and to provide an overview of current advantages and disadvantages of the Google Scholar search interface in scientific literature retrieval. General and MEDLINE-specific search strategies were retrieved from 14 Cochrane systematic reviews. Cochrane systematic review search strategies were translated to Google Scholar search expression as good as possible under consideration of the original search semantics. The references of the included studies from the Cochrane reviews were checked for their inclusion in the result sets of the Google Scholar searches. Relative recall and precision were calculated. We investigated Cochrane reviews with a number of included references between 11 and 70 with a total of 396 references. The Google Scholar searches resulted in sets between 4,320 and 67,800 and a total of 291,190 hits. The relative recall of the Google Scholar searches had a minimum of 76.2% and a maximum of 100% (7 searches). The precision of the Google Scholar searches had a minimum of 0.05% and a maximum of 0.92%. The overall relative recall for all searches was 92.9%, the overall precision was 0.13%. The reported relative recall must be interpreted with care. It is a quality indicator of Google Scholar confined to an experimental setting which is unavailable in systematic retrieval due to the severe limitations of the Google Scholar search interface. Currently, Google Scholar does not provide necessary elements for systematic scientific literature retrieval such as tools for incremental query optimization, export of a large number of references, a visual search builder or a history function. Google Scholar is not ready as a professional searching tool for tasks where structured retrieval methodology is necessary.
    BMC Medical Research Methodology 10/2013; 13(1):131. DOI:10.1186/1471-2288-13-131 · 2.27 Impact Factor
Show more