Identifying studies for systematic reviews of diagnostic tests was difficult due to the poor sensitivity and precision of methodologic filters and the lack of information in the abstract

University of Queensland, School of Medicine, Center for General Practice, Herston Road, Herston, Brisbane, Qld 4006, Australia.
Journal of Clinical Epidemiology (Impact Factor: 3.42). 06/2005; 58(5):444-9. DOI: 10.1016/j.jclinepi.2004.09.011
Source: PubMed


Methods to identify studies for systematic reviews of diagnostic accuracy are less well developed than for reviews of intervention studies. This study assessed (1) the sensitivity and precision of five published search strategies and (2) the reliability and accuracy of reviewers screening the results of the search strategy.
We compared the results of the search filters with the studies included in two systematic reviews, and assessed the interobserver reliability of two reviewers screening the list of articles generated by a search strategy.
In the first review, the search strategy published by van der Weijden had the greatest sensitivity, and in the second, four search strategies had 100% sensitivity. There was "substantial" agreement between two reviewers, but in the first review each reviewer working on their own would have missed one paper eligible for inclusion in the review. Ascertainment intersection techniques indicate that it is unlikely that further papers have been missed in the screening process.
Published search strategies may miss papers for reviews of diagnostic test accuracy. Papers are not easily identified as studies of diagnostic test accuracy, and the lack of information in the abstract makes it difficult to assess the eligibility for inclusion in a systematic review.

1 Read
  • Source
    • "For example, the included studies in a systematic review represent the total number of relevant documents retrieved by various sources working as whole, which refers to the various databases and other sources of information searched when conducting a systematic review. The relative recall method has been used by other researchers in the context of developing and validating various search strategies [11] [12] [14]. Although this method may exaggerate recall or sensitivity if the searches for the systematic reviews miss some studies, it has many benefits. "
    [Show abstract] [Hide abstract]
    ABSTRACT: To develop, assess, and maximize the sensitivity of a search strategy to identify diet and nutrition trials in PubMed using relative recall. We developed a search strategy to identify diet and nutrition trials in PubMed. We then constructed a gold standard reference set to validate the identified trials using the relative recall method. Relative recall was calculated by dividing the number of references from the gold standard our search strategy identified by the total number of references in the gold standard. Our gold standard comprised 298 trials, derived from 16 included systematic reviews. The initial search strategy identified 242 of 298 references, with a relative recall of 81.2% [95% confidence interval (CI): 76.3%, 85.5%]. We analyzed titles and abstracts of the 56 missed references for possible additional terms. We then modified the search strategy accordingly. The relative recall of the final search strategy was 88.6% (95% CI: 84.4%, 91.9%). We developed a search strategy to identify diet and nutrition trials in PubMed with a high relative recall (sensitivity). This could be useful for establishing a nutrition trials register to support the conduct of future research, including systematic reviews. Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.
    Full-text · Article · Feb 2015 · Journal of Clinical Epidemiology
  • Source
    • "A systematic search was performed from inception until 14 April 2012 in MEDLINE (Ovid), EMBASE (Ovid), DARE [6], and Medion [7]. No diagnostic search filter [8,9] or language restrictions were used. Details of the search are given in Additional file 1. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The prognostic accuracy of 1H (proton) magnetic resonance spectroscopy (MRS) in neonatal hypoxic-ischemic encephalopathy has been assessed by a criticized study-based meta-analysis. An individual patient data meta-analysis may overcome some of the drawbacks encountered in the aggregate data meta-analysis. Moreover, the prognostic marker can be assessed quantitatively and the effect of covariates can be estimated. Diagnostic accuracy studies relevant to the study topic were retrieved. The primary authors will be invited to share the raw de-identified study data. These individual patient data will be analyzed using logistic regression analysis. A prediction tool calculating the individualized risk of very adverse outcome will be devised. The proposed individual patient data meta-analysis provides several advantages. Inclusion and exclusion criteria can be applied more uniformly. Furthermore, adjustment is possible for confounding factors and subgroup analyses can be conducted. Our goal is to develop a prediction model for outcome in newborns with hypoxic-ischemic encephalopathy.
    Full-text · Article · Oct 2013 · Systematic Reviews
  • Source
    • "Google Scholar was also examined for any grey literature that was not represented within the database. Because computerized search results for sensitivity and specificity frequently omit many relevant articles [16] "
    [Show abstract] [Hide abstract]
    ABSTRACT: Diagnosis of patellofemoral pain syndrome (PFPS) is commonly performed using a myriad of clinical and imaging-based criteria. The objective of this systematic literature review was to summarize the research on accuracy of individual clinical tests/findings for PFPS. MEDLINE, ProQuest Nursing and Allied Health, Cochrane Trials, PEDro, and CINAHL. STUDY SELECTION OR ELIGIBILITY CRITERIA: PRISMA guidelines were followed for this review. To be considered for review, the study required: (1) a description of a clinical test or tests used for diagnosing PFPS (including a test that was combined with another finding such as patient history), (2) a report of the diagnostic accuracy of the measures (e.g., sensitivity and specificity), and (3) an acceptable reference standard for comparison. STUDY APPRAISAL OR SYNTHESIS METHODS: Quality Assessment of Studies of Diagnostic Accuracy (QUADAS) scores were completed on each selected article. Sensitivity, specificity, and negative and positive likelihood ratios (LR-/LR+) were calculated for each diagnostic test described. The systematic search strategy and hand search revealed 704 potential articles, 9 of which met the criteria for this review; analysing a total of 22 PFPS clinical tests. After assessment using the QUADAS score, 1 of the 9 articles was of high quality. The tests with the highest reported diagnostic value were also associated with studies that had the lowest QUADAS values. A majority of the studies that have investigated diagnostic accuracy of clinical tests for PFPS demonstrate notable design or reporting biases, and at this stage, determining the best tests for diagnosis of PFPS is still difficult.
    Full-text · Article · Jun 2012 · Physiotherapy
Show more