Benchmarking of reported search and selection methods of systematic reviews by dental specialty

Goldman School of Dental Medicine, Boston University, Boston, USA.
Evidence-Based Dentistry 02/2007; 8(3):66-70. DOI: 10.1038/sj.ebd.6400504
Source: PubMed


Previous evaluations showed that literature-search and selection methods reported in dental systematic reviews (SR) have improved since 2000. It is not known, however, whether these differences are consistent between the SR of the different dental specialities.
SR in dentistry published in the English language between 1 January 2000 and 14 June 2006 were located and then categorised by American Dental Association (ADA) recognised specialities. Search and selection methods were evaluated following an adaptation of the Cochrane Handbook for Systematic Reviews. Search and selection methods between dental specialities were compared using Pearson's chi-squared analysis and ranked.
There were significant differences between specialities in the following criteria: documentation of search dates (P 0.003); inclusion-exclusion documentation (P 0.017); article selection by two or more reviewers (P 0.001); and inclusion of all languages (P 0.014). Periodontics SR met the most criteria followed by oral and maxillofacial surgery (OMS) and then dental public heath (DPH). Prosthodontics along with the area of oral and maxillofacial radiology (OMR) met the fewest. All dental SR had low compliance with four criteria, only 65.7% searched more than Medline, 50.4% had a search strategy documented with Boolean operators, 51.5% had article selection carried out by two or more reviewers and only 25.7% included all languages.
Some dental specialities are better at reporting search and selection methods than others but all dental SR need some improvement in their reporting.

Download full-text


Available from: Michael Major
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The Cochrane Handbook for Systematic Reviews of Interventions provides instructions for documenting a systematic review's electronic database search strategy, listing elements that should be in the description. Complete documentation of the search strategy allows readers to evaluate the search when critically appraising a review's quality. The research analyzed recently published Cochrane reviews to determine whether instructions for describing electronic database search strategies were being followed. Eighty-three new reviews added to the Cochrane Database of Systematic Reviews in the first quarter of 2006 were selected for analysis. Eighteen were subsequently excluded because their searches were conducted only in the specialized registers of Cochrane review groups. The remaining sixty-five reviews were analyzed for the seven elements of an electronic database search strategy description listed in the Cochrane Handbook, using dual review with consensus. Of the 65 reviews analyzed, none included all 7 recommended elements. Four reviews (6%) included 6 elements. Thirty-two percent (21/65) included 5 or more elements, with 68% (44/65) including 4 or fewer. Three included only 2 elements. The 65 reviews represented 41 different Cochrane review groups. The instructions from the Cochrane Handbook for reporting search strategies are not being consistently employed by groups producing Cochrane reviews.
    Full-text · Article · Feb 2009 · Journal of the Medical Library Association JMLA
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Systematic reviews and meta-analyses are essential to summarise evidence relating to efficacy and safety of healthcare interventions accurately and reliably. The clarity and transparency of these reports, however, are not optimal. Poor reporting of systematic reviews diminishes their value to clinicians, policy makers, and other users. Since the development of the QUOROM (quality of reporting of meta-analysis) statement—a reporting guideline published in 1999—there have been several conceptual, methodological, and practical advances regarding the conduct and reporting of systematic reviews and meta-analyses. Also, reviews of published systematic reviews have found that key information about these studies is often poorly reported. Realising these issues, an international group that included experienced authors and methodologists developed PRISMA (preferred reporting items for systematic reviews and meta-analyses) as an evolution of the original QUOROM guideline for systematic reviews and meta-analyses of evaluations of health care interventions. The PRISMA statement consists of a 27-item checklist and a four-phase flow diagram. The checklist includes items deemed essential for transparent reporting of a systematic review. In this explanation and elaboration document, we explain the meaning and rationale for each checklist item. For each item, we include an example of good reporting and, where possible, references to relevant empirical studies and methodological literature. The PRISMA statement, this document, and the associated website ( should be helpful resources to improve reporting of systematic reviews and meta-analyses.
    Full-text · Article · Feb 2009 · BMJ (online)
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The study reported in this article had three objectives: 1) identify the challenges faced by authors of dental systematic reviews (SR) during the process of literature search and selection; 2) determine whether dental SR authors' responses to survey questions about their study methodology were consistent with the reported published methodology; and 3) assess whether dental SR authors' evidence-based publication experience was associated with reported methodology. Seventy-eight authors (53 percent) of dental SRs out of 147 potential authors published from 2000 to 2006 responded to an online survey. According to the respondents, the most challenging aspects of literature search and selection were the initial design and performing extended literature searches. Agreement between the protocol identified by SR authors on the survey and the actual protocol described in their publications was fair to moderate. There were virtually no correlations between authors' publication experience, systematic review literature search, and selection thoroughness except for the number of past SRs published, and no differences in thoroughness between SRs written by clinicians (dental practitioners in the community) and dental school faculty members. Dental SR authors do not appear to fully appreciate the importance of extensive literature searches as central to the validity of their systematic review methods and potential findings.
    Full-text · Article · May 2009 · Journal of dental education
Show more