Guidelines for Guidelines: Are They Up to the Task? A Comparative Assessment of Clinical Practice Guideline Development Handbooks

Old Dominion University, United States of America
PLoS ONE (Impact Factor: 3.23). 11/2012; 7(11):e49864. DOI: 10.1371/journal.pone.0049864
Source: PubMed

ABSTRACT We conducted a comparative review of clinical practice guideline development handbooks. We aimed to identify the main guideline development tasks, assign weights to the importance of each task using expert opinions and identify the handbooks that provided a comprehensive coverage of the tasks.
We systematically searched and included handbooks published (in English language) by national, international or professional bodies responsible for evidenced-based guideline development. We reviewed the handbooks to identify the main guideline development tasks and scored each handbook for each task from 0 (the handbook did not mention the task) to 2 (the task suitably addressed and explained), and calculated a weighted score for each handbook. The tasks included in over 75% of the handbooks were considered as 'necessary' tasks.
Nineteen guideline development handbooks and twenty seven main tasks were identified. The guideline handbooks' weighted scores ranged from 100 to 220. Four handbooks scored over 80% of the maximum possible score, developed by the National Institute for Health and Clinical Excellence, Swiss Centre for International Health, Scottish Intercollegiate Guidelines Network and World Health Organization. Necessary tasks were: selecting the guideline topic, determining the guideline scope, identifying relevant existing guidelines, involving the consumers, forming guideline development group,, developing clinical questions, systematic search for evidence, selecting relevant evidence, appraising identifies research evidence, making group decision, grading available evidence, creating recommendations, final stakeholder consultation, guideline implementation strategies, updating recommendations and correcting potential errors.
Adequate details for evidence based development of guidelines were still lacking from many handbooks. The tasks relevant to ethical issues and piloting were missing in most handbooks. The findings help decision makers in identifying the necessary tasks for guideline development, provide an updated comparative list of guideline development handbooks, and provide a checklist to assess the comprehensiveness of guideline development processes.

Download full-text


Available from: Arash Rashidian, Oct 20, 2014
  • Source
    • "In 2010, AGREE II was translated into Korean and distributed to Korean medical societies. The usefulness of AGREE II have been verified through various quality assessment studies of specific diseases (8, 9, 10, 11), international comparison of the level of guideline development (12, 13), and overall quality assessment of CPGs developed in specific countries (14, 15, 16, 17). "
    [Show abstract] [Hide abstract]
    ABSTRACT: The Korean translated Appraisal of Guidelines for Research and Evaluation II (Korean AGREE II) instrument was distributed into Korean medical societies in 2011. However, inter-rater disagreement issues still exist. The Korean AGREE II scoring guide was therefore developed to reduce inter-rater differences. This study examines the effects of the Korean AGREE II scoring guide to reduce inter-rater differences. Appraisers were randomly assigned to two groups (Scoring Guide group and Non-Scoring Guide group). The Korean AGREE II instrument was provided to both groups. However, the scoring guide was offered to Scoring Guide group only. Total 14 appraisers were participated and each guideline was assessed by 8 appraisers. To evaluate the reliability of the Korean AGREE II scoring guide, correlation of scores among appraisers and domain-specific intra-class correlation (ICC) were compared. Most scores of two groups were comparable. Scoring Guide group showed higher reliability at all guidelines. They showed higher correlation among appraisers and higher ICC values at almost all domains. The scoring guide reduces the inter-rater disagreement and improves the overall reliability of the Korean-AGREE II instrument. Graphical Abstract
    Journal of Korean Medical Science 06/2014; 29(6):771-5. DOI:10.3346/jkms.2014.29.6.771 · 1.25 Impact Factor
  • Source
    • "In the last decade, guideline organizations faced new problems [1]. First, both guideline developers and implementation experts reported a lack of methodological support in producing high-quality guidelines and felt a need for specific tools to help them in improving the quality of the guidelines [3]. A second problem is that the development and updating of high-quality guidelines require substantial resources, while government is cutting back spending on guideline development. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In the last decade, guideline organizations faced a number of problems, including a lack of standardization in guideline development methods and suboptimal guideline implementation. To contribute to the solution of these problems, we produced a toolbox for guideline development, implementation, revision, and evaluation. All relevant guideline organizations in the Netherlands were approached to prioritize the topics. We sent out a questionnaire and discussed the results at an invitational conference. Based on consensus, twelve topics were selected for the development of new tools. Subsequently, working groups were composed for the development of the tools. After development of the tools, their draft versions were pilot tested in 40 guideline projects. Based on the results of the pilot tests, the tools were refined and their final versions were presented. The vast majority of organizations involved in pilot testing of the tools reported satisfaction with using the tools. Guideline experts involved in pilot testing of the tools proposed a variety of suggestions for the implementation of the tools. The tools are available in Dutch and in English at a web-based platform on guideline development and implementation ( A collaborative approach was used for the development and evaluation of a toolbox for development, implementation, revision, and evaluation of guidelines. This approach yielded a potentially powerful toolbox for improving the quality and implementation of Dutch clinical guidelines. Collaboration between guideline organizations within this project led to stronger linkages, which is useful for enhancing coordination of guideline development and implementation and preventing duplication of efforts. Use of the toolbox could improve quality standards in the Netherlands, and might facilitate the development of high-quality guidelines in other countries as well.
    Implementation Science 03/2014; 9(1):34. DOI:10.1186/1748-5908-9-34 · 3.47 Impact Factor
  • Source
    • "A lack of compliance with state of the art methodology for developing CPGs has been found, and hence the methodological quality of CPGs remained very similar over the last two decades [17,18]. Little is known about the guidance for updating CPGs included in these handbooks [19,20]. Therefore, we systematically reviewed CPGs methodological handbooks to identify and describe the methodological guidance about updating. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Updating clinical practice guidelines (CPGs) is a crucial process for maintaining the validity of recommendations. Methodological handbooks should provide guidance on both developing and updating CPGs. However, little is known about the updating guidance provided by these handbooks. We conducted a systematic review to identify and describe the updating guidance provided by CPG methodological handbooks and included handbooks that provide updating guidance for CPGs. We searched in the Guidelines International Network library, US National Guidelines Clearinghouse and MEDLINE (PubMed) from 1966 to September 2013. Two authors independently selected the handbooks and extracted the data. We used descriptive statistics to analyze the extracted data and conducted a narrative synthesis. We included 35 handbooks. Most handbooks (97.1%) focus mainly on developing CPGs, including variable degrees of information about updating. Guidance on identifying new evidence and the methodology of assessing the need for an update is described in 11 (31.4%) and eight handbooks (22.8%), respectively. The period of time between two updates is described in 25 handbooks (71.4%), two to three years being the most frequent (40.0%). The majority of handbooks do not provide guidance for the literature search, evidence selection, assessment, synthesis, and external review of the updating process. Guidance for updating CPGs is poorly described in methodological handbooks. This guidance should be more rigorous and explicit. This could lead to a more optimal updating process, and, ultimately to valid trustworthy guidelines.
    Implementation Science 01/2014; 9(1):3. DOI:10.1186/1748-5908-9-3 · 3.47 Impact Factor
Show more