[Show abstract][Hide abstract] ABSTRACT: Continuous escalation in methodological and procedural rigor for evidence-based processes in guideline development is associated with increasing costs and production delays that threaten sustainability. While health research methodologists are appropriately responsible for promoting increasing rigor in guideline development, guideline sponsors are responsible for funding such processes.
This paper acknowledges that other stakeholders in addition to methodologists should be more involved in negotiating trade-offs between methodological procedures and efficiency in guideline production to produce guidelines that are 'good enough' to be trustworthy and affordable under specific circumstances. The argument for reasonable methodological compromise to meet practical circumstances is consistent with current implicit methodological practice. This paper proposes a conceptual tool as a framework to be used by different stakeholders in negotiating, and explicitly reporting, reasonable compromises for trustworthy as well as cost-worthy guidelines. The framework helps fill a transparency gap in how methodological choices in guideline development are made. The principle, 'when good is good enough' can serve as a basis for this approach. The conceptual tool 'Efficiency-Validity Methodological Continuum' acknowledges trade-offs between validity and efficiency in evidence-based guideline development and allows for negotiation, guided by methodologists, of reasonable methodological compromises among stakeholders. Collaboration among guideline stakeholders in the development process is necessary if evidence-based guideline development is to be sustainable.
Full-text · Article · Dec 2015 · Implementation Science
[Show abstract][Hide abstract] ABSTRACT: Background: Health system guidance (HSG) provides recommendations aimed to address health system challenges. However, there is a paucity of methods to direct, appraise, and report HSG. Earlier research identified 30 candidate criteria (concepts) that can be used to evaluate the quality of HSG and guide development and reporting requirements. The objective of this paper was to describe two studies aimed at evaluating the importance of these 30 criteria, design a draft HSG appraisal tool, and test its usability. Methods: This study involved a two-step survey process. In step 1, respondents rated the 30 concepts for appropriateness to, relevance to, and priority for health system decisions and HSG. This led to a draft tool. In step 2, respondents reviewed HSG documents, appraised them using the tool, and answered a series of questions. Descriptive analyses were computed. Results: Fifty participants were invited in step 1, and we had a response rate of 82 %. The mean response rates for each concept within each survey question were universally favorable. There was also an overall agreement about the need for a high-quality tool to systematically direct the development, appraisal, and reporting of HSG. Qualitative feedback and a consensus process by the team led to refinements to some of the concepts and the creation of a beta (draft) version of the HSG tool. In step 2, 35 participants were invited and we had a response rate of 74 %. Exploratory analyses showed that the quality of the HSGs reviewed varied as a function of the HSG item and the specific document assessed. A favorable consensus was reached with participants agreeing that the HSG items were easy to understand and easy to apply. Moreover, the overall agreement was high for the usability of the tool to systematically direct the development (85 %), appraisal (92 %), and reporting (81 %) of HSG. From this process, version 1.0 of the HSG appraisal tool was generated complete with 32 items (and their descriptions) and 4 domains. Conclusions: The final tool, named the Appraisal of Guidelines for Research and Evaluation for Health Systems (AGREE-HS) (version 1), defines expectations of HSG and facilitates informed decisions among policymakers on health system delivery, financial, and governance arrangements.
Preview · Article · Dec 2015 · Implementation Science
[Show abstract][Hide abstract] ABSTRACT: Practice guideline (PG) implementability refers to PG features that promote their use. While there are tools and resources to promote PG implementability, none are based on an evidence-informed and multidisciplinary perspective. Our objectives were to (i) create a comprehensive and evidence-informed model of PG implementability, (ii) seek support for the model from the international PG community, (iii) map existing implementability tools on to the model, (iv) prioritize areas for further investigation, and (v) describe how the model can be used by PG developers, users, and researchers.
A mixed methods approach was used. Using our completed realist review of the literature of seven different disciplines as the foundation, an iterative consensus process was used to create the beta version of the model. This was followed by (i) a survey of international stakeholders (guideline developers and users) to gather feedback and to refine the model, (ii) a content analysis comparing the model to existing PG tools, and (iii) a strategy to prioritize areas of the model for further research by members of the research team.
The Guideline Implementability for Decision Excellence Model (GUIDE-M) is comprised of 3 core tactics, 7 domains, 9 subdomains, 44 attributes, and 40 subattributes and elements. Feedback on the beta version was received from 248 stakeholders from 34 countries. The model was rated as logical, relevant, and appropriate. Seven PG tools were selected and compared to the GUIDE-M: very few tools targeted the Contextualization and Deliberations domain. Also, fewer of the tools addressed PG appraisal than PG development and reporting functions. These findings informed the research priorities identified by the team.
The GUIDE-M provides an evidence-informed international and multidisciplinary conceptualization of PG implementability. The model can be used by PG developers to help them create more implementable recommendations, by clinicians and other users to help them be better consumers of PGs, and by the research community to identify priorities for further investigation.
Full-text · Article · Dec 2015 · Implementation Science
[Show abstract][Hide abstract] ABSTRACT: Breast and prostate cancers are the most commonly diagnosed non-dermatologic malignancies in Canada. Agents including endocrine therapies (e.g., aromatase inhibitors, gonadotrophin-releasing hormone analogs, anti-androgens, tamoxifen) and chemotherapy have improved survival for both conditions. As endocrine manipulation is a mainstay of treatment, it is not surprising that hot flashes are a common and troublesome adverse effect. Hot flashes can cause chills, night sweats, anxiety, and insomnia, lessening patients' quality of life. These symptoms impact treatment adherence, worsening prognosis. While short-term estrogen replacement therapy is frequently used to manage hot flashes in healthy menopausal women, its use is contraindicated in breast cancer. Similarly, testosterone replacement therapy is contraindicated in prostate cancer. It is therefore not surprising that non-hormonal pharmacological treatments (anti-depressants, anti-epilectics, anti-hypertensives), physical/behavioral treatments (e.g., acupuncture, yoga/exercise, relaxation techniques, cognitive behavioral therapy), and natural health products (e.g., black cohosh, flax, vitamin E, ginseng) have been studied for control of hot flashes. There is a need to identify which interventions minimize the frequency and severity of hot flashes and their impact on quality of life. This systematic review and network meta-analysis of randomized studies will synthesize available evidence addressing this knowledge gap.
An electronic search of Medline, Embase, AMED, PsycINFO, and the Cochrane Register of Controlled Trials has been designed by an information specialist and peer reviewed by a second information specialist. Study selection and data collection will be performed by two reviewers independently. Risk of bias assessments will be completed using the Cochrane Risk of Bias Scale. Outcomes of interest will include validated measures of hot flash severity, hot flash frequency, quality of life, and harms. Bayesian network meta-analyses will be performed where judged appropriate based on review of clinical and methodologic features of included studies.
Our review will include a broad range of interventions that patients with breast and prostate cancer have attempted to use to manage hot flashes. Our work will establish the extent of evidence underlying these interventions and will employ an inclusive approach to analysis to inform comparisons between them. Our findings will be shared with Cancer Care Ontario for consideration in the development of guidance related to supportive care in these patients.
Full-text · Article · Dec 2015 · Systematic Reviews
[Show abstract][Hide abstract] ABSTRACT: Health care professionals (HCPs) are able to make effective decisions regarding patient care through the use of systematically developed clinical practice guidelines (CPGs). These recommendations are especially important in a cancer health care context as patients are exposed to a multitude of interdisciplinary HCPs offering high-quality care throughout diagnosis, treatment, survivorship and palliative care. Although a large number of CPGs targeted towards cancer are widely disseminated, it is unknown whether implementation strategies targeting the use of these guidelines are effective in effecting HCP behaviour and patient outcomes in the cancer care context. The purpose of this systematic review will be to determine the effectiveness of different CPG dissemination and implementation interventions on HCPs' behaviour and patient outcomes in the cancer health care context.
Five electronic databases (CINAHL, the Cochrane Controlled Trials Register, MEDLINE via Ovid, EMBASE via Ovid and PsycINFO via Ovid) will be searched to include all studies examining the dissemination and/or implementation of CPGs in a cancer care setting targeting all HCPs. CPG implementation strategies will be included if the CPGs were systematically developed (e.g. literature review/evidence-informed, expert panel, evidence appraisal). The studies will be limited to randomized controlled trials, controlled clinical trials and quasi-experimental (interrupted time series, controlled before-and-after designs) studies. Two independent reviewers will assess articles for eligibility, data extraction and quality appraisal.
The aim of this review is to inform cancer care health care professionals and policymakers about evidence-based implementation strategies that will allow for effective use of CPGs.
Full-text · Article · Aug 2015 · Systematic Reviews
[Show abstract][Hide abstract] ABSTRACT: There is an urgent need to prevent excessive pregnancy weight gain, a contributor to both maternal and child obesity. However, the majority of women had reported not being counseled to gain an appropriate amount of gestational weight by their health care providers. We developed a knowledge translation (KT) tool designed to facilitate the clinical interaction between pregnant women and their health care providers (HCPs). We piloted the tool on the impact on women's knowledge of gestational weight gain (GWG) goals, and evaluated its potential in promoting appropriate knowledge about GWG within the 2009 Institute of Medicine guidelines.
We conducted a prospective cohort study, comparing women's knowledge about GWG after the KT tool to women from the same clinics and care providers the year prior. Our primary outcome was the proportion of women who reported receiving an appropriate GWG recommendation from their care provider. We evaluated knowledge on a survey conducted at enrollment in the cohort at ≤ 20 weeks gestation and evaluated participant satisfaction with the KT tool in the third trimester. We performed univariate and multivariable logistic regression analyses for differences in outcomes with historical controls from the same clinics. Our a priori sample size calculation required 130 participants to demonstrate a 15% increase in reported counseling about gestational weight gain.
One hundred and forty-six women were recruited and 131 (90%) completed the enrollment survey. Women who received the KT tool were more likely to report receiving a specific GWG recommendation from their HCP (adjusted odds ratio [AOR] 3.45, 95% confidence interval [CI] 2.22-5.37) and discussing GWG topics with their HCP (AOR 7.96, 95% CI 4.41-14.37), and believing that there were risks to their infants with inadequate GWG (AOR 2.48, 95% CI 1.14-5.37). Half of women (49.5%) indicated that they would recommend the tool to a friend.
Women who received the KT tool reported receiving more counseling on GWG from their HCPs and were more aware of the risks of gaining outside appropriate GWG recommendations. The association between GWG education and GWG requires further research.
[Show abstract][Hide abstract] ABSTRACT: Providers and patients are most likely to use and benefit from guidelines accompanied by implementation support. Guidelines published in 2007 and earlier assessed with the Appraisal of Guidelines, Research and Evaluation (AGREE) instrument scored poorly for applicability, which reflects the inclusion of implementation instructions or tools. The purpose of this study was to examine the applicability of guidelines published in 2008 or later and identify factors associated with applicability.
Systematic review of studies that used AGREE to assess guidelines published in 2008 or later.
MEDLINE and EMBASE were searched from 2008 to July 2014, and the reference lists of eligible items. Two individuals independently screened results for English language studies that reviewed guidelines using AGREE and reported all domain scores, and extracted data. Descriptive statistics were calculated across all domains. Multilevel regression analysis with a mixed effects model identified factors associated with applicability.
Of 245 search results, 53 were retrieved as potentially relevant and 20 studies were eligible for review. The mean and median domain scores for applicability across 137 guidelines published in 2008 or later were 43.6% and 42.0% (IQR 21.8-63.0%), respectively. Applicability scored lower than all other domains, and did not markedly improve compared with guidelines published in 2007 or earlier. Country (UK) and type of developer (disease-specific foundation, non-profit healthcare system) appeared to be associated with applicability when assessed with AGREE II (not original AGREE).
Despite increasing recognition of the need for implementation tools, guidelines continue to lack such resources. To improve healthcare delivery and associated outcomes, further research is needed to establish the type of implementation tools needed and desired by healthcare providers and consumers, and methods for developing high-quality tools.
Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.
[Show abstract][Hide abstract] ABSTRACT: Research shows that guidelines featuring implementation tools (GItools) are more likely to be used than those without GItools, however few guidelines offer GItools and guidance on developing GItools is lacking. The objective of this study was to identify common processes and considerations for developing GItools.
Interviews were conducted with developers of 4 types of GItools (implementation, patient engagement, point-of-care decision-making and evaluation) accompanying guidelines on various topics created in 2008 or later identified in the National Guideline Clearinghouse. Participants were asked to describe the GItool development process and related considerations. A descriptive qualitative approach was used to collect and analyze data.
Interviews were conducted with 26 GItool developers in 9 countries. Participants largely agreed on 11 broad steps, each with several tasks and considerations. Response variations identified issues lacking uniform approaches that may require further research including timing of GItool development relative to guideline development; decisions about GItool type, format and content; and whether and how to engage stakeholders. Although developers possessed few dedicated resources, they relied on partnerships to develop, implement and evaluate GItools.
GItool developers employed fairly uniform and rigorous processes for developing GItools. By supporting GItool development, the GItool methods identified here may improve guideline implementation and use.
[Show abstract][Hide abstract] ABSTRACT: Collaboration among researchers (clinician, non-clinician) and decision makers (managers, policy-makers, clinicians), referred to as integrated knowledge translation (IKT), enhances the relevance and use of research, leading to improved decision-making, policies, practice, and health care outcomes. However IKT is not widely practiced due to numerous challenges. This research explored how context influenced IKT as a means of identifying how IKT could be strengthened.
This research investigated IKT in three health services programs for colon cancer screening, prostate cancer diagnosis, and the treatment of pancreatic cancer. Qualitative methods were used to explore contextual factors that influenced how IKT occurred, and its impact. Data were collected between September 1, 2012 and May 15, 2013 from relevant documents, observation of meetings, and interviews with researchers and decision-makers, analyzed using qualitative methods, and integrated.
Data were analyzed from 39 documents, observation of 6 meetings, and 36 interviews. IKT included interaction at meetings, joint undertaking of research, and development of guidelines. IKT was most prevalent in one program with leadership, clear goals, dedicated funding and other infrastructural resources, and an embedded researcher responsible for, and actively engaged in IKT. This program achieved a variety of social, research and health service outcomes despite mixed individual views about the value of IKT and the absence of a programmatic culture of IKT. Participants noted numerous challenges including lack of time and incentives, and recommendations to support IKT. A conceptual framework of factors that influence IKT and associated outcomes was generated, and can be used by others to plan or evaluate IKT.
The findings can be applied by researchers, clinicians, managers or policy-makers to plan or improve collaborative decision-making for health services planning, delivery, evaluation or quality improvement. Further research is needed to explore whether these findings are widespread, and further understand how IKT can be optimized.
Full-text · Article · Dec 2014 · BMC Health Services Research
[Show abstract][Hide abstract] ABSTRACT: Guidelines are meant to facilitate evidence-based clinical decision-making but vary in methodological rigor and quality of reporting. We assessed the quality of guidelines published in major pediatric surgery journals. A MEDLINE search of 4 key pediatric surgery journals was performed. Included studies had guidelines, clinical practice guidelines, and consensus statements as a subject heading or keyword. Evaluations of guidelines were excluded. Eligible guidelines were assessed by three reviewers using the Appraisal of Guidelines for Research and Evaluation (AGREE II) Instrument. Our search identified ten guidelines for review. Agreement for study selection was excellent [K = 0.81 (95 % CI 0.63-0.99)]. The mean AGREE II score for individual guidelines was 18 % (SD 5.7 %). The best-scored quality domains were "scope and purpose" [mean score 49 % (SD 8.7 %)] and "clarity of presentation" [mean score 40 % (SD 18.7 %)]. The poorest score was for "editorial independence" [mean score 2 % (SD 3.7 %)]. The overall quality of guidelines in pediatric surgery, using AGREE II, is poor and may lead to inappropriate clinical decisions. Increased awareness of proper reporting and the methodological requirements for guideline development are needed to optimize the potential of guideline recommendations to improve practice.
No preview · Article · Oct 2014 · Pediatric Surgery International
[Show abstract][Hide abstract] ABSTRACT: Objectives
New evidence continues to emerge and requires attention after the release of a clinical practice guideline (CPG). The objective of this article is to describe the Document Assessment and Review (DAR) strategy designed to ensue that the CPGs remain current and their quality maintained and to present the results of two iteration of its implementation.
Study Design and Setting
The DAR process involves an annual assessment of our CPGs and a review of documents that require an update search. Two questionnaires are used to conduct the annual assessment and the review. The review involves evidence search, evidence review, and review approval.
In 2011, 109 documents were assessed; 22 (20%) were archived, 1 (1%) was deferred for assessment in 2012, 24 (22%) were considered special cases and 62 (57%) needed a new systematic review of the evidence. Of those 62, 19 (31%) were categorized as urgent, 16 (26%) as high, and others as medium or low priority. In 2012, 88 total documents were assessed; 15 (17%) were archived, 32 (36%) deferred, 3 (3%) were considered special cases, and 38 (43%) were prioritized for review.
Assessment and prioritization of existing CPGs are effective ways of ensuring that resources are directed toward the upkeep of those that are relevant and of highest priority.
[Show abstract][Hide abstract] ABSTRACT: Background
The original AGREE (Appraisal of Guidelines for REsearch and Evaluation) Instrument was published in 2003, and its revision, the AGREE II, in 2009. Together, they filled an important gap in the guideline and quality of care fields. Ten years later, the AGREE Enterprise reflects on a trajectory of projects and international collaboration that have contributed to advancing the science and quality of practice guidelines and the uptake of AGREE/AGREE II.FindingsThe AGREE Enterprise has undertaken activities to improve the tool and to develop resources to support its use. Since 2003, the uptake and adoption of AGREE by the international community has been swift and broad. A total of 33 language translations of the original AGREE Instrument and the current AGREE II are available and were initiated by the international community. A recent scan of the published literature identified over 600 articles that referenced the AGREE tools. The AGREE tools have been widely received and applied, with several organizations having incorporated the AGREE as part of their formal practice guideline programs. Since its redevelopment in 2010, the AGREE Enterprise website (www.agreetrust.org) continues to experience steady increases in visitors per month and currently has over 10,000 registered users.Conclusions
The AGREE Enterprise has contributed to the advancements of guidelines through research activities and international participation by scientific and user communities. As we enter a new decade, we look forward to ongoing collaborations and contributing to further advancements to improve quality of care and health care systems.
Preview · Article · Aug 2014 · Implementation Science
[Show abstract][Hide abstract] ABSTRACT: Background
Guidelines are the foundation for healthcare planning, delivery and quality improvement but are not consistently implemented. Few guidelines are accompanied by guideline implementation tools (GItools). Users have requested GItools, and developers have requested guidance on how to develop GItools. First it is necessary to characterize GItools. The purpose of this research was to generate a framework of desirable features of GItools.Methods
Items representing desirable GItool features were generated by a cross-sectional survey of the international guideline community. Items were confirmed by 31 guideline developers, implementers and researchers in a two-round Delphi survey administered on the Internet. The resulting GItool framework was applied with a sample of GItools accompanying guidelines identified in the National Guideline Clearinghouse.ResultsThe cross-sectional survey was completed by 96 respondents from Australia, Canada, the United Kingdom, the United States, The Netherlands, and various other countries. Seven of nine items were rated by the majority as desirable. A total of 31 panelists from 10 countries including Australia, Canada, Germany, New Zealand, Peru, Saudi Arabia, Spain, the United Kingdom, and the United States took part in a two-round Delphi survey. Ten items achieved consensus as desirable GItool features in round #1, and two additional items in round #2. A total of 13 GItools for Resource Planning, Implementation and Evaluation were identified among 149 guidelines on a variety of clinical topics (8.7%). Many GItools did not possess features considered desirable.Conclusions
Inclusion of higher quality GItools in guidelines is needed to support user adoption of guidelines. The GItool framework can serve as the basis for evaluating and adapting existing GItools, or developing new GItools. Further research is needed to validate the framework, develop and implement instruments by which developers can apply the framework, and specify which guidelines should be accompanied by GItools.
Preview · Article · Aug 2014 · Implementation Science
[Show abstract][Hide abstract] ABSTRACT: Practice pattern data demonstrate regional variation and lower than expected rates of adherence to practice guideline (PG) recommendations for the treatment of stage II/IIIA resected and stage IIIA/IIIB unresected non-small cell lung cancer (NSCLC) patients in Ontario, Canada. This study sought to understand how clinical decisions are made for the treatment of these patients and the role of PGs.
Surveys and key informant interviews were undertaken with clinicians and administrators.
Participants reported favorable ratings for PGs and the evidentiary bases underpinning them. The majority of participants agreed more patients should have received treatment and that regional variation is problematic. Participants estimated that up to 30% of patients are not good candidates for treatment and up to 20% of patients refuse treatment. The most common barrier to implementing PGs was the lack of organizational support by clinical administrative leadership. There was concern that the trial results underpinning the PG recommendations were not generalizable to the typical patients seen in clinic. The qualitative analysis yielded five themes related to physicians' decision making: the unique patient, the unique physician, the family, the clinical team, and the clinical evidence. A dynamic interplay between these factors exists.
Our study demonstrates the challenges inherent in (i) the complexity of clinical decision making; (ii) how quality of care problems are perceived and operationalized; and (iii) the clinical appropriateness and utility of PG recommendations. We argue that systematic and rigorous methodologies to help decision makers mitigate or negotiate these challenges are warranted.
Full-text · Article · Mar 2014 · Implementation Science
[Show abstract][Hide abstract] ABSTRACT: Inter-professional collaborative care (ICC) for cancer leads to multiple system, organizational, professional, and patient benefits, but is limited by numerous challenges. Empirical research on interventions that promote or enable ICC is sparse so guidance on how to achieve ICC is lacking. Research shows that ICC for diagnosis could be improved. Diagnostic assessment programs (DAPs) appear to be a promising model for enabling ICC. The purpose of this study was to explore how DAP structure and function enable ICC, and whether that may be associated with organizational and clinical outcomes.
A case study approach will be used to explore ICC among eight DAPs that vary by type of cancer (lung, breast), academic status, and geographic region. To describe DAP function and outcomes, and gather information that will enable costing, recommendations expressed in DAP standards and clinical guidelines will be assessed through retrospective observational study. Data will be acquired from databases maintained by participating DAPs and the provincial cancer agency, and confirmed by and supplemented with review of medical records. We will conduct a pilot study to explore the feasibility of estimating the incremental cost-effectiveness ratio using person-level data from medical records and other sources. Interviews will be conducted with health professionals, staff, and referring physicians from each DAP to learn about barriers and facilitators of ICC. Qualitative methods based on a grounded approach will be used to guide sampling, data collection and analysis.
Findings may reveal opportunities for unique structures, interventions or tools that enable ICC that could be developed, implemented, and evaluated through future research. This information will serve as a formative needs assessment to identify the nature of ongoing or required improvements, which can be directly used by our decision maker collaborators, and as a framework by policy makers, cancer system managers, and DAP managers elsewhere to strategically plan for and implement diagnostic cancer services.
Full-text · Article · Jan 2014 · Implementation Science
[Show abstract][Hide abstract] ABSTRACT: Realist reviews offer a rigorous method to analyze heterogeneous data emerging from multiple disciplines as a means to develop new concepts, understand the relationships between them, and identify the evidentiary base underpinning them. However, emerging synthesis methods such as the Realist Review are not well operationalized and may be difficult for the novice researcher to grasp. The objective of this paper is to describe the development of an analytic process to organize and synthesize data from a realist review.
Clinical practice guidelines have had an inconsistent and modest impact on clinical practice, which may in part be due to limitations in their design. This study illustrates the development of a transparent method for organizing and analyzing a complex data set informed by a Realist Review on guideline implementability to better understand the characteristics of guidelines that affect their uptake in practice (e.g., clarity, format). The data organization method consisted of 4 levels of refinement: 1) extraction and 2) organization of data; 3) creation of a conceptual map of guideline implementability; and 4) the development of a codebook of definitions.
This new method is comprised of four steps: data extraction, data organization, development of a conceptual map, and operationalization vis-a-vis a codebook. Applying this method, we extracted 1736 guideline attributes from 278 articles into a consensus-based set of categories, and collapsed them into 5 core conceptual domains for our guideline implementability map: Language, Format, Rigor of development, Feasibility, Decision-making.
This study advances analysis methods by offering a systematic approach to analyzing complex data sets where the goals are to condense, organize and identify relationships.
Full-text · Article · Sep 2013 · BMC Medical Research Methodology