Audit and Feedback: Effects on Professional Practice and Healthcare Outcomes

Department of Family Medicine, Women’s College Hospital, Toronto, Canada. 2Norwegian Knowledge Centre for the Health Services,Oslo, .
Cochrane database of systematic reviews (Online) (Impact Factor: 6.03). 06/2012; 6(6):CD000259. DOI: 10.1002/14651858.CD000259.pub3
Source: PubMed


Audit and feedback is widely used as a strategy to improve professional practice either on its own or as a component of multifaceted quality improvement interventions. This is based on the belief that healthcare professionals are prompted to modify their practice when given performance feedback showing that their clinical practice is inconsistent with a desirable target. Despite its prevalence as a quality improvement strategy, there remains uncertainty regarding both the effectiveness of audit and feedback in improving healthcare practice and the characteristics of audit and feedback that lead to greater impact.
To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes and to examine factors that may explain variation in the effectiveness of audit and feedback.
We searched the Cochrane Central Register of Controlled Trials (CENTRAL) 2010, Issue 4, part of The Cochrane Library., including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register (searched 10 December 2010); MEDLINE, Ovid (1950 to November Week 3 2010) (searched 09 December 2010); EMBASE, Ovid (1980 to 2010 Week 48) (searched 09 December 2010); CINAHL, Ebsco (1981 to present) (searched 10 December 2010); Science Citation Index and Social Sciences Citation Index, ISI Web of Science (1975 to present) (searched 12-15 September 2011).
Randomised trials of audit and feedback (defined as a summary of clinical performance over a specified period of time) that reported objectively measured health professional practice or patient outcomes. In the case of multifaceted interventions, only trials in which audit and feedback was considered the core, essential aspect of at least one intervention arm were included.
All data were abstracted by two independent review authors. For the primary outcome(s) in each study, we calculated the median absolute risk difference (RD) (adjusted for baseline performance) of compliance with desired practice compliance for dichotomous outcomes and the median percent change relative to the control group for continuous outcomes. Across studies the median effect size was weighted by number of health professionals involved in each study. We investigated the following factors as possible explanations for the variation in the effectiveness of interventions across comparisons: format of feedback, source of feedback, frequency of feedback, instructions for improvement, direction of change required, baseline performance, profession of recipient, and risk of bias within the trial itself. We also conducted exploratory analyses to assess the role of context and the targeted clinical behaviour. Quantitative (meta-regression), visual, and qualitative analyses were undertaken to examine variation in effect size related to these factors.
We included and analysed 140 studies for this review. In the main analyses, a total of 108 comparisons from 70 studies compared any intervention in which audit and feedback was a core, essential component to usual care and evaluated effects on professional practice. After excluding studies at high risk of bias, there were 82 comparisons from 49 studies featuring dichotomous outcomes, and the weighted median adjusted RD was a 4.3% (interquartile range (IQR) 0.5% to 16%) absolute increase in healthcare professionals' compliance with desired practice. Across 26 comparisons from 21 studies with continuous outcomes, the weighted median adjusted percent change relative to control was 1.3% (IQR = 1.3% to 28.9%). For patient outcomes, the weighted median RD was -0.4% (IQR -1.3% to 1.6%) for 12 comparisons from six studies reporting dichotomous outcomes and the weighted median percentage change was 17% (IQR 1.5% to 17%) for eight comparisons from five studies reporting continuous outcomes. Multivariable meta-regression indicated that feedback may be more effective when baseline performance is low, the source is a supervisor or colleague, it is provided more than once, it is delivered in both verbal and written formats, and when it includes both explicit targets and an action plan. In addition, the effect size varied based on the clinical behaviour targeted by the intervention.
Audit and feedback generally leads to small but potentially important improvements in professional practice. The effectiveness of audit and feedback seems to depend on baseline performance and how the feedback is provided. Future studies of audit and feedback should directly compare different ways of providing feedback.

Download full-text


Available from: Signe Agnes Flottorp, Jan 06, 2015
  • Source
    • "Although we are the first to have empirically investigated step 1 in the underlying feedback mechanism, earlier studies aimed at identifying design characteristics and contextual factors that influence the outcome of the mechanism (i.e., change in quality of care). A Cochrane review [1] concluded that low performance scores increased A&F effectiveness. The authors attributed this to greater intention to take action, or absence of ceiling effects [12]; our findings seem to support the first. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Audit and feedback (A&F) is widely used to aid healthcare professionals in improving clinical performance, but there is little understanding of the underlying mechanism that determines its effectiveness. The aim of this paper is to investigate the process by which healthcare professionals select indicators as improvement targets based on A&F. We performed a laboratory study among 41 healthcare professionals in the context of a web-based A&F intervention designed to improve the quality of cardiac rehabilitation care in the Netherlands. Feedback was provided on eighteen quality indicators, including a score and a colour (representing a recommendation for selection (red and yellow) or non-selection (green)). Indicators with more room for improvement were more likely to be selected, although this varied substantially between participants. In more than a quarter of the cases, participants did not select indicators with obvious room for improvement (yellow or red colour), or selected indicators without apparent room for improvement (green colour). We conclude that personal preferences and beliefs concerning quality and performance targets may dilute the efficiency of A&F.
    Studies in health technology and informatics 08/2015; 216:424-8.
  • Source
    • "Currently, the specific features of audit and feedback interventions that can improve emergency care are not well understood [2]. The Cochrane review focused only on randomized controlled trials and did not examine the effectiveness of audit and feedback specific to the ED setting [7]. There is a rapidly growing literature on audit and feedback in the ED setting that has not been systematically reviewed in 6 years [8]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Audit and feedback can decrease variation and improve the quality of care in a variety of health care settings. There is a growing literature on audit and feedback in the emergency department (ED) setting. Because most studies have been small and not focused on a single clinical process, systematic assessment could determine the effectiveness of audit and feedback interventions in the ED and which specific characteristics improve the quality of emergency care. The objective of the study is to assess the effect of audit and feedback on emergency physician performance and identify features critical to success. We adhered to the PRISMA statement to conduct a systematic review of the literature from January 1994 to January 2014 related to audit and feedback of physicians in the ED. We searched Medline, EMBASE, PsycINFO, and PubMed databases. We included studies that were conducted in the ED and reported quantitative outcomes with interventions using both audit and feedback. For included studies, 2 reviewers independently assessed methodological quality using the validated Downs and Black checklist for nonrandomized studies. Treatment effect and heterogeneity were to be reported via meta-analysis and the I(2) inconsistency index. The search yielded 4332 articles, all of which underwent title review; 780 abstracts and 131 full-text articles were reviewed. Of these, 24 studies met inclusion criteria with an average Downs and Black score of 15.6 of 30 (range, 6-22). Improved performance was reported in 23 of the 24 studies. Six studies reported sufficient outcome data to conduct summary analysis. Pooled data from studies that included 41124 patients yielded an average treatment effect among physicians of 36% (SD, 16%) with high heterogeneity (I(2) = 83%). The literature on audit and feedback in the ED reports positive results for interventions across numerous clinical conditions but without standardized reporting sufficient for meta-analysis. Characteristics of audit and feedback interventions that were used in a majority of studies were feedback that targeted errors of omission and that was explicit with measurable instruction and a plan for change delivered in the clinical setting greater than 1 week after the audited performance using a combination of media and types at both the individual and group levels. Future work should use standardized reporting to identify the specific aspects of audit or feedback that drive effectiveness in the ED. Copyright © 2015. Published by Elsevier Inc.
    The American journal of emergency medicine 07/2015; 33(10). DOI:10.1016/j.ajem.2015.07.039 · 1.27 Impact Factor
  • Source
    • "Although the effect of feedback involves uncertainty, AF may have larger effects under specific conditions, including when baseline performance is low, when feedback is provided by a supervisor or peer, when feedback is provided for more than one performance interval, and when it is provided both verbally and in writing [16]. AF is frequently used as a complement to other interventions, such as educational outreach and clinical reminders [16]. Multifaceted interventions using AF have been demonstrated to improve care in low-resource settings [1] [19]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Sub-optimal performance of healthcare providers in low-income countries is a critical and persistent global problem. The use of electronic health information technology (eHealth) in these settings is creating large-scale opportunities to automate performance measurement and provision of feedback to individual healthcare providers, to support clinical learning and behavior change. An electronic medical record system (EMR) deployed in 66 antiretroviral therapy clinics in Malawi collects data that supervisors use to provide quarterly, clinic-level performance feedback. Understanding barriers to provision of eHealth-based performance feedback for individual healthcare providers in this setting could present a relatively low-cost opportunity to significantly improve the quality of care. The aims of this study were to identify and describe barriers to using EMR data for individualized audit and feedback for healthcare providers in Malawi and to consider how to design technology to overcome these barriers. We conducted a qualitative study using interviews, observations, and informant feedback in eight public hospitals in Malawi where an EMR system is used. We interviewed 32 healthcare providers and conducted seven hours of observation of system use. We identified four key barriers to the use of EMR data for clinical performance feedback: provider rotations, disruptions to care processes, user acceptance of eHealth, and performance indicator lifespan. Each of these factors varied across sites and affected the quality of EMR data that could be used for the purpose of generating performance feedback for individual healthcare providers. Using routinely collected eHealth data to generate individualized performance feedback shows potential at large-scale for improving clinical performance in low-resource settings. However, technology used for this purpose must accommodate ongoing changes in barriers to eHealth data use. Understanding the clinical setting as a complex adaptive system (CAS) may enable designers of technology to effectively model change processes to mitigate these barriers. Copyright © 2015. Published by Elsevier Ireland Ltd.
    International Journal of Medical Informatics 07/2015; 84(10). DOI:10.1016/j.ijmedinf.2015.07.003 · 2.00 Impact Factor
Show more