ArticlePDF Available
Title
Implementing better healthcare: using audit and feedback to change therapists’
practice
Introduction
The quality of healthcare, including care by physiotherapists, is influenced by a range
of factors. For example, a recent review (Flottorp et al. 2013) identified 57 clusters of
factors, ranging from system and organisational factors (e.g. workload, leadership),
to professionals’ practice (e.g. care actions, skills), to service user characteristics
(e.g. social support, adherence). These clusters of factors provide evidence-based
pathways to implementing better healthcare.
Of the possible clusters of factors, targeting professionals’ practice is one of the most
important (Greenhalgh et al. 2004, Powell et al. 2009). Professionals’ practice
explains substantial variation in patient care (Kolehmainen et al. 2010) even after
accounting for organisational and service user factors (Brookhart et al. 2006). In
simple terms, implementing better care requires a change in what professionals do
(Boaden 2009).
There is a wealth of evidence about how to effectively support professionals to
change what they do, i.e. evidence on healthcare professionals’ behaviour change.
One of the techniques with strongest evidence for effectiveness is audit and
feedback (Ivers et al. 2012). While audit and feedback is used widely in healthcare in
general, its use in physiotherapy has been limited (Scott et al. 2012). Instead,
physiotherapists, alongside other allied health professions, tend to rely on the
traditional educational approaches which have been shown to have limited
effectiveness (Scott et al. 2012). This is likely to be, at least in part, because of lack
of applied, practical knowledge and examples about how to use audit and feedback
1
to change practice (as opposed to merely using it to evaluate the quality of practice).
Sharing examples of their use is an effective way to increase this knowledge.
The purpose of the present article is to briefly describe what audit and feedback as a
change technique consists of, and to provide practice-based examples of its use and
effects within allied health, including physiotherapy.
Audit and feedback as a technique to change practice
Audit and feedback to change practice involves four parts.
1: Selecting a clinical practice standard, e.g. “programmes for managing spasticity in
children should be goals focussed” (National Institute for Health and Care Excellence
2012), and from that standard specifying a practice behaviour to be targeted for
change, e.g. “therapist documents goals for children with spasticity” (National
Institute for Health and Care Excellence 2012). The targeted behaviours should be
within the professionals’ capabilities and control, and specific in terms of who does
what to whom (Michie&Johnston 2004, Ivers et al. 2012).
2: Measuring the professionals’ current practice in terms of the target behaviour (e.g.
auditing whether or not therapist has documented goals for children with spasticity).
The measuring should be carried out routinely and focus on a recent behaviour that
is observable in principle (National Institute for Health and Clinical Excellence 2007,
Ivers et al. 2012).
3: Feeding back the gathered information to the professional (e.g. feed back to the
professional the proportion of children with spasticity on their caseload for whom the
therapist has documented goals). This provides the professional with information
2
about their actual practice in relation to the target behaviour and the standard. In this,
feedback delivered both in writing and verbally is more effective than verbal feedback
alone, and feedback delivered by people known to and familiar with the professional
is more effective than feedback from an unknown investigator (Ivers et al. 2012).
Effectiveness of feedback can be further enhanced by including the target behaviour
within the feedback information and focusing the feedback on behaviours for which
professionals’ performance is low (Ivers et al. 2012).
4: The professional uses the feedback: to compare their practice with the target
behaviour and the standard; to identify any discrepancy between their practice and
the target; and to identify actions they can take to change their practice and thus to
achieve the standard (Carver&Scheier 2001, Ivers et al. 2012). The effectiveness of
this can be enhanced by providing the professional with support to set specific goals
and develop clear action plans for change (Ivers et al. 2012).
Practice-based examples: methods and results
We present three examples of using audit and feedback to change therapists’
practice to implement better care in three United Kingdom National Health Service
(NHS) therapy teams. The standards related to working with service users,
maintaining clinical records, and managing service user’s posture. Each example is
presented in relation to the four steps above, describing methods used in each
therapy team, results, and a brief discussion of the impact on practice.
Working with service users
The standard was working in partnership with service users (National Institute for
Health and Clinical Excellence 2012, National Institute for Health and Care
Excellence 2013, National Institute for Health and Care Excellence 2015), and within
3
that the target behaviour was that the community therapists have direct
communication with the service users on their caseloads at least once every three
months. The standard and target behaviour were selected by the lead clinician (JM)
and the wider team in response to service user requests to improve the timeliness of
communication. Fifteen therapists within one therapy team managing approximately
500 cases were included. Therapists’ current practice was measured as a daily snap
shot approximately once every quarter for one year (2014-15). All service users on
the service’s collective caseload were audited by an administrator, through the
electronic health records, for whether or not the service user had had direct
communication from a therapist in the last three months. All service users who had
not had direct contact from a therapist were further reviewed by the lead clinician to
confirm whether direct communication had been recorded and whether there were
clinical reasons for no communication to have taken place (e.g. because a review
date of greater than three months had been agreed). Therapists’ practice was rated
in terms of how closely it aligned with the target behaviour: red (communicated
directly with <85% of service users in the last three months), amber (communicated
directly with 85-94% of service users) or green (communicated directly ≥95% of
service users).
After each audit cycle, therapists received anonymised, visual, team-level verbal and
written feedback from the lead clinician at a team meeting. This feedback showed a
graph of therapists’ practice at each of the quarterly audits to date (Figure 1, no
quarter 4 data available for therapist 5 due to staff changes), and was used to prompt
discussion about current barriers and supports to timely communication with service
users. Each therapist then also received individualised written feedback. This
consisted of: the anonymised graph presented at the team meeting, but with an
indication of that therapist’s individual position within the graph (Figure 1); the
therapist’s rating (above); and a list of service users with whom they should arrange
4
direct communication. The feedback also included praise for those achieving a green
rating. The individualised feedback enabled therapists to compare their current
practice with the target behaviour, their past practice, and their (anonymised) peers’
practice. Each therapist’s clinical supervisor received a copy of the individual
feedback, and therapists were encouraged to use clinical supervision for support to
improve or maintain their practice. Therapists with a red rating received further
support from the lead clinician.
In terms of impact on practice, from quarter 1 to quarter 3, the proportion of
therapists with a red rating reduced from 53% to 0%, and the proportion with a green
rating increased from 13% to 67%. There was reduced variability in practice: the
range was reduced from 40-100% at quarter 1 to 86-100% at quarter 3. Practice
against the standard dipped from quarter 3 to quarter 4 (21% in red, 36% in green,
range 80-100%). The team attributed this to the impact of significant organisational
changes.
Figure 1: Feedback graph to all therapists in the team displaying each therapists’
communication with service users over the last four quarters audited
Maintaining clinical records
The standard was that all clinical records should be traceable to individual service
users (Department of Health 2013), and within that the target behaviour was that
therapists document on every page of the clinical record the service user’s name,
hospital number, and NHS number. The standard and the behaviour were selected
by the service manager (JF) with the Trust’s clinical governance team. Seventy-three
therapists across ten therapy teams were included. Therapists’ practice was
measured by auditing a random sample (minimum of 2 per therapist, range 2-5
5
depending on head count per team) of clinical records for the included therapists.
The first audit was completed in April 2013, a total of 191 records were included, and
each included record was scored from 0 to 4 on performance of the target behaviour
(0 = therapist has not documented any of pages with the required information; 1 =
therapist has documented a quarter of the pages, 2 = therapist has documented half
of the pages, 3 = therapist has documented two thirds of the pages, 4 = therapist has
documented all of the pages). The data were collected by therapists, with each
therapist auditing records for which they were not the case-holder, and entered into
Microsoft Excel. Team managers accessed the data during the audit, for live visual
information about their team’s performance. The therapists received team-level
verbal and written feedback (Figure 2) on the team’s aggregate practice from the
managers at team meetings; and individual written feedback on their own practice
alongside comparative information about their colleagues’ practice. Therapists
compared their practice against that of their colleagues and identified the
discrepancy between their actual practice (percentage score) and the target
performance (100%). From the comparisons, actions to improve practice were
identified, e.g. the use of sticker labels with the service user’s name, hospital number
and NHS number. In terms of impact on practice, in 2013, 69% of the records audited
met the standard. At follow up, in 2014, 81% of the records audited (103/128 records
from 66 therapists) met the standard. At this follow up audit, the scoring was
simplified and made stricter. Scores 1, 2 and 3 were eliminated, and 0 = therapist has
not documented the required information on every page, 4 = therapist has
documented the required information on every page. Through regular audit and
feedback, using clearly defined criteria and linking results to individual therapist
behaviours, the standard of clinical records improved resulting in safer and more
effective care.
6
Whe re c an we im pr ove ?
S c or es < 7 5 %
Pat ie nt d e t ail s on e ve r y sh e e t = 6 9 %
Al te r at io ns t o r e co r d s ar e t im e d = 0 %
Al te r at io ns a re d at ed = 5 0%
Al l e nt r ie s a re ti me d = 33 %
Full na me & s ig na tur e = 5 0 %
Loc a tio n = 6 5 %
Ma nua l h and lin g ri sk a ss e s s me n t
Co ns e nt
2 c m ga ps s c or e t h ro ug h = 5 9 %
Re s ult s
• 19 1 s e t s o f not e s aud it e d in t ot al
• O ve r all c om plianc e ac r os s t he s e r vic e
= 8 4 % ( down f r om 9 1% la st y e ar )
– Mains t re a m = 8 1%
– S p e cial S c h oo l = 88 %
– DC D = 9 2%
Figure 2: An example of feedback provided verbally and in writing to therapists
through manager-led team meetings
Managing service user’s postures
The standard was that children with therapeutic postural management programmes
are moved or repositioned regularly during the school day (Dabbs et al. 2004), and
within that the target behaviour was that children who have a therapeutic postural
management programme are moved or repositioned by classroom staff, using sitting,
standing and lying positions, at least 3 times per day in school. The standard and the
target behaviour were selected by a school therapist with a service manager (JF) and
was agreed with teachers. Classroom staff’s current practice in moving or
repositioning children was measured in line with the guidance from the
implementation toolkit on postural management programmes in special schools
(Dabbs et al. 2004). This consisted of sending a questionnaire to classroom staff
(n=9) and school senior managers (n=3) to ask ‘In your class/school, how often are
children with therapeutic postural management programmes moved or repositioned
7
at least 3 times a day?’, with response options: ‘Always’, ‘Sometimes’, ‘Never’. Staff
had two weeks to complete the questionnaires and received one verbal reminder.
Questionnaires were completed anonymously, and 100% were returned. Responses
were collated by the school therapist (see acknowledgements) using Microsoft Excel.
Classroom staff and school senior management received collective verbal and visual
feedback on the aggregate performance of staff within the school. The feedback was
provided by the school therapist at a mandatory staff training day focused on the
delivery of the standard. Written aggregate feedback was also provided in a format of
a full report of results. The school therapist supported classroom staff to collectively
generate action plans for changing practice. Action plans included, e.g.: therapist to
provide the classroom staff written instructions and pictures of positions for children
with postural management programmes; classroom staff to identify key times within
daily timetables for moving and repositioning children; and therapy and classroom
staff to work together in lessons where children could be moved or repositioned often
(swimming, physical education etc.). In terms of impact on practice, at the first audit
cycle, 23.8% of classroom staff reported children with postural management
programmes were always repositioned or moved at least 3 times per day (Figure 3).
At follow-up cycle, 62.5% of classroom staff reported children were always moved or
repositioned at least 3 times a day.
Figure 3. Percentages of staff who reported children with postural management
programmes are always, sometimes, or never moved or repositioned at least 3 times
a day
Discussion
Three practice-based examples of using audit and feedback to change therapists’
practice and improve healthcare were presented. The examples illustrate specific
aspects of applying audit and feedback, published examples of which, to our
8
knowledge, do not yet exist. For example, they show how audit and feedback can be
applied to different types of practice standards and settings, from therapist-service
user interactions, to care process management, and school based clinical
interventions. This is particularly relevant for paediatric therapy providers as explicit
standards for practice continue to be rare, and are traditionally heavily focused on
process factors (such as record keeping). The examples used here show that other
standards beyond process factors can meaningfully be identified and used, and that
doing so is advantageous for care.
The examples also illustrate a range of ways for implementing the two ‘feedback’
related steps of audit and feedback (i.e. the provision of feedback and the use of
feedback by professionals), e.g. interactive methods involving peers and supervisors,
electronic individualised methods, and team approaches. This is important, as the
feedback steps are the ones that transform an audit from merely a quality evaluation
to ‘audit and feedback’ as a change technique. As the demands and constraints of
practice settings vary, it is important to consider a range of ways in which feedback
can be provided, in order to ensure the feasibility and acceptability of the feedback
within the specific setting.
Conclusion
Audit and feedback is an evidence-based but underused technique to change allied
health, including physiotherapy, practice. The examples provided increase applied
knowledge about how to implement audit and feedback to improve diverse standards
in therapy practice, and through that will enable therapy providers to use audit and
feedback more widely.
9
Key Message: Audit and feedback, an effective, evidence-based technique for
implementing better care, can be used by physiotherapists – examples are provided
on how to do it.
REFERENCES
Boaden, R. 2009, "Quality improvement: theory and practice", British Journal of
Healthcare Management, vol. 15, no. 1, pp. 12-16.
Brookhart, M.A., Solomon, D.H., Wang, P., Glynn, R.J., Avorn, J. & Schneeweiss, S.
2006, "Explained variation in a model of therapeutic decision making is
partitioned across patient, physician, and clinic factors", Journal of Clinical
Epidemiology, vol. 59, no. 1, pp. 18-25.
Carver, C.S. & Scheier, M.F. 2001, On the self-regulation of behaviour, Cambridge
University Press, Cambridge.
Dabbs, H., Richardson, M. & Harrison, C. 2004, Joint health education review into
the delivery of postural management programmes in special schools.,
Nottingham County Council.
Department of Health 2013, Information: to share or not to share? The information
governance review., Crown Copyright.
Flottorp, S.A., Oxman, A.D., Krause, J., Musila, N.R., Wensing, M., Godycki-Cwirko,
M., Baker, R. & Eccles, M.P. 2013, "A checklist for identifying determinants of
practice: a systematic review and synthesis of frameworks and taxonomies of
factors that prevent or enable improvements in healthcare professional practice",
Implementation Science, [Online], vol. 8, no. 35, pp. 03/09/2013.
Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P. & Kyriakidou, O. 2004, "Diffusion
of innovations in service organizations: systematic review and
recommendations", The Milbank Quarterly, vol. 82, no. 4, pp. 581-629.
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J., Odgaard-Jensen, J., French, S.,
O'Brien, M., Johansen, M., Grimshaw, J. & Oxman, A.:. 2012, "Audit and
feedback: effects on professional practice and healthcare outcomes", Cochrane
Database of Systematic Reviews, , no. 6, pp. Art. No.: CD000259.
Kolehmainen, N., MacLennan, G., Francis, J. & Duncan, E.A.S. 2010, "Clinicians'
caseload management behaviours as explanatory factors in patient's length of
time on caseloads: a predictive multilevel study in community paediatric
occupational therapy", BMC Health Services Research, vol. 10, no. 249, pp.
doi:10.1186/1472-6963-10-249.
Michie, S. & Johnston, M. 2004, "Changing clinical behaviour by making guidelines
specific", British Medical Journal, vol. 328, no. 7435, pp. 343-345.
National Institute for Health and Care Excellence 2015, "Challenging behaviour and
learning disabilities: prevention and interventions for people with learning
disabilities whose behaviour challenges.", .
National Institute for Health and Care Excellence 2013, CG170 Autism -
management of autism in children and young people: NICE guideline, National
Institute for Health and Care Excellence, Manchester.
National Institute for Health and Care Excellence 2012, CG145: Spasticity in children
and young people with non-progressive brain disorders: management of
spasticity and co-existing motor disorders and their early musculoskeletal
complications: NICE Guideline, National Institute for Health and Clinical
Excellence, Manchester.
10
National Institute for Health and Clinical Excellence 2012, "Spasticity in children and
young people with non-progressive brain disorders", .
National Institute for Health and Clinical Excellence 2007, Behaviour change at
population, community and individual levels, National Institute for Health and
Clinical Excellence, London.
Powell, A.E., Rushmer, R.K. & Davies, H.T.O. 2009, A systematic narrative review of
quality improvement models in health care, Quality Improvement Scotland,
Edinburgh.
Scott, S.D., Albrecht, L., O’Leary, K., Ball, G.D.C., Hartling, L. & Hofmeyer, A. 2012,
"Systematic review of knowledge translation strategies in the allied health
professions", Implementation Science, [Online], vol. 7, no. 70.
11
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.