Measuring how well the NHS looks after its own staff: methodology of the first national clinical audits of occupational health services in the NHS

Journal of Evaluation in Clinical Practice (Impact Factor: 1.58). 03/2012; 18(2):283 - 289. DOI: 10.1111/j.1365-2753.2010.01574.x

ABSTRACT Rationale, aims and objectives Little is known about the quality of occupational health care provided to National Health Service (NHS) staff. We designed the first national clinical audits of occupational health care in England. We chose to audit depression and back pain as health care workers have high levels of both conditions compared with other employment sectors. The aim of the audits was to drive up quality of care for staff with these conditions. The object of this paper is to describe how we developed an audit methodology and overcame challenges presented by the organization and delivery of occupational health care for NHS staff.Methods We designed two retrospective case note audits which ran simultaneously. Sites submitted up to 40 cases for each audit. We used duplicate case entry to test inter-rater reliability and performed selection bias checks. Participants received their site's audit results, benchmarked against the national average, within 4 months of the end of the data entry period. We used electronic voting at a results dissemination conference to inform implementation activities.Results Occupational Health departments providing services to 278 (83%) trusts in England participated in one or both audits. Median kappa scores were above 0.7 for both pilot and full audits, indicating ‘good’ levels of inter-rater reliability.In total, 79% of participants at a dissemination conference said that they had changed their clinical practice either during data collection (52%) or following receipt of their audit results (27%).Conclusions Clinical audit can be conducted successfully in the occupational health setting. We obtained meaningful data that have stimulated local and national quality improvement activities. Our methodology would be transferable to occupational health settings outside the NHS and in other countries.

  • [Show abstract] [Hide abstract]
    ABSTRACT: The aim of this study was to assure the validity and reliability of the Intercollegiate Stroke Audit Package as used in the National Sentinel Audit of Stroke. The Intercollegiate Working Party for Stroke, which included most stakeholders, including patients, devised the audit standards. These were submitted to a formal consensus (modified Delphi) survey before the audit questions were developed and piloted for validity and reliability. Following the pilot, Help Booklets were developed to promote the involvement of all disciplines as auditors in the national sentinel audit of stroke and ensure inter-rater reliability. During the national audit each Trust was asked to double rate the first five cases with auditors of different disciplines working independently. A total of 886 case notes were double-rated in 184 separate sites (median 5, range 1-5 per site). Trusts used auditors from different disciplines in 77% of cases. After excluding the 'No answer' cases the kappa score for items ranged from 0.49 to 0.87 (median 0.70, IQR 0.63-0.78). Very good agreement was found for seven of the 45 items, good agreement for 30 items, and moderate agreement for eight items. This large study, across a range of hospital sites and involving many disciplines, demonstrates that careful piloting of audit tools, with use of clear instructions to auditors, promotes the reliability of data.
    Journal of Evaluation in Clinical Practice 03/2001; 7(1):1-11. · 1.58 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: To describe the standards of care for stroke patients in England, Wales and Northern Ireland and to determine the power of national audit, coupled with an active dissemination strategy to effect change. A national audit of organisational structure and retrospective case note audit, repeated within 18 months. Separate postal questionnaires were used to identify the types of change made between the first and second round and to compare the representativeness of the samples. 157 trusts (64% of eligible trusts in England, Wales, and Northern Ireland) participated in both rounds. 5589 consecutive patients admitted with stroke between 1 January 1998 and 31 March 1998 (up to 40 per trust) and 5375 patients admitted between 1 August 1999 and 31 October 1999 (up to 40 per trust). Audit tool-Royal College of Physicians Intercollegiate Working Party stroke audit. The proportion of patients managed on stroke units rose between the two audits from 19% to 26% with the proportion managed on general wards falling from 60% to 55% and those managed on general rehabilitation wards falling from 14% to 11%. Standards of assessment, rehabilitation, and discharge planning improved equally on stroke units and general wards, but in many aspects remained poor (41% formal cognitive assessment, 46% weighed once during admission, 67% physiotherapy assessment within 72 hours, 24% plan documented for mood disturbance, 36% carers' needs assessed separately). Nationally conducted audit linked to a comprehensive dissemination programme was effective in stimulating improvements in the quality of care for patients with stroke. More patients are being managed on stroke units and multidisciplinary care is becoming more widespread. There remain, however, many areas where standards of care are low, indicating a need for investment of skills and resources to achieve acceptable levels.
    Quality in Health Care 10/2001; 10(3):141-51.
  • [Show abstract] [Hide abstract]
    ABSTRACT: Rationale, aims and objectives This national clinical audit aimed to develop and implement a methodology to assess the appropriateness of prescribing for patients over the age of 65 in hospitals, general practice and nursing homes. Methods Organizations providing health care in the National Health Service in these three sectors were recruited into multi-disciplinary and inter-organizational local coalition teams. Prescription data and relevant clinical data were collected electronically on a customized database. The appropriateness of prescribing for specific conditions among the patients sampled was assessed by simple computerized algorithms, and users were provided with feedback to stimulate discussion and change. Use of the software tool was demonstrated to be feasible and its data reliable. Participants were re-audited, after a period of nationally guided and locally driven intervention, to evaluate levels of change. Local efforts to stimulate change and barriers to change were collected qualitatively. Results and conclusions The investigation revealed encouraging results and demonstrated the ability of audit to improve the quality of clinical services in given circumstances, although a multiplicity of questions relating to cost and methodology remain to be addressed.
    Journal of Evaluation in Clinical Practice 06/2002; 8(2):189 - 198. · 1.58 Impact Factor


Available from
May 23, 2014