Improving the reliability of physician performance assessment: identifying the "physician effect" on quality and creating composite measures.

Center for Health Policy Research and Department of Medicine, School of Medicine, University of California, Irvine, California 92697, USA.
Medical care (Impact Factor: 2.94). 04/2009; 47(4):378-87. DOI: 10.1097/MLR.0b013e31818dce07
Source: PubMed

ABSTRACT The proliferation of efforts to assess physician performance underscore the need to improve the reliability of physician-level quality measures.
Using diabetes care as a model, to address 2 key issues in creating reliable physician-level quality performance scores: estimating the physician effect on quality and creating composite measures.
Retrospective longitudinal observational study.
A national sample of physicians (n = 210) their patients with diabetes (n = 7574) participating in the National Committee on Quality Assurance-American Diabetes Association's Diabetes Provider Recognition Program.
Using 11 diabetes process and intermediate outcome quality measures abstracted from the medical records of participants, we tested each measure for the magnitude of physician-level variation (the physician effect or "thumbprint"). We then combined measures with a substantial physician effect into a composite, physician-level diabetes quality score and tested its reliability.
We identified the lowest target values for each outcome measure for which there was a recognizable "physician thumbprint" (ie, intraclass correlation coefficient > or =0.30) to create a composite performance score. The internal consistency reliability (Cronbach's alpha) of the composite score, created by combining the process and outcome measures with an intraclass correlation coefficient > or =0.30, exceeded 0.80. The standard errors of the composite case-mix adjusted score were sufficiently small to discriminate those physicians scoring in the highest from those scoring in the lowest quartiles of the quality of care distribution with no overlap.
We conclude that the aggregation of well-tested quality measures that maximize the "physician effect" into a composite measure yields reliable physician-level quality of care scores for patients with diabetes.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Given rising health care costs, there has been a renewed interest in using utilization measures to profile physicians. Despite the measures' common use, few studies have examined their reliability and whether they capture true differences among physicians.
    06/2013; 1(1-2):22-29. DOI:10.1016/j.hjdsi.2013.04.002
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: OBJECTIVE To investigate the quality of type 2 diabetes care according to sex.RESEARCH DESIGN AND METHODS Clinical data collected during the year 2009 were extracted from electronic medical records; quality-of-care indicators were evaluated. Multilevel logistic regression analysis was applied to estimate the likelihood of women versus men to be monitored for selected parameters, to reach clinical outcomes, and to be treated with specific classes of drugs. The intercenter variability in the proportion of men and women achieving the targets was also investigated.RESULTSOverall, 415,294 patients from 236 diabetes outpatient centers were evaluated, of whom 188,125 (45.3%) were women and 227,169 (54.7%) were men. Women were 14% more likely than men to have HbA1c >9.0% in spite of insulin treatment (odds ratio 1.14 [95% CI 1.10-1.17]), 42% more likely to have LDL cholesterol (LDL-C) ≥130 mg/dL (1.42 [1.38-1.46]) in spite of lipid-lowering treatment, and 50% more likely to have BMI ≥30 kg/m(2) (1.50 [1.50-1.54]). Women were less likely to be monitored for foot and eye complications. In 99% of centers, the percentage of men reaching the LDL-C target was higher than in women, the proportion of patients reaching the HbA1c target was in favor of men in 80% of the centers, and no differences emerged for blood pressure.CONCLUSIONS Women show a poorer quality of diabetes care than men. The attainment of the LDL-C target seems to be mainly related to pathophysiological factors, whereas patient and physician attitudes can play an important role in other process measures and outcomes.
    Diabetes care 07/2013; 36(10). DOI:10.2337/dc13-0184 · 8.57 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Background To describe the implementation and initial results of an audit-feedback quality improvement (QI) initiative in Belgian diabetic foot clinics (DFCs). Methods Using self-developed software and questionnaires, DFCs collected data in 2005, 2008 and 2011, covering characteristics, history, and ulcer severity, management and outcome of the first 52 patients presenting with a Wagner grade ≥2 diabetic foot ulcer (DFU) or acute neuropathic osteoarthropathy that year. QI was encouraged by meetings and by anonymous benchmarking of DFCs. ResultsThe first audit-feedback cycle was a pilot study. Subsequent audits, with a modified methodology, had increasing rates of participation and data completeness.Over 85% of DFCs participated and 3,372 unique patients were sampled between 2005 and 2011 (3,312 with a DFU and 111 with acute neuropathic osteoarthropathy). Median age was 70 years, median diabetes duration was 14 years and 64% were male. Fifty-one percent of DFUs were plantar and 29% were both ischemic and deeply infected. Ulcer healing rate at 6 months significantly increased from 49% to 54% between 2008 and 2011. Management of DFUs varied between DFCs: 88% of plantar mid-foot ulcers were off-loaded (P10-P90: 64-100%) and 42% of ischemic limbs were revascularized (P10-P90: 22-69%) in 2011. ConclusionsA unique, nationwide QI initiative was established among DFCs, covering ulcer healing, lower-limb amputation and many other aspects of diabetic foot care. Data completeness increased, partly thanks to questionnaire revision. Benchmarking remains challenging, given the many possible indicators and limited sample size. The optimized questionnaire allows future quality of care monitoring in DFCs. This article is protected by copyright. All rights reserved.
    Diabetes/Metabolism Research and Reviews 07/2014; 30(5). DOI:10.1002/dmrr.2524 · 3.59 Impact Factor