Meaningful variation in performance: a systematic literature review.

Division of Research, Kaiser Permanente Medical Care Program, Oakland, CA, USA.
Medical care (Impact Factor: 2.94). 02/2010; 48(2):140-8. DOI: 10.1097/MLR.0b013e3181bd4dc3
Source: PubMed

ABSTRACT Recommendations for directing quality improvement initiatives at particular levels (eg, patients, physicians, provider groups) have been made on the basis of empirical components of variance analyses of performance.
To review the literature on use of multilevel analyses of variability in quality.
Systematic literature review of English-language articles (n = 39) examining variability and reliability of performance measures in Medline using PubMed (1949-November 2008).
Variation was most commonly assessed at facility (eg, hospital, medical center) (n = 19) and physician (n = 18) levels; most articles reported variability as the proportion of total variation attributable to given levels (n = 22). Proportions of variability explained by aggregated levels were generally low (eg, <19% for physicians), and numerous authors concluded that the proportion of variability at a specific level did not justify targeting quality interventions to that level. Few articles based their recommendations on absolute differences among physicians, hospitals, or other levels. Seven of 12 articles that assessed reliability found that reliability was poor at the physician or hospital level due to low proportional variability and small sample sizes per unit, and cautioned that public reporting or incentives based on these measures may be inappropriate.
The proportion of variability at levels higher than patients is often found to be "low." Although low proportional variability may lead to poor measurement reliability, a number of authors further suggested that it also indicates a lack of potential for quality improvement. Few studies provided additional information to help determine whether variation was, nevertheless, clinically meaningful.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Greater insight into the effectiveness of usual dietetic care will contribute to the ongoing development of dietetic services. The present study examined the change in body mass index (BMI) in overweight patients after dietetic treatment in primary care, the sources of variability and factors associated with BMI change. This population-based observational study was based on data from a Dutch registration network of dietitians in primary health care. Data were derived from electronic medical records concerning 3960 overweight adult patients (BMI ≥ 25 kg m(-2) ) who received usual care from 32 registered dietitians between 2006 and 2012. Multilevel linear regression analyses were conducted. Patients' BMI significantly (P < 0.001) decreased by 0.94 kg m(-2) on average during treatment. An additional reduction of 0.8 kg m(-2) was observed in patients treated for longer than 6 months. BMI decreased by 0.06 kg m(-2) for each additional unit in initial BMI above 31.6. Most (97%) variability in BMI change was attributed to patients and 3% to dietitians. Part of the variance between patients (11%) and dietitians (30%) was explained by patient sociodemographic characteristics, nutrition-related health aspects, initial body weight and treatment duration. Dietetic treatment in primary care lowers BMI in overweight patients. Patients' change in BMI was rather similar between dietitians. Greater BMI reductions were observed in those with a high initial BMI and those treated for at least 6 months. Future research is necessary to study the long-term effects of weight loss after treatment by primary healthcare dietitians, especially because many patients drop out of treatment prematurely.
    Journal of Human Nutrition and Dietetics 11/2013; DOI:10.1111/jhn.12175 · 2.07 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: To determine the extent to which hospitals vary in the use of intensive care, and the proportion of variation attributable to differences in hospital practice that is independent of known patient and hospital factors. Hospital discharge data in the State Inpatient Database for Maryland and Washington States in 2006. Cross-sectional analysis of 90 short-term, acute care hospitals with critical care capabilities. DATA COLLECTION/METHODS: We quantified the proportion of variation in intensive care use attributable to hospitals using intraclass correlation coefficients derived from mixed-effects logistic regression models after successive adjustment for known patient and hospital factors. The proportion of hospitalized patients admitted to an intensive care unit (ICU) across hospitals ranged from 3 to 55 percent (median 12 percent; IQR: 9, 17 percent). After adjustment for patient factors, 19.7 percent (95 percent CI: 15.1, 24.4) of total variation in ICU use across hospitals was attributable to hospitals. When observed hospital characteristics were added, the proportion of total variation in intensive care use attributable to unmeasured hospital factors decreased by 26-14.6 percent (95 percent CI: 11, 18.3 percent). Wide variability exists in the use of intensive care across hospitals, not attributable to known patient or hospital factors, and may be a target to improve efficiency and quality of critical care.
    Health Services Research 10/2012; 47(5):2060-80. DOI:10.1111/j.1475-6773.2012.01402.x · 2.49 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Variability in processes of care and outcomes has been reported widely in high-income settings (at geographic, hospital, physician group and individual physician levels); however, such variability and the factors driving it are rarely examined in low-income settings. Using data from a cross-sectional survey undertaken in 22 hospitals (60 case records from each hospital) across Kenya that aimed at evaluating the quality of routine hospital services, we sought to explore variability in four binary inpatient paediatric process indicators. These included three prescribing tasks and use of one diagnostic. To examine for sources of variability, we examined intra-class correlation coefficients (ICC) and their changes using multi-level mixed models with random intercepts for hospital and clinician levels and adjusting for patient and clinician level covariates. Levels of performance varied substantially across indicators and hospitals. The absolute values for ICCs also varied markedly ranging from a maximum of 0.48 to a minimum of 0.09 across the models for HIV testing and prescription of zinc, respectively. More variation was attributable at the hospital level than clinician level after allowing for nesting of clinicians within hospitals for prescription of quinine loading dose for malaria (ICC = 0.30), prescription of zinc for diarrhoea patients (ICC = 0.11) and HIV testing for all children (ICC = 0.43). However, for prescription of correct dose of crystalline penicillin, more of the variability was explained by the clinician level (ICC = 0.21). Adjusting for clinician and patient level covariates only altered, marginally, the ICCs observed in models for the zinc prescription indicator. Performance varied greatly across place and indicator. The variability that could be explained suggests interventions to improve performance might be best targeted at hospital level factors for three indicators and clinician factors for one. Our data suggest that better understanding of performance and sources of variation might help tailor improvement interventions although further data across a larger set of indicators and sites would help substantiate these findings.
    Implementation Science 04/2015; 10(1):60. DOI:10.1186/s13012-015-0245-x · 3.47 Impact Factor


Available from