Meaningful Variation in Performance: A Systematic Literature Review

Division of Research, Kaiser Permanente Medical Care Program, Oakland, CA, USA.
Medical care (Impact Factor: 3.23). 02/2010; 48(2):140-8. DOI: 10.1097/MLR.0b013e3181bd4dc3
Source: PubMed


Recommendations for directing quality improvement initiatives at particular levels (eg, patients, physicians, provider groups) have been made on the basis of empirical components of variance analyses of performance.
To review the literature on use of multilevel analyses of variability in quality.
Systematic literature review of English-language articles (n = 39) examining variability and reliability of performance measures in Medline using PubMed (1949-November 2008).
Variation was most commonly assessed at facility (eg, hospital, medical center) (n = 19) and physician (n = 18) levels; most articles reported variability as the proportion of total variation attributable to given levels (n = 22). Proportions of variability explained by aggregated levels were generally low (eg, <19% for physicians), and numerous authors concluded that the proportion of variability at a specific level did not justify targeting quality interventions to that level. Few articles based their recommendations on absolute differences among physicians, hospitals, or other levels. Seven of 12 articles that assessed reliability found that reliability was poor at the physician or hospital level due to low proportional variability and small sample sizes per unit, and cautioned that public reporting or incentives based on these measures may be inappropriate.
The proportion of variability at levels higher than patients is often found to be "low." Although low proportional variability may lead to poor measurement reliability, a number of authors further suggested that it also indicates a lack of potential for quality improvement. Few studies provided additional information to help determine whether variation was, nevertheless, clinically meaningful.

30 Reads
  • Source
    • "Furthermore, the results show that 3% of the variance in patients' mean BMI change can be attributed to differences between dietitians. This is relatively low compared to other studies in primary health care (Fung et al., 2010). Low proportional variances have often been interpreted as indicating little potential for quality improvement efforts. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Greater insight into the effectiveness of usual dietetic care will contribute to the ongoing development of dietetic services. The present study examined the change in body mass index (BMI) in overweight patients after dietetic treatment in primary care, the sources of variability and factors associated with BMI change. This population-based observational study was based on data from a Dutch registration network of dietitians in primary health care. Data were derived from electronic medical records concerning 3960 overweight adult patients (BMI ≥ 25 kg m(-2) ) who received usual care from 32 registered dietitians between 2006 and 2012. Multilevel linear regression analyses were conducted. Patients' BMI significantly (P < 0.001) decreased by 0.94 kg m(-2) on average during treatment. An additional reduction of 0.8 kg m(-2) was observed in patients treated for longer than 6 months. BMI decreased by 0.06 kg m(-2) for each additional unit in initial BMI above 31.6. Most (97%) variability in BMI change was attributed to patients and 3% to dietitians. Part of the variance between patients (11%) and dietitians (30%) was explained by patient sociodemographic characteristics, nutrition-related health aspects, initial body weight and treatment duration. Dietetic treatment in primary care lowers BMI in overweight patients. Patients' change in BMI was rather similar between dietitians. Greater BMI reductions were observed in those with a high initial BMI and those treated for at least 6 months. Future research is necessary to study the long-term effects of weight loss after treatment by primary healthcare dietitians, especially because many patients drop out of treatment prematurely.
    Journal of Human Nutrition and Dietetics 11/2013; 27(5). DOI:10.1111/jhn.12175 · 1.99 Impact Factor
  • Source
    • "ion guidelines for secondary prevention of myocardial infarction,(Bennett-Guerrero et al. 2010; Huesch 2011; Rasmussen et al. 2008; Rogers et al. 2009) and up to 18% of the variation in the quality of care for acute stroke.(Reeves et al. 2010) These hospital level effects also exceed the proportion of variation generally attributable to providers.(Fung et al. 2010) Importantly, as in other areas of medicine, this degree of variance attribution may be responsive to targeted efforts to reduce practice variability.(Selby et al. 2010) Although we are unable to determine whether intensive care was over-or under-utilized within hospitals, it seems very unlikely that this degree of variation can be expl"
    [Show abstract] [Hide abstract]
    ABSTRACT: To determine the extent to which hospitals vary in the use of intensive care, and the proportion of variation attributable to differences in hospital practice that is independent of known patient and hospital factors. Hospital discharge data in the State Inpatient Database for Maryland and Washington States in 2006. Cross-sectional analysis of 90 short-term, acute care hospitals with critical care capabilities. DATA COLLECTION/METHODS: We quantified the proportion of variation in intensive care use attributable to hospitals using intraclass correlation coefficients derived from mixed-effects logistic regression models after successive adjustment for known patient and hospital factors. The proportion of hospitalized patients admitted to an intensive care unit (ICU) across hospitals ranged from 3 to 55 percent (median 12 percent; IQR: 9, 17 percent). After adjustment for patient factors, 19.7 percent (95 percent CI: 15.1, 24.4) of total variation in ICU use across hospitals was attributable to hospitals. When observed hospital characteristics were added, the proportion of total variation in intensive care use attributable to unmeasured hospital factors decreased by 26-14.6 percent (95 percent CI: 11, 18.3 percent). Wide variability exists in the use of intensive care across hospitals, not attributable to known patient or hospital factors, and may be a target to improve efficiency and quality of critical care.
    Health Services Research 10/2012; 47(5):2060-80. DOI:10.1111/j.1475-6773.2012.01402.x · 2.78 Impact Factor
  • Source
    • "Secondly, given that quality improvement activities take place throughout the hospital, the relationship between hospital-wide and departmental-specific quality improvement activities should be explored. Considering that departmental level activities are more proximal to outcomes, it might be possible to detect stronger associations and prevent attenuation of within-hospital variations in quality outcomes [40]. Thirdly, assessments of outcomes should be accompanied by assessments of clinical processes against evidence-based standards for effective, safe and patient-centred care. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Previous research addressed the development of a classification scheme for quality improvement systems in European hospitals. In this study we explore associations between the 'maturity' of the hospitals' quality improvement system and clinical outcomes. The maturity classification scheme was developed based on survey results from 389 hospitals in eight European countries. We matched the hospitals from the Spanish sample (113 hospitals) with those hospitals participating in a nation-wide, voluntary hospital performance initiative. We then compared sample distributions and explored associations between the 'maturity' of the hospitals' quality improvement system and a range of composite outcomes measures, such as adjusted hospital-wide mortality, -readmission, -complication and -length of stay indices. Statistical analysis includes bivariate correlations for parametrically and non-parametrically distributed data, multiple robust regression models and bootstrapping techniques to obtain confidence-intervals for the correlation and regression estimates. Overall, 43 hospitals were included. Compared to the original sample of 113, this sample was characterized by a higher representation of university hospitals. Maturity of the quality improvement system was similar, although the matched sample showed less variability. Analysis of associations between the quality improvement system and hospital-wide outcomes suggests significant correlations for the indicator adjusted hospital complications, borderline significance for adjusted hospital readmissions and non-significance for the adjusted hospital mortality and length of stay indicators. These results are confirmed by the bootstrap estimates of the robust regression model after adjusting for hospital characteristics. We assessed associations between hospitals' quality improvement systems and clinical outcomes. From this data it seems that having a more developed quality improvement system is associated with lower rates of adjusted hospital complications. A number of methodological and logistic hurdles remain to link hospital quality improvement systems to outcomes. Further research should aim at identifying the latent dimensions of quality improvement systems that predict quality and safety outcomes. Such research would add pertinent knowledge regarding the implementation of organizational strategies related with quality of care outcomes.
    BMC Health Services Research 12/2011; 11:344. DOI:10.1186/1472-6963-11-344 · 1.71 Impact Factor
Show more


30 Reads
Available from