Empirically Derived Composite Measures of Surgical Performance

Department of Economics and the Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth College, Hanover, New Hampshire, USA.
Medical care (Impact Factor: 3.23). 03/2009; 47(2):226-33. DOI: 10.1097/MLR.0b013e3181847574
Source: PubMed


Individual quality measures have significant limitations for assessing surgical performance. Despite growing interest in composite measures, empirically-based methods for combining multiple domains of surgical quality are not well established.
To develop and validate a composite measure of surgical performance that best describes variation in hospital mortality rates and forecasts future performance.
Using the national Medicare claims database, we identified all patients undergoing aortic valve replacement in 2000 to 2001 (n = 53,120). To serve as input variables, we identified hospital-level predictors of mortality with aortic valve replacement, including hospital volume, complication rates, and mortality with other procedures. Hospital-specific predicted mortality rates were then determined using Bayesian-derived modeling techniques and assessed against subsequent hospital mortality (2002-2003).
Our composite measure explained 78% of the variation in aortic valve replacement mortality rates (2000-2001). The most important input variables were hospital volume, mortality with aortic valve replacement, and mortality for other high-risk cardiac procedures. The composite measure forecasted 70% of future hospital-level variation in mortality rates (2002-2003), and was substantially better in this regard than individual measures. Hospitals scoring in the bottom quintile on the composite measure in 2000 to 2001 had 2-fold higher mortality rates in 2002 to 2003 than hospitals in the top quintile (adjusted odds ratio, 1.97; 95% CI, 1.73-2.23).
Compared with individual surgical quality indicators, empirically derived composite measures are superior in explaining variation in hospital mortality rates and in forecasting future performance. Such measures could be useful for public reporting, value-based purchasing, or benchmarking for quality improvement purposes.

30 Reads
  • Source
    • "Hospital quality was characterized by a previously validated composite score, which combines operational mortality with hospital volume information using Bayesian techniques [18]. Hospital procedure volume was assessed as the total number of procedures performed by each hospital. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Introduction To assess excess use of coronary angiography prior to coronary artery bypass graft surgery and its association with mortality, health care costs, and hospital quality in Turkey. Methods Using Turkish National Health Insurance Data (2009–2011) that included patients who underwent cardiac surgery, coronary angiography utilization was identified. Propensity score matching was used to compare survival rates and annual health care costs of patients in a coronary angiography excess-use group (>1 angiogram) and in a standard-therapy group (1 angiogram). The empirical Bayesian approach was used to combine mortality and hospital volume for quality index. The relationship between hospital quality and excess use of coronary angiography was assessed using Chi-squared tests. Results Out of 20,126 patients identified, 7.27% of patients underwent excessive coronary angiography procedures (excess-use group), with an average annual cost at 9.7% higher than those who had a single angiography (standard-therapy group; P < 0.01). Operational mortality associated with excessive use was significantly higher as well (7.4% versus 5.4%, P < 0.02). There exists variation in the use of coronary angiography across cities and hospitals. Patients who underwent cardiac surgery in high-quality hospitals were less likely to have excessive angiography use than those in low-quality hospitals (7.0% versus 9.5%, P < 0.01). Conclusion In Turkey, excess use of coronary angiography prior to coronary artery bypass graft surgery is associated with higher operational mortality, higher expenditures, and lower hospital quality.
    Full-text · Article · Dec 2013
  • Source
    • "In other fields, such as the fields of medical care and education, adjustments of performance measures are sometimes applied as well. Typical performance measures that are adjusted are of a logistic nature, like mortality rates in hospitals (e.g., Drösler et al., 2012; Silber, Rosenbaum & Ross, 1995; Landon et al., 1996; Staiger et al., 2009). These statistical adjustments are performed with the use of prior evidence on the relation between the performance measure and certain selected factors. "

    Full-text · Article · Sep 2013
  • Source
    • "variance . We first used an " empty model " that contained only patient variables for risk adjustment . We then entered each historical quality measure ( assessed in 2005 – 2006 ) into the model . We then calculated the degree to which the historical quality measures reduced the hospital - level variance , an approach described in our prior work ( Staiger et al . 2009 ) . All statistical analy - ses were conducted using STATA 10 . 0 ( College Station , Texas ) ."
    [Show abstract] [Hide abstract]
    ABSTRACT: To assess the value of a novel composite measure for identifying the best hospitals for major procedures. We used national Medicare data for patients undergoing five high-risk surgical procedures between 2005 and 2008. For each procedure, we used empirical Bayes techniques to create a composite measure combining hospital volume, risk-adjusted mortality with the procedure of interest, risk-adjusted mortality with other related procedures, and other variables. Hospitals were ranked based on 2005-2006 data and placed in one of three groups: 1-star (bottom 20 percent), 2-star (middle 60 percent), and 3-star (top 20 percent). We assessed how well these ratings forecasted risk-adjusted mortality rates in the next 2 years (2007-2008), compared to other measures. For all five procedures, the composite measures based on 2005-2006 data performed well in predicting future hospital performance. Compared to 1-star hospitals, risk-adjusted mortality was much lower at 3-star hospitals for esophagectomy (6.7 versus 14.4 percent), pancreatectomy (4.7 versus 9.2 percent), coronary artery bypass surgery (2.6 versus 5.0 percent), aortic valve replacement (4.5 versus 8.5 percent), and percutaneous coronary interventions (2.4 versus 4.1 percent). Compared to individual surgical quality measures, the composite measures were better at forecasting future risk-adjusted mortality. These measures also outperformed the Center for Medicare and Medicaid Services (CMS) Hospital Compare ratings. Composite measures of surgical quality are very effective at predicting hospital mortality rates with major procedures. Such measures would be more informative than existing quality indicators in helping patients and payers identify high-quality hospitals with specific procedures.
    Preview · Article · Oct 2012 · Health Services Research
Show more