Empirically Derived Composite Measures of Surgical Performance

Department of Economics and the Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth College, Hanover, New Hampshire, USA.
Medical care (Impact Factor: 3.23). 03/2009; 47(2):226-33. DOI: 10.1097/MLR.0b013e3181847574
Source: PubMed


Individual quality measures have significant limitations for assessing surgical performance. Despite growing interest in composite measures, empirically-based methods for combining multiple domains of surgical quality are not well established.
To develop and validate a composite measure of surgical performance that best describes variation in hospital mortality rates and forecasts future performance.
Using the national Medicare claims database, we identified all patients undergoing aortic valve replacement in 2000 to 2001 (n = 53,120). To serve as input variables, we identified hospital-level predictors of mortality with aortic valve replacement, including hospital volume, complication rates, and mortality with other procedures. Hospital-specific predicted mortality rates were then determined using Bayesian-derived modeling techniques and assessed against subsequent hospital mortality (2002-2003).
Our composite measure explained 78% of the variation in aortic valve replacement mortality rates (2000-2001). The most important input variables were hospital volume, mortality with aortic valve replacement, and mortality for other high-risk cardiac procedures. The composite measure forecasted 70% of future hospital-level variation in mortality rates (2002-2003), and was substantially better in this regard than individual measures. Hospitals scoring in the bottom quintile on the composite measure in 2000 to 2001 had 2-fold higher mortality rates in 2002 to 2003 than hospitals in the top quintile (adjusted odds ratio, 1.97; 95% CI, 1.73-2.23).
Compared with individual surgical quality indicators, empirically derived composite measures are superior in explaining variation in hospital mortality rates and in forecasting future performance. Such measures could be useful for public reporting, value-based purchasing, or benchmarking for quality improvement purposes.

30 Reads
  • Source
    • "Hospital quality was characterized by a previously validated composite score, which combines operational mortality with hospital volume information using Bayesian techniques [18]. Hospital procedure volume was assessed as the total number of procedures performed by each hospital. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Introduction To assess excess use of coronary angiography prior to coronary artery bypass graft surgery and its association with mortality, health care costs, and hospital quality in Turkey. Methods Using Turkish National Health Insurance Data (2009–2011) that included patients who underwent cardiac surgery, coronary angiography utilization was identified. Propensity score matching was used to compare survival rates and annual health care costs of patients in a coronary angiography excess-use group (>1 angiogram) and in a standard-therapy group (1 angiogram). The empirical Bayesian approach was used to combine mortality and hospital volume for quality index. The relationship between hospital quality and excess use of coronary angiography was assessed using Chi-squared tests. Results Out of 20,126 patients identified, 7.27% of patients underwent excessive coronary angiography procedures (excess-use group), with an average annual cost at 9.7% higher than those who had a single angiography (standard-therapy group; P < 0.01). Operational mortality associated with excessive use was significantly higher as well (7.4% versus 5.4%, P < 0.02). There exists variation in the use of coronary angiography across cities and hospitals. Patients who underwent cardiac surgery in high-quality hospitals were less likely to have excessive angiography use than those in low-quality hospitals (7.0% versus 9.5%, P < 0.01). Conclusion In Turkey, excess use of coronary angiography prior to coronary artery bypass graft surgery is associated with higher operational mortality, higher expenditures, and lower hospital quality.
    12/2013; 2(2). DOI:10.1007/s40119-013-0018-z
  • Source
    • "variance . We first used an " empty model " that contained only patient variables for risk adjustment . We then entered each historical quality measure ( assessed in 2005 – 2006 ) into the model . We then calculated the degree to which the historical quality measures reduced the hospital - level variance , an approach described in our prior work ( Staiger et al . 2009 ) . All statistical analy - ses were conducted using STATA 10 . 0 ( College Station , Texas ) ."
    [Show abstract] [Hide abstract]
    ABSTRACT: To assess the value of a novel composite measure for identifying the best hospitals for major procedures. We used national Medicare data for patients undergoing five high-risk surgical procedures between 2005 and 2008. For each procedure, we used empirical Bayes techniques to create a composite measure combining hospital volume, risk-adjusted mortality with the procedure of interest, risk-adjusted mortality with other related procedures, and other variables. Hospitals were ranked based on 2005-2006 data and placed in one of three groups: 1-star (bottom 20 percent), 2-star (middle 60 percent), and 3-star (top 20 percent). We assessed how well these ratings forecasted risk-adjusted mortality rates in the next 2 years (2007-2008), compared to other measures. For all five procedures, the composite measures based on 2005-2006 data performed well in predicting future hospital performance. Compared to 1-star hospitals, risk-adjusted mortality was much lower at 3-star hospitals for esophagectomy (6.7 versus 14.4 percent), pancreatectomy (4.7 versus 9.2 percent), coronary artery bypass surgery (2.6 versus 5.0 percent), aortic valve replacement (4.5 versus 8.5 percent), and percutaneous coronary interventions (2.4 versus 4.1 percent). Compared to individual surgical quality measures, the composite measures were better at forecasting future risk-adjusted mortality. These measures also outperformed the Center for Medicare and Medicaid Services (CMS) Hospital Compare ratings. Composite measures of surgical quality are very effective at predicting hospital mortality rates with major procedures. Such measures would be more informative than existing quality indicators in helping patients and payers identify high-quality hospitals with specific procedures.
    Health Services Research 10/2012; 47(5):1861-79. DOI:10.1111/j.1475-6773.2012.01407.x · 2.78 Impact Factor
  • Source
    • "Individual quality measures have significant limitations for assessing performance. Despite growing interest in composite measures, methods for combining multiple domains of surgical quality are not well established [37]. Composite scores are an aggregation of underlying performance indicators into a single index and have been used widely in the public sector to create one more easily understood number or rating [38,39]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Although performance measurement for assessing care quality is an emerging area, a system for measuring the quality of cancer care at the hospital level has not been well developed. The purpose of this study was to develop organization-based core measures for colorectal cancer patient care and apply these measures to compare hospital performance. The development of core measures for colorectal cancer has undergone three stages including a modified Delphi method. The study sample originated from 2004 data in the Taiwan Cancer Database, a national cancer data registry. Eighteen hospitals and 5585 newly diagnosed colorectal cancer patients were enrolled in this study. We used indicator-based and case-based approaches to examine adherences simultaneously. The final core measure set included seventeen indicators (1 pre-treatment, 11 treatment-related and 5 monitoring-related). There were data available for ten indicators. Indicator-based adherence possesses more meaningful application than case-based adherence for hospital comparisons. Mean adherence was 85.8% (79.8% to 91%) for indicator-based and 82.8% (77.6% to 88.9%) for case-based approaches. Hospitals performed well (>90%) for five out of eleven indicators. Still, the performance across hospitals varied for many indicators. The best and poorest system performance was reflected in indicators T5-negative surgical margin (99.3%, 97.2%-100.0%) and T7-lymph nodes harvest more than twelve(62.7%, 27.6%-92.2%), both of which related to surgical specimens. In this nationwide study, quality of colorectal cancer care still shows room for improvement. These preliminary results indicate that core measures for cancer can be developed systematically and applied for internal quality improvement.
    BMC Health Services Research 01/2010; 10(1):27. DOI:10.1186/1472-6963-10-27 · 1.71 Impact Factor
Show more