Hospital performance reports based on severity adjusted mortality rates in patients with cirrhosis depend on the method of risk adjustment

Liver Unit, Division of Gastroenterology and Hepatology, Department of Medicine, University of Calgary, Alberta, Canada.
Annals of hepatology: official journal of the Mexican Association of Hepatology (Impact Factor: 2.07). 07/2012; 11(4):526-35.
Source: PubMed


Hospital outcome report cards are used to judge provider performance, including for liver transplantation. We aimed to determine the impact of the choice of risk adjustment method on hospital rankings based on mortality rates in cirrhotic patients.
We identified 68,426 cirrhotic patients hospitalized in the Nationwide Inpatient Sample database. Four risk adjustment methods (the Charlson/Deyo and Elixhauser algorithms, Disease Staging, and All Patient Refined Diagnosis Related Groups) were used in logistic regression models for mortality. Observed to expected (O/E) death rates were calculated for each method and hospital. Statistical outliers with higher or lower than expected mortality were identified and rankings compared across methods.
Unadjusted mortality rates for the 553 hospitals ranged from 1.4 to 30% (overall, 10.6%). For 163 hospitals (29.5%), observed mortality differed significantly from expected when judged by one or more, but not all four, risk adjustment methods (25.9% higher than expected mortality and 3.6% lower than expected mortality). Only 28% of poor performers and 10% of superior performers were consistently ranked as such by all methods. Agreement between methods as to whether hospitals were flagged as outliers was moderate (kappa 0.51-0.59), except the Charlson/Deyo and Elixhauser algorithms which demonstrated excellent agreement (kappa 0.75).
Hospital performance reports for patients with cirrhosis require sensitivity to the method of risk adjustment. Depending upon the method, up to 30% of hospitals may be flagged as outliers by one, but not all methods. These discrepancies could have important implications for centers erroneously labeled as high mortality outliers.

Full-text preview

Available from:
  • [Show abstract] [Hide abstract]
    ABSTRACT: The aim of the study is to assess the validity of three measures of illness severity (prior year's hospital expenditures, Charlson and Elixhauser indices), by analysing the effect of introducing report cards on hospitals treating patients with acute myocardial infarction (AMI). Medicare claims data were obtained for 1992-1997 for AMI patients aged 65+. We used differences-in-differences regression analysis to assess the impact of report cards introduced in New Jersey and Pennsylvania on the illness severity of AMI patients with and without coronary artery bypass graft (CABG) surgery (relative to states without report cards). The analysis was conducted at the hospital level. For validation we used raw mortality and re-admission trends for AMI patients. While prior hospital expenditures suggest a considerable change in the illness severity of AMI patients in Pennsylvania relative to other states, raw mortality and re-admission trends in Pennsylvania are relatively consistent with the trend in the rest of the USA. In line with raw mortality and re-admission data, the Charlson and Elixhauser indices do not imply a considerable change in the severity of AMI patients in Pennsylvania. For CABG patients, illness severity - as measured by all three severity measurement methods - decreased after introduction of report cards, particularly in Pennsylvania. In conclusion, for AMI patients the Charlson and Elixhauser indices are a more valid measure of illness severity than prior year's hospital expenditures. After report cards were introduced, healthier AMI patients were more likely to receive CABG surgery, while sicker patients were avoided.
    Health Services Management Research 08/2012; 25(3):138-43. DOI:10.1258/hsmr.2012.012025
  • [Show abstract] [Hide abstract]
    ABSTRACT: International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes capture comorbidities that can be used to risk adjust nonrandom patient groups. We explored the accuracy of capturing comorbidities associated with one risk adjustment method, the Elixhauser Comorbidity Measure (ECM), in patients with chronic heart failure (CHF) at one Veterans Affairs (VA) medical center. We explored potential reasons for the differences found between the original codes assigned and conditions found through retrospective review. This descriptive, retrospective study used a cohort of patients discharged with a principal diagnosis coded as CHF from one VA medical center in 2003. One admission per patient was used in the study; with multiple admissions, only the first admission was analyzed. We compared the assignment of original codes assigned to conditions found in a retrospective, manual review of the medical record conducted by an investigator with coding expertise as well as by physicians. Members of the team experienced with assigning ICD-9-CM codes and VA coding processes developed themes related to systemic reasons why chronic conditions were not coded in VA records using applied thematic techniques. In the 181-patient cohort, 388 comorbid conditions were identified; 305 of these were chronic conditions, originally coded at the time of discharge with an average of 1.7 comorbidities related to the ECM per patient. The review by an investigator with coding expertise revealed a total of 937 comorbidities resulting in 618 chronic comorbid conditions with an average of 3.4 per patient; physician review found 872 total comorbidities with 562 chronic conditions (average 3.1 per patient). The agreement between the original and the retrospective coding review was 88 percent. The kappa statistic for the original and the retrospective coding review was 0.375 with a 95 percent confidence interval (CI) of 0.352 to 0.398. The kappa statistic for the retrospective coding review and physician review was 0.849 (CI, 0.823-0.875). The kappa statistic for the original coding and the physician review was 0.340 (CI, 0.316-0.364). Several systemic factors were identified, including familiarity with inpatient VA and non-VA guidelines, the quality of documentation, and operational requirements to complete the coding process within short time frames and to identify the reasons for movement within a given facility. Comorbidities within the ECM representing chronic conditions were significantly underrepresented in the original code assignment. Contributing factors potentially include prioritization of codes related to acute conditions over chronic conditions; coders' professional training, educational level, and experience; and the limited number of codes allowed in initial coding software. This study highlights the need to evaluate systemic causes of underrepresentation of chronic conditions to improve the accuracy of risk adjustment used for health services research, resource allocation, and performance measurement.
    Perspectives in health information management / AHIMA, American Health Information Management Association 10/2013; 10:1b.
  • [Show abstract] [Hide abstract]
    ABSTRACT: Background & Aims: Little is known about geographic variations in health care for patients with cirrhosis. We studied geographic and hospital-level variations in care of patients with cirrhosis in the United States by using inpatient mortality as an outcome for comparing hospitals. We also aimed to identify features of patients and hospitals associated with lower mortality. Methods: We used the 2009 U.S. Nationwide Inpatient Sample to identify patients with cirrhosis, which were based on the International Classification of Diseases, 9th Revision-Clinical Modification diagnosis codes for cirrhosis or 1 of its complications (ascites, hepatorenal syndrome, upper gastrointestinal bleeding, portal hypertension, or hepatic encephalopathy). Multilevel modeling was performed to measure variance among hospitals. Results: There were 102,155 admissions for cirrhosis in 2009, compared with 74,417 in 2003. Overall inpatient mortality was 6.6%. On multivariable-adjusted logistic regression, patients hospitalized in the Midwest had the lowest odds ratio (OR) of inpatient mortality (OR, 0.54; P < .001). Patients who were transferred from other hospitals (OR, 1.49; P < .001) or had hepatic encephalopathy (OR, 1.28; P < .001), upper gastrointestinal bleeding (OR, 1.74; P < .001), or alcoholic liver disease (OR, 1.23; P= .03) had higher odds of inpatient mortality than patients without these features. Those who received liver transplants had substantially lower odds of inpatient mortality (OR, 0.21; P < .001). Multilevel modeling showed that 4% of the variation in mortality could be accounted for at the hospital level (P < .001). Adjusted mortality among hospitals ranged from 1.2% to 14.2%. Conclusions: Inpatient cirrhosis mortality varies considerably among U.S. hospitals. Further research is needed to identify hospital-level and provider-level practices that could be modified to improve outcomes.
    Clinical Gastroenterology and Hepatology 09/2014; 13(3). DOI:10.1016/j.cgh.2014.09.038 · 7.90 Impact Factor
Show more