Validation of a Risk Stratification Index and Risk Quantification Index for Predicting Patient Outcomes

ArticleinAnesthesiology 119(3):525-40 · September 2013with11 Reads
Impact Factor: 5.88 · DOI: 10.1097/ALN.0b013e31829ce6e6 · Source: PubMed


    External validation of published risk stratification models is essential to determine their generalizability. This study evaluates the performance of the Risk Stratification Indices (RSIs) and 30-day mortality Risk Quantification Index (RQI).
    108,423 adult hospital admissions with anesthetics were identified (2006–2011). RSIs for mortality and length-of-stay endpoints were calculated using published methodology. 91,128 adult, noncardiac inpatient surgeries were identified with administrative data required for RQI calculation.
    RSI in-hospital mortality and RQI 30-day mortality Brier scores were 0.308 and 0.017, respectively. RSI discrimination, by area under the receiver operating curves, was excellent at 0.966 (95% CI, 0.963–0.970) for in-hospital mortality, 0.903 (0.896–0.909) for 30-day mortality, 0.866 (0.861–0.870) for 1-yr mortality, and 0.884 (0.882–0.886) for length-of-stay. RSI calibration, however, was poor overall (17% predicted in-hospital mortality vs. 1.5% observed after inclusion of the regression constant) as demonstrated by calibration plots. Removal of self-fulfilling diagnosis and procedure codes (20,001 of 108,423; 20%) yielded similar results. RQIs were calculated for only 62,640 of 91,128 patients (68.7%) due to unmatched procedure codes. Patients with unmatched codes were younger, had higher American Society of Anesthesiologists physical status and 30-day mortality. The area under the receiver operating curve for 30-day mortality RQI was 0.888 (0.879–0.897). The model also demonstrated good calibration. Performance of a restricted index, Procedure Severity Score + American Society of Anesthesiologists physical status, performed as well as the original RQI model (age + American Society of Anesthesiologists + Procedure Severity Score).
    Although the RSIs demonstrated excellent discrimination, poor calibration limits their generalizability. The 30-day mortality RQI performed well with age providing a limited contribution.