Predicting death: An empirical evaluation of predictive tools for mortality

Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
Archives of internal medicine (Impact Factor: 17.33). 07/2011; 171(19):1721-6. DOI: 10.1001/archinternmed.2011.334
Source: PubMed


The ability to predict death is crucial in medicine, and many relevant prognostic tools have been developed for application in diverse settings. We aimed to evaluate the discriminating performance of predictive tools for death and the variability in this performance across different clinical conditions and studies.
We used Medline to identify studies published in 2009 that assessed the accuracy (based on the area under the receiver operating characteristic curve [AUC]) of validated tools for predicting all-cause mortality. For tools where accuracy was reported in 4 or more assessments, we calculated summary accuracy measures. Characteristics of studies of the predictive tools were evaluated to determine if they were associated with the reported accuracy of the tool.
A total of 94 eligible studies provided data on 240 assessments of 118 predictive tools. The AUC ranged from 0.43 to 0.98 (median [interquartile range], 0.77 [0.71-0.83]), with only 23 of the assessments reporting excellent discrimination (10%) (AUC, >0.90). For 10 tools, accuracy was reported in 4 or more assessments; only 1 tool had a summary AUC exceeding 0.80. Established tools showed large heterogeneity in their performance across different cohorts (I(2) range, 68%-95%). Reported AUC was higher for tools published in journals with lower impact factor (P = .01), with larger sample size (P = .01), and for those that aimed to predict mortality among the highest-risk patients (P = .002) and among children (P < .001).
Most tools designed to predict mortality have only modest accuracy, and there is large variability across various diseases and populations. Most proposed tools do not have documented clinical utility.

Download full-text


Available from: George CM Siontis,
11 Reads
  • Source
    • "ICU) settings constitute a broad area of research. Siontis et al. [16] reviewed 94 studies with 240 assessments of 118 mortality prediction tools from 2009 alone. Many of these studies evaluated established clinical decision rules for predicting mortality, such as APACHE [9], SAPS-II [10], and SOFA [17] (with median reported AUCs of 0.77, 0.77, and 0.84, respectively). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Accurate knowledge of a patient's disease state and trajectory is critical in a clinical setting. Modern electronic healthcare records contain an increasingly large amount of data, and the ability to automatically identify the factors that influence patient outcomes stand to greatly improve the efficiency and quality of care. We examined the use of latent variable models (viz. Latent Dirichlet Allocation) to decompose free-text hospital notes into meaningful features, and the predictive power of these features for patient mortality. We considered three prediction regimes: (1) baseline prediction, (2) dynamic (time-varying) outcome prediction, and (3) retrospective outcome prediction. In each, our prediction task differs from the familiar time-varying situation whereby data accumulates; since fewer patients have long ICU stays, as we move forward in time fewer patients are available and the prediction task becomes increasingly difficult. We found that latent topic-derived features were effective in determining patient mortality under three timelines: inhospital, 30 day post-discharge, and 1 year post-discharge mortality. Our results demonstrated that the latent topic features important in predicting hospital mortality are very different from those that are important in post-discharge mortality. In general, latent topic features were more predictive than structured features, and a combination of the two performed best. The time-varying models that combined latent topic features and baseline features had AUCs that reached 0.85, 0.80, and 0.77 for in-hospital, 30 day post-discharge and 1 year post-discharge mortality respectively. Our results agreed with other work suggesting that the first 24 hours of patient information are often the most predictive of hospital mortality. Retrospective models that used a combination of latent topic features and structured features achieved AUCs of 0.96, 0.82, and 0.81 for in-hospital, 30 day, and 1-year mortality prediction. Our work focuses on the dynamic (time-varying) setting because models from this regime could facilitate an on-going severity stratification system that helps direct care-staff resources and inform treatment strategies.
  • Source
    • "Recently a Multidimensional Prognostic Index (MPI) derived from a standardized CGA has been developed and validated in several independent cohorts of hospitalized [8] and community-dwelling [9] elderly patients. The good accuracy and calibration of the MPI as predictive tool for mortality have been recently confirmed by independent reviews and meta-analysis [10] [11]. Moreover, the MPI demonstrated a significant higher predictive power for BioMed Research International short-and long-term all-cause mortality than other frailty instruments in a multicentre study on hospitalized older patients [12]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Aim: To identify the characteristics associated with multidimensional impairment, evaluated through the Multidimensional Prognostic Index (MPI), a validated predictive tool for mortality derived from a standardized Comprehensive Geriatric Assessment (CGA), in a cohort of elderly diabetic patients treated with oral hypoglycemic drugs. Methods and results: The study population consisted of 1342 diabetic patients consecutively enrolled in 57 diabetes centers distributed throughout Italy, within the Metabolic Study. Inclusion criteria were diagnosis of type 2 diabetes mellitus (DM), 65 years old or over, and treatment with oral antidiabetic medications. Data concerning DM duration, medications for DM taken during the 3-month period before inclusion in the study, number of hypoglycemic events, and complications of DM were collected. Multidimensional impairment was assessed using the MPI evaluating functional, cognitive, and nutritional status; risk of pressure sores; comorbidity; number of drugs taken; and cohabitation status. The mean age of participants was 73.3 ± 5.5 years, and the mean MPI score was 0.22 ± 0.13. Multivariate analysis showed that advanced age, female gender, hypoglycemic events, and hospitalization for glycemic decompensation were independently associated with a worse MPI score. Conclusion: Stratification of elderly diabetic patients using the MPI might help to identify those patients at highest risk who need better-tailored treatment.
    02/2014; 2014:906103. DOI:10.1155/2014/906103
  • Source
    • " Accepted for publication 28 February 2013 ª 2013 John Wiley & Sons Ltd and the Anatomical Society Assessment (Pilotto et al., 2008; Siontis et al., 2011; Yourman et al., 2012 "
    [Show abstract] [Hide abstract]
    ABSTRACT: A combination of several metabolic and hormonal adaptations has been proposed to control aging. Little is known regarding the effects of multiple deregulations of these metabolic and hormonal systems in modulating frailty and mortality in hospitalized elderly patients. We measured 17 biological serum parameters from different metabolic/hormonal pathways in 594 hospitalized elderly patients followed up to 1 year who were stratified into 3 groups according to their multidimensional impairment, evaluated by a comprehensive geriatric assessment (CGA)-based Multidimensional Prognostic Index (MPI). The mortality incidence rates were 7% at 1-month and 21% at 1-year. Our data show that frailty and mortality rate were positively associated with chronic inflammation, and with a down-regulation of multiple endocrine factors. Of the 17 biomarkers examined, blood levels of IGF-1, triiodothyronine, C-reactive protein, erythrocyte sedimentation rate, white blood cell and lymphocyte counts, iron, albumin, total cholesterol, and LDL-c were significantly associated with both MPI severity grade and mortality. In multivariate Cox proportional-hazard model, the following biomarkers most strongly predicted the risk of mortality (adjusted hazard ratio (HR) per 1 quintile increment in predictor distribution): IGF-1 HR=0.71 (95%CI: 0.63-0.80), CRP HR=1.48 (95%CI: 1.32-1.65), hemoglobin HR=0.82 (95%CI: 0.73-0.92) and glucose HR=1.17 (95%CI: 1.04-1.30). Multidimensional impairment assessed by MPI is associated with a distinctive metabolic "signature". The concomitant elevation of markers of inflammation, associated with a simultaneous reduction of multiple metabolic and hormonal factors predicts mortality in hospitalized elderly patients. © 2013 The Authors Aging Cell © 2013 Blackwell Publishing Ltd/Anatomical Society of Great Britain and Ireland.
    Aging cell 03/2013; 12(3). DOI:10.1111/acel.12068 · 6.34 Impact Factor
Show more