Conference Paper

Progressive prediction of hospitalisation and patient disposition in the emergency department

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Hospitals face high occupation rates resulting in a longer boarding time and more complex bed management. This task could be facilitated by anticipating the unscheduled admissions. We study the capability of information from French electronic health records of an emergency department (ED) to predict patient disposition decisions. We compare the performances of five learning models in predicting the admission of a patient visiting an emergency department and in predicting the patient's place of admission at two progressive time points throughout the ED care process: triage and initial assessment. Medical and administrative data were retrospectively collected on 53,608 visits to the Groupe Hospitalier Bretagne Sud, France, from July 2020 to June 2021. Our best model achieve a ROC-AUC equal to 88% and F1-score equal to 75% for admission prediction. Regarding medical unit admission prediction, the global ROC-AUC equals to 87% and F1-score ranges from 38% to 77% for the four admission classes, i.e., intensive care unit (6% of the dataset), medicine units (45%), surgery units (13.2%), and observation unit (35.8%). A validation with a posterior dataset indicates constant results.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

Article
Background The optimization of patient care pathways is crucial for hospital managers in the context of a scarcity of medical resources. Assuming unlimited capacities, the pathway of a patient would only be governed by pure medical logic to meet at best the patient’s needs. However, logistical limitations (eg, resources such as inpatient beds) are often associated with delayed treatments and may ultimately affect patient pathways. This is especially true for unscheduled patients—when a patient in the emergency department needs to be admitted to another medical unit without disturbing the flow of planned hospitalizations. Objective In this study, we proposed a new framework to automatically detect activities in patient pathways that may be unrelated to patients’ needs but rather induced by logistical limitations. Methods The scientific contribution lies in a method that transforms a database of historical pathways with bias into 2 databases: a labeled pathway database where each activity is labeled as relevant (related to a patient’s needs) or irrelevant (induced by logistical limitations) and a corrected pathway database where each activity corresponds to the activity that would occur assuming unlimited resources. The labeling algorithm was assessed through medical expertise. In total, 2 case studies quantified the impact of our method of preprocessing health care data using process mining and discrete event simulation. Results Focusing on unscheduled patient pathways, we collected data covering 12 months of activity at the Groupe Hospitalier Bretagne Sud in France. Our algorithm had 87% accuracy and demonstrated its usefulness for preprocessing traces and obtaining a clean database. The 2 case studies showed the importance of our preprocessing step before any analysis. The process graphs of the processed data had, on average, 40% (SD 10%) fewer variants than the raw data. The simulation revealed that 30% of the medical units had >1 bed difference in capacity between the processed and raw data. Conclusions Patient pathway data reflect the actual activity of hospitals that is governed by medical requirements and logistical limitations. Before using these data, these limitations should be identified and corrected. We anticipate that our approach can be generalized to obtain unbiased analyses of patient pathways for other hospitals.
Article
Full-text available
Several approaches exist today for developing predictive models across multiple clinical sites, yet there is a lack of comparative data on their performance, especially within the context of EHR-based prediction models. We set out to provide a framework for prediction across healthcare settings. As a case study, we examined an ED disposition prediction model across three geographically and demographically diverse sites. We conducted a 1-year retrospective study, including all visits in which the outcome was either discharge-to-home or hospitalization. Four modeling approaches were compared: a ready-made model trained at one site and validated at other sites, a centralized uniform model incorporating data from all sites, multiple site-specific models, and a hybrid approach of a ready-made model re-calibrated using site-specific data. Predictions were performed using XGBoost. The study included 288,962 visits with an overall admission rate of 16.8% (7.9–26.9%). Some risk factors for admission were prominent across all sites (e.g., high-acuity triage emergency severity index score, high prior admissions rate), while others were prominent at only some sites (multiple lab tests ordered at the pediatric sites, early use of ECG at the adult site). The XGBoost model achieved its best performance using the uniform and site-specific approaches (AUC = 0.9–0.93), followed by the calibrated-model approach (AUC = 0.87–0.92), and the ready-made approach (AUC = 0.62–0.85). Our results show that site-specific customization is a key driver of predictive model performance.
Article
Full-text available
Objective: This paper presents a deep learning method of predicting where in a hospital emergency patients will be admitted after being triaged in the Emergency Department (ED). Such a prediction will allow for the preparation of bed space in the hospital for timely care and admission of the patient as well as allocation of resource to the relevant departments, including during periods of increased demand arising from seasonal peaks in infections. Methods: The problem is posed as a multi-class classification into seven separate ward types. A novel deep learning training strategy was created that combines learning via curriculum and a multi-armed bandit to exploit this curriculum post-initial training. Results: We successfully predict the initial hospital admission location with area-under-receiver-operating-curve (AUROC) ranging between 0.60 to 0.78 for the individual wards and an overall maximum accuracy of 52% where chance corresponds to 14% for this seven-class setting. Our proposed network was able to interpret which features drove the predictions using a 'network saliency' term added to the network loss function. Conclusion: We have proven that prediction of location of admission in hospital for emergency patients is possible using information from triage in ED. We have also shown that there are certain tell-tale tests which indicate what space of the hospital a patient will use. Significance: It is hoped that this predictor will be of value to healthcare institutions by allowing for the planning of resource and bed space ahead of the need for it. This in turn should speed up the provision of care for the patient and allow flow of patients out of the ED thereby improving patient flow and the quality of care for the remaining patients within the ED.
Article
Full-text available
Using artificial intelligence and machine learning techniques in different medical fields, especially emergency medicine is rapidly growing. In this paper, studies conducted in the recent years on using artificial intelligence in emergency medicine have been collected and assessed. These studies belonged to three categories: prediction and detection of disease; prediction of need for admission, discharge and also mortality; and machine learning based triage systems. In each of these categories, the most important studies have been chosen and accuracy and results of the algorithms have been briefly evaluated by mentioning machine learning techniques and used datasets.
Article
Full-text available
We investigate the capability of information from electronic health records of an emergency department (ED) to predict patient disposition decisions for reducing “boarding” delays through the proactive initiation of admission processes (e.g., inpatient bed requests, transport, etc.). We model the process of ED disposition decision prediction as a hierarchical multiclass classification while dealing with the progressive accrual of clinical information throughout the ED caregiving process. Multinomial logistic regression as well as machine learning models are built for carrying out the predictions. Utilizing results from just the first set of ED laboratory tests along with other prior information gathered for each patient (2.5 h ahead of the actual disposition decision on average), our model predicts disposition decisions with positive predictive values of 55.4%, 45.1%, 56.9%, and 47.5%, while controlling false positive rates (1.4%, 1.0%, 4.3%, and 1.4%), with AUC values of 0.97, 0.95, 0.89, and 0.84 for the four admission (minor) classes, i.e., intensive care unit (3.6% of the testing samples), telemetry unit (2.2%), general practice unit (11.9%), and observation unit (6.6%) classes, respectively. Moreover, patients destined to intensive care unit present a more drastic increment in prediction quality at triage than others. Disposition decision classification models can provide more actionable information than a binary admission vs. discharge prediction model for the proactive initiation of admission processes for ED patients. Observing the distinct trajectories of information accrual and prediction quality evolvement for ED patients destined to different types of units, proactive coordination strategies should be tailored accordingly for each destination unit.
Article
Full-text available
Importance While machine learning approaches may enhance prediction ability, little is known about their utility in emergency department (ED) triage. Objectives To examine the performance of machine learning approaches to predict clinical outcomes and disposition in children in the ED and to compare their performance with conventional triage approaches. Design, Setting, and Participants Prognostic study of ED data from the National Hospital Ambulatory Medical Care Survey from January 1, 2007, through December 31, 2015. A nationally representative sample of 52 037 children aged 18 years or younger who presented to the ED were included. Data analysis was performed in August 2018. Main Outcomes and Measures The outcomes were critical care (admission to an intensive care unit and/or in-hospital death) and hospitalization (direct hospital admission or transfer). In the training set (70% random sample), using routinely available triage data as predictors (eg, demographic characteristics and vital signs), we derived 4 machine learning–based models: lasso regression, random forest, gradient-boosted decision tree, and deep neural network. In the test set (the remaining 30% of the sample), we measured the models’ prediction performance by computing C statistics, prospective prediction results, and decision curves. These machine learning models were built for each outcome and compared with the reference model using the conventional triage classification information. Results Of 52 037 eligible ED visits by children (median [interquartile range] age, 6 [2-14] years; 24 929 [48.0%] female), 163 (0.3%) had the critical care outcome and 2352 (4.5%) had the hospitalization outcome. For the critical care prediction, all machine learning approaches had higher discriminative ability compared with the reference model, although the difference was not statistically significant (eg, C statistics of 0.85 [95% CI, 0.78-0.92] for the deep neural network vs 0.78 [95% CI, 0.71-0.85] for the reference; P = .16), and lower number of undertriaged critically ill children in the conventional triage levels 3 to 5 (urgent to nonurgent). For the hospitalization prediction, all machine learning approaches had significantly higher discrimination ability (eg, C statistic, 0.80 [95% CI, 0.78-0.81] for the deep neural network vs 0.73 [95% CI, 0.71-0.75] for the reference; P < .001) and fewer overtriaged children who did not require inpatient management in the conventional triage levels 1 to 3 (immediate to urgent). The decision curve analysis demonstrated a greater net benefit of machine learning models over ranges of clinical thresholds. Conclusions and Relevance Machine learning–based triage had better discrimination ability to predict clinical outcomes and disposition, with reduction in undertriaging critically ill children and overtriaging children who are less ill.
Article
Full-text available
Objective To predict hospital admission at the time of ED triage using patient history in addition to information collected at triage. Methods This retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model. Results A total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86–0.87), 0.87 for XGBoost (95% CI 0.87–0.88) and 0.87 for DNN (95% CI 0.87–0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86–0.87), 0.87 for XGBoost (95% CI 0.87–0.87) and 0.87 for DNN (95% CI 0.87–0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91–0.91), 0.92 for XGBoost (95% CI 0.92–0.93) and 0.92 for DNN (95% CI 0.92–0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91–0.91). Conclusion Machine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.
Article
Full-text available
Crowding within Emergency Departments (EDs) can have significant negative consequences for patients. EDs therefore need to explore the use of innovative methods to improve patient flow and prevent overcrowding. One potential method is the use of data mining using machine learning techniques to predict ED admissions. This study uses routinely collected administrative data (120,600 records) from two major acute hospitals in Northern Ireland to compare contrasting machine learning algorithms in predicting the risk of admission from the ED. We use three algorithms to build the predictive models: (1) logistic regression, (2) decision trees, and (3) gradient boosted machines (GBM). The GBM performed better (accuracy=80.31%, AUC-ROC=0.859) than the decision tree (accuracy=80.06%, AUC-ROC=0.824) and the logistic regression model (accuracy=79.94%, AUC-ROC=0.849). Drawing on logistic regression, we identify several factors related to hospital admissions including hospital site, age, arrival mode, triage category, care group, previous admission in the past month, and previous admission in the past year. This study highlights the potential utility of three common machine learning algorithms in predicting patient admissions. Practical implementation of the models developed in this study in decision support tools would provide a snapshot of predicted admissions from the emergency department at a given time, allowing for advance resource planning and the avoidance bottlenecks in patient flow, as well as comparison of predicted and actual admission rates. When interpretability is a key consideration, EDs should consider adopting logistic regression models, although GBM’s will be useful where accuracy is paramount. OAPA
Article
Full-text available
Objective: To describe and compare logistic regression and neural network modeling strategies to predict hospital admission or transfer following initial presentation to Emergency Department (ED) triage with and without the addition of natural language processing elements. Methods: Using data from the National Hospital Ambulatory Medical Care Survey (NHAMCS), a cross-sectional probability sample of United States EDs from 2012 and 2013 survey years, we developed several predictive models with the outcome being admission to the hospital or transfer vs. discharge home. We included patient characteristics immediately available after the patient has presented to the ED and undergone a triage process. We used this information to construct logistic regression (LR) and multilayer neural network models (MLNN) which included natural language processing (NLP) and principal component analysis from the patient's reason for visit. Ten-fold cross validation was used to test the predictive capacity of each model and receiver operating curves (AUC) were then calculated for each model. Results: Of the 47,200 ED visits from 642 hospitals, 6,335 (13.42%) resulted in hospital admission (or transfer). A total of 48 principal components were extracted by NLP from the reason for visit fields, which explained 75% of the overall variance for hospitalization. In the model including only structured variables, the AUC was 0.824 (95% CI 0.818-0.830) for logistic regression and 0.823 (95% CI 0.817-0.829) for MLNN. Models including only free-text information generated AUC of 0.742 (95% CI 0.731- 0.753) for logistic regression and 0.753 (95% CI 0.742-0.764) for MLNN. When both structured variables and free text variables were included, the AUC reached 0.846 (95% CI 0.839-0.853) for logistic regression and 0.844 (95% CI 0.836-0.852) for MLNN. Conclusions: The predictive accuracy of hospital admission or transfer for patients who presented to ED triage overall was good, and was improved with the inclusion of free text data from a patient's reason for visit regardless of modeling approach. Natural language processing and neural networks that incorporate patient-reported outcome free text may increase predictive accuracy for hospital admission.
Article
Full-text available
Background Disposition decisions are critical to the functioning of Emergency Departments. The objectives of the present study were to derive and internally validate a prediction model for inpatient admission from the Emergency Department to assist with triage, patient flow and clinical decision making. Methods This was a retrospective analysis of State-wide Emergency Department data in New South Wales, Australia. Adult patients (age ≥ 16 years) were included if they presented to a Level five or six (tertiary level) Emergency Department in New South Wales, Australia between 2013 and 2014. The outcome of interest was in-patient admission from the Emergency Department. This included all admissions to short stay and medical assessment units and being transferred out to another hospital. Analyses were performed using logistic regression. Discrimination was assessed using area under curve and derived risk scores were plotted to assess calibration. Results1,721,294 presentations from twenty three Level five or six hospitals were analysed. Of these 49.38% were male and the mean (sd) age was 49.85 years (22.13). Level 6 hospitals accounted for 47.70% of cases and 40.74% of cases were classified as an in-patient admission based on their mode of separation. The final multivariable model including age, arrival by ambulance, triage category, previous admission and presenting problem had an AUC of 0.82 (95% CI 0.81, 0.82). Conclusion By deriving and internally validating a risk score model to predict the need for in-patient admission based on basic demographic and triage characteristics, patient flow in ED, clinical decision making and overall quality of care may be improved. Further studies are now required to establish clinical effectiveness of this risk score model.
Article
Full-text available
To create and validate a simple clinical score to estimate the probability of admission at the time of triage. This was a multicentre, retrospective, cross-sectional study of triage records for all unscheduled adult attendances in North Glasgow over 2 years. Clinical variables that had significant associations with admission on logistic regression were entered into a mixed-effects multiple logistic model. This provided weightings for the score, which was then simplified and tested on a separate validation group by receiving operator characteristic (ROC) analysis and goodness-of-fit tests. 215 231 presentations were used for model derivation and 107 615 for validation. Variables in the final model showing clinically and statistically significant associations with admission were: triage category, age, National Early Warning Score (NEWS), arrival by ambulance, referral source and admission within the last year. The resulting 6-variable score showed excellent admission/discharge discrimination (area under ROC curve 0.8774, 95% CI 0.8752 to 0.8796). Higher scores also predicted early returns for those who were discharged: the odds of subsequent admission within 28 days doubled for every 7-point increase (log odds=+0.0933 per point, p<0.0001). This simple, 6-variable score accurately estimates the probability of admission purely from triage information. Most patients could accurately be assigned to 'admission likely', 'admission unlikely', 'admission very unlikely' etc., by setting appropriate cut-offs. This could have uses in patient streaming, bed management and decision support. It also has the potential to control for demographics when comparing performance over time or between departments.
Book
Full-text available
During the last decade, the French-speaking scientific community developed a very strong research activity in the field of Knowledge Discovery and Management (KDM or EGC for “Extraction et Gestion des Connaissances” in French), which is concerned with, among others, Data Mining, Knowledge Discovery, Business Intelligence, Knowledge Engineering and SemanticWeb. The recent and novel research contributions collected in this book are extended and reworked versions of a selection of the best papers that were originally presented in French at the EGC 2009 Conference held in Strasbourg, France on January 2009. The volume is organized in four parts. Part I includes five papers concerned by various aspects of supervised learning or information retrieval. Part II presents five papers concerned with unsupervised learning issues. Part III includes two papers on data streaming and two on security while in Part IV the last four papers are concerned with ontologies and semantic.
Conference Paper
Full-text available
We present a new evaluation criterion for the induction of decision trees. We exploit a parameter-free Bayesian approach and propose an analytic formula for the evaluation of the posterior probability of a decision tree given the data. We thus transform the training problem into an optimization problem in the space of decision tree models, and search for the best tree, which is the maximum a posteriori (MAP) one. The optimization is performed using top-down heuristics with pre-pruning and post-pruning processes. Extensive experiments on 30 UCI datasets and on the 5 WCCI 2006 performance prediction challenge datasets show that our method obtains predictive performance similar to that of alternative state-of-the-art methods, with far simpler trees.
Article
Full-text available
Hospital admission delays in the Emergency Department (ED) reduce volume capacity and contribute to the nation's ED diversion problem. This study evaluated the accuracy of a Bayesian network for the early prediction of hospital admission status using data from 16,900 ED encounters. The final model included nine variables that are commonly available in many ED settings. The area under the receiver operating characteristic curve was 0.894 (95% CI: 0.887-0.902) for the validation set. The system had high accuracy an may be used to alert clinicians to initiate admission processes earlier during a patient's ED encounter.
Conference Paper
The study of urgent patient hospitalisation pathways is necessary to tackle the difficulty of management of unscheduled hospitalisations. The emergency department overcrowding is often the entry point of these studies. The following of the pathway and the discharge of the patients admitted from the emergency department are little investigated. Health data seem to be an adequate resource to study patient pathways. The sources of data are diverse and rich of information but little accessible and underused. The perspectives of research are the study of whole pathways and discharges using hospital data. Key words: patient pathways, hospital data, emergency department.
Article
Overcrowding in emergency department (ED) causes lengthy waiting times, reduces adequate emergency care and increases rate of mortality. Accurate prediction of daily ED visits and allocating resources in advance is one of the solutions to ED overcrowding problem. In this paper, a deep stacked architecture is being proposed and applied to the daily ED visits prediction problem with deep components such as Long Short Term Memory (LSTM), Gated Recurrent Units (GRU) and simple Recurrent Neural Network (RNN). The proposed architecture achieves very high mean accuracy level (94.28–94.59%) in daily ED visits predictions. We have also compared the performance of this architecture with non-stacked deep models and traditional prediction models. The results indicate that deep stacked models outperform (4–7%) the traditional prediction models and other non-stacked deep learning models (1–2%) in our prediction tasks. The application of deep neural network in ED visits prediction is novel as this is one of the first studies to apply a deep stacked architecture in this field. Importantly, our models have achieved better prediction accuracy (in one case comparable) than the state-of-the-art in the literature.
Article
This work proposes a framework for optimizing machine learning algorithms. The practicality of the framework is illustrated using an important case study from the healthcare domain, which is predicting the admission status of emergency department (ED) patients (e.g., admitted vs. discharged) using patient data at the time of triage. The proposed framework can mitigate the crowding problem by proactively planning the patient boarding process. A large retrospective dataset of patient records is obtained from the electronic health record database of all ED visits over three years from three major locations of a healthcare provider in the Midwest of the US. Three machine learning algorithms are proposed: T-XGB, T-ADAB, and T-MLP. T-XGB integrates extreme gradient boosting (XGB) and Tabu Search (TS), T-ADAB integrates Adaboost and TS, and T-MLP integrates multi-layer perceptron (MLP) and TS. The proposed algorithms are compared with the traditional algorithms: XGB, ADAB, and MLP, in which their parameters are tunned using grid search. The three proposed algorithms and the original ones are trained and tested using nine data groups that are obtained from different feature selection methods. In other words, 54 models are developed. Performance was evaluated using five measures: Area under the curve (AUC), sensitivity, specificity, F1, and accuracy. The results show that the newly proposed algorithms resulted in high AUC and outperformed the traditional algorithms. The T-ADAB performs the best among the newly developed algorithms. The AUC, sensitivity, specificity, F1, and accuracy of the best model are 95.4%, 99.3%, 91.4%, 95.2%, 97.2%, respectively.
Conference Paper
Overcrowding in Emergency Departments (ED) is considered as an international issue, which could have adverse impacts on multiple care outcomes such as the length of stay for example. Part of the solution could lie in the early prediction of the patient outcome as discharge or hospitalization. This study applies Deep Learning to this end. A large-scale dataset of about 260K ED records was provided by the Amiens-Picardy University Hospital in France. In general, our approach is based on integrating structured data with unstructured textual notes recorded at the triage stage. The key idea is to apply a multi-input of mixed data for training a classification model to predict hospitalization. In a simultaneous manner, the model training utilizes the numeric features along with textual data. On one hand, a standard Multi-Layer Perceptron (MLP) model is used with the standard set of features (i.e. numeric and categorical). On the other hand, a Convolutional Neural Network (CNN) is used to operate over the textual data. The two components of learning are conducted independently in parallel. The empirical results demonstrated that the classifier could achieve a very good accuracy with ROC-AUC≈0.83. The study is conceived to contribute to the mounting efforts of applying Natural Language Processing in the healthcare domain.
Article
Background Emergency departments (ED) are a portal of entry into the hospital and are uniquely positioned to influence the health care trajectories of older adults seeking medical attention. Older adults present to the ED with distinct needs and complex medical histories, which can make disposition planning more challenging. Machine Learning (ML) approaches have been previously used to inform decision-making surrounding ED disposition in the general population. However, little is known about the performance and utility of ML methods in predicting hospital admission among older ED patients. We applied a series of ML algorithms to predict ED admission among older adults and discuss their clinical and policy implications. Materials and Methods We analyzed the Canadian data from the interRAI multinational ED study, the largest prospective cohort study of older ED patients to date. The data included 2,274 ED patients 75 years of age and older from eight ED sites across Canada between November 2009 and April 2012. Data were extracted from the interRAI ED Contact Assessment, with predictors including a series of geriatric syndromes, functional assessments, and baseline care needs. We applied a total of five ML algorithms. Models were trained, assessed, and analyzed using 10-fold cross-validation. The performance of predictive models was measured using the area under the receiver operating characteristic curve (AUC). We also report the accuracy, sensitivity, and specificity of each model to supplement performance interpretation. Results Gradient boosted trees was the most accurate model to predict older ED patients who would require hospitalization (AUC = 0.80). The five most informative features include home intravenous therapy, time of ED presentation, a requirement for formal support services, independence in walking, and the presence of an unstable medical condition. Conclusion To the best of our knowledge, this is the first study to predict hospital admission in older ED patients using a series of geriatric syndromes and functional assessments. We were able to predict hospital admission in older ED patients with good accuracy using the items available in the interRAI ED Contact Assessment. This information can be used to inform decision-making about ED disposition and may expedite admission processes and proactive discharge planning.
Article
Background Emergency department (ED) overcrowding has been a serious issue and demands effective clinical decision-making of patient disposition. In previous studies, emergency clinical narratives provide a rich context for clinical decision. We aimed to develop the disposition prediction model using deep learning modeling strategy with the heterogeneous data including the physicians' narratives. Methods We constructed a retrospective cohort of all 104,083 ED visits of non-trauma adults during 2017-18 from an academically affiliated ED in Taiwan. 18,308 visits were excluded based on the completeness of each record and the unpredictable dispositions, such as out-of-hospital cardiac arrest, against-advice discharge, and escapes. We integrated subjective section of the first physicians' clinical narratives and structured data (e.g., demographics, triage vital signs, etc.) as available predictors at the first physician-patient encounter. To predict final patient disposition (i.e., hospitalization or discharge), a deep neural network (DNN) model was developed with word embedding, a common natural language processing method. We compared the proposed model to a reference model using the Rapid Emergency Medicine Score, a logistic regression model with structured data, and a DNN model with paragraph vectors. F1 score was used to measure the predictive performance for each model. Results The F1 score (with 95% CI) for the proposed model, the reference model, the logistic regression model with structured data, and the DNN model with paragraph vectors were 0.674 (0.669-0.679), 0.474 (0.469-0.479), 0.547 (0.543-0.551), and 0.602 (0.596-0.607), respectively. While analyzing the relationship between context length and predictive performance under the proposed model, the F1 score at 95th percentile of the word counts was higher than that at 25th percentile of the word counts in chief complaint [0.634 (0.629-0.640) vs. 0.624 (0.620-0.628)] and in present illness [0.671 (0.667-0.674) vs. 0.654 (0.651-0.658)], but not in past medical history [0.674 (0.669-0.679) vs. 0.673 (0.666-0.679)]. Conclusions The proposed deep learning model with the usage of the first physicians' clinical narratives and structured data based on natural language processing outperformed the commonly used ones in terms of F1 score. It also evidenced the importance of the subjective section of clinical narratives, which serve as vital predictors for ED clinical decision-making.
Article
Background: Nursing triage documentation is the first free-form text data created at the start of an emergency department (ED) visit. These 1-3 unstructured sentences reflect the clinical impression of an experienced nurse and are key in gauging a patient's illness. We aimed to predict final ED disposition using three commonly-employed natural language processing (NLP) techniques of nursing triage notes in isolation from other data. Methods: We constructed a retrospective cohort of all 260,842 consecutive ED encounters in 2015-16, from three clinically heterogeneous academically-affiliated EDs. After exclusion of 3964 encounters based on completeness of triage, and disposition data, we included 256,878 encounters. We defined the outcome as: 1) admission, transfer, or in-ED death [68,092 encounters] vs. 2) discharge, "left without being seen," and "left against medical advice" [188,786 encounters]. The dataset was divided into training and testing subsets. Neural network regression models were trained using bag-of-words, paragraph vectors, and topic distributions to predict disposition and were evaluated using the testing dataset. Results: Area under the curve for disposition using triage notes as bag-of-words, paragraph vectors, and topic distributions were 0.737 (95% CI: 0.734 - 0.740), 0.785 (95% CI: 0.782 - 0.788), and 0.687 (95% CI: 0.684 - 0.690), respectively. Conclusions: Nursing triage notes can be used to predict final ED patient disposition, even when used separately from other clinical information. These findings have substantial implications for future studies, suggesting that free text from medical records may be considered as a critical predictor in research of patient outcomes.
Article
Background: Long boarding time in emergency department (ED) leads to increased morbidity and mortality. Prediction of admissions upon triage could improve ED care efficiency and decrease boarding time. Objective: To develop a real-time automated model (MA) to predict admissions upon triage and compare this model with triage nurse prediction (TNP). Patients and methods: A cross-sectional study was conducted in four EDs during 1 month. MA used only variables available upon triage and included in the national French Electronic Emergency Department Abstract. For each patient, the triage nurse assessed the hospitalization risk on a 10-point Likert scale. Performances of MA and TNP were compared using the area under the receiver operating characteristic curves, the accuracy, and the daily and hourly mean difference between predicted and observed number of admission. Results: A total of 11 653 patients visited the EDs, and 19.5-24.7% were admitted according to the emergency. The area under the curves (AUCs) of TNP [0.815 (0.805-0.826)] and MA [0.815 (0.805-0.825)] were similar. Across EDs, the AUCs of TNP were significantly different (P<0.001) in all EDs, whereas AUCs of MA were all similar (P>0.2). Originally, using daily and hourly aggregated data, the percentage of errors concerning the number of predicted admission were 8.7 and 34.4%, respectively, for MA and 9.9 and 35.4%, respectively, for TNP. Conclusion: A simple model using variables available in all EDs in France performed well to predict admission upon triage. However, when analyzed at an hourly level, it overestimated the number of inpatient beds needed by a third. More research is needed to define adequate use of these models.
Article
Objective To further develop and refine an Emergency Department (ED) in‐patient admission prediction model using machine learning techniques. Methods This was a retrospective analysis of state‐wide ED data from New South Wales, Australia. Six classification algorithms (Bayesian networks, decision trees, logistic regression, naïve Bayes, neural networks and nearest neighbour) and five feature selection techniques (none, manual, correlation‐based, information gain and wrapper) were examined. Presenting problem was categorised using broad (n = 20) and specific (n = 100) representations. Models were evaluated based on Area Under the Curve (AUC) and accuracy. The results were compared with the Sydney Triage to Admission Risk Tool (START), which uses logistic regression and six manually selected features. Results Sixty admission prediction models were trained and validated using data from 1 721 294 patients. Under the broad representation of presenting problem, the nearest neighbour algorithm with manual feature selection had the best AUC of 0.8206 (95% CI ±0.0006), while the decision tree with no feature selection had the best accuracy of 74.83% (95% CI ±0.065). Under the specific representation, almost all models improved; the nearest neighbour with information gain feature selection had the best AUC of 0.8267 (95% CI ±0.0006), while the decision tree with wrapper or no feature selection had the best accuracy of 75.24% (95% CI ±0.064). Eleven of the machine learning models had slightly better AUC than the START model. Conclusion Machine learning methods demonstrate similar performance to logistic regression for ED disposition prediction models using basic triage information. This should be investigated further, especially for larger data sets with more complex clinical information.
Article
Background: Emergency department (ED) overcrowding is a growing international patient safety issue. A major contributor to overcrowding is long wait times for inpatient hospital admission. The objective of this study is to create a model that can predict a patient's need for hospital admission at the time of triage. Methods: Retrospective observational study of electronic clinical records of all ED visits over ten years to a large urban hospital in Singapore. The data was randomly divided into a derivation set and a validation set. We used the derivation set to develop a logistic regression model that predicts probability of hospital admission for patients presenting to the ED. We tested the model on the validation set and evaluated the performance with receiver operating characteristic (ROC) curve analysis. Results: A total of 1,232,016 visits were included for final analysis, of which 38.7% were admitted. Eight variables were included in the final model: age group, race, postal code, day of week, time of day, triage category, mode of arrival, and fever status. The model performed well on the validation set with an area under the curve of 0.825 (95% CI 0.824–0.827). Increasing age, increasing triage acuity, and mode of arrival via private patient transport were most predictive of the need for admission. Conclusions: We developed a model that accurately predicts admission for patients presenting to the ED using demographic, administrative, and clinical data routinely collected at triage. Implementation of the model into the electronic health record could help reduce the burden of overcrowding. Keywords: Admission; Emergency department; Triage; Prediction
Article
Introduction: One of the factors contributing to ED crowding is the lengthy delay in transferring an admitted patient from the ED to an inpatient department (ie, boarding time). An earlier start of the admission process using an automatic hospitalisation prediction model could potentially shorten these delays and reduce crowding. Methods: Clinical, operational and demographic data were retrospectively collected on 80 880 visits to the ED of Rambam Health Care Campus in Haifa, Israel, from January 2011 to January 2012. Using these data, a logistic regression model was developed to predict patient disposition (hospitalisation vs discharge) at three progressive time points throughout the ED visit: within the first 10 min, within an hour and within 2 hours. The algorithm was trained on 50% of the data (n=40 440) and tested on the remaining 50%. Results: During the study time period, 58 197 visits ended in discharge and 22 683 in hospitalisation. Within 1 hour of presentation, our model was able to predict hospitalisation with a specificity of 90%, sensitivity of 94% and an AUCof 0.97. Early clinical decisions such as testing for calcium levels were found to be highly predictive of hospitalisations. In the Rambam ED, the use of such a prediction system would have the potential to save more than 250 patient hours per day. Conclusions: Data collected by EDs in electronic medical records can be used within a progressive modelling framework to predict patient flow and improve clinical operations. This approach relies on commonly available data and can be applied across different healthcare settings.
Article
Objective: Emergency department (ED) overcrowding is a serious issue for hospitals. Early information on short-term inward bed demand from patients receiving care at the ED may reduce the overcrowding problem, and optimize the use of hospital resources. In this study, we use text mining methods to process data from early ED patient records using the SOAP framework, and predict future hospitalizations and discharges. Design: We try different approaches for pre-processing of text records and to predict hospitalization. Sets-of-words are obtained via binary representation, term frequency, and term frequency-inverse document frequency. Unigrams, bigrams and trigrams are tested for feature formation. Feature selection is based on χ2 and F-score metrics. In the prediction module, eight text mining methods are tested: Decision Tree, Random Forest, Extremely Randomized Tree, AdaBoost, Logistic Regression, Multinomial Naïve Bayes, Support Vector Machine (Kernel linear) and Nu-Support Vector Machine (Kernel linear). Measurements: Prediction performance is evaluated by F1-scores. Precision and Recall values are also informed for all text mining methods tested. Results: Nu-Support Vector Machine was the text mining method with the best overall performance. Its average F1-score in predicting hospitalization was 77.70%, with a standard deviation (SD) of 0.66%. Conclusions: The method could be used to manage daily routines in EDs such as capacity planning and resource allocation. Text mining could provide valuable information and facilitate decision-making by inward bed management teams.
Article
The objective was to test the generalizability, across a range of hospital sizes and demographics, of a previously developed method for predicting and aggregating, in real time, the probabilities that emergency department (ED) patients will be admitted to a hospital inpatient unit. Logistic regression models were developed that estimate inpatient admission probabilities of each patient upon entering an ED. The models were based on retrospective development (n = 4,000 to 5,000 ED visits) and validation (n = 1,000 to 2,000 ED visits) data sets from four heterogeneous hospitals. Model performance was evaluated using retrospective test data sets (n = 1,000 to 2,000 ED visits). For one hospital the developed model also was applied prospectively to a test data set (n = 910 ED visits) coded by triage nurses in real time, to compare results to those from the retrospective single investigator-coded test data set. The prediction models for each hospital performed reasonably well and typically involved just a few simple-to-collect variables, which differed for each hospital. Areas under receiver operating characteristic curves (AUC) ranged from 0.80 to 0.89, R(2) correlation coefficients between predicted and actual daily admissions ranged from 0.58 to 0.90, and Hosmer-Lemeshow goodness-of-fit statistics of model accuracy had p > 0.01 with one exception. Data coded prospectively by triage nurses produced comparable results. The accuracy of regression models to predict ED patient admission likelihood was shown to be generalizable across hospitals of different sizes, populations, and administrative structures. Each hospital used a unique combination of predictive factors that may reflect these differences. This approach performed equally well when hospital staff coded patient data in real time versus the research team retrospectively.
Conference Paper
In supervised machine learning, the partitioning of the values (also called grouping) of a categorical attribute aims at constructing a new synthetic attribute which keeps the information of the initial attribute and reduces the number of its values. In case of very large number of values, the risk of overfitting the data increases sharply and building good groupings becomes difficult. In this paper, we propose two new grouping methods founded on a Bayesian approach, leading to Bayes optimal groupings. The first method exploits a standard schema for grouping models and the second one extends this schema by managing a “garbage” group dedicated to the least frequent values. Extensive comparative experiments demonstrate that the new grouping methods build high quality groupings in terms of predictive quality, robustness and small number of groups.
Article
While real data often comes in mixed format, discrete and continuous, many supervised induction algorithms require discrete data. Efficient discretization of continuous attributes is an important problem that has effects on speed, accuracy and understandability of the induction models. In this paper, we propose a new discretization method MODL1, founded on a Bayesian approach. We introduce a space of discretization models and a prior distribution defined on this model space. This results in the definition of a Bayes optimal evaluation criterion of discretizations. We then propose a new super-linear optimization algorithm that manages to find near-optimal discretizations. Extensive comparative experiments both on real and synthetic data demonstrate the high inductive performances obtained by the new discretization method.
Article
With the rapid growth of computer storage capacities, available data and demand for scoring models both follow an increasing trend, sharper than that of the processing power. However, the main limitation to a wide spread of data mining solutions is the non-increasing availability of skilled data analysts, which play a key role in data preparation and model selection. In this paper, we present a parameter-free scalable classification method, which is a step towards fully automatic data mining. The method is based on Bayes optimal univariate conditional density estimators, naive Bayes classification enhanced with a Bayesian variable selection scheme, and averaging of models using a logarithmic smoothing of the posterior distribution. We focus on the complexity of the algorithms and show how they can cope with data sets that are far larger than the available central memory. We finally report results on the Large Scale Learning challenge, where our method obtains state of the art performance within practicable computation time.
Article
To be able to predict, at the time of triage, whether a need for hospital admission exists for emergency department (ED) patients may constitute useful information that could contribute to systemwide hospital changes designed to improve ED throughput. The objective of this study was to develop and validate a predictive model to assess whether a patient is likely to require inpatient admission at the time of ED triage, using routine hospital administrative data. Data collected at the time of triage by nurses from patients who visited the ED in 2007 and 2008 were extracted from hospital administrative databases. Variables included were demographics (age, sex, and ethnic group), ED visit or hospital admission in the preceding 3 months, arrival mode, patient acuity category (PAC) of the ED visit, and coexisting chronic diseases (diabetes, hypertension, and dyslipidemia). Chi-square tests were used to study the association between the selected possible risk factors and the need for hospital admission. Logistic regression was applied to develop the prediction model. Data were split for derivation (60%) and validation (40%). Receiver operating characteristic curves and goodness-of-fit tests were applied to the validation data set to evaluate the model. Of 317,581 ED patient visits, 30.2% resulted in immediate hospital admission. In the developed predictive model, age, PAC status, and arrival mode were most predictive of the need for immediate hospital inpatient admission. The c-statistic of the receiver operating characteristic (ROC) curve was 0.849 (95% confidence interval [CI] = 0.847 to 0.851). The goodness-of-fit test showed that the predicted patients' admission risks fit the patients' actual admission status well. A model for predicting the risk of immediate hospital admission at triage for all-cause ED patients was developed and validated using routinely collected hospital data. Early prediction of the need for hospital admission at the time of triage may help identify patients deserving of early admission planning and resource allocation and thus potentially reduce ED overcrowding.
Article
The ability to predict patient visits to emergency departments (ED) is crucial for designing strategies aimed at avoiding overcrowding. A good working knowledge of the mathematical models used to predict patient volume and of their results is therefore essential. Articles retrieved by a Medline search were reviewed for studies designed to predict patient attendance at ED or walk-in clinics. Nine studies were identified. Most of the models used to predict patient volume were either linear regression models including calendar variables or time series models. These models explained 31-75% of patient-volume variability. Although the day of the week had the strongest effect, this variable explained only part of the variability. Other causes of this variability are to be defined. However, the performance of the models was good, with errors ranging from 4.2% to 14.4%. Adding meteorological data failed to improve model performance. The mathematical methods developed to predict ED visits have a low rate of error, but the prediction of daily patient visits should be used carefully and therefore does not allow day-to-day adjustments of staff. ED directors or managers should be aware of the model limitations. These models should certainly be used on a larger scale to assess future needs.
Fiche 25 -La médecine d'urgence
DREES, Fiche 25 -La médecine d'urgence, 2021. [Online]. Available: https://drees.solidaritessante.gouv.fr/sites/default/files/2021-07/Fiche%2025%20-%20La%20m%C3%A9decine%20d%E2%80%99urgence.pdf
Les hospitalisations après passage aux urgences moins nombreuses dans le secteur privé, DREES - Etudes & Résultats
  • L Ricroch
  • A Vuagnat
L. Ricroch and A. Vuagnat, Les hospitalisations après passage aux urgences moins nombreuses dans le secteur privé, DREES -Etudes & Résultats, no. 0997, Sep. 2017.
Urgences: sept patients sur dix attendent moins d’une heure avant le début des soins
  • Ricroch
L. Ricroch and A. Vuagnat, Urgences : sept patients sur dix attendent moins d'une heure avant le début des soins. no. 0929, Aug. 2015. [Online]. Available: https://drees.solidaritessante.gouv.fr/sites/default/files/2020-08/er929.pdf
Explaining Unscheduled Patient Pathways, unpublished
  • L Uhl
L. Uhl, Explaining Unscheduled Patient Pathways, unpublished.
Khiops: outil d’apprentissage supervisé automatique pour la fouille de grandes bases de données multi-tables
  • Boullé
M. Boullé, Khiops: outil d'apprentissage supervisé automatique pour la fouille de grandes bases de données multi-tables, Revue des Nouvelles Technologies de l'Information, vol. 16ème Journées Francophones Extraction et Gestion des Connaissances, no. E-30, pp. 505-510, 2016.
A Bayes Evaluation Criterion for Decision Trees&#x00E9;, Advances in Knowledge Discovery and Management (AKDM-1)
  • voisine
La démographie des professionnels de santé de 1999
  • Drees
  • La