Project

Combining physiological and biomedical data into a novel computer-aided risk score to support near real-time clinical decision making and determine its impact on quality/safety of care in two hospitals.

Goal: 1. To develop and validate CARS in two hospitals.
2. To embed to CARS into routine clinical practice without adverse

Updates
0 new
0
Recommendations
0 new
0
Followers
0 new
1
Reads
0 new
27

Project log

Muhammad Faisal
added a research item
Background: Although the National Early Warning Score (NEWS) and its latest version NEWS2 are recommended for monitoring for deterioration in patients admitted to hospital, little is known about their performance in COVID-19 patients. We analysed the performance of National Early Warning Score (NEWS2) during the first phase of the COVID-19 pandemic. Methods: Adult non-elective admissions discharged between 11-March-2020 to 13-June-2020 with an index NEWS2 electronically recorded within 24 hours of admission are used to predict mortality at four time points (in-hospital, 24hours, 48hours, and 72hours) in COVID-19 versus non-COVID-19 admissions. Results: Out of 6480 non-elective admissions, 620 (9.6%) had a diagnosis of COVID-19. They were older (73.3 vs 67.7yrs), more often male (54.7% vs 50.1%), had higher index NEWS (4 vs 2.5) and NEWS2 (4.6 vs 2.8) scores and higher in-hospital mortality (32.1% vs 5.8%). The c-statistics for predicting in-hospital mortality in COVID-19 admissions was significantly lower using NEWS (0.64 vs 0.74) or NEWS2 (0.64 vs 0.74), however these differences reduced at 72hours (NEWS: 0.75 vs 0.81; NEWS2: 0.71 vs 0.81), 48 hours (NEWS: 0.78 vs 0.81; NEWS2: 0.76 vs 0.82) and 24hours (NEWS: 0.84 vs 0.84; NEWS2: 0.86 vs 0.84). Increasing NEWS2 values reflected increased mortality, but for any given value the absolute risk was on average 24% higher (e.g.NEWS2=5: 36% vs 9%). Interpretation: NEWS2 is a valid predictor of the mortality risk but substantially underestimates the absolute mortality risk in COVID-19 patients. Clinical staff
Muhammad Faisal
added a research item
Objectives In the English National Health Service, the patient’s vital signs are monitored and summarised into a National Early Warning Score (NEWS) to support clinical decision making, but it does not provide an estimate of the patient’s risk of death. We examine the extent to which the accuracy of NEWS for predicting mortality could be improved by enhanced computer versions of NEWS (cNEWS). Design Logistic regression model development and external validation study. Setting Two acute hospitals (YH—York Hospital for model development; NH—Northern Lincolnshire and Goole Hospital for external model validation). Participants Adult (≥16 years) medical admissions discharged over a 24-month period with electronic NEWS (eNEWS) recorded on admission are used to predict mortality at four time points (in-hospital, 24 hours, 48 hours and 72 hours) using the first electronically recorded NEWS (model M0) versus a cNEWS model which included age+sex (model M1) +subcomponents of NEWS (including diastolic blood pressure) (model M2). Results The risk of dying in-hospital following emergency medical admission was 5.8% (YH: 2080/35 807) and 5.4% (NH: 1900/35 161). The c-statistics for model M2 in YH for predicting mortality (in-hospital=0.82, 24 hours=0.91, 48 hours=0.88 and 72 hours=0.88) was higher than model M0 (in-hospital=0.74, 24 hours=0.89, 48 hours=0.86 and 72 hours=0.85) with higher Positive Predictive Value (PPVs) for in-hospital mortality (M2 19.3% and M0 16.6%). Similar findings were seen in NH. Model M2 performed better than M0 in almost all major disease subgroups. Conclusions An externally validated enhanced computer-aided NEWS model (cNEWS) incrementally improves on the performance of a NEWS only model. Since cNEWS places no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated to determine if it can improve care in hospitals that have eNEWS systems.
Muhammad Faisal
added a research item
Objectives To compare the performance of a validated automatic computer-aided risk of mortality (CARM) score versus medical judgement in predicting the risk of in-hospital mortality for patients following emergency medical admission. Design A prospective study. Setting Consecutive emergency medical admissions in York hospital. Participants Elderly medical admissions in one ward were assigned a risk of death at the first post-take ward round by consultant staff over a 2-week period. The consultant medical staff used the same variables to assign a risk of death to the patient as the CARM (age, sex, National Early Warning Score and blood test results) but also had access to the clinical history, examination findings and any immediately available investigations such as ECGs. The performance of the CARM versus consultant medical judgement was compared using the c-statistic and the positive predictive value (PPV). Results The in-hospital mortality was 31.8% (130/409). For patients with complete blood test results, the c-statistic for CARM was 0.75 (95% CI: 0.69 to 0.81) versus 0.72 (95% CI: 0.66 to 0.78) for medical judgements (p=0.28). For patients with at least one missing blood test result, the c-statistics were similar (medical judgements 0.70 (95% CI: 0.60 to 0.81) vs CARM 0.70 (95% CI: 0.59 to 0.80)). At a 10% mortality risk, the PPV for CARM was higher than medical judgements in patients with complete blood test results, 62.0% (95% CI: 51.2 to 71.9) versus 49.2% (95% CI: 39.8 to 58.5) but not when blood test results were missing, 50.0% (95% CI: 24.7 to 75.3) versus 53.3% (95% CI: 34.3 to 71.7). Conclusions CARM is comparable with medical judgements in discriminating in-hospital mortality following emergency admission to an elderly care ward. CARM may have a promising role in supporting medical judgements in determining the patient’s risk of death in hospital. Further evaluation of CARM in routine practice is required.
Muhammad Faisal
added a research item
Background: In English hospitals, the patient’s vital signs are monitored and summarised into a National Early Warning Score (NEWS). NEWS is more accurate than the quick sepsis related organ failure assessment (qSOFA) score at identifying patients with sepsis. We investigate the extent to which the accuracy of the NEWS is enhanced by developing computer-aided NEWS (cNEWS) models. We compared three cNEWS models (M0=NEWS alone; M1=M0 + age + sex; M2=M1 + subcomponents of NEWS + diastolic blood pressure) to predict the risk of sepsis. Methods: All adult emergency medical admissions discharged over 24-months from two acute hospitals (YH–York Hospital for model development; NH–Northern Lincolnshire and Goole Hospital for external model validation). We used a validated Canadian method for defining sepsis from administrative hospital data. Findings: The prevalence of sepsis was lower in YH (4.5%=1596/35807) than NH (8.5%=2983/35161). The c-statistic increased across models (YH: M0: 0.705, M1:0.763, M2:0.777; NH:M0: 0.708, M1:0.777, M2:0.791). At NEWS 5+, sensitivity increased (YH: 47.24% vs 50.56% vs 52.69%; NH: 37.91% vs 43.35% vs 48.07%)., the positive likelihood ratio increased (YH: 2.77 vs 2.99 vs 3.06; NH: 3.18 vs 3.32 vs 3.45) and the positive predictive value increased (YH: 11.44% vs 12.24% vs 12.49%; NH: 22.75% vs 23.55% vs 24.21%). Interpretation: From the three cNEWS models, Model M2 is the most accurate. Since it places no additional data collection burden on clinicians and can be automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.
Muhammad Faisal
added a research item
Objectives: The Computer Aided Risk Score (CARS) estimates the risk of death following emergency admission to medical wards using routinely collected vital signs and blood test data. Our aim was to elicit the views of health care practitioners (staff) and Service Users and Carers (SU/C) on i) the potential value, unintended consequences and concerns associated with CARS and practitioner views on ii) the issues to consider before embedding CARS into routine practice. Setting: This study was conducted in two NHS Hospital Trusts in the North of England. Both had inhouse IT development teams, mature IT infrastructure with electronic National Early Warning Score (NEWS) and were capable of integrating NEWS with blood test results. The study focused on emergency medical and elderly admissions units. There were 60 and 39 acute medical/elderly admissions beds at the two NHS hospital Trusts. Participants: We conducted eight focus groups with 45 health care practitioners and two with 11 service users and carers (SU/Cs) in two NHS acute hospitals. Results: Staff and SU/Cs recognised the potential of CARS but were clear the score should not replace or undermine clinical judgments. Staff recognised CARS could enhance clinical decision making/judgments and aid communication with patients. They wanted to understand the components of CARS and be reassured about its accuracy but were concerned about the impact on intensive care and blood tests. Conclusion: Risk scores are widely used in healthcare, but their development and implementation does not usually involve input from practitioners and SU/Cs. We contributed to the development of CARS by eliciting views of staff and SU/Cs who provided important, often complex, insights to support the development and implementation of CARS to ensure successful implementation in routine clinical practice.
Muhammad Faisal
added a research item
Background: The National Early Warning Score (NEWS) is being replaced with NEWS2 which adds 3 points for new confusion or delirium. We estimated the impact of adding delirium on the number of medium/high level alerts that are triggers to escalate care. Methods: Analysis of emergency medical admissions in two acute hospitals (York Hospital (YH) and Northern Lincolnshire and Goole NHS Foundation Trust hospitals (NH)) in England. Twenty per cent were randomly assigned to have delirium. Results: The number of emergency admissions (YH: 35584; NH: 35795), mortality (YH: 5.7%; NH: 5.5%), index NEWS (YH: 2.5; NH: 2.1) and numbers of NEWS recorded (YH: 879193; NH: 884072) were similar in each hospital. The mean number of patients with medium level alerts per day increased from 55.3 (NEWS) to 69.5 (NEWS2), a 25.7% increase in YH and 64.1 (NEWS) to 77.4 (NEWS2), a 20.7% increase in NH. The mean number of patients with high level alerts per day increased from 27.3 (NEWS) to 34.4 (NEWS2), a 26.0% increase in YH and 29.9 (NEWS) to 37.7 (NEWS2), a 26.1% increase in NH. Conclusions: The addition of delirium in NEWS2 will have a substantial increase in medium and high level alerts in hospitalised emergency medical patients. Rigorous evaluation of NEWS2 is required before widespread implementation because the extent to which staff can cope with this increase without adverse consequences remains unknown.
Muhammad Faisal
added 2 research items
We compare the performance of logistic regression with several alternative machine learning methods to estimate the risk of death for patients following an emergency admission to hospital based on the patients' first blood test results and physiological measurements using an external validation approach. We trained and tested each model using data from one hospital ( n = 24,696) and compared the performance of these models in data from another hospital ( n = 13,477). We used two performance measures - the calibration slope and area under the receiver operating characteristic curve. The logistic model performed reasonably well - calibration slope: 0.90, area under the receiver operating characteristic curve: 0.847 compared to the other machine learning methods. Given the complexity of choosing tuning parameters of these methods, the performance of logistic regression with transformations for in-hospital mortality prediction was competitive with the best performing alternative machine learning methods with no evidence of overfitting.
Objectives There are no established mortality risk equations specifically for emergency medical patients who are admitted to a general hospital ward. Such risk equations may be useful in supporting the clinical decision-making process. We aim to develop and externally validate a computer-aided risk of mortality (CARM) score by combining the first electronically recorded vital signs and blood test results for emergency medical admissions. Design Logistic regression model development and external validation study. Setting Two acute hospitals (Northern Lincolnshire and Goole NHS Foundation Trust Hospital (NH)—model development data; York Hospital (YH)—external validation data). Participants Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic National Early Warning Score(s) and blood test results recorded on admission. Results The risk of in-hospital mortality following emergency medical admission was 5.7% (NH: 1766/30 996) and 6.5% (YH: 1703/26 247). The C-statistic for the CARM score in NH was 0.87 (95% CI 0.86 to 0.88) and was similar in an external hospital setting YH (0.86, 95% CI 0.85 to 0.87) and the calibration slope included 1 (0.97, 95% CI 0.94 to 1.00). Conclusions We have developed a novel, externally validated CARM score with good performance characteristics for estimating the risk of in-hospital mortality following an emergency medical admission using the patient’s first, electronically recorded, vital signs and blood test results. Since the CARM score places no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.
Muhammad Faisal
added a project goal
1. To develop and validate CARS in two hospitals.
2. To embed to CARS into routine clinical practice without adverse