Content uploaded by Sæmundur Óskar Haraldsson
Author content
All content in this area was uploaded by Sæmundur Óskar Haraldsson on Jun 04, 2019
Content may be subject to copyright.
A preview of the PDF is not available
Nýsköpun: Getur gervigreind gert endurhæfingu skilvirkari? [Novel Innovation: Can Artificial Intelligence make Rehabilitation more Efficient?]
Abstract
Demand for Vocational Rehabilitation in Iceland has been steadily rising in recent years where the presence of young patients has increased proportionally the most. It is essential that public spending is efficient without compromising the treatment quality. It is worth exploring if a solution for increasing the efficiency in this healthcare section is to use Artificial Intelligence (AI). An innovative project on developing, testing, and implementing specialised AI software in its services is being performed in Janus Rehabilitation. The software, named Völvan in Icelandic, can identify latent areas of possible interest in patient's circumstances which might affect the outcome of their treatment, and assist specialists in providing timely and appropriate interventions. The accuracy, precision, and recall of its predictions have been verified in two recent publications. Völvan seems to be a promising tool for individualised rehabilitation, where patients are dealing with difficult and complex problems. Janus Rehabilitation is in the process of launching Völvan as an unbiased member of the interdisciplinary teams of specialists. The aim of this report is to introduce Völvan and the associated research.
ResearchGate has not been able to resolve any citations for this publication.
Background
Emergency admissions are a major source of healthcare spending. We aimed to derive, validate, and compare conventional and machine learning models for prediction of the first emergency admission. Machine learning methods are capable of capturing complex interactions that are likely to be present when predicting less specific outcomes, such as this one.
Methods and findings
We used longitudinal data from linked electronic health records of 4.6 million patients aged 18–100 years from 389 practices across England between 1985 to 2015. The population was divided into a derivation cohort (80%, 3.75 million patients from 300 general practices) and a validation cohort (20%, 0.88 million patients from 89 general practices) from geographically distinct regions with different risk levels. We first replicated a previously reported Cox proportional hazards (CPH) model for prediction of the risk of the first emergency admission up to 24 months after baseline. This reference model was then compared with 2 machine learning models, random forest (RF) and gradient boosting classifier (GBC). The initial set of predictors for all models included 43 variables, including patient demographics, lifestyle factors, laboratory tests, currently prescribed medications, selected morbidities, and previous emergency admissions. We then added 13 more variables (marital status, prior general practice visits, and 11 additional morbidities), and also enriched all variables by incorporating temporal information whenever possible (e.g., time since first diagnosis). We also varied the prediction windows to 12, 36, 48, and 60 months after baseline and compared model performances. For internal validation, we used 5-fold cross-validation. When the initial set of variables was used, GBC outperformed RF and CPH, with an area under the receiver operating characteristic curve (AUC) of 0.779 (95% CI 0.777, 0.781), compared to 0.752 (95% CI 0.751, 0.753) and 0.740 (95% CI 0.739, 0.741), respectively. In external validation, we observed an AUC of 0.796, 0.736, and 0.736 for GBC, RF, and CPH, respectively. The addition of temporal information improved AUC across all models. In internal validation, the AUC rose to 0.848 (95% CI 0.847, 0.849), 0.825 (95% CI 0.824, 0.826), and 0.805 (95% CI 0.804, 0.806) for GBC, RF, and CPH, respectively, while the AUC in external validation rose to 0.826, 0.810, and 0.788, respectively. This enhancement also resulted in robust predictions for longer time horizons, with AUC values remaining at similar levels across all models. Overall, compared to the baseline reference CPH model, the final GBC model showed a 10.8% higher AUC (0.848 compared to 0.740) for prediction of risk of emergency admission within 24 months. GBC also showed the best calibration throughout the risk spectrum. Despite the wide range of variables included in models, our study was still limited by the number of variables included; inclusion of more variables could have further improved model performances.
Conclusions
The use of machine learning and addition of temporal information led to substantially improved discrimination and calibration for predicting the risk of emergency admission. Model performance remained stable across a range of prediction time windows and when externally validated. These findings support the potential of incorporating machine learning models into electronic health records to inform care and service planning.
Background
Studies have demonstrated that the current US guidelines based on American College of Cardiology/American Heart Association (ACC/AHA) Pooled Cohort Equations Risk Calculator may underestimate risk of atherosclerotic cardiovascular disease (CVD) in certain high‐risk individuals, therefore missing opportunities for intensive therapy and preventing CVD events. Similarly, the guidelines may overestimate risk in low risk populations resulting in unnecessary statin therapy. We used Machine Learning (ML) to tackle this problem.
Methods and Results
We developed a ML Risk Calculator based on Support Vector Machines (SVMs) using a 13‐year follow up data set from MESA (the Multi‐Ethnic Study of Atherosclerosis) of 6459 participants who were atherosclerotic CVD‐free at baseline. We provided identical input to both risk calculators and compared their performance. We then used the FLEMENGHO study (the Flemish Study of Environment, Genes and Health Outcomes) to validate the model in an external cohort. ACC/AHA Risk Calculator, based on 7.5% 10‐year risk threshold, recommended statin to 46.0%. Despite this high proportion, 23.8% of the 480 “Hard CVD” events occurred in those not recommended statin, resulting in sensitivity 0.76, specificity 0.56, and AUC 0.71. In contrast, ML Risk Calculator recommended only 11.4% to take statin, and only 14.4% of “Hard CVD” events occurred in those not recommended statin, resulting in sensitivity 0.86, specificity 0.95, and AUC 0.92. Similar results were found for prediction of “All CVD” events.
Conclusions
The ML Risk Calculator outperformed the ACC/AHA Risk Calculator by recommending less drug therapy, yet missing fewer events. Additional studies are underway to validate the ML model in other cohorts and to explore its ability in short‐term CVD risk prediction.
Adaptive systems will become increasingly important for health care in coming years as costs and workload grow. The need for efficient rehabilitation will expand which will be fulfilled by information technologies. This paper presents a novel implementation and application of a dynamic prediction software in vocational rehabilitation. The software is made adaptable with a Genetic Improvement of software methodology and utilised to predict fluctuations in patient's perceived quality of life. Results of accuracy, recall and precision were better than 90% for the classification of the shifts and the mean absolute error in predictions of the quantity of the shifts was low. The findings of the present study support that it is possible to predict fluctuations in quality of life on average based on the status six months prior. Professionals could therefore intervene accordingly and increase the possibility of successful rehabilitation. The significant long term effect on health care from applying the prediction tool might be reduced cost and overall improved quality of life.
With the expanding load on healthcare and consequent strain on budget, the demand for tools to increase efficiency in treatments is rising. The use of prediction models throughout the treatment to identify risk factors might be a solution. In this paper we present a novel implementation of a prediction tool and the first use of a dynamic predictor in vocational rehabilitation practice. The tool is periodically updated and improved with Genetic Improvement of software. The predictor has been in use for 10 months and is evaluated on predictions made during that time by comparing them with actual treatment outcome. The results show that the predictions have been consistently accurate throughout the patients' treatment. After approximately 3 week learning phase, the predictor classified patients with 100% accuracy and precision on previously unseen data. The predictor is currently being successfully used in a complex live system where specialists have used it to make informed decisions.
Background:
Information regarding the determinants of successful vocational rehabilitation (VR) is scarce.
Objective:
Investigate whether sex, duration, quality of life and financial circumstances influence the success of VR.
Methods:
The study group consisted of 519 participants, (293 women, 56%) who finished VR in the period 2000-2014. The group was divided into the following subgroups: dropouts, unsuccessful and successful VR. Data were collected by questionnaire.
Results:
Income had the most impact on whether the outcome was successful. Having supplemental income when entering the VR program increased the likelihood of a successful conclusion, odds ratio (OR) 5.60 (95% CI; 2.43-13.59) (p < 0.001), being on sick leave OR 5.02 (95% CI 1.93-13.79) (p < 0.001) or rehabilitation pension OR 1.93 (95% CI 1.07-3.52) (p < 0.03). The participants in the successful sub-group were older (p < 0.06) and stayed in rehabilitation longer (p < 0.001), compared to those who were unsuccessful. However, the effect on OR was limited: 1.03 (95% CI 1.01-1.06) and 1.04 (95% CI 1.02-1.07), respectively.
Conclusions:
For this sample, supplemental income appears to be the most important factor for a successful rehabilitation outcome. Checking financial status at the beginning of the rehabilitation process could minimize financial strain and increase the likelihood of success.
We conducted a qualitative study in the Emergency Departments (EDs) of four hospitals in order to investigate the perceived scope and causes of 'diagnostic overshadowing' - the misattribution of physical symptoms to mental illness - and other challenges involved in the diagnostic process of people with mental illness who present in EDs with physical symptoms. Eighteen doctors and twenty-one nurses working in EDs and psychiatric liaisons teams in four general hospitals in the UK were interviewed. Interviewees were asked about cases in which mental illness interfered with diagnosis of physical problems and about other aspects of the diagnostic process. Interviews were transcribed and analysed thematically. Interviewees reported various scenarios in which mental illness or factors related to it led to misdiagnosis or delayed treatment with various degrees of seriousness. Direct factors which may lead to misattribution in this regard are complex presentations or aspects related to poor communication or challenging behaviour of the patient. Background factors are the crowded nature of the ED environment, time pressures and targets and stigmatising attitudes held by a minority of staff. The existence of psychiatric liaison team covering the ED twenty-four hours a day, seven days a week, can help reduce the risk of misdiagnosis of people with mental illness who present with physical symptoms. However, procedures used by emergency and psychiatric liaison staff require fuller operationalization to reduce disagreement over where responsibilities lie.
Objective:
The aim of this study was to examine the outcome of a vocational rehabilitation programme for patients with brain tumours and to determine whether the outcome could be predicted at point of referral to the service.
Methods:
Data was collected for 34 patients with brain tumours referred to the Macmillan vocational rehabilitation service. Work status at baseline (time of referral) and at discharge was compared. Logistic regression analyses were computed to identify which variables (demographic, tumour and treatment, functional and vocational) predicted work status at discharge from the service.
Results:
Significantly, more patients were working at discharge from the service than at baseline. Having at least some physical disability decreased the likelihood of being in work at discharge from the service.
Conclusions:
The vocational rehabilitation programme for brain tumour survivors showed significant improvement over time. Functional ability affected the likelihood of working to some extent. Vocational rehabilitation services should be available to patients with brain tumours and should focus on supporting patients wishing to return to or maintain their current work. However, more support for brain tumour patients with physical impairments is needed.
We describe the architecture, problems and lessons learned from building a Health Kiosk from commercial, off-the shelf Personal Health Devices and a computer with a touch-screen interface. The kiosk is used autonomously by patients to measure vital data prior to a consultation, in the scope of a population screening, or for routinely monitoring. The prototype was tested in multiple events with different user groups, aided by observer reporting and user questionnaires to assess problems and difficulties with the interface and device usage.
Background:
Antidepressant treatment efficacy is low, but might be improved by matching patients to interventions. At present, clinicians have no empirically validated mechanisms to assess whether a patient with depression will respond to a specific antidepressant. We aimed to develop an algorithm to assess whether patients will achieve symptomatic remission from a 12-week course of citalopram.
Methods:
We used patient-reported data from patients with depression (n=4041, with 1949 completers) from level 1 of the Sequenced Treatment Alternatives to Relieve Depression (STAR*D; ClinicalTrials.gov, number NCT00021528) to identify variables that were most predictive of treatment outcome, and used these variables to train a machine-learning model to predict clinical remission. We externally validated the model in the escitalopram treatment group (n=151) of an independent clinical trial (Combining Medications to Enhance Depression Outcomes [COMED]; ClinicalTrials.gov, number NCT00590863).
Findings:
We identified 25 variables that were most predictive of treatment outcome from 164 patient-reportable variables, and used these to train the model. The model was internally cross-validated, and predicted outcomes in the STAR*D cohort with accuracy significantly above chance (64·6% [SD 3·2]; p<0·0001). The model was externally validated in the escitalopram treatment group (N=151) of COMED (accuracy 59·6%, p=0.043). The model also performed significantly above chance in a combined escitalopram-buproprion treatment group in COMED (n=134; accuracy 59·7%, p=0·023), but not in a combined venlafaxine-mirtazapine group (n=140; accuracy 51·4%, p=0·53), suggesting specificity of the model to underlying mechanisms.
Interpretation:
Building statistical models by mining existing clinical trial data can enable prospective identification of patients who are likely to respond to a specific antidepressant.
Funding:
Yale University.