Jim Warren’s research while affiliated with University of Auckland and other places

We introduce PropNEAT, a fast backpropagation implementation of NEAT that uses a bidirectional mapping of the genome graph to a layer-based architecture that preserves the NEAT genomes whilst enabling efficient GPU backpropagation. We test PropNEAT on 58 binary classification datasets from the Penn Machine Learning Benchmarks database, comparing the performance against logistic regression, dense neural networks and random forests, as well as a densely retrained variant of the final PropNEAT model. PropNEAT had the second best overall performance, behind Random Forest, though the difference between the models was not statistically significant apart from between Random Forest in comparison with logistic regression and the PropNEAT retrain models. PropNEAT was substantially faster than a naive backpropagation method, and both were substantially faster and had better performance than the original NEAT implementation. We demonstrate that the per-epoch training time for PropNEAT scales linearly with network depth, and is efficient on GPU implementations for backpropagation. This implementation could be extended to support reinforcement learning or convolutional networks, and is able to find sparser and smaller networks with potential for applications in low-power contexts.

Background Automated clinical decision support for risk assessment is a powerful tool in combating cardiovascular disease (CVD), enabling targeted early intervention that could avoid issues of overtreatment or undertreatment. However, current CVD risk prediction models use observations at baseline without explicitly representing patient history as a time series. Objective The aim of this study is to examine whether by explicitly modelling the temporal dimension of patient history event prediction may be improved. Methods This study investigates methods for multivariate sequential modelling with a particular emphasis on long short-term memory (LSTM) recurrent neural networks. Data from a CVD decision support tool is linked to routinely collected national datasets including pharmaceutical dispensing, hospitalization, laboratory test results, and deaths. The study uses a 2-year observation and a 5-year prediction window. Selected methods are applied to the linked dataset. The experiments performed focus on CVD event prediction. CVD death or hospitalization in a 5-year interval was predicted for patients with history of lipid-lowering therapy. Results The results of the experiments showed temporal models are valuable for CVD event prediction over a 5-year interval. This is especially the case for LSTM, which produced the best predictive performance among all models compared achieving AUROC of 0.801 and average precision of 0.425. The non-temporal model comparator ridge classifier (RC) trained using all quarterly data or by aggregating quarterly data (averaging time-varying features) was highly competitive achieving AUROC of 0.799 and average precision of 0.420 and AUROC of 0.800 and average precision of 0.421, respectively. Conclusion This study provides evidence that the use of deep temporal models particularly LSTM in clinical decision support for chronic disease would be advantageous with LSTM significantly improving on commonly used regression models such as logistic regression and Cox proportional hazards on the task of CVD event prediction.

Background Receiver operating characteristic (ROC) analysis is commonly used for comparing models and humans; however, the exact analytical techniques vary and some are flawed. Objectives The aim of the study is to identify common flaws in ROC analysis for human versus model performance, and address them. Methods We review current use and identify common errors. We also review the ROC analysis literature for more appropriate techniques. Results We identify concerns in three techniques: (1) using mean human sensitivity and specificity; (2) assuming humans can be approximated by ROCs; and (3) matching sensitivity and specificity. We identify a technique from Provost et al using dominance tables and cost-prevalence gradients that can be adapted to address these concerns. Conclusion Dominance tables and cost-prevalence gradients provide far greater detail when comparing performances of models and humans, and address common failings in other approaches. This should be the standard method for such analyses moving forward.

Background Wide-ranging concerns exist regarding the use of black-box modelling methods in sensitive contexts such as healthcare. Despite performance gains and hype, uptake of artificial intelligence (AI) is hindered by these concerns. Explainable AI is thought to help alleviate these concerns. However, existing definitions for explainable are not forming a solid foundation for this work. Methods We critique recent reviews on the literature regarding: the agency of an AI within a team; mental models, especially as they apply to healthcare, and the practical aspects of their elicitation; and existing and current definitions of explainability, especially from the perspective of AI researchers. On the basis of this literature, we create a new definition of explainable, and supporting terms, providing definitions that can be objectively evaluated. Finally, we apply the new definition of explainable to three existing models, demonstrating how it can apply to previous research, and providing guidance for future research on the basis of this definition. Results Existing definitions of explanation are premised on global applicability and don’t address the question ‘understandable by whom?’. Eliciting mental models can be likened to creating explainable AI if one considers the AI as a member of a team. On this basis, we define explainability in terms of the context of the model, comprising the purpose, audience, and language of the model and explanation. As examples, this definition is applied to regression models, neural nets, and human mental models in operating-room teams. Conclusions Existing definitions of explanation have limitations for ensuring that the concerns for practical applications are resolved. Defining explainability in terms of the context of their application forces evaluations to be aligned with the practical goals of the model. Further, it will allow researchers to explicitly distinguish between explanations for technical and lay audiences, allowing different evaluations to be applied to each.

INTRODUCTIONNew Zealand veterans may have complex mental and physical complaints related to multiple exposures to war environments. They are entitled to, but often do not, access a range of physical, mental health and social services funded through Veterans’ Affairs New Zealand. eCHAT (electronic Case-finding and Help Assessment Tool) is a self-completed electronic holistic screen for substance misuse, problem gambling, anger control, physical inactivity, depression, anxiety, exposure to abuse; and assesses whether help is wanted for identified issues. AIMA proof-of-concept study was conducted to develop a modified version of eCHAT (VeCHAT) with remote functionality for clinical assessment of mental health and lifestyle issues of contemporary veterans, and assesses acceptability by veterans and Veterans’ Affairs staff, and feasibility of implementation. METHODS We used a co-design approach to develop VeCHAT. Veterans’ Affairs and service organisations invited veterans to remotely complete VeCHAT and a subsequent short online acceptability survey. Veterans’ Affairs medical and case manager staff underwent semi-structured interviews on feasibility and acceptability of VeCHAT use. RESULTSThirty-four veterans completed VeCHAT. The tool proved acceptable to veterans and Veterans’ Affairs staff. Key emergent themes related to tool functionality, design, ways and barriers to use, and suggested improvements. Veterans’ Affairs staff considered VeCHAT use to be feasible with much potential. DISCUSSIONCapacity of Veterans’ Affairs to respond if their engagement with veterans increases and employment of VeCHAT is scaled up, is unknown. Work is needed to assess how introducing VeCHAT as a standard procedure might influence Veterans’ Affairs case management processes.

Multivariate time series provide many challenges for analysis, especially if there are additional requirements of being able to translate analytical results easily to the clinical context, and scale analyses to large numbers of records. The Clinically Comprehensible Clustering of Change in Time Series (C4TS) algorithm is a novel approach to reduce anaesthetic records from a multivariate physiological time series to a set of features, directly relatable to the clinical context, scalable to large data sets, and compatible with a wide range of statistical analyses. These features are windows of the records, located at points of change detected via a drift detector and clustered into cohesive groups. A proof of concept version of C4TS was implemented and evaluated on a retrospective set of 9,246 cardiac, thoracic and otorhinological electronic anaesthetic records from one hospital in New Zealand. C4TS captured distinct features that were interpretable by clinicians, and were amenable to statistical analysis relating them to 30-day mortality yielding plausible results that would be worth exploring with a more robust implementation of the algorithm.

The Healthcare system is exposed to the increasing impact of chronic diseases including cardiovascular diseases; it is of much importance to analyze and understand the health trajectories for efficient planning and fair allotment of resources. This work proposes an approach based on mining clinical data to support the exploration of health trajectories related to cardiovascular diseases. As the health data are highly confidential, we aimed to conduct our experiments using a large, synthetic, longitudinal dataset, constituted to represent the CVD risk factors distribution and temporal sequence of events related to heart failure hospitalization and readmission. This research work analyses and represents the temporal events or states of the patient's trajectory with the aim of understanding the patient's journey in the management of the chronic condition and its complications by using data mining techniques. This study focuses on developing an efficient algorithm to find cohesive clusters for handling the temporal events. Clustering health trajectories have been carried out by proposing an improved version of the Ant-based clustering algorithm. Insights from this study can potentially result in evidence that these approaches are useful in understanding and analyzing patient's health trajectories for better management of the chronic condition and its progression.

Chronic conditions, especially cardiovascular disease account for a large burden on modern healthcare systems. These conditions are by their nature ones that unfold over a long period of time, typically involving many healthcare events, treatments and changes of patient status. The gold standard in public health informatics for risk assessment is regression-based. While these techniques are effective in identifying factors contributing to risk, they produce reductive scores (e.g. probability of a specific class of event, like a heart attack) or binary prediction results, and moreover, they are sequence agnostic. In the area of long-term chronic disease management, multivariate sequential modeling offers an opportunity to forecast disease progression and treatment trajectory in a fine-grained manner in order to aid clinical decision making. This paper investigates the suitability of Long short-term memory, a type of recurrent neural network, in conducting multivariate sequential modeling in the healthcare domain, specifically in the task of forecasting. The eventual goal is to apply this technique to linked New Zealand health data through the Vascular Informatics using Epidemiology and the Web (VIEW) research project. This paper presents initial experiments and results for modeling patients' treatment trajectories during hospitalization using the Medical Information Mart for Intensive Care (MIMIC-III) data set.

Risky health behaviours and mental health problems in adolescents constitute a global concern, and are relatively common in New Zealand. A self-administered screening tool, YouthCHAT, has been developed and validated for general practices and school clinics. As YouthCHAT is more widely deployed, an opportunity emerges to analyse aggregated amassed data to extend epidemiological knowledge of psychosocial issues and facilitate monitoring health of the youth population. The present research designs and prototypes a data analytics portal to facilitate such analysis. The design process is supported by analysis of a dataset generated by a YouthCHAT trial, a scanning study of existing tools that aggregate and analyse health survey data, stakeholder interviews, development of personas and scenarios of use, and creation of a simulated dataset on the scale of an expected future YouthCHAT database. The result is a prototype providing descriptive and inferential analyses and visualisations meeting expected requirements of future users.

This paper describes development of a prototype data analytics portal for analysis of accumulated screening results from eCHAT (electronic Case-finding and Help Assessment Tool). eCHAT allows individuals to conduct a self-administered lifestyle and mental health screening assessment, with usage to date chiefly in the context of primary care waiting rooms. The intention is for wide roll-out to primary care clinics, including secondary school based clinics, resulting in the accumulation of population-level data. Data from a field trial of eCHAT with sexual health questions tailored to youth were used to support design of a data analytics portal for population-level data. The design process included user personas and scenarios, screen prototyping and a simulator for generating large-scale data sets. The prototype demonstrates the promise of wide-scale self-administered screening data to support a range of users including practice managers, clinical directors and health policy analysts.