Conference PaperPDF Available

Explainable Machine Learning Models for Suicidal Behavior Prediction

  • Faculty of Medicine Universiti Kebangsaan Malaysia
Explainable Machine Learning Models for Suicidal Behavior
Noratikah NORDIN
School of Computer Sciences, Universiti Sains Malaysia,
11800 Pulau Pinang, Malaysia
Zurinahni ZAINOL
School of Computer Sciences, Universiti Sains Malaysia,
11800 Pulau Pinang, Malaysia
Mohd Halim MOHD NOOR
School of Computer Sciences, Universiti Sains Malaysia,
11800 Pulau Pinang, Malaysia
Lai Fong CHAN
Department of Psychiatry, Universiti Kebangsaan
Malaysia Medical Centre, Jalan Yaacob Latif, Bandar Tun
Razak, 56000 Cheras, Kuala Lumpur, Malaysia
In the healthcare setting, suicidal behavior prediction plays an
important role in clinical decision making due to the suicide rate
increasing day by day contributes to a decrease in productivity and
increase in national expenditure. Several machine learning models
are being developed to generate accurate predictions in a suicide
attempt. However, there is a lack of interpretability, explainability
and transparency with these predictive models. Therefore, the aim
of this study is to improve explanations of machine learning models
for predicting suicidal behavior based on clinical data using the
Shapley Additive exPlanations (SHAP) approach. The experiment
shows that machine learning models with SHAP are able to interpret
and understand the nature of an individual’s predictions of suicidal
Applied computing
Life and medical sciences; Health infor-
matics;; Computing methodologies Machine learning.
Explainable AI, Machine Learning, Suicidal Behavior, Prediction
Model, Data Mining
ACM Reference Format:
Noratikah NORDIN, Zurinahni ZAINOL, Mohd Halim MOHD NOOR,
and Lai Fong CHAN. 2022. Explainable Machine Learning Models for Sui-
cidal Behavior Prediction. In 2022 6th International Conference on Medical
and Health Informatics (ICMHI 2022), May 13–15, 2022, Virtual Event, Japan.
ACM, New York, NY, USA, 6 pages.
Suicide remains a serious public health issue and one of the world’s
leading causes of death. Nearly one million individuals have died by
suicide around the world, and the number of suicide attempts is now
Corresponding author:
Corresponding author:
Corresponding author:
Publication rights licensed to ACM. ACM acknowledges that this contribution was
authored or co-authored by an employee, contractor or aliate of a national govern-
ment. As such, the Government retains a nonexclusive, royalty-free right to publish or
reproduce this article, or to allow others to do so, for Government purposes only.
ICMHI 2022, May 13–15, 2022, Virtual Event, Japan
©2022 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-9630-1/22/05. . . $15.00
estimated to be ten to twenty times higher for every suicide [
]. In
addition, low- and middle-income countries account for more than
75 percent of suicides. As a result, the United Nations Sustainable
Development Goals (SDGs), which aim to reduce the worldwide
suicide rates by 2030, have made suicide prevention a global health
Developing models to predict suicidal behavior is one of the
strategies to prevent suicide and improve health systems. Suicidal
behavior includes suicide, suicide attempt, suicidal ideation, and
suicide planning are all terms used to describe thoughts and behav-
iors that lead someone to intentionally or unintentionally ending
their own life [
]. Recent studies on the prediction of suicidal be-
havior have examined a wide range of machine learning techniques
including logistic regression, decision trees, k-nearest neighbors,
support vector machines, random forests, and gradient boosting
[3], [4].
The developed predictive models were evaluated with machine
learning techniques and showed good performance and the ability
to distinguish an individual with suicidal behavior from an individ-
ual without suicidal behavior [
]. However, these predictive models
were poorly interpretable and dicult to trust in the healthcare
system. In fact, the causes of suicidal behavior are very complex
and rely on dynamic interactions among multiple risk factors, in-
cluding sociodemographic, psychological, and biological factors.
The common risk for suicidal behavior is increased among young
people, unmarried people, and people with low income and social
disadvantage [
]. Although a variety of risk factors for suicidal be-
havior predictive models have been discovered, it is not clear how
or why these factors interact to increase the risk for this behavior.
Explainable Articial Intelligence (XAI) oers a technique that
provides more interpretable and explainable predictions. Several
studies have proposed explanatory models for healthcare to help
clinicians and healthcare providers make wise and interpretable
decisions, such as explaining cardiovascular disease risk in patients
and interpreting lung-cancer mortality in electronic health records
]. However, there is no study that has addressed the explanation
of suicidal behavior prediction models. Therefore, this study aims
to improve explanations of machine learning models for predict-
ing suicidal behavior based on clinical data. This study focuses on
explanations using the state-of-the-art XAI post-hoc explainabil-
ity approaches, namely Shapley Additive exPlanations (SHAP), to
explain individuals with suicidal behavior.
The paper is organized as follows: Section 2 discusses the cur-
rent approaches of suicidal behavior prediction models. Section 3
ICMHI 2022, May 13–15, 2022, Virtual Event, Japan NORATIKAH NORDIN et al.
discusses the method for explaining the machine learning model
and Section 4 presents the machine learning model’s outcomes, and
nally, Section 5 summarizes the conclusion with future develop-
Data mining and machine learning approaches have been used in
clinical psychology and psychiatry research to overcome the con-
ventional statistical techniques, particularly in the area of suicidal
behavior [
], [
]. In order to predict suicidal behavior, researchers
frequently employ logistic regression and decision trees [
], [
Su et al. [
] used logistic regression to predict suicide risk in chil-
dren and adolescents. The study found that the model performed
better with an accuracy of 0.81. Edgcomb et al. [
] proposed a de-
cision tree classier for predicting suicidal behavior and self-injury
in adults with severe mental illness. The results showed that the
classication model is able to predict the risk of suicide attempt
with good performance (accuracy 0.80).
In addition, articial neural networks (ANN) and support vector
machines (SVM) have also been shown to be good at predicting sui-
cidal behavior [
]. The study by Amini et al. [
] found that SVM
performed better than other models (decision tree, naïve Bayes,
logistic regression) in assessing the high-risk group for suicide with
an accuracy of 0.68. Most researchers have attempted to develop
models to predict suicidal behavior using machine learning tech-
niques and found that the performance is good and gives excellent
results. However, these models are dicult to interpret when try-
ing to understand the risk of suicidal behavior that may lead to
Explainable Articial Intelligence (XAI) was introduced to im-
prove the interpretability and explainability of machine learning
predictions. There are two methods for interpretability, namely
intrinsic approaches and post-hoc approaches. Intrinsic explain-
ability approaches, also known as transparent models, are a set of
methods that are inherently transparent and easy to understand
such as linear/logistic regression, decision trees, and rule-based
learning [
]. Post-hoc explainability approaches such as Local
Interpretable Model-Agnostic Explanations (LIME), Anchors, and
Shapley Additive exPlanations (SHAP) are used to explain complex
machine learning techniques or opaque models such as random
forests, support vector machines, and articial neural networks.
After machine learning models are trained, the explanation method
is applied to extract information from the trained models without
interfering or disturbing the training process [7].
Although previous studies have shown that both methods of
interpretability XAI are able to provide useful information and un-
derstanding of predictions, the post-hoc explainability approaches
have received more research attention due to their exibility that
works separately from the machine learning models [
]. In fact, no
study examines the post-hoc explainability approaches that explain
the clinical understanding of suicidal behavior prediction models.
Therefore, the main contribution of this study is to utilize Shapley
Additive ExPlanations (SHAP) that can explain the machine learn-
ing models that provide predictions about individuals with suicidal
Models for predicting suicidal behavior are explained in three steps
in this study, as shown in Figure 1. The data is rst collected and
pre-processed, and then three machine learning models (logistic
regression, decision tree, support vector machine) are used to clas-
sify those who have attempted suicide and those who have not at-
tempted suicide. Finally, the machine learning model that achieves
the highest performance is explained comprehensively by the SHAP
approach based on global explanation and local explanation.
3.1 Data Collection and Pre-processing
Suicidal behavior is dened as a set of thoughts and behaviors that
result in someone taking their own life. Therefore, the term suicidal
behavior in this study refers to suicide attempts. A suicide attempt
is a non-fatal, self-inicted destructive act in which a person inten-
tionally harms himself or herself with the purpose of dying and
survives [
]. The study used clinical research data from Universiti
Kebangsaan Malaysia Medical Centre (UKMMC), Malaysia. There
are 75 inpatient psychiatric patients with depressive disorders in
the dataset [
]. The dataset contains 18 variables, including patient
demographic, and clinical information, as well as patient classi-
cation (suicide attempt or no suicide attempt) as shown in Table
3.2 Machine Learning Models
The predictive model for suicidal behavior was developed using
three machine learning models: logistic regression, decision tree,
and support vector machine. The reasons for choosing these ma-
chine learning models are robust results and good predictive power.
Logistic regression and decision tree are known as transparent
models, while support vector machines are an opaque model that
is dicult to understand [7].
Logistic regression is the common machine learning for predict-
ing suicidal behavior. It is a classication model that categorizes
observations into separate groups. To obtain a probability value,
logistic regression transforms its output using a more complex cost
function, known as the logistic sigmoid function [
]. This model
assumes a linear dependence between the predicted and predicted
variables, which makes it dicult to exibly t the data. Therefore,
logistic regression model rigidity makes this model transparent
A decision tree is a non-parametric and non-linear classication
dened as a classication scheme that generates a tree and a set
of rules from a given dataset [
]. Decision trees are recursively
generated from a dataset to form ‘nodes’ and ‘leaves’. The nodes
(also called decision nodes) dene splitting conditions for features,
and the leaves, called terminal nodes, are labelled with a class.
Classication starts from the root node to one of the leaves by
moving from one node to another. The leaves represent the risk of
suicide attempt (yes or no) to be inferred in the context of suicidal
behavior, while the nodes reect the features that lead to this suicide
risk in suicidal behavior.
Support Vector Machines (SVMs) are type of models rooted
deeply in geometrical approaches which is to identify the optimal
hyperplanes that separate any class of input spaces. The optimal
hyperplanes have a maximum margin resulting from discriminant
Explainable Machine Learning Models for Suicidal Behavior Prediction ICMHI 2022, May 13–15, 2022, Virtual Event, Japan
Figure 1: Framework of explaining suicidal behavior prediction models
Table 1: Data description
Data category Data items (risk factors) Values
Demographic Gender 0 Male, 1 Female
Ethnicity/ Race 0 Malay, 1 Chinese, 2 - Indian
Religion 0 Muslim, 1 Buddhist, 2 Hindu, 3 - Christian
Marital status 0 Unmarried, 1 Married
Clinical information Suicide ideation 0 No, 1 Yes
Melancholic features 0 No, 1 Yes
Psychosis features 0 No, 1 Yes
Anxiety disorder 0 No, 1 Yes
Severity of depression 0 Mild, 1 Moderate, 2 Severe
Medical problem 0 No, 1 Yes
Nicotine dependence 0 No, 1 Yes
Alcohol abuse 0 No, 1 Yes
Any substance abuse 0 No, 1 Yes
Sexual abuse 0 No, 1 Yes
Mood stabilizer use 0 No, 1 Yes
History of hospitalization 0 No, 1 Yes
Past suicide attempts 0 No, 1 Yes
Family history of suicide attempts 0 No, 1 Yes
Output Suicide attempt 0 No (no suicide attempt), 1 Yes (suicide attempt)
boundaries (dividing lines). Statistical importance weights can be
assigned to the features using a variety of kernels and parameters.
Due to their excellent prediction and generalization capabilities,
SVMs are among the most widely used machine learning models,
however they are very complex and opaque models [13].
3.3 Explainable Articial Intelligence (XAI)
The main goal of this study is to improve explanations for machine
learning models predicting suicidal behavior by applying post-hoc
explainability approaches. Post-hoc explainability approaches can
be divided into two techniques, model-specic and model-agnostic.
Model-specic is the technique that design to explain and exploit
the parameters based on their internal model, such as structure
or weights, and is not readily transferable to other models, while
model-agnostic is the technique that extracts post-hoc explanations
by treating the original model as a black box and not depending
on the structure of the internal models. This study focused on the
model-agnostic technique because of its model exibility, explana-
tion exibility and representation exibility [18].
SHapley Additive exPlanations (SHAP) is a model-agnostic ex-
planation that gives each input feature an important value for a
given prediction [
]. SHAP is based on cooperative game theory
principles and the important value calculated probabilistically using
the Shapley value to determine the contribution of players in the
game to the nal game outcome. In a coalition game, for example,
each subset of players is referred to as a “coalition” and the game
assigns a value to each coalition based on the quality of its outcome.
The Shapley value is a formula constructed to fairly measure each
player’s contribution to the coalition of all players. This formula is
completed by considering all possible player orders. Each player’s
Shapley value is the average marginal contribution to all possible
coalitions. The Shapley values can be calculated to learn to dis-
tribute the pay-out fairly for all the features by formulating the
features as players in a coalition game. This indicates that the SHAP
for the suicide attempt features is used to determine
the ratio of a single feature’s contribution based on the weight of
ICMHI 2022, May 13–15, 2022, Virtual Event, Japan NORATIKAH NORDIN et al.
Table 2: Performance results of machine learning models
Classication Logistic regression Decision tree Support vector machines
Accuracy 0.75 0.71 0.82
Precision 0.79 0.74 0.80
Specicity 0.73 0.72 0.72
Sensitivity (recall) 0.75 0.75 0.83
all features’ contribution, as stated in Equation 1.
𝑗 (1)
represents the number of features;
, set of features
with order;
, contribution of a set of features with order;
𝑗 {𝑗})
, contribution of a set of features with feature
This study aims to improve the explanations of machine learning
models for predicting suicidal behavior based on clinical data. We
used logistic regression, decision tree and support vector machine
as machine learning models. Because the number of samples is lim-
ited, three-fold cross-validation was utilized to evaluate the models
in order to increase generalization and avoid overtting [
]. Cross-
validation is a widely used method to overcome the limitation of a
small dataset when evaluation machine learning models [
]. For
the development of predictive models, the analysis was carried out
in the Python programming language (version 3.8.5). The perfor-
mance of three machine learning models was evaluated based on
accuracy, precision, specicity, and sensitivity for predicting the
classes (non-suicide attempters and suicide attempters).
The performance results of each machine learning model are
shown in Table 2. Overall, the machine learning models achieved
moderate performance (accuracy
0.71 0.82) for predicting suici-
dal behavior from the clinical data. As shown in Table 2, the best
performance was achieved by the support vector machine which
had a precision of 0.80, followed by logistic regression (0.79) and
decision tree (0.74). However, the logistic regression has a slightly
higher specicity of 0.73 than the decision tree and the support
vector machine, both with 0.72. Although support vector machine
has a moderate specicity, the model is able to classify an individual
with suicidal behavior who has a negative test result due to the
high sensitivity value (0.83), resulting in low specicity value.
Therefore, it can be concluded that support vector machine out-
performs logistic regression and decision tree in predicting suicidal
behavior. Although the dataset used in this study diers due to con-
dentiality, this study aligns with a recent work in which support
vector machine was able to predict suicidal behavior [
]. However,
the support vector machine is a black box model. The explanation
of support vector machine is necessary for suicidal behavior to gain
insight and understand which risk factors aect the predictions
while maintaining good prediction performance. Thus, we integrate
Shapley Additive ExPlanations (SHAP) method with a complex ma-
chine learning model (support vector machine) and provide two
types of explanations: global explanation, and local explanation.
4.1 Global Explanation
The importance of features that inuence the suicidal behavior
can be interpretated using SHAP, which indicated using absolute
Shapley values. Thus, we obtain a global explanation by computing
the mean Shapley values for each feature across patient samples.
Figure 2 illustrates the results of global explanation for the support
vector machine in predicting suicidal behavior.
Figure 2 shows the features in descending order based on the im-
portance values for the predictions (outcomes). The color indicates
that the value of the feature aects the prediction. Blue represents
class 0 (no suicide attempt) and red represents class 1 (suicide at-
tempt). The feature importance shows that religion, past suicide
attempt, depression severity, medical problems, and ethnicity are
the most important feature that inuences the prediction of suicidal
behavior. This can also be seen in Figure 2, where family history of
suicide attempts, hospitalization history, and mood stabilizers use
are the least important in predicting suicidal behavior.
4.2 Local Explanation
SHAP provide a great understanding of local explanation. For com-
plex machine learning models, local explanations give a local under-
standing of how and why a particular prediction was made. Using
SHAP, the explanations for support vector machine prediction are
represented in a force plot. Figure 3 is the force plot for an example
of an individual with suicidal behavior.
According to Figure 3, the function
is the model’ output
(the predicted probability for this patient), and the base value is
the model’s average prediction (0.31). The features that drive the
prediction higher are represented by red arrows, while the features
that the prediction lower are represented by blue arrows. In Figure
3, despite having mild depression (blue arrow), this patient has a
high probability of attempting suicide (0.54) due to the features that
increase the prediction, such as religion, history of suicide attempt,
medical problems, and suicidal ideation. These descriptions of an
individual with suicidal behavior are based on Table 1, which shows
that the rate of suicidal ideation is 1 for presence, while the rate of
medical problem is 1 for presence. In addition, the rates for religion
and race/ethnicity are Muslim and Malay, respectively. The rate of
past suicide attempts is categorized as presence (1) in Figure 3, as
describe Table 1, whereas the rate of depression severity is 0 for
mild severity.
The current predictive model for suicidal behavior has shown
features importance based on ranking score [
], but this feature
Explainable Machine Learning Models for Suicidal Behavior Prediction ICMHI 2022, May 13–15, 2022, Virtual Event, Japan
Figure 2: SHAP feature importance - SHAP values per feature averaged over all the samples
Figure 3: Local explanation for an individual with suicidal behavior
importance did not adequately interpret and explain the features
that drive into the prediction. Using the global explanation and
the local explanation from SHAP method for complex models, we
can make reliable decisions and develop a clear understanding of
which individuals are at risk for attempting suicide. With the given
complex features and predictors, this study will help clinicians in
treating patients. In addition, it has been shown that the importance
of features by the SHAP improves the understanding of model
performance compared to conventional machine learning models.
In this study, we improve explanations for machine learning models
in predicting suicidal behavior from clinical dataset. We found that
support vector machine achieves higher performance in predicting
suicidal behavior compared to logistic regression and decision tree.
In addition, we provide insights into explaining the prediction
suicidal behavior by using SHAP, model-agnostic explainability
approach. Religion, past suicide attempt, medical problems, severity
of depression, and race were identied as the most importance
features in predicting suicidal behavior. The explanatory power
of the model is necessary to ensure truthful and accurate results
before it is used, particularly in health care settings. However, the
disadvantage of SHAP is that as the number of features increases,
the computational time for calculating the Shapley values increases.
This problem will be addressed in the future when the risk factors
for suicidal behavior are dynamic and high dimensional (many
clinical features) and there are dependencies among features.
The authors would like to thank all parties that support and involve
directly or indirectly into this research, especially Universiti Sains
Malaysia, Pulau Pinang and Universiti Kebangsaan Malaysia Medi-
cal Centre, Kuala Lumpur. This research is supported by the Min-
istry of Higher Education Malaysia for Fundamental Research Grant
Scheme (FRGS) with Project Code: FRGS/1/2020/ICT02/USM/02/5.
World Health Organization. Suicide. Retrieved Aug 20, 2021 from https://www. sheets/detail/suicide
Rory C O’Connor and Matthew K Nock. The psychology of suicidal behaviour.
Lancet Psychiatry. 1, 1 (Jun. 2014), 73–85. doi: 10.1016/S2215-0366(14)70222-6.
Jun Su Jung, Sung Jin Park, Eun Young Kim, Kyoung-Sae Na, Young Jae Kim, and
Kwang Gi Kim. Prediction models for high risk of suicide in Korean adolescents
ICMHI 2022, May 13–15, 2022, Virtual Event, Japan NORATIKAH NORDIN et al.
using machine learning techniques. PLOS ONE. 14, 6 (June 2019), e0217639. doi:
Bumjo Oh, Je-Yeon Yun, Eun Chong Yeo, Dong-Hoi Kim, Jin Kim, and Bum-
Joo Cho. Prediction of suicidal ideation among korean adults using machine
learning: A cross-sectional study. Psychiatry Investig. 17, 4 (April 2020), 331–340.
doi: 10.30773/pi.2019.0270.
Taylor A Burke, Brooke A Ammerman, and Ross Jacobucci. The use of machine
learning in the study of suicidal and non-suicidal self-injurious thoughts and
behaviors: A systematic review. J. Aect. Disord. 245 (Februari 2019), 869–884.
doi: 10.1016/j.jad.2018.11.073.
Joseph. C. Franklin, Jessica D. Ribeiro et al. Risk factors for suicidal thoughts and
behaviors: A meta-analysis of 50 years of research. Psychol. Bull. 143, 2 (January
2017), 187–232. doi: 10.1037/bul0000084.
Talal A. A. Abdullah, Mohd Soperi Mohd Zahid, and Waleed Ali. A review of
interpretable ml in healthcare: taxonomy, applications, challenges, and future
directions. Symmetry. 13, 12 (December 2021), 2439. doi: 10.3390/sym13122439.
Sumitra Velupillai, Gerfo Hadlaczky et al. Risk assessment tools and data-driven
approaches for predicting and preventing suicidal behavior. Front. Psychiatry. 10,
36 (February 2019), 1-8. doi: 10.3389/fpsyt.2019.00036.
Payan Amini, Hasan Ahmadinia, Jalal Poorolajal, and Mohammad Moqaddasi
Amiri. Evaluating the high-risk groups for suicide: A comparison of logistic
regression, support vector machine, decision tree and articial neural network.
Iran. J. Public Health. 45, 9 (September 2016), 1179–1187.
[10] Juliet Beni Edgcomb, Trevor Shaddox, Gerhard Hellemann, and John O. Brooks.
Predicting suicidal behavior and self-harm after general hospitalization of adults
with serious mental illness. J. Psychiatr. Res., 136 (April 2021), 515–521. doi:
Chang Su, Robert Aseltine, Riddhi Doshi, Kun Chen, Steven C. Rogers, and Fei
Wang. Machine learning for suicide risk prediction in children and adolescents
with electronic health records. Transl. Psychiatry. 10, 1 (December 2020). 41. 1-10.
doi: 10.1038/s41398-020-01100-0.
Adam Horvath, Mark Dras, Catie C. W. Lai, and Simon Boag. Predicting suicidal
behavior without asking about suicidal ideation: machine learning and the role
of borderline personality disorder criteria. Suicide Life. Threat. Behav. 51, 3 (June
2021), 455–466. doi: 10.1111/sltb.12719.
Vaishak Belle and Ioannis Papantonis. Principles and practice of ex-
plainable machine learning. Front. Big Data. 4 (July 2021), 688969. doi:
Diego De Leo, Benjamin Goodfellow et al. International study of denitions of
English-language terms for suicidal behaviours: A survey exploring preferred
terminology. BMJ Open. 11, 2 (February 2021), e043409. doi: 10.1136/bmjopen-
Lai Fong Chan, T. Maniam, and A. S. Shamsul. Suicide attempts among depressed
inpatients with depressive disorder in a Malaysian sample: Psychosocial and
clinical risk factors. Crisis. 32, 5 (September 2011), 283–287. doi: 10.1027/0227-
Frank Iorno, Nicholas Ho et al. Predicting self-harm within six months after
initial presentation to youth mental health services: A machine learning study.
PLOS ONE. 15, 12, (December 2020), e0243467. doi: 10.1371/journal.pone.0243467.
Jorge Barros, Susana Morales et al. Suicide detection in Chile: proposing a predic-
tive model for suicide risk in a clinical sample of patients with mood disorders.
Rev. Bras. Psiquiatr. 39, 1 (October 2016), 1–11. doi: 10.1590/1516-4446-2015-1877.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Model-agnostic
interpretability of machine learning. In Proceedings of 2016 ICML Workshop
on Human Interpretability in Machine Learning, New York, USA, 91 -95. doi:
Scott M Lundberg and Su-In Lee. 2017. A unied approach to interpret-
ing model predictions. In Proceedings of 31
Conference on Neural Infor-
mation Processing Systems (NIPS 2017), Long Beach, CA, USA, 1 -10. doi:
[20] Noratikah Nordin, Zurinahni Zainol, Mohd Halim Mohd Noor, and C. Lai Fong.
A comparative study of machine learning techniques for suicide attempts pre-
dictive model. Health Informatics J. 27, 1 (January 2021), 146045822198939. doi:
Max Kuhn and Kjell Johnson. 2013. Applied predictive modeling. Springer. New
York, USA. doi:10.1007/978-1-4614-6849-3.
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
We have witnessed the impact of ML in disease diagnosis, image recognition and classification, and many more related fields. Healthcare is a sensitive field related to people’s lives in which decisions need to be carefully taken based on solid evidence. However, most ML models are complex, i.e., black-box, meaning they do not provide insights into how the problems are solved or why such decisions are proposed. This lack of interpretability is the main reason why some ML models are not widely used yet in real environments such as healthcare. Therefore, it would be beneficial if ML models could provide explanations allowing physicians to make data-driven decisions that lead to higher quality service. Recently, several efforts have been made in proposing interpretable machine learning models to become more convenient and applicable in real environments. This paper aims to provide a comprehensive survey and symmetry phenomena of IML models and their applications in healthcare. The fundamental characteristics, theoretical underpinnings needed to develop IML, and taxonomy for IML are presented. Several examples of how they are applied in healthcare are investigated to encourage and facilitate the use of IML models in healthcare. Furthermore, current limitations, challenges, and future directions that might impact applying ML in healthcare are addressed.
Full-text available
Artificial intelligence (AI) provides many opportunities to improve private and public life. Discovering patterns and structures in large troves of data in an automated manner is a core component of data science, and currently drives applications in diverse areas such as computational biology, law and finance. However, such a highly positive impact is coupled with a significant challenge: how do we understand the decisions suggested by these systems in order that we can trust them? In this report, we focus specifically on data-driven methods—machine learning (ML) and pattern recognition models in particular—so as to survey and distill the results and observations from the literature. The purpose of this report can be especially appreciated by noting that ML models are increasingly deployed in a wide range of businesses. However, with the increasing prevalence and complexity of methods, business stakeholders in the very least have a growing number of concerns about the drawbacks of models, data-specific biases, and so on. Analogously, data science practitioners are often not aware about approaches emerging from the academic literature or may struggle to appreciate the differences between different methods, so end up using industry standards such as SHAP. Here, we have undertaken a survey to help industry practitioners (but also data scientists more broadly) understand the field of explainable machine learning better and apply the right tools. Our latter sections build a narrative around a putative data scientist, and discuss how she might go about explaining her models by asking the right questions. From an organization viewpoint, after motivating the area broadly, we discuss the main developments, including the principles that allow us to study transparent models vs. opaque models, as well as model-specific or model-agnostic post-hoc explainability approaches. We also briefly reflect on deep learning models, and conclude with a discussion about future research directions.
Full-text available
Current suicide risk assessments for predicting suicide attempts are time consuming, of low predictive value and have inadequate reliability. This paper aims to develop a predictive model for suicide attempts among patients with depression using machine learning algorithms as well as presents a comparative study on single predictive models with ensemble predictive models for differentiating depressed patients with suicide attempts from non-suicide attempters. We applied and trained eight different machine learning algorithms using a dataset that consists of 75 patients diagnosed with a depressive disorder. A recursive feature elimination was used to reduce the features via three-fold cross validation. An ensemble predictive models outperformed the single predictive models. Voting and bagging revealed the highest accuracy of 92% compared to other machine learning algorithms. Our findings indicate that history of suicide attempt, religion, race, suicide ideation and severity of clinical depression are useful factors for prediction of suicide attempts.
Full-text available
Objectives Explore international consensus on nomenclatures of suicidal behaviours and analyse differences in terminology between high-income countries (HICs) and low/middle-income countries (LMICs). Design An online survey of members of the International Organisation for Suicide Prevention (IASP) used multiple-choice questions and vignettes to assess the four dimensions of the definition of suicidal behaviour: outcome, intent, knowledge and agency. Setting International. Participants Respondents included 126 individuals, 37 from 30 LMICs and 89 from 33 HICs. They included 40 IASP national representatives (65% response rate), IASP regular members (20% response rate) and six respondents from six additional countries identified by other organisations. Outcome measures Definitions of English-language terms for suicidal behaviours. Results The recommended definition of ‘suicide’ describes a fatal act initiated and carried out by the actors themselves. The definition of ‘suicide attempt’ was restricted to non-fatal acts with intent to die, whereas definition of ‘self-harm’ more broadly referred to acts with varying motives, including the wish to die. Almost all respondents agreed about the definitions of ‘suicidal ideation’, ‘death wishes’ and ‘suicide plan’. ‘Aborted suicide attempt’ and ‘interrupted suicide attempt’ were not considered components of ‘preparatory suicidal behaviour’. There were several differences between representatives from HICs and LMICs. Conclusion This international opinion survey provided the basis for developing a transcultural nomenclature of suicidal behaviour. Future developments of this nomenclature should be tested in larger samples of professionals, including LMICs may be a challenge.
Full-text available
Background A priority for health services is to reduce self-harm in young people. Predicting self-harm is challenging due to their rarity and complexity, however this does not preclude the utility of prediction models to improve decision-making regarding a service response in terms of more detailed assessments and/or intervention. The aim of this study was to predict self-harm within six-months after initial presentation. Method The study included 1962 young people (12–30 years) presenting to youth mental health services in Australia. Six machine learning algorithms were trained and tested with ten repeats of ten-fold cross-validation. The net benefit of these models were evaluated using decision curve analysis. Results Out of 1962 young people, 320 (16%) engaged in self-harm in the six months after first assessment and 1642 (84%) did not. The top 25% of young people as ranked by mean predicted probability accounted for 51.6% - 56.2% of all who engaged in self-harm. By the top 50%, this increased to 82.1%-84.4%. Models demonstrated fair overall prediction (AUROCs; 0.744–0.755) and calibration which indicates that predicted probabilities were close to the true probabilities (brier scores; 0.185–0.196). The net benefit of these models were positive and superior to the ‘treat everyone’ strategy. The strongest predictors were (in ranked order); a history of self-harm, age, social and occupational functioning, sex, bipolar disorder, psychosis-like experiences, treatment with antipsychotics, and a history of suicide ideation. Conclusion Prediction models for self-harm may have utility to identify a large sub population who would benefit from further assessment and targeted (low intensity) interventions. Such models could enhance health service approaches to identify and reduce self-harm, a considerable source of distress, morbidity, ongoing health care utilisation and mortality.
Full-text available
Accurate prediction of suicide risk among children and adolescents within an actionable time frame is an important but challenging task. Very few studies have comprehensively considered the clinical risk factors available to produce quantifiable risk scores for estimation of short- and long-term suicide risk for pediatric population. In this paper, we built machine learning models for predicting suicidal behavior among children and adolescents based on their longitudinal clinical records, and determining short- and long-term risk factors. This retrospective study used deidentified structured electronic health records (EHR) from the Connecticut Children’s Medical Center covering the period from 1 October 2011 to 30 September 2016. Clinical records of 41,721 young patients (10–18 years old) were included for analysis. Candidate predictors included demographics, diagnosis, laboratory tests, and medications. Different prediction windows ranging from 0 to 365 days were adopted. For each prediction window, candidate predictors were first screened by univariate statistical tests, and then a predictive model was built via a sequential forward feature selection procedure. We grouped the selected predictors and estimated their contributions to risk prediction at different prediction window lengths. The developed predictive models predicted suicidal behavior across all prediction windows with AUCs varying from 0.81 to 0.86. For all prediction windows, the models detected 53–62% of suicide-positive subjects with 90% specificity. The models performed better with shorter prediction windows and predictor importance varied across prediction windows, illustrating short- and long-term risks. Our findings demonstrated that routinely collected EHRs can be used to create accurate predictive models for suicide risk among children and adolescents.
Full-text available
Mental illnesses, such as depression, are highly prevalent and have been shown to impact an individual’s physical health. Recently, artificial intelligence (AI) methods have been introduced to assist mental health providers, including psychiatrists and psychologists, for decision-making based on patients’ historical data (e.g., medical records, behavioral data, social media usage, etc.). Deep learning (DL), as one of the most recent generation of AI technologies, has demonstrated superior performance in many real-world applications ranging from computer vision to healthcare. The goal of this study is to review existing research on applications of DL algorithms in mental health outcome research. Specifically, we first briefly overview the state-of-the-art DL techniques. Then we review the literature relevant to DL applications in mental health outcomes. According to the application scenarios, we categorize these relevant articles into four groups: diagnosis and prognosis based on clinical data, analysis of genetics and genomics data for understanding mental health conditions, vocal and visual expression data analysis for disease detection, and estimation of risk of mental illness using social media data. Finally, we discuss challenges in using DL algorithms to improve our understanding of mental health conditions and suggest several promising directions for their applications in improving mental health diagnosis and treatment.
Individuals with psychiatric disorders are vulnerable to adverse mental health outcomes following physical illness. This longitudinal cohort study defined risk profiles for readmission for suicidal behavior and self-harm after general hospitalization of adults with serious mental illness. Structured electronic health record data were analyzed from 15,644 general non-psychiatric index hospitalizations of individuals with depression, bipolar, and psychotic disorders admitted to an urban health system in the southwestern United States between 2006 and 2017. Using data from one-year prior to and including index hospitalization, supervised machine learning was implemented to predict risk of readmission for suicide attempt and self-harm in the following year. The Classification and Regression Tree algorithm produced a classification prediction with an area under the receiver operating curve (AUC) of 0.86 (95% confidence interval (CI) 0.74–0.97). Incidence of suicide-related behavior was highest after general non-psychiatric hospitalizations of individuals with prior suicide attempt or self-harm (18%; 69 cases/389 hospitalizations) and lowest after hospitalizations associated with very high medical morbidity burden (0 cases/3090 hospitalizations). Predictor combinations, rather than single risk factors, explained the majority of risk, including concomitant alcohol use disorder with moderate medical morbidity, and age ≤55-years-old with low medical morbidity. Findings suggest that applying an efficient and highly interpretable machine learning algorithm to electronic health record data may inform general hospital clinical decision support, resource allocation, and preventative interventions for medically ill adults with serious mental illness.
Identifying factors that predict who may be at risk of suicide could help prevent suicides via targeted interventions. It is difficult at present, however, to predict which individuals are likely to attempt suicide, even in high ­risk populations such as Borderline Personality Disorder (BPD) sufferers. The complexity of person-­situation dynamics means that relying on known risk factors may not yield accurate enough results for prevention strategies to be successful. Furthermore, risk models typically rely on suicidal thoughts, even though it has been shown that people often intentionally withhold this information. To address these challenges, this study compared the performance of six machine learning and categorisation models in terms of accurately identifying suicidal behaviour in a prison population (n = 353), by including or excluding questions about previous suicide attempts and suicidal ideation. Results revealed that modern machine learning algorithms, specially gradient tree boosting (AUC = .875, F1 = .846), can accurately identify individuals with suicidal behaviour, even without relying on questions about suicidal thoughts, and this accuracy can be maintained with as low as 29 risk factors. Additionally, based on this evidence, it may be possible to implement a decision tree model using known predictors to assess individuals at risk of suicide. These findings highlight that modern classification algorithms do not necessarily require information about suicidal thoughts for modelling suicide and self­ harm behaviour.
Objective: Suicidal ideation (SI) precedes actual suicidal event. Thus, it is important for the prevention of suicide to screen the individuals with SI. This study aimed to identify the factors associated with SI and to build prediction models in Korean adults using machine learning methods. Methods: The 2010-2013 dataset of the Korea National Health and Nutritional Examination Survey was used as the training dataset (n=16,437), and the subset collected in 2015 was used as the testing dataset (n=3,788). Various machine learning algorithms were applied and compared to the conventional logistic regression (LR)-based model. Results: Common risk factors for SI included stress awareness, experience of continuous depressive mood, EQ-5D score, depressive disorder, household income, educational status, alcohol abuse, and unmet medical service needs. The prediction performances of the machine learning models, as measured by the area under receiver-operating curve, ranged from 0.794 to 0.877, some of which were better than that of the conventional LR model (0.867). The Bayesian network, LogitBoost with LR, and ANN models outperformed the conventional LR model. Conclusion: A machine learning-based approach could provide better SI prediction performance compared to a conventional LR-based model. These may help primary care physicians to identify patients at risk of SI and will facilitate the early prevention of suicide.