No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Abstract
Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust in a model. Trust is fundamental if one plans to take action based on a prediction, or when choosing whether or not to deploy a new model. Such understanding further provides insights into the model, which can be used to turn an untrustworthy model or prediction into a trustworthy one. In this work, we propose LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally around the prediction. We further propose a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem. We demonstrate the flexibility of these methods by explaining different models for text (e.g. random forests) and image classification (e.g. neural networks). The usefulness of explanations is shown via novel experiments, both simulated and with human subjects. Our explanations empower users in various scenarios that require trust: deciding if one should trust a prediction, choosing between models, improving an untrustworthy classifier, and detecting why a classifier should not be trusted.
... Feature importance explanations, a significant focus of XAI research, aim to identify the most salient parts of model input given an output. Locally Interpretable Model-agnostic Explanations (LIME) [11] have emerged as a popular method in this domain, providing explanations in the form of simple linear models that approximate the decision surface of a classifier. Recent methodologies have adapted LIME to interpret time series classifiers [12], [13], [14], [15]. ...
... However, many of these explanations tend to be developer-focused and rely on latent layer activations, whereas users often require higher-level abstract explanations [15]. In contrast, model-agnostic methods such as LIME [11] and SHAP [22] provide a greater applicability. LIME approximates complex model predictions by creating a locally interpretable model, such as a linear classifier, around the instance to be explained. ...
... From each dataset, 100 instances are randomly selected from the test set, and the average fidelity score is reported alongside the corresponding 95% confidence interval. To compare the effectiveness of LOMATCE with other XAI methods, including LIME [11], Integrated Gradients (IG) [33], and Shapley Additive Explanations (SHAP) [22], we use performance decrease metrics. These metrics assess how accurately each method identifies critical time steps in time series data that significantly influence model predictions. ...
... The field of EXplainable Artificial Intelligence (XAI) aims to elucidate the decision-making processes of complex ML models and address these challenges. * Equal contribution A prominent class of XAI methods is that of surrogate models which approximate complex, opaque models through simpler, interpretable ones [1]. Surrogate models serve as interpretable proxies, allowing operators to understand how input variables influence the ML model's predictions. ...
... II. RELATED WORK Surrogate models in XAI: Surrogate models have been used in the literature as a means to approximate the predictions of black-box AI models, and they are categorized into global and local surrogates. Global surrogate models [2] aim at approximating the behavior of the black-box in the entire dataset, while local surrogate models [1] are trained to provide explanations for individual instances of the dataset. ...
... To evaluate the approximation capability of our approach in a local explainability setting, we assess whether a surrogate model can more accurately approximate the black-box model f θ * obtained by Algorithm 1, compared to black-box models obtained by the baseline algorithms we use in this paper, for each test instance separately. Specifically, we first train a black-box model f θ * with our approach and the baseline algorithms, and then we select a local surrogate method (e.g., LIME [1] or SHAP [14]) to approximate its output for each test instance. ...
Explainable AI is a crucial component for edge services, as it ensures reliable decision making based on complex AI models. Surrogate models are a prominent approach of XAI where human-interpretable models, such as a linear regression model, are trained to approximate a complex (black-box) model's predictions. This paper delves into the balance between the predictive accuracy of complex AI models and their approximation by surrogate ones, advocating that both these models benefit from being learned simultaneously. We derive a joint (bi-level) training scheme for both models and we introduce a new algorithm based on multi-objective optimization (MOO) to simultaneously minimize both the complex model's prediction error and the error between its outputs and those of the surrogate. Our approach leads to improvements that exceed 99% in the approximation of the black-box model through the surrogate one, as measured by the metric of Fidelity, for a compromise of less than 3% absolute reduction in the black-box model's predictive accuracy, compared to single-task and multi-task learning baselines. By improving Fidelity, we can derive more trustworthy explanations of the complex model's outcomes from the surrogate, enabling reliable AI applications for intelligent services at the network edge.
... Interpretability has become a crucial research topic in machine learning. Methods such as Shapley Additive Explanations (SHAPs) [5], Local Interpretable Model-agnostic Explanations (LIMEs) [6], and TimeSHAP [7] provide insights into feature importance for individual predictions. However, these methods are primarily designed for static datasets or classification tasks, and their effectiveness is limited when applied to time series regression models. ...
... While RFE has proven effective for static datasets, it often disregards temporal order and dependencies in time series. Interpretability methods such as SHAP [5], LIME [6], and TimeSHAP [7] help explain model decisions by estimating the contribution of input features. However, these approaches target classification tasks or static tabular data, limiting their direct applicability to time series regression. ...
... In our research, we propose a gradient-based feature importance method as an innovative explainability framework to address the challenge of interpretability in time-seriesforecasting models. Traditional model interpretation methods, such as SHAP [5], LIME [6], and Anchors [19], while influential in many domains, encounter limitations when interpreting time series models. These methods typically ignore the sequential nature of data or are designed for classification contexts. ...
In the wave of digital transformation and Industry 4.0, accurate time series forecasting has become critical across industries such as manufacturing, energy, and finance. However, while deep learning models offer high predictive accuracy, their lack of interpretability often undermines decision-makers’ trust. This study proposes a linear time series model architecture based on seasonal decomposition. The model effectively captures trends and seasonality using an additive decomposition, chosen based on initial data visualization, indicating stable seasonal variations. An augmented feature generator is introduced to enhance predictive performance by generating features such as differences, rolling statistics, and moving averages. Furthermore, we propose a gradient-based feature importance method to improve interpretability and implement a gradient feature elimination algorithm to reduce noise and enhance model accuracy. The approach is validated on multiple datasets, including order demand, energy load, and solar radiation, demonstrating its applicability to diverse time series forecasting tasks.
... First, SHAP solutions satisfy three essential properties for any additive feature attribution model: local accuracy, missingness, and consistency [17]. Second, SHAP combines the local interpretable model-agnostic explanations (LIME) [18] method, which explains individual predictions using an interpretable model, with Shapley values. This integration allows for faster computation of explanations for complex learning models than directly calculating Shapley values. ...
... As a remedy, SHAP provides a computationally efficient way to calculate Shapley values by iterating over a small subset of possible feature permutations using sampling approximations [37]. It combines previously proposed explainability methods, such as LIME [18] and Deep Learning Important FeaTures (DeepLIFT) [38], to approximate the Shapley values. As an additive feature attribution method, SHAP approximates the output f (x) of the original complex model by summing the scores attributed to each feature, ϕ l , as follows: ...
Artificial intelligence (AI) is expected to significantly
enhance radio resource management (RRM) in sixth-generation
(6G) networks. However, the lack of explainability in complex
deep learning (DL) models poses a challenge for practical implementation. This paper proposes a novel explainable AI (XAI)-
based framework for feature selection and model complexity
reduction in a model-agnostic manner. Applied to a multi-agent
deep reinforcement learning (MADRL) setting, our approach
addresses the joint sub-band assignment and power allocation
problem in cellular vehicle-to-everything (V2X) communications.
We propose a novel two-stage systematic explainability framework leveraging feature relevance-oriented XAI to simplify the
DRL agents. While the former stage generates a state feature
importance ranking of the trained models using Shapley additive
explanations (SHAP)-based importance scores, the latter stage
exploits these importance-based rankings to simplify the state
space of the agents by removing the least important features
from the model’s input. Simulation results demonstrate that the
XAI-assisted methodology achieves∼97% of the original MADRL
sum-rate performance while reducing optimal state features by
∼28%, average training time by∼11%, and trainable weight
parameters by∼46% in a network with eight vehicular pairs.
... The basic LIME method provides explanations by locally approximating the classifier with an interpretable model. To achieve this, a synthetic neighborhood is generated around a given instance by perturbing it, and then, the explanation is obtained by learning an interpretable model using the neighbors (see [4] for more details). ...
This paper focuses on researching and proposing perturbation-based model-agnosticmethods to explain time series classification models. The main objective of this study is to explainthe predictions of the model, or in other words, to give reasons why it classifies a time series intoa particular label in a set of labels. In this work, we aim to provide the reliability of the decisionand the importance of features in the model. Moreover, in real-world time series, variations in thespeed or scale of a particular action can determine the class, so modifying this type of feature leadsto arbitrary explanations of the time series. To achieve the set objectives, we provide two methods,each with its own strategies and advantages: the LIME-based method and the SHAP method, withthe novelty of using them in combination with data perturbation techniques, especially the ones thataffect the above-mentioned characteristics of the time series.
... Since our physicsinformed reward shaping bears distinct physics principles depending on the specific link states, we opt for post-hoc local XAI methods that provide explanations for specific instances instead of creating a white-box surrogate model, e.g., linear and rule-based models [43]- [45], to explain the global navigation behaviors over the entire map. Among existing posthoc local XAI approaches, model-agnostic methods, such as LIME [46], SHAP [47], and Ancors [48], enjoy border applicability since they apply to generic machine learning models. However, these methods often need to generate auditing data under stringent requirements and additional training and computation to examine the black-box model and key features, which is challenging to fulfill in our digital twin environment. ...
Millimeter-wave (mmWave) communication is a vital component of future generations of mobile networks, offering not only high data rates but also precise beams, making it ideal for indoor navigation in complex environments. However, the challenges of multipath propagation and noisy signal measurements in indoor spaces complicate the use of mmWave signals for navigation tasks. Traditional physics-based methods, such as following the angle of arrival (AoA), often fall short in complex scenarios, highlighting the need for more sophisticated approaches. Digital twins, as virtual replicas of physical environments, offer a powerful tool for simulating and optimizing mmWave signal propagation in such settings. By creating detailed, physics-based models of real-world spaces, digital twins enable the training of machine learning algorithms in virtual environments, reducing the costs and limitations of physical testing. Despite their advantages, current machine learning models trained in digital twins often overfit specific virtual environments and require costly retraining when applied to new scenarios. In this paper, we propose a Physics-Informed Reinforcement Learning (PIRL) approach that leverages the physical insights provided by digital twins to shape the reinforcement learning (RL) reward function. By integrating physics-based metrics such as signal strength, AoA, and path reflections into the learning process, PIRL enables efficient learning and improved generalization to new environments without retraining. Digital twins play a central role by providing a versatile and detailed simulation environment that informs the RL training process, reducing the computational overhead typically associated with end-to-end RL methods. Our experiments demonstrate that the proposed PIRL, supported by digital twin simulations, outperforms traditional heuristics and standard RL models, achieving zero-shot generalization in unseen environments and offering a cost-effective, scalable solution for wireless indoor navigation.
... To mitigate the opacity of high-performing black-box models, several post-hoc explanation techniques have been proposed. Methods such as LIME (Local Interpretable Model-agnostic Explanations) [32] and SHAP (SHapley Additive exPlanations) [25] approximate complex models locally with simpler, interpretable surrogates that assign importance scores to individual features. These approaches have proven effective in providing insights into the local behavior of classifiers, albeit sometimes at the cost of stability and global consistency. ...
In recent years, there has been a growing interest in explainable AI methods. We want not only to make accurate predictions using sophisticated neural networks but also to understand what the model's decision is based on. One of the fundamental levels of interpretability is to provide counterfactual examples explaining the rationale behind the decision and identifying which features, and to what extent, must be modified to alter the model's outcome. To address these requirements, we introduce HyConEx, a classification model based on deep hypernetworks specifically designed for tabular data. Owing to its unique architecture, HyConEx not only provides class predictions but also delivers local interpretations for individual data samples in the form of counterfactual examples that steer a given sample toward an alternative class. While many explainable methods generated counterfactuals for external models, there have been no interpretable classifiers simultaneously producing counterfactual samples so far. HyConEx achieves competitive performance on several metrics assessing classification accuracy and fulfilling the criteria of a proper counterfactual attack. This makes HyConEx a distinctive deep learning model, which combines predictions and explainers as an all-in-one neural network. The code is available at https://github.com/gmum/HyConEx.
... As one of the prevailing practices, the rationalization has been extended from the NLP community [104] to the Graph [139] and Vision field [169]. Recently, this development also stems from the multi-modal community. ...
After a decade of prosperity, the development of video understanding has reached a critical juncture, where the sole reliance on massive data and complex architectures is no longer a one-size-fits-all solution to all situations. The presence of ubiquitous data imbalance hampers DNNs from effectively learning the underlying causal mechanisms, leading to significant performance drops when encountering distribution shifts, such as long-tail imbalances and perturbed imbalances. This realization has prompted researchers to seek alternative methodologies to capture causal patterns in video data. To tackle these challenges and increase the robustness of DNNs, causal modeling emerged as a principle to discover the true causal patterns behind the observed correlations. This thesis focuses on the domain of semantic video understanding and explores the potential of causal modeling to advance two fundamental tasks: Video Relation Detection (VidVRD) and Video Question Answering (VideoQA).
... Past works have highlighted the benefit and necessity of moving away from black-box models toward more interpretable approaches. 24,25 PHIL models excel in dataconstrained environments, 22 leverage domain-specific knowledge, and incorporate physics into the deep learning framework as shown in Fig. 1. 21,26,27 This not only yields accurate predictions with less data, or smaller model sizes, but also has the potential to uncover the underlying physics, granting us a deeper understanding of AEM phenomena. 1,3,6,28 Beyond enhancing data-driven models in scientific regression tasks, PHIL approaches have also been utilized to discover unknown physics from data. ...
The advent of artificial intelligence—deep neural networks (DNNs) in particular—has transformed traditional research methods across many disciplines. DNNs are data driven systems that use large quantities of data to learn patterns that are fundamental to a process. In the realm of artificial electromagnetic materials (AEMs), a common goal is to discover the connection between the AEM's geometry and material properties to predict the resulting scattered electromagnetic fields. To achieve this goal, DNNs usually utilize computational electromagnetic simulations to act as ground truth data for the training process, and numerous successful results have been shown. Although DNNs have many demonstrated successes, they are limited by their requirement for large quantities of data and their lack of interpretability. The latter results because DNNs are black-box models, and therefore, it is unknown how or why they work. A promising approach which may help to mitigate the aforementioned limitations is to use physics to guide the development and operation of DNNs. Indeed, this physics-informed learning (PHIL) approach has seen rapid development in the last few years with some success in addressing limitations of conventional DNNs. We overview the field of PHIL and discuss the benefits of incorporating knowledge into the deep learning process and introduce a taxonomy that enables us to categorize various types of approaches. We also summarize deep learning principles which are critical to PHIL understanding and the Appendix covers some of the physics of AEMs. A few specific PHIL works are highlighted and serve as examples of various approaches. Finally, we provide an outlook detailing where the field is currently and what we can expect in the future.
... It is difficult to obtain details regarding model weights that are used in objective functions to generate adversarial samples. [8]. To understand how the model arrive at a decision, we required a representation that is understandable to humans, regardless of the features that were used for training the model. ...
Explainable AI is a strong strategy implemented to understand complex black-box model predictions in a human interpretable language. It provides the evidence required to execute the use of trustworthy and reliable AI systems. On the other hand, however, it also opens the door to locating possible vulnerabilities in an AI model. Traditional adversarial text attack uses word substitution, data augmentation techniques and gradient-based attacks on powerful pre-trained Bidirectional Encoder Representations from Transformers (BERT) variants to generate adversarial sentences. These attacks are generally whitebox in nature and not practical as they can be easily detected by humans E.g. Changing the word from "Poor" to "Rich". We proposed a simple yet effective Grey-box cum Black-box approach that does not require the knowledge of the model while using a set of surrogate Transformer/BERT models to perform the attack using Explainable AI techniques. As Transformers are the current state-of-the-art models for almost all Natural Language Processing (NLP) tasks, an attack generated from BERT1 is transferable to BERT2. This transferability is made possible due to the attention mechanism in the transformer that allows the model to capture long-range dependencies in a sequence. Using the power of BERT generalisation via attention, we attempt to exploit how transformers learn by attacking a few surrogate transformer variants which are all based on a different architecture. We demonstrate that this approach is highly effective to generate semantically good sentences by changing as little as one word that is not detectable by humans while still fooling other BERT models.
... The visualization process is similar to that of the Grad-CAM algorithm, as explained by the pseudocode in Table 6. However, the map was generated using the LIME algorithm [44]. The experimental results obtained in this study were evaluated and compared with those obtained in previous studies. ...
Glioma brain tumors are malignant diseases for which early detection and instant treatment will increase the survival rate. Several studies have reported the efficiency of deep learning convolutional neural networks (CNN) in diagnosing brain tumors using magnetic resonance imaging (MRI). In this study, we investigated the potential of state-of-the-art classifiers to achieve the highest accuracy in the detection of brain tumors using MRI. For this purpose, we introduced a comparative study of eight state-of-the-art classifiers. The methodology comprised three different approaches: (i) an imbalanced dataset, (ii) a balanced dataset using image augmentation, and (iii) ensemble learning using the best of the top five models for majority hard and soft voting. The dataset comprised converted MRI data from the repository of molecular brain neoplasia data (REMBRANDT) and brain tumor segmentation 2021 (BraTS) databases. An increasing number of MRI images and datasets has prevented overfitting. Initially, a preprocessing stage morphological operation and contrast-limited adaptive histogram equalization (CLAHE) algorithm were used to remove skeletons and artifacts and optimize the image contrast for readiness classification. The stochastic gradient descent with the momentum algorithm option was used to train the network. The trained model was used to predict the testing dataset, and the results from each pretrained network were evaluated. The experimental results demonstrated that the prediction accuracy of the trained network was significantly improved using a balanced training dataset. The discriminative image region used to interpret the predicted result using the gradient-weighted class activation mapping (Grad-CAM) algorithm was proposed in the final stage for trustworthiness. The experimental results showed that the best approach was inceptionV3 with a balanced dataset. The accuracy, sensitivity, specificity, and area under the curve were 99.73%, 99.61%, 100%, and 1.00, respectively.
... However, realizing this potential will require addressing several challenges. These include ensuring robust data privacy and security [84], building trust in AI-driven quality assessments [85], and bridging the skill gap in data management professionals. Moreover, ethical considerations in AI deployment for data quality management will need careful attention [86]. ...
This paper presents a theoretical framework for an AI-driven data quality monitoring system designed to address the challenges of maintaining data quality in high-volume environments. We examine the limitations of traditional methods in managing the scale, velocity, and variety of big data and propose a conceptual approach leveraging advanced machine learning techniques. Our framework outlines a system architecture that incorporates anomaly detection, classification, and predictive analytics for real-time, scalable data quality management. Key components include an intelligent data ingestion layer, adaptive preprocessing mechanisms, context-aware feature extraction, and AI-based quality assessment modules. A continuous learning paradigm is central to our framework, ensuring adaptability to evolving data patterns and quality requirements. We also address implications for scalability, privacy, and integration within existing data ecosystems. While practical results are not provided, it lays a robust theoretical foundation for future research and implementations, advancing data quality management and encouraging the exploration of AI-driven solutions in dynamic environments.
... This situation has led to the emergence of the concept of explainable artificial intelligence (XAI) in the field of AI. XAI was essentially created to improve the interpretability of ML and to produce solutions like LIME (Local Interpretable Model-Agnostic Explanations) (Ribeiro et al. 2016), SHAP (SHapley Additive exPlanations) (Lundberg and Lee 2017), and GradCam (Gradient-weighted Class Activation Mapping) (Selvaraju et al. 2020) to increase trust in these systems (Xiong, Li et al. 2024). With the prevalence of neurological and mental health studies that take this issue into account, like studies (Mishra et al. 2021;Ammar and Shaban-Nejad 2020;Jaber et al. 2022) eliminating the deficiencies in AI-based studies and increasing trust in AI will be inevitable. ...
Mental and neurological disorders significantly impact global health. This systematic review examines the use of artificial intelligence (AI) techniques to automatically detect these conditions using electroencephalography (EEG) signals. Guided by Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA), we reviewed 74 carefully selected studies published between 2013 and August 2024 that used machine learning (ML), deep learning (DL), or both of these two methods to detect neurological and mental health disorders automatically using EEG signals. The most common and most prevalent neurological and mental health disorder types were sourced from major databases, including Scopus, Web of Science, Science Direct, PubMed, and IEEE Xplore. Epilepsy, depression, and Alzheimer's disease are the most studied conditions that meet our evaluation criteria, 32, 12, and 10 studies were identified on these topics, respectively. Conversely, the number of studies meeting our criteria regarding stress, schizophrenia, Parkinson's disease, and autism spectrum disorders was relatively more average: 6, 4, 3, and 3, respectively. The diseases that least met our evaluation conditions were one study each of seizure, stroke, anxiety diseases, and one study examining Alzheimer's disease and epilepsy together. Support Vector Machines (SVM) were most widely used in ML methods, while Convolutional Neural Networks (CNNs) dominated DL approaches. DL methods generally outperformed traditional ML, as they yielded higher performance using huge EEG data. We observed that the complex decision process during feature extraction from EEG signals in ML‐based models significantly impacted results, while DL‐based models handled this more efficiently. AI‐based EEG analysis shows promise for automated detection of neurological and mental health conditions. Future research should focus on multi‐disease studies, standardizing datasets, improving model interpretability, and developing clinical decision support systems to assist in the diagnosis and treatment of these disorders.
... Chain-of-Thought Prompting, by encouraging models to articulate their reasoning processes, can enhance model interpretability [24]. Contextual Prompting, by providing models with richer background information and task-speci c context, can improve their understanding of task requirements, leading to more accurate sleep disorder classi cations. ...
Sleep disorders are a major global health issue, affecting nearly a third of the world’s population. This research explores using the sophisticated language and reasoning abilities of large language models (LLMs) to automatically identify these disorders. We trained LLMs on a dataset of sleep patterns, lifestyle choices, and related health factors, employing three novel prompting approaches to guide their design, training, and evaluation of classifiers. Our results show that a support vector machine classifier, identified through decomposed prompting, achieved an impressive 91.9 percent accuracy (F1-score: 0.919), significantly better than traditional zero-shot and few-shot methods. This work demonstrates a powerful integration of LLM’s understanding and reasoning with automated machine learning, offering a promising new direction for sleep disorder classification in health informatics.
Deep learning has significantly aided current advancements in artificial intelligence. Deep learning techniques have significantly outperformed more than typical machine learning approaches, in various fields like Computer Vision, Natural Language Processing (NLP), Robotics Science, and Human-Computer Interaction (HCI). Deep learning models are ineffective in outlining their fundamental mechanism. That's the reason the deep learning model mainly consider as Black-Box. To establish confidence and responsibility, deep learning applications need to explain the model's decision in addition to the prediction of results. The explainable AI (XAI) research has created methods that offer these interpretations for already trained neural networks. It's highly recommended for computer vision tasks relevant to medical science, defense system, and many more. The proposed study is associated with XAI for Sign Language Recognition. The methodology uses an attention-based ensemble learning approach to create a prediction model more accurate. The proposed methodology used ResNet50 with the Self Attention model to design ensemble learning architecture. The proposed ensemble learning approach has achieved remarkable accuracy at 98.20%. In interpreting ensemble learning prediction, the author has proposed SignExplainer to explain the relevancy (in percentage) of predicted results. SignExplainer has illustrated excellent results, compared to other conventional Explainable AI models reported in state of the art.
Sentiment classification is a widely studied problem in natural language processing (NLP) that focuses on identifying the sentiment expressed in text and categorizing it into predefined classes, such as positive, negative, or neutral. As sentiment classification solutions are increasingly integrated into real-world applications, such as analyzing customer feedback in business reviews ( e.g ., hotel reviews) or monitoring public sentiment on social media, the importance of both their accuracy and explainability has become widely acknowledged. In the Turkish language, this problem becomes more challenging due to the complex agglutinative structure of the language. Many solutions have been proposed in the literature to solve this problem. However, it is observed that the solutions are generally based on black-box models. Therefore the explainability requirement of such artificial intelligence (AI) models has become as important as the accuracy of the model. This has further increased the importance of studies based on the explainability of the AI model’s decision. Although most existing studies prefer to explain the model decision in terms of the importance of a single feature/token, this does not provide full explainability due to the complex lexical and semantic relations in the texts. To fill these gaps in the Turkish NLP literature, in this article, we propose a graph-aware explainability solution for Turkish sentiment analysis named TurkSentGraphExp. The solution provides both classification and explainability for sentiment classification of Turkish texts by considering the semantic structure of suffixes, accommodating the agglutinative nature of Turkish, and capturing complex relationships through graph representations. Unlike traditional black-box learning models, this framework leverages an inherent graph representation learning (GRL) model to introduce rational phrase-level explainability. We conduct several experiments to quantify the effectiveness of this framework. The experimental results indicate that the proposed model achieves a 10 to 40% improvement in explainability compared to state-of-the-art methods across varying sparsity levels, further highlighting its effectiveness and robustness. Moreover, the experimental results, supported by a case study, reveal that the semantic relationships arising from affixes in Turkish texts can be identified as part of the model’s decision-making process, demonstrating the proposed solution’s ability to effectively capture the agglutinative structure of Turkish.
Although social determinants of health (SDH) significantly impact health outcomes, reliable data on SDH has not been widely accessible. Despite recent efforts to integrate more individual-level social determinants of health (I-SDH) data into electronic health records, the available sources of SDH that can be accessed at scale remain predominantly community-level, structured SDH (S-SDH). An alternative, though underutilized, strategy for collecting individual-level social determinants of health (I-SDH) is to link with third-party consumer databases that have accumulated socioeconomic status data over an extended period. In this pilot study, we aimed to systematically evaluate the quality and fitness-for-use (FFU) of a widely used consumer database for the prediction of 30-day unplanned hospital-wide, all-cause readmission (HW-ACR). We found that the inclusion of I-SDH not only provided significantly more predictive power over S-SDH but also allowed for the opportunity to explicitly uncover potential social risk factors that would otherwise remain hidden.
Machine learning has significantly advanced breast cancer diagnosis, yet challenges such as high-dimensional data, severe class imbalance, and limited interpretability persist. To address these issues, we proposed a Grammatical Evolution (GE)-based Feature Selection (FS) approach, integrated with a class-balancing technique called STEM, which combines Synthetic Minority Oversampling Technique, Edited Nearest Neighbour and Mixup, effectively handling both inter-class and intra-class imbalance. Our study evaluates the performance of the GE-based FS method against other FS models, including Logistic Regression (LR) and Extreme Gradient Boosting (XGBoost), in identifying critical features for breast cancer diagnosis. The results demonstrate that the GE-based FS method effectively identifies critical features and achieves superior Area Under the Curve (AUC) scores, particularly with smaller subsets of features, unlike LR and XGBoost, which perform optimally with the full feature set. The analysis was conducted on the Digital Database for Screening Mammography and Wisconsin Breast Cancer datasets, which originally contained 52 and 30 features, respectively. The GE-based FS produces the highest AUC with subsets of 10 and 15 features, while LR and XGBoost achieve their best results using the entire feature set, underscoring the superiority of the GE-based FS method.
The success of OpenAI's ChatGPT in 2023 has spurred financial enterprises into exploring Generative AI applications to reduce costs or drive revenue within different lines of businesses in the Financial Industry. While these applications offer strong potential for efficiencies, they introduce new model risks, primarily hallucinations and toxicity. As highly regulated entities, financial enterprises (primarily large US banks) are obligated to enhance their model risk framework with additional testing and controls to ensure safe deployment of such applications. This paper outlines the key aspects for model risk management of generative AI model with a special emphasis on additional practices required in model validation.
Non-small cell lung cancer (NSCLC) constitutes about 85% of all lung cancers and is a leading cause of cancer-related deaths globally. Within the spectrum of lung cancer, Solitary Pulmonary Nodules (SPNs) have become a focal point of research due to their significant implications for mortality. Estimating the malignancy of SPNs is usually performed by medicine experts considering multiple screening methods (Computerised Tomography and Positron Emission Tomography). Machine Learning may simplify this time-consuming procedure and highlight potential human errors. The study presents an efficient methodology for the classification of Solitary Pulmonary Nodule malignancy, addressing critical limitations in existing SPN classification approaches by emphasizing on the synergistic use of PET and CT image features, clinical data, and validation against biopsy-confirmed data. Patient data recorded from a hybrid PET/CT scanner at the University Hospital of Patras, Greece were examined. Human readers annotated 360 SPNs, which were used to train and internally validate the proposed model. SPNs (96) with confirmed histopathological results were used as an external test set. The classification methodology relied on an XGBoost model, which uses manually-extracted SPN features from both imaging modalities. Feature selection was performed to reduce the dimensions of the data and identify the most important predictors. The proposed method exhibited an agreement of 97% with the human readers on the training and validation set. On the external set, the accuracy was 86% (81% sensitivity, 100% specificity). The SUVmax predictor exhibited lower scores (92% agreement on the training and validation set, 85% accuracy on the external set). The model was superior to the advised SUVmax threshold of 2.5 (85% accuracy, 80% sensitivity, 100% specificity).
The accurate prediction of travel time in India is essential for efficient transportation planning and management, especially in busy cities with heavy traffic. Recent research in transportation engineering, computer science, and data analytics has demonstrated the effectiveness of data-driven approaches in predicting travel times more accurately. Accurate road travel time prediction can benefit both transportation service providers and travelers by improving trip planning, reducing travel times, and increasing operational efficiency. Driven by a vision to make a significant contribution to this expansive realm of research, this paper suggests a methodology for forecasting taxi travel times for Bangalore city using data from Uber Movement to increase the effectiveness and dependability of urban transport networks. The model combines regression approaches with spatiotemporal data analysis to capture complicated geographical and temporal trends. The model adds pertinent data, such as weather from Wunderground, and is trained using Uber Movement’s data on cab trips. This report provides a thorough trend analysis of Bangalore city’s commuters. With a Mean Absolute Error (MAE) of 103 s on the Uber data, the ExtraTree regressor outperforms the other regression methods, according to a comparison of their findings. Based on Uber Movement data, the suggested algorithm can accurately predict the time needed to travel between places in Bangalore city. Furthermore, a comprehensive evaluation is undertaken to analyze the suitability of eXplainable Artificial Intelligence (XAI) in effectively addressing the interpretability of black box machine learning models for predicting travel time. The XAI methods, SHAP and LIME are used to evaluate the interpretability of the trained models. As part of future work, this model can be applied to other cities and improve traffic management by taking proactive measures in advance.
Applications of artificial intelligence in predictive analytics of the energy complex meets with risks of incorrect predictions caused by data errors, lack of data, and overtraining of models. Risks can be minimized if to use meta-models which could interpret and explain the provided predictions. One of the most popular methods of the machine learning prediction explanation (XAI) is the SHapley Additive exPlanations method (SHAP). An imprecise SHAP as a modification of the original SHAP is proposed for cases when the class probability distributions are imprecise and represented by sets of distributions. The first idea behind the imprecise SHAP is a new approach for computing the marginal contribution of a feature, which fulfils the important efficiency property of Shapley values. The second idea is an attempt to consider a general approach to calculating and reducing interval-valued Shapley values, which is similar to the idea of reachable probability intervals in the imprecise probability theory. A simple special implementation of the general approach in the form of linear optimization problems is proposed. It is based on using the Kolmogorov–Smirnov distance and imprecise contamination models. Numerical examples with synthetic and real data illustrate the imprecise SHAP.
In contemporary organizational research, when dealing with large heterogeneous datasets and complex relationships, statistical modeling focused on developing substantive explanations typically results in low predictive accuracy. In contrast, machine learning (ML) exhibits remarkable strength for prediction, but suffers from an unexplainable analytical process and output—thus ML is often known as a “black box” approach. The recent development of explainable machine learning (XML) integrates high predictive accuracy with explainability, which combines the advantages inherent in both statistical modeling and ML paradigms. This paper compares XML with statistical modeling and the traditional ML approaches, focusing on an advanced application of XML known as evolving fuzzy system (EFS), which enhances model transparency by clarifying the unique contribution of each modeled predictor. In an illustrative study, we demonstrate two EFS-based XML models and conduct comparative analyses among XML, ML, and statistical models with a commonly-used database in organizational research. Our study offers a thorough description of analysis procedures for implementing XML in organizational research, along with best-practice recommendations for each step as well as Python code to aid future research using XML. Finally, we discuss the benefits of XML for organizational research and its potential development.
This study employed a hybrid methodological approach that integrated machine learning, natural language processing, and deep learning to evaluate AI algorithms for real-time misinformation detection. Using a dataset of 10,000 entries balanced across true, false, and uncertain claims, models were trained and tested on accuracy, precision, recall, F1-score, and receiver operating characteristic area under the curve metrics. Real-time capabilities were assessed on 5,000 live social media posts collected during the Trump versus Harris debate. This allowed for a critical evaluation of the models in real-world settings. Data were sourced from reputable news outlets, misinformation sites, and social media platforms, employing relevant hashtags and keywords related to misinformation narratives. The results show that transformer-based models, particularly bidirectional encoder representations from transformer (BERT) and generative pretrained transformer, outperformed traditional machine learning models like support vector machines, Naive Bayes, and Random Forest, demonstrating superior accuracy, precision, and contextual understanding. BERT achieved the highest performance with an accuracy of 94.8% and a precision of 93.5%. However, the computational demands of these models posed significant challenges for real-time deployment, thus, highlighting the need for optimization strategies such as hyperparameter tuning and model compression. The study also addressed ethical concerns, using adversarial testing and interpretability tools like local interpretable model-agnostic explanations to ensure fairness and transparency. Models trained on fact-checked datasets outperformed those trained on unverified social media data, underscoring the impact of training data quality on model performance.
Introduction: Timely care in a specialised neuro-intensive therapy unit (ITU) reduces mortality and hospital stays, with planned admissions being safer than unplanned ones. However, post-operative care decisions remain subjective. This study used artificial intelligence (AI), specifically natural language processing (NLP) to analyse electronic health records (EHRs) and predict ITU admissions for elective surgery patients. Methods: This study analysed the EHRs of elective neurosurgery patients from University College London Hospital (UCLH) using NLP. Patients were categorised into planned high dependency unit (HDU) or ITU admission; unplanned HDU or ITU admission; or ward / overnight recovery (ONR). The Medical Concept Annotation Tool (MedCAT) was used to identify SNOMED-CT concepts within the clinical notes. We then explored the utility of these identified concepts for a range of AI algorithms trained to predict ITU admission. Results: The CogStack-MedCAT NLP model, initially trained on hospital-wide EHRs, underwent two refinements: first with data from patients with Normal Pressure Hydrocephalus (NPH) and then with data from Vestibular Schwannoma (VS) patients, achieving a concept detection F1-score of 0.93. This refined model was then used to extract concepts from EHR notes of 2,268 eligible neurosurgical patients. We integrated the extracted concepts into AI models, including a decision tree model and a neural time-series model. Using the simpler decision tree model, we achieved a recall of 0.87 (CI 0.82 - 0.91) for ITU admissions, reducing the proportion of unplanned ITU cases missed by human experts from 36% to 4%. Conclusion: The NLP model, refined for accuracy, has proven its efficiency in extracting relevant concepts, providing a reliable basis for predictive AI models to use in clinically valid applications.
As predictive machine learning models become increasingly adopted and advanced, their role has evolved from merely predicting outcomes to actively shaping them. This evolution has underscored the importance of Trustworthy AI, highlighting the necessity to extend our focus beyond mere accuracy and toward a comprehensive understanding of these models' behaviors within the specific contexts of their applications. To further progress in explainability, we introduce Poem, Prefetched Offline Explanation Model, a model-agnostic, local explainability algorithm for image data. The algorithm generates exemplars, counterexemplars and saliency maps to provide quick and effective explanations suitable for time-sensitive scenarios. Leveraging an existing local algorithm, \poem{} infers factual and counterfactual rules from data to create illustrative examples and opposite scenarios with an enhanced stability by design. A novel mechanism then matches incoming test points with an explanation base and produces diverse exemplars, informative saliency maps and believable counterexemplars. Experimental results indicate that Poem outperforms its predecessor Abele in speed and ability to generate more nuanced and varied exemplars alongside more insightful saliency maps and valuable counterexemplars.
In reaction to concerns about a broad range of potential ethical issues, dozens of proposals for addressing ethical aspects of artificial intelligence (AI) have been published. However, many of them are too abstract for being easily translated into concrete designs for AI systems. The various proposed ethical frameworks can be considered an instance of principlism that is similar to that found in medical ethics. Given their general nature, principles do not say how they should be applied in a particular context. Hence, a broad range of approaches, methods, and tools have been proposed for addressing ethical concerns of AI systems. This paper presents a systematic analysis of more than 100 frameworks, process models, and proposed remedies and tools for helping to make the necessary shift from principles to implementation, expanding on the work of Morley and colleagues. This analysis confirms a strong focus of proposed approaches on only a few ethical issues such as explicability, fairness, privacy, and accountability. These issues are often addressed with proposals for software and algorithms. Other, more general ethical issues are mainly addressed with conceptual frameworks, guidelines, or process models. This paper develops a structured list and definitions of approaches, presents a refined segmentation of the AI development process, and suggests areas that will require more attention from researchers and developers.
Model building in machine learning is an iterative process. The performance analysis and debugging step typically involves a disruptive cognitive switch from model building to error analysis, discouraging an informed approach to model building. We present ModelTracker, an interactive visualization that subsumes information contained in numerous traditional summary statistics and graphs while displaying example-level performance and enabling direct error examination and debugging. Usage analysis from machine learning practitioners building real models with ModelTracker over six months shows ModelTracker is used often and throughout model building. A controlled experiment focusing on ModelTracker's debugging capabilities shows participants prefer ModelTracker over traditional tools without a loss in model performance.
In machine learning often a tradeoff must be made between accuracy and intelligibility. More accurate models such as boosted trees, random forests, and neural nets usually are not intelligible, but more intelligible models such as logistic regression, naive-Bayes, and single decision trees often have significantly worse accuracy. This tradeoff sometimes limits the accuracy of models that can be applied in mission-critical applications such as healthcare where being able to understand, validate, edit, and trust a learned model is important. We present two case studies where high-performance generalized additive models with pairwise interactions (GA2Ms) are applied to real healthcare problems yielding intelligible models with state-of-the-art accuracy. In the pneumonia risk prediction case study, the intelligible model uncovers surprising patterns in the data that previously had prevented complex learned models from being fielded in this domain, but because it is intelligible and modular allows these patterns to be recognized and removed. In the 30-day hospital readmission case study, we show that the same methods scale to large datasets containing hundreds of thousands of patients and thousands of attributes while remaining intelligible and providing accuracy comparable to the best (unintelligible) machine learning methods.