FIGURE 5 - available via license: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
Content may be subject to copyright.
The spatial information loss caused by spatial pooling operations in a pyramid network. The highlighted region shows valuable context which is missed in visual explanation of top convolution layer.
Source publication
With advances in NanoSat (CubeSat) and high-resolution sensors, the amount of raw data to be analyzed by human supervisors has been explosively increasing for satellite image analysis. To reduce the raw data, the satellite onboard AI processing with low-power COTS (Commercial, Off-The-Shelf) HW has emerged from a real satellite mission. It filters...
Contexts in source publication
Context 1
... identify the reason why this background bias occurs, we extract visual explanation of intermediate convolution layers in the pyramid feature blocks. The result is shown in Fig. 5. In the pyramid feature blocks in CNN, there are spatial pooling operations between the blocks, which reduces the width and height of the feature map. The lower convolution layers seem to only focus on the small contexts over a local region (see Residual Block 1 and 2). As mentioned in LayerCAM [24], these layers can highlight the ...
Context 2
... mechanisms of the proposed CPANet between the onboard and ground station. In this section, we describe the active learning-based data sampling for finding valuable samples to improve visual explainability of CPANet. To this end, it needs to resolve the problem of filtering the samples showing ambiguous explanation automatically. As shown in Fig. 5, visual explanations which inconsistent in different blocks may indicate the information loss about the target object. Inspired by this phenomenon, we introduce the criteria about how inconsistent visual explanations are with respect to the pyramid feature ...
Citations
... Other studies also evaluate the localization ability of CAM methods by turning the attributions into segmentation masks and comparing the IoU or classification accuracy [132,135,155]. Additionally, [156] compare attention networks and CAM variants on the metrics max-sensitivity and average % drop/increase in confidence. Regarding other xAI approaches, the attention weights are evaluated in [144] by inspecting drops in the accuracy for crop mapping when the transformer model is trained on a subset of dates with the highest attention values. ...
... These insights are used to improve the prediction performance by fine-tuning the model on a hybrid synthetic-real dataset that accounts for these patterns. Further, Kim et al. propose an iterative classification model improvement for satellite onboard processing through a weakly supervised human-in-the-loop process [156]. An inconsistency metric is introduced to measure the similarity of the attribution maps emphasizing commonly highlighted regions to identify uncertain explanations across the attention blocks. ...
... For instance, in the works identified in our review, different model types [103,142,209,214,238,239,248], model seeds or configurations [152,214], and xAI method seeds [188] were used. Furthermore, the outcomes of different xAI methods can be compared, as it is done in [99,108,109,115,132,135,137,143,155,156,193,214,238,261]. For example, Jing et al. apply Integrated Gradients (IG), Expected Gradients (EG), and DeepLift to exhaustively interpret different Recurrent Neural Network (RNN)-based architectures (RNN, LSTM, GRU), also varying their numbers of neurons [214]. ...
In recent years, black-box machine learning approaches have become a dominant modeling paradigm for knowledge extraction in remote sensing. Despite the potential benefits of uncovering the inner workings of these models with explainable AI, a comprehensive overview summarizing the used explainable AI methods and their objectives, findings, and challenges in remote sensing applications is still missing. In this paper, we address these issues by performing a systematic review to identify the key trends of how explainable AI is used in remote sensing and shed light on novel explainable AI approaches and emerging directions that tackle specific remote sensing challenges. We also reveal the common patterns of explanation interpretation, discuss the extracted scientific insights in remote sensing, and reflect on the approaches used for explainable AI methods evaluation. As such, our review provides a complete summary of the state-of-the-art of explainable AI in remote sensing. Further, we give a detailed outlook on the challenges and promising research directions, representing a basis for novel methodological development and a useful starting point for new researchers in the field.
... Further exploration of the explanations provided by the attention mechanisms in these new transformer architectures is still missing in the EO literature and could be a promising direction for latent space analysis-based xAI. It should be noted that attention mechanisms are not solely used in transformers but also in convolutional and recurrent DNNs [17,27,29,41,48,53,59,60]. ...
Training deep learning models on remote sensing imagery is an increasingly popular approach for addressing pressing challenges related to urbanization extreme weather events food security deforestation or poverty reduction. Although explainable AI is getting more frequently utilized to uncover the workings of these models a comprehensive summary of how the fundamental challenges in remote sensing are tackled by explainable AI is still missing. By conducting a scoping review we identify the current works and key trends in the field. Next we relate them to recent developments and challenges in remote sensing and explainable AI. By doing so we also point to novel strategies and promising research directions such as the work on self-interpretable deep learning models and explanation evaluation.
... XAI's compatibility with IoT applications offers up novel paths for upholding standards of ethics and transparency in multiple sectors. Given the device' resource limits, it is arduous to develop comprehensive, reliable applications for users [15], [40]. To be viable, the XAI model calls for strong computational resources [15], [35]. ...
... The two widely reported outcomes of XAI are enhanced user trust [4], [7], [38] and improved performance [4], [7], [13]. XAI makes AI systems more trustworthy for users and boosts user confidence [40] in employing AI systems for decision-making by elaborating to users how the algorithm arrives at an outcome [18], [20], [39]. XAI facilitates users to validate and optimize the decisions taken by the system, enabling adoption [15] and appropriation of technology [4], [7]. ...
... For the foregoing reason, users must be able to validate and optimise their decisions, resulting in more widespread adoption of AI in everyday situations [20], [21]. Using XAI additionally serves in mitigating biases in AI-based automated decision making, rendering the system more fair and reliable [4], [39], [40]. The primary intent of XAI is to elevate AI methods' transparency and persuade users that these techniques may yield reliable decisions [7], [39]. ...
... This is the author's version which has not been fully edited and • Memory occupation • Computation complexity (FLOPs) Resource optimized DL model deployment FIGURE 10. Developed prototype of on-board system [31] with proposed layer-wise channel pruning optimization model. mem-opt, flop-opt and s-ls-global for each sparsity level, which implies the importance of considering layerwise computational characteristics together. ...
... In order to evaluate the feasibility of the proposed methods on the restricted computing environments such as satellite onboard computing system [11], the embedded system board [31] in which inference of the deep learning models can be served in low power management is used as a test environment. Fig. 10 shows the hardware prototype of our embedded system board [31], and it consists of NVIDIA Jetson Nano chipset for managing host/GPU and ASIC chip that is designed to process the inference of the deep learning models. ...
... In order to evaluate the feasibility of the proposed methods on the restricted computing environments such as satellite onboard computing system [11], the embedded system board [31] in which inference of the deep learning models can be served in low power management is used as a test environment. Fig. 10 shows the hardware prototype of our embedded system board [31], and it consists of NVIDIA Jetson Nano chipset for managing host/GPU and ASIC chip that is designed to process the inference of the deep learning models. In the system, 4GB DDR4 memory is available, and ASIC chip is prototyped under Samsung foundry 28nm CMOS process with 200mW power consumption and minimum 7.5W in the entire on-board system. ...
In the constrained computing environments such as mobile device or satellite on-board system, various computational factors of hardware resource can restrict the processing of deep learning (DL) services. Recent DL models such as satellite image analysis mainly require larger resource memory occupation for intermediate feature map footprint than the given memory specification of hardware resource and larger computational overhead (in FLOP) to meet service-level objective in the sense of hardware accelerator. As one of the solutions, we propose a new method of controlling the layer-wise channel pruning in a single-shot manner that can decide how much channels to prune in each layer by observing dataset once without full pretraining. To improve the robustness of the performance degradation, we also propose a layer-wise sensitivity and formulate the optimization problems for deciding layer-wise pruning ratio under target computational constraints. In the paper, the optimal conditions are theoretically derived, and the practical optimum searching schemes are proposed using the optimal conditions. On the empirical evaluation, the proposed methods show robustness on performance degradation, and present feasibility on DL serving under constrained computing environments by reducing memory occupation, providing acceleration effect and throughput improvement while keeping the accuracy performance.
On-board image processing represents a practical application for satellite imagery, aiming to optimize or prioritize data transmission to ground stations, thereby enhancing bandwidth utilization. In addition to their low cost and reduced development time, nanosatellites have experienced significant growth in applications and capabilities as technology advances. Presently, the implementation of on-board image processing has been successfully tested on real nanosatellite missions. Earth observation stands as the most utilized application for nanosatellites equipped with on-board imagers, and integrating image processing in those nanosatellites would enhance the overall performance of the mission. Hence, this article is a systematic review that investigates the current advancements and trends in terms of applications and technologies in the context of image processing on-board nanosatellites. To conduct the study systematically, Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were utilized. After the search and screening process, 73 relevant publications were analyzed to extract pertinent data for the review. The findings outlined a particular interest toward the application of deep learning methods on-board nanosatellites, especially for applications such as image classification or image segmentation.
This systematic literature review employs the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology to investigate recent applications of explainable AI (XAI) over the past three years. From an initial pool of 664 articles identified through the Web of Science database, 512 peer-reviewed journal articles met the inclusion criteria—namely, being recent, high-quality XAI application articles published in English—and were analyzed in detail. Both qualitative and quantitative statistical techniques were used to analyze the identified articles: qualitatively by summarizing the characteristics of the included studies based on predefined codes, and quantitatively through statistical analysis of the data. These articles were categorized according to their application domains, techniques, and evaluation methods. Health-related applications were particularly prevalent, with a strong focus on cancer diagnosis, COVID-19 management, and medical imaging. Other significant areas of application included environmental and agricultural management, industrial optimization, cybersecurity, finance, transportation, and entertainment. Additionally, emerging applications in law, education, and social care highlight XAI’s expanding impact. The review reveals a predominant use of local explanation methods, particularly SHAP and LIME, with SHAP being favored for its stability and mathematical guarantees. However, a critical gap in the evaluation of XAI results is identified, as most studies rely on anecdotal evidence or expert opinion rather than robust quantitative metrics. This underscores the urgent need for standardized evaluation frameworks to ensure the reliability and effectiveness of XAI applications. Future research should focus on developing comprehensive evaluation standards and improving the interpretability and stability of explanations. These advancements are essential for addressing the diverse demands of various application domains while ensuring trust and transparency in AI systems.
Deep neural networks (DNNs) have recently gained significant prominence in various real-world applications such as image recognition, natural language processing, and autonomous vehicles. However, due to their black-box nature in system, the underlying mechanisms of DNNs behind the inference results remain opaque to users. In order to address this challenge, researchers have focused on developing explainable artificial intelligence (AI) algorithms. Explainable AI aims to provide a clear and human-understandable explanation of the model’s decision, thereby building more reliable systems. However, the explanation task differs from well-known inference and training processes as it involves interactions with the user. Consequently, existing inference and training accelerators face inefficiencies when processing explainable AI on edge devices. This article introduces explainable processing unit (EPU), the first hardware accelerator designed for explainable AI workloads. The EPU utilizes a novel data compression format for the output heat maps and intermediate gradients to enhance the overall system performance by reducing both memory footprint and external memory access. Its sparsity-free computing core efficiently handles the input sparsity with negligible control overhead, resulting in a throughput boost of up to 9.48×. It also proposes a dynamic workload scheduling with a customized on-chip network for distinct inference and explanation tasks to maximize internal data reuse hence reducing external memory access by 63.7%. Furthermore, the EPU incorporates point-wise gradient pruning (PGP) that can significantly reduce the size of heat maps by a factor of 7.01× combined with the proposed compression format. Finally, the EPU chip fabricated in a 28 nm CMOS process achieves a remarkable heat map generation rate of 367 frames/s for ResNet-34 while maintaining the state-of-the-art area and energy efficiency of 112.3 GOPS/mm
and 26.55 TOPS/W, respectively.
