Fig 1 - uploaded by Sucheta Chauhan
Content may be subject to copyright.
Similar publications
The paper introduces a multilayer long short-term memory (LSTM) based auto-encoder network to spot abnormalities in fetal ECG. The LSTM network was used to detect patterns in the time series, reconstruct errors and classify a given segment as an anomaly or not. The proposed anomaly detection method provides a filtering procedure able to reproduce E...
Citations
... In a similar manner, CNNs that are popular in image recognition are also used in ECG signal and have been developed from advanced DL ventures [32]. Also, the recurrent neural network (RNN), one of which is LSTM, is beneficial in analyzing time series data and then making it suitable for ECG classification [33]. There have been significant advances in the application of DL in ECG signal classification as well as feature extraction. ...
Machine learning (ML)-based in-home electrocardiogram (ECG) systems have emerged as transformative tools, advancing beyond traditional cardiology methods by offering innovative techniques for cardiac care. These systems enable sustained data collection, real-time monitoring of cardiac status, and individualized treatment plans, all while minimizing the need for frequent clinic visits. By leveraging advanced analytics and ML algorithms, in-home ECG systems analyze large-scale datasets to detect patterns and anomalies that might otherwise go unnoticed, providing early alerts and improving patient outcomes. This review examines the latest trends in ML-enhanced in-home ECG technology, emphasizing its functionality in anomaly detection, continuous monitoring, and decision-making processes. The integration of ML not only enhances diagnostic precision but also opens avenues for scalable, personalized, and remote healthcare solutions. Despite these advancements, significant challenges remain, including issues related to data privacy, algorithmic biases, and the reliability of real-world implementations. Addressing these challenges is essential for optimizing the performance and ethical use of these systems. This review also explores opportunities for future research, particularly in improving algorithm robustness and addressing biases to ensure equitable and accurate cardiac care for diverse populations. By integrating state-of-the-art ML techniques, in-home ECG systems are poised to revolutionize contemporary cardiology, reducing healthcare costs and enabling a progressive shift toward accessible, patient-centered care. This comprehensive exploration highlights the potential of ML-based in-home ECG systems to redefine cardiac monitoring and treatment, contributing to the broader transformation of modern healthcare. Received: 13 September 2024 | Revised: 11 December 2024 | Accepted: 31 December 2024 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement Data sharing is not applicable to this article as no new data were created or analyzed in this study. Author Contribution Statement Aqsa Bibi: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Resources, Data curation, Writing - original draft, Writing - review & editing, Visualization, Project administration. Jawwad Sami Ur Rahman: Methodology, Validation, Investigation, Resources, Writing - review & editing, Supervision, Project administration.
... In a similar manner, CNNs that are popular in image recognition are also used in ECG signal and have been developed from advanced DL ventures [32]. Also, the recurrent neural network (RNN), one of which is LSTM, is beneficial in analyzing time series data and then making it suitable for ECG classification [33]. There have been significant advances in the application of DL in ECG signal classification as well as feature extraction. ...
Machine learning (ML)-based in-home electrocardiogram (ECG) systems have emerged as transformative tools, advancing beyond traditional cardiology methods by offering innovative techniques for cardiac care. These systems enable sustained data collection, real-time monitoring of cardiac status, and individualized treatment plans, all while minimizing the need for frequent clinic visits. By leveraging advanced analytics and ML algorithms, in-home ECG systems analyze large-scale datasets to detect patterns and anomalies that might otherwise go unnoticed, providing early alerts and improving patient outcomes. This review examines the latest trends in ML-enhanced in-home ECG technology, emphasizing its functionality in anomaly detection, continuous monitoring, and decision-making processes. The integration of ML not only enhances diagnostic precision but also opens avenues for scalable, personalized, and remote healthcare solutions. Despite these advancements, significant challenges remain, including issues related to data privacy, algorithmic biases, and the reliability of real-world implementations. Addressing these challenges is essential for optimizing the performance and ethical use of these systems. This review also explores opportunities for future research, particularly in improving algorithm robustness and addressing biases to ensure equitable and accurate cardiac care for diverse populations. By integrating state-of-the-art ML techniques, in-home ECG systems are poised to revolutionize contemporary cardiology, reducing healthcare costs and enabling a progressive shift toward accessible, patient-centered care. This comprehensive exploration highlights the potential of ML-based in-home ECG systems to redefine cardiac monitoring and treatment, contributing to the broader transformation of modern healthcare.
... ES [226] Forecasting-based -I Se ✗ DES [226] Forecasting-based -I Se ✗ TES [226] Forecasting-based -I U ✗ ARIMA [211] Forecasting-based ARIMA I U ✓ NoveltySVR [149] Forecasting-based SVM I U ✓ PCI [263] Forecasting-based ARIMA I U ✓ OceanWNN [246] Forecasting-based -I Se ✗ MTAD-GAT [267] Forecasting-based GRU M Se ✓ AD-LTI [252] Forecasting-based GRU M Se ✓ CoalESN [172] Forecasting-based ESN M Se ✓ MoteESN [49] Forecasting-based ESN I Se ✓ HealthESN [51] Forecasting-based ESN I Se ✗ Torsk [96] Forecasting-based ESN M U ✓ LSTM-AD [153] Forecasting-based LSTM M Se ✗ DeepLSTM [50] Forecasting-based LSTM I Se ✗ DeepAnT [167] Forecasting-based LSTM M Se ✗ Telemanom★ [103] Forecasting-based LSTM M Se ✗ RePAD [127] Forecasting-based LSTM M U ✗ NumentaHTM [3] Forecasting-based HTM I U ✓ MultiHTM [249] Forecasting-based HTM M U ✓ RADM [59] Forecasting-based HTM M Se ✓ MAD-GAN [129] Reconstruction-based GAN M Se ✓ VAE-GAN [171] Reconstruction-based GAN M Se ✗ TAnoGAN [17] Reconstruction-based GAN M Se ✗ USAD [8] Reconstruction-based GAN M Se ✗ EncDec-AD [152] Reconstruction-based AE M Se ✗ LSTM-VAE [194] Reconstruction-based AE M Se ✓ DONUT [254] Reconstruction-based AE I Se ✗ BAGEL [130] Reconstruction-based AE I Se ✗ OmniAnomaly [231] Reconstruction-based AE M Se ✗ MSCRED [265] Reconstruction-based AE I U ✗ VELC [264] Reconstruction-based AE I Se ✗ CAE [72,73] Reconstruction-based AE I Se ✗ DeepNAP [117] Reconstruction-based AE M Se ✓ STORN [227] Reconstruction ...
Recent advances in data collection technology, accompanied by the ever-rising volume and velocity of streaming data, underscore the vital need for time series analytics. In this regard, time-series anomaly detection has been an important activity, entailing various applications in fields such as cyber security, financial markets, law enforcement, and health care. While traditional literature on anomaly detection is centered on statistical measures, the increasing number of machine learning algorithms in recent years call for a structured, general characterization of the research methods for time-series anomaly detection. This survey groups and summarizes anomaly detection existing solutions under a process-centric taxonomy in the time series context. In addition to giving an original categorization of anomaly detection methods, we also perform a meta-analysis of the literature and outline general trends in time-series anomaly detection research.
... Wei et al. [17] proposed an anomaly detection method using LSTM-autoencoder architecture, training their model on normal timestamps to identify later anomalies based on error thresholds for indoor quality data. Similarly, in [18], four different types of outliers have been measured in ECG data by applying RNN-LSTM approach. Combining the Wavelet and Hilbert transform with a deep learning algorithm is proposed in [19] to detect irregularities within temporal data patterns. ...
Anomaly detection is critical in various sectors, offering significant advantages by precisely identifying and mitigating system failures and errors, thus preventing severe losses. This study provides a comprehensive comparative anomaly detection analysis through two sophisticated deep learning models: Autoencoder and Long Short-Term Memory (LSTM) Autoencoder, explicitly focusing on temperature and sound time series data. The paper starts with a detailed theoretical foundation, elaborating on both models' mechanics and mathematical formulations. We then advance to the empirical phase, where these models are rigorously trained and tested against a robust dataset. The effectiveness of each model is meticulously assessed through a suite of metrics that gauge their accuracy, sensitivity, and robustness in anomaly detection scenarios. Additionally, we explore the deployment of these models in a real-time environment, where they actively engage in anomaly detection on incoming data streams. The anomalies detected are dynamically displayed on a user-friendly graphical interface, making the results readily accessible and interpretable for users at all levels of technical expertise. Quantitative evaluations of the models are conducted using key performance metrics such as accuracy, precision, recall, and F1-score. Our analysis reveals that the LSTM Autoencoder model excels with an impressive accuracy rate of 99%, while other metrics also affirm its superior performance, marking it as exceptionally effective and reliable. This study highlights the LSTM Autoencoder's advanced anomaly detection capabilities and establishes its superiority over the traditional Autoencoder model in processing complex time series data. The insights gained here are crucial for industries focused on predictive maintenance and quality control, where early anomaly detection is key to maintaining operational efficiency and safety.
... Autoencoders have become one of the leading approaches for unsupervised anomaly detection in sequential data (see e.g. Zimmerer et al. (2018); Chauhan and Vig (2015); Malhotra et al. (2016)). This provides valuable insights into how the previously described methods can be applied to improve the quality of life insurance data. ...
Life insurance, like other forms of insurance, relies heavily on large volumes of data. The business model is based on an exchange where companies receive payments in return for the promise to provide coverage in case of an accident. Thus, trust in the integrity of the data stored in databases is crucial. One method to ensure data reliability is the automatic detection of anomalies. While this approach is highly useful, it is also challenging due to the scarcity of labeled data that distinguish between normal and anomalous contracts or inter\-actions. This manuscript discusses several classical and modern unsupervised anomaly detection methods and compares their performance across two different datasets. In order to facilitate the adoption of these methods by companies, this work also explores ways to automate the process, making it accessible even to non-data scientists.
... Anomaly detection techniques, such as Isolation Forests [80] and One-Class SVM [109], are used to identify data points that deviate significantly from the norm, indicating potential errors or outliers in the data. More recent methods, like Deep Anomaly Detection using Autoencoders and LSTM-based models [20], provide enhanced detection capabilities for complex, high-dimensional data. Automated validation tools are increasingly used to maintain data quality in large-scale ML systems. ...
Artificial intelligence (AI) has transformed various fields, significantly impacting our daily lives. A major factor in AI success is high-quality data. In this paper, we present a comprehensive review of the evolution of data quality (DQ) awareness from traditional data management systems to modern data-driven AI systems, which are integral to data science. We synthesize the existing literature, highlighting the quality challenges and techniques that have evolved from traditional data management to data science including big data and ML fields. As data science systems support a wide range of activities, our focus in this paper lies specifically in the analytics aspect driven by machine learning. We use the cause-effect connection between the quality challenges of ML and those of big data to allow a more thorough understanding of emerging DQ challenges and the related quality awareness techniques in data science systems. To the best of our knowledge, our paper is the first to provide a review of DQ awareness spanning traditional and emergent data science systems. We hope that readers will find this journey through the evolution of data quality awareness insightful and valuable.
... LSTMs also found application in anomaly detection. The study in [14] uses an LSTM network to predict healthy electrocardiogram signals. A further variation of LSTMs has been developed by [15] where LSTM layers are stacked. ...
Traditional anomaly detection methods in time series data often struggle with inherent uncertainties like noise and missing values. Indeed, current approaches mostly focus on quantifying epistemic uncertainty and ignore data-dependent uncertainty. However, consideration of noise in data is important as it may have the potential to lead to more robust detection of anomalies and a better capability of distinguishing between real anomalies and anomalous patterns provoked by noise. In this paper, we propose LSTMAE-UQ (Long Short-Term Memory Autoencoder with Aleatoric and Epistemic Uncertainty Quantification), a novel approach that incorporates both aleatoric (data noise) and epistemic (model uncertainty) uncertainties for more robust anomaly detection. The model combines the strengths of LSTM networks for capturing complex time series relationships and autoencoders for unsupervised anomaly detection and quantifies uncertainties based on the Bayesian posterior approximation method Monte Carlo (MC) Dropout, enabling a deeper understanding of noise recognition. Our experimental results across different real-world datasets show that consideration of uncertainty effectively increases the robustness to noise and point outliers, making predictions more reliable for longer periodic sequential data.
... Technique AD MVP #Params [13,75,78,81,111,112,[114][115][116] LSTM and variants Y N >2 [117][118][119] Other LSTM variants N N >2 [110] Deep ANN Y N >2 [120,121] Deep Embedding Models Y N >2 [122][123][124][125] RNNs Y N >2 [126] GAT Y N >2 [127] GRU Y N >2 [128] DLAE Y N >2 [129] Federated Learning Y P >2 [130] Deep Belief Network (DBN) Y N >2 [131] Normalizing Flow N N >2 [132] DeepAnT N N >2 [133] STORN N N >2 [134][135][136][137] ESNs N N >2 [138] DeepNAP N N >2 [139] DANN N N >2 [140] MTLED N N >2 [141] Hybrid KNN N N >2 [142] Hybrid DAE N N >2 [143] ELM-HTM N N >2 [144] TCN-AE N N >2 [145] LTI N N >2 [146,147] VarAE N N >2 [148] OMES/MTAD-GAT N N >2 [149] HTM/RADM N N >2 [150] MSCRED N N >2 [151] MEGA N N >2 [152] Hybrid ARIMA-WNN N N >2 [153] DL image-based N N >2 [154] GAN-based N N >2 [155] Hybrid VAELSTM N N >2 [156] D 2 S N N >2 [157] GC-ADS N N >2 [158] XceptionTimePlus/Telemanom N N >2 [159] HS-VAE N N >2 [160] FluxEV N N >2 Table 5. Statistical methods. ...
The Internet’s default inter-domain routing system, the Border Gateway Protocol (BGP), remains insecure. Detection techniques are dominated by approaches that involve large numbers of features, parameters, domain-specific tuning, and training, often contributing to an unacceptable computational cost. Efforts to detect anomalous activity in the BGP have been almost exclusively focused on single observable monitoring points and Autonomous Systems (ASs). BGP attacks can exploit and evade these limitations. In this paper, we review and evaluate categories of BGP attacks based on their complexity. Previously identified next-generation BGP detection techniques remain incapable of detecting advanced attacks that exploit single observable detection approaches and those designed to evade public routing monitor infrastructures. Advanced BGP attack detection requires lightweight, rapid capabilities with the capacity to quantify group-level multi-viewpoint interactions, dynamics, and information. We term this approach advanced BGP anomaly detection. This survey evaluates 178 anomaly detection techniques and identifies which are candidates for advanced attack anomaly detection. Preliminary findings from an exploratory investigation of advanced BGP attack candidates are also reported.
... TID detection may be characterized as a distinct anomaly detection problem involving multivariate time-series data. The advancements of deep learning and the development of new network architectures have led to improvements in the performance of time series anomaly detection solutions (Bontemps et al., 2017;Chauhan & Vig, 2015;Hundman et al., 2018;Malhotra et al., 2015Malhotra et al., , 2016Nanduri & Sherry, 2016;Taylor et al., 2016) which-together with new data transformation approaches (Wang & Oates, 2015)-may present an alternative approach for TID detection over Random Forests. Automatic TID detection approaches have the potential to greatly improve the detection of tsunami waves and the characterization of tsunami events by providing open-ocean detection capability unconstrained by a fixed geographic locations such as the DART system. ...
Global Navigation Satellite System Ionospheric Seismology investigates the ionospheric response to earthquakes and tsunamis. These events are known to generate Traveling Ionospheric Disturbances (TIDs) that can be detected through GNSS‐derived Total Electron Content (TEC) observations. Real‐time TID identification provides a method for tsunami detection, improving tsunami early warning systems (TEWS) by extending coverage to open‐ocean regions where buoy‐based warning systems are impractical. Scalable and automated TID detection is, hence, essential for TEWS augmentation. In this work, we present an innovative approach to perform automatic real‐time TID monitoring and detection, using deep learning insights. We utilize Gramian Angular Difference Fields (GADFs), a technique that transforms time‐series into images, in combination with Convolutional Neural Networks (CNNs), starting from VARION (Variometric Approach for Real‐time Ionosphere Observation) real‐time TEC estimates. We select four tsunamigenic earthquakes that occurred in the Pacific Ocean: the 2010 Maule earthquake, the 2011 Tohoku earthquake, the 2012 Haida‐Gwaii, the 2015 Illapel earthquake. The first three events are used for model training, whereas the out‐of‐sample validation is performed on the last one. The presented framework, being perfectly suitable for real‐time applications, achieves 91.7% of F1 score and 84.6% of recall, highlighting its potential. Our approach to improve false positive detection, based on the likelihood of a TID at each time step, ensures robust and high performance as the system scales up, integrating more data for model training. This research lays the foundation for incorporating deep learning into real‐time GNSS‐TEC analysis, offering a joint and substantial contribution to TEWS progression.
... Conventional methods such as ARIMA [11], distance-based models [12], and distributed approaches [13] have formed the backbone of research in this field for many years. Classical methodologies like PCA [14] and deep learning approaches like DeepLSTM [15], which employs advanced LSTM networks for tasks like ECG signal anomaly detection, have shown considerable effectiveness. Similarly, MTAD-GAT [16] utilizes dual graph attention layers for capturing anomalies in multivariate time series. ...
With the rapid development of sensor technology, the anomaly detection of multi-source time series data becomes more and more important. Traditional anomaly detection methods deal with the temporal and spatial information in the data independently, and fail to make full use of the potential of spatio-temporal information. To address this issue, this paper proposes a novel integration method that combines sensor embeddings and temporal representation networks, effectively exploiting spatio-temporal dynamics. In addition, the graph neural network is introduced to skillfully simulate the complexity of multi-source heterogeneous data. By applying a dual loss function—consisting of a reconstruction loss and a prediction loss—we further improve the accuracy of anomaly detection. This strategy not only promotes the ability to learn normal behavior patterns from historical data, but also significantly improves the predictive ability of the model, making anomaly detection more accurate. Experimental results on four multi-source sensor datasets show that our proposed method performs better than the existing models. In addition, our approach enhances the ability to interpret anomaly detection by analyzing the sensors associated with the detected anomalies.
