Conference PaperPDF Available

Abstract

While commonly used approach for disease localization, we propose an approach to detect anomalies by differentiating them from reliable models of anatomies without pathologies. The method is based on a Variational Auto Encoder to learn the anomaly free distribution of the anatomy and a novel image subtraction approach to obtain pixel-precise segmentation of the anomalous regions. The proposed model has been trained with the MOOD dataset. Evaluation is done on BraTS 2019 dataset and a subset of the MOOD, which contain anomalies to be detected by the model.
Unsupervised reconstruction based anomaly detection using a Variational Auto
Encoder










Department of Biomedical Magnetic Resonance, Otto von Guericke University, Magdeburg, Germany,
Data and Knowledge Engineering Group, Otto von Guericke University, Magdeburg, Germany,
Faculty of
Computer Science, Otto von Guericke University, Magdeburg, Germany,
MedDigit, Department of Neurology, Medical Faculty, University Hopspital, Magdeburg, Germany,
German Centre for Neurodegenerative
Diseases, Magdeburg, Germany,
Center for Behavioral Brain Sciences, Magdeburg, Germany,
Leibniz Institute for Neurobiology, Magdeburg, Germany
Synopsis
While commonly used approach for disease localization, we propose an approach to detect anomalies by di�erentiating them from
reliable models of anatomies without pathologies. The method is based on a Variational Auto Encoder to learn the anomaly free
distribution of the anatomy and a novel image subtraction approach to obtain pixel-precise segmentation of the anomalous regions. The
proposed model has been trained with the MOOD dataset. Evaluation is done on BraTS 2019 dataset and a subset of the MOOD, which
contain anomalies to be detected by the model.
Introduction








Methods









μσ




σ
θσθγ









Results and Discussion








Conclusion and future work





Acknowledgements









1
2
3
4
5
6
7



Firefox https://submissions2.mirasmart.com/ISMRM2021/Vie...
1 of 2 12/17/20, 1:09 PM
References














Figures

μσ










Firefox https://submissions2.mirasmart.com/ISMRM2021/Vie...
2 of 2 12/17/20, 1:09 PM
... VAEs are a class of generative model that can be used for dimensionality reduction (Sun, et al., 2018) and anomaly detection (Chatterjee, et al., 2021). Similar to other autoencoder models, VAEs pass data through an encoder, which produces a dense latent representation. ...
... Recurrent architectures, such as long short-term memory networks (LSTMs) or gated recurrent units (GRUs), are often combined with VAEs to improve their performance on sequential data (Lin, et al., 2020). VAEs can be used directly for anomaly detection by considering the reconstruction error (Chatterjee et al., 2021;Lin et al., 2020). The latent representations can also be used as a dense representation of the data (Sun et al., 2018). ...
Article
Full-text available
In recent years deep learning methods, based on reconstruction errors, have facilitated huge improvements in unsupervised anomaly detection. These methods make the limiting assumption that the greater the distance between an observation and a prediction the lower the likelihood of that observation. In this paper we propose conDENSE, a novel anomaly detection algorithm, which does not use reconstruction errors but rather uses conditional density estimation in masked autoregressive flows. By directly estimating the likelihood of data, our model moves beyond approximating expected behaviour with a single point estimate, as is the case in reconstruction error models. We show how conditioning on a dense representation of the current trajectory, extracted from a variational autoencoder with a gated recurrent unit (GRU VAE), produces a model that is suitable for periodic datasets, while also improving performance on non-periodic datasets. Experiments on 31 time-series, including real-world anomaly detection benchmark datasets and synthetically generated data, show that the model can outperform state-of-the-art deep learning methods.
... The methods were compared based on their accuracy in segmenting the anomalies, calculated with Sørensen-Dice coefficient. Initial experiments were performed using a vanilla VAE (Chatterjee et al., 2021), following that, different state-ofthe-art models (the ones mentioned in Sec. 2) were compared. Finally, the three best-performing models in this current experimental setup were chosen as baselines, and final comparisons were performed against the proposed method. ...
... The initial approach of using the vanilla VAE (Chatterjee et al., 2021) was able to detect simple contrast anomaliesanomalies having very different contrast than the rest (e.g. MOOD toy dataset), but in most cases failed to generate an acceptable reconstruction. ...
Article
Full-text available
Expert interpretation of anatomical images of the human brain is the central part of neuro-radiology. Several machine learning-based techniques have been proposed to assist in the analysis process. However, the ML models typically need to be trained to perform a specific task, e.g., brain tumour segmentation or classification. Not only do the corresponding training data require laborious manual annotations, but a wide variety of abnormalities can be present in a human brain MRI — even more than one simultaneously, which renders a representation of all possible anomalies very challenging. Hence, a possible solution is an unsupervised anomaly detection (UAD) system that can learn a data distribution from an unlabelled dataset of healthy subjects and then be applied to detect out-of-distribution samples. Such a technique can then be used to detect anomalies — lesions or abnormalities, for example, brain tumours, without explicitly training the model for that specific pathology. Several Variational Autoencoder (VAE) based techniques have been proposed in the past for this task. Even though they perform very well on controlled artificially simulated anomalies, many of them perform poorly while detecting anomalies in clinical data. This research proposes a compact version of the “context-encoding” VAE (ceVAE) model, combined with pre and post-processing steps, creating a UAD pipeline (StRegA), which is more robust on clinical data and shows its applicability in detecting anomalies such as tumours in brain MRIs. The proposed pipeline achieved a Dice score of 0.642 ± 0.101 while detecting tumours in T2w images of the BraTS dataset and 0.859 ± 0.112 while detecting artificially induced anomalies, while the best performing baseline achieved 0.522 ± 0.135 and 0.783 ± 0.111, respectively.
... The methods were compared based on their accuracy in segmenting the anomalies, calculated with Sørensen-Dice coefficient. Initial experiments were performed using a vanilla VAE (Chatterjee et al., 2021), following that, different state-ofthe-art models (the ones mentioned in Sec. 1.1) were compared. ...
... The initial approach of using the vanilla VAE (Chatterjee et al., 2021) was able to detect simple contrast anomaliesanomalies having very different contrast than the rest (e.g. MOOD toy dataset), but in most cases failed to generate an acceptable reconstruction as it was unable to provide proper latent space representation. ...
Preprint
Full-text available
Expert interpretation of anatomical images of the human brain is the central part of neuro-radiology. Several machine learning-based techniques have been proposed to assist in the analysis process. However, the ML models typically need to be trained to perform a specific task, e.g., brain tumour segmentation or classification. Not only do the corresponding training data require laborious manual annotations, but a wide variety of abnormalities can be present in a human brain MRI - even more than one simultaneously, which renders representation of all possible anomalies very challenging. Hence, a possible solution is an unsupervised anomaly detection (UAD) system that can learn a data distribution from an unlabelled dataset of healthy subjects and then be applied to detect out of distribution samples. Such a technique can then be used to detect anomalies - lesions or abnormalities, for example, brain tumours, without explicitly training the model for that specific pathology. Several Variational Autoencoder (VAE) based techniques have been proposed in the past for this task. Even though they perform very well on controlled artificially simulated anomalies, many of them perform poorly while detecting anomalies in clinical data. This research proposes a compact version of the "context-encoding" VAE (ceVAE) model, combined with pre and post-processing steps, creating a UAD pipeline (StRegA), which is more robust on clinical data, and shows its applicability in detecting anomalies such as tumours in brain MRIs. The proposed pipeline achieved a Dice score of 0.642±\pm0.101 while detecting tumours in T2w images of the BraTS dataset and 0.859±\pm0.112 while detecting artificially induced anomalies, while the best performing baseline achieved 0.522±\pm0.135 and 0.783±\pm0.111, respectively.
... The rationale for this is that anomalies lack the common features that are highly compressed in the latent space, therefore their reconstruction is less precise than that of regular observations [16,17,18,19]. Variational autoencoders (VAE) prove to be especially interesting in this context [20,21,22,23]. Application of RNN-based autoencoders is also considered [24]. ...
Preprint
Full-text available
We propose an End-to-end Convolutional Activation Anomaly Analysis (E2E-CA3^3), which is a significant extension of A3^3 anomaly detection approach proposed by Sperl, Schulze and B\"ottinger, both in terms of architecture and scope of application. In contrast to the original idea, we utilize a convolutional autoencoder as a target network, which allows for natural application of the method both to image and tabular data. The alarm network is also designed as a CNN, where the activations of convolutional layers from CAE are stacked together into k+1k+1-dimensional tensor. Moreover, we combine the classification loss of the alarm network with the reconstruction error of the target CAE, as a "best of both worlds" approach, which greatly increases the versatility of the network. The evaluation shows that despite generally straightforward and lightweight architecture, it has a very promising anomaly detection performance on common datasets such as MNIST, CIFAR-10 and KDDcup99.
... Since the introduction of the VAE model in 2014 by Kingma and Welling, it has been used in a variety of studies for voxel-wise anomaly detection (e.g. [47][48][49]). The ceVAE model has similar architecture as VAE but a more complex definition of the loss. ...
Article
Full-text available
Additive Manufacturing (AM) has emerged as a manufacturing process that allows the direct production of samples from digital models. To ensure that quality standards are met in all samples of a batch, X-ray computed tomography (X-CT) is often used in combination with automated anomaly detection. For the latter, deep learning (DL) anomaly detection techniques are increasingly used, as they can be trained to be robust to the material being analysed and resilient to poor image quality. Unfortunately, most recent and popular DL models have been developed for 2D image processing, thereby disregarding valuable volumetric information. Additionally, there is a notable absence of comparisons between supervised and unsupervised models for voxel-wise pore segmentation tasks. This study revisits recent supervised (UNet, UNet++, UNet 3+, MSS-UNet, ACC-UNet) and unsupervised (VAE, ceVAE, gmVAE, vqVAE, RV-VAE) DL models for porosity analysis of AM samples from X-CT images and extends them to accept 3D input data with a 3D-patch approach for lower computational requirements, improved efficiency and generalisability. The supervised models were trained using the Focal Tversky loss to address class imbalance that arises from the low porosity in the training datasets. The output of the unsupervised models was post-processed to reduce misclassifications caused by their inability to adequately represent the object surface. The findings were cross-validated in a 5-fold fashion and include: a performance benchmark of the DL models, an evaluation of the post-processing algorithm, an evaluation of the effect of training supervised models with the output of unsupervised models. In a final performance benchmark on a test set with poor image quality, the best performing supervised model was UNet++ with an average precision of 0.751 ± 0.030, while the best unsupervised model was the post-processed ceVAE with 0.830 ± 0.003. Notably, the ceVAE model, with its post-processing technique, exhibited superior capabilities, endorsing unsupervised learning as the preferred approach for the voxel-wise pore segmentation task.
... Since the introduction of the VAE model in 2014 by Kingma and Welling, it has been used in a variety of studies for voxel-wise anomaly detection (e.g. [37,38,39]). The implemented VAE architecture follows the one described in a successive paper [20]. ...
Preprint
Full-text available
Additive Manufacturing (AM) has emerged as a manufacturing process that allows the direct production of samples from digital models. To ensure that quality standards are met in all manufactured samples of a batch, X-ray computed tomography (X-CT) is often used combined with automated anomaly detection. For the latter, deep learning (DL) anomaly detection techniques are increasingly, as they can be trained to be robust to the material being analysed and resilient towards poor image quality. Unfortunately, most recent and popular DL models have been developed for 2D image processing, thereby disregarding valuable volumetric information. This study revisits recent supervised (UNet, UNet++, UNet 3+, MSS-UNet) and unsupervised (VAE, ceVAE, gmVAE, vqVAE) DL models for porosity analysis of AM samples from X-CT images and extends them to accept 3D input data with a 3D-patch pipeline for lower computational requirements, improved efficiency and generalisability. The supervised models were trained using the Focal Tversky loss to address class imbalance that arises from the low porosity in the training datasets. The output of the unsupervised models is post-processed to reduce misclassifications caused by their inability to adequately represent the object surface. The findings were cross-validated in a 5-fold fashion and include: a performance benchmark of the DL models, an evaluation of the post-processing algorithm, an evaluation of the effect of training supervised models with the output of unsupervised models. In a final performance benchmark on a test set with poor image quality, the best performing supervised model was MSS-UNet with an average precision of 0.808 ±\pm 0.013, while the best unsupervised model was the post-processed ceVAE with 0.935 ±\pm 0.001. The VAE/ceVAE models demonstrated superior capabilities, particularly when leveraging post-processing techniques.
... Masaki et al. confirmed that abnormal and normal conditions could be separated based on FST by using variational autoencoder(VAE) [4], a deep generative model [5]. Quantitative evaluation of anomaly detection performance has not yet been done. ...
Article
Full-text available
Facial skin temperature (FST) has also gained prominence as an indicator for detecting anomalies such as fever due to the COVID-19. When FST is used for engineering applications, it is enough to be able to recognize normal. We are also focusing on research to detect some anomaly in FST. In a previous study, it was confirmed that abnormal and normal conditions could be separated based on FST by using a variational autoencoder (VAE), a deep generative model. However, the simulations so far have been a far cry from reality. In this study, normal FST with a diurnal variation component was defined as a normal state, and a model of normal FST in daily life was individually reconstructed using VAE. Using the constructed model, the anomaly detection performance was evaluated by applying the Hotelling theory. As a result, the area under the curve (AUC) value in ROC analysis was confirmed to be 0.89 to 1.00 in two subjects.
Chapter
Novelty detection (ND) is a crucial task in machine learning to identify anomalies in the test data in some respects different from the training data. As an anomaly detection method, novelty detection only uses normal samples for model learning, which can well fit most of the natural scenes that the amount of abnormal samples is in fact strongly insufficient, such as network intrusion detection, industrial fault detection, and so on, due to the rareness of abnormal events or the high cost of abnormal samples collection. This paper proposes a reconstruction-based ND scheme by introducing an optimized deep generative model (ODGM), which combines the concept of Variational Auto-encoder (VAE) and the generative adversarial network (GAN) model jointly to efficiently and stably learn the essential characteristics from normal samples. A novelty index is established by combining signal reconstruction loss and feature loss between the original signal of the reconstructed signal based on the ODGM on normal samples for anomaly point identification in the test data. The effectiveness and superiority of the proposed model is validated and compared with other representative deep learning-based novelty detection models on two public data sets.
Article
Full-text available
Deep reinforcement learning augments the reinforcement learning framework and utilizes the powerful representation of deep neural networks. Recent works have demonstrated the remarkable successes of deep reinforcement learning in various domains including finance, medicine, healthcare, video games, robotics, and computer vision. In this work, we provide a detailed review of recent and state-of-the-art research advances of deep reinforcement learning in computer vision. We start with comprehending the theories of deep learning, reinforcement learning, and deep reinforcement learning. We then propose a categorization of deep reinforcement learning methodologies and discuss their advantages and limitations. In particular, we divide deep reinforcement learning into seven main categories according to their applications in computer vision, i.e. (i) landmark localization (ii) object detection; (iii) object tracking; (iv) registration on both 2D image and 3D image volumetric data (v) image segmentation; (vi) videos analysis; and (vii) other applications. Each of these categories is further analyzed with reinforcement learning techniques, network design, and performance. Moreover, we provide a comprehensive analysis of the existing publicly available datasets and examine source code availability. Finally, we present some open issues and discuss future research directions on deep reinforcement learning in computer vision.
Conference Paper
Full-text available
The detection of anomalous structures in natural image data is of utmost importance for numerous tasks in the field of computer vision. The development of methods for unsupervised anomaly detection requires data on which to train and evaluate new approaches and ideas. We introduce the MVTec Anomaly Detection (MVTec AD) dataset containing 5354 high-resolution color images of different object and texture categories. It contains normal, i.e., defect-free, images intended for training and images with anomalies intended for testing. The anomalies manifest themselves in the form of over 70 different types of defects such as scratches, dents, contaminations, and various structural changes. In addition, we provide pixel-precise ground truth regions for all anomalies. We also conduct a thorough evaluation of current state-of-the-art unsupervised anomaly detection methods based on deep architectures such as convolutional autoencoders, generative adversarial networks, and feature descriptors using pre-trained convolutional neural networks, as well as classical computer vision methods. This initial benchmark indicates that there is considerable room for improvement. To the best of our knowledge, this is the first comprehensive, multi-object, multi-defect dataset for anomaly detection that provides pixel-accurate ground truth regions and focuses on real-world applications.
Article
Full-text available
In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients – manually annotated by up to four raters – and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all subregions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.
Variational Autoencoder based Anomaly Detection using Reconstruction Probability
  • Jinwon An
  • S Cho
An, Jinwon and S. Cho. "Variational Autoencoder based Anomaly Detection using Reconstruction Probability." (2015).