Figure 4 - uploaded by Raghavendra Selvan
Content may be subject to copyright.
Box plot of test set performance for the baseline and proposed models using block masking and diffused noise data augmentations (last two rows of Table 1)
Source publication
Pulmonary opacification is the inflammation in the lungs caused by many respiratory ailments, including the novel corona virus disease 2019 (COVID-19). Chest X-rays (CXRs) with such opacifications render regions of lungs imperceptible , making it difficult to perform automated image analysis on them. In this work, we focus on segmenting lungs from...
Context in source publication
Context 1
... best dice overlap (p < 0.05) and binary accuracy (p < 0.001) is obtained by the proposed model with variational data imputation when augmented with block masking and diffused noise, reported in the last row of Table 1. Box plots with performance measures comprising all 30 test set images for the two models used with block masking and diffused noise augmentations are shown in Figure 4. Predicted segmentations for three test set images are visualized in Figure 5 along with the ground truth annotations. ...
Similar publications
The problem of missing data, usually absent incurated and competition-standard datasets, is an unfortunate reality for most machine learning models used in industry applications. Recent work has focused on understanding the nature and the negative effects of such phenomena, while devising solutions for optimal imputation of the missing data, using...
Citations
... In our implementation, lung segmentation has been carried out using the solution proposed by Selvan et al. [37], based on an U-net architecture and a variational encoder for data imputation, trained on public datasets labeled for tuberculosis detection [24], and specifically the model provided by the authors 1 . The segmentation masks are post-processed using connected component analysis to exclude small erroneous regions and binary morphological closing to fill any small holes in the masks (see Figure 3 for examples). ...
... The segmentation masks are post-processed using connected component analysis to exclude small erroneous regions and binary morphological closing to fill any small holes in the masks (see Figure 3 for examples). Figure 3: Examples of lung mask obtained using [37] before (a) and after (b) the postprocessing step. (c) Example of an inversely-segmented image used, in Section 3.5, to train a model excluding the lung region data (evidence of external confoundig factors). ...
Deep learning technologies have already demonstrated a high potential to build diagnosis support systems from medical imaging data, such as Chest X-Ray images. However, the shortage of labeled data in the medical field represents one key obstacle to narrow down the performance gap with respect to applications in other image domains. In this work, we investigate the benefits of a curricular Self-Supervised Learning (SSL) pretraining scheme with respect to fully-supervised training regimes for pneumonia recognition on Chest X-Ray images of Covid-19 patients. We show that curricular SSL pretraining, which leverages unlabeled data, outperforms models trained from scratch, or pretrained on ImageNet, indicating the potential of performance gains by SSL pretraining on massive unlabeled datasets. Finally, we demonstrate that top-performing SSLpretrained models show a higher degree of attention in the lung regions, embodying models that may be more robust to possible external confounding factors in the training datasets, identified by previous works.
... For example, in [31], researchers achieve state-of-the-art performance and can provide clinically interpretable saliency maps, which are very useful for COVID-19 diagnosis. In addition, in [32], researchers also applied the image segmentation method on CXR to extend it to cases with extreme abnormalities. CT-scans, in addition to CXR, have been shown in previous studies to be an extremely useful tool for diagnosing COVID-19 [33][34][35]. ...
COVID-19 is the disease that has spread over the world since December 2019. This disease has a negative impact on individuals, governments, and even the global economy, which has caused the WHO to declare COVID-19 as a PHEIC (Public Health Emergency of International Concern). Until now, there has been no medicine that can completely cure COVID-19. Therefore, to prevent the spread and reduce the negative impact of COVID-19, an accurate and fast test is needed. The use of chest radiography imaging technology, such as CXR and CT-scan, plays a significant role in the diagnosis of COVID-19. In this study, CT-scan segmentation will be carried out using the 3D version of the most recommended segmentation algorithm for bio-medical images, namely 3D UNet, and three other architectures from the 3D UNet modifications, namely 3D ResUNet, 3D VGGUNet, and 3D DenseUNet. These four architectures will be used in two cases of segmentation: binary-class segmentation, where each architecture will segment the lung area from a CT scan; and multi-class segmentation, where each architecture will segment the lung and infection area from a CT scan. Before entering the model, the dataset is preprocessed first by applying a minmax scaler to scale the pixel value to a range of zero to one, and the CLAHE method is also applied to eliminate intensity in homogeneity and noise from the data. Of the four models tested in this study, surprisingly, the original 3D UNet produced the most satisfactory results compared to the other three architectures, although it requires more iterations to obtain the maximum results. For the binary-class segmentation case, 3D UNet produced IoU scores, Dice scores, and accuracy of 94.32%, 97.05%, and 99.37%, respectively. For the case of multi-class segmentation, 3D UNet produced IoU scores, Dice scores, and accuracy of 81.58%, 88.61%, and 98.78%, respectively. The use of 3D segmentation architecture will be very helpful for medical personnel because, apart from helping the process of diagnosing someone with COVID-19, they can also find out the severity of the disease through 3D infection projections.
... Segmented lung-only CXR images were used in the subsequent tasks of bone suppression, super-resolution, and COVID-19 classification. Interested readers can refer to the paper by Selvan et al. (62) on the detailed design of the segmentation model with a variational autoencoder. ...
Background:
The coronavirus disease 2019 (COVID-19) led to a dramatic increase in the number of cases of patients with pneumonia worldwide. In this study, we aimed to develop an AI-assisted multistrategy image enhancement technique for chest X-ray (CXR) images to improve the accuracy of COVID-19 classification.
Methods:
Our new classification strategy consisted of 3 parts. First, the improved U-Net model with a variational encoder segmented the lung region in the CXR images processed by histogram equalization. Second, the residual net (ResNet) model with multidilated-rate convolution layers was used to suppress the bone signals in the 217 lung-only CXR images. A total of 80% of the available data were allocated for training and validation. The other 20% of the remaining data were used for testing. The enhanced CXR images containing only soft tissue information were obtained. Third, the neural network model with a residual cascade was used for the super-resolution reconstruction of low-resolution bone-suppressed CXR images. The training and testing data consisted of 1,200 and 100 CXR images, respectively. To evaluate the new strategy, improved visual geometry group (VGG)-16 and ResNet-18 models were used for the COVID-19 classification task of 2,767 CXR images. The accuracy of the multistrategy enhanced CXR images was verified through comparative experiments with various enhancement images. In terms of quantitative verification, 8-fold cross-validation was performed on the bone suppression model. In terms of evaluating the COVID-19 classification, the CXR images obtained by the improved method were used to train 2 classification models.
Results:
Compared with other methods, the CXR images obtained based on the proposed model had better performance in the metrics of peak signal-to-noise ratio and root mean square error. The super-resolution CXR images of bone suppression obtained based on the neural network model were also anatomically close to the real CXR images. Compared with the initial CXR images, the classification accuracy rates of the internal and external testing data on the VGG-16 model increased by 5.09% and 12.81%, respectively, while the values increased by 3.51% and 18.20%, respectively, for the ResNet-18 model. The numerical results were better than those of the single-enhancement, double-enhancement, and no-enhancement CXR images.
Conclusions:
The multistrategy enhanced CXR images can help to classify COVID-19 more accurately than the other existing methods.
... The combination of U-net and VAE in [20] is conducted in this work. The block diagram of this model is illustrated in Figure 3. Firstly, the training for lung area segmentation is conducted. ...
... Since the U-net need to be trained for lung segmentation before removing the decoder part to merge with RepVGG model. This work is applied by [20] which is conducted by the Pulmonary CXR Abnormalities dataset [29]. The training and validation ratio is 75/25 and the augmentation for training data is also employed for enriching the number of training images. ...
Millions of fatal cases have been reported worldwide as a result of the Coronavirus disease 2019 (COVID-19) outbreak. In order to stop the spreading of disease, early diagnosis and quarantine of infected people are one of the most essential steps. Therefore, due to the similar symptoms of SARS-CoV-2 virus and other pneumonia, identifying COVID-19 still exists some challenges. Reverse transcription-polymerase chain reaction (RT-PCR) is known as a standard method for the COVID-19 diagnosis process. Due to the shortage of RT-PCR toolkit in global, Chest X-Ray (CXR) image is introduced as an initial step to support patient’s classification. Applying deep learning in medical imaging becomes an advanced research trend in many applications. In this research, RepVGG pre-trained model is considered to be used as the main backbone of the network. Besides, variational autoencoder (VAE) is firstly trained to perform lung segmentation. Afterwards, the encoder part in VAE is preserved as an additional feature extractor to combine with RepVGG performing classification. A COVID-19 radiography database consisting of 3 classes COVID-19, Normal and Viral Pneumonia is conducted. The obtained average accuracy of the proposed model is 95.4% and other evaluation metrics also show better results compared with the original RepVGG model.
... Therefore, global-level histogram equalization was applied on the whole dataset for preprocessing. After histogram equalization, a lung segmentation module was implemented using a pre-trained model (41) to generate lung region masks, which were later used to generate lung region class activation maps (LCAMs). To exclude small erroneous regions, connected component analysis was applied after mask generation, and binary morphological closing was performed to fill any small holes of radius up to 10 pixels in the segmentation. ...
Background:
Accurate assessment of coronavirus disease 2019 (COVID-19) lung involvement through chest radiograph plays an important role in effective management of the infection. This study aims to develop a two-step feature merging method to integrate image features from deep learning and radiomics to differentiate COVID-19, non-COVID-19 pneumonia and normal chest radiographs (CXR).
Methods:
In this study, a deformable convolutional neural network (deformable CNN) was developed and used as a feature extractor to obtain 1,024-dimensional deep learning latent representation (DLR) features. Then 1,069-dimensional radiomics features were extracted from the region of interest (ROI) guided by deformable CNN's attention. The two feature sets were concatenated to generate a merged feature set for classification. For comparative experiments, the same process has been applied to the DLR-only feature set for verifying the effectiveness of feature concatenation.
Results:
Using the merged feature set resulted in an overall average accuracy of 91.0% for three-class classification, representing a statistically significant improvement of 0.6% compared to the DLR-only classification. The recall and precision of classification into the COVID-19 class were 0.926 and 0.976, respectively. The feature merging method was shown to significantly improve the classification performance as compared to using only deep learning features, regardless of choice of classifier (P value <0.0001). Three classes' F1-score were 0.892, 0.890, and 0.950 correspondingly (i.e., normal, non-COVID-19 pneumonia, COVID-19).
Conclusions:
A two-step COVID-19 classification framework integrating information from both DLR and radiomics features (guided by deep learning attention mechanism) has been developed. The proposed feature merging method has been shown to improve the performance of chest radiograph classification as compared to the case of using only deep learning features.
... These methods can be used in combination with our proposed method for feature disentanglement. In our experiments we used the recent SOTA lung VAE model [27] to create lung masks and implementation of histogram equalization from OpenCV [28]. ...
In response to the COVID-19 global pandemic, recent research has proposed creating deep learning based models that use chest radiographs (CXRs) in a variety of clinical tasks to help manage the crisis. However, the size of existing datasets of CXRs from COVID-19+ patients are relatively small, and researchers often pool CXR data from multiple sources, for example, using different x-ray machines in various patient populations under different clinical scenarios. Deep learning models trained on such datasets have been shown to overfit to erroneous features instead of learning pulmonary characteristics in a phenomenon known as shortcut learning. We propose adding feature disentanglement to the training process. This technique forces the models to identify pulmonary features from the images and penalizes them for learning features that can discriminate between the original datasets that the images come from. We find that models trained in this way indeed have better generalization performance on unseen data; in the best case we found that it improved AUC by 0.13 on held out data. We further find that this outperforms masking out non-lung parts of the CXRs and performing histogram equalization, both of which are recently proposed methods for removing biases in CXR datasets.
... As opposed to GANS, VAEs have witnessed a limited interest to perform DA and were mostly used for audio applications [9,14,23]. There have also been some other attempts on medical images for both classification [27,13] and segmentation tasks [18,19]. ...
We propose a new data augmentation technique for semi-supervised learning settings that emphasizes learning from the most challenging regions of the feature space. Starting with a fully supervised reference model, we first identify low confidence predictions. These samples are then used to train a Variational AutoEncoder (VAE) that can generate an infinite number of additional images with similar distribution. Finally, using the originally labeled data and the synthetically generated labeled and unlabeled data, we retrain a new model in a semi-supervised fashion. We perform experiments on two benchmark RGB datasets: CIFAR-100 and STL-10, and show that the proposed scheme improves classification performance in terms of accuracy and robustness, while yielding comparable or superior results with respect to existing fully supervised approaches
... Accordingly, segmenting the lung in these cases can be a challenging task. To overcome this, we followed the idea proposed in [15]. The proposed segmentation architecture involves a variational encoder for data imputation, and a Unet shape network with encoder and decoder for segmentation. ...
... To further process the dataset, we normalized the images using histogram equalization and used the model in [15] to segment the lungs. Then, we cropped the images centralizing the lungs by considering the maximum and minimum values for (x,y) pixel coordinates in the extracted mask. ...
During the COVID-19 pandemic, the sheer volume of imaging performed in an emergency setting for COVID-19 diagnosis has resulted in a wide variability of clinical CXR acquisitions. This variation is seen in the CXR projections used, image annotations added and in the inspiratory effort and degree of rotation of clinical images. The image analysis community has attempted to ease the burden on overstretched radiology departments during the pandemic by developing automated COVID-19 diagnostic algorithms, the input for which has been CXR imaging. Large publicly available CXR datasets have been leveraged to improve deep learning algorithms for COVID-19 diagnosis. Yet the variable quality of clinically-acquired CXRs within publicly available datasets could have a profound effect on algorithm performance. COVID-19 diagnosis may be inferred by an algorithm from non-anatomical features on an image such as image labels. These imaging shortcuts may be dataset-specific and limit the generalisability of AI systems. Understanding and correcting key potential biases in CXR images is therefore an essential first step prior to CXR image analysis. In this study, we propose a simple and effective step-wise approach to pre-processing a COVID-19 chest X-ray dataset to remove undesired biases. We perform ablation studies to show the impact of each individual step. The results suggest that using our proposed pipeline could increase accuracy of the baseline COVID-19 detection algorithm by up to 13%.
... In addition to tuberculosis, we collected 750 cases of consolidation patients from publicly available data [33], of which 600 cases were used for training and the remaining 150 cases for testing. We segmented lung regions from each CXR image using a publicly available pretrained lung segmentation network [34] based on a variational autoencoder, which achieves a higher generalization performance than other networks. We used the JSRT [35] and NLM [36] datasets to evaluate the lung segmentation performance of the pretrained network in terms of the intersection over union Table 3: AUC of GAN-IT models (CycleGAN and CUT) for patient-wise anomaly score H(v t , τ ) 2 in tuberculosis cases. ...
Image translation based on a generative adversarial network (GAN-IT) is a promising method for precise localization of abnormal regions in chest X-ray images (AL-CXR). However, heterogeneous unpaired datasets undermine existing methods to extract key features and distinguish normal from abnormal cases, resulting in inaccurate and unstable AL-CXR. To address this problem, we propose an improved two-stage GAN-IT involving registration and data augmentation. For the first stage, we introduce an invertible deep-learning-based registration technique that virtually and reasonably converts unpaired data into paired data for learning registration maps. This novel approach achieves high registration performance. For the second stage, we apply data augmentation to diversify anomaly locations by swapping the left and right lung regions on the uniform registered frames, further improving the performance by alleviating imbalance in data distribution showing left and right lung lesions. Our method is intended for application to existing GAN-IT models, allowing existing architecture to benefit from key features for translation. By showing that the AL-CXR performance is uniformly improved when applying the proposed method, we believe that GAN-IT for AL-CXR can be deployed in clinical environments, even if learning data are scarce.
... Carvalho Souza et al. [42] have used a first NN to derive an initial segmentation that excludes lung opacities, and a second NN to reconstruct the lung regions "lost" due to pulmonary abnormalities. Selvan et al. [43] have treated high opacity regions as missing data and designed an image segmentation network that utilizes a generative model based on a variational encoder for data imputation. Motamed et al. [20] and Munawar et al. [15] have used generative adversarial networks (GANs) for both lung segmentation and classification. ...
We have developed a two-module pipeline for the detection of SARS-CoV-2 from chest X-rays (CXRs). Module 1 is a traditional convnet that generates masks of the lungs overlapping the heart and large vasa. Module 2 is a hybrid convnet that preprocesses CXRs and corresponding lung masks by means of the Wavelet Scattering Transform, and passes the resulting feature maps through an Attention block and a cascade of Separable Atrous Multiscale Convolutional Residual blocks to produce a class assignment as Covid or non-Covid. Module 1 was trained on a public dataset of 6395 CXRs with radiologist annotated lung contours. Module 2 was trained on a dataset of 2362 non-Covid and 1435 Covid CXRs acquired at the Henry Ford Health System Hospital in Detroit. Six distinct cross-validation models, were combined into an ensemble model that was used to classify the CXR images of the test set. An intuitive graphic interphase allows for rapid Covid vs. non-Covid classification of CXRs, and generates high resolution heat maps that identify the affected lung regions.
