Article

Segmentation of multiple sclerosis lesions in brain MRI: A review of automated approaches

If you want to read the PDF, try requesting it from the authors.

Abstract

Automatic segmentation of multiple sclerosis (MS) lesions in brain MRI has been widely investigated in recent years with the goal of helping MS diagnosis and patient follow-up. However, the performance of most of the algorithms still falls far below expert expectations. In this paper, we review the main approaches to automated MS lesion segmentation. The main features of the segmentation algorithms are analysed and the most recent important techniques are classified into different strategies according to their main principle, pointing out their strengths and weaknesses and suggesting new research directions. A qualitative and quantitative comparison of the results of the approaches analysed is also presented. Finally, possible future approaches to MS lesion segmentation are discussed.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Such features include discrete lesions or tissue atrophy in addition to changes in intrinsic tissue intensity. As implemented here, ane registration of bispectral intensity histograms reduces the contribution of factors of no interest (e.g., dierences in scanner characteristics, head coil dierences) while allowing 19 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. ...
... Image segmentation, particularly WM lesion (WML) segmentation, is a well developed technique [4,12,30,32]. Algorithms for automatic tissue segmentation use a variety of approaches including probabilistic mapping and machine learning (for review, see [19,12]). Tissue segmentation tools (e.g., FAST, FreeSurfer [11,14]) as well as many WML segmentation tools (e.g., using machine learning algorithms [4,19]) use both tissue intensity and anatomical priors. Here, we demonstrate tissue segmentation with minimal dependence on anatomical priors. ...
... Algorithms for automatic tissue segmentation use a variety of approaches including probabilistic mapping and machine learning (for review, see [19,12]). Tissue segmentation tools (e.g., FAST, FreeSurfer [11,14]) as well as many WML segmentation tools (e.g., using machine learning algorithms [4,19]) use both tissue intensity and anatomical priors. Here, we demonstrate tissue segmentation with minimal dependence on anatomical priors. ...
Preprint
Full-text available
Assessment of intrinsic tissue integrity is commonly accomplished via quantitative relaxometry or other specialized imaging, which requires sequences and analysis procedures not routinely available in clinical settings. We detail an alternative technique for extraction of quantitative tissue biomarkers based on intensity normalization of T1- and T2-weighted images. We develop the theoretical underpinnings of this approach and demonstrate its utility in imaging of multiple sclerosis.
... Interest in lesions that appear hypointense on T1-weighted images (T1L) ("black holes") has grown because T1L provide more specificity for axonal loss and a closer link to neurologic disability (Andermatt et al., 2017;Katdare and Ursekar, 2015). The technical difficulty of T1L segmentation has led investigators to rely on time-consuming manual assessments prone to inter-and intra-rater variability (Garcia-Lorenzo et al., 2013;Llado et al., 2012 Multi-modal automatic segmentation approaches often yield a probability map to which a threshold is applied to create lesion segmentation masks. Unfortunately, few approaches systematically determine the threshold employed; many methods use a manually selected threshold (Sweeney et al., 2013, thus introducing human error and bias into the automated procedure. ...
... Manual segmentation is the gold standard approach for WML quantification and requires an expert to analyze scans visually. Unfortunately, this process is costly, time-consuming, and prone to intraand inter-rater variability (Garcia-Lorenzo et al., 2013;Llado et al., 2012;Sweeney et al., 2014). ...
... Manual segmentation is the gold standard approach and requires a neuroradiologist or imaging expert to inspect scans visually and delineate lesions. Due to difficulties associated with manual segmentation such as cost, time, and large intra-and inter-rater variability, many automatic segmentation methods have been developed (Carass et al., 2017b;Egger et al., 2017;Garcia-Lorenzo et al., 2013;Llado et al., 2012). Unfortunately, since lesions present heterogeneously on MRI scans, automatic segmentation remains a difficult task, though numerous methods have been proposed. ...
Article
Multi-modal neuroimaging, where several high-dimensional imaging variables are collected, has enabled the visualization and analysis of the brain structure and function in unprecedented detail. Due to methodological and computational challenges, the vast number of imaging studies evaluate data from each modality separately and do not consider information encoded in the relationships between imaging types. In this work, we propose methods that quantify the complex relationships between multiple imaging modalities and map how these relationships vary spatially across different anatomical regions of the brain. In order to understand relationships between several high-dimensional imaging variables, we use novel multi-modal image analysis techniques for feature development and image fusion in conjunction with machine learning techniques to develop automatic approaches for multiple sclerosis lesion detection. Additionally, we use multi-modal image analysis to understand the association between high-dimensional imaging variables with phenotypes of interest to investigate structure-function relationships in development, aging, and pathology of the brain. We find that by leveraging the relationship between imaging modalities, we can more accurately detect neuropathology and delineate brain trajectories to provide complementary characterizations of healthy development. We provide publicly available R packages to allow easy access and implemention of the proposed methods in new data and contexts.
... Therefore, the voxel intensity of tissues is a good feature to follow intensity changes and detect lesions. For MS patients, we can use four different MRI images: T1-weighed (T1-w), T2weighed (T2-w), Fluid-Attenuated Inversion Recovery (FLAIR), and Proton-Density weighted (PD-w) [3]. Fig. 1 shows these images. ...
... The figure displays T1-w, T2-w, PD-w, and FLAIR images, from left to right, respectively. Lesions are more detectable in the FLAIR image, where they clearly appear as brighter areas, and soft tissues are more easily recognized in the T1-w image [3]. ...
... However, unsupervised learning refers to algorithms in which the machine tries to discover patterns among data and cluster them without labeled inputs. A list of supervised and unsupervised approaches which have been proposed for MS lesion segmentation up to 2012 can be found in Ref. [3]. Some examples of supervised learning methods are random forests, ensemble methods, non-local means, and k-nearest neighbors. ...
Article
Full-text available
Multiple Sclerosis (MS) is a Central Nervous System (CNS) disease that Magnetic Resonance Imaging (MRI) system can detect and segment its lesions. Artificial Neural Networks (ANNs) recently reached a noticeable performance in finding MS lesions from MRI. U-Net and Attention U-Net are two of the most successful ANNs in the field of MS lesion segmentation. In this work, we proposed a framework to segment MS lesions in Fluid-Attenuated Inversion Recovery (FLAIR) and T2 MRI images by modified U-Net and modified Attention U-Net. For this purpose, we developed some extra preprocessing on MRI scans, made modifications in the loss function of U-Net and Attention U-Net, and proposed using the union of FLAIR and T2 predictions to reach a better performance. Results show that the union of FLAIR and T2 predicted masks by the modified Attention U-Net reaches the performance of 82.30% in terms of Dice Similarity Coefficient (DSC) in the test dataset, which is a considerable improvement compared to the previous works.
... However, the WM lesions have to be segmented for further analysis, which is always time-consuming and suffers from high intra-and inter-observer variabilities. [33] To overcome these limitations of manual segmentation, different DL architectures and networks have been used, yielding dice coefficients ranging from 0.48 to 0.95 for WM lesion segmentation. [33][34][35][36] Therefore, in a recent publication, it was shown that the use of regression networks for generating distance maps of the lesions might improve the WM lesion segmentation process [37]. ...
... [33] To overcome these limitations of manual segmentation, different DL architectures and networks have been used, yielding dice coefficients ranging from 0.48 to 0.95 for WM lesion segmentation. [33][34][35][36] Therefore, in a recent publication, it was shown that the use of regression networks for generating distance maps of the lesions might improve the WM lesion segmentation process [37]. This could provide more information about lesion geometry, structure, and changes similar to lesion probability mapping. ...
... After binarizing the probability maps, the dice coefficient was 0.61 ± 0.09 for the test data, which is comparable to the intra-observer variability of the manual drawer ( 0.68 ± 0.23 ) and is comparable to literature (0.47-0.95). [33,36] The CNN might be more robust compared with manual annotations because the network has no variations for multiple annotations. We have shown that the network only predicts lesion probability maps for the loss functions MAE and MSE. ...
Article
Full-text available
Background To develop a regression neural network for the reconstruction of lesion probability maps on Magnetic Resonance Fingerprinting using echo-planar imaging (MRF-EPI) in addition to $$T_1$$ T 1 , $${T_2}^*$$ T 2 ∗ , NAWM, and GM- probability maps. Methods We performed MRF-EPI measurements in 42 patients with multiple sclerosis and 6 healthy volunteers along two sites. A U-net was trained to reconstruct the denoised and distortion corrected $$T_1$$ T 1 and $${T_2}^*$$ T 2 ∗ maps, and to additionally generate NAWM-, GM-, and WM lesion probability maps. Results WM lesions were predicted with a dice coefficient of $$0.61\pm 0.09$$ 0.61 ± 0.09 and a lesion detection rate of $$0.85\pm 0.25$$ 0.85 ± 0.25 for a threshold of 33%. The network jointly enabled accurate $$T_1$$ T 1 and $${T_2}^*$$ T 2 ∗ times with relative deviations of 5.2% and 5.1% and average dice coefficients of $$0.92\pm 0.04$$ 0.92 ± 0.04 and $$0.91\pm 0.03$$ 0.91 ± 0.03 for NAWM and GM after binarizing with a threshold of 80%. Conclusion DL is a promising tool for the prediction of lesion probability maps in a fraction of time. These might be of clinical interest for the WM lesion analysis in MS patients.
... Notably, advances in machine learning technologies over the past 20 years [16] have made supervised training of statistical models highly attractive, particularly given the availability of robust non-linear image co-registration algorithms [17,18]. Supervised learning is also typically easier to conduct and may perform better than unsupervised learning [19], but the ground truth is not always available [20]. ...
... Our overall outcomes also indicate that proper lesion detection in MRI is critical for improving model accuracy. While we applied manual lesion masks, there is extensive literature on the topic of lesion identification using unsupervised approaches [20], which however is out of the scope of the present study. ...
... However, cohort-level analysis following strict longitudinal co-registration served as an alternative approach, particularly for this established animal model, which is highly predictable in both disease course and lesion location [21,47]. In the future, we seek to extend our GMRF model to a full Gaussian random field or beyond [36,48], replace it with a multi-scale approach [49], or modify it with outlier detection [20,50] to improve performance. Notably, our approach was purposefully designed as stepwise and modular, to enable further validation and extension. ...
Article
Full-text available
Myelin plays a critical role in the pathogenesis of neurological disorders but is difficult to characterize in vivo using standard analysis methods. Our goal was to develop a novel analytical framework for estimating myelin content using T2-weighted magnetic resonance imaging (MRI) based on a de- and re-myelination model of multiple sclerosis. We examined 18 mice with lysolecithin induced demyelination and spontaneous remyelination in the ventral white matter of thoracic spinal cord. Cohorts of 6 mice underwent 9.4T MRI at days 7 (peak demyelination), 14 (ongoing recovery), and 28 (near complete recovery), as well as histological analysis of myelin and the associated cellularity at corresponding timepoints. Our MRI framework took an unsupervised learning approach, including tissue segmentation using a Gaussian Markov random field (GMRF), and myelin and cellularity feature estimation based on the Mahalanobis distance. For comparison, we also investigated 2 regression-based supervised learning approaches, one using our GMRF results, and another using a freely available generalized additive model (GAM). Results showed that GMRF segmentation was 73.2% accurate, and our unsupervised learning method achieved a correlation coefficient of 0.67 (top quartile: 0.78) with histological myelin, similar to 0.70 (top quartile: 0.78) obtained using supervised analyses. Further, the area under the receiver operator characteristic curve of our unsupervised myelin feature (0.883, 95% CI: 0.874–0.891) was significantly better than any of the supervised models in detecting white matter myelin as compared to histology. Collectively, metric learning using standard MRI may prove to be a new alternative method for estimating myelin content, which ultimately can improve our disease monitoring ability in a clinical setting.
... As the manual detection of WML is time-consuming and prone to inter-rater variability 7 , a myriad of automated or semi-automated approaches have been developed to facilitate this task 8 , representing some of the earliest uses of machine learning techniques applied to MRI. These methods were initially based primarily on MRI intensity features and probabilistic atlases 8 , whereas more recently, the vast majority use deep learning (DL) approaches 9 , the latter without prior feature extraction. ...
... As the manual detection of WML is time-consuming and prone to inter-rater variability 7 , a myriad of automated or semi-automated approaches have been developed to facilitate this task 8 , representing some of the earliest uses of machine learning techniques applied to MRI. These methods were initially based primarily on MRI intensity features and probabilistic atlases 8 , whereas more recently, the vast majority use deep learning (DL) approaches 9 , the latter without prior feature extraction. Substantial effort is now being made towards reproducibility of the results and open science 10 . ...
Preprint
Full-text available
The current multiple sclerosis (MS) diagnostic criteria lack specificity, and this may lead to misdiagnosis, which remains an issue in present-day clinical practice. In addition, conventional biomarkers only moderately correlate with MS disease progression. Recently, advanced MS lesional imaging biomarkers such as cortical lesions (CL), the central vein sign (CVS), and paramagnetic rim lesions (PRL), visible in specialized magnetic resonance imaging (MRI) sequences, have shown higher specificity in differential diagnosis. Moreover, studies have shown that CL and PRL are potential prognostic biomarkers, the former correlating with cognitive impairments and the latter with early disability progression. As machine learning-based methods have achieved extraordinary performance in the assessment of conventional imaging biomarkers, such as white matter lesion segmentation, several automated or semi-automated methods have been proposed for CL, CVS, and PRL as well. In the present review, we first introduce these advanced MS imaging biomarkers and their imaging methods. Subsequently, we describe the corresponding machine learning-based methods that were used to tackle these clinical questions, putting them into context with respect to the challenges they are still facing, including non-standardized MRI protocols, limited datasets, and moderate inter-rater variability. We conclude by presenting the current limitations that prevent their broader deployment and suggesting future research directions.
... Lesion characteristics, such as number and volume, are principal imaging metrics for both MS clinical trials and monitoring of the disease in clinical practice [2]. To this end, automatic and accurate MS lesion segmentation in Magnetic Resonance (MR) imaging can critically enhance both MS research and patient management [3]- [6]. ...
... In addition, we conducted experiments in which lesion volume was employed for local-level re-weighting on the task learning, referred to as the 'Ours-vol' method in Table V. Specifically, the volume ratio vr i in Equation 6 is replaced by the total number of lesion voxels v i . Due to the inaccurate estimation of the true MS lesion distributions in brain MRI We think the reasons for this phenomenon are two folds: 1) each client of the Scenario 2 has more data than those in Scenario 1; 2) the multi-client MS dataset in Scenario 2 is constructed by various datasets from in-house scanners and the public resources, which brings more distinctions for the cross-client data distributions. ...
Preprint
Full-text available
Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's performance is limited for multiple sclerosis (MS) lesion segmentation tasks, due to variance in lesion characteristics imparted by different scanners and acquisition parameters. In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training. Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by outperforming other FL methods significantly. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can exceed centralised training with all the raw data. The extensive evaluation also indicated the superiority of our method when estimating brain volume differences estimation after lesion inpainting.
... Radiologists or other imaging scientists visually assess scans and manually delineate lesions on each slice in order to report total number and volume of WML. Not only is this costly and time-consuming, but it is prone to large inter-and intra-observer variability due to the challenge of incorporating 3D information from several MRI modalities [5] [6]. However, these WML metrics are vital in clinical trials where lesion number and volume are important outcomes for assessing disease-related changes and treatment effects [7]. ...
... Automated methods additionally introduce stability and consistency into lesion segmentation as they eliminate human bias and error. Though many automated approaches and methods exist, no currently available algorithm is able to outperform manual lesion segmentation in terms sensitivity and specificity across subjects and scanning centers [5] [8][9]. As a result, no particular automated segmentation algorithm is accepted as the gold standard in practice. ...
Preprint
Magnetic resonance imaging (MRI) is crucial for in vivo detection and characterization of white matter lesions (WML) in multiple sclerosis. While these lesions have been studied for over two decades using MRI technology, automated segmentation remains challenging. Although the majority of statistical techniques for the automated segmentation of WML are based on a single imaging modality, recent advances have used multimodal techniques for identifying WML. Complementary imaging modalities emphasize different tissue properties, which can help identify and characterize interrelated features of lesions. However, prior work has ignored relationships between imaging modalities, which may be informative in this clinical context. To harness the coherent changes in these measurements, we utilized inter-modal coupling regression (IMCo) to estimate the covariance structure across modalities. We then used a local logistic regression, MIMoSA, which leverages new covariance features from IMCo regression as well as the mean structure of each imaging modality in order to model the probability that any voxel is part of a lesion. Finally, we introduced a novel thresholding algorithm to fully automate the estimation of the probability maps to generate fully automated segmentations masks for 94 subjects. To evaluate the performance of the automated segmentations generated using MIMoSA we compared results with gold standard manual segmentations. We demonstrate the superiority of MIMoSA to other automated segmentation techniques by comparing it to the OASIS algorithm as well as LesionTOADS. MIMoSA resulted in statistically significant improvement in lesion segmentation.
... This makes the automatic segmentation of MS lesions a challenging problem, so in [4] they focused on differentiating between active MS and cold-spot lesion from brain MRI. MRI is a cornerstone in current diagnosis standard by enabling to show the distribution of WM lesions in space and time at high specificity and sensitivity [5]. The challenge was in identification of MS in MR Images since the lesions have different size, shape and also different locations with anatomical variability [6]. ...
... When comparing these results with previous work, we found that the proposed method has the minimum FPR and FNR than other work. Various researches have been accomplished to obtain the relation between the different gray levels and texture features [5]. In [22] texture analysis based gray level run length matrix (RLM) was performed on 110 Patients with classification accuracy of Multi Sclerosis 96.9%. ...
... [ WM lesion segmentation. [33][34][35][36] Therefore, in a recent publication, it was shown that this processing step can be improved by regression by also generating distance maps of the lesions. ...
... 68 ± 0.23) and is comparable to literature (0.47-0.95).[33,36] The CNN might be more robust compared with manual annotations because the network has no variations for multiple annotations. ...
Preprint
Full-text available
Purpose To develop a regression neural network for the reconstruction of lesion probability maps on Magnetic Resonance Fingerprinting using echo-planar imaging (MRF-EPI) in addition to T 1 , T 2 * , NAWM, and GM- probability maps. Methods We performed MRF-EPI measurements in 42 patients with multiple sclerosis and 6 healthy volunteers along two sites. A U-net was trained to reconstruct the denoised and distortion corrected T 1 and T 2 * maps, and to additionally generate NAWM-, GM-, and WM lesion probability maps. Results WM lesions were predicted with a dice coefficient of 0.61±0.09 and a lesion detection rate of 0.85±0.25 for a threshold of 33%. The network jointly enabled accurate T 1 and T 2 * times with relative deviations of 5.2% and 5.1% and average dice coefficients of 0.92±0.04 and 0.91±0.03 for NAWM and GM after binarizing with a threshold of 80%. Conclusion DL is a promising tool for the prediction of lesion probability maps in a fraction of time. These might be of clinical interest for the WM lesion analysis in MS patients.
... Detecting abnormalities is indeed a very challenging task in itself and the first step to analyse lesions. Therefore, much justifiable research has been conducted to detect brain lesions on MR images [13,110,159]. However, far less attention has been given to subsequent analysis strategies to derive clinically valuable information from segmented brain lesions. ...
Thesis
Neuroimaging studies are becoming increasingly bigger, and multi-centre collaborations to collect data under similar protocols, but different scanning sites, are now commonplace.However, with increasing sample size the complexity of databases and the entailed data management as well as computational burden are growing. This thesis aims to highlight and address challenges faced by large multi-centre magnetic resonance imaging(MRI) studies. The methods implemented are then applied to traumatic brain injury (TBI) data.Firstly, a pre-processing pipeline for both anatomical and diffusion MRI was proposed, that allows for a high throughput of MRI scans. After describing the choices for processing tools,the performance of the integrated quality assurance was assessed based on the results from a large multi-centre dataset for TBI. Secondly, the applicability of the pipelines for processing mild TBI (mTBI) data from three sites was shown in a case study. For this, volumetric and diffusion metrics in the acute phase are analysed for their prognostic potential. Further-more, the cohort was examined for longitudinal changes. Thirdly, independent scan-rescan datasets are examined to gain a better understanding of the degree of reproducibility which can be achieved in imaging studies. This involves analysing the robustness of brain parcellations based on structural or diffusion imaging. The effect of using different MRI scanners or imaging protocols was also assessed and discussed. Fourthly, sources of diffusion MRI variability and different approaches to cope with these are reviewed. Using this foundation,state-of-the art methods for diffusion MRI harmonisation were compared against each other using both a benchmark dataset and mTBI cohort. Lastly, a solution to localise brain lesions was proposed. Its implications for lesion analysis, are assessed in the light of an application to a more severe TBI patient cohort, imaged on two different scanners. Furthermore, a lesion matching algorithm was introduced to automatically examine lesion evolution with time post-injury. In summary, this thesis explored different options for MRI data analysis in the context of large multi-centre studies. Different approaches are studied and compared using a number of different MRI datasets, including scan-rescan data across different MRI scanners and imaging protocols. The potential of the optimised solutions was illustrated through applications to TBI data.
... To help in the selection of the WMH segmentation methods and discuss their applicability, other systematic literature reviews have been published, but on focused topics specific to diseases, e.g. MS lesion segmentation (García-Lorenzo et al., 2013;Llado et al., 2011Llado et al., , 2012Miller et al., 1998;Mortazavi, Kouzani & Soltanian-Zadeh, 2012 (Caligiuri et al., 2015). Two other non-overlapping reviews Blair et al., 2017) discussed different approaches, published up to 2016, not only for segmenting WMH, but also for assessing other neuroimaging markers of SVD. ...
Preprint
Background: White matter hyperintensities (WMH), of presumed vascular origin, are visible and quantifiable neuroradiological markers of brain parenchymal change. These changes may range from damage secondary to inflammation and other neurological conditions, through to healthy ageing. Fully automatic WMH quantification methods are promising, but still, traditional semi-automatic methods seem to be preferred in clinical research. We systematically reviewed the literature for fully automatic methods developed in the last five years, to assess what are considered state-of-the-art techniques, as well as trends in the analysis of WMH of presumed vascular origin. Method: We registered the systematic review protocol with the International Prospective Register of Systematic Reviews (PROSPERO), registration number - CRD42019132200. We conducted the search for fully automatic methods developed from 2015 to July 2020 on Medline, Science direct, IEE Explore, and Web of Science. We assessed risk of bias and applicability of the studies using QUADAS 2. Results: The search yielded 2327 papers after removing 104 duplicates. After screening titles, abstracts and full text, 37 were selected for detailed analysis. Of these, 16 proposed a supervised segmentation method, 10 proposed an unsupervised segmentation method, and 11 proposed a deep learning segmentation method. Average DSC values ranged from 0.538 to 0.93, being the highest value obtained from a deep learning segmentation method. Only four studies validated their method in longitudinal samples, and eight performed an additional validation using clinical parameters. Only 8/37 studies made available their method in public repositories. Conclusions: Although deep learning methods reported highly accurate results, we found no evidence that favours them over the more established k-NN, linear regression and unsupervised methods in this task. Data and code availability, bias in study design and ground truth generation influence the wider validation and applicability of these methods in clinical research.
... In brain MRI segmentation, there has been a significant effort to partition the whole of the image into sub-portions like WM, GM, and CSF of the brain [1,2]. In contrast, some works concentrate on the extraction of a specific structure like brain tumors [3], multiple sclerosis lesions [4], or ischemic stroke lesion [5]. However, the segmentation of brain MRI is a challenging task [6,7] due to the presence of artifacts, such as intensity non-uniformity, noise, partial volume effect, motion artifacts, and bias field, non-linear in nature which gets in during the image acquisition process. ...
Article
Background and objectives: Accurate segmentation of critical tissues from a brain MRI is pivotal for characterization and quantitative pattern analysis of the human brain and thereby, identifies the earliest signs of various neurodegenerative diseases. To date, in most cases, it is done manually by the radiologists. The overwhelming workload in some of the thickly populated nations may cause exhaustion leading to interruption for the doctors, which may pose a continuing threat to patient safety. A novel fusion method called U-Net inception based on 3D convolutions and transition layers is proposed to address this issue. Methods: A 3D deep learning method called Multi headed U-Net with Residual Inception (MhURI) accompanied by Morphological Gradient channel for brain tissue segmentation is proposed, which incorporates Residual Inception 2-Residual (RI2R) module as the basic building block. The model exploits the benefits of morphological pre-processing for structural enhancement of MR images. A multi-path data encoding pipeline is introduced on top of the U-Net backbone, which encapsulates initial global features and captures the information from each MRI modality. Results: The proposed model has accomplished encouraging outcomes, which appreciates the adequacy in terms of some of the established quality metrices when compared with some of the state-of-the-art methods while evaluating with respect to two popular publicly available data sets. Conclusion: The model is entirely automatic and able to segment gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) from brain MRI effectively with sufficient accuracy. Hence, it may be considered to be a potential computer-aided diagnostic (CAD) tool for radiologists and other medical practitioners in their clinical diagnosis workflow.
... MS is known as one of the most important diseases of the central nervous system of the brain. The detection, segmentation, and quantification of the MS lesions is an important task as it can help to characterize the progression of the disease and monitor the efficacy of a candidate treatment [62]. ...
Thesis
Multiple Sclerosis (MS) is a demyelinating disease of the central nervous system which causes lesions in brain tissues, especially visible in white matter with magnetic resonance imaging (MRI). The diagnosis of MS lesions, which is often performed visually with MRI, is an important task as it can help characterizing the progression of the disease and monitoring the efficacy of a candidate treatment. automatic detection and segmentation of MS lesions from MRI images offer the potential for a faster and more cost-effective performance which could also be immune to expert bias segmentation. In this thesis, we study automated approaches to segment MS lesions from MRI images. The thesis begins with a review of the existing literature on MS lesion segmentation and discusses their general limitations. We then propose three novel approaches that rely on Convolutional Neural Networks (CNNs) to segment MS lesions. The first approach demonstrates that the parameters of a CNN learned from natural images, transfer well to the tasks of MS lesion segmentation. In the second approach, we describe a novel multi-branch CNN architecture with end-to-end training that can take advantage of each MRI modalities individually. In that work, we also investigated the combination of MRI modalities leading to the best segmentation performance. In the third approach, we show an effective and novel generalization method for MS lesion segmentation when data are collected from multiple MRI scanning sites and as suffer from (site-)domain shifts. Finally, this thesis concludes with open questions that may benefit from future work. This thesis demonstrates the potential role of CNNs as a common methodological building block to address clinical problems in MS segmentation.
... There are many ingenious approaches proposed in the literature which could be highlighted the automatic segmentation techniques based on image patterns analysis [7,20,1,16,18], statistical models [4,6,9] and machine learning approaches [3,19,11]. Much more details are presented in recent scientific reviews, being a few examples here suggested [13,5,12]. ...
Article
Full-text available
Recently, the scientific community has been proposing several automatic algorithms to biomedical image segmentation procedure, being an interesting and helpful approach to assist both technicians and radiologists in this time-consuming and subjective task. One of these interesting and widely used image segmentation method could be the voxel intensity-based algorithms, e.g. image histogram threshold methods, which have been intensively improved in the past decades. Recently, an interesting approach that gained focus is the logistic classification (LC) for object detection in biomedical images. Even though the general concept behind the LC method is fairly known, the proper method's optimization still commonly adjusted by hand which naturally adds a level of uncertainty and subjectivity in the general segmentation performance. Therefore, an empirical LC optimization is presented, offering a ITK class that performs the LC parameters optimization based on empirical input data analysis. It is worth mentioning that the LogisticContrastEnhancementImageFilter class showed here is also applied on others computational problems, being briefly explained in this document.
... Many algorithms have been developed for MS lesion segmentation [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21]. The mentioned references are examples of different techniques of digital image processing (DIP) applied to detect MS lesions captured by MRI. ...
Article
Full-text available
Medical imaging has improved the diagnostic capability of many diseases, as well as surgical planning and monitoring of chronic degenerative diseases. This allows specialists to improve accuracy by applying image segmentation. Recent research proposes new techniques and algorithms to facilitate this work, reducing execution time and inter-intra operator errors during the analysis procedure. In fact, most of these new techniques are still on validation procedures, due to the performance and accuracy variations presented in results when different variables modify the image quality. Nowadays many researchers propose segmentation solutions by applying Artificial Neural Networks (ANN), wavelets, classifiers, thresholding and hybrid integrations, which improve the accuracy of results. The bio-inspired algorithms are scarcely used for segmenting medical images. These tools are frequently useful to solve optimization problems rather than image segmentation. Clustering algorithms such as K-means can be visualized as optimization problems. The scope of bio-inspired algorithms in this case, is to evaluate their clustering solutions behavior, by comparing their results with K-means. The bio-inspired algorithms selected to evaluate are Genetic Algorithm (GA) and Artificial Bee Colony (ABC). Both techniques have similarities on their evolutive behavior, and their results can be compared between them and with K-means clustering. This comparison allows to determine if heuristic centroids have better results than adaptive ones, adjusted by a fitness function (Adaptive Clustering).
... Magnetic resonance imaging (MRI) plays a vital role in diagnosing and assessing disease progression in people with MS. For instance, contrast-enhanced lesions and chronic T1weighted hypointensities reflect inflammation, and axonal loss, respectively, but are at best only semi-quantitative (Ge, 2006;Ceccarelli et al., 2012;Llado et al., 2012). Axonal loss is a critical mechanism of irreversible neurological disability (Kornek et al., 2000;Wujek et al., 2002;Medana and Esiri, 2003). ...
Article
Full-text available
Optic neuritis is a frequent first symptom of multiple sclerosis (MS) for which corticosteroids are a widely employed treatment option. The Optic Neuritis Treatment Trial (ONTT) reported that corticosteroid treatment does not improve long-term visual acuity, although the evolution of underlying pathologies is unclear. In this study, we employed non-invasive diffusion basis spectrum imaging (DBSI)-derived fiber volume to quantify 11% axonal loss 2 months after corticosteroid treatment (vs. baseline) in experimental autoimmune encephalomyelitis mouse optic nerves affected by optic neuritis. Longitudinal DBSI was performed at baseline (before immunization), after a 2-week corticosteroid treatment period, and 1 and 2 months after treatment, followed by histological validation of neuropathology. Pathological metrics employed to assess the optic nerve revealed axonal protection and anti-inflammatory effects of dexamethasone treatment that were transient. Two months after treatment, axonal injury and loss were indistinguishable between PBS- and dexamethasone-treated optic nerves, similar to results of the human ONTT. Our findings in mice further support that corticosteroid treatment alone is not sufficient to prevent eventual axonal loss in ON, and strongly support the potential of DBSI as an in vivo imaging outcome measure to assess optic nerve pathology.
... To help select WMH segmentation methods and discuss their applicability, other systematic literature reviews have been published, but on focused topics specific to diseases, e.g. MS lesion segmentation (García-Lorenzo et al., 2013;Llado et al., 2011Llado et al., , 2012Miller et al., 1998;Mortazavi, Kouzani & Soltanian-Zadeh, 2012). Methods which work for MS may only perform moderately if applied to individuals with SVD or to the normal elderly (Table 1). ...
Article
Full-text available
Background White matter hyperintensities (WMH), of presumed vascular origin, are visible and quantifiable neuroradiological markers of brain parenchymal change. These changes may range from damage secondary to inflammation and other neurological conditions, through to healthy ageing. Fully automatic WMH quantification methods are promising, but still, traditional semi-automatic methods seem to be preferred in clinical research. We systematically reviewed the literature for fully automatic methods developed in the last five years, to assess what are considered state-of-the-art techniques, as well as trends in the analysis of WMH of presumed vascular origin. Method We registered the systematic review protocol with the International Prospective Register of Systematic Reviews (PROSPERO), registration number - CRD42019132200. We conducted the search for fully automatic methods developed from 2015 to July 2020 on Medline, Science direct, IEE Explore, and Web of Science. We assessed risk of bias and applicability of the studies using QUADAS 2. Results The search yielded 2327 papers after removing 104 duplicates. After screening titles, abstracts and full text, 37 were selected for detailed analysis. Of these, 16 proposed a supervised segmentation method, 10 proposed an unsupervised segmentation method, and 11 proposed a deep learning segmentation method. Average DSC values ranged from 0.538 to 0.91, being the highest value obtained from an unsupervised segmentation method. Only four studies validated their method in longitudinal samples, and eight performed an additional validation using clinical parameters. Only 8/37 studies made available their methods in public repositories. Conclusions We found no evidence that favours deep learning methods over the more established k-NN, linear regression and unsupervised methods in this task. Data and code availability, bias in study design and ground truth generation influence the wider validation and applicability of these methods in clinical research.
... MS is a chronic disease that changes the morphology and structure of the brain due to the harm to the myelin sheath . More importantly, MS can cause disability in young adults (Lladó et al., 2012). MS is relatively common in Europe, New Zealand, the United States, and parts of Australia. ...
Article
Full-text available
In recent years, there have been multiple works of literature reviewing methods for automatically segmenting multiple sclerosis (MS) lesions. However, there is no literature systematically and individually review deep learning-based MS lesion segmentation methods. Although the previous review also included methods based on deep learning, there are some methods based on deep learning that they did not review. In addition, their review of deep learning methods did not go deep into the specific categories of Convolutional Neural Network (CNN). They only reviewed these methods in a generalized form, such as supervision strategy, input data handling strategy, etc. This paper presents a systematic review of the literature in automated multiple sclerosis lesion segmentation based on deep learning. Algorithms based on deep learning reviewed are classified into two categories through their CNN style, and their strengths and weaknesses will also be given through our investigation and analysis. We give a quantitative comparison of the methods reviewed through two metrics: Dice Similarity Coefficient (DSC) and Positive Predictive Value (PPV). Finally, the future direction of the application of deep learning in MS lesion segmentation will be discussed.
... In addition, spinal cord diseases exhibit more variations in their morphology and signals in sagittal MRI. [12][13][14][15] Only a few studies have investigated spinal cord diseases on MRI using CNN models. Gros et al. conducted a study that utilized a sequence of two CNNs to segment the spinal cord and/or intramedullary multiple sclerosis lesions based on a multi-site clinical dataset, and their segmentation methods showed a better result compared to previous CNN models. ...
Article
Full-text available
Magnetic resonance imaging (MRI) can indirectly reflect microscopic changes in lesions on the spinal cord; however, the application of deep learning to MRI to classify and detect lesions for cervical spinal cord diseases has not been sufficiently explored. In this study, we implemented a deep neural network for MRI to detect lesions caused by cervical diseases. We retrospectively reviewed the MRI of 1,500 patients irrespective of whether they had cervical diseases. The patients were treated in our hospital from January 2013 to December 2018. We randomly divided the MRI data into three groups of datasets: disc group (800 datasets), injured group (200 datasets), and normal group (500 datasets). We designed the relevant parameters and used a faster‐region convolutional neural network (Faster R‐CNN) combined with a backbone convolutional feature extractor using the ResNet‐50 and VGG‐16 networks, to detect lesions during MRI. Experimental results showed that the prediction accuracy and speed of Faster R‐CNN with ResNet‐50 and VGG‐16 in detecting and recognizing lesions from a cervical spinal cord MRI were satisfactory. The mean average precisions (mAPs) for Faster R‐CNN with ResNet‐50 and VGG‐16 were 88.6 and 72.3%, respectively, and the testing times was 2.2 and 2.4 s/image, respectively. Faster R‐CNN can identify and detect lesions from cervical MRIs. To some extent, it may aid radiologists and spine surgeons in their diagnoses. The results of our study can provide motivation for future research to combine medical imaging and deep learning.
... Medical image analysis is greatly performed with automated methods, mostly involving deep learning [42]. Automated MS lesion identification/segmentation is still an active field of research and several methods have been provided in the last decade and well reviewed along time [19,21,23,37,43,46,75] and the role of AI-based methods is emerging [2]. Automated strategies can be classified in three main groups: methods using pre-selected features (PSFM), methods using a-priori information (APIM) and methods using deep learning (DLM). ...
Preprint
Full-text available
To date, several automated strategies for identification/segmentation of Multiple Sclerosis (MS) lesions by Magnetic Resonance Imaging (MRI) have been presented which are either outperformed by human experts or, at least, whose results are well distinguishable from humans. This is due to the ambiguity originated by MRI instabilities, peculiar MS Heterogeneity and MRI unspecific nature with respect to MS. Physicians partially treat the uncertainty generated by ambiguity relying on personal radiological/clinical/anatomical background and experience. We present an automated framework for MS lesions identification/segmentation based on three pivotal concepts to better emulate human reasoning: the modeling of uncertainty; the proposal of two, separately trained, CNN, one optimized with respect to lesions themselves and the other to the environment surrounding lesions, respectively repeated for axial, coronal and sagittal directions; the ensemble of the CNN output. The proposed framework is trained, validated and tested on the 2016 MSSEG benchmark public data set from a single imaging modality, FLuid-Attenuated Inversion Recovery (FLAIR). The comparison, performed on the segmented lesions by means of most of the metrics normally used with respect to the ground-truth and the 7 human raters in MSSEG, prove that there is no significant difference between the proposed framework and the other raters. Results are also shown for the uncertainty, though a comparison with the other raters is impossible.
... Various automated MS lesion detection approaches have been proposed in the literature. 10,11 Researchers performed an automated algorithm based on intensity normalization and subtraction of volumes in MR images to detect MS lesions. 12 In one study, MS lesions have been detected from multimodal MR images (T1w, FLAIR, T2w) ...
Article
Full-text available
Background Multiple sclerosis (MS) is an immune-mediated inflammatory disease that attacks myelinated axons in the central nervous system, destroying myelin and axons to variable degrees and, resulting in significant physical disability. Magnetic resonance imaging (MRI) is useful in the diagnosis of MS, surpassing all other imaging techniques in terms of prediction accuracy. Depending on the number and location of lesions, however, the success of MR can vary significantly in terms of sensitivity and specificity in the diagnosis of MS. Adverse effects of various intensity and residual artifacts in the MRI data make it challenging to compute MS lesion volume to assess the progression of MS. Therefore, the development of robust and automated MS lesion detection methods has been a challenge. Objectives This study aims to develop a novel, robust, and simple image segmentation method to perform quantitative analysis of MS lesions from multimodal MRI data. Methods An algorithm based on a supervised minimum Euclidean distance-based clustering method employing three 2D MRI modalities, T1-weighted (T1w), fluid-attenuated inversion recovery (FLAIR), and T2-weighted (T2w) MRI was developed for classification of significant brain tissues and MS lesions. The developed method was applied to an MRI dataset from six MS patients. Results The developed method classifies various brain tissues and detects MS lesions with over 90% accuracy and specificity, and 62%-65% sensitivity, on average. Conclusions Segmentation of different brain tissues using our proposed algorithm results in superior MS lesion-detection accuracy, comparable with the recent deep-learning classification results in the literature.
... For example, lesion segmentation is a classification problem and solved by training statistical classifiers (e.g. logistic regression, support vector machine, or random forest) given the voxel-level intensity information from sMRI (Pham et al., 2000;Balafar et al., 2010;Lladó et al., 2012;Sweeney et al., 2013b). Another example is the study of the association between lesion localization and health outcomes (e.g. ...
Thesis
As both clinical and cognitive neuroscience matures, the need for sophisticated neuroimaging analyses becomes more important. The use of imaging markers to predict clinical outcomes, or even imaging outcomes, can have great impact on public health. However, such analyses are still under development since it is challenging for several reasons: 1) the images are of high dimension, and 2) the images may exhibit complex spatial correlation structure. Bayesian methods play an important role in solving these problems by dealing with spatial data flexibly and applying efficient sampling algorithms. This dissertation aims to develop spatial Bayesian models to predict either scalar or imaging outcomes by using imaging predictors and seeks computationally efficient approaches. In Chapter I, we propose a Bayesian scalar-on-image regression model with application to Multiple Sclerosis (MS) Magnetic Resonance Imaging (MRI) data. Specifically, we build up a multinomial logistic regression model to predict the clinical subtypes of MS patients by using their 3D MRI lesion data. Parameters corresponding to MRI predictors are spatially varying in the image space and are assumed to have a Gaussian Process (GP) prior distribution. Since the covariates are highly correlated, we use the Hamiltonian Monte Carlo algorithm, which is more statistically efficient than other Markov Chain Monte Carlo methods when the parameters are highly correlated. Finally, to reduce computational burden, we code the problem to run in parallel on a graphical processing unit. Results from simulation studies and a real MS data set show that our method has high prediction accuracy as evaluated by leave-one-out cross validation using an importance-sampling scheme. In Chapter II, we propose a novel image-on-image regression model, by extending a spatial Bayesian latent factor model to neuroimaging data, where low dimensional latent factors are adopted to make connections between high-dimensional image outcomes and image predictors. We assign GP priors to the spatially varying regression coefficients in the model, which capture the complex spatial dependence among image outcomes as well as that among the image predictors. We perform simulation studies to evaluate the out-of-sample prediction performance of our method compared with linear regression and voxel-wise regression methods for different scenarios. We apply the proposed method to analysis of multimodal image data in the Human Connectome Project (HCP) where we predict task-related contrast maps using sub-cortical volumetric seed maps. The proposed method achieves a better prediction accuracy than simpler models by effectively accounting for the spatial dependence and efficient reduction of image dimension with latent factors. In Chapter III, we extend the image-on-image regression model proposed in Chapter II to the case where outcome is a cortical surface image and predictors images are volumetric seed maps. We expand the surface image on a set of spherical harmonics basis functions, where coefficients are linked to image predictors through a latent factor model. We assign GP priors to the spatially varying regression coefficients of the volumetric predictor images. Compared to ridge regression, the proposed method performs better in prediction according to simulation studies, and it can identify active brain regions in spherical z-score images from the HCP.
... To help select WMH segmentation methods and discuss their applicability, other systematic literature reviews have been published, but on focused topics specific to diseases, e.g. MS lesion segmentation (García-Lorenzo et al., 2013;Llado et al., 2011Llado et al., , 2012Miller et al., 1998;Mortazavi, Kouzani & Soltanian-Zadeh, 2012). Methods which work for MS may only perform moderately if applied to individuals with SVD or to the normal elderly (Table 1). ...
Preprint
Background: White matter hyperintensities (WMH), of presumed vascular origin, are visible and quantifiable neuroradiological markers of brain parenchymal change. These changes may range from damage secondary to inflammation and other neurological conditions, through to healthy ageing. Fully automatic WMH quantification methods are promising, but still, traditional semi-automatic methods seem to be preferred in clinical research. We systematically reviewed the literature for fully automatic methods developed in the last five years, to assess what are considered state-of-the-art techniques, as well as trends in the analysis of WMH of presumed vascular origin. Method: We registered the systematic review protocol with the International Prospective Register of Systematic Reviews (PROSPERO), registration number - CRD42019132200. We conducted the search for fully automatic methods developed from 2015 to July 2020 on Medline, Science direct, IEE Explore, and Web of Science. We assessed risk of bias and applicability of the studies using QUADAS 2. Results: The search yielded 2327 papers after removing 104 duplicates. After screening titles, abstracts and full text, 37 were selected for detailed analysis. Of these, 16 proposed a supervised segmentation method, 10 proposed an unsupervised segmentation method, and 11 proposed a deep learning segmentation method. Average DSC values ranged from 0.538 to 0.91, being the highest value obtained from an unsupervised segmentation method. Only four studies validated their method in longitudinal samples, and eight performed an additional validation using clinical parameters. Only 8/37 studies made available their method in public repositories. Conclusions: We found no evidence that favours deep learning methods over the more established k-NN, linear regression and unsupervised methods in this task. Data and code availability, bias in study design and ground truth generation influence the wider validation and applicability of these methods in clinical research.
... Automatic segmentation algorithms have therefore become a crucial need for the clinical community to simplify the clinician's task. A large literature of automatic segmentation methods has been devised ( García-Lorenzo et al., 2013;Lladó et al., 2012;Mortazavi et al., 2012 ) for which a common ground for their evaluation is more and more required. This evaluation is performed using databases where manual delineation was performed by one or several expert radiologists. ...
Article
Full-text available
MRI plays a crucial role in multiple sclerosis diagnostic and patient follow-up. In particular, the delineation of T2-FLAIR hyperintense lesions is crucial although mostly performed manually - a tedious task. Many methods have thus been proposed to automate this task. However, sufficiently large datasets with a thorough expert manual segmentation are still lacking to evaluate these methods. We present a unique dataset for MS lesions segmentation evaluation. It consists of 53 patients acquired on 4 different scanners with a harmonized protocol. Hyperintense lesions on FLAIR were manually delineated on each patient by 7 experts with control on T2 sequence, and gathered in a consensus segmentation for evaluation. We provide raw and preprocessed data and a split of the dataset into training and testing data, the latter including data from a scanner not present in the training dataset. We strongly believe that this dataset will become a reference in MS lesions segmentation evaluation, allowing to evaluate many aspects: evaluation of performance on unseen scanner, comparison to individual experts performance, comparison to other challengers who already used this dataset, etc.
... As 3D-FLAIR 2 may also improve automatic lesion segmentation, it could easily be implemented in future automatic MS lesion detection algorithms [18]. Automatized segmentation and Artificial intelligence (AI) of MRI images have great potential in monitoring disease activity in demyelinating diseases of the central nervous system and guiding diagnostic pathways [1,16,19], 3D-FLAIR 2 may further improve the quality of diagnosing and monitoring these patients non-invasively. Likewise, it will also As WM lesion load only in part explains clinical disease progression, conversion and cognitive decline, cortical lesions increasingly become a focus of research [5]. ...
Article
Full-text available
Background Technical improvements in magnetic resonance imaging (MRI) acquisition, such as higher field strength and optimized sequences, lead to better multiple sclerosis (MS) lesion detection and characterization. Multiplication of 3D-FLAIR with 3D-T2 sequences (FLAIR²) results in isovoxel images with increased contrast-to-noise ratio, increased white–gray-matter contrast, and improved MS lesion visualization without increasing MRI acquisition time. The current study aims to assess the potential of 3D-FLAIR² in detecting cortical/leucocortical (LC), juxtacortical (JC), and white matter (WM) lesions. Objective To compare lesion detection of 3D-FLAIR² with state-of-the-art 3D-T2-FLAIR and 3D-T2-weighted images. Methods We retrospectively analyzed MRI scans of thirteen MS patients, showing previously noted high cortical lesion load. Scans were acquired using a 3 T MRI scanner. WM, JC, and LC lesions were manually labeled and manually counted after randomization of 3D-T2, 3D-FLAIR, and 3D-FLAIR² scans using the ITK-SNAP tool. Results LC lesion visibility was significantly improved by 3D-FLAIR² in comparison to 3D-FLAIR (4 vs 1; p = 0.018) and 3D-T2 (4 vs 1; p = 0.007). Comparing LC lesion detection in 3D-FLAIR² vs. 3D-FLAIR, 3D-FLAIR² detected on average 3.2 more cortical lesions (95% CI − 9.1 to 2.8). Comparing against 3D-T2, 3D-FLAIR² detected on average 3.7 more LC lesions (95% CI 3.3–10.7). Conclusions 3D-FLAIR² is an easily applicable time-sparing MR post-processing method to improve cortical lesion detection. Larger sampled studies are warranted to validate the sensitivity and specificity of 3D-FLAIR².
... Multiple sclerosis (MS) is a chronic inflammatory disorder of the central nervous system characterized by the presence of demyelinated lesions and axonal loss. [1][2][3] The presence of white matter (WM) lesions in the brain is one of the radiological features of MS. 4 Conventional magnetic resonance imaging (MRI) techniques including T2-and T1-weighted images as well as fluid attenuated inversion recovery (FLAIR) are sensitive for the detection of MS lesions. 5,6 However, this technology is limited due to a lack of pathologic specificity and a poor correlation with disability. ...
Article
Background: Normal-appearing white matter (NAWM) lesions are known to be present in multiple sclerosis (MS); however, it is not easy to distinguish these lesions from others in MRI. This study aimed to investigate the most useful value for estimating NAWM damage using fractional anisotropy (FA) histograms analysis. Methods: Data from patients with relapsing-remitting MS and healthy controls were analyzed using a 1.5T MRI system with SENSE-Head-8 coil. FA maps with diffusion- weighted images were acquired using a single-shot echo-planar imaging sequence. The median, standard deviation (SD), kurtosis, and skewness of white matter (WM) of each subject were compared between MS and healthy controls using an in-house application. Results: FA decrease in 8 patients with MS was observed upon comparison with 12 controls and leaned toward the left side. While the SDs of the healthy controls were not significantly different from those of patients with MS, patients with MS expressed significantly lower median values, and higher kurtosis and skewness compared to healthy controls. A trend for inverse associations existed between median and expanded disability status scale scores. Conclusion: Our data suggests that median FA values can allow for distinguishing between patients with MS and healthy controls with high accuracy.
... Generally, traditional supervised machine learning approaches are dependent on hand-crafted or low-level features. So far, plenty of supervised techniques for MS lesion segmentation have been proposed, such as decision random forests [7] [8], ensemble methods [9], non-local means [10], k-nearest neighbors, [11] [12] and combined inference from patient and healthy populations [13]. Another group of automated methods is unsupervised which extracts patterns from unlabeled data. ...
Preprint
Full-text available
Objective: Multiple Sclerosis (MS) is an autoimmune, and demyelinating disease that leads to lesions in the central nervous system. This disease can be tracked and diagnosed using Magnetic Resonance Imaging (MRI). Up to now a multitude of multimodality automatic biomedical approaches is used to segment lesions which are not beneficial for patients in terms of cost, time, and usability. The authors of the present paper propose a method employing just one modality (FLAIR image) to segment MS lesions accurately. Methods: A patch-based Convolutional Neural Network (CNN) is designed, inspired by 3D-ResNet and spatial-channel attention module, to segment MS lesions. The proposed method consists of three stages: (1) the contrast-limited adaptive histogram equalization (CLAHE) is applied to the original images and concatenated to the extracted edges in order to create 4D images; (2) the patches of size 80 * 80 * 80 * 2 are randomly selected from the 4D images; and (3) the extracted patches are passed into an attention-based CNN which is used to segment the lesions. Finally, the proposed method was compared to previous studies of the same dataset. Results: The current study evaluates the model, with a test set of ISIB challenge data. Experimental results illustrate that the proposed approach significantly surpasses existing methods in terms of Dice similarity and Absolute Volume Difference while the proposed method use just one modality (FLAIR) to segment the lesions. Conclusions: The authors have introduced an automated approach to segment the lesions which is based on, at most, two modalities as an input. The proposed architecture is composed of convolution, deconvolution, and an SCA-VoxRes module as an attention module. The results show, the proposed method outperforms well compare to other methods.
... Generally, traditional supervised machine learning approaches are dependent on hand-crafted or low-level features. So far, plenty of supervised techniques for MS lesion segmentation have been proposed, such as decision random forests [7], [8], ensemble methods [9], non-local VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ ...
Article
Full-text available
Objective: Multiple Sclerosis (MS) is an autoimmune and demyelinating disease that leads to lesions in the central nervous system. This disease can be tracked and diagnosed using Magnetic Resonance Imaging (MRI). A multitude of multimodality automatic biomedical approaches are used to segment lesions that are not beneficial for patients in terms of cost, time, and usability. The authors of the present paper propose a method employing just one modality (FLAIR image) to segment MS lesions accurately. Methods: A patch-based Convolutional Neural Network (CNN) is designed, inspired by 3D-ResNet and spatial-channel attention module, to segment MS lesions. The proposed method consists of three stages: (1) the Contrast-Limited Adaptive Histogram Equalization (CLAHE) is applied to the original images and concatenated to the extracted edges to create 4D images; (2) the patches of size $80\times 80\times 80\times2$ are randomly selected from the 4D images; and (3) the extracted patches are passed into an attention-based CNN which is used to segment the lesions. Finally, the proposed method was compared to previous studies of the same dataset. Results: The current study evaluates the model with a test set of ISIB challenge data. Experimental results illustrate that the proposed approach significantly surpasses existing methods of Dice similarity and Absolute Volume Difference while the proposed method uses just one modality (FLAIR) to segment the lesions. Conclusion: The authors have introduced an automated approach to segment the lesions, which is based on, at most, two modalities as an input. The proposed architecture comprises convolution, deconvolution, and an SCA-VoxRes module as an attention module. The results show, that the proposed method outperforms well compared to other methods.
... The former ones rely on a manually labeled training set and aim at learning a function that maps the input to the desired output. The latter do not require manual annotations as they are based on generative models that rely on modeling the MRI intensities values of different brain tissues and lesions [11]. ...
Article
Full-text available
The presence of cortical lesions in multiple sclerosis patients has emerged as an important biomarker of the disease. They appear in the earliest stages of the illness and have been shown to correlate with the severity of clinical symptoms. However, cortical lesions are hardly visible in conventional magnetic resonance imaging (MRI) at 3T, and thus their automated detection has been so far little explored. In this study, we propose a fully-convolutional deep learning approach, based on the 3D U-Net, for the automated segmentation of cortical and white matter lesions at 3T. For this purpose, we consider a clinically plausible MRI setting consisting of two MRI contrasts only: one conventional T2-weighted sequence (FLAIR), and one specialized T1-weighted sequence (MP2RAGE). We include 90 patients from two different centers with a total of 728 and 3856 gray and white matter lesions, respectively. We show that two reference methods developed for white matter lesion segmentation are inadequate to detect small cortical lesions, whereas our proposed framework is able to achieve a detection rate of 76% for both cortical and white matter lesions with a false positive rate of 29% in comparison to manual segmentation. Further results suggest that our framework generalizes well for both types of lesion in subjects acquired in two hospitals with different scanners.
... /fnins. . clinical practice, MRI data can be used to diagnose and assess MS lesions, which helps physicians better understand the natural history of MS (Lladó et al., 2012;Combès et al., 2021). Fluid Attenuated Inversion Recovery (FLAIR) is an MRI technique that provides images in which WM lesions emerge as highintensity areas, allowing for tracking of the disease progression (Rovira et al., 2015). ...
Article
Full-text available
Multiple sclerosis (MS) is an autoimmune disease that causes lesions in the central nervous system of humans due to demyelinating axons. Magnetic resonance imaging (MRI) is widely used for monitoring and measuring MS lesions. Automated methods for MS lesion segmentation have usually been performed on individual MRI scans. Recently, tracking lesion activity for quantifying and monitoring MS disease progression, especially detecting new lesions, has become an important biomarker. In this study, a unique pipeline with a deep neural network that combines U-Net, attention gate, and residual learning is proposed to perform better new MS lesion segmentation using baseline and follow-up 3D FLAIR MR images. The proposed network has a similar architecture to U-Net and is formed from residual units which facilitate the training of deep networks. Networks with fewer parameters are designed with better performance through the skip connections of U-Net and residual units, which facilitate information propagation without degradation. Attention gates also learn to focus on salient features of the target structures of various sizes and shapes. The MSSEG-2 dataset was used for training and testing the proposed pipeline, and the results were compared with those of other proposed pipelines of the challenge and experts who participated in the same challenge. According to the results over the testing set, the lesion-wise F1 and dice scores were obtained as a mean of 48 and 44.30%. For the no-lesion cases, the number of tested and volume of tested lesions were obtained as a mean of 0.148 and 1.488, respectively. The proposed pipeline outperformed 22 proposed pipelines and ranked 8th in the challenge.
... Additional techniques used to classify multiple sclerosis state or subtype on the basis of non-MRS data sets have included neural networks 18,25,26,[35][36][37][38][39] , K-nearest neighbors 17,20,25,27,37 (KNN), decision trees 17,18,26,40 , logistic regression 17,27 , Naïve Bayes 25 , and least squares 27 or maximum likelihood estimation 41 . A range of classifiers has additionally been employed, also on non-MRS data, to characterize or predict disease conversion 42 , symptom severity [43][44][45][46][47][48][49][50][51] , or treatment effect [52][53][54][55] , and especially to automatically segment MRI-visible multiple sclerosis lesions 56 . ...
Article
Full-text available
Multiple sclerosis (MS) is a heterogeneous autoimmune disease for which diagnosis continues to rely on subjective clinical judgment over a battery of tests. Proton magnetic resonance spectroscopy (1H MRS) enables the noninvasive in vivo detection of multiple small-molecule metabolites and is therefore in principle a promising means of gathering information sufficient for multiple sclerosis diagnosis and subtype classification. Here we show that supervised classification using 1H-MRSvisible normal-appearing frontal cortex small-molecule metabolites alone can indeed differentiate individuals with progressive MS from control (held-out validation sensitivity 79% and specificity 68%), as well as between relapsing and progressive MS phenotypes (held-out validation sensitivity 84% and specificity 74%). Post hoc assessment demonstrated the disproportionate contributions of glutamate and glutamine to identifying MS status and phenotype, respectively. Our finding establishes 1H MRS as a viable means of characterizing progressive multiple sclerosis disease status and paves the way for continued refinement of this method as an auxiliary or mainstay of multiple sclerosis diagnostics.
... Several automated segmentations that apply a supervised learning strategy have been proposed such as [19-231. On the other hand, unsupervised segmentation-based attempts to separate the brain image to multiple tissue clusters, for example white matter or gray matter [24][25][26][27], Lastly, a semi-supervised learning method that uses a small chunk of training images to teach a segmentation algorithm can be found in |28| Regardless of training methodology, the briefly details of some example algorithms are explained as following. ...
Article
The Stockwell Transform has the potential to perform multi-resolution texture analysis in magnetic resonance imaging (MRI). However, it is computationally intensive and memory demanding. The polar Stockwell Transform (PST) is rotation-invariant and relatively memory efficient, but still computationally demanding. The new Discrete Orthogonal Stockwell Transform (DOST) appears to have addressed both the computation and storage challenges; however, its utility in localized texture analysis remains unclear. Our goal was to investigate the theory and texture analysis ability of the DOST versus PST using both synthetic and MR images, and explore the relative importance of the associated texture features using a simple classification example based on clinical brain MRI of six multiple sclerosis patients. MRI texture analysis focused on FLAIR images, and the classification used a machine learning algorithm, random forest, that differentiated regions of interest (ROIs) into 2 classes: white matter lesions, and the contralateral normal-appearing white matter (control). Our results showed that the PST features had a greater ability in detecting subtle changes in image structure than the DOST and polar-index DOST (PDOST). Quantitatively, based on 187 lesion and 187 control ROIs, both the PST and the rotation-invariant radial PST performed better in the classification than the DOST and PDOST, where the latter were no better than guessing (p = 0.65 and 0.98). Further analysis using a hierarchical random forest showed that combining MRI signal intensity with the PST or DOST predictions increased the classification performance, with the accuracy, sensitivity, and specificity all improved to >85% in the tests. Collectively, the DOST is less competitive than the PST in localized image texture analysis. The PST features may help with texture-based lesion classification in MS based on clinical brain MRI scans following further verification.
Preprint
We propose a novel numerical approach to separate multiple tissue compartments in image voxels and to estimate quantitatively their nuclear magnetic resonance (NMR) properties and mixture fractions, given magnetic resonance fingerprinting (MRF) measurements. The number of tissues, their types or quantitative properties are not a-priori known, but the image is assumed to be composed of sparse compartments with linearly mixed Bloch magnetisation responses within voxels. Fine-grid discretisation of the multi-dimensional NMR properties creates large and highly coherent MRF dictionaries that can challenge scalability and precision of the numerical methods for (discrete) sparse approximation. To overcome these issues, we propose an off-the-grid approach equipped with an extended notion of the sparse group lasso regularisation for sparse approximation using continuous (non-discretised) Bloch response models. Further, the nonlinear and non-analytical Bloch responses are approximated by a neural network, enabling efficient back-propagation of the gradients through the proposed algorithm. Tested on simulated and in-vivo healthy brain MRF data, we demonstrate effectiveness of the proposed scheme compared to the baseline multicompartment MRF methods.
Chapter
Medical image processing techniques have revolutionized the visual depictions of within a body medicinal examination and visual description of certain organs or tissues. Medical imaging exploring brain anatomy and functions to analyze brain injuries have always been a complex task as there are large amount of datasets, which require computerized methods to extract the useful information. This paper presents the clustering-based segmentation approach using soft computing techniques with the aid of morphology to segment the region of interest from the MRI images. This work proposes a method of processing the MRI images using Particle Swarm Optimization (PSO)-based clustering approach for the segmentation of brain tissues, as to avoid the limitation of initial cluster definition and to improve the accuracy in comparison with the Fuzzy C Means (FCM)-based segmentation technique. The proposed method has been validated and accuracy is calculated as performance evaluation parameter, with compared to ground truth of given dataset in different images.
Book
This book is a collection of best selected high-quality research papers presented at the International Conference on Advances in Energy Management (ICAEM 2019) organized by the Department of Electrical Engineering, Jodhpur Institute of Engineering & Technology (JIET), Jodhpur, India, during 20–21 December 2019. The book discusses intelligent energy management technologies which are cost effective compared to the high cost of fossil fuels. This book also explains why these systems have beneficial impact on environmental, economic and political issues of the world. The book is immensely useful for research scholars, academicians, R&D institutions, practicing engineers and managers from industry.
Chapter
Nowadays, tumor detection in the brain’s ground truth image is a difficult task for the medical practitioner as the brain’s spatial and structural variables like region of occurrence, contrast, intensity, size, and shape vary for every image due to different image acquisition methods, modality, and patients’ age. Manual segmentation of tumor region in magnetic resonance image (MRI) is tedious and time taking process for the medical examiner. To overcome this, many semi and fully automated brain tumor detection systems were developed in the literature, and still, the evolution continues to have improved efficient and accurate diagnosis. The main intention of this paper is to give an in-depth review of various advanced segmentation methods developed in the literature over the period. Though numerous review papers exist, we focus on the most recent soft computing techniques used in MRI brain analysis. The segmentation algorithms reviewed in this work are the neural network model, self-organizing maps, backpropagation, fuzzy C-means, deformable models, and genetic and hybrid algorithms. Initially, the methods are classified based on supervised and unsupervised categories, then advanced techniques developed in each method are discussed, and lastly, comments on future directions are discussed.
Article
This paper presents a new fuzzy-based method for the segmentation of brain structures from noisy magnetic resonance (MR) images, in the presence of noise. Our algorithm is a new extension of the fuzzy C-means (FCM) algorithm. The proposed algorithm is developed by modifying the objective function in the FCM using double estimation by incorporating both the original and denoised images in place of using solely the denoised image. To the best of our knowledge, the proposed algorithm is the first extension of the FCM method that is capable of segmenting images (per pixel) based on both noisy and denoised image estimates. In this algorithm we: (a) introduce a novel formulation that assigns weights for each estimation using spatial image information and (b) apply a kernel distance metric for image segmentation. This formulation is highly applicable in segmenting images corrupted by high levels of noise. Experimental results on both simulated and original MR images are presented to demonstrate the robustness and effectiveness of our proposed algorithm in the presence of noise. These results are compared to the nonlocal fuzzy C-means method (LNLFCM), discrete cosine transform-LNLFCM (DCT-LNLFCM), kernel weighted fuzzy local information C-means (KWFLICM), and bias correction embedded fuzzy C-means with spatial constraint (BCEFCM-S) algorithm.
Article
Multiple Sclerosis (MS) is a chronic and autoimmune neurological disease that is frequently seen especially in young people. MS lesions that can be seen with magnetic resonance imaging (MRI) findings are important biomarkers that provide information about the clinical prognosis and activity of the disease. The presence of new MS lesions is associated with future disease activity. This study aims to predict the future activity of MS using the 3D discrete wavelet transform (DWT) as a feature extraction method from 3D MRI. The 3D-DWT can be used as it provides spatial and spectral location features of MS lesions without losing their relationship between MRI slices. Ten different wavelet families of DWT are used individually, each of them is classified by six machine learning algorithms, and their feature extraction performances are compared. The highest F1-score, Precision, and Recall of 95.0% are obtained by the support vector machine algorithm on the SYM4, SYM8, and Haar wavelet families in the 3D MRI dataset consisting of 40 patients based on 5-fold cross validation. The results show that the 3D-DWT method is an effective method for feature extraction in predicting the future activity of MS.
Thesis
Quantitative magnetic resonance imaging (MRI) is a non-invasive and versatile tool for the assessment of anatomical structures. In recent years, MRI has evolved rapidly and is of high clinical interest because of its potential to distinguish diseased from healthy tissue. A variety of methods have been proposed for quantitative cardiac MRI, but insufficient precision and practicality limit its clinical use. One objective of this work was to analyze the effects of blood flow in relation to T1 relaxation times of blood for conventional inversion recovery (IR) and saturation recovery (SR) methods. Simulations, phantom, and in vivo experiments were performed to validate the effects of flow. The in-flow of non-prepared spins resulted in decreased T1 times, and thus SR methods were found to be more resistant to flow effects. Based on this, a sequence was developed for simultaneous quantification of T1, T2, and T2*. Phantom measurements were performed with high accuracy in agreement with simulations and good visual image quality was observed in the myocardium compared to reference methods and in patients. In the second part of the work, a novel renal magnetic resonance fingerprinting (MRF) approach was developed for the simultaneous quantification of T1 and T2* within four slices. Simulations showed good agreement with phantom measurements and a convergence of the reconstructed relaxation times. In vivo measurements benefited from a 10-fold speedup compared to conventional methods and good reproducibility for repeated measurements. Additionally, this technique has been used in brain scans at two centers to study white matter lesions in patients with multiple sclerosis. Complex and computationally costly data processing was replaced by a neural network combining noise reduction, T1 and T2* reconstruction, distortion correction, and white matter, gray matter and lesion segmentation. Robust and accurate parameter maps provide reconstructions with a 100-fold speed up, and therefore ideal for clinical applications.
Article
Purpose This study aimed at introducing short-T1/T2 compartment to MR fingerprinting (MRF) at 3 T. Water that is bound to myelin macromolecules have significantly shorter T1 and T2 than free water and can be distinguished from free water by multi-compartment analysis. Methods We developed a new multi-inversion-recovery (mIR) water mapping-MRF based on an unbalanced steady-state coherent sequence (FISP). mIR pulses with an interval of 400 or 500 repetition times (TRs) were inserted into the conventional FISP MRF sequence. Data from our proposed mIR MRF was used to quantify different compartments, including myelin water, gray matter free water, and white matter free water, of brain water by virtue of the iterative non-negative least square (NNLS) with reweighting. Three healthy volunteers were scanned with mIR MRF on a clinical 3 T MRI. Results Using an extended phase graph simulation, we found that our proposed mIR scheme with four IR pulses allowed differentiation between short and long T1/T2 components. For in vivo experiments, we achieved the quantification of myelin water, gray matter water, and white matter water at an image resolution of 1.17 × 1.17 × 5 mm³/pixel. As compared to the conventional MRF technique with single IR, our proposed mIR improved the detection of myelin water content. In addition, mIR MRF using spiral-in/out trajectory provided a higher signal level compared with that with spiral-out trajectory. Myelin water quantification using mIR MRF with 4 IR and 5 IR pulses were qualitatively similar. Meanwhile, 5 IR MRF showed fewer artifacts in myelin water detection. Conclusion We developed a new mIR MRF sequence for the rapid quantification of brain water compartments.
Article
Brain tissue segmentation in multi-modal magnetic resonance (MR) images is significant for the clinical diagnosis of brain diseases. Due to blurred boundaries, low contrast, and intricate anatomical relationships between brain tissue regions, automatic brain tissue segmentation without prior knowledge is still challenging. This paper presents a novel 3D fully convolutional network (FCN) for brain tissue segmentation, called APRNet. In this network, we first propose a 3D anisotropic pyramidal convolutional reversible residual sequence (3DAPC-RRS) module to integrate the intra-slice information with the inter-slice information without significant memory consumption; secondly, we design a multi-modal cross-dimension attention (MCDA) module to automatically capture the effective information in each dimension of multi-modal images; then, we apply 3DAPC-RRS modules and MCDA modules to a 3D FCN with multiple encoded streams and one decoded stream for constituting the overall architecture of APRNet. We evaluated APRNet on two benchmark challenges, namely MRBrainS13 and iSeg-2017. The experimental results show that APRNet yields state-of-the-art segmentation results on both benchmark challenge datasets and achieves the best segmentation performance on the cerebrospinal fluid region. Compared with other methods, our proposed approach exploits the complementary information of different modalities to segment brain tissue regions in both adult and infant MR images, and it achieves the average Dice coefficient of 87.22% and 93.03% on the MRBrainS13 and iSeg-2017 testing data, respectively. The proposed method is beneficial for quantitative brain analysis in the clinical study, and our code is made publicly available.
Article
Purpose: For the planning and navigation of neurosurgery, we have developed a fully convolutional network (FCN)-based method for brain structure segmentation on magnetic resonance (MR) images. The capability of an FCN depends on the quality of the training data (i.e., raw data and annotation data) and network architectures. The improvement of annotation quality is a significant concern because it requires much labor for labeling organ regions. To address this problem, we focus on skip connection architectures and reveal which skip connections are effective for training FCNs using sparsely annotated brain images. Methods: We tested 2D FCN architectures with four different types of skip connections. The first was a U-Net architecture with horizontal skip connections that transfer feature maps at the same scale from the encoder to the decoder. The second was a U-Net++ architecture with dense convolution layers and dense horizontal skip connections. The third was a full-resolution residual network (FRRN) architecture with vertical skip connections that pass feature maps between each downsampled scale path and the full-resolution scale path. The last one was a hybrid architecture with a combination of horizontal and vertical skip connections. We validated the effect of skip connections on medical image segmentation from sparse annotation based on these four FCN architectures, which were trained under the same conditions. Results: For multi-class segmentation of the cerebrum, cerebellum, brainstem, and blood vessels from sparsely annotated MR images, we performed a comparative evaluation of segmentation performance among the above four FCN approaches: U-Net, U-Net++, FRRN, and hybrid architectures. The experimental results show that the horizontal skip connections in the U-Net architectures were effective for the segmentation of larger-sized objects, while the vertical skip connections in the FRRN architecture improved the segmentation of smaller-sized objects. The hybrid architecture with both horizontal and vertical skip connections achieved the best results of the four FCN architectures. We then performed an ablation study to explore which skip connections in the FRRN architecture contributed to the improved segmentation of blood vessels. In the ablation study, we compared the segmentation performance between architectures with a horizontal path (HP), a horizontal path and vertical up paths (HP+VUPs), a horizontal path and vertical down paths (HP+VDPs), and a horizontal path and vertical up and down paths (FRRN). We found that the vertical up paths were effective in improving the segmentation of smaller-sized objects. Conclusions: This paper investigated which skip connection architectures were effective for multi-class brain segmentation from sparse annotation. Consequently, using vertical skip connections with horizontal skip connections allowed FCNs to improve segmentation performance. This article is protected by copyright. All rights reserved.
Article
Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patients neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.
Chapter
Disease occurrence rates in humans are rapidly rising for various reasons and so timely detection and treatment implementation are necessary to ensure successful cures. The disease diagnosis procedure varies based on the type and severity of the disease. Further, this scheme also depends on the infected area under examination. In most cases, verification by a doctor, followed by signal-based prescreening and image-based postscreening is widely recommended to confirm the disease and its severity. This chapter provides a detailed overview of image-supported disease screening procedures adopted in hospitals to examine the various internal organs, such as the breasts, heart, and brain. The various imaging modalities considered to examine the abnormalities in these organs are examined and the advantages and disadvantages of the considered imaging schemes are discussed. Compared to other imaging methods, magnetic resonance imaging is confirmed as one of the most widely adopted radiological procedures to identify disease in these organs with improved visibility.
Article
We propose a novel numerical approach to separate multiple tissue compartments in image voxels and to estimate quantitatively their nuclear magnetic resonance (NMR) properties and mixture fractions, given magnetic resonance fingerprinting (MRF) measurements. The number of tissues, their types or quantitative properties are not a-priori known, but the image is assumed to be composed of sparse compartments with linearly mixed Bloch magnetisation responses within voxels. Fine-grid discretisation of the multi-dimensional NMR properties creates large and highly coherent MRF dictionaries that can challenge scalability and precision of the numerical methods for (discrete) sparse approximation. To overcome these issues, we propose an off-the-grid approach equipped with an extended notion of the sparse group lasso regularisation for sparse approximation using continuous (non-discretised) Bloch response models. Further, the nonlinear and non-analytical Bloch responses are approximated by a neural network, enabling efficient back-propagation of the gradients through the proposed algorithm. Tested on simulated and in-vivo healthy brain MRF data, we demonstrate effectiveness of the proposed scheme compared to the baseline multi-compartment MRF methods.
Preprint
Full-text available
Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patient's neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to extract and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.
Article
Full-text available
We present a fully automated framework for identifying mul-tiple sclerosis (MS) lesions from multispectral human brain magnetic res-onance images (MRIs). The brain tissue intensities and lesions are both modeled using Markov Random Fields (MRFs) to incorporate local spa-tial variations and neighborhood information. In this work, we model all brain tissues, including lesions, as separate classes as opposed to the common approach of modelling the lesions as outliers of the brain tissues. A maximum probability estimate is obtained by arriving at the global convergence of the MRFs using Simulated Annealing. Finally, probabil-ity surface discontinuities due to noise and local intensity variations are avoided by incorporating a spline based smoothing function following the MRF modelling. The algorithm is validated on a set of real MRI brain volumes of MS patients with widely varying lesion loads by comparing the results against a silver standard derived from manual expert labellings. The algorithm yields favorable results, including in the posterior fossa where few methods have proved successful. Further, our algorithm yields fewer false negatives than is usual in practice.
Article
Full-text available
Different sets of criteria are currently used for the diagnosis of multiple sclerosis (MS). Some are based on clinical features, while others are related to imaging findings. Among the image processing systems, specific criteria include spatial dissemination of lesions in one image or their temporal dissemination in images acquired at different time points. In addition, the evolution of the lesion load can be used to evaluate treatment efficiency in MS clinical research. Consequently, obtaining a precise segmentation of the MS lesion appears to be crucial. In the literature, a number of semi-automated or completely automated approaches have been proposed enabling a reduction of the inter- and intra-expert variability for manual delineations. A comprehensive state-of-the-art classification of the most representative systems is presented here.
Article
Full-text available
This paper describes the setup of a segmentation competition for the automatic extraction of Multiple Sclerosis (MS) lesions from brain Magnetic Resonance Imaging (MRI) data. This competition is one of three competitions that make up a comparison workshop at the 2008 Medical Image Computing and Computer Assisted Intervention (MICCAI) conference and was modeled after the successful comparison workshop on liver and caudate segmentation at the 2007 MICCAI conference. In this paper, the rationale for organizing the competition is discussed, the training and test data sets for both segmentation tasks are described and the scoring system used to evaluate the segmentation is presented.
Article
Full-text available
Intensity inhomogeneity (IIH) is often encountered in MR imaging, and a number of techniques have been devised to correct this artifact. This paper attempts to review some of the recent developments in the mathematical modeling of IIH field. Low-frequency models are widely used, but they tend to corrupt the low-frequency components of the tissue. Hypersurface models and statistical models can be adaptive to the image and generally more stable, but they are also generally more complex and consume more computer memory and CPU time. They are often formulated together with image segmentation within one framework and the overall performance is highly dependent on the segmentation process. Beside these three popular models, this paper also summarizes other techniques based on different principles. In addition, the issue of quantitative evaluation and comparative study are discussed.
Article
Full-text available
PurposeTo assess the reproducibility and accuracy compared to radiologists of three automated segmentation pipelines for quantitative magnetic resonance imaging (MRI) measurement of brain white matter signal abnormalities (WMSA).Materials and MethodsWMSA segmentation was performed on pairs of whole brain scans from 20 patients with multiple sclerosis (MS) and 10 older subjects who were positioned and imaged twice within 30 minutes. Radiologist outlines of WMSA on 20 sections from 16 patients were compared with the corresponding results of each segmentation method.ResultsThe segmentation method combining expectation-maximization (EM) tissue segmentation, template-driven segmentation (TDS), and partial volume effect correction (PVEC) demonstrated the highest accuracy (the absolute value of the Z-score was 0.99 for both groups of subjects), as well as high interscan reproducibility (repeatability coefficient was 0.68 mL in MS patients and 1.49 mL in aging subjects).Conclusion The addition of TDS to the EM segmentation and PVEC algorithms significantly improved the accuracy of WMSA volume measurements, while also improving measurement reproducibility. J. Magn. Reson. Imaging 2002;15:203–209. © 2002 Wiley-Liss, Inc.
Conference Paper
Full-text available
Graph Cuts have been shown as a powerful interactive segmentation technique in several medical domains. We propose to automate the Graph Cuts in order to automatically segment Multiple Sclerosis (MS) lesions in MRI. We replace the manual interaction with a robust EM-based approach in order to discriminate between MS lesions and the Normal Appearing Brain Tissues (NABT). Evaluation is performed in synthetic and real images showing good agreement between the automatic segmentation and the target segmentation. We compare our algorithm with the state of the art techniques and with several manual segmentations. An advantage of our algorithm over previously published ones is the possibility to semi-automatically improve the segmentation due to the Graph Cuts interactive feature.
Article
Full-text available
The purpose of this study is to develop a reproducible method for quantifying brain lesions in traumatic brain injury (TBI). Quantifying the effects of neuropathology is an important goal in the study of brain injury and disease, yet examiners have encountered significant difficulty quantifying brain lesions in neurotrauma where there may exist multiple, overlapping forms of injury including large focal lesions and more subtle, diffuse hemorrhage and/or shear injury. In the current study, we used conventional MRI to quantify brain lesion volume at separate time points in individuals with severe TBI. We present an automated method (ISODATA) for quantifying brain lesions that is compared against a standard semi-automated volumetric approach. The ISODATA method makes no assumptions about the location or extent of brain lesions, instead identifying areas of neuropathology via voxelwise comparisons of MRI signal intensity. The data reveal that ISODATA overlaps significantly with a semi-automated approach, is reliable across multiple observations, and is sensitive to change in lesion size during recovery from TBI. This study validates a reproducible, automated lesion quantification method used here to determine the location and extent of brain pathology following TBI. This approach may be used in conjunction with advanced imaging techniques to characterize the relationship between brain lesions and neurometabolism and function.
Chapter
Full-text available
Intensity-based classification of MR images has proven problematic, even when advanced techniques are used. Intra-scan and interscan intensity inhomogeneities are a common source of difficulty. While reported methods have had some success in correcting intra-scan inhomogeneities, such methods require supervision for the individual scan. This paper describes a new method called adaptive segmentation that uses knowledge of tissue intensity properties and intensity inhomogeneities to correct and segment MR images. Use of the EM algorithm leads to a fully automatic method that allows for more accurate segmentation of tissue types as well as better visualization of MRI data, that has proven to be effective in a study that includes more than 1000 brain scans.
Conference Paper
Full-text available
We have developed a 4D connected component labeling algorithm to identify automatically individual lesions on MRI brain images of multiple sclerosis patients, and to follow the changes in each lesion over time. We were also able to use this algorithm as a noise reduction filter to remove misclassified components from the binarized segmentations of the images.
Article
Full-text available
This paper aims to present a review of recent as well as classic image registration methods. Image registration is the process of overlaying images (two or more) of the same scene taken at different times, from different viewpoints, and/or by different sensors. The registration geometrically align two images (the reference and sensed images). The reviewed approaches are classified according to their nature (area-based and feature-based) and according to four basic steps of image registration procedure: feature detection, feature matching, mapping function design, and image transformation and resampling. Main contributions, advantages, and drawbacks of the methods are mentioned in the paper. Problematic issues of image registration and outlook for the future research are discussed too. The major goal of the paper is to provide a comprehensive reference source for the researchers involved in image registration, regardless of particular application areas.
Article
Full-text available
Multiple sclerosis is an idiopathic inflammatory disease characterized by multiple focal lesions in the white matter of the central nervous system. Multiple sclerosis patients are usually treated with interferon-β, but disease activity decrease in only 30–40% of patients. In the attempt to differentiate between responders and non-responders, we screened the main genes involved in the interferon signaling pathway for 38 single nucleotide polymorphisms (SNPs) in a multiple sclerosis Caucasian population from South Italy. We then analyzed the data using a multilayer perceptron neural network-based approach, in which we evaluated the global weight of a set of SNPs localized in different genes and their association with response to interferon therapy through a feature selection procedure (a combination of automatic relevance determination and backward elimination). The neural approach appears to be a useful tool in identifying gene polymorphisms involved in the response of patients to interferon therapy: 2 out of 5 genes were identified as containing 4 out of 38 significant single nucleotide polymorphisms, with a global accuracy of 70% in predicting responder and non-responder patients.
Conference Paper
Full-text available
Over the last five years, new "voxel-based" approaches have allowed important progress in multimodal image registration, notably due to the increasing use of information-theoretic similarity measures. Their wide success has led to the progressive abandon of measures using standard image statistics (mean and variance). Until now, such measures have essentially been based on heuristics. In this paper, we address the determination of a new measure based on standard statistics from a theoretical point of view. We show that it naturally leads to a known concept of probability theory, the correlation ratio. In our derivation, we take as the hypothesis the functional dependence between the image intensities. Although such a hypothesis is not as general as possible, it enables us to model the image smoothness prior very easily. We also demonstrate results of multimodal rigid registration involving Magnetic Resonance (MR), Computed Tomography (CT), and Positron Emission Tomography (PET) images. These results suggest that the correlation ratio provides a good trade-o between accuracy and robustness.
Article
Multiple sclerosis is primarily an inflammatory disorder of the brain and spinal cord in which focal lymphocytic infiltration leads to damage of myelin and axons. Initially, inflammation is transient and remyelination occurs but is not durable. Hence, the early course of disease is characterised by episodes of neurological dysfunction that usually recover. However, over time the pathological changes become dominated by widespread microglial activation associated with extensive and chronic neurodegeneration, the clinical correlate of which is progressive accumulation of disability. Paraclinical investigations show abnormalities that indicate the distribution of inflammatory lesions and axonal loss (MRI); interference of conduction in previously myelinated pathways (evoked electrophysiological potentials); and intrathecal synthesis of oligoclonal antibody (examination by lumbar puncture of the cerebrospinal fluid). Multiple sclerosis is triggered by environmental factors in individuals with complex genetic-risk profiles. Licensed disease modifying agents reduce the frequency of new episodes but do not reverse fixed deficits and have questionable effects on the long-term accumulation of disability and disease progression. We anticipate that future studies in multiple sclerosis will provide a new taxonomy on the basis of mechanisms rather than clinical empiricism, and so inform strategies for improved treatment at all stages of the disease.
Conference Paper
Many applications of MRI are facilitated by segmenting the volume spanned by the imagery into the various tissue types that are present. Intensity-based classification of MR images has proven to be problematic, even when advanced techniques such as non- parametric multi-channel methods are used. A persistent difficulty has been accommodating the spatial intensity inhomogeneities that are due to the equipment. This paper describes a statistical method that uses knowledge of tissue properties and intensity inhomogeneities to correct for these intensity inhomogeneities. Use of the Expectation-Maximization algorithm leads to a method (EM segmentation) for simultaneously estimating tissue class and the correcting gain field. The algorithm iterates two components to convergence: tissue classification, and gain field estimation. The result is a powerful new technique for segmenting and correcting MR images. An implementation of the method is discussed, and results are reported for segmentation of white matter and gray matter in gradient-echo and spin-echo images. Examples are shown for axial, coronal and sagittal (surface coil) images. For a given type of acquisition, intensity variations across patients, scans, and equipment have been accommodated without manual intervention in the segmentation. In this sense, the method is fully automatic for segmenting healthy brain tissue. An accuracy assessment was made in which the method was compared to manual segmentation, and to a method based on supervised multi-variate classification, in segmenting white matter and gray matter. The method was found to be consistent with manual segmentation, and closer to manual segmentation than the supervised method.
Article
This paper is concerned with the development of entropy-based registration criteria for automated 3D multi-modality medical image alignment. In this application where misalignment can be large with respect to the imaged field of view, invariance to overlap statistics is an important consideration. Current entropy measures are reviewed and a normalised measure is proposed which is simply the ratio of the sum of the marginal entropies and the joint entropy. The effect of changing overlap on current entropy measures and this normalised measure are compared using a simple image model and experiments on clinical image data. Results indicate that the normalised entropy measure provides significantly improved behaviour over a range of imaged fields of view.
Article
A fuzzy c-means approach is described for tissue segmentation of X-ray computed tomography (CT) and T1 weighted magnetic resonance (MR) images of the same crosssection of the human brain. A fuzzy set approach is then utilized to obtain a fused classification displaying the salient features of image data of the individual modalities.
Article
In progressive neurological disorders, such as multiple sclerosis (MS), magnetic resonance imaging (MRI) follow-up is used to monitor disease activity and progression and to understand the underlying pathogenic mechanisms. This article presents image postprocessing methods and validation for integrating multiple serial MRI scans into a spatiotemporal volume for direct quantitative evaluation of the temporal intensity profiles. This temporal intensity signal and its dynamics have thus far not been exploited in the study of MS pathogenesis and the search for MRI surrogates of disease activity and progression. The integration into a four-dimensional data set comprises stages of tissue classification, followed by spatial and intensity normalization and partial volume filtering. Spatial normalization corrects for variations in head positioning and distortion artifacts via fully automated intensity-based registration algorithms, both rigid and nonrigid. Intensity normalization includes separate stages of correcting intra- and interscan variations based on the prior tissue class segmentation. Different approaches to image registration, partial volume correction, and intensity normalization were validated and compared. Validation included a scan-rescan experiment as well as a natural-history study on MS patients, imaged in weekly to monthly intervals over a 1-year follow-up. Significant error reduction was observed by applying tissue-specific intensity normalization and partial volume filtering. Example temporal profiles within evolving multiple sclerosis lesions are presented. An overall residual signal variance of 1.4% +/- 0.5% was observed across multiple subjects and time points, indicating an overall sensitivity of 3% (for axial dual echo images with 3-mm slice thickness) for longitudinal study of signal dynamics from serial brain MRI.
Article
The automatic analysis of subtle changes between MRI scans is an important tool for assessing disease evolution over time. Manual labeling of evolutions in 3D data sets is tedious and error prone. Automatic change detection, however, remains a challenging image processing problem. A variety of MRI artifacts introduce a wide range of unrepresentative changes between images, making standard change detection methods unreliable. In this study we describe an automatic image processing system that addresses these issues. Registration errors and undesired anatomical deformations are compensated using a versatile multiresolution deformable image matching method that preserves significant changes at a given scale. A nonlinear intensity normalization method is associated with statistical hypothesis test methods to provide reliable change detection. Multimodal data is optionally exploited to reduce the false detection rate. The performance of the system was evaluated on a large database of 3D multimodal, MR images of patients suffering from relapsing remitting multiple sclerosis (MS). The method was assessed using receiver operating characteristics (ROC) analysis, and validated in a protocol involving two neurologists. The automatic system outperforms the human expert, detecting many lesion evolutions that are missed by the expert, including small, subtle changes.
Article
A fully automated magnetic resonance (MR) segmentation method for identification and volume measurement of demyelinated white matter has been developed. Spin-echo MR brain scans were performed in 38 patients with multiple sclerosis (MS) and in 46 healthy subjects. Segmentation of normal tissues and white matter lesions (WML) was obtained, based on their relaxation rates and proton density maps. For WML identification, additional criteria included three-dimensional (3D) lesion shape and surrounding tissue composition. Segmented images were generated, and normal brain tissues and WML volumes were obtained. Sensitivity, specificity, and reproducibility of the method were calculated, using the WML identified by two neuroradiologists as the gold standard. The average volume of “abnormal” white matter in normal subjects (false positive) was 0.11 ml (range 0–0.59 ml). In MS patients the average WML volume was 31.0 ml (range 1.1–132.5 ml), with a sensitivity of 87.3%. In the reproducibility study, the mean SD of WML volumes was 2.9 ml. The procedure appears suitable for monitoring disease changes over time. J. Magn. Reson. Imaging 2000;12:799–807. © 2000 Wiley-Liss, Inc.
Article
PurposeTo evaluate the accuracy, reproducibility, and speed of two semiautomated methods for quantifying total white matter lesion burden in multiple sclerosis (MS) patients with respect to manual tracing and to other methods presented in recent literature.Materials and Methods Two methods involving the use of MRI for semiautomated quantification of total lesion burden in MS patients were examined. The first method, geometrically constrained region growth (GEORG), requires user specification of lesion location. The second technique, directed multispectral segmentation (DMSS), requires only the location of a single exemplar lesion. Test data sets included both clinical MS data and MS brain phantoms.ResultsThe mean processing times were 60 minutes for manual tracing, 10 minutes for region growth, and 3 minutes for directed segmentation. Intra- and interoperator coefficients of variation (CVs) were 5.1% and 16.5% for manual tracing, 1.4% and 2.3% for region growth, and 1.5% and 5.2% for directed segmentation. The average deviations from manual tracing were 9% for region growth and 5.7% for directed segmentation.Conclusion Both semiautomated methods were shown to have a significant advantage over manual tracing in terms of speed and precision. The accuracy of both methods was acceptable, given the high variability of the manual results. J. Magn. Reson. Imaging 2003;17:300–308. © 2003 Wiley-Liss, Inc.
Conference Paper
We propose to segment Multiple Sclerosis (MS) lesions overtime in multidimensional Magnetic Resonance (MR) sequences. We use a robust algorithm that allows the segmentation of the abnormalities using the whole time series simultaneously and we propose an original rejection scheme for outliers. We validate our method using the BrainWeb simulator. To conclude, promising preliminary results on longitudinal multi-sequences of clinical data are shown.
Conference Paper
Combining image segmentation based on statistical classification with a geometric prior has been shown to significantly increase robustness and reproducibility. Using a probabilistic geometric model of sought structures and image registration serves both initialization of probability density functions and definition of spatial constraints. A strong spatial prior, however, prevents segmentation of structures that are not part of the model. In practical applications, we encounter either the presentation of new objects that cannot be modeled with a spatial prior or regional intensity changes of existing structures not explained by the model. Our driving application is the segmentation of brain tissue and tumors from three-dimensional magnetic resonance imaging (MRI). Our goal is a high-quality segmentation of healthy tissue and a precise delineation of tumor boundaries. We present an extension to an existing expectation maximization (EM) segmentation algorithm that modifies a probabilistic brain atlas with an individual subject’s information about tumor location obtained from subtraction of post- and pre-contrast MRI. The new method handles various types of pathology, space-occupying mass tumors and infiltrating changes like edema. Preliminary results on five cases presenting tumor types with very different characteristics demonstrate the potential of the new technique for clinical routine use for planning and monitoring in neurosurgery, radiation oncology, and radiology.
Article
A variant of the popular nonparametric nonuniform intensity normalization (N3) algorithm is proposed for bias field correction. Given the superb performance of N3 and its public availability, it has been the subject of several evaluation studies. These studies have demonstrated the importance of certain parameters associated with the B -spline least-squares fitting. We propose the substitution of a recently developed fast and robust B-spline approximation routine and a modified hierarchical optimization scheme for improved bias field correction over the original N3 algorithm. Similar to the N3 algorithm, we also make the source code, testing, and technical documentation of our contribution, which we denote as ??N4ITK,?? available to the public through the Insight Toolkit of the National Institutes of Health. Performance assessment is demonstrated using simulated data from the publicly available Brainweb database, hyperpolarized <sup>3</sup>He lung image data, and 9.4T postmortem hippocampus data.