Intuitive visualization for convolutional neural networks detecting brain diseases in MRI scans

Abstract

Introduction:
• Although deep learning approaches achieve high diagnostic accuracy to automatically detect neurodegenerative diseases-such as Alzheimer's disease-based on MRI and PET, they are currently not part of clinically applied diagnostic systems.
• The main reason for this lack of clinical use is the shortcoming in proper methods for model comprehensibility and interpretability for clinical users.
Aims:
i. Development of a convolutional neural network (CNN) model to achieve competitive diagnostic accuracy for detecting Alzheimer's disease in patients with dementia and mild cognitive impairment (MCI).
ii. Intuitive visualization to aid model comprehensibility and clinical utility using class activation mapping approaches to highlight contributing brain regions.

Methods:
• MRI data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) (Tab.1) were used for model training by applying a six-fold cross-validation scheme.
• Only patients with positive amyloid-β biomarker and controls with negative finding were included to improve the diagnostic confidence of the training sample.
• Twelve coronal slices covering the hippocampus area were selected, corrected for effects of age and gender using linear regression, and fed into the CNN model as separate channels (Fig.1).
• MRI data from the DZNE-Longitudinal Cognitive Impairment and Dementia Study (DELCODE) (Tab.1) were used as independent validation set.

Results:
• Area under the curve and diagnostic accuracy are given in Fig.2.
• Group mean CNN activation maps indicate hippocampal areas as most informative for the model (Fig.3 left).
• Individual subject's activation maps show more distributed cortical and subcortical regions to contribute to the model's decision (Fig.3 right).

Conclusion:
• Results applying 2D convolutional layers provide high diagnostic accuracy and promising results for the visualization of individual subject's CNN activity maps.
• Extension of the CNN toolbox for 3D convolutional layers recently became available in MATLAB R2019a and will provide activation maps with higher spatial information.

Full text

Intuitive visualization for convolutional neural networks
detecting brain diseases in MRI scans
Eman N. Marzban1,2, Stefan Teipel1,3, Slawek Altenstein4,5, Claudia Bartels6,7, Frederic Brosseron8,9, Martina Buchmann10,11, Katharina Buerger12,13, Cihan Catak13,
Laura Dobisch14, Klaus Fließbach8,9, Michael T. Heneka8,9, Enise Incesoy15, Daniel Janowitz13, Pascal Kalbhen9, Ingo Kilimann1,3, Christoph Laske10,11, Siyao Li15,
Dix Meiberth8,16, Felix Menne4,15, Coraline D. Metzger4,17,18, Oliver Peters4,15, Alexandra Polcher8, Josef Priller4,5, Janna Rudolph8, Anja Schneider8,9, Annika Spottke8,19,
Eike J. Spruth4,5, Michael Wagner8,9, Jens Wiltfang6,7, Emrah Düzel14,17, Frank Jessen8,16, Martin Dyrba1,
the Alzheimer’s Disease Neuroimaging Initiative, and the DELCODE study group
1German Center for Neurodegenerative Diseases (DZNE), Site Rostock/Greifswald, Rostock, Germany; 2Biomedical Engineering and Systems Department, Faculty of Engineering, Cairo University, Giza, Egypt; 3Clinic for Psychosomatics and Psychotherapeutic Medicine,
University Medicine Rostock, Rostock, Germany; 4German Center for Neurodegenerative Diseases (DZNE), Berlin, Germany; 5Department of Psychiatry and Psychotherapy, Charité, Berlin, Germany; 6German Center for Neurodegenerative Diseases (DZNE), Goettingen,
Germany; 7Department of Psychiatry and Psychotherapy, University Medical Center Goettingen, University of Goettingen, Goettingen, Germany; 8German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany; 9Department of Neurodegenerative Diseases and
Geriatric Psychiatry, University Hospital Bonn, Bonn, Germany; 10 German Center for Neurodegenerative Diseases (DZNE), Tübingen, Germany; 11 Section for Dementia Research, Hertie Institute for Clinical Brain Research and Department of Psychiatry and Psychotherapy,
University of Tübingen, Tübingen, Germany; 12 German Center for Neurodegenerative Diseases (DZNE), Munich, Germany; 13 Institute for Stroke and Dementia Research (ISD), University Hospital, LMU Munich, Munich, Germany; 14 German Center for Neurodegenerative
Diseases (DZNE), Magdeburg, Germany; 15 Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Institute of Psychiatry and Psychotherapy, Berlin, Germany; 16 Department of
Psychiatry, University of Cologne, Medical Faculty, Cologne, Germany; 17 Institute of Cognitive Neurology and Dementia Research (IKND), Otto-von-Guericke University, Magdeburg, Germany; 18 Department of Psychiatry and Psychotherapy, Otto-von-Guericke University,
Magdeburg, Germany; 19 Department of Neurology, University of Bonn, Bonn, Germany
• Although deep learning approaches achieve high diagnostic accuracy to
automatically detect neurodegenerative diseases -such as Alzheimer’s disease-
based on MRI and PET, they are currently not part of clinically applied diagnostic
systems.
• The main reason for this lack of clinical use is the shortcoming in proper methods
for model comprehensibility and interpretability for clinical users.
• MRI data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (Tab.1) were
used for model training by applying a six-fold cross-validation scheme.
• Only patients with positive amyloid-βbiomarker and controls with negative finding
were included to improve the diagnostic confidence of the training sample.
• Twelve coronal slices covering the hippocampus area were selected, corrected for
effects of age and gender using linear regression, and fed into the CNN model as
separate channels (Fig.1).
• MRI data from the DZNE – Longitudinal Cognitive Impairment and Dementia Study
(DELCODE) (Tab.1) were used as independent validation set.
• Area under the curve and diagnostic accuracy are given in Fig.2.
• Group mean CNN activation maps indicate hippocampal areas as most informative
for the model (Fig.3 left).
• Individual subject’s activation maps show more distributed cortical and subcortical
regions to contribute to the model’s decision (Fig.3 right).
• Results applying 2D convolutional layers provide high diagnostic accuracy and
promising results for the visualization of individual subject’s CNN activity maps.
• Extension of the CNN toolbox for 3D convolutional layers recently became available
in MATLAB R2019a and will provide activation maps with higher spatial information.
i. Development of a convolutional neural network (CNN) model to achieve competitive
diagnostic accuracy for detecting Alzheimer’s disease in patients with dementia and
mild cognitive impairment (MCI).
ii.Intuitive visualization to aid model comprehensibility and clinical utility using class
activation mapping approaches to highlight contributing brain regions.
Methods
Results
Introduction Aim of the study:
Discussion and conclusion
References
1. Hendricks, et al. Generating Visual Explanations. ECCV (2016).
Table 1 Sample characteristics.
This project was supported by the German Academic Exchange
Service (DAAD) and grants from the University Medicine Rostock.
Acknowledgement
Figure 1 Input data (top) and deep learning model layout (bottom). Twelve coronal slices covering the hippocampus enter the model as separate input channels.
Binarization
Max Pooling 2x2
ReLU
Dropout 50%
2D Convolution
5 3x3x12
Input 242x242x12
2D Convolution
5 3x3x5
Binarization
Max Pooling 2x2
ReLU
Dropout 50%
Fully connected
Softmax 2 classes
Likelihood
MCI/AD vs. normal
Male/Female Age Years of
education MMSE
ADNI (N=294)
HC (n=126) 65/61(48.4%) 72.7±6.4 16.8±2.5 29.1±1.2
MCI (n=93) 50/43(46.2%) 72.3±7.4 16.4±2.8 27.1±1.9
AD (n=75) 40/35(46.7%) 75.0±8.5 15.6±2.8 22.9±2.1
DELCODE (N=332)
HC (n=182) 75/107(58.8%) 69.0±5.3 14.8±2.7 29.4±0.9
MCI (n=89) 54/35(39.3%) 72.3±5.1 14.0±3.0 28.0±1.7
AD (n=61) 27/34(55.7%) 74.0±6.4 13.3±3.3 23.5±3.3
Training &
cross-
validation set
Independent
validation set
Abbreviations: AD – Alzheimer’s dementia, MCI – mild cognitive impairment, HC – healthy controls,
MMSE – mini mental status examination.
Mean HC Mean MCI Mean AD
Figure 3 Group average (left) and individual subject‘s activation maps (right) as well as likelihood scores for Alzheimer‘s disease returned by the CNN model.
DAAD
AD
HC
HC1(2%) HC2(3%) MCI1(86%) MCI2(87%) MCI3(69%) AD1(98%) AD2(99%)
Contact: eman.marzban@eng1.cu.edu.eg & martin.dyrba@dzne.de
• Prospectively, we will focus on generating textual explanations from the input
images [1] to enhance model interpretability and clinical utility.
Covariate cleaning
(linear regression
with age + gender)
Image upscaling
0.97 0.88 0.95 0.94
0.83 0.77
0.94
0.85 0.92 0.91
0.82 0.78
0.50
0.60
0.70
0.80
0.90
1.00
ADNI AD-Aβ+
vs. HC-Aβ–ADNI MCI-Aβ+
vs. HC-Aβ–DELCODE AD-
Aβ+ vs. HC-Aβ–DELCODE AD
vs. HC DELCODE MCI-
Aβ+ vs. HC-Aβ–DELCODE MCI
vs. HC
AUC
Accuracy
Figure 2 Area under the curve (AUC) and diagnostic accuracy.
Abbreviations: AD – Alzheimer’s dementia, MCI – mild cognitive impairment, HC – healthy controls, Aβ+/- – amyloid-β
biomarker positive/negative.