Development of an MRI rating scale for multiple brain regions: comparison with volumetrics and with voxel-based morphometry

Abstract

IntroductionWe aimed to devise a rating method for key frontal and temporal brain regions validated against quantitative volumetric methods
and applicable to a range of dementia syndromes.

MethodsFour standardised coronal MR images from 36 subjects encompassing controls and cases with Alzheimer’s disease (AD) and frontotemporal
dementia (FTD) were used. After initial pilot studies, 15 regions produced good intra- and inter-rater reliability. We then
validated the ratings against manual volumetry and voxel-based morphometry (VBM) and compared ratings across the subject groups.

ResultsValidation against both manual volumetry (for both frontal and temporal lobes), and against whole brain VBM, showed good correlation
with visual ratings for the majority of the brain regions. Comparison of rating scores across disease groups showed involvement
of the anterior fusiform gyrus, anterior hippocampus and temporal pole in semantic dementia, while anterior cingulate and
orbitofrontal regions were involved in behavioural variant FTD.

ConclusionThis simple visual rating can be used as an alternative to highly technical methods of quantification, and may be superior
when dealing with single cases or small groups.

Full text

Page 1

DIAGNOSTIC NEURORADIOLOGY
Development of an MRI rating scale for multiple brain
regions: comparison with volumetrics and with voxel-based
morphometry
R. Rhys Davies & Victoria L. Scahill & Andrew Graham &
Guy B. Williams & Kim S. Graham & John R. Hodges
Received: 25 November 2008 /Accepted: 4 March 2009 /Published online: 24 March 2009
# Springer-Verlag 2009
Abstract
Introduction We aimed to devise a rating method for key
frontal and temporal brain regions validated against
quantitative volumetric methods and applicable to a range
of dementia syndromes.
Methods Four standardised coronal MR images from 36
subjects encompassing controls and cases with Alzheimer’s
disease (AD) and frontotemporal dementia (FTD) were used.
After initial pilot studies, 15 regions produced good intra- and
inter-rater reliability. We then validated the ratings against
manual volumetry and voxel-based morphometry (VBM) and
compared ratings across the subject groups.
Results Validation against both manual volumetry (for both
frontal and temporal lobes), and against whole brain VBM,
showed good correlation with visual ratings for the majority
of the brain regions. Comparison of rating scores across
disease groups showed involvement of the anterior fusiform
gyrus, anterior hippocampus and temporal pole in semantic
dementia, while anterior cingulate and orbitofrontal regions
were involved in behavioural variant FTD.
Conclusion This simple visual rating can be used as an
alternative to highly technical methods of quantification,
and may be superior when dealing with single cases or
small groups.
Keywords Visualratingscale.MRI.VBM
Introduction
Visual rating of MR scans can provide anatomical data for
clinical diagnosis or for research [1–3]. Advantages of the
rating approach include speed, a flexibility and clinical
applicability. Other approaches, however, provide more
precise quantification. For instance, MRI volumetry, the
‘gold standard’ of structural imaging, typically involves
tracing boundaries by hand [3–5]. Various automated
techniques devised in recent years, notably voxel-based
morphometry (VBM), lend objectivity to brain measure-
ment and permit assessment of the whole brain [6, 7].
Disadvantages of these methods, however, are that they
require sophisticated and time-consuming post-imaging
analysis. They also stipulate specific scanning protocols.
By contrast, clinical scans are often available from cases
where instructive behavioural data have been collected but
where further scanning is not possible. Valuable neuropsy-
chological data may also be held by workers without access
to automated imaging resources who may wish to perform
studies involving structure–function correlation.
A number of MRI rating scales have been devised but
have focused on certain lesion types (e.g. infarcts [2, 8]) or
on individual regions, e.g. hippocampus [1, 9], which limits
their applicability. There is scope for extension of this
approach as illustrated by Galton et al. who successfully
established a method for assessing atrophy of temporal lobe
Neuroradiology (2008) 51:491–503
DOI 10.1007/s00234-009-0521-z
R. R. Davies:A. Graham:G. B. Williams:J. R. Hodges
Department of Clinical Neurosciences, University of Cambridge,
R3 Box 83, Addenbrooke’s Hospital,
Cambridge CB2 2QQ, UK
V. L. Scahill:A. Graham:K. S. Graham:J. R. Hodges
MRC Cognition and Brain Sciences Unit, Cambridge and Wales
Institute of Cognitive Neuroscience, School of Psychology,
Cardiff University,
Cardiff CB2 7EF, UK
J. R. Hodges (*)
Cognitive Neurology, Prince of Wales Medical Research Institute,
Barker Street Randwick,
Sydney, NSW 2031, Australia
e-mail: j.hodges@powmri.edu.au

Page 2

sub-regions [3]. In the light of recent insights concerning
localisation of brain function and distribution of atrophy in
neurodegenerative disease [4, 10], we wanted to develop a
more extensive rating scale which could be readily applied
to good quality clinical MRI images. We also intended the
scheme to encompass brain abnormalities other than
neurodegeneration, including post-operative or post-
encephalitic appearances, where a given brain structure
may be completely destroyed.
The primary aim of this study was to devise a reliable
rating scheme focusing on temporal and frontal lobe
regions important in diagnosis, in localisation of function,
or both. Secondly, because pilot studies were conducted
over many regions, we were able to gauge the limits of
rating as an approach. Thirdly, as the method was
developed using scans from patients with Alzheimer’s
disease (AD), behavioural variant FTD (bvFTD) and
semantic dementia (SD), we were able to draw inferences
about distribution of atrophy in those syndromes. Finally,
we examined the relationship between rating and the
automated structural imaging method of VBM. This
included using VBM for the validation of rating scores
and, also, a comparison of rating data with VBM analyses
across the neurodegenerative disease groups.
Materials and methods
Development of rating method
The following candidate regions were selected: basal
ganglia, orbitofrontal, dorsolateral frontal, ventrolateral
frontal, cingulated (anterior and posterior), temporal pole,
amygdala, hippocampus (anterior, mid and posterior),
parahippocampal gyrus (anterior, mid and posterior),
banks of the collateral sulcus, fusiform gyrus (anterior,
mid and posterior), lateral temporal, insula, superior
temporal, posterior temporal and inferior parietal lobule.
Landmarks for the regions were taken from standard
reference works [11–13]. To maximise the efficiency of
the scheme, the number of images used in the protocol
was limited to four easily identifiable T1 coronal scan
slices. All regions of interest, apart from the amygdala,
could be viewed in one of the four slices; the amygdala
was consequently removed from the protocol. In order to
maximise applicability, scan slice thickness and brain
orientation were not specified beyond the fact that the
slices must recognisably be in the coronal plane (perpen-
dicular to the anterior commissure–posterior commissure
axis, ±10°).
The four slices defined by the protocol were, in antero-
posterior sequence: (I) the most anterior slice in which the
internal capsule may be seen in the corpus striatum; (II) the
most anterior slice in which the medial opening of the
intralimbic gyrus is visible; (III) the slice midway between
II and IV, at the same antero-posterior level as the lateral
genicluate nucleus; (IV) the most anterior slice in which the
crus of the fornix may be seen in continuity alongside the
pulvinar (Fig. 1). The slices were determined individually
for each hemisphere to minimise the confounding effects of
variation in brain orientation.
Initial pilot studies showed that rating was not viable for
eight of the initial candidate regions because of a high degree
of variability among controls or local degradation of image
quality. These were: slice I—dorsolateral and ventrolateral
frontal cortex (later combined to give the large ‘lateral frontal
cortex’ region); slice III—mid-parahippocampal gyrus, mid-
fusiform gyrus; slice IV—posterior parahippocampal, poste-
rior fusiform, inferior parietal lobule, posterior cingulate
cortex. The final selection of 15 regions is listed in Table 1
(see also Fig. 1).
The rating scheme encompasses four stages of atrophy
[1–4] with 0 indicating no atrophy and 4 being most
abnormal. Rating 2 indicates that a structure is undoubtedly
abnormal while rating 1 implies borderline appearances that
are not categorically abnormal. In the cases of the basal
ganglia, hippocampus and temporal pole, the absence of
discernible tissue is rated 4. The presence of only skeletal
remnants of white matter without identifiable grey qualifies
for a rating of 4 for the remaining cortical structures. A
rating of 3 indicates severe atrophy falling short of criteria
for 4. For consistency, appearances seeming to fall between
two levels were always given the higher rating.
For certain regional ratings, specific criteria were
adopted. The basal ganglia rating criteria were: (0) convex
contour of basal ganglia with normal-sized ventricle; (1)
prominence of the lateral ventricle but contour remaining
convex; (2) contour of basal ganglia flattened; (3) concave
basal ganglia; and (4) basal ganglia no longer discernible.
The anterior hippocampal criteria were: (0) no significant
cerebrospinal fluid (CSF) spaces visible; (1) mild promi-
nence of CSF but hippocampal structure essentially normal;
(2) convolutions of anterior hippocampus clearly separated
by CSF; (3) remnant of hippocampus at the medial tip of
the temporal horn; and (4) hippocampus no longer
discernible. The mid-hippocampal criteria were: (0) no
more than a small crescent of CSF visible; (1) mild
prominence of CSF but hippocampal structure essentially
normal; (2) temporal horn of ventricle taking up larger area
on slice than hippocampus; (3) remnant of hippocampus at
the medial tip of the temporal horn; and (4) hippocampus
no longer discernible. The posterior hippocampal criteria
were: (0) no significant CSF space between the fornix and
the retrosplenial cortex; (1) prominence of CSF but
hippocampal structure essentially normal; (2) CSF taking
up larger area under fornix than the hippocampus; (3)
492Neuroradiology (2008) 51:491–503

Page 3

remnant of hippocampus at the base of the fornix; and (4)
hippocampus no longer discernible.
A key source of reference during the rating process was
an array of pre-rated images with a guide to the target
regions for rating (Fig. 1) and illustrations of levels of
atrophy taken from patients and controls selected to
encompass the whole range of pathologies likely to be
encountered by clinicians using the scale. The pamphlet
images were consulted during each rating session.
Evaluation of rating method
Methodological analyses were based on the rating of 36
MR scans. These included controls (n=8, age=63.8±6.0),
Alzheimer’s disease (AD) cases (n=8, age=64.9±4.6) and
frontotemporal dementia (FTD) cases [14, 15] (n=20;
semantic dementia n=9, age=61.1±7.8; behavioural vari-
ant FTD n=11, age=57.8±5.6). The scans selected had all
been used in previous volumetric studies, allowing compar-
Fig. 1 Reference pamphlet sli-
ces 1 to 4. Guide for identifying
regions to be rated (upper left
hand corner) and images for
comparison while rating show-
ing range of severities (0–4) for
each rated region. Slice 1: temp
(ant) anterior temporal pole; b
ganglia basal ganglia; orbital fr
orbitofrontal cortex; dorsal fr
dorsolateral frontal cortex;
cingulate anterior cingulate cor-
tex. Slice 2: HPC (ant) anterior
hippocampus; ERC entorhinal
cortex (anterior parahippocam-
pal gyrus); PRC perirhinal cor-
tex; fus (ant) anterior fusiform
gyrus; temp (lat) lateral temporal
region; insula. Slice 3: HPC
(mid) mid-hippocampus; temp
(sup) superior temporal gyrus.
Slice 4: post HPC posterior
hippocampus; 37 area 37
Neuroradiology (2008) 51:491–503493

Page 4

isons across data-sets [3, 4, 16]. All scans were also
available in electronic form (necessary for VBM analysis).
Agreement analyses
Agreement analyses were based on assessment of all scans
by three independent raters blinded to clinical details and,
after further blinding, repeat rating by one of the three (rater
1, RD), a behavioural neurologist, with extensive experi-
ence of reviewing scans. Rater 2 (VS), a neuropsychologist,
was involved with selecting the regions for rating and the
preparation of reference material. Rater 3(AG), a trained
neurologist, had not been involved with the initial phase of
devising the scheme.
The rating of 36 scans yielded 72 measurements in total
for each region (ratings for left and right for each of the 36
Fig. 1 (continued)
494Neuroradiology (2008) 51:491–503

Page 5

cases). Agreement between raters was measured overall,
by region, and by subject group, using Cohen’s kappa
statistic and the intra-class correlation coefficient. For
each pair of raters, three measures of agreement were
calculated: the raw proportion of ratings in absolute
agreement and kappa, unweighted and weighted. The
weighted kappa statistic is more appropriate than the
unweighted as discrepancies carry greater weight if raters
have disagreed by a greater margin. The intraclass
correlation coefficient (ICC), unlike kappa, permits simul-
taneous analysis of agreement across all raters but has the
disadvantage of being influenced more by variance in the
sample assessed (more homogeneous samples will yield
lower ICCs for the same measure). Scoring from the two
sessions undertaken by rater 1 were compared in a similar
manner to the inter-rater data.
Fig. 1 (continued)
Neuroradiology (2008) 51:491–503495