ADAPTIVE SEGMENTATION OF INTERNAL BRAIN STRUCTURES IN PATHOLOGICAL MR IMAGES DEPENDING ON TUMOR TYPES
ABSTRACT This paper introduces a novel methodology for the segmentation of internal brain structures in MRI volumes in the presence of a tumor. The proposed method relies on an initial segmentation of the tumor. Based on the tumor's type, a set of spatial relations between internal structures, remaining stable even in presence of the pathology, is established. Segmentation and recognition of surrounding anatomical structures are based on prior knowledge about their spatial arrangement. Segmentation results on tumors inducing small or large deformations are provided to illustrate the potential of the approach.
- SourceAvailable from: perso.telecom-paristech.fr[Show abstract] [Hide abstract]
ABSTRACT: Segmenting the heart in medical images is a challenging and important task for many applications. In particular, segmenting the heart in CT images is very useful for cardiology and oncological applications such as radiotherapy. Although the majority of methods in the literature are designed for ventricle segmentation, there is a real interest in segmenting the heart as a whole in this modality. In this paper, we address this problem and propose an automatic and robust method, based on anatomical knowledge about the heart, in particular its position with respect to the lungs. This knowledge is represented in a fuzzy formalism and it is used both to define a region of interest and to drive the evolution of a deformable model in order to segment the heart inside this region. The proposed method has been applied on non-contrast CT images and the obtained results have been compared to manual segmentations of the heart, showing the good accuracy and high robustness of our approach.Pattern Recognition. 01/2008;
- [Show abstract] [Hide abstract]
ABSTRACT: We propose a new general method for segmenting brain tumors in 3D magnetic resonance images. Our method is applicable to different types of tumors. First, the brain is segmented using a new approach, robust to the presence of tumors. Then a first tumor detection is performed, based on selecting asymmetric areas with respect to the approximate brain symmetry plane and fuzzy classification. Its result constitutes the initialization of a segmentation method based on a combination of a deformable model and spatial relations, leading to a precise segmentation of the tumors. Imprecision and variability are taken into account at all levels, using appropriate fuzzy models. The results obtained on different types of tumors have been evaluated by comparison with manual segmentations.Fuzzy Sets and Systems 05/2009; 160(10):1457-1473. · 1.75 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Sequential methods for knowledge-based recognition of structures re-quire to define in which order the structures have to be recognized. We propose to address this problem by integrating pre-attention mech-anisms, in the form of a saliency map, in the determination of the or-der. As pre-attention mechanisms extract knowledge from an image without object recognition in ad-vance and do not require any a pri-ori knowledge on the image, they provide useful knowledge for guid-ing object segmentation and recog-nition. Additionally, we make use of generic knowledge of the scene, expressed as spatial relations, since they play a crucial role in model-based image recognition and inter-pretation due to their stability com-pared to many other image appear-ance characteristics. Graphs are well adapted to represent this infor-mation, and finding an order then amounts to find a path in a graph. The proposed algorithms are applied on brain image understanding.01/2008;
ADAPTIVE SEGMENTATION OF INTERNAL BRAIN STRUCTURES IN PATHOLOGICAL
MR IMAGES DEPENDING ON TUMOR TYPES
Hassan Khotanlou(1), Jamal Atif(1), Elsa Angelini(1), Hugues Duffau(2), Isabelle Bloch(1)
(1) Ecole Nationale Sup´ erieure des T´ el´ ecommunications (GET - T´ el´ ecom Paris)
CNRS UMR 5141 LTCI, Paris, France. Isabelle.Bloch@enst.fr
(2) Department of Neurosurgery, Hˆ opital Gui de Chauliac, CHU de Montpellier, France.
This paper introduces a novel methodology for the segmenta-
tion of internal brain structures in MRI volumes in the pres-
ence of a tumor. The proposed method relies on an initial seg-
mentation of the tumor. Based on the tumor’s type, a set of
spatial relations between internal structures, remaining stable
even in presence of the pathology, is established. Segmen-
tation and recognition of surrounding anatomical structures
are based on prior knowledge about their spatial arrangement.
Segmentation results on tumors inducing small or large de-
formations are provided to illustrate the potential of the ap-
In brain oncology, especially when dealing with brain tumors,
it is desirable to have a descriptive human brain model that
as its location, its type (cf. WHO classification ), its shape,
its anatomo-functional positioning, as well as its influence
over the surrounding brain structures (through their spatial
relations for example). There is a large literature reporting
works on segmentation of either cerebral structures or tumors
but rarely both at the same time. This paper tries to fill this
gap, by addressing the problem of segmenting internal brain
structures in the presence of a tumor.
Few methods for segmentation of brain structures in the
on atlas registration. Tumor growth and tissue deformation
models are used to constrain atlas registration. For example, a
biomedical model using finite element method was proposed
in [2, 3] and a simpler model based on optical-flow algorithms
was introduced in . Recently Cuadra et al. and Pollo et
al. proposed a model based on radial tumor growth and us-
ing seeded atlas deformation. Nowinski and Belov  intro-
duced a tumor growth model based on geometric assumption
and Talairach space registration. Model computation is a time
consuming process and does not apply to all types of tumor.
This work has been partially funded by GET, ANR and INCA grants. J.
Atif is now with Universit´ e des Antilles et de la Guyane, Guyane, France.
on previous work from our group  introducing a frame-
work for the integration of spatial relations into a deformable
model, to segment normal brain structures on MRI data. Spa-
tial relations, such as directions and distances, were repre-
sented as fuzzy subsets of the image space and incorporated
into a deformable model as external forces. In this paper we
extend this framework to pathological cases, where the pres-
ence of a tumor may induce important alterations of the iconic
and morphometric characteristics of the surrounding struc-
tures. By understanding the spatial behavior of the tumor and
its incidence on the surrounding structures (induces small or
large deformations for example), we discuss the preservation
of some spatial relations used for the recognition and segmen-
2, while the proposed method for segmentation of internal
brain structures is detailed in Section 3, which is the core of
the paper. The approach has been applied on eight MRI data
sets. In Section 4, we present their quantitative evaluation on
2. SEGMENTATION OF BRAIN TUMORS
Since tumor segmentation is not the focus of this paper (nor
the comparison with other methods), we only briefly sum-
marize our method. We recently proposed an original tu-
mor segmentation approach that combines symmetry analysis
and a parametric deformable model . It does not require
user supervision, runs on standard 3D contrast-enhanced T1-
weighted MRI data, and does not make any assumption on
the tumor’s type. This method has been successfully applied
to more than 20 cases exhibiting different types of tumors and
is illustrated in Figure 1 for two different types.
3. ADAPTIVE SEGMENTATION OF INTERNAL
A deformable model constrained with spatial relations.
Our segmentation approach combines a deformable model
5881424406722/07/$20.00 ©2007 IEEEISBI 2007
tumors: (a) fully-enhanced tumor; (b) its segmentation; (c) non-
enhanced tumor (low grade glioma); (d) its segmentation.
Automated detection and segmentation of two types of
with spatial relations between brain structures . These spa-
tial relations are represented as fuzzy subsets of the image
space . Their integration in the evolution scheme of the
deformable model relies on the introduction of a new force
computed from a fuzzy set representing the fusion of spatial
relations of interest.
following usual dynamic force equation: γ∂X
Fext(X), where X is the deformable surface, Fintis the in-
ternal force that constrains the regularity of the surface and
Fextis the external force. The external force combines edge
and spatial relation information. It is defined as: Fext =
λFC+ νFR, where FCis a classical data term that drives
the model towards edges in the image, FRis a force derived
from spatial relations and λ and ν are weighting coefficients.
The role of FRis to force the deformable model to stay within
regions where specific spatial relations are fulfilled.
Several types of spatial relations are necessary to fully as-
sess the structure of a given scene. We consider two main
classes of spatial relations: topological relations, which in-
clude part-whole relations such as inclusion or exclusion, and
adjacency; and metric relations such as distances and orien-
tations. More complex relations can also be useful, such as
“between”, “around” or “along”. Fuzzy representations are
appropriate to cope with imprecision in images, in knowl-
edge description, and to define vague relations. Our previous
work in this domain was mainly based on fuzzy mathemati-
cal morphology, which allowed us to represent, in a unified
framework, various spatial relations .
In , several methods to compute FRfrom a fuzzy set
μRwere proposed. For instance, if μR(x) denotes the degree
of satisfaction of the fuzzy relation at point x (with μR(x) ∈
[0,1]), and supp(R) the support of μR(i.e. the set of points
with non-zero membership values), then we can derive the
following potential: PR(x) = 1 − μR(x) + dsupp(R)(x),
where dsupp(R)is the distance to the support of μR, used to
have non-zero force values outside the support. The force
FRassociated with the potential PRis derived as follows:
FR(x) = −(1 − μR(x))
ple of a spatial relation and its corresponding force.
The adaptation of this framework to pathological cases
requires addressing the fundamental question: given a pathol-
ogy, what kinds of spatial relations do remain consistent, with
∂t= Fint(X) +
?∇PR(x)?. Figure 2 shows an exam-
Fig. 2. (a) Fuzzy subset μR representing the spatial relation “out-
side the third ventricle and below the right lateral ventricle” (highest
grey level values correspond to regions where the spatial relation is
best satisfied). (b) Force FRcomputed from μR.
respect to the set of relevant relations defined for normal cases
in ? The answer depends on the type of tumor.
Tumor classification. The two main brain tumor classifica-
tions used in clinical neurology are based on radiometric ap-
pearances  or degrees of malignancy (cf. WHO criteria
). The first one, also known as the Sainte-Anne classifica-
tion, is partly based on the contrast enhancement of tumors,
while the WHO classification is exclusively based on histo-
logical criteria. As an alternative, we consider in this work a
classification of brain tumors according to their spatial char-
acteristics and the nature of the potential alterations of the
brain structural organization they induce (location, infiltra-
tion, destruction, edema...). We distinguish two main types:
• Small deforming tumors (SD). In this category we in-
clude tumors that are principally infiltrating without necro-
sis and small necrotic tumors. The whole structural brain
arrangement is not significantly altered (Figures 1 (c) and 4
(a)). A further distinction is made, into subcortical (SD-SC)
or peripheral (SD-P) tumors, according to their distance to
the inter-hemispheric plane and depending on whether they
involve deep grey nuclei or not.
• Large deforming tumors (LD). Tumors and lesions in
this category significantly alter the surrounding brain struc-
ture arrangement. These tumors are necrotic and can be sur-
rounded by edema (Figure 3 (a)).
Identifying the type of tumor is based on its segmentation
on T1-weighted data.
Tumor-specific spatial relations. Some spatial relations are
more stable than others in the presence of a tumor. Intuitively,
topological relations imply less instability than metric ones.
For example, an adjacency relation can be preserved even if
large deformations are considered in a given structural orga-
nization; on the contrary metric relations, even if formulated
with fuzzy sets, are prone to significant modifications in the
case of large tumors and should therefore be avoided or ma-
nipulated with great care, by introducing more flexibility in
Reasoning about distances requires to take into account
the granularity level of the relation expression. For example
the distance predicates “far from” and “near” are naturally
more vague than the predicate “at a distance of about 1cm”,
which makes them more stable. In the case of tumor-specific
spatial relations, if the tumor is large deforming, only rela-
tions such as “far from” and “near” are retained. The choice
of cancelling or maintaining a spatial relation in the pres-
ence of a tumor is first motivated by clinical considerations,
namely the location, size and type of the tumor. Moreover,
in , we designed a computational framework for learn-
ing spatial relations stability, from a database constituted of
healthy and pathological MRI, where the main anatomical
structures were manually segmented. The degree of stabil-
ity was inferred from the comparison between learned spatial
relations for pathological cases and for healthy ones.
Table 1 summarizes our current list of tumor-based spatial
in the segmentation of the caudate nuclei in cases of LD and
SD tumors. In these examples, tumors and ventricles are first
segmented. Based on the tumor’s type, specific relations to
the ventricles are modeled and guide the recognition of the
Large deforming (LD)
Distance (far, near)
Distance (far, near)
Small deforming (SD) Peripheral
Table 1. Spatial relations for internal brain structures depending on
the tumor’s type.
Fig. 3. Case of a LD tumor. (a) Segmentation of the ventricles. (b)
Spatial relation “near the ventricles”, used in conjunction with di-
rectional relations for segmenting the caudate nuclei. (c) Segmented
structures close to the ventricles.
Fig. 4. Case of a SD-P tumor. (a) Approximate symmetry plane
(useful in case of SD to process each hemisphere separately and de-
fine relationswith respect to it). (b) Fusion ofspatial relations“tothe
left of the left ventricle” and “near the left ventricle”. (c) Segmented
structures on an axial slice. (d) A coronal slice.
ets of various origins and types. We illustrate the results on
four cases, for which manual segmentation of several struc-
tures was available, and which exhibit tumors with different
shapes, locations, sizes, intensities and contrasts (the results
on the other cases are qualitatively similar). Evaluation of
the segmentation results was performed through quantitative
comparisonswithmanual segmentations, usingtheerrormea-
sures proposed in . Let us denote by A the segmented
object obtained manually and B the object segmented by the
proposed method. We compute: (i) the overlap:
the Hausdorff distance between A and B (very severe eval-
uation), (iii) the average distance between the surfaces of A
and B. Segmentation results are illustrated in Figure 5 and
quantitative evaluations are provided in Table 2, showing high
accuracy. The voxel size is typically 1×1×1.3 mm3, so that
the average error is less than one voxel. The Hausdorff dis-
tance represents the error for the worst point, which explains
its higher values. For overlap measures, values above 70%
are satisfactory . Although the first case exhibits strong
deformations of the normal structures, the results are similar
to those obtained with less deformed objects, illustrating the
robustness of the adaptive approach.
Table 2. Segmentation evaluation for the caudate nuclei, the lateral
ventricles and the tumor on four 3D MR datasets.
5. DISCUSSION AND CONCLUSION
knowledge on tumoral physiology in a new and original way.
ematical models to quantitatively describe the growth rates of
gliomas visualized radiologically [15, 16]. The model in 
takes into account the two major biological phenomena un-
derlying the growth of gliomas at the cellular scale: prolifer-
ation and migration. Initially, this model was suggested for
high-grade gliomas. Most of these anaplastic tumors have an
important proliferation index, inducing a mass effect on the
normal brain structures, especially in cases of large space-
occupying lesions. Thus, internal cerebral structures can be
Fig. 5. Segmentation results. (a) Original MRI data. (b) Manual
segmentation. (c) Segmentation with the proposed method. (d) Su-
perimposition of results in (c) on MRI data (From top to bottom,
tumors are LD, SD-SC, SD-P and SD-SC (cf. Table 1).
distorted, with a preservation of their spatial relations despite
mechanical deformations. In the event of necrosis, very fre-
quent in WHO grade IV gliomas (glioblastomas), it is pos-
sible that normal brain tissue is destroyed and not only dis-
torted, eliciting neurological deficit: in these cases, topol-
ogy may be modified. More recently, the same biomathe-
matical model has also been applied to the WHO grade II
gliomas, showing linear growth of mean tumor diameter 
and anisotropic migration along white matter tract . For
these infiltrating tumors, there are no necrosis (i.e. no brain
destruction), and the mass effect is most of the time very
limited. Therefore, spatial relations of the internal cerebral
structures (ventricles, deep grey nuclei and white matter bun-
dles) are preserved, whatever the size and the location of the
low-grade gliomas. Finally, it is worth noting that in addi-
tion to the natural history of the tumor, the treatment can have
an impact on the segmentation of the cerebral structures and
the tumor (thus their relations). This is especially true for
surgery, which may induce brain deformation and changes on
the functional anatomy via the resection of cerebral structures
. The paradigm of the proposed method should be able
to handle such drastic modifications of the cerebral anatomy.
Additional work, focusing on a quantitative analysis of the
consistency of spatial relations over large datasets with dif-
ferent tumor types and behaviors is ongoing, in order to refine
and complete the list of tumor-based spatial relations for each
individual internal brain structure to be segmented.
 P. Kleihues and W. Cavenee, “(eds.) Pathology and genetics of tumours
of the nervous system,” USA: WHO publications Center, 2000.
 S.K. Kyriacou and C. Davatzikos, “Nonlinear elastic registration of
brain images with tumor pathology using a biomechanical model,”
IEEE Trans. on Medical Imaging, vol. 18, no. 7, pp. 580–592, 1999.
 A. Mohamed, D. Shen, and C. Davatzikos, “Deformable registration of
brain tumor images via a statistical model of tumor-induced deforma-
tion,” in MICCAI, Oct. 2005, pp. 263–270.
 M. Dawant, S.L. Hartmann, Shiyan Pan, and S. Gadamsetty, “Brain
atlas deformation in the presence of small and large space-occupying
tumors,” Computer Aided Surgery, vol. 7, pp. 1–10, 2002.
 M. Bach Cuadra, M. De Craene, V. Duay, B. Macq, C. Pollo, and J.-
Ph. Thiran, “Dense deformation field estimation for atlas-based seg-
mentation of pathological MR brain images,” Computer Methods and
Programs in Biomedicine, vol. 84, no. 2-3, pp. 66–75, 2006.
 C. Pollo, M. Bach Cuadra, O. Cuisenaire, J. Villemure, and J. Ph. Thi-
lesion,” Neuroimage, vol. 24, no. 4, pp. 990–996, February 2005.
 W.L. Nowinski and D. Belov, “Toward atlas-assisted automatic inter-
pretation of MRI morphological brain scans in the presence of tumor,”
Acad. Radiol., vol. 12, no. 8, pp. 1049–1057, 2005.
 O. Colliot, O. Camara, and I. Bloch, “Integration of Fuzzy Spatial
Relations in Deformable Models - Application to Brain MRI Segmen-
tation,” Pattern Recognition, vol. 39, pp. 1401–1414, 2006.
 H. Khotanlou, O. Colliot, and I. Bloch,
Segmentation using Symmetry Analysis and Deformable Models,” in
ICAPR, Kolkata, India, jan 2007, pp. 198–202.
 I. Bloch, “Fuzzy Spatial Relationships for Image Processing and Inter-
pretation: A Review,” Image and Vision Computing, vol. 23, no. 2, pp.
 C. Daumas-Duport, “Histological grading of gliomas,” Current Opin-
ion in Neurology and neurosurgery, vol. 5, no. 6, pp. 924–931, 1992.
 J. Atif, C. Hudelot, G. Fouquier, I. Bloch, and E. Angelini, “From
Generic Knowledge to Specific Reasoning for Medical Image Inter-
pretation using Graph-based Representations,” in IJCAI, Hyderabad,
India, jan 2007, pp. 224–229.
 G. Gerig, M. Jomier, and M. Chakos, “VALMET: a new validation
tool for assessing and improving 3D object segmentation,” in MICCAI,
2001, vol. 2208, pp. 516–523.
 A.P. Zijdenbos, B.M. Dawant, Richard A. Margolin, and Andrew C.
Palmer, “Morphometric analysis of white matter lesions in MR images:
Method and validation,” IEEE Trans. on Medical Imaging, vol. 13, no.
4, pp. 716–724, 1994.
 K.R. Swanson, C. Bridge, J.D. Murray, and E.C. Alvord, “Virtual and
real brain tumors: using mathematical modeling to quantify glioma
growth and invasion,” J Neurol Sci, vol. 216, pp. 1–10, 2003.
 O. Clatz, M. Sermesant, P-Y. Bondiau, H. Delingette, S. K. Warfield,
G. Malandain, and N. Ayache, “Realistic simulation of the 3D growth
of brain tumors in MR images coupling diffusion with mass effect,”
IEEE Trans. on Medical Imaging, vol. 24, no. 10, pp. 1334–1346, 2005.
 E. Mandonnet, J.Y. Delattre, M.L. Tanguy, K.R. Swanson, A.F. Car-
pentier, H. Duffau, Ph. Cornu, R. Van Effenterre, E.C. Alvord, and
L. Capelle, “Continuous growth of mean tumor diameter in a subset
of grade II gliomas,” Ann Neurol, vol. 53, pp. 524–528, 2003.
 S. Jbabdi, E. Mandonnet, H. Duffau, L. Capelle, K.R. Swanson,
M. Pelegrini-Issac, R. Guillevin, and H. Benali,
anisotropic growth of low-grade gliomas using diffusion tensor imag-
ing,” Magn Reson Med, vol. 54, pp. 616–624, 2005.
 H. Duffau,“Lessons from brain mapping in surgery for low-grade
glioma: insights into associations between tumour and brain plastic-
ity,” Lancet Neurol, vol. 4, pp. 476–486, 2005.
“Automatic Brain Tumor