Simultaneous segmentation and correspondence establishment for statistical shape models

Abstract

Statistical Shape Models have been proven to be valuable tools for segmenting anatomical structures of arbitrary topology. Being based on the statistical description of representative shapes, an initial segmentation is required - preferably done by an expert. For this purpose, mostly manual segmentation methods followed by a mesh generation step are employed. A prerequisite for generating the training data based on these segmentations is the establishment of correspondences between all training meshes. While existing approaches decouple the expert segmentation from the correspondence establishment step, we propose in this work a segmentation approach that simultaneously establishes the landmark correspondences needed for the subsequent generation of shape models. Our approach uses a reference segmentation given as a regular mesh. After an initial placement of this reference mesh, it is manually deformed in order to best match the boundaries of the considered anatomical structure. This deformation is coupled with a real time optimization that preserves point correspondences and thus ensures that a pair of landmark points in two different data sets represents the same anatomical feature. We applied our new method to different anatomical structures: vertebra of the spinal chord, kidney, and cardiac left ventricle. In order to perform a visual evaluation of the degree of correspondence between different data sets, we have developed well adapted visualization methods. From our tests we conclude that the expected correspondences are established during the manual mesh deformation. Furthermore, our approach considerably speeds up the shape model generation, since there is no need for an independent correspondence establishment step. Finally, it allows the creation of shape models of arbitrary topology and removes potential error sources of landmark and correspondence optimization algorithms needed so far.

Full text

Page 1

Simultaneous Segmentation and Correspondence
Establishment for Statistical Shape Models
Marius Erdt1, Matthias Kirschner2, and Stefan Wesarg2
1Fraunhofer Institute for Computer Graphics, Darmstadt, Germany,
Department of Cognitive Computing & Medical Imaging
marius.erdt@igd.fhg.de
2Technische Universit¨ at Darmstadt, Graphisch-Interaktive Systeme, Germany
matthias.kirschner@gris.tu-darmstadt.de,
stefan.wesarg@gris.tu-darmstadt.de
Abstract. Statistical Shape Models have been proven to be valuable
tools for segmenting anatomical structures of arbitrary topology. Being
based on the statistical description of representative shapes, an initial
segmentation is required – preferably done by an expert. For this pur-
pose, mostly manual segmentation methods followed by a mesh genera-
tion step are employed. A prerequisite for generating the training data
based on these segmentations is the establishment of correspondences be-
tween all training meshes. While existing approaches decouple the expert
segmentation from the correspondence establishment step, we propose in
this work a segmentation approach that simultaneously establishes the
landmark correspondences needed for the subsequent generation of shape
models.
Our approach uses a reference segmentation given as a regular mesh.
After an initial placement of this reference mesh, it is manually deformed
in order to best match the boundaries of the considered anatomical struc-
ture. This deformation is coupled with a real time optimization that pre-
serves point correspondences and thus ensures that a pair of landmark
points in two different data sets represents the same anatomical feature.
We applied our new method to different anatomical structures: ver-
tebra of the spinal chord, kidney, and cardiac left ventricle. In order to
perform a visual evaluation of the degree of correspondence between dif-
ferent data sets, we have developed well adapted visualization methods.
From our tests we conclude that the expected correspondences are estab-
lished during the manual mesh deformation. Furthermore, our approach
considerably speeds up the shape model generation, since there is no need
for an independent correspondence establishment step. Finally, it allows
the creation of shape models of arbitrary topology and removes potential
error sources of landmark and correspondence optimization algorithms
needed so far.
1Introduction
State-of-the-art clinical diagnosis, therapy planning, and intra-operative naviga-
tion is based on three-dimensional image data. For this, anatomical structures
N. Magnenat-Thalmann (Ed.): 3DPH 2009, LNCS 5903, pp. 25–35, 2009.
c ? Springer-Verlag Berlin Heidelberg 2009

Page 2

26 M. Erdt, M. Kirschner, and S. Wesarg
have to be extracted from the images. This segmentation can be done employ-
ing a wide range of techniques: region-based approaches, contour-oriented algo-
rithms, atlas-based approaches, and techniques incorporating prior knowledge
about the typical shape of these structures and their most likely variation. The
latter methods typically make use of statistical shape models (SSM) [1] that
have been proven to be a valuable tool for segmenting organs like the liver [6],
the heart [15], and the pelvic bones [12].
For deriving statistical information from a set of meshes created from a num-
ber of data sets, the so-called correspondence problem has to be solved. This
means, that all meshes must contain the same number of nodes (called land-
marks), and for two different meshes each node of data set A is required to have
a corresponding node in data set B, both representing the same anatomical
feature.
The first statistical shape models were constructed from landmarks which
were manually placed on training images [1]. This manual annotation process is
very time-consuming and regarded as intractable in 3D, due to the size and com-
plexity of the shapes. Therefore, many researchers focused on the development
of algorithms that establish correspondence automatically or semi-automatically.
Recent overviews of automatic correspondence algorithms can be found in [3,5].
In typical semi-automatic methods, a sparse set of manual landmarks is de-
fined which corresponds to predominant and unambiguously identifiable features.
Additional landmarks are then automatically placed in between, either equally
spaced according to the contour length in 2D [10] or by subdivision surfaces in
3D [13].
Beyond the tractability problem, the process of manual landmarking is often
criticized for suffering from inter- and intra-observer variability. However, this
does not imply that manually placed landmarks have lower quality than their
automatically determined counterparts, as it remains unclear whether the ob-
jective functions applied in automatic methods really measure ‘true’ correspon-
dence. Evaluation studies give an inconsistent picture: In the study of Styner
et al. [13], a semi-automatic method based on manually defined landmarks and
subdivision surfaces produced worse landmarks than optimization algorithms
based on DetCov [8] and MDL [2]. It is unclear whether the manually defined
landmarks or rather the automatic subdivision scheme accounts for the poor per-
formance of the semi-automatic method in this evaluation. On the other hand,
in a study from Ericsson and Karlsson [4], models learned from manually de-
fined landmarks performed better than those constructed from automatically
established landmarks. Though the latter study was restricted to 2D shapes and
therefore excluded all complications that occur in 3D, the results indicate that
manual landmarks may be better than their reputation.
(Semi-)automatic algorithms require that the training shapes are provided as
surfaces. In practice, these surfaces are reconstructed from segmentations, which
requires that either an expert delineates contours on training images manually
or that automatic segmentation algorithms are used. Manual delineation is time-
consuming and tedious, though by far easier than consistently placing landmarks.

Page 3

Simultaneous Segmentation and Correspondence Establishment for SSM27
Fig.1. Workflow of shape model generation. One triangulated reference mesh is created
for every organ. The reference mesh is deformed by the user and simultaneously opti-
mized to enforce correspondence of the deformed models with the reference shape. The
resulting surfaces are then used as training data input for the shape model generation.
If the segmentations are generated automatically, the resulting training shapes
are restricted by the accuracy of the applied segmentation algorithm.
While segmentation of training shapes and establishing correspondence are
treated independently in case of (semi-)automatic correspondence algorithms,
manual placement of landmarks integrates both aspects. Hence it is promising
to develop tools which support the user during placement of landmarks, thereby
making the process tractable.
In this work we present a method for simultaneous segmentation and point
correspondence establishment for SSMs. In our approach a reference mesh is
manually deformed and at the same time optimized in real time to preserve
point correspondence. The resulting meshes can be directly used for building a
shape model. In addition, SSMs of arbitrary topology can be easily constructed
using our method, whereas many automatic, parameterization-based methods
are either restricted to shapes of specific topology, like genus-0 surfaces [7], or
require an artificial decomposition of the training shapes into several patches
[12]. In the latter approach, correspondence is established independently on the
patches and the results are merged afterwards, which introduces discontinuities
at the cuts.
2 Methods
An overview of the shape model generation process is given in figure 1. The first
step is the construction of a polygonal reference organ model that can be taken
as a basis for all training data sets of the shape model. The next step is a user
guided segmentation. Here, the mesh is three dimensionally deformed by the user

Page 4

28 M. Erdt, M. Kirschner, and S. Wesarg
(a) (b)(c) (d)
Fig.2. (a) Generated mesh models for user guided adaptation: vertebra (top), kidney
(lower left) and cardiac left ventricle (lower right). (b) 3D mesh deformation using
a Gaussian weighting force. (c) Global shape preservation during lateral movement.
(d) Local weights on the kidney shape: a soft area which maps the vessel and ureter
connection region is embedded into a stiff capsule.
to match the organ boundaries in the data set. In order to ensure that points
in the reference mesh and the deformed mesh denote the same feature points
and therefore correspond to the same region, the mesh is globally optimized in
each deformation step. The results are deformed training meshes with regularly
distributed points that can be directly taken for shape model generation in the
last step.
2.1 Reference Mesh Construction
In order to create a segmentation by using manual mesh deformation, a reference
model is needed for every organ that can be adapted to the data set by the user.
This model should be represented by a regularly distributed point cloud with an
adequate number of points in order to ensure that all local organ features in the
current data set can be mapped to the shape model.
The organ models are constructed based on a clinically validated reference
segmentation — in our case taken from an organ atlas [14]. For simple organ
shapes like the left ventricle we uniformly sample points along the volume’s
main axis in order to create a regular point grid. In the case of complex organ
shapes, the binary segmentation is first resampled in order to remove the typical
staircase artifacts resulting from image reconstruction in CT or MRI. Secondly,
morphological closing and opening is applied to close holes inside the organ
(e.g. vessels that are not classified as organ tissue). Afterwards, the Marching
Cubes Algorithm generates a polygonal tessellation from the binary mask. Since
the points of the resulting mesh are in most cases not regularly distributed, a

Page 5

Simultaneous Segmentation and Correspondence Establishment for SSM29
Laplacian smoothing is applied iteratively until all polygons are of comparable
size. Fig. 2(a) shows the results of the model generation of vertebra, kidney and
cardiac left ventricle.
In order to preserve an anatomic correct shape during deformation and to pre-
vent the user from mapping smooth regions to areas of high curvature, we added
local weight constraints to the kidney model (compare fig. 2(d)). Here, the kidney
capsule is modeled with a 5 times higher rigidity than the area containing ureter
and vesselconnections. This leads to a more robust adaptation while being able to
map the well known regions of high frequency boundaries at the same time.
2.2 Model Deformation
User Guided Adaptation. The first step of the manual segmentation process
is the selection of an organ and the placement of the according model in the
data set by the user. After placement, the model can be scaled and rotated in
order to ease the adaptation and to roughly align important feature points of
the model to the underlying data (e.g. inferior and superior renal capsule, which
denote the lower and upper boundaries of the kidney capsule, respectively).
The subsequent step is a fine grained segmentation by directly deforming the
mesh. This is done by pulling the boundaries of the mesh towards the real image
boundaries in the three 2D standard views of medical imaging (axial, sagittal
and coronal image planes). The user driven force at a given point is propagated
to adjacent points using a 3D Gaussian Gσ(x,y,z) (compare fig. 2(b)). The
user can switch between three different scales of the standard deviation σ of Gσ.
Initially, it is suitable to select the highest value of σ, which results in a non-local
or stiff deformation of the mesh around the user movement vector (fig. 3(a)).
In order to adapt to areas of high curvature, σ may be lowered which results in
a softer deformation until only points in a vicinity are affected (fig. 3(b)). This
procedure is repeated in a couple of different slices using the standard views and
changing the value of σ until the mesh is properly fitted to the data (fig. 3(c)).
In order to keep areas of corresponding points matched during the deformation
process, lateral movement does not change the global shape of the organ, i.e. a
local deformation can be pulled back and forth on the surface (compare fig. 2(c)).
This also prevents self-intersection or folding of the surface.
Optimization. Relying only on the described mesh deformation would lead to
highly irregularly distributed point clouds after adaptation. Moreover, the qual-
ity of the resulting meshes would directly depend on the number of refinement
steps, since no position correction is applied.
In order to use the user guided segmentation meshes as training input for
the shape model generation, we optimize all point coordinates in real time such
that the global shape of the reference mesh as well as the point distances are
preserved.
Similar to [9], we define two energies Eshapeand Eforceto regularize the
mesh deformation. Eshapedenotes a shape preservation energy defined as