1286 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
of Cervical Carcinoma Invasion Fronts From
Histological Serial Sections
Ulf-Dietrich Braumann, Member, IEEE, Jens-Peer Kuska, Member, IEEE, Jens Einenkel, Lars-Christian Horn,
Markus Löffler, and Michael Höckel
Abstract—The analysis of the three-dimensional (3-D) structure
of tumoral invasion fronts of carcinoma of the uterine cervix is
the prerequisite for understanding their architectural-functional
relationship. The variation range of the invasion patterns known
so far reaches from a smooth tumor-host boundary surface to
more diffusely spreading patterns, which all are supposed to have
a different prognostic relevance. As a very decisive limitation
of previous studies, all morphological assessments just could be
done verbally referring to single histological sections. Therefore,
the intention of this paper is to get an objective quantification
of tumor invasion based on 3-D reconstructed tumoral tissue
data. The image processing chain introduced here is capable to
reconstruct selected parts of tumor invasion fronts from histo-
logical serial sections of remarkable extent (90–500 slices). While
potentially gaining good accuracy and reasonably high resolution,
microtome cutting of large serial sections especially may induce
severe artifacts like distortions, folds, fissures or gaps. Starting
from stacks of digitized transmitted light color images, an overall
of three registration steps are the main parts of the presented
algorithm. By this, we achieved the most detailed 3-D reconstruc-
tion of the invasion of solid tumors so far. Once reconstructed, the
invasion front of the segmented tumor is quantified using discrete
Index Terms—Biological tissues, image color analysis, image
processing, image registration, image segmentation, image shape
analysis, scientific visualization, tumors.
views of tissue organization, e.g., tumor morphology and tumor
O our understanding, to really consider volumes but not
just single slices is essential in order to get new insight
Manuscript received December 28, 2004; revised July 12, 2005. This work
was supported by the German Research Foundation (DFG) under Grant BIZ-6
and recommending its publication was N. Ayache. Asterisk indicates corre-
*U.-D. Braumann is with the Interdisciplinary Center for Bioinformatics,
University Leipzig, Härtelstraße 16-18, 04107 Leipzig, Germany (e-mail: brau-
Leipzig, 04107 Leipzig, Germany (e-mail: firstname.lastname@example.org).
J. Einenkel and M. Höckel are with the Department of Gynecology
and Obstetrics, University Leipzig, 04103 Leipzig, Germany (e-mail:
L.-C. Horn is with the Institute of Pathology, University Leipzig, 04103
Leipzig, Germany (e-mail: email@example.com).
M. Löffler is with the Institute for Medical Informatics, Statistics and Epi-
demiology, the Coordination Center for Clinical Trials, and the Interdiscipli-
nary Center for Bioinformatics, University Leipzig, 04107 Leipzig, Germany
Digital Object Identifier 10.1109/TMI.2005.855437
growth. The three-dimensional (3-D) characterization of inva-
sion patterns of squamos cell carcinoma of the uterine cervix
using histological serial sections is a current clinical question.
This gives demand for both high level image processing and
analysis. Properties of the hitherto observed two-dimensional
(2-D) tumor invasion fronts are supposed to have relevance for
the further prognosis of the respective patient –. Doing
those quantitative analyses in 3-D, a new quality for the struc-
tural and morphological assessment of the considered tumors
can be expected.
Three-dimensional imaging modalities like computed to-
mography (CT), cone beam computed tomography (CBCT),
nuclear magnetic resonance imaging (MRI), positron emis-
sion tomography (PET), single photon emission computed
tomography (SPECT), etc. have become state of the art in
many fields of medical diagnostics and research. Besides
those macroscopic in vivo 3-D techniques, for more detailed
analyses nondestructive 3-D microscopy is available for in vitro
(partially applicable in vivo), as e.g., scanning transmission
ion microscopy (STIM) or particle induced X-ray emission
(PIXE), scanning force microscopy (SFM), 3-D electron mi-
croscopy (3DEM), miniaturized computed tomography (µ
miniaturized nuclear magnetic resonance imaging (µ
confocal LASER scanning microscopy (CLSM), etc. For a
concise review on current high-resolution imaging techniques
of (living) tissue, see .
CLSM could be successfully applied on precancerous cer-
vical epithelial lesions  both ex vivo on biopsies in 3-D, as
well as in vivo in 2-D using a CLSM-variant referred to as “con-
focal microendoscope.” The limited range of CLSM of about
, 200 µm is acceptable for these epithelial lesions. The
whole epithelium’s thickness is 200,
ysis of cervical tumors, the CLSM’s penetration range unfortu-
nately is too short.
Even though other in vivo techniques would be desirable for
detailed uterine cervix analyses, spatial resolutions of
as achieved e.g., in MRIs from the pelvic region unfortunately
do not appear sufficient for conclusions on tumor invasion and
infiltration which necessitates resolutions
far beyond typical cell diameters of
clinical diagnostics of cervical cancer (e.g., tumor staging),
MRI and partially CT are indispensable ,  and partially
can be used to “predict” histopathologic features. The “ground
truth” for tumor typing, however, only can be obtained by
histopathology both using visual inspection and especially
, 300 µm. For the anal-
mm, but not
10 µm. However, for
0278-0062/$20.00 © 2005 IEEE
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS 1287
transmitted light microscopy once the tumor was surgically
Among the microscopic techniques, apart from those with
typically very limited fields of view (FOVs) and/or far sub-cel-
lular spatial resolutions there is no well established 3-D proce-
dure or protocol for tumor imaging providing appropriate con-
data acquisition using conventional transmitted-light imaging
based on HE-stained histological serial sections. The reasons
for taking this modality are twofold: at this stage of our work
we wanted to stay as close as possible to the procedure usually
applied by the pathologist when assessing single slices in rou-
tine. Moreover, both the achievable spatial resolution (
voxel edge length) and the effective FOV (typ. 0.1 cm ) can be
However, this necessarily requires solving the 3-D tissue
reconstruction problem based on huge registered serial sections
without having a reference data set available to co-register. On
principle, thiskind ofproblem has a long history and infact was
tions using manually digitised line drawings. More recent work
essentially could benefit from much improved computational
power (especially concerning CPU and RAM) but also from
(analogue, later digital) imaging improvements. Consequently,
partially treated as a co-registration problem –. Even
though a completely different level of quality was reached
meanwhile, due to its nature such 3-D reconstruction from
serial sections remain complex and time-consuming. What is
the central problem we are facing is the absence of some hard
quality criterion to refer to. This means, we cannot utilize some
reference data set since unfortunately there is none. This kind
of dilemma has motivated us to strictly follow a coarse-to-fine
strategy, i.e., we do the reconstruction in a stepwise manner
and apply registration schemes with an increasing order of
complexity, first a rigid one, then a polynomial nonlinear one,
and finally a curvature-based nonlinear one.
The nine-criteria based classification of the required registra-
tion method(s) according to  would comprise: 2-D/2-D (ad-
jacent image pairs), intrinsic-direct (pixel/voxel property based
only), curved (to compensate for nonlinear distortions, how-
ever, rigid registrations might be required in addition), global
(affect entire images), automatic (no user interaction), with pa-
procedures using explicitly computable parameters might be
additionally applied), monomodal (histological sections only),
intra-subject (no pair of data sets), and refers to pelvic organs
(specimen of cervical tissue). What is crucial is the ill-posed-
ness of the required underlying registrations , . That
means, registration results might be decisively affected from
small changes in the images. And, for this work, some appli-
cable algorithmic solution has to cope with a broad range of
different tumor invasion patterns—without knowing their char-
acteristics a priori. This paper introduces our newly developed
dedicated processing chain. For an overview of the processing
chain, see Fig. 4. It further elucidates quantitative results as-
sessing the tumor growth based on 3-D data, and also discusses
the above mentioned related work.
considered in this work. TNM nomenclature tumor stage T1b (see left)
is defined as a lesion greater than a micro-invasive cancer, which has a
microscopically measured invasion of stroma 5.0 mm or less in depth and no
wider than 7.0 mm (as for T1a1 or T1a2), and as a tumor confined to cervix.
Stage T1b is subdivided into tumors of 4.0 cm or less (T1b1) and more than 4.0
cm in size (T1b2). Stage T2 (see right) is defined as tumor invasion beyond the
uterus but not to the pelvic wall or to the lower third of vagina. It is subdivided
into cases without (T2a) and with (T2b) parametrial invasion. C: cervix uteri
(neck of uterus), Co: corpus uteri (body of uterus), I: isthmus uteri (constricted
part of the uterus between neck and body), Ca: cavum uteri (uterine cavity), F:
openings of the uterine tubes (fallopian tube), V: vagina.
Sketch of cut sections of the uterus depicting all tumor stages
Main objective of this paper is to provide an automated algo-
rithm objectively assessing the cervical tumor invasion based
on 3-D reconstructed tissue volumes using serial sections of
cervical specimen of resected uteri. Papers focussing on the
II. THE TISSUE RECONSTRUCTION PROCESS
A. Tissue Samples and Digitization
This paper comprises an overall of thirteen specimens of
squamous cell carcinoma of the uterine cervix (anatomic
tumor stages T1b1, T1b2, T2a, and T2b, according to TNM
nomenclature , , see Fig. 1), surgically managed by total
mesometrial resection . Three selected samples basically
exemplify the different tumor invasion patterns observed in
pathology routine (see Fig. 2).
The resected and formalin-fixed cervix was radially cut into
specimens (thickness: 6–8 mm) which were paraffin-embedded
(see Fig. 3), then serially sliced using a microtome HM355S by
MICROM GmbH, Germany [Fig. 2(a): 500 @ 5 µm, Fig. 2(b):
100 @ 10 µm, Fig. 2(c): 230 @ 10 µm], and finally stained
with hematoxylin-eosin (HE) using a staining machine. Sec-
tions as parallel planes starting from one of the radial cutting
planes typically have a rough extent of 2.5 cm
raw digitization area is 1300
8.28 mm0.865 cm at a nominal pixelsize of
8.04 µm . The digitization of the serial sections was carried
out manually using the AxioVision 3.1 controlling PC software
directly reading from a digital 2/3 one chip CCD-camera Ax-
ioCam MRc mounted on an Axioskop 2 plus transmitted light
1.5 cm. The
1030 pixels corresponding to
1288 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
based on single HE-stained slices.
Typical three-tiered verbal quantification of tumor invasion patterns
and the Video adapter 60C 2/3 0.63
made by Carl Zeiss, AG, Germany).
(all mentioned products
paraffin ready for slicing on a microtome.
View onto a typical unstained cervix specimen embedded within
Under these conditions, due to the still limited FOV the dig-
itization practically can be considered as a rough selection of a
region of interest (ROI) within the tumor invasion front. There-
fore, one also could refer to the placement of the microscope
slides as a zero-order registration step in order to maximize the
effectively reconstructible volume of interest (VOI) along the
tumor invasion front.
Some fiducials would be difficult to apply in our framework.
Since it remains unclear where the ROI/VOI within the spec-
imen is located (without staining one cannot reasonably locate
the tumor invasion front) one would need to have some stained
reference section for “navigation.” Then one would have to in-
directly place some fiducials (e.g., four—one nearby each ROI
corner). The choice of material is crucial (soft, but not too soft
rods). However, we expect the drawbacks are greater than the
benefits. Even if one would succeed placing the fiducials, one
themicrotome blade could be much more worn).The benefit re-
mains very limited, one could not spare any of the registrations
which are detailed in the following.
B. Rigid Registration
In the first stage, a serial section undergoes a successive pair-
wise rigid co-registration of all slices. By this, the data set is
restricted to an effectively captured VOI. The method is com-
puted on scalar (gray-leveled) images obtained based on the
of the original
the (old) International Telecommunication Union’s recommen-
dation ITU-R BT.601-5 as linear combination of the color pri-
color images following
The applied AxioCam MRc provides linear primaries (i.e.,
without gamma correction).
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS1289
Fig. 4. An overview of the processing chain towards 3-D tumor invasion front reconstruction at the example of specimen 8. Just starting from the unregistered
image stack, finally an appropriate basis for a subsequent automated 3-D invasion front quantification is provided. The second column consists of three orthogonal
planes (two reconstructions: the x-z planes above and the ?–? planes at right). In the third column, cutouts of the right half of the ?–? planes (second column) are
magnified, while in the fourth column even further zoomed cutouts are depicted (bottom-left quarters of the third column).
1290 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
What has tobe solvedis thefollowing transformation consid-
ering the parameters
tween the two scalar images
, (translation) be-
In fact, since
sake of simplicity in the notation any real existing differences
of the images are neglected.
The approach we are using is a noniterative two-step al-
gorithm consisting of a combination of the polar-logarithmic
Fourier-Mellin invariant (FMI) descriptor,  and phase-only
matched filtering (POMF) . Variants thereof have suc-
cessfully been applied in  and . FMI basically is
a polar-logarithmic transform accomplished on the Fourier
transformed images converting both rotations and scalings
into shifts. POMF is a matching technique representing an
extension of the matched filtering approach. However, the latter
is highly depending on the image energy rather than the spatial
structures within. A solution, therefore, is to take a transfer
function equal to the spectral phase as done by POMF. While
the pure cross-correlation technique as, e.g., applied in ,
tends to result in quite broad/flat maxima, POMF will yield
much narrower maxima. This method provides a reasonable
compromise utilizing both energy and phase of the Fourier
and . Experiments using a symmetric
POMF as proposed as SPOMF in  resulted in even more
narrow, sharper fits, however, with our histological data with
a variety of slight damages (fissures, missing parts, folds)
SPOMF turned out to be too susceptible compared with POMF.
The first part of the FMI-POMF-based scheme treats the ro-
tational registration, while the second part takes this determined
angle and computes the translation by means of another POMF.
Hence, the goal of the very first step is to determine the angle
by which the image
andare images of adjacent sections, for the
is rotated with respect to image
unit. While the spectral phase
closely depending on both translation and scaling, the spectral
magnitude is translation invariant
denotes the Fourier transform andthe imaginary
of the imageis
Now, since what is of interest is a rotational angle, the spectral
(orientation) and(wave number)
which is abbreviated in the following as:
and are referred to as the FMIs of the images
(2). By Fourier transforming (6) one obtains
as phase shift
We determine this phase shift under the constraint that no
scaling is assumed
by the following POMF (the star
denotes the complex conjugate)
Finishing the first part, the intermediate result is
For this part, the choice of the used rotational centre is arbitrary,
however, since the images are naturally of limited extend, it is
recommended to always take the physical centre of
minimize boundary effects.
While the rotational part of the rigid registration is finished,
the principle for solving the second part
in order to
Now, applying (2) by inserting the results of (8) and (11) this
rigid registration part is formally solved.
Further, for the implementation the following was applied.
• The images need to have an appropriate contrast. We take
the following method for local contrast enhancement 
“Local” refers to a squared vicinity centered around
with a side length of 55 which is about the max-
imum width of fissures in pixels.
obtain a strong but not maximum effect.
The images should be windowed in order to reduce
leakage artifacts of the fast Fourier transform (FFT) .
We used a Hann window
anddepict the global and local mean,
is the local standard deviation.
was set to 0.9 to
andare the image extents.
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS1291
• For all re-sampling steps throughout this paper at least
first order interpolation is recommended. We use higher
order polynomial interpolation (cubic splines).
In (5), only the half of the spectrum has to be considered,
and are of real data, so that their Fourier
transforms are mirror-symmetric w.r.t.
window is, therefore, halved as well.
The resolution of the angle
tation of (5); we take an angular scale of 720 for
sulting in a quarter-degree resolution; this does not imply
any iterative angle search, (8) requires one single max-
imum search, where the index of the maximum is a direct
measure for the angle.
Due to the applied FFTs, the time complexity of the rigid reg-
withdenoting the number of pixels,
the memory complexity is
rithm runs about 1 s on a standard PC for image sizes around
For an illustrative view on this processing step, for a typ-
ical slice-to-slice transition we have depicted the displacement
vector field using line integral convolution , , see Fig. 5.
This kind of visualization of directions and strenghs in vector
fields is considered more illustrative than any direct plot of the
vectors even though the respective sense of directions cannot be
or ; the Hann
depends on the implemen-
. For one image pair the algo-
C. Color Adaptation
Once the first registration step is carried out, the effectively
available tissue volume is more or less restricted to a core re-
gion depending on the accuracy of the slice placement during
the digitization. Since the staining is going to have further im-
achieved staining wrt. to its constancy. Even though applying a
staining machine, the number of sections per series by far ex-
ceeds its capacity, so that series only can be stained by stages,
thus unfortunately introducing fluctuations. Another reason for
a similar effect are some very slight thickness variations which
also can appear as fluctuations mainly affecting the color satu-
This adaptation procedure is accomplished as second step,
since the completely unregistered data set is inappropriate for
doing a section-wise adaptation. Once the data has passed a
first rough registration, for every section we can assume to have
corresponding ROIs for all images which is not the case just
after initial digitization. Hence, in this second step within the
reconstruction procedure we are going to treat possible fluctu-
ations of the staining along the serial sections. The idea behind
the simple but effective procedure is as follows: the concerned
sample image’s staining is subsequently adapted using a linear
color transform based on statistical distribution parameters.
So, the essence of the scheme we are proposing is just to
force all sample images to have the same mean and covariance
matrix applying a linear transform. In principle, what has to be
the displacement vector field to adapt (b) onto (a) according to the obtained
parameters ? , ?
is depicted using line integral convolution. The
(cyclic) color codes the absolute value of the underlying displacements from
purple-red (high) via blue, cyan, green, yellow to orange-red (low). The
maximum displacement is 1184.3 µm (lower right), the minimum is 254.1 µm
(upper left) which is located closely to the “rotational center” (outside the
Rigid registration examples: For a pair of adjacent slices (a) and (b)
1292 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
(in homogeneous coordinates), where
denote the transform matrix, the sample section
image, the transformed sample section image and the reference
section image,respectively,whilethelatteris manuallyselected
for each series.
values of the red, green and blue image channel, respectively,
: We consider allpixels belonging to the finite set
90% of the maximum intensity as well as with a minimum in-
tensity above 0. This restricts the transform to nonbright image
regions, since one can assume to have a background consisting
of either bright or black image regions due to fissures or artifi-
cially filled black margins, respectively.
in (14) consists of the following matrix
, , and are the mean
Its individual factors are obtained as follows.
and denote the offsets (referring to
tively) and are determined as
and , respec-
and in analogy for
of rank 4.
, with representing the identity matrix
(wrt. decreasing order of their corresponding eigenvalues) of
the covariance matrix
of the centered color value data (in
is the matrix of sorted eigenvectors
This real symmetric and orthonormal matrix and the mean
vector represent an estimated multivariate distribution of the
color values in
by solving the following eigenvalue problem:
has the (full) rank 3,
one obtains the three eigenvalues
reduction to get a tridiagonal form and then based upon this
using the QL algorithm (with implicit shifts).
and their corresponding
. The problem was solved using Householder
denotes the scalings along the principal axes, which are
The time complexity of the color adaptation is
denoting the number of pixels, the memory complexity is also
. For one image the algorithm runs less than 1 s on a stan-
dard PC for image sizes around megapixels.
Two examples are given in Fig. 6. Although the method is
simple, the results can be considered adequate for our purposes
since mainly staining-related outliers with small fluctuations
are targeted. To the knowledge of the authors, even if simple,
this method is not implemented in standard image manipulation
D. Polynomial Nonlinear Registration
This third stage basically does the compensation for
slice-global distortions using polynomial warping  based
on sparsely populated displacement vector fields taken from
automatically determined control points. Its basic form is
represents an undistorted reference image and
the already rigidly registered but still distorted coun-
terpart. The unknown coefficients
th degree for each independent variable)
respectively, can offhand be found once displacement vectors
are available. Those displacement vectors rely on the pair-wise
correlate of partially overlapping image tiles (i.e., subimages).
These tiles are sized 128
128 pixels and overlap 96 pixels
in both directions. 128 was taken as the most appropriate
power of 2 (lots of FFTs have to be computed). The overlap
results in a density of control points of one per 32 pixels which
results in reasonable numbers of control points in the order of
1000. To prepare the tiling, we ensure the image dimensions
to be multiples of the nonoverlapping tile size by adding an
appropriate black margin. Again, we use POMF applied to all
computing the correspondencies to the control points
th degree polynomials
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS 1293
registration. Insomecases,appearingasintenselystained as(c)amore obviousimprovementcan be achieved(d).Note thatforthiscomparison(c/d) isnotadjacent
to (a/b), there was just one slice skipped in between in order to be more illustrative. The reference slice for this serial section is depicted in (e).
Color adaptation examples: Usually the adaptation only leads to minor changes as from (a) to (b), whereas (a) is corresponding to Fig. 5(a) after rigid
So, for a tile pair
vector is, e.g.,
and the corresponding displacement
a multivariate linear regression using a least-squares (LS) error
minimization is done. The multivariate model is
representing the matrix of displacement vector end points and
an arranged matrix of all coefficients. For compactness reasons
of the derivation, we have further introduced the matrix
ferred to as design matrix which is build up from
products of combinations of the control point coordinates (i.e.,
the displacement vector start points)
wise linearly independent row vectors are supposedto represent
1294 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
as well as a covariance matrix
The general form of the sum of squared errors is
which has to be minimized.
differentiating (27) with respect to
zero in order to find an extremum one achieves
denotes an estimate of
and setting this expression
Reforming (28), one easily can isolate
full rank) and gets
is positive definite which holds true if
ready assumed above.Thus, the result of (29) can be considered
valid. The two columns
and , respectively, which now can be inserted in (21).
For the implementation of (29), even with a simple
Gauß-elimination method and
lems throughout the matrix inversion have occurred. Unwanted
warping effects as reported in  have occasionally occured,
but could be managed as we have introduced the following
blocks) we artificially haveadded some
control points (associated with zero-length displacement vec-
tors) and placed them just along the image margin as a “frame”
around the existing grid of control points. Under the condition
that the rigid registration step was successful, this kind of
“framing” is warrantable. This extension did decisively im-
prove the method so that no “collapsing” or other unwanted
warpings thereupon did occur. Further, differing to the previous
rigid registration, this could be accomplished based on the
luminance of the color adapted images.
The time complexity of the polynomial registration is
due to the FFTs with
pixels, the memory complexity is
the algorithm runs about 3 s on a standard PC for image sizes
Corresponding to Fig. 5 also for this polynomial registration
step we give an illustration of the resulting displacement vector
field for the same typical slice-to-slice transition, see Fig. 7
using line integral convolution.
has the full rank as al-
ofare the estimates of
no singularity prob-
denoting the number of
. For one image pair
E. Staining-Based Tumor Probability
Now, while two registration steps are done, the serial section
is fairly re-aligned. Most of the slice-to-slice transitions can be
pair of adjacent images (a) and (b) according to the estimated polynomial
coefficients the resulting displacement vector fields is visualised (c). The
(cyclic) color codes the absolute value of the underlying displacements from
purple-red (high) via blue, cyan, green, yellow to orange-red (low). The
maximum displacement is 84.4 µm, the minimum is 0 µm. What is visible
at the first glance is the inhomogeneity of the field, whereas on the right and
left there are two distinct local maxima of the displacement, in this case the
registrations leads to some contraction from the left/right/upper part toward a
region around below the slice center. Two vortices (upper right and lower left)
occur outside the physical slice and have very small strenghts.
Polynomial registration examples: Corresponding to Fig. 5, for a
treat those remaining registration errors, we subsequently need
to apply yet another registration step. Just like for the previous
registration steps, this one also applies to scalar data. Despite
of taking some luminance-related images, we, therefore, want
to use scalar images highlighting the tumor regions. We gen-
erate such images simply by computing staining-based tumor
probability maps relying on the HE staining applied to all slices
short after sectioning. The probability maps are necessarily re-
quired for threshold-based tumor segmentation. The reason for
swapping these two steps is mainly that by this we can further
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS 1295
and polynomially registered images (a) and (c) and have determined the respective tumor probability maps (b) and (d). This example also illustrates the usefulness
of the color correction by what the two probability maps appear qualitatively equal. Note that the suddenly emerging margin was artificially introduced in order to
get image dimensions as multiples of primes up to 5 which is useful for the many FFTs accomplished in the following curvature registration.
Staining-based tumor probability computation examples: Corresponding to the image pairs of Fig. 6 we have taken the same two but now color corrected
attenuate artifacts mainly occurring outside the tumor regions,
which facilitates the final registration step.
Basically, it is required to manually obtain representa-
tive tumor color samples from the respective serial section.
Precisely, we arbitrarily select a number of ten slices equidis-
tantly along the series and let the pathologist draw in the
tumor boundary which provides us both with
for tumor and nontumor. Now,
for the tumor probability we adopt normalized color values
leading to a projec-
tion onto a sphere sector fitting within the RGB cube, where
. One of the normalized components is
redundant, so we can restrict to use
assume that for the tumor as well as for the nontumor
components follow multivariate normal distributions. So, we
estimate the multivariate densities for both sets
. We further
means, respectively. Finally, the probability for a pixel to
exhibit the color of tumor at
and denoting the covariance matrices and
Fig. 8 illustrates the tumor probability computation for two
images. The results indicate that the previous color-adaptation
(compare Fig. 6) is justifiable.
The time complexity of the staining-based tumor probability
withdenoting the number of pixels, the
memory complexity is
as well. For one image the algo-
rithm runs about 5 s on a standard PC for image sizes around
F. Curvature-Based Nonlinear Registration
In this processing stage, remaining local registration errors
are diminished and the image-to-image transitions are further
smoothed. What generally has to be computed is a local dis-
placement field as vector function
considered as representation of the mis-
alignments. The nonparametric procedure we are applying for
this nonlinear registration uses a regularization term approxi-
mating local curvature, which was introduced in  and re-
cently studied in , . While the authors state that the al-
gorithm would include an automatic rigid alignment, with our
data we in fact could not benefit from this effect. This can be
explained as follows: their images strictly cover some complete
more or less assumed to successfully find all correspondences
one-by-one. However, in our image material we do not have
isolated objects with some delimited boundary. In fact, since
the digitized regions usually do not comprise any background
1296 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
but generally tissue, around the image margin for a consider-
able portion of the images pair-wise correspondences will be
missing. Therefore, from the viewpoint of this algorithm the
previous registration steps can be considered as preprocessing
in order to drastically improve a priori image-to-image corre-
Basically, the distance measure to be minimized for this reg-
tensities [here, taken as tumor probabilities using (32)]. Again,
represents an undistorted reference image and
the already both rigidly as well as polynomially registered but
still distorted counterpart, while the registration should do the
we define a joint registration criterion consisting of the sum of
and the smoothing term
From the calculus of variations we know that a function
imizing (34) necessarily should be a solution for the Euler-La-
[see (37)] an artificial time parameter is introduced as
with the boundary condition of
To solve (39) the time dependence is discretized using an
implicit midpoint rule for the linear operator
. For the integration over a single time step
being periodic across the
for the propagation from
to. Defining the
the solution for the next time step is found by
Denoting the discrete Fourier components
and using [35, equation (25.3.33)] for the discrete version of the
biharmonic operator, (42) in the Fourier domain is given by
whereand and for a
is used to update . Every time integration step needs a
the two components of
. We have chosen periodic boundary
conditions for the nonlinear registration. Other boundary con-
ditions, e.g., with zero displacement on the boundary or zero
normal derivative for
can also be used. It shoud be noted that
we always could find a single transformation without the need
Fig. 9 illustrates the curvature registration for an image pair
using line integral convolution. Concerning the parametrization
of the algorithm, for all specimen we have applied a fixed max-
imum number of 32 iteration steps, whereas
with an iteration time step
of 2.0. The solution is computed
in the Fourier space. We have iterated some certain fix number
of steps and applied both a fixed stepwidth and smoothing co-
efficient. Remember, reference-free registration is an ill-posed
problem, so the difficulty is especially to avoid removing all
differences between two adjacent sections/images. The specific
choice of these mentioned parameters was made as follows: we
is computed the backward Fourier trans-
was set to 5.0
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS1297
to Figs. 5 and 7, for a pair of adjacent images (a) and (b) the determined
displacement vector field is visualised (c) using line integral convolution.
The (cyclic) color codes the absolute value of the underlying displacements
from purple-red (high) via blue, cyan, green, yellow to orange-red (low).
The maximum displacement is 36.2 µm, the minimum is 0 µm. Compared
to Fig. 7 this displacement vector field is even more inhomogeneous and the
displacements are at most half as far as for the polynomial step.
Curvature-based nonlinear registration example: Corresponding
wanted to have the curvature term get weighted as five times as
the squared differences. The time step and iteration steps were
selected based on visual inspections of tests making sure that no
eye-catching unwanted warpings occur. At the present stage of
our work we did not implement any time dependency of these
To further improve the performance of those nonlinear
schemes, ,  have proposed algorithms what they have
called Pair-and-Smooth registrations which combine geometric
matching with intensity-based registration. This together with a
multigrid implementation will be a future direction of our work.
Because of the FFT-based implementation, the time com-
plexity of the curvature-based nonlinear registration step is
, with denoting the number of pixels, the memory
. For one image pair the algorithm runs
about four minutes on a standard PC for image sizes around
megapixels. Comparing this CPU time with the time neces-
sary to do the microtome sectioning, the staining, the manual
digitization, and a number of previous processing steps, we
still consider some minutes acceptable for one image-to-image
transition. With the above mentioned multigrid implementation
(this is ongoing work) we expect a decisive computational
G. Total Variation Filtering
Due to the pixel based color segmentation typically the data
is affected by a significant amount of noise. While this noise
is not essentially affecting the previous registration step, we
consider the necessity for an intermediate processing on the
reconstructed 3-D data step in order to facilitate the succeeding
thresholding-based segmentation. Non-linear filters are in
general much better in preserving image structures compared
to linear ones. So, e.g., median filtering perfoming a ranking
operation will keep edges but remove outliers while, on the
other hand, linear binomial filtering will damp both. However,
while the median filter  is appropriate in case of simple
salt-and-pepper noise, its homogenizing properties remain
limited. More sophisticated schemes like nonlinear diffusion
filtering ,  have been proposed, however these basically
require some certain stopping criterion, otherwise the image
structures get lost.
Instead, we have decided to apply nonlinear total variation
filtering . This filter minimizes the functional
the probability that at
, for the scalar 3-D image that contains
a cancer voxel can be
be the original noisy image with Gaussian
exhibiting the following properties:
the Euler-Lagrange equation
minimizing (45) generally can be obtained solving
For this extremely nonlinear equation several solution methods
are known. Since the considered volume data are very large,
memory intensive methods are not feasible, because a nonlinear
solver would require several gigabytes of temporary memory.
1298 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
transformed by the curvature-registration, and now were TV filtered attaining an edge-preserving smoothing as depicted in (b) and (d). Note that the TV filtering
is applied as 3-D operation, which is why we show this adjacent pair here (the 3-D effect can be best seen in the nontumour background).
Total variation filtering examples: This example again refers to the same pair as in Fig. 9. The two images of the adjacent slices (a) and (c) were
However, an appropriate solution method with low memory de-
mands was proposed by Osher et. al. . It transforms the
problem into a time dependent problem for
. So, instead of (46) we apply
ishes, the equation becomes singular so that the gradient must
be regularized as
. In regions where the gradient van-
. For the discrete solution
the summation runs over all next neighbors of
. For inner pointsin thevolume a 6-neighborhood
stencil is used. Boundary points cover a reduced neighborhood
as respective grid points exist. This yields a nonlinear filter for
every mesh point
which is de-
lation one simultaneously needs to store at least three data sets
, and. The main advantage of the filter is its rel-
atively quick convergence toward the denoised result. The only
free parameter is . Its choice is of importance for the denoising
quality. Following  we use
of mesh points
. After 5–10 iterations a new
and used for the update of the
The time complexity of the total variation filtering is
with denoting the number of voxels, and also the memory
. For one image series with typically 300
sections the algorithm runs about half an hour on a standard PC
for imagesizes around megapixels.Thisstep is a 3-Doperation,
so it may have exorbitant RAM requirements, since
andhave to be accessible simultaneously.
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS 1299
Fig. 11. Tumor segmentation example: The two images (a) and (b) show the results after thresholding the TV filtered images in Fig. 10(b) and (d), respectively
Fig. 12. (left) Multiplanar reconstruction and (right) 3-D surface rendering of the invasion front for specimen 8.
Fig. 10 illustrates the 3-D TV filtering effect on a pair of ad-
H. Tumor Segmentation
In this final reconstruction step, just the TV filtered volume
data is binarized. According to (32) an illustrative criterion for
thresholding is where the two estimated densities for tumor and
nontumor exhibit the same magnitude so that the tumor proba-
bility is 0.5. Let
represent the previously TV filtered scalar
image, the binarized result
is obtained as
Both time and memory complexity of thresholding of course is
with denoting the number of voxels. It takes a fraction
of 1 s on standard PC hardware.
III. INVASION FRONT QUANTIFICATION
Once the smoothing by means of total variation filtering and
the segmentation was accomplished, following the 3-D recon-
struction process the tumor invasion front within the volume
data is going to be assessed. Hence, the invasion front firstly
is visualized and subsequently quantified.
A. Three-Dimensional Tumor Visualization
What is of basic interest is the topology of the invasion front.
One of the interesting questions at hand is how the tumor inva-
sion front is shaped. Another question is the presence of pos-
sibly separated tumor islets apart from the main tumor. Some
virtually have occured but turned out to have direct contact to
the data set outskirt. These were sorted out since it cannot be
decided if separated or not. The rest, however, was not straight-
forward to be verified or falsified, due to the limitations of the
HE staining. HE in fact is just enhancing image contrast with
respect to the averaged local cell kernel density. In tumor cells,
sifications might occur as, e.g., for smaller inflamational cells
or some other dense tissue parts. We have let the pathologist
check all suspected tumor islets using a much larger magnifica-
instead of 1.25 ) but got none of them verified to
consist of tumor. What has remained for all our specimen was
one large connected tumor segment, a kind of “massif” VOI of
the tumor invasion front.
Therefore, in order to give a 3-D illustration of the recon-
structed tumor invasion, we do a surface rendering applying the
1300 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
Fig. 13.Views onto the 3-D surface renderings of tumor invasion fronts, part I.
faces uses the well-known algorithm from  with the mesh
displacement modification , . A detailed discussion of
rendering algorithms can be found in .
The gallery of tumor invasions of our 13 specimens is shown
in the Figs. 12–14. These are the first-ever visualizations of
a solid tumor’s invasion front with a resolution of
The renderings have been generated using MathGL3d ,
an OpenGL-based interactive viewer for Mathematica’s 3-D
graphics. Typical numbers of (potentially nonconnected) sur-
face polygons occur from
series with 300 sections and image sizes around megapixels
. For a typical image
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS 1301
Fig. 14.Views onto the 3-D surface renderings of tumor invasion fronts, part II.
the algorithm runs about 5 min on a standard PC with OpenGL
graphics hardware, but of course this will closely depend on
the respective invasion front morphology.
B. Invasion Quantification
Another highly important goal of this paper is to give a quan-
titative characterization of the 3-D reconstructed invasion front.
Just doing a verbal assessment of the invasion obviously is not
not want to provide something closely mimicking the pathol-
ogist’s assessments, e.g., as in cytology. Just conversely, the
recently pointed out in a review on the parameters of prognosis
in cervical cancer by Singh and Arif , there is too much
1302 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
confusion in the assessments of the importance of the invasion
pattern: “The importance of pattern of invasion as an indepen-
dent prognostic factor is extremely difficult to evaluate due to
varying definitions and parameters.” Obviously, this clear state-
ment is articulating the relevance of our work.
In principle, there are a couple of possibilities feasible for
quantification, e.g., descriptions with components basically
consisting of tumor surface areas or volumes, or differen-
tial-geometric properties. However, considering fractal proper-
ties would basically suffer from far too small tumor/VOI size
to cell diameter ratios rather than being primarily a problem
of digitization resolution. At most, four orders of magnitude
are not considered to be sufficient for reliably analyzing some
Alternatively, more feasible and even more promising should
be a description relying on the sizes of both tumor surface
and volume. A pretty much known description consisting
of just these two components is compactness. This is an in-
trinsic 3-D object property and is dimensionless defined as
volume with the sphere as that object providing the
absolute minimum at
. Direct compactness implementa-
tions, however, do lack of sufficient robustness i.e., surface
enlargements due to noise could lead to the same compact-
nesses like real surface shape changes. A new way to determine
a compactness which far less can be irritated is discrete com-
introduced in . Instead of directly considering
surface and volume,
relies to internal voxelcontact surfaces
and is defined simply as
a 3-D object consisting of
imum of contact surfaces achieved with a cubic object also con-
voxels (isotropic case). Contrasting to , we de-
, in order to consistently allow for objects con-
sisting of neighboring voxels even without contact surfaces, so
for a “cube” and
obviously would be evaluated little less compact than a cube.
complexity is again
. For one image series with typically
300 sections the algorithm runs several seconds on a standard
PC for image sizes around megapixels.
, voxels, whereas cor-
is the theoretical max-
for a diagonal
So far, the processing chain meanwhile was applied to an
cervix. No special selection criterion was applied except the
tumor stage to be within T1b1 and T2b. Despite utmost care
due to unavoidable but inacceptable damages, e.g., large folds
and/or fissures up to 2% of the sections had to be sorted out,
those were replaced by adjacent sections.
The overview given in Fig. 4 is intended to illustratively ex-
emplify how the histological image data is effected for each
processing step. The three-plane orthogonal reconstructions (all
cuboid object consisting of a sequence of 33 artificial 256?256 slices which
underwent some disturbance consisting of a shift, a rotation, and finally a
superimposition with Perlin noise. The three registration steps can restore the
original cuboid (not shown) quite good. Note that in each case the registrations
were accomplished from bottom toward the top of the “stack” of slices. The
outcomes of the rigidly registered and the polynomial step do not differ that
much for this test case. This is since the underlying control point related
displacement vectors are predominantly zero for the polynomial registration
due to the binary data.
The three registration steps applied on synthetic data: The binary
another synthetic data set with 33 distorted squares when registering using
the curvature-based method alone. The left figure shows the outline of the
stack of unregistered images and the position of the corners labeled as red,
green, blue and yellow points. The right image shows the outline after the
curvature registration as well as the corresponding transformed corners. The
superimposition in the middle shows the path of the corner points along the
?-direction before (thin lines) and after the registration (thick lines) as well as
the connecting straight lines for the corners in the upmost and lowest image
plane. The registration was done with ? ? ? and ? ? ??? over not more than
1024 steps for every image along the stack.
This figure demonstrates the preservation of the corners for
structed images may illustrate how the tumor segment bound-
aries are getting smoother. It should be noted, however, that de-
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS1303
DETAILS OF SERIAL SECTION PROPERTIES, REGISTRATION QUALITY ASSESSMENTS AND INVASION QUANTIFICATION
spite the nominal plane coordinates remain fixed, the progress
registration just might correct for some a priori unknown re-
Due to this kind of difficulties in assessing especially the
with synthetic data shown in Fig. 15. For a binary cuboid ob-
ject the zeroth (artificial) slice remained an undistorted square
128 pixels, not depicted) while all following 33 slices
underwent a progressive change consisting of a shift, a rotation,
and finally a deformation using a Perlin noise superimposition.
Sowe gained somespecialbutdifficultcase wherethedeforma-
tion is quasi-continuous along the stack. What is visible is that
compared to the preceding rigid registration. The reason is that
for this binary object too few nonzero length displacement vec-
tors can be obtained. The curvature registration in turn, which
plete displacement vector field and by this is quite satisfactorily
reconstructing the cuboid shape. Differing to the procedure on
kind of smoothness is not adequate. While for our example our
goal was to demonstrate to what degree the curvature registra-
tion can be operated, we deliberately take a much lower number
of steps, i.e., our choice with 32 steps is considered to be quite
conservative and will decisively reduce unwanted warping.
Some important aspect is documented in Fig. 16. In this
second example, we focus on the consistency of the curvature
registration. We demonstrate this with respect to the corners
of the synthetic images before and after the processing for
the whole stack. It becomes visible that even for this most
flexible nonlinear registration corners remain just corners and
no unwanted warping occurs. For the other two registrations
such warping is not really an issue. While impossible for the
rigid step, this is also no serious problem for the polynomial
step. The latter basically can degenerate, but by this the trans-
formation would completely corrupt the image. We actually did
not observe such corruptions using polynomials with degrees
up to five.
To illustrate the basic effects of the registration steps on real
data we would again like to refer back to the line integral con-
volutions in Figs. 5(c), 7(c), and 9(c). Note that in these im-
ages the length of the displacement field lines are not exhibiting
the respective displacements but just the course of the field. In-
stead, the length is color coded (with each example using an
own, adapted scale). Both displacement lengths as well as the
courses of the displacements exemplify each step’s main focus:
the general movement to roughly align the pair, the slice-global
dewarping, and the local nonlinear adjustments, respectively.
Concerning whole serial sections, several quantitative details
on the 13 data sets are given in Table I. For the three respec-
tive registration steps, mean residual registration deviations are
given. For a serial section with
adjacent pairsresidual displacement vectors represented as
slices, for each of the
referring to a set of
obtained. The displacements are computed in analogy to (22).
The mean residual deviation of a series is summed and normal-
grid-like arranged control points can be
the goal is not to eliminate any differences of adjacent slices),
in all of the 13 cases this error effectively can be further de-
creased by means of the second and third registration step. Note
the numbers put in parentheses in the third “mean residual devi-
ation” column of Table I: these were the mean residual registra-
tion deviations if the polynomial nonlinear registration would
1304 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
a rather homogeneous but not a three-tiered distribution as used for the verbal
assessments (compare Fig. 2). Specimen 2 and 9 holds the most and least
compactnesses, respectively. Dots are labeled with consecutive specimen
numbers (see Table I). The terms next to the specimen numbers are from the
pathologist’s second assessment, while the abscissa has no other meaning than
to qualitatively rank the compactness numbers.
The compactness values of the 3-D reconstructed specimen exhibit
be skipped evidencing how the processing chain benefits from
The compactnesses for all specimen are rather equally dis-
tributed between 0.881 (diffuse invasion) and 0.995 (closed in-
vasion, see, also Fig. 17). A corresponding linear regression
with a verbal three-tiered 2-D based clinical routine assessment
(cf. Table I) based on single slices out of the same specimen
yielded a correlation coefficient of 0.73. One should note, how-
ever, that this is not a correlation with some hard reference,
but illustrates that the new results are in a reasonable relation
with the “traditional” ones, while being far superior concerning
their degree of reliance. As the decisive benefit of the compact-
ness-based quantification now these numbers can be used for
clear more precise comparisons with other clinical parameters,
as the expression of immunohistochemical markers or the sur-
vival period etc. To do the latter is still to early, while the first is
to emphasizewhythisprocessing chainis legitimatefor the3-D
reconstruction process. In the following, we try to discuss as-
pects of known previous work in this or related fields.
Ourselin et al.  have introduced some single-step algo-
rithm for the reference-free 3-D reconstruction of a rat’s brain
from serial sections, whereas we have set up a multistep pro-
cessing scheme. Their implicit assumption is that after some
rigid registration remaining distortions are not too important for
the 3-D reconstruction, which is basically not considered ac-
ceptable for our work. However, applying some nonlinear reg-
istration on our data without assuring that throughout the slides
the image correspondencies are accessible (irrespective the in-
herent differences between adjacent sections) would cause lots
of severe problems. That is why an initial rigid registration step
is considered essential, otherwise we would risk the failure of
the following nonlinear steps. So we indeed have decided to
as possible. However, what was even more crucial is the mag-
nitude at hand of the shifts/rotations to compensate, so the re-
quired block size would be quite large to sufficiently cover the
appearing dislocations. In summary, these aspects have tipped
the scales toward taking a nontiling approach for the rigid step.
Ourselin et al.’s block-matching strategy applied for some ba-
sically more sophisticated rigid registration approach has some
similarities to the image tiling setup for our second (polyno-
mial) registration step. Besides the worthwile multiresolution
implementation, they use a Manhattan distance based regres-
norm) which has turned out to be more robust against
(even some small portion of) outliers among the displacement
vectors. The LS original (
norm) in general is known to be
decisivelyaffected fromoutliers.However,unlike LS,their
Ourselin et al.’s work, outlier-induced problems have not been
an issue in our work, neither we have got unwanted warpings
with our setup (even with polynomials of fifth degree) nor we
were in need to treat all distortions with the polynomial regis-
Bardinet et al.  have reconstructed parts of the human
brain (basal ganglia) using histological serial sections but could
benefit from the existence of a reference incident light pho-
tographs data set which was taken just before sectioning. The
registration algorithm is in principle that from  but applied
as coregistration. As we would highly appreciate a reference
data set for our work, we do not have, so we respect that the
some nonlinear registration step we have doubts that the result
Basically, for our work we have had plans to provide us with
some reference data just by taking incident light photographs
from the cutting area immediatly before the respective micro-
tome cut was obtained, as done by Schormann et al. . How-
ever, this could not be applied for our data, since the general
contrast was far too low to reasonably use them as reference
pictures. In particular, looking on the unstained specimen one
cannot reliably distinguish tumor from normal tissue. For ex-
ample, the dark structures visible in Fig. 3 on the cutting area
related reconstruction and analyses staining is essential, but it
can only applied on the sliced tissue, not on the tissue block
embedded in paraffin. Contrasting to the vast majority of pa-
pers reporting on histological serial section reconstructions we
cannot exploit the availability of a more or less clearly visible
structures (e.g., some organ outline), since on the microscope
used for digitization we cannot capture the full slice
but only some limited FOV of about 1 cm . Our images always
cover some ROIs amidst the sections. Even if we would have
covered (at least parts of) the organ outline, this would be a too
weak reference for ourdata ofinterest whichis thetumour inva-
reconstructing the human cortical brain from histological serial
sections and could utilize both an incident light data set as well
as a MRI 3-D data set (even though not coplanar to those of the
other two). Moreover, they did scan the whole sections which
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS1305
points to the important aspect that their work operates two or-
ders of magnitude above our’s, i.e., with voxels of about 1 mm
extent—even though the microtome sectioning was done with
30 µm thickness. This and the Nissl staining has allowed them
to register the incident light data with the slices. Instead, for our
unsectioned tissue no relevant tumor structures essential for use
as reference are visible. What is common to our work is that the
processing wasdone instepsdoing firsta affineregistration(ex-
tended principal axis transform) and following some nonlinear
For Mega et al.’s paper  we have to state something very
similar as for Schormann’s. Besides utilizing a reference data
set available (FDG-PET in this case), they also benefit from the
incident light photographs taken before the physical sectioning,
and again, they process full human brains with a spatial reso-
lution in the order of 0.5 mm. What is especially interesting
in this paper is that the 2-D-registrations (stained histological
section onto the presectioning photo) were accomplished using
pairs of correspondingly drawn contour sections along the brain
outline which serve as registration anchors which are required
from the elastic registration algorithm. It has to be assumed that
this explicit focussing on the outline was done mainly due to
contrast deficits within the tissue. As stated above, for our work
we cannot exploit the specimen outline.
In , Ali and Cohen try to map 2-D histological sections
onto an existing 3-D atlas of the rat’s brain. The problem is
treated as some outline based approach and utilizes geometric
curve invariants (i.e., some inflection points) which are taken
as kind of landmarks for the affine transform. Even if operating
on a 20 µm scale this outline focussed approach is not at all
the outline is an intuitive but weak anchor since it is frequently
damaged from the microtome sectioning.
Last butnotleast,since additionallighting fluctuations are no
matter here, since we can assure reproducible conditions on the
microscope. The question is rather if it is maintainable to es-
pecially treat those staining fluctuation within our framework.
Malandain andBardinet are addressingtherelatedproblem
of intensity fluctuations along serial sections. Besides the cen-
is crucial being capable to invert the respective error functions,
which is no problem for the one-dimensional case. For three di-
mensions, however, this is no longer straightforward, as long
as the PDF is not a (multivariate) Gaussian, so that the correc-
Having said that, to treat the three bands of
dependent scalar images to circumvent this would not be a good
Lehmann et al.’s work  is introducing the cepstrum
technique into the Fourier-Mellin based rotation-, scaling-,
and translation-invariant image descriptor, which among others
is applied on histological images. It is closely related to the
scheme we are applying for the rigid registration. Their main
argument for doing this with the cepstrum technique is the
robustness against uncorrelated noise and intensity distortions.
The latter, however, is not a problem for our data. Moreover,
-images as in-
even though the Fourier-Mellin technique as basically applied
for our work too is capable to treat image scalings, there is no
certain reason that the sections would be affected by forces
requiring scaling to compensate them. All relevant image
acquisition parameters including magnification were kept
constant. Even if some slight shrinking or stretching of some
sections would appear, this would be be treated as a special
(isotropic) case of distortion later on and can be compensated
by the nonlinear polynomial registration.
Fischer and Modersitzki  do present a very interesting
work, a nonrigid image registration algorithm. This curvature-
based nonlinear registration was adopted within this work for
the third registration step. What is claimed is the needlessness
of some rigid preregistration step as applied in various other ap-
proaches.As thismightholdtruefor caseswherethewholesec-
tions are visible, for our cases with even an initial FOV from
within the sections this is not applicable. This means, in case of
too much inexistent correspondencies, that algorithm will intro-
duce unwanted warpings. That is why we have introduced some
preregistrations. The reason why we did not just the rigid one
but also the polynomial is that we wanted to act as careful as
possible to avoid the loss of the 3-D integrity. However, what
now seems to be worthwhile is to skip the polynomial registra-
we expect a further improved performance of the curvature reg-
istration if applied as multigrid scheme.
Compared to these sketched known related approaches, our
conditions are probably more complicated, especially because
oftheabsenceofa referencedataset.Anotherpointis theuseof
but on the other hand is guaranteeing utmost reproducibility in
lighting and optical accuracy at a “nominal” resolution of 3160
dpi (8.04 µm per pixel). A motorized table was unfortunately
not available to us, however accurate stitching not necessarily
would be trivial. Under the given conditions, we do not see al-
ternatives as with the dedicated multistep scenario for tissue re-
construction aiming to preserve the structures of interest, i.e.,
the tumoral invasion fronts.
Form our point of view, our contribution might be considered
as some preparation toward a virtual microscopy framework
for histology and pathology. The upcoming purely digital mi-
croscopy technology sounds appealing for problems like our’s.
Products like MiraxScan by Zeiss or CoolScope by Nikon, to
mention just two, are available meanwhile and can reach pixel
extents down to 0.25 µm which corresponds to about 100000
dpi. Those can operate at FOVs up to the full (!) glass slide at
using some built-in automatic image stitching. Resolu-
tion, however, is important but is not the only criterion. For ex-
ample, for our 3-D reconstructions, we have been operating not
far below the 2 GB RAM usage level. Even in future, irrespec-
tive of the most appropriate resolution which for our problem
is about 3000 dpi, for this kind of problems one probably will
operate in ROIs rather than within the whole respective FOV.
Withtheabovedetailed schemeanobjective quantificationof
the invasion of cervical tumors based on 3-D reconstructed
1306IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 10, OCTOBER 2005
data could be achieved for the first time. To reach this
main objective firstly we had to set up a dedicated image
processing chain enabling us to provide a very detailed tissue
reconstruction. The reason why transmitted light microscopy
instead of flatbed scanning was used for this work is just to
assure an excellent reproducible image quality at a required
Our primary interest was not to extend a 2-D microscopic
technique to 3-D, but to suffice a requirement of providing spa-
tial resolutions of approx. 0.1 µm
modality which reliably can be utilized to segment the tumor.
At present, even though desirable, due to computer memory
limitations not even tumor stage T1b1 cervical carcinoma vol-
umes can completely be reconstructed at this resolution. Self-
evidently, the trade-off we found was just to select a typical
region within the tumor invasion front which is considered to
be representative for the whole respective tumor. Although we
hardly can proof the latter—we rarely can accomplish a (tile-
wise) reconstruction for a whole specimen—we consider that
the obtained compactnesses are of particular interest for new
detailed assessments of various prognostic factors of carcinoma
of the uterine cervix.
The scheme we have introduced emphasizes 3-D, but
requires a very extensive procedure. What will be subject
of further investigations is the possible applicability of the
discrete 2-D compactness referring to single sections as an
alternative solution. Possibly, satisfactory conclusions from
2-D to 3-D compactness-based assessments can be made so
that the expensive 3-D reconstruction can be circumvented for
using a well established
The authors are indebted to R. Scherling who carefully ac-
complished all sectioning and staining essential for this work.
They thank MICROM GmbH for kindly providing them with
the electrically cooled object clamp “Cool-Cut” and the sec-
tion transfer system “STS,” both decisively facilitating and im-
proving the sectioning of the large histological serial sections
throughout this work. They would also like to thank T. Frohn
for the digitizations. Finally, the authors would like to thank the
referees for their careful and constructive suggestions.
 L.-C. Horn, U. Fischer, and K. Bilek, “Morphologic factors associated
näkologie, vol. 123, pp. 266–274, May 2001.
 J. Baltzer, K. J. Lohe, W. Köpcke, and J. Zander, “Histological criteria
for the prognosis in patients with operated squamous cell carcinoma of
the cervix,” Gynecologic Oncol., vol. 13, pp. 184–194, Apr. 1982.
 V. Maaßen, R. Dieckhoff, J. Baltzer, and G. Kindermann, “Risk of
hematogenous metastasis of cancer of the uterine cervix,” Zeitschrift für
Geburtshilfe und Frauenheilkunde, vol. 54, pp. 656–661, Dec. 1994.
 G. B. Kristensen, V. M. Abeler, B. Risberg, C. Trop, and M. Bryne,
“Tumor size, depth of invasion, and grading of the invasive tumor front
are the main prognostic factors in early squamous cell cervical carci-
noma,” Gynecologic Oncol., vol. 74, pp. 245–251, Aug. 1999.
 P. J. Tadrous. (2002) Methods for Imaging the Structure and
 B. Luck, K. Carlson, T. Collier, and K.-B. Sung, “Confocal microscopy
[detecting and diagnosing cancers],” IEEE Potentials Mag, vol. 23, no.
1, pp. 14–17, Feb. 2004.
 S. M. Boles, H. Hricak, and P. Rubin, “Carcinoma of the cervix and
endometrium,” in Oncologic Imaging, 2nd ed, B. G. Bragg, P. Rubin,
and H. Hricak, Eds. Philadelphia, PA: Saunders, 2002, ch. 26, pp.
 Y. Okamoto, Y. O. Tanaka, M. Nishida, H. Tsunoda, H. Yoshikawa, and
Y. Itai, “MR imaging of the uterine cervix: Imaging-pathologic correla-
tion,” Radiographics, vol. 23, no. 2, pp. 425–445, Mar. 2003.
sections,” Nature, vol. 236, pp. 207–210, Mar. 1972.
 S. J. Young, S. M. Royer, P. M. Groves, and J. C. Kinnamon, “Three-
dimensional reconstruction from serial micrographs using the IBM PC,”
J. Electron Microsc. Tech., vol. 6, pp. 207–217, 1987.
 S. Ourselin, A. Roche, G. Subsol, X. Pennec, and N. Ayache, “Recon-
structing a 3-D structure from serial histological sections,” Image Vis.
Comput., vol. 19, no. 1–2, pp. 25–31, Jan. 2001.
 É. Bardinet, S. Ourselin, G. Malandain, D. Tande, K. Parain, N. Ayache,
and J. Yelnik, “Three dimensional functional cartography of the human
basal ganglia by registration of optical and histological serial sections,”
in Proc. IEEE Int. Symp. Biomedical Imaging (ISBI’02), Washington,
DC, Jul. 2002, pp. 329–332.
 T. Schormann and K. Zilles, “Limitations of the principal-axes theory,”
IEEE Trans. Med. Imag., vol. 16, no. 6, pp. 942–947, Dec. 1997.
, “Three-dimensional linear and nonlinear transformations: An in-
tegration of light microscopical and MRI data,” Hum. Brain Mapp., vol.
6, no. 5–6, pp. 339–347, 1998.
 M. S. Mega, S. S. Chen, P. M. Thompson, R. P. Woods, T. J. Karaca,
A. Tiwari, H. V. Vinters, G. W. Small, and A. W. Toga, “Mapping his-
tology to metabolism: Coregistration of stained whole-brain sections to
premortem PET in Alzheimer’s disease,” Neuroimage, vol. 5, no. 2, pp.
147–153, Feb. 1997.
 T. M. Lehmann, C. Goerke, A. Kaupp, W. Schmitt, and R. Repges, “A
rotation-extended cepstrum technique and its application to medical im-
ages,” Pattern Recognit. Image Anal., vol. 6, no. 3, pp. 592–604, 1996.
 W. S. I. Ali and F. S. Cohen, “Registering coronal histological 2-D sec-
tions of a rat brain with coronal sections of a 3-D brain atlas using geo-
metric curve invariants and B-spline representation,” IEEE Trans. Med.
Imag., vol. 17, no. 6, pp. 957–966, Dec. 1998.
 J. B. A. Maintz and M. A. Viergever, “A survey of medical image regis-
tration,” Med. Image Anal., vol. 2, pp. 1–37, Mar. 1998.
 F. Maes, D. Vandermeulen, and P. Suetens, “Medical image registration
using mutual information,” Proc. IEEE, vol. 91, pp. 1699–1722, Oct.
Oxford Univ. Press, 2004.
 L. H. Sobin and C. Wittekind, Eds., TNM Classification of Malignant
Tumours, 6th ed.New York: Wiley, 2002.
 M. Höckel, L.-C. Horn, B. Hentschel, S. Höckel, and G. Naumann,
terectomy based on developmentally defined surgical anatomy,” Int. J.
Gynecological Cancer, vol. 13, pp. 791–803, Nov. 2003.
correlation,” Appl. Opt., vol. 15, pp. 1795–1799, Jul. 1976.
 J. L. Horner and P. D. Gianino, “Phase-only matched filtering,” Appl.
Opt., vol. 23, pp. 812–816, Mar. 1984.
 Q.-S. Chen, M. Defrise, and F. Deconinck, “Symmetric phase-only
matched filtering of Fourier-Mellin transforms for image registration
and recognition,” IEEE Trans. Pattern Anal. Machine Intell., vol. 16,
no. 12, pp. 1156–1168, Dec. 1994.
 B. S. Reddy and B. N. Chatterji, “An FFT-based technique for trans-
lation, rotation, and scale-invariant image registration,” IEEE Trans.
Image Processing, vol. 5, no. 8, pp. 1266–1271, Aug. 1996.
 R. C. Gonzales and R. E. Woods, Digital Image Processing.
MA: Addison Wesley, 1992.
 M. McGuire. (1998) An Image Registration Technique for Recov-
ering Rotation, Scale and Translation Parameters. [Online]. Available:
 B. Cabral and L. C. Leedom, “Imaging vector fields using line integral
convolution,” in Proc. SIGGRAPH ’93), vol. 27, 1993, pp. 263–272.
 D. Stalling, “Fast texture-based algorithms for vector field visualiza-
tion,” Ph.D. thesis, Fachbereich Mathematik und Informatik der Freien
Universität Berlin, Berlin, Germany, 1998.
 E. L. Hall, Computer Image Processing and Recognition.
Academic, 1979, ch. 4.3.1.
BRAUMANN et al.: 3-D-RECONSTRUCTION AND QUANTIFICATION OF CERVICAL CARCINOMA INVASION FRONTS 1307 Download full-text
 L. G. Brown, “A survey of image registration techniques,” ACM Com-
puting Surveys, vol. 24, no. 4, pp. 325–376, Dec. 1992.
 Y. Amit, “A nonlinear variational problem for image matching,” SIAM
J. Scientific Computing, vol. 15, no. 1, pp. 207–224, Jan. 1994.
 B. Fischer and J. Modersitzki, “Curvature based image registration,” J.
Math. Imag. Vis., vol. 18, pp. 81–85, Jan. 2003.
 M. Abramowitz and I. A. Stegun, Pocketbook of Mathematical Func-
tions. Frankfurt, Germany: Verlag Harri Deutsch, 1984.
 P. Cachier and N. Ayache, “Regularization in Image Non-Rigid Regis-
tration: I. Trade-Off Between Smoothness and Intensity Similarity,” The
FrenchNationalInstitutefor ResearchinComputer Science andControl
(INRIA Sophia Antipolis), Res. Rep. 4188, 2001.
, “Regularization Methods in Non-Rigid Registration: II. Isotropic
Energies, Filters and Splines,” The French National Institute for Re-
Rep. 4243, 2001.
 P. Perona and J. Malik, “Scale-space and edge detection using
anisotropic diffusion,” IEEE Trans. Pattern Anal. Machine Intell., vol.
12, no. 7, pp. 629–639, Jul. 1990.
many: Teubner-Verlag, 1998.
 T. F. Chan, S. Osher, and J. Shen, “The digital TV filter and nonlinear
denoising,” IEEE Trans. Image Processing, vol. 10, no. 2, pp. 231–241,
 W. E. Lorensen and H. E. Cline, “Marching cubes: A high resolution
3-D surface construction algorithm,” Comput. Graphics, vol. 21, no. 4,
pp. 163–169, 1987.
, “Compact isocontours from sampled data,” in Graphics Gems III,
D. Kirk, Ed. Boston, MA: AP Professional, 1992, pp. 23–28.
 J.-P. Kuska, “Volume rendering with MathGL3d,” in Challenging the
Boundaries of Symbolic Computation, Proc. 5th Mathematica Symp., P.
Mitic, P. Ramsden, and J. Carne, Eds., 2003, pp. 341–348.
, (1997–2005) MathGL3d—Interactive OpenGL viewer for
 N. Singh and S. Arif, “Histopathologic parameters of prognosis in cer-
vical cancer—A review,” Int. J. Gynecological Cancer, vol. 14, no. 5,
pp. 741–750, Sep. 2004.
 E. Bribiesca, “A measure of compactness for 3-D shapes,” Comput.
Math. Applicat., vol. 40, pp. 1275–1284, Nov./Dec. 2000.
 P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier Detec-
tion.New York: Wiley, 1987, ch. 5.4, pp. 208–214.
 G. Malandain and É. Bardinet, “Intensity compensation within series
of images,” in Lecture Notes in Computer Science.
Springer-Verlag, Nov. 2003,vol. 2879,Proc. Medical ImageComputing
and Computer-Assisted Intervention (MICCAI2003), pp. 41–49.