ArticlePDF Available

Imaging and Modelling from Serial Microscopic Sections for the Study of Anatomy



A system is considered for segmenting noisy intensity images and consequent three-dimensional object reconstruction from a set of planar contours. A new semi-automatic method for the extraction of contours from a sequence of cross-sectional images based on an active contour model (ACM) is proposed. The dynamic ACM proceeds along the sequence of cross-sections following a non-rigid motion, in accordance with the organ boundary. Image texture information is also employed in the model. Problems associated with topological reconstruction from planar contours are addressed, and several criteria promoting semi-automatic topological reconstruction are introduced. The proposed system is successfully applied to the processing of real data related to animal embryonic organs, proving that the system allows detailed modelling of irregular objects. The reconstructed models can be observed in wire-frame, solid, transparent or stereoscopic semi-transparent format. The human-computer interaction implemented in the procedure assists with problems of feature identification and object manipulation about an arbitrary axis.
the study of
R. Durikovid* K. Kaneda H. Yamashita
Electric Machinery Laboratory, Faculty of Engineering, Hiroshima University, Japan
Abstract-A system
is considered
segmenting noisy intensity images
and consequent
three-dimensional object reconstruction
from a set of planar contours. A new semi-
automatic method for the extraction of contours from a seouence of cross-sectional
images based on an active contour model (ACM) ls proposed.
The dynamic ACM
the sequence of cross-sections
a non-rigid motion, in accordance
with the organ boundary.
lmage texture
information is also employed in the model.
contours are addressed,
and several criteria
semi-automatic topological reconstruction
are introduced.
The proposed
system is successfully applied to the processing
of real data related
animal embryonic organs,
that the system allows detailed
of irregular
objects. The reconstructed models
can be observed
in wireJrame, solid,
transparent or
stereoscopic semi-transparent format. The human--computer
interaction implemented
assists with
of feature
and object
an arbitrary
Three-dimensional modelling, Topological reconstruction,
e atu re ide ntif i catio n
Med. Biol. Eng.
1998,36, 276-284
1 Introduction
ANAToMISTS oFTEN undertake the task of observing the shape
of a specific embryo organ contaminated by a test drug. The
primary source of information
during this observation
dure is a series of thin microscopic slices that render the inner
organs visible. For observing the overall shape of organs,
however, the slices
rather limited, as
they provide
2D information.
Conversely, 3D models,
give more complete
information about the shape of the structure. Reconstruction of
a 3D surface or volume model from 2D information is there-
fore an indispensable technique for better understanding
shape and the position
of an organ inside an organism.
Although 3D reconstruction is widely used in computer
tomography (CT) and magnetic resonance imaging (MRI), the
methods do not fulfil all the needs in the field of anatomy.
Anatomists seek
information about the exact overall shape
try to ascertain the features that build it. The resolution in the
CT and MRI methods is not high enough for the process
scanning the small structures used in embryology (around
4.5 mm height in our applications). For this reason, manual
sectioning followed by serial microscopy scanning is used. In
anatomical fields that implement
serial microscopy, an image-
registration method is necessary.
However, the type of hard-
Correspondence should be addressed to Dr. Kaneda; email:
ki n@ e m l. h i rosh i m a- u. ac.j
*On leave from Department of Computer Graphics and lmage
Faculty of Mathematics
and Physics,
Mlynskd dolina,
820'13 Bratislava, Slovakia.
First received
27 November 1995 and in final form 27 Januarv 1998
@ IFMBE:1998
ware used in the CT and MRI methods is designed
to minimise
the image-registration
Moreover, in the case of CT
and MRI, voxel (volume) representation is used mainly
portion and volume are primary, whereas
representation is most imporlant in dealing with the exact
shapes of anatomical structures.
2 Problems of 3D reconstruction and previous work
Several systems are available for reconstructing organs
from serial sections
and Macoowsrr, 1990;
et al., 1990} STERECON is a system
(MARKo and LnttH,
1.992; PrmINs et al., 1993) with the capability of using
stereoscopic contour tracing with a digitising pad. Another
well-known package
based on manual contouring was devel-
oped at Columbia University (Ar-reN and Lrvrxrnar-, 1990).
Manual segmentation
through the process
of tracking organ
lines via mouse and cursor is a time-consuming
task. Segmentation
algorithms for automating this
procedure are therefore desirable, but conventional image-
methods (Pnas and VBNBrsANopouLos, 1990)
using only local information are often inadequate.
2.1 Registration
The most
obvious solution in embryology
is to set common
external reference
marks in the paraffin block before section-
ing. Four
can be made with a laser
the object
to be
The marks
on each
then determine
by an
automatic image-registration
(BnoN and GRslarr-
& Biological
Engineering & Computing May 1998
LEr, 1992).ln a system
by CanlaoH.r
et a\. (1994),
microscopy images
are aligned using an interactive
digital blink comparator.
As one image is held stationary, the
rotates the other image,
with the stationary
and moving images
shown alternately
on a graphics
An illusion of movement
is created
when the images
the movement
is minimised
as the images
again brought into alignment.
2.2 Segmentation
In the work of Leylaarue (1990),
the movement
of living
on a planar
surface is observed
by processing
an anima-
et al.,1987) are
used to track the
moving cell and to follow the deformations
that occur as
cell moves.
et al. (1994] introduced
the methodol-
ogy of tracking boundaries of neural dendrites from serial
Here, the snakes
are used in an inter-
active technique consistent with manual tracking methods.
This interactive
automatically from
slice to slice, nor can it follow highly concave structures
those whose topology changes (branching or merging)
two successive
of the complexity of microscopic
of embryo
the aforementioned
only in reconstructing
such higher contrast regions
as the outer skin of an embryo.
The techniques
do not appear
to be useful in inner
areas. The
aim ofthis paper
is therefore to
introduce a semi-automatic method that can track both the
and the topology
ofan organ in a section-by-section
2.3 Reconstruction
and visualisation
The traces
can be displayed as
a set ofcontours or
by a triangulation technique to form a surface that
can be displayed as a solid object (KeNEDA et al., 1987).
Before the surface can be formed, it is essential to know how
the contours are grouped together (the contours must be
to each
other) in the formation ofthe object topology,
a step called
topological reconstruction.
The simple techniques
use the geometric relationships
between two contours,
such as
the overlapping area or the distance between them. Two
contours are then assigned if their distance is less then the
predefined threshold (Murr-nn and KrrNcBnr, 1993). Addi-
tional information available is the topological position of
contours on the slice (nesting of contours), formally repre-
by a tree of nesting
et al., l99l). The root
of the tree corresponds
to an imaginary outer contour (image
boundary) that envelops
all the other contours. The successors
of a vertex of the tree correspond
to the contours immediately
contained in the contour of that vertex.
the above methods fail to reconstruct holes,
which are considered
as a particular characteristic
of branch-
ing but which connect the contours with different topological
positions (different levels of nesting, see Fig. 4a) within the
slice. In our work. we extend the above ideas to automatic
topological reconstruction
of branches
and holes.
Supposing that the topology has been previously recon-
structed, then the next step to be solved is the problem of
filling the space
planar contours. The surface is most
often approximated by a set of triangles based on either a
global criterion, minimum surface area (Fucus et al., 1977),
maximum volume (KnerEI-, 1975)
or a local one, minimum
of edges
and Saonneenc, 1978).
For a global criterion, the problem of finding the optimum
solution is reduced
to a problem of finding the minimum cost
cycles in a directed toroidal graph. As global methods are
Medical & Biological Engineering
& Computing May 1998
unable to deal with branches,
been treated (EKouLE et al., l99l; Csor and panr. 1994)
the introduction
of an intermediate
is based
on reduoing
the problem
to a 2D interpola_
tion, by building as many intermediate
us n...irury
for a reconstruction
of the object
et at., 1996j.
Several methods using local criteria were irnplemented
they can be adapted to solve the branchiirg problern,
and their calculation
time is negligible compared
with global
However, they failed in some carefully designed
cases. The proposal
offered in this research
is therefore
most effective
among the available
local methods.
3 Data acquisition
Using routine
an embryo
with an ultramicrotome at an average
thickness of between 7
pm. The sections were further processed,
and then mounted
on glass.
of a certain magnification were taken by a camera
an optical microscope.
The images were then stored into
compact discs after they had been converted into digital
form with a resolution
of 720 x 580 pixels.
To reconstruct
a 3D strucfure
from the series of cross-
sections, it is necessary
for all sections
to share
a common
in other words,
they should be
The most obvious solution is to use common
external reference marks set into the paraffin block before
and Grcuu-rET, 1992).
some important image data used in our experiments were
recorded some time ago, when no reference marks had been
a manual
image registration process
was employed.
3.1 Image registration
Sections from serial microscopy are not only translated or
rotated. Often they are deformed by the heating required to
the tissue for display on microscopic slices. Thus,
requires a combination
of linear transforma-
tion processes
to bring the sections into approximate
ment. We therefore adopted a combination of two methods:
the landmark method (BnowN, 1992) given by a user at
distinctive points; and the colour combination method, used
to conform quickly if the results are acceptable. These
methods have been employed separately
in different systems
with limited success.
-Howevei theii combination brings
together reference knowledge about sections and a quick
preview of results.
A pair of associated
were registered
by using two
selected on each section
at a distinctive point, such
as the point of a heart or the tip of a limb bud, indicated by the
X and * in Fig. l. The vectors
to each
cross-section, defined by the landmarks, can be used to
determine the translation and rotation of sections simply by
being aligned. For preview pu{poses,
initially, one of the full-
colour sections was tinted in green,
and the other was tinted in
magenta (see
Figs. la and b). While the user manipulated the
the colours of the two sections
were combined to
show the overlapping regions in some shade
of grey, as shown
in Fig. lc. The resultant
colour differences
were instructive
with regard to moving the landmarks
to achieve better section
registration. To provide better interactions with landmarks, a
preview transformation of colour-marked images was con-
ducted, but without using an anti-aliasing method. After
landmarks had been selected
correctly, the final, transformed
image was calculated
by taking into account the anti-aliasing
Fig. I Registration of two cross-sections of mouse embryo. (a), (b) Static image and its successive image with selected landmarks (symbols X
and t) placed at neural tube and anterior tip of embryo. (c) Colour merging of transformed and static images. (d) Best registration
of image rotation
Fig. ld). The series of sections can be
by repeating the registration of image
pairs from top
to bottom, or vice versa.
4 Gontour extraction: the ACM model
We used a semi-automatic approach
that extends the well-
known active-contour model (KASS
et al., 1987) to produce a
method that tracks both the boundary and the topology of an
organ in section-by-section
fashion. The ACM provided sig-
nificant assistance to the user in the accurate location of
structure boundaries.
The sequence of slice images obtained
was generalised in a video, where each frame had a time
component. The user quickly traced a contour to approximate
roughly the strucfure boundary for a few key frames from a
sequence. The dynamic simulation then conformed the con-
tour to the true structure boundary and proceeded through the
entire time sequence to extract quickly a sequence of profiles
of the same structure.
An ACM is a deformable curve composed
of abstract elastic
materials minimising its potential energy. Consider an ACM
o(s, t) : (x(s,
i), y(s, r)), with a spatial parameter
s and section
number i defined on spaces
O and N, respectively. Let l(x,y)
denote the image intensifi at position (x,y). The potential
energy function of the ACM, originally defined in Kess er a/.
(1987), consists of two components. First, the deformation
energy is written as
E*,(u): w1ft)lo'(s)12
* w2(s)la"(s)12 (l)
where the first and second derivatives of contour position z;
with respect to a parameter s are denoted a' and o. The
weighting functions 4(s) control the tension and rigidify of
the contour over the space O. Secondly, the image energy
computed from image intensities l,(x,y) is written as
: Go x li{u) (2)
where Gor. denotes
the convolution with a smoothing Gaus-
sian filter of width o.
In our previous work lDumr<ovre et al., 1995), we solved
four problems not addressed in the earlier work of K-a,ss et al.
(1987) or CeRtBoN et al. (1994). First, we defined the area
energy Eo,"o, giving us a contour that tends to minimise its
delimited area against the length minimisation, as is found in
e/ al. (1987) and Te,Rzoporos
and MeraxRs (1991).
Secondly, the structures in our sections are expected
to maintain average texture properties.
Therefore the problem
ofpoorly contrasting images is solved by a texture representa-
tion E,"rrr" of the image against the representation of the
image scale space only. Thirdly, the profile of a structure
consists of multiple regions is found using a contour-splitting
operation. Fourthly, the problem until now was to select model
but, with our approach, the parameters
the elastic process can be set automatically, based on the
initial estimation of the structure boundary. The total energy
functional as proposed by Dururovte et al. (1995) is then
expressed by
E,nokn: " l{Ei",@) * V{/1Ei*r"(a)
+ WrE,o,u,"1u1
* W1Eo,"o(u)lds (3)
The weight functions Wr, Wr and Wu control the image,
texture and area energy, respectively.
4.1 Contour operations
The most fundamental contour operations of adding and
deleting a single point are introduced- If the distance between
two neighbouring points is less or greater
than a given t}resh-
old, a single point from or between them is either deleted or
added" For the entire contour, dividing the contour into n parts
is proposed. Provided that the contour points are in a near-
equilibrium configuration with respect
to the internal energy,
the contour is divided into two parts using the segment
points z, and u, connected by vectors rii, rii, satisfu-the
followins conditions:
' lai
- oj1
- d*,
where dru, is a constant.
o The projections of vectors YEo,"o at points o, and o, onto
the vectors ru and 1r, respectively, have opposite direc-
tions (see
Fig. 2).
Medical & Biological Engineer-ng & Computing May 1998
Fig. 2 Conditions
for cutting ACM into two parts
Fig. 3 Automatic
of structure
boundary through
of sections
Moreover, a contour is divided recursively into n parts, if
n - 1 pairs of points obeying the above
conditions exist. Fig. 3
shows the significance of this property for speeding up the
tracing ofprofiles through the sequence ofslices ordered along
the time axis. The initialisation of a contour position for a
single (bottom) slice has been traced manually, whereas all
other contours were obtained automaticallv bv our ACM.
5 Data structure and topology
A hierarchical polygon data structure consists of objects,
sub-object surfaces, contours, and contour points. A simple
example is shown in Fig. 4a. To identify branches uniquely,
each contour line is associated with a sub-object surface. The
surface is part of the structure
that includes neither
branch nor confluence. The object consists of several sub-
object surfaces,
but only the connections
between them create
the branching (see Fig. 46). Therefore, if a pair of contorrs
lying in the same
belong to different sub-object
a branch must exit between the corresponding
A hole in the surface of the object is represented
as a
branching (see Fig. 4). For example, representation
of a vein passing
through the hole is shown in Fig. 4b, where
the inner and outer surfaces of the vein form two different
objects Ob6 and, Ob6, each consisting only of a single sub-
object surface. On the other side, the object with the hole
of five sub-object
...,,Sos7), forming
two branches
on the top and the bottom of the hole.
6 Automatic topological reconstruction
The problem of topological reconstruction is particularly
difficult if there are several contours on each
plane, and if it is
known that the coresponding sub-object
can have branches or
Medical & Biological
& Computing May 1998
Sos, Sos,
Fig. 4 Hierarchical data structure and corresponding assignment
graphs. (a) Three objects reconstructed front four cross-
sectional images: Ob1 consisting of sub-object surfaces
Ob6 and Ob6, each consisting
of a single
sub-object surface Sosl and Sos2, respectively. (b) Assign-
ment graph for objects shown in (a), where solid lines
connect contours of a single sub-object surface, and
broken lines denote connection between
sub-obiect surfaces
forming hole
Both geometrical
and topological
two contours
are used to solve the problem of topological
If there is a sufficient
of cross-sections
for geometrical reconstruction, the proposed topological
method also works well.
6.1 Assignment
Let us consider a sequence
of parallel planes in space
consisting of a collection of non-intersecting, 2D contours.
The aim is to find the relationships
between the contours such
that they will group into the sub-object surfaces and, finally,
after determination of branches and holes, into the entire
object. The topological reconstruction creates an oulput in
the form of an assignment
graph. The vertices of the assign-
ment graph correspond to contours, and the edges connecting
the vertices form connected
parts of the object, as shown in
Fig. 4b. When considering the reconstruction between two
different types
ofconnection are
These three comprise a connection between two contours (a
cylindrical connection), a connection between more
than two sub-object surfaces forming an open branch, and a
between the sub-object surfaces forming a hole (a
brarich see Fig. 4).
The proposed automatic construction of an assignment
graph can be summarised in two essential steps. First, the
tree (SnrNecAwA et al., l99l) was created
for each of
two successive sections. Secondly, pairs of contours, each
from different cross-sections,
were assigned
to each other if
they satisfied a connectivity criterion (see definition below)
and occupied the same level in the nesting tree. As multiple
cylindrical connections can generate branches, they were
further examined based on the criteria for open branches. At
this point, the cylindrical connections
and open branches were
determined. The connected branches (holes) were given the
lowest priority, and therefore the criterion for this kind of
branching was examined
as the last step. The criteria used for
cylindrical connections between contours and for the two
kinds ofbranch between sub-obiect surfaces
are as follows.
6.2 Connectivitlt criterion
Recently, MULLER
and KrrNceRr (1993) proposed criteria
for assigning two contours, based on the average distance
between contours P and,
Q, having ly'p and Nn number of
Sos, Sos6
Sos, Sos,
obB obc
The contours
are assumed to be planar
and belong to different parallel planes.
The asyrnnrctric
between P and
Q is then expressed as
t Nr
q: ;+ Itp, - plt
Llzl\ p :-
where the point Qi e Q is the nearest
point to P, e P, and A--
denotes the distance
between two sections.
Afterwards, two
contours are assigned
when l[J(P,9)]-' is greater
than a
threshold value.
Note that, for a particular
of i, Q, can
appear more than once in the summation. it is also worth
noting that 6(P,O is without dimension and not metric.
Unfortunately, ,X., .) satisfies only one of a totai of four
metric conditions, namely,
the non-negativity
condition. The
been used extensively
and works well for partially
It is not sufficient, horvever, for con-
having no common area
or of very different
To solve this
two quasi-metrics
are proposed
in our approach,
one measuring the separation
and other measuring
the relative overlap between two con-
Defnition -1.'
Suppose two non-self-intersecting planar
tours P and Q, having /y'p and ly'n number of points and
delimited areas A(P) and A(Q), respectively.
Their separation
is defined by
L(P,q:;Q*t6(P,Ol'-[a(0,p)] ') (4)
and their relative
overlao is measured bv
::ry* +:!9: (5)
2 A(P\ 2 A(Q)
where A(Int\ is the area of intersection
contours P
and Q.
note that, for any P and Q, L(P,q e [0, 1] and
A(P,O [0, 1], respectively.
Both I and A are symmetric.
Separation I has a value
of0 when contours P and Qhavethe
shape and are close to each other. Similarly, the relative
overlap A is zero for no overlap and I for complete overlap.
When experimenting with Z and
A, we realised that, for
large contours,
the criterion
ofrelative overlap should be used
primarily, whereas, for small contours, the separation
is preferred. What contour is considered small is strongly
on image size and application area.
Defnition 2.' A non-self-intersecting
contour P lying inside
an image t having area A(P) and contour length i" is
to be small relative
to the image 1, if
:'# . T, (6)
A\I )I I
l(1) is the image
area, and
I is the length
of the image
In the above definition, S(P) -+ 0 if the area
delimited by P
is small, similarly S(P) : I for the largest
i.e. the
contour identical with the image border. A threshold value
4 e [0, 1] was selected based on the application field by
selecting the representative
set of'small' contours and calcu-
lating the maximum of their respective value of S(P). The
following definition of a connectivity
criterion accommodates
the idea of selecting one of the quasi-metrics
A or L, based on
the relative size
of contours.
J: The connectivity criterion for two contours
p is a quasi-metric
defined as
&e'q+llf;Ot |
if .s(P),
: 4
4ffiO {*ft"" P'
itttP,Q't+t-A(P.q\ if .t(P), S(Q)
- r,
6.2.1 C)tlindrical connection and open branch: Al1 pairs of
contours from consecutive
both occupying the same
level in the nesting tree and satisfying C(P,q < Q were
assigned to each other in an assignment
These criteria
reconstruct both the cylindrical connections and open
6.2.2 Connected branch'. As the criteria for assigning the
connected branch (a hole), the small overlapping
area and
the distance between
the inner contour and the one on the
following or preceding
section are used
Fig. 5). In Fig. 5,
the two contours A and C lying at different levels within the
nesting tree are connected to a single contour B of ttre
adjoining section. It is intuitively assumed
that the shape
the hole C is similar to the open part of .8. Based on this
reasoning, the criteria for assigning the connected
the three contours A, B, and C are the mbasure
of the largest
overlapping area and the smallest distance between the hole
and the open
Additionally, contours A and
B must satisfy
the criteria for cylindrical
connection. When there
open parts in C, all parts are compared with the hole
and the best match is selected.
A fully accurate
reconstruction from contours
is not always
of many valid possibilities
that are unaccep-
For example, ernploying automatic
topology reconstruc-
tion on our data sets of a mouse embryo and a frog's organs,
the program
correctly reconstructed 90-94% of all branches.
The remaining or effoneous
branches had to be corrected
manually by an expert
in the field.
7 Reconstruction from contour data
7.1 Cylindrical connection
Let us consider the problem of filling the space
pair of planar contours. In the proposed system, several
methods using local criteria were implemented to solve the
filling problem,
as they can be adapted to solve
the branching
problem and have negligible calculation time compared
global methods. However, they failed in some carefully
The proposal
discussed in this Section
is the
most effective selection
among available local triangulation
The best selection can be made based
on one of the
open area
Fig. 5 Measure that assigns connected branch to three contours
based on similarity between open part and hole C
Medical & Biological Engineering
& Computing May 1998
such as the minimisation
oftotal surface
or the maximisation of volume
of the reconstructed
We have considered
the shortest length of spans as a
good choice
for a selection criterion.
A span is the edge of a
triangle that does not belong to either planar
The filler triangular patches
using two local
The first was the Christiansen local method
and SeoERenRc,
which is effective for con-
tours with similar shapes,
but ineffective when the shapes
(see Figs. 6a--c). The second method is a modification of the
above method that was designed to reconstruct
rapid changes
in shape
Figs. 6d-ll:
(i) Starting
from an initial base span ab, a new base
span is
(ii) For the two forward points
d and F6, the nearest
Nro and Nou in the neighbourhood
of the adjoined contour are
Fig. 6@.
(iii) Comparing the length of spans NooFo uni lyr,fi the
shorter span is taken as the new base span.
(iv) The triangles between the old and new base spans are
then generated
to minimise the total area of triangulation in
this area. Next, the new base span becomes the initial base
span, and the process
is repeated from step i.
Using this modification,
such as those in Fig. 6e can
be generated.
The Christiansen
method fails in this type of
case, as
in Fig. 6b. Conversely,
however, the modified
method fails in Fig. 6f, when two contours are thin, even if
they are of the same shape. In contrast, the Christiansen
method engenders
good results in this case. The combination
of the above methods represents a good solution, as shown in
Figs. 69 and ft. Using more than two local methods can greatly
improve results, a significant payoff for the additional time
that must be invested in computation.
7.2 Open branching
Tire multiple-branching
is often
by introdu-
cing an intennediate contour
and by splitting the
several double-branching
Let the area between
multiple contour be called a channel
and the inner chain of
triangles filling this channel be a channel polygon. The
entering and exiting edges of the channel are called the
bridge segments. Our particular focus is qn_lhe 'opt1lq4ll
determination of bridge segments, denoted B2rB2, and BtrAIo
in Fig. 7. When the bridge is found,
reconstructing the channel
by horizontal triangular
within bridge segments is the
simplest method.
Finding the bridge as a convex hull of the convex
hulls of
two contours, as
by CHot and Panr (1994),
is often
as the channel area is larger than actually necessa-
rily. First, in this approach, the convex
hulls of two contours
are identified, and then the convex hull of those convex
is calculated. There are two new segments
(bridge segments)
having end points in the different convex hulls of polygons.
We call this solution the maximum bridge because
of its
maximum channel area. The minimum segment between two
contours determines
the minimum bridge. Examples of a
maximum bridge denoted by segmetts M/M2o and MIML
and of a minimum bridge noted
by mg, mD are shown in Fig.
7. The 'optimum' selection of the bridge lies somewhere
between the rninimum and maximum bridges. The candidates
for bridge segments are all segments (Xc, Yd having
X, e lM/M[] and Y, etu[tr'l'o]. To speed up the search of
bridge segments, it is sufficient to search only the set of
segrnents filling the channel determined by the maximum
bridges generated
by the Christiansen method. The optimum
bridge is one in which the contour joined through the bridge
segments is similar to the contour in the next section. Two
contours were considered similar (GoNzelez and WINTZ,
total length=4.557
proposed method
total lengith=6.815
c total lenglh=3.625 t total length=8.961
Fig. 6 Generation of triangles: (a) Christiansen algorithm. (b), (c) Failure and success of Christiansen algorithm- (d) Modifed algorithm. (e)
(fl Success and failure of modified algorithm. (d, @ Proposed method: selection based on span length between Christiansen and
modified method
Medical & Biological Engineering & Computing May 1998 281
Fig. 7 Findinq o{ bridge points on brctnch
fi.on ubot'e.
and Bt Bto are automoticall,v generated
bridge segntents
when the factor 12
I@EA) had a similar value for both
contours, where / and A are the perimeter and area, respec-
tively, of the surrounded
8 Results
The system described for the reconstruction
and analysis of
optical microscope sections reduced the effort and time
required to reconstruct embryological structures from a few
months to only several days. As an example, a mouse embryo
(4.45mm height) was sectioned with an average section
thickness of about 7 pm using an ultramicrotome. The 636
obtained were further processed.
The proposed ACM was applied by the authors to the
location of target organs within the microscopic images and
to the three-dimensional reconstruction of a mouse embryo.
The new ACM offers a way to reduce the sensitivity of active
contours to localised noise. The method is shown to be
effective for refining estimates of an object's shape and
location and tracking any topology changes. The outlines of
organs that consist of multiple branches or holes can be
followed automatically from slice to slice. This phenomenon
occurs within the series
of cross-sections
as the splitting of a
contour when there is a different number of contours in two
The result of automatic topology reconstruction is demon-
strated in Fig. 8. These data contain several open and con-
branches, and, in the worst case,90Yo
(total on average
94%) of them were reconstructed
correctly when measuring
for each organ separately.
f for successful topo-
logical reconstruction
was calculated
as the maximum value of
S(P) on a representative
set of small contours; the result was
4 : 0.03113.
Another set of sections
used in the study was that of a frog's
area. This set was
because ofthe complicated
of a frog's semicircular
canals. The set consisted of
64 cross-sections with a thickness
of 35
pm. Outlines of the
semicircular canals were extracted from the sections by
initialising outlines for only a single section within the
entire set. With methods using only geometrical information,
reconstructing the topology of the obtained contour data is
extremely difficult, if not impossible.
the proposed
reconstruction method reconstructed the topology of the
Fig. 8 Reconstructed topolog,t of 54 objects oJ'mouse embryts. Qy",
structure is neural tube, light brown stntcture in middle is
heart tyith blood vessels coloured dat* brown. Stomach is
structure behind heart, coloured mapenta
strucfure without any misassignments.
as seen in Fig. 9b.
when the triangular patches
are generated,
parent and semi-transparent
et al., 1987) visualisa-
tion techniques can be applied. The transparent
of the heart and blood vessels
is shown in Fie. 9a.
9 Conclusions
In this paper,
a system for extracting contours from multiple
cross-sections and techniques for reconstruction of three-
dimensional structures have been proposed. A new adaptive
ACM was implemented in the proposed
system. The proposed
ACM can automatically follow slice to slice and can adapt its
shape to highly concave
or to those whose topology
changes (branching or merging) between fwo successive
sections. The process of topology reconstruction was
improved by incorporating information about the nesting of
contours, and by the definition of quasi-metric criteria regard-
ing when to assign two contours to each other in an assign-
ment graph.
Methods for topology reconstruction of both open and
branching have been proposed.
At the reconstruc-
tion step, the space between two contours was filled by
triangular patches resulting from the optimum solution, i.e.
the most effective solution among several local optimum
methods. The reconstruction of open branches can be pro-
cessed more accurately using discrete optimisation of the
bridge location limited to the minimum and maximum
The proposed system was successfully applied to the
processing of actual data related to human and animal
embryonic organs, proving that the system allows detailed
modelling of complicated
authors wish to thank Professor Mineo
Yasuda and Professor
Akinao G. Sato for their advice and for
the sample data of a mouse
We also wish to
thank Professor Yasuo
Harada for inspiration
tion of a seniicircular
canal. which has
in both
and 3D slnrctures.
and also ior providing
the cross-
section images.
Our thanks
also to the reviewers for their helpful
Medical & Biological Engineering
& Computing May 1998
Fig.9 Transparent visualisation ofreconstructed structures. (a) Mouse embryo heart and blood vessels. (b) Contplicated topologlt ofleft and
right frog semicircular canals
B.A., and LovrNrHAL,
'CARTOS II semi-automated
nerve tracing: three-dimensional
from serial section
Comput. Med.
Graph.,14,pp.319 329
C., and Gnetttr-ler, P.
'3-D reconstructions
by image-
of serial
sections in electron microscopy',
ir? KRIETE, A.
(Ed.): 'Visualization
in biomedical microscopies:
3-D imaging and
computer applications'
(VCH Publishers,
pp. 75 105
BRowN, J.P. (1992): 'A survey of image registration
pp. 321376
CHot, Y.K., and Pnnrc, K.H. (1994): 'A heuristic triangulation
algorithm for multiple planar contours using an extended double
pp. 372-387
H.N., and Seoeneenc, T.W. (1978): 'Conversion
complex contour line definitions into polygonal element mosaics',
Comput. Graph., (SIGGRAPH '78 Proceedings),
pp. 187-192
CARLBoM, I., TERZopouLos,
D., and H.rnnrs, K.M. (1994): 'Com-
registration, segmentation, and 3D reconstruction
from images of neural tissue sections',IEEE Trans. Med. Imaging,
- 13,
pp. 351
Dunrrcovri, R., K-q,Nnpe,
H. (1995): 'Dynamic
contour: A texfure approach
and contour operations', Vis. Comput.,
ll. pp. 217-289
EKouLE, A.B., PEyRrN, F.C.
and Ooer, C.L. (1991):
'A triangulation
algorithm from arbitrary shaped multiple planar contours', ACM
Graph.,10, pp. 182-199
FucHS, H., KeoeN, 2.M., and.
UsELroN, S.P. (1977): 'Optimal
surface reconstruction from planar contours', Communi. ACM,
20, pp. 693-102
and WlNrz, P. (1987):
'Digital image
edn', (Addison-Wesley Publishing Company, Reading, Massachu-
E., YASUDA, M., and Saro,
A.G. (1987): 'Reconstruction and semi-transparent
display method
for observing inner structure of an object consisting of multiple
Vis. Comput.,3, pp. 137-144
KASS, M., WrrKrN, A., and TEnzopout-os, D. (1987): 'Snakes:
Active contour models', Int. J. Comput. Vis., 4, pp. 321131
KEPPEL, E. (1975): 'Approximating complex surfaces by triangula-
tion of contour lines', IBM J. Res. Dev., 19, pp. 2-ll
KRtErE, A., and MAGDowsKr, G. (1990): 'Computerized three-
dimensional image processing
of serial sections in electron micro-
scopy', Ultramicroscopy,32, pp. 48 54
F. (1990): 'Tracking and describing deformable objects
using active contour models', Master's
thesis, McGill University,
Montreal, Canada
MARKo, M., and LErrH, A. (1992): 'Contour-based
3-D surface
reconstruction using stereoscopic contouring and digitized
in Knle,te,
A. (Ed.): 'Visualization
in biomedical
Medical & Biological Engineering & Computing May 1998
3-D imaging
and computer
applications' (VCH publishers,
Weinheim), pp.
45 73
H., and KUNGERT,
A. (1993): 'Surface
cross sections',
H., and MULLER,
on scientific
Heidelberg), pp. 139 189
OLrvA, J.M., PERRIN, M., and CoeurLLARr, S. (1996): '3D recon-
struction of complex polyhedral shapes from contours using a
simplified generalized
Voronoi diagram', ir RossrcN,qc, J., and
SIlr-roN, F. (Eds.): 'EUROGRAPHICS '96' (Blackwell Publish-
pp. 397 408
PITAS, I., and VENErsANopouLos,
A.N. (1990): 'Nonlinear digital
filters: principles and applications' (Kluwer Academic, Boston,
Dordrecht, London), pp. 27 6-344
PenrcrNs, W.J.,
HASHMT, S., and JonoaN. M. (1993): 'Three-dimen-
sional computer modelling system for the study of biological
structures', Med. Biol. Eng. Comput.,31, pp. 557-561
Y., KrrNu,
T.L., and KERGoSTEN,
Y.L. (1991): 'Surface
coding based on Morse theory', IEEE Comput. Graph. Appli., ll,
pp. 66-78
Tetsuvr, H., TAKAOKT, E., Ouunn, K., and FuJrrA, H. (1990): 'A
new method for three-dimensional
reconstruction from serial sec-
tions by computer graphics using meta-balls: reconstruction
of the
hepatoskeletal system formed by Ito cells in the cod liver',
Comput. Biomed. Res., 23, pp. 3745
D. and METAXAS, D. (1991): 'Dynamic 3D models
with local and global deformations: Deformable superquadrics',
IEEE Trans. Pattern Anal. Mach. Intell., 13, pp.703Jl4
Authors' biographies
Roman Durikovid graduated from Comenius University Bratislava,
Slovakia, where he received MD and MS degrees in Numerical
Analysis, in 1989. He obtained his PhD from the Faculty of
Engineering of Hiroshima University, Japan, in 1996. He was a
visiting scholar at the Groningen University, Netherlands, in 1991.
He is currently a visiting researcher
and member of the computer
graphics group at Hiroshima University. His research interests
include image processing, 3D reconstnrction, artificial life and
He is a member
of the IEEE.
Kazufumi Kaneda is an Associate Professor in the Faculty of
Engineering at Hiroshima University. He worked at the Chugoku
Power Company Ltd, Japan, from 1984 to 1986. He joined
Hiroshima University in 1986. He was a visiting researcher in the
Engineering Computer Graphics Laboratory at Brigham Young Uni-
versity in 1991. His research
include computer graphics
image processing.
his BE, ME, and DE, in 1982,
and 1991, respectively,
from Hiroshima University. He is a
member of the ACM, IEE of Japan, IPS of Japan and IEICE of Japan.
is a Professor
at Hiroshinra University. Depaftment
of Engineering,
E,lectric Machinery Laboratory, where he teaches
out research
into the visualisation
of rra.gnetic
linite element ar-ralysis.
He received
his BE and ME. degrees
Electrical Er-rgineering
from Hiroshima University, Japan,
in 1964
and 1968,
and Engineering
in 19l'1
, from Waseda
Tokyo, Japan.
He was appointed
in 196g,
in 1978,
of the Faculty
of Engineering,
shirra {Jnive
rsity. He was an Associate Researcher
at Clarkson
York in l98l-1982. He is a member
the IEEE, ACM. the
IEE of Japan, the IECE of Japan,
the IpS of
244 Medical
& Biological
& Computing May 1998
... Computer-automated extraction of contours from biological objects of interest has been attempted by others to reduce the labor and subjectivity of manual contour tracing. Popular methods of selecting areas of interest use segmentation or edge detection algorithms (Toga, 1990;Montgomery and Ross, 1993;Durikovic et al., 1998). These algorithms attempt to select objects of interest based on their specific and uniform pixel intensity or by their distinct boundaries from the surrounding tissue. ...
Full-text available
We are interested in three-dimensionally visualizing soft tissues and their internal structures through a process of computer image reconstruction from serial histological sections. We demonstrated that three-dimensional (3D) reconstruction permits a level of anatomical evaluation not possible with conventional histology. Three-dimensional imaging allows one to graphically manipulate a tissue providing unlimited vantage points as well as the ability to peer inside biological features of interest. We primarily three-dimensionally visualized segments of spinal cord because of its particular pathology after injury. Necrosis occurs mainly in the central grey matter. Therefore, 3D reconstruction approaches permitted an unique view into the site of injury that has only been possible previously by two-dimensional (2D) sectioning. We have been interested in the morphology of cysts and the character of the lesion which forms in more chronic spinal cord injuries. Three-dimensional reconstruction allowed us to investigate the intricacy and spatial relationship of these structures to the rest of the spinal cord segment. To pursue these goals, we have evaluated and compared three different algorithmic methods to produce three-dimensional images from data sets of histological sections. Two of the novel algorithms we have employed allowed us to query the surface area and volume of the 3D surfaces we have reconstructed. These methods were compared to older 2D morphometric and formulaic quantitative approaches cited in the literature using injured and intact spinal cord data sets. We found 3D quantitation to be more conclusive at measuring complicated structures, like the spinal cord injury site, than standard quantitative approaches which make estimates from few data points. Our intention was to demonstrate the utility and practicality of 3D visualization in evaluating biological data. In our final study we applied one 3D technique to evaluate the efficacy of an experimental treatment for acute compression injury in the mammalian spinal cord. We used 3D reconstructions to examine and measure changes in the three-dimensional histopathology of control and experimental groups.
... Usually, the microscopic cross-sections are used to reconstruct the polygonal representation of an embryo, which is exact but complicated process. In case of such destructive approach often a mouse embryo is used instead of the human embryo [1]. To control the shape metamorphosis between two mesh objects become a problem when they have different topology and geometry. ...
Full-text available
The growth of the organs of human embryo is changing significantly over a short period of time in the mother body. The shape of the human organs is organic and has many folds that are difficult to model or animate with conventional techniques. Convolution surface and function representation are a good choice in modelling such organs as human embryo stomach and brain. Two approaches are proposed for animating the organ growth: First, uses a simple line segment skeleton demonstrated on a stomach model and the other method uses a tubular skeleton calculated automatically from a 2D object outline. The growth speed varies with the position within the organ and thus the model is divided into multiple geometric primitives that are later glued by a blending operation. Animation of both the embryo stomach and brain organs is shown.
Topology based watermarking method is proposed in combination with some unique feature extraction useful for retrieval of 2D/3D models such as those reconstructed from CT/MRI data. Watermark message is cut into several pieces and each piece of message is embedded at different spots so that if a piece of message is lost in one spot, the same information can be potentially retrieved from other spots by error correct decoding. The method compares height of the vertices of a triangle lying in the same layer. A watermark message is converted into a binary bit sequence, a parity bit is added with every consecutive 8 bits, and then embedded into the model in either way that the first vertex of a triangle in the upper level or in the lower level, that carries information 1 or 0, respectively. It is robust against translation, rotation, arbitrary re-sectioning, local deformation, scaling, and unauthorized alteration of a single bit in every consequent 8-bits length. It is useful for shape sensitive 3D geometric models. It left some unique artifacts even after local or global number rearrangement which is useful for data retrieval.
To reconstruct three-dimensional medical images in Internet based on Web and to achieve highly realistic display, getting two-dimensional image of human organs from ultrafast CT as sources, applying the volume rendering technique we rebuild and display three-dimensional images in Java Applet and Java Application program. This reconstruction can be run in Web browser on many different kinds of computers. The anatomic structure of human organs can be displayed clearly in reconstructed three-dimensional images, especially the gross morphology of the heart and the trail of the coronary artery. Three-dimensional medical image reconstruction in Web browser implemented by Java Applet is feasible, which will prompt clinical use of three-dimensional images. The solid conformation of human organs, especially the anatomic structure of the coronary artery, can be displayed by using three-dimensional reconstruction techniques, which may offer great references to clinic.
A new topology-based watermarking method is proposed to embed information in objects with layered 3D triangular meshes such as those reconstructed from CT or MRI data. The main idea of the method is to compare the heights of the vertices of a triangle lying in the same layer. A watermark message is converted into a binary bit sequence, and then embedded into the modelin such a way that the first vertex of a triangle in the upper level carries information 1, and the first vertex of a triangle in the lower level carries information 0. For experimental purposes, a watermark message is embedded in a mouse embryo model. It is robust against translation, rotation, re-sectioning, local deformation and scaling. It left some artifacts after re-arrangement of local or global numbering. It is useful for shape sensitive 3D geometric models.
The growth of the organs of the human embryo changes significantly over a short period of time in the mother's body. The shape of the human organs is organic and has many folds that are difficult to model or animate using conventional techniques. Convolution surface and function representation are a good choice in modelling such organs as human embryo stomach and brain. Two approaches are proposed for animating organ growth: the first uses a simple line segment skeleton demonstrated on a stomach model and the other method uses a tubular skeleton calculated automatically from a 2D object outline. Growth speed varies with the position within the organ and thus the model is divided into multiple geometric primitives that are later glued by a blending operation. Animation of both the embryo stomach and brain is shown. Copyright © 2002 John Wiley & Sons, Ltd.
Conference Paper
Topology based watermarking method is proposed in combination with triangle strip peeling symbol sequence embedding to embed information in objects with triangular mesh surface such as those reconstructed from CT/MRI data. The hybrid method enhances the performance in many aspects. With the integration of parity checking, it improves the robustness against unauthorized alteration of a single bit in every consecutive 8-bits of length. Watermark message is cut into several pieces and each piece of message is embedded at different spots so that if a piece of message is lost in one spot, the same information can be potentially retrieved from other spots by error correct decoding. The method compares height of the vertices of a triangle lying in the same layer. A watermark message is converted into a binary bit sequence, a parity bit is added with every consecutive 8 bits, and then embedded into the model in either way that the first vertex of a triangle in the upper level or in the lower level, that carries information 1 or 0, respectively. It is robust against translation, rotation, arbitrary re-sectioning, local deformation, scaling, and unauthorized alteration of a single bit in every consequent 8-bits length. It is useful for shape sensitive 3D geometric models.
The "Visible Animal Project" (VAP) is comprised of axial anatomic cryosections and corresponding CT and MR images of a mature dog. The digital database is used for the creation of three-dimensional computer graphics of canine anatomy. The technique of cryodissection is described in detail. The combining of the corresponding CT and MR images, and cryosections as well as the data processing for the creation of three-dimensional reconstructions is presented and examples are shown. For the first time a complete high-resolution three-dimensional database of a dog is available, which can be used as the base for further high quality three-dimensional reconstructions, similar to the "Visible Human Project" (VHP).
Using a new method derived from the 'visible human project' (Spitzer et al., 1996, Journal of the American Medical Informatics Association, 3, 118-130), we were able to establish a simple and low-cost tool which produces high-quality cryosections of macroscopic specimens down to 1-mm slice thickness, based on a milling process. For the first time, a macroscopic cryotome is available to veterinary anatomists, which can be used on cutting faces up to 25 cm high and 50 cm wide and with a minimal slice thickness of 1 mm without any gap. The method employs a modified wood circular saw. Recording of the cutting faces is carried out 'online' by a high-resolution digital camera. The process has been tested extensively and produces high-quality sections of very hard material (teeth) as well as of very soft tissues (brain). It is now possible in veterinary medicine to provide three-dimensional anatomical databases of high resolution and of tissue-specific colour as an additional tool for high-quality two- and three-dimensional anatomical reconstructions for use in science and education.
A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours, motion tracking, and stereo matching. The authors have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.
To improve reconstructive 3D electron microscopy novel methods are discussed to represent and process serial section images in a cuberille environment. This includes the analysis of the transfer characteristics of the image detection system, the use of laser-induced fiducials for deformation correction and alignment, the control of section thickness by EELS and the use of ESI to image thick sections.
An algorithm is described for obtaining an optimal approximation, using triangulation, of a three-dimensional surface defined by randomly distributed points along contour lines. The combinatorial problem of finding the best arrangement of triangles is treated by assuming an adequate objective function. The optimal triangulation is found using classical methods of graph theory. An illustrative example gives the procedure for triangulation of contour lines of a human head for use in radiation therapy planning.
A simple algorithm is presented for processing complex contour arrangements to produce polygonal element mosaics which are suitable for line drawing and continuous tone display. The program proceeds by mapping adjacent contours onto the same unit square and, subject to ordering limitations, connecting nodes of one contour to their nearest neighbors in the other contour. While the mapping procedure provides a basis for branching decisions, highly ambiguous situations are resolved by user interaction. The program was designed to interface a contour definition of the components of a human brain. These brain data are a most complex definition and, as such, serve to illustrate both the capabilities and limitations of the procedures.
In many scientific and technical endeavors, a three-dimensional solid must be reconstructed from serial sections, either to aid in the comprehension of the object's structure or to facilitate its automatic manipulation and analysis. This paper presents a general solution to the problem of constructing a surface over a set of cross-sectional contours. This surface, to be composed of triangular tiles, is constructed by separately determining an optimal surface between each pair of consecutive contours. Determining such a surface is reduced to the problem of finding certain minimum cost cycles in a directed toroidal graph. A new fast algorithm for finding such cycles is utilized. Also developed is a closed-form expression, in terms of the number of contour points, for an upper bound on the number of operations required to execute the algorithm. An illustrated example which involves the construction of a minimum area surface describing a human head is included.
Principles of Optics is one of the classic science books of the twentieth century, and probably the most influential book in optics published in the past forty years. This edition has been thoroughly revised and updated, with new material covering the CAT scan, interference with broad-band light and the so-called Rayleigh-Sommerfeld diffraction theory. This edition also details scattering from inhomogeneous media and presents an account of the principles of diffraction tomography to which Emil Wolf has made a basic contribution. Several new appendices are also included. This new edition will be invaluable to advanced undergraduates, graduate students and researchers working in most areas of optics.