Content uploaded by Roman Ďurikovič
Author content
All content in this area was uploaded by Roman Ďurikovič on Jul 22, 2014
Content may be subject to copyright.
lS'?sxmsxX
, \
Y
elt*letf
q,qlnrtluf*r
Dynamic
contour:
a texture
approach
and
1 Introduction
Powerful methods
for representing
the shape
of
a
region
of interest
are essential
towards image
understanding. A
contour represented
by the out-
line
of a region is
useful for
analyzing
the shape,
size, and motion
of an object.
To complete
the
shape information,
one can also
use color
or
texture. Conventional
edge
selections
(Pitas
and
Venetsanopoulos 1990)
based
on methods
using
only local information
are often inadequate
to
trace the outline
contour
of a region.
Physically
based techniques
using external
forces
and
smoothness restrictions
on
the selection
of the
region's
outline have
been
proposed
recently.
The
actiue
contour model
(ACM),
first introduced
in
Kass
et al.
(1987),
to
trace the outline
of a region,
has
been developed in
recent
years
and has
been
used in
several applications
(Terzopoulos
and
Fleischer
1988; Terzopoulos
and Metaxas
1991;
Turner
and Thalmann
1993), including
medical
imaging
(Cootes
et al. 1993;
Nastar
and Ayache
1993).
This
work aims at segmenting
2D
objects from 2D
images.
The
objects are
taken from a microscope
with
problems
such as noise
and
poor
contrast.
Objects
such as embryo
organs are
expected
to
tend towards having
ayerage
texture
properties.
One can take advantage
of these
tendencies
when
designing
a method
of
fitting
2D shapes
based
on
Newton's law
using the texture
approach
while
allowing contour
operations
such as contour
split-
ting. The ACM considers
an object
undergoing
elastic deformation
as a
set of masses linked
by
springs; the natural lengths
of
the springs are
set
to zero.
We also
propose
a method for
automatic
estimation
of some model
parameters
based on
the calculated histogram
of image forces
along
a contour.
In
this
paper
we demonstrate
the examples
of
image
segmentation
using a new
contour model
on mouse embryo
cross-secl.ions.
Concerning the
contour tracking
problem,
our
proposed
ACM
differs from,
but is related
to, the
one used in
previous
work by Kass
et at.
(1987),
Terzopoulos
and Metaxas
(1991),
and
Williams
and Shah
(1992).
We have
solved four
problems
not
addressed by their
work. First,
we define
an
area
force
giving
us
the contour
that tends
to
minimize its
delimited
area against
the length
minimization
as is found
in Terzopoulos
and
Williams. Second, the
problem
of a
poorly
con-
trasted image is
solved by
a texture representation
The Visual
Computer
(1995)
11:217 289
@
Springer-Verlag
1995
contour
operations
Roman
Durikovidl,
Kazufumi
Kaneda,
and
Hideo
Yamashita
Electric
Machinery
Laboratory,
Faculty of Engineering,
Hiroshima
University,
1-4-1,
Kagamiyama,
Higashihiroshima,
7 24, J apan
The large
morphometric
variability
in
biomedical
organs requires
an
accurate
fltting method
for
a
pregenerated
contour
model.
We
propose
a
physically
based
ap-
proach
to fitting
2D
shapes using
texture
feature
vectors
and
contour operations
that allow
even
automatic
contour
solit-
ting. To
support
shrinkage
of the .oniou.
and
obtain a
better fit for
the concave
parts,
an
area force is
introduced.
When
two
parts
of
the active
contour
approach
each
other, it
divides. The
contour
under-
going
elastic
deformation is
considered
as
a
set of masses linked
by springs
with their
natural lengths
set to zero.
We also
pro-
pose
a method
for
automatic estimation
of
some model
parameters
based on a histo-
gram
of image
forces
along
a contour.
Key
words: Elasticity
-
Active
contour
methods
-
Segmentation
Image
processing
Correspondence
lo: R. Durikovid
t
On
leave
from
the Department
of
Computer
Graphics
and Image
Processing, Faculty
of Mathe-
matics
and Physics,
Comenius
University, Bratislava,
Slovakia
277
-u=1iPgttt,*
t,orngrufer
of
the
image
against
the
representation
of the
at
each
contour
point.
We
impose
on
D a simplifi-
image
scale
spaie
only.
Third, the
outline
of
an
ed case
of
a deformation
energy
functional
written
object
that
consists
of
multiregions
can be
found
as:
by a
contour
splitting
operation.
Fourth,
the auto-
matic
setting
of
pari^eters controlling
the elastic
1
1
process
is shown.
This
is a
problem no,
O#iffi
EQI
:
r
l
{'tG)l''(s)l'
+
w2(s)lu"(s)12}ds
(2)
solved.
Groshong
(1992)
and
Staib
and
Duncan
(1992)
use
closed
contour
shape
models,
based
on elliptic
Fourier
representation
for applications
in medical
images.
However,
the basis
functions
used
cannot
represent
the
most complex
shapes,
such
as those
with
thin
appendages.
Celniker
and
Gossard
(1991)
and
Cohen
et al.
(1.992) implemented
finite element
models
of
flex-
ible objects.
However,
like
Fourier
representation,
the basis
functions
do
not have
the
desired
varia-
bility
for a
particular set of
shapes.
Revankar
and
Sher
(1993)
worked
with
a
closed
contour
shape
in
polar
coordinates
and searched
the
graph of
"locally
optimal
contour
locations."
The
method
is
very
local and
has
no
real suitabi-
lity
for a
large set
of optimal
locations.
2
Equation
of
motion
An
ACM
is described
by
its
positions u(s, /)
:
(x(s,
r),
y(s,
t)), velocities
H6,t),
and accelerations
#(t,t)
of
its mass
elements
as
function of
para-
meter
s
(0 <
s
<
1) and
time
f.
The dynamics
of
each
vertex
u(s,
f) according
to
Newton's
law
is
given
by
Lagrange's
equations
of
motion
(Lanczos
1,974)
A2u
Au
,
d1$.r)
+
?;(s,t)
+
Fi,t:
F"*,
(1)
where
pr(s)
and
y(s)
are the
mass
density
and
the
damping
factor,
respectively.
Force
F;n, is created
to
make the
contour
continuous,
while
F"', corres-
ponds to external
forces.
Both
4',
and
Fn,, depend
on
the
contour
position
u(s)
and
time
r.
2.1
Elasticity
The
internal
force
fln,
is derived
from a
non-nega-
tive
total
elastic
energy
E(u)
as
its variational
derivate
Fi,t
:
W
and
represents
the elastic
force
278
where
the
first and
second
derivations
of contour
position u
according
to
parameter s are
noted
as u'
and
u".
Thus d
is a
functional
with
the
weighting
functions
w;(s),
/
:1,2,
controlling
the
tension
and
rigidity
of
the
deformable
model
over
the
parametric
space
of
s.
1o
simulate
the
physical significance,
the use
of
Terzopoulos
et
al.
(1987) is suggested
to
define
w, and
w2
with
respect
to
the
natural
arc
length
9o
and
the
natural
curuature
ffu,
respectively'
Thus,
the weighting
functions
controlling
the ten-
sion
and
rigidity
can
be
fixed
as
follows:
wr(s):
9o(s)
-
g(s)
wz(s):.:{o(s)-tr(s)
where
9(s)
and
tr(s)
stand
for the
actual
arc
length
and
actual
curuature
at
active
contour
point
u(s),
respectively.
The variational
derivation
of the
deformation
en-
ergy,
Eq.
2, can
be derived
as
the
equation:
6E
d
..
d2
Frnt:;:
-
fr(w'(s)
u'(s))
*
fuWrtt\u"(s)).
(3)
2.2 Simplification
Equation
1 can
be simplified
while
preserving the
dynamics
by setting
the
mass
density
p(s)
to
zero
to
obtain:
Au
lfils,tl
+
Fi,t:
F",t.
g)
Experiments
in Sect.
6 demonstrate
the
sufficiency
of using
the simplified
ACM,
Eq.
4, in computer
vision
applications
involving
the
fltting
of a con-
tour to
image data.
Contours
yielded by
Eq. 4 rest
as soon
as
all the
applied
forces
vanish
or come
to
equilibrium.
-jl\isual
t,ompufer
2.3
Discretization
and
solution
To
solve
the
set of
differential
equations
in
Eq.
4 numerically,
we
discretize
the function
u(s)
ac-
cording
to
parameter
s
by
sampling
it
with
N
points.
The
result
is
the
N-dimensional
vector
denoted
&s
o
:
(x,y)
where
x and
y
are
discrete
representations
of the functions
x(s)
and
y(s),
re-
spectively.
The
single
node
of discretized
contour
o
is
denoted
as
ai:@t,!i):(x(si),
y(s;))
for
j:
1. ....N.
Similarly,
we discretize
the forces
Fin,,
F",,
as the
N-dimensional
vectors
F1*
and{'.
According
to
Eq.
3, the
discrete
vector Fin
may
be
written
in
the
matrix
form
Fi,r:
K'ts
(5)
where K is
an
N
x
N matrix
known
as
a stiffness
matrix.
This
yields
the
discrete
form
of
the equa-
tions
of motion:
da
T
*(t)
*
K'a
:
F",t
The
discrete
equations
of motion
may
therebv
be integrated
through
time
by explicit
methodi.
We implemented
both
the
sixth-order
Runge-
Kutta
method
and
the
flrst-order
Euler
method
(Berezin
and
Shidkov
1965).
For
obtaining
a
sufficiently precise
solution
in
the image process-
ing
applications
quickly,
the
Euler
method
is
sufficient.
Assuming
discretized
time
with
time
step /t,
the
Euler
method
updates
the
position
of
an
active
contour
u(s,
/)
according
to the
formula
t)t+lt
-
at
+
/ty-
L1F!,,
-
Kar1.
3 Externally
applied
forces
The
process
of
dynamic
active
contour
fitting
is
controlled
by
a field
of externally
applied
forces
F",,
formed
from
the
image
data. The
externally
applied force
is
therefore
a composition
of all
the
elementary
external
and
constrained
forces
acting
on the
active
contour
F",t:
W,i,
+
Wrf',
+
Wnio
0)
(6)
where
f
is ttre
image
force.
Texture
force
f'7
and
area
force
fo
in
Eq.
7
are
proposed
to
extract
edges
between
two
textures
and
minimize
the
ac-
tive
contour
area.
The
weight
functions
Wr,
Wr,
and
Wa control
the strength
of the
forces.
3.1 lmage
force
The first
term in
8q,.7,
ir,
it
the image
force
expressed
as the
gradient
of
the
potential
function
P(a(s)),
calculated
from
the
image
intensity
I(*,y)
AS
i,
:
Y
P(u)
where
Y
:
G+,$)r.
Suitable
potential
functions
for
various
applications
are described
in
Kass
et
al.
(1987)
and Terzopoulos
and Metaxas (1991).
Considering
the results,
we construct
the
potential
function
P(x,y):
w1I(x,y)
+
w6lV(G,xI(x,y))1, (g)
where
the
constants
wr e
(
-
1, 1)
and
we e
(-
1,0),
controlling
the
respective
contri-
butions
of image
intensity
and
the
gradient
of
intensity,
are
appropriate
selected
weights.
G"x,
denotes
convolution
with
a
Gaussian
filter
of
width
o ranging
from
1.0
to 4.0.
When
w, <
0 the
ACM
tends
to follow
the
bright
areas
in
the im-
age,
and
when
w7 >
0 the ACM
converselv
fol-
lows
the dark
areas.
To
calculat
e
i1
atthe location
(x, y),
we first
calcu-
late
the
discrete
gradients
VP,(i
:
l, ...
,4)
using
finite
differences
at the four pixels
that
surround
(x,y)
as
shown
in Fig.
1.
Then,
we use
bilinear
interpolation
of those gradients
to
calculate
a
gradient
VP(x,
y)
that is
defined
between
the
pixels.
3.2
Texture
force
In
the following,
a texture
force
is
derived
on
the
basis
of
texture
feature
uectors.
The
elements
of
these
vectors
are
given
by
a random
field
model
defined
at two
kinds
of nieghbourhoods.
A kind
of 2D
noncausal
random
field
model
called
the
simultaneous
autoregressive
model
(SAR),
has
been
used
by many
researchers
279
_
rrX,
t
ta
--v tstfatE
d rtllTlp$t*tr
l-1
,1
)
(o,1)
(1,1)
(-1,o)
x
(1,o)
-1
,-1
)
(o,-1)
1,-1)
2a
Fig. 1.
Calculation ofimage force/, as bilinear interpolation
between the surrounding
discrete
gradients,
VPb...,VP4
Fig. 2a,
b.
Texture lattices:
a neighbors of the
point
X; b The
lattice regions
covering the image in d
pixel-wide
steps
(Kashyap
and Chellappa 1983; Mengyang
et al.
1992). The
SAR model is
parsimonious,
i.e., it is
characterized by less
parameters
than the equiva-
lent
conditional Markov
(CM)model (Cross
and
Jain 1983). In this correspondence
we consider the
second-order SAR model
as an appropriate tool
for
efficient texture
description.
Assume
that the image I(x,y)
obeys the SAR
model in Eq.
9, with the lattice O
:
{
(x,}),
0
<
x,y
<
M
-
Ij, and
with associated neighbor set
N:
I(x,y)-
Fa:
I
@(i,j)ll(x@i,y@,r)
-
Fal
(t,
j)€N
+
^,/ilw(x,y)
(9)
In Eq.
9, @(i,
j),
((t,"r)
e N) and
pN
are unknown
parameters,
pa
is
the mean
of the
intensities
in O,
280
and w(') is a noise sequence
approximated by
a Gaussian random variable
with
zero mean
and
unit variance.
The
symbol'@'means addition in
module M.We refer the reader
to
Kashyap
and
Chellappa
(1983)
for a more
precise
description of
the SAR
model.
Two
types of SAR
models
used in our texture
description are defined on two kinds of neihbor-
hoods N1
:
{(1,0),
(0,1),(-1,0),(0,
-1)}
and
t[,
:
{(1,1),
(-
1, 1),
(
-
1,
-
1),
(1,
-
1)}
asshown
in Fig. 2a. Our interesr is in
{@(0.
1).
@(1,0).
pN,}
parameters
obtaining the horizontal and
vertical
texture
information from
the N1
neighborhood
and
in
parameters
{@(1,l).
@(
-
1.1),
pN,}
obtaining the diagonal texture
information
from
the N,
neighborhood. In
our approach,
para-
meters of the SAR are used in obtaining measures
for
the classification of textures.
3.2.1 Parameter estimation and the texture
feature
uector
We consider that the
notation
collAl
gives
us
a column representation
of
matrix A. In
this sense,
we represent the image intensities in neighbor-
hood
N as a column vector
Z(x,y):
colll(x
@
i,y
@/),
(i,i)
e Nl.
Thus, the least squares method
yields
the esti-
mates @ and
plof
the exact
parameters
@ and
p*
by the expressions
fM
LM 1
I
7
6:l
t \
ztx.vlZr(x.vll
t4-l
Lx-O
y-O
-l
/M-rM-l \
x{I
Lz(*,y)I(x,y)l
\x=0
y=0
/
1 M-lM-l
lrr,,,
fi
:
in
L I
Ut".
il
-
6Ztx.y)\2
tvr
x_o r_0
The consequent estimate 6 is a
column vector
with elements 61i,;)
written
as 6
:
coll@(i,),
(i,.i)
e Nl where the
neighborhood
N
:
Nr or Nr.
Definition l. Let 6,
fiu fi,,
bu the estimates
of
unknown in the SAR model
representing
the texture
lattice Q, and
let
ps
be
the mean in
this lattice.
-ri;KE {
,'_vlstlfll
q,GHn$lutsr
The
texture
feature
uector
on
e
is a
collection
of
estimates
denoted
by:
F
:
(fr,
...
,
fr):
(6(0,
\,
60,0),
@(1,
1),
6(
-
t,1),
i;,,
fi,,
t
o)
3.2.2
Segmentation
and
texture
Potential
Function
In
a
segmentation
process,
the image
is
scanned
from
top to
bottom
and
from
left
to right
by
the
M x
M lattice
region.
The
lattice
regions
cover
the
image
in
d
pixel-wide
steps
both
horizontally
and
vertically
overlapping
each
other
to
smooth
the
borders
between
texture
regions,
as Fig.
2b
shows.
For
each
of
these
regions,
we calculate
the feature
vector.
The places
where
texture
features
vary
dramatically
are
edge
locations
of
various
regions.
A normalized
Euclidean
distance
measure
iJ
used
as a
similarity
measure
between
texture
feature
vectors
! 7
tfi.
.
,"fi1
and
Fi
:
(-fi,
.
,fil
with
the following
definition
7
rt
u(Fi. Fit: f Vl
-fl)
'o?,,uir-+G# (10)
Therefore,
a
new
type
of
external
force
..texture
Jbrce",
f7,
can
be
expressed
as the
gradient
of the
texture potential
function
T(u(s))
calculated
from
the field
of texture
feature
vectors
as:
ir:V
161
We construct
the
potential
function
T
(u)
:
p(F,,
F;
) ftil
where
pr
is
the
Euclidean
distance
measure
defined
in
Eq.
10
and
F;,
F,
shown
in
Fig.
3 are
the
texture
feature
vectors
corresponding
to
both
the
auxiliary
point
V)
and
the
contour point
u(s),
respectively.
If
6
>
0 is
a
sufficiently
small
con-
stant
and
N. is
a
normal
vector
to
the
contour
qiv^en qt
point
u(s),
an
auxiliary
point
is
simply
deflned
by:
Vl(s):
u(s)
+
(N"
The
potential
function
has
large
changes
at
places
where
the
feature
vector
Vf
deviates
sufficlently
from
vector
d.
Active
Contour
4b
4c
Fig.
3. The
similarity
measure
between
the texture
feature
vectors
Fland
F, define
a texture potential
function
Fig. 4a-c.
Extraction
of a circle
sampled
by texture
on a rexture
background:
a
given
initial
contour;
b Final
contour
equilibrium
obtained
with
texture
edge
energy
only;
c contour
equilibrium
gained
in the
image
space
only
The
texture potential
Eq. 11
was
applied
to
the
active
contour
from
Fig.
4a,
resulting
in
the
equi-
librium
shown
in
Fig.
4b.
The
unsolvability
of
this
problem
using
only
image
space
or
scale
space
in
Eq.
8, i.e.,
images
smoothed
by
a
Gaussian
filter
considered
in
Carlbom
et al.
(1992),
Kass
et
al.
(1987)
and
Kita
(1992),
is
obvious
from
Fig.4c.
3.3 Area
force
Unfortunately,
the
dynamic
ACM given
by
Eq.
1
tends
to minimize
its
length
because
of
the
fiist-
order
term in
Eq.
2
controlled
by
wr(s).
This
property
in
general
does
not
allow
us
to fit
the
contour
to
the
concave parts.
To
avoid
this,
researchers
introduced
interactive
constraints
(shown
in Fig.
5a) for
some
contour
points
to
achieve
the
desired
contour
shape
(Carlbom
et al.
1992;
Celnlker
and
Gossard
l99I;
Terzopoulos
281
ra.[ )L t
-,.t
lstf**
(,rlrnp$feH
Thus, the forces
f)
are defined at each
point
r;
in
the sense of a discrete
approximation of Eq. 12
given
by the
expression
To
avoid
paftially
the
problem
of contour ori-
entation and the
problem
of self
intersection,
the
minimization
of Sfr(o) is required. Desired
forces
can be used
in
the
from
TL
JA:
Fig.
5a, b. Classical active contours
do
not
allow fitting to
concave
parts.
Two methods
describe how to avoid this
problem.
a addition of a
user defined constraint is
a well
known approach;
b
proposed
method using
the automatic
selection of area forces
and Metaxas 1991). These
constraints
are deflned
manually,
supported
by a user interface.
Our approach is to develop
such a contour model
that will
minimize
the
delimited area, thereby
rendering fitting
to concave
parts
easier. This is
possible
simply
by adding
a set of
new
forces
with
the same direction
as the inner normal
to the
contour u(s), as Fig. 5b
shows.
3.4
Area force
approximation
We assume a functional
S(u) to be
an area de-
limited
by the contour
u(s)
:
(x(s), y(s)).
Concern-
i{rg
tfr. necessary
condition for
area minimizing
r,^
:
0, area
forces
fa,
yieldingthe
minimized
con-
tour area are defined for
each
point
ofa contour by:
The measure
of delimited
area S(u) and area force
fa
are both derived from
the discrete representa-
tion of u(s).
Let
us
give
a set
of vertices vi
:
@i, !i)
(i
:
L, ...
,
N)
as a discrete representation
of
a clockwise oriented
contour that does not inter-
sect
itself
(Sect.
2.3). Therefore,
the area
delimited
by the set of
vertices
{v;}
is derived
as a sum
of
determinants
It is easy to
see
that
the area force
f;is
akind of
discrete approximation of
an
inner normal
vector
to the contour
u(s) at the
point
u;.
Application
of the area force in
the
point
u; moves
it inside
the delimited
area and
makes
this area
smaller. One
question
which still remains open is
how
to control the masnitude
of this
force.
4
Strength of forces
Figure
6 traces out a
simple example
with
the
clockwise active contour u(s) and
a structure con-
taining two regions
whose contours
we wish to
trace.
The
set of contour
points
in Fig.
6 is divided
into
two subsets, the first, consisting
of contour
points
between
points
ri, vi o.nd between uo, u, is
denoted as
a subset 51.
The
second subset,
de-
noted as
Sr,
is
the complement of subset
51
for
all
the contour
points.
The
points
of 52 are close
to
the edges of the object and have large image
forces
f1
in Eq.7,
while the
points
of 51 are far from
the
edges, having image forces
close to zero. The
shrinking of the active contour in
the area of
points
51
is
expected to be
greater
than that of
subset 52. Therefore,
the area
force
for the
points
of St is
scaled by the weight function
to
get
a
greater
magnitude,
while,
in
the
case of the
points
forming 52, the
area
force
is scaled
to
get
a lesser one.
4.1 Weight
of the area force
The
part
of the active
contour
in
which the image
forces
sain
less maenitude
can
be called the
f,
:
:(
11,::r'l;i,11,,"')
]s"r,r(
-|::,_;:::',)
i":(#,x)
(r2)
So(tv,, i
:
r,
|
(lxt
2\lY'
282
"''N))
,,1
*
l"'
".1
*
...
+
l'"
*'l)
lzl
llz
./s
I l./iv
!tl/
f1
f4
,"{
01L931L6+L7L
-T
-->-
7
i-T
e
Fig.
6.
Contour
close
to
two parts
of an
objects.
The
area
forces/.
along
the
shrinking
region
S1 are
iareer
than
those
alongLhe
subset
Sr. while
the
image
forcesfr
are
conversely,
larger
for
the points
of
S" than
for
tho;
Sl
Fig.
7.
Graphs
of
the
weight
t'unction
IV,
of
the
area
forcc
andthe
weight
function
II{ of
the
image-iorce
used
in
our
lmplementation.
The graphs
are
drawn
for
the
parameter
L=8
-,,&E*qw$*eg-
q,&$s*glt*fftrtr
a. contour
point
u(s).
A
graph
of
the
weight
func_
tion
W
a
is
illustrated
in
Fis.
7.
4.2
Weight
of
the
image
force
The
image
intensity
in
a
small
neighbourhood
of
the
shrinking
contour
part
is
usually
not
a
con_
stant
value,
but
varies
slightly
due
to
the
noise
distribution.
It.arises
in
the
small
magnitudes
of
image
forces
1
7:,l
tnatare
neglected
till
i-hev
exceed
a
certain
threshold
value
16.
Precisely,
the
continuous
image
force
weight
func-
tion
W1(0)
also
introduced
in
Eq.
7 is
dehned
for
all
points
of
u(s)
as:
W,(0):
t-wA(r+(?-LoD
Lo<0<L
,
O<Lo
otherwise
arccos
0.8
(14)
where
Lo
and
L
are parameters
controlling
the
vanishing
interval
of
image
force,
and
0,
deiotes
the
norm
of
the image
force.
A graph
of the
weight
function
trT, is
illustrated
in
Fie.
7.
Parameter
I,e
in
our
implemeitation
is
selected
such
that
the
image
forces
are
neglected
for
all
active
contour
points
where
the
wEieht
function
Wo(0) >
0.9.
This
gives
us
the
.*pr..rion
for
the
estimation
of
parameter
-Ln
as:
, -L
LO--
n
shrinking
part
of the
contour.
As
mentioned,
we
control
the
strength
of
area
force
along
the
active
contour
by
anonzero
areaforce
magniTudefor
the
shrinking
part
of
the
contour,
while
for
other
parts
the
area
force
is
set
to zero.
Thus,
the
continuous
weight
function
W
o@)
of
the
area force,
introduced
in
Eq.
7, is
definid'for
all
the
points
of
active
contour
u(s)
as:
r, ,n\ fL
+jcosf
e
<
L
w
Atat
:
1- ;
"
^,'^^-..1^^
(t3)
I
0
orherwise
w^he1e
I is
a
parameter
controlling
the
vanishing
of
the
area
force
strength,
and.
0:
ll(r(r))'i
denotes
the
norm
of
the
image
to.."
glu""
i;
This^gives
4se
to
the
following
situation:
the
im_
age
lorce
J,
at
a certain
point
is
neglected
till
at
least
90o/o
of
the
area
force
magnitude
at
that
p-oint
is
applied
in Eq.
4.
Moreovei,
the
area
force
l)
is
neglected
if
the
image
force
is
fully
applied
in
Eq.
4.The
superposition
of
the
weight
functions
Wa
and
Wl
used
in
our
implementaiion
is
traced
out
in
Fig.
7.
4.3
Estimation
of
parameter
L
To
avoid
the
trouble
of
selecting
a
parameter
L,
an
automatic
selection
method
based
on a
histogram
of
image
forces
along
the
contour
is
283
::ee
d\e
ftLti;cE
,
q
,.{}ffi&}€?€Ba#tr
UT
E
o
Q)
a
F
i
P,
v^
P,
Norm of
Image Force
4
o
9g
o
$z
tr
-1
Fig, 8.
a
An ideal image force norm
histo-
gram
corresponding to a contour
passing
the
objects edges and shrinking
region.
Corresponding
values at
peaks
and valley are
f;, \
and
V-; b Smooth
histogram of the
image force norm
ffil
along
the contour after
few
iterations of an initial contour based on
a
real image. The estimation
of
parameter
L, at
valley V-=$.lQ
Fig. 9. Conditions for cutting
the contour
into two oarls
8.9 10
12 13.8
N orm
of
I
mage
F
o
rce
8b
ojectlons
proposed.
Let
us suppose the
initial
contour
is
close
to the edge of the object, but does not
necessarily
follow
the concave
parts.
This
assump-
tion
is fulfilled, for example, by a few iterations in
Eq. 6. We observe
in Fig.
6
mainly
two
parts
of the
active contour.
The first
part
follows the object
edges and the second does
not, i.e., it
passes
right
through the shrinking
region. An appropriate
measure that the contour
point
lies on the edge is
the
norm of the image force
fl
at this
point.
To
find the
probability
of the contour
point
being on
the edge of an object, the
histogram
of
image force
norm
along
the contour is built.
There are two main
probabilities
in
the
histo-
gram
that the contour
point
lies
on the object's
edge
or that the
point
lies in the shrinking region.
284
Both are high
peaks
with different
dominant
values
P., P"
of the
image force norm, as shown
in
Fig. 8a. One
valley
corresponding to
the
value
V^
is between them.
This
value
can be used as a thre-
shold
for
splitting
contour
points
set
in
subsets 51
and 52.
The
set of
the contour
points
is split into
two collections
of
points:
those with an
image
force norm below
the threshold V*
(a
subset 51)
and those with
an image force norm above
it
(a
subset S2).
In
practice
the histogram
is not
as smooth as
shown
in Fig. 8a.
It is,
therefore,
useful to apply
a smoothing operator,
such as an averaging oper-
ator, before
finding
peaks
and valleys.
High
peaks
and deep valleys are
located
by
thresholding their
levels. This is followed by selection of the
most
-
tliW *
S
. *9
$$tfi*i*
€)iffi$i?kt*;r
prominent
valley
surrounded
by high peaks,
and
the
set
of
points
is
split
according
to
ihe
corres_
ponding
threshold
V^
at the
valley.
The
result
of
valley
selection
for
the
histogram
.irown
in Fig.
gb
is
Z.
:
3.16.
Usually, points
with
an image
force
norm
below
the V^
are
points
away
from
the
edge.
Therefore,
the
area
force,
rather
that
image
forci,
dominates.
Remembering
Eq.
13
and its
graph
in
Fig.
7, it
is
wise to
set
the
parameter
L:
V-.
To
implement
dynamic parameter
estimation,
threshold
V*
from
the
histogram
can
be
recal-
culated
during
the
iteration
process
of
Eq.
6.
In
this
case.
the
use
of
median
filtering
applied
on
a
calculated
sequence
of
values
,L ii
necessary.
because
of
their
variation
due
to
the imase
noise.
5 Basic
contour
operations
This
section
describes
some
basic
operations
for
the
automatic
creation
of an
active
cbntour
tooo-
logy
and
for
the
shaping
of
this
contour.
The
most
basic
operations
are
adding,
deleting
a simple
contour
point,
and
diuidinq
a
contour
into
two
parts.
The
basic
componentJof
our
algo-
rithm
extension
are
based
on heuristic
rules
c6n-
trolling
the
addition
and
deletion
of
a
point.
These
rules
assume
that
the
contour points
are in
a near-
equilibrium
configuration
with respect
to
the for-
ces
of internal
potentials.
The
first
rule
checks
if
two
neighboring
points
have
a large
enough
distance
between
themto
add
a new
one.
If
two
points
vi,
vi+1
are
separated
such
that
d*i,
(
ly;
-
y;+rl
S
d^o,,
d new point
is
cre-
ated
at the
midpoint.
If
we denote
the
averase
interpoint
distance
as
d,
typically
d^i, x
t.7
tr
aid
d*o* N
2.5
d.
Here
d^o*
paiameter
ii
intended
to
prevent
a contour
from
growing
infinitely.
The
second
rule
allows points
to
be deleted.
If
two
neighbors
are
separated
by
a distance
D
such
that
D
1
D^r,,
one
of
these
points
is
deleted.
We have
experimented
with
D*;, x
td.
A
heuristic
rule
controlling
the
diuiding
of an
active
contour
allows
the
contour
to
be
but into
two
parts
by
separating
it
with
an
additional
segment.
Let
two
points
u;,
v;
exist,
as shown
in
Fig.
9,
obeying
the
following
conditions:
l.
lv,
-
vil
I
d,ut
where
d,u,
is
a
constant.
2. The projections
of area
force
vectors
at
points
vi
and
v;
onto
the
vectors
v;\and
y,t,
respec-
tively.
have
opposite
directions.
An
active
contour
is
cut
into
two
parts
by
the
segment
u;v;
if
conditions
1
and
2 hold,
and
more_
over,
both parts
of
the
split
contour
have
at
least
five
vertices.
The
last
restriction,
a minimum
num-
ber
of
vertices,
is
selected
because
of the
pen-
tadiagonality
of
the
stiffness
matrix,
described
in
Sect. 2.3,
Eq.
5.
The
rule
of division
can
be
generalized
to
a con-
tour
n-split,
i.e.,
contour
division
into
n
parts
as
a sequence
of n recursive
divisions
into
two parts.
Figure
10
illustrates
the
states
of active
contour
splitting.
The
flrst
estimate
of
the
paramerer
I,
used
in
weight functions
W1
and.
Wo
is
cal-
culated
from
initial
state
of
an
active
contour
shown
in
Fig.
10a.
Figure
10b
shows
a
contour
fulfilling
the heuristic
rule
of
contour
division.
A
contour
separated
by an
additional
segment
is
also
shown.
Postsplitting
contours
are
shown
in
Fig.
10c.
6 Experiments
The
elastic
ACM
has
been
implemented
on
a Sili-
con
Graphics
Iris
workstation.
We
evaluated
our
approach
in
simulations
involving
both
synthetic
and
real
image
data.
6.1
Synthetic
image
data
For
synthetic
images,
the
true
boundary
is known,
and
thus,
a
quantitative
measure
of
error
can
be
selected.
The
appropriate
error
measure
we used
is
the
average
distance
between
the
corresponding
points
of
the
two
contours.
The
correspondence
can
be
determined
by
the nearest
distanie
of
two
corresponding
points.
In
discrete
form.
we use
simple
linear
interpolation
between
the
points
to
find
the
corresponding
points.
The
averige
error
e is
then
defined
by
1{
e(D.t)):,V,L,
.TJl,lu(si)
-
t(s)l
where
d is
the
true
boundary,
t) is
the
calculated
boundary,
and
S is
the
parametrization
of d.
285
10a
....:
ia,'
'
'.::il'
11a
Fig.
10a-c. Contour splitting
on a molrse embryo
cross-section:
a Initial state
of the contour
model on a 1eg of a
mouse embryo;
b Splitting
of the contour; c
Final reconstructed contours
of the
leg
Fig. 11a-c.
The extraction of
a circle contour when
poor
contrast
exists
between
circle and background: a
synthetic image
with
Figure 11 shows a circle that
is difficult to distin-
guish
from
its
background.
The initial contour
agrees
only
roughly
with the target
shape and the
location.
The
extraction of
feature vectors
from
lattices
of
size l2xl2
pixels
(M
:12)
and a 4-
pixel
step
(d:0
is
achieved as
a
preprocessing
step. The weight
parameter
W
7
of the texture
force
can vary
from 0.2 to 0.3.
The
elastic
para-
286
roughly
given
initial contour; b
Final contour of a circle
from tex-
ture information; c
contour in equilibrium
obtained by employing
scale
space
Fig. 12a, b. Optical
microscope
image of a mouse embryo; a
Ini-
tial
contours of the
mouse embryo stomach; b
final contours of the
stomach of
the embryo obtained
by using texture constraints
meters
from Eq. 2 are fixed to constants
ranging
in
the
interval
(0,1),
nominally set to w1
:
0.1,
and
w2
:0.3
in this example.
The
obtained
equi-
librium applied
on the initial contour of
Fig. 11a,
using the
texture energy,
is shown in Fig. 11b.
To
compare
the results with
the method employing
only the scale space
information
from trig. 11c, we
set
parameter
W
7
:
0 at
zero,
while
the remaining
14b
Fig.
13, Tracking the
brain of
the mouse embryo
through
slices
visualized
as a stack of contours
with their interior
Fig.14a,
b. Sequence
of 15 serial
sections segmented
with
the
proposed
active contour
model:
a the reconstructed
head;
b the
internal structure
(brain)
of the
mouse embryo
are visualized
by a
transparent
technique
parameters
wr
and wG are
fixed at
0.41 and
-
0.33, respectively, giving
in
the best result
ob-
tained. The
final
curve'delineates
the target
with
an average
pixel
error s of approximately
1.1
per
contour
point,
while
methods
not using
texture
information
(Carlbom
et
al.1992; Kass
et al. 1987)
sketch the target
with an average
pixel
error of 7.2.
6.2
Real
image
data
Figure
12 shows
a selected
slice from
a mouse
embryo
sequence of
cross-sections
with the initial
and final
stages of the
contour model
using
tex-
ture information
during
the iteration
process.
The
process
of active
contour
splitting can
be
observed in Fig. 10.
New
vertices are
added auto-
matically,
while
parts
of the contour
shrink
to-
ward each
other. After
the contour
splitting,
the
ACM iteration
runs
on each
part
separately,
while
contour
points
are deleted
according
to the rules
already discussed.
We
have
tested
our
method
on a set of microscop-
ic images
of mouse
embryo
cross-sections.
Our
2D
ACM might
be used
to segment
structures
from
3D
volume images.
The initialization
of
a contour
position
for
a single
slice is traced
manually.
We
refine
the initial
position
using the
proposed
ACM.
The result
generated
by the
ACM is mapped
onto the next
slice of the
set of
images
to
provide
an initial
contour
position
of
a structure in
the
next
slice. All
the active
contour
parameters
of a new
position
are
copied from
the
previous
slice. Figure 13
shows
the result
of ap-
plying
this approach
to a
sequence of microscopic
images
of the mouse
embryo
brain.
The
transparent
visualization
of the reconstructed
head
and brain
of an embryo
using our method
is
shown in Fig. 14.
7
Conclusions
and
future
work
We
have
described
a
physically
based contouring
system
that facilitates
the use of
several higher-
level
constraints
to
govern
edge selection
and
linking. The
iterative
approach is
even effective for
locating
objects
such as
organs,
given
an initial
estimate of their
position,
scale, and orientation.
The
stiffness matrix remains
constant at
all
iter-
ations
except those
to which
the basic
contour
14a
287
_..*
%
*
* *'i!$
ffisg.ffgRg
"
q
,dbsBEE$€.gtrtrtr
operations such as adding, deleting
of a simple
contour
point,
and diuiding a contour into two
parts
are applied,
changing the contour topology.
The
natural
active contour length is
expected to
be
zero
(Terzopoulos
and Fleischer 1988). Thanks
to this, a set of linear
differential equations de-
coupled
in
x and
y
allow even large deformations.
Also,
we
have
shown how to take into account the
texture
approach
contouring
problem,
which is
a
powerful
tool when working with
poorly
con-
trasted images. Our
techniques can be applied to
a wide variety of objects in
various
imaging mo-
dalities.
One of the limitations
of the current
implementa-
tion is the finite difference
(Eqs.
3 and 5) based
solution, which
has
topological
restrictions,
mak-
ing it difficult to create
contours with thin ap-
pendages.
We must reduce the amount of user
assistance
required in
such cases. Careful finite
element
discretization
should
remove
these
re-
strictions.
Acknowledgemenrs. The authors thank Prof. Mineo Yasuda
and
Prof. Akinao G. Sato for their advice and for
providing
the
sample images, and also thank Prof. Tomoyuki Nishita for most
helpful discussions.
References
Berezin IS, Shidkov NP
(1965)
Computing
methods
(Volume
II).
Pergamon Press, London
Carlbom
I, Terzopoulos D,
Harris KM
(1991)
Reconstructing
and
visualizing
models
of
neuronal
dendrites.
In:
Pat-
rikalakis NM
(ed)
Scientific visualization of
physical phe-
nomena
(Proceedings
of CG
International
'91).
Springer,
Tokyo,
pp
623 638
Celniker G, Gossard D
(1991)
Deformable curve and surface
finite-elements for free-form
shape design. In: Comput
Graph,25:257
266
Cohen I, Cohen DL, Ayache N
(1992)
Using
deformable surfaces
to segment 3-D images and inler
differential structures. Pro-
ceedings of the
Second
European
Conference on Computer
Vision, Santa Margherita Ligure, Italy,
pp
648-651
Cootes TF, Hill A, Taylor
CJ,
Haslam
J
(1993)
The
use
of active
shape models for locating
structures
in
medical images. In
Barratt
HH,
Gnitro AF
(eds)
Information
processing
in
medical imaging, 13th International
Conference, Springer,
Berlin Heidelberg,
New York,
pp
33
46
Cross RG, Jain KA
(1983)
Random Markov field texture models.
IEEE Trans Patt Anal Machine Intell 5:25-39
Groshong BR
(1992)
Estimating
simple
closed
contours in im-
ages. Proceedings of 5th Annual IEEE Symposium on
Com-
puter-Based
Medical
Systems,
IEEE
Computer Society
Press, Los Alamitos,
California,
pp
35
43
Kashyap
RL,
Chellappa R
(1983)
Estimation and choice of
neighbors in spatial interaction models of images. IEEE
Trans Information Theorv 29:60
72
288
Kass M, Witkin A, Terzopoulos D
(1987)
Snakes:
active contour
models.
Int. J. Comput Vis
4:321
331
Kita Y
(1992)
Model-driven contour extraction for
physically
deformed objects application to analysis of stomach X-ray
images. Proceedings of the 1 1th International
Conference on
Pattern Recognition
(Volume
I). IEEE Computer Society
Press, The Hague,
pp
280-285
Lanczos
C
(1974)
The
variational
principles
of
mechanics, 4th
edn. University of
Toronto Press, Toronto,
Canada
Mengyang L, Jiamei
Q,
Yanni
'f
(1992)
Texture classification
and segmentation using simultaneous autoregressive
ran-
dom model. Proceedings of the 5th IEEE Symposium on
Computer-Based
Medical
Systems, IEEE Comp. Society
Press, Los Alamitos, California,
pp
398 401
Nastar
Ch,
Ayache N
(1993)
Non-rigid motion analysis in medi-
cal images: a
physically
based approach. Barrett HH,
Gmitro AF
(eds)
Information
processing
in medical imaging.
13th International
Conference
IPMI,
Springer,
Berlin
Heidelberg New York,
pp
17 32
Pitas I, Venetsanopoulos AN
(1990)
Nonlinear digital filters
principles
and applications. Kluwer Academic, Boston Dor-
drecht London
pp
276 344
Revankar
S, Sher
D
(1993)
Constrained
contouring in
polar
coordinates. Proceedings of Computer Vision and Pattern
Recognition, New York,
pp
687-689
Staib LH, Duncan JS
(1992)
Boundary finding with
paramet-
rically
deformable
models. IEEE Trans Patt Anal Machine
Intell 14:1061
1075
Terzopoulos D, Platt
J,
Barr A, Fleischer K
(1987)
Elastically
deformable models. Comput Graph 2l:205-214
Terzopoulos D, Fleischer K
(1988)
Modeling inelastic deforma-
tion: viscoelasticity,
plasticity,
fracture.
Comput Graph
22:269-278
Terzopoulos D, Metaxas D
(1991)
Dynamic 3D models with
local and
global
deformations: deformable
superquadrics.
IEEE Trans Patt
Anal
Machine
Intell 13:703
714
Turner R, Thalmann D
(1993)
The elastic surface layer model for
animated character construction. In: Thalmann NM, Thal-
mann D
(eds)
Communicating with virtual worlds,
Proceed-
ings CG International
'93,
Springer, Tokyo,
pp
399412
Williams DJ,
Shah
M
(1992)
A fast algorithm for active contours
and curvature estimation.
CVGIP:
Imag.e
Understanding
55:14
26
RoMAN DuRtKovIa srud-
ied numerical analysis at the
Comenius University,
Bratis-
lava.
Slovakia. where
he received
a M.S. degree in Numerical
Analysis in 1989. During the
spring of 1991 he followed
grad-
uate
courses in
systems
and con-
trol theory at Groningen Uni-
versity, The Netherlands. In
1992 he
joined
the computer
graphics group
of Hiroshima
University, Japan,
as a visiting
scholar.
He is
currently a
Ph. D.
student in Electrical Engineering
at this university. His research
interests are image
processing,
computer
graphics,
and
physical
modeling.
'r*-I-lb
a
-,.,}'iffiral
1",{)filpiltr.}f
KAZUFUMI
KANEDA
is a
research
associate
in
Faculty
of
Engineering
at
Hiroshima
Uni-
versity.
He
worked at
the
Chugoku
Electric
Power,
Japan
from
1984
to 1986.
He
ioined
Hiroshima
University in- 1986.
He
was a
visiting researcher
in
the
Engineering
Computer
Graphics
Laboratory
at
Brigham
Young
University
in
1991.
His research
interests
in-
clude
computer
graphics
and
image processing.
He received
the
BE, ME,
and
DE in 1982,
1984,
and 1991,
respectively,
He is
a member
of ACM,
IEE of
HIDEo
YATvTASHITA
is a
Professor
at
Hiroshima
Univer-
sity. Department
of Engineering,
Electric
Machinery
Laboratory,
where he
teaches
and does
re-
search in
visualization
of mas-
netic fields
and finite
elemeit
analysis.
He
received his
B.E.
and
M.E. degrees
in
Electrical
Engineering
from
Hiroshima
University,
Japan in 1964
and
1968,
and
DE in
1977 from
Waseda
University,
Tokyo,
Ja-
pan.
He
was appointed
as a Re-
search Assistant
in 1968
and
an
Associate
Professor
in 1978
to
Japan, IPS of
Japan and
IEICE
of Japan.
the Faculty
of Engineering,
Hiroshima
University.
He
was an
Associate Researcher
at
Clarkson
Universitv.
Potsdam.
N.Y. in
1981-1982.
His research
interests
lie in the
aieas
of the masnelic
field analysis
that use the
finite element
method
and
the b6und-
ary element
method
and visualization
techniques
of multidimen-
sional data.
He is a member
of
the
IEEE.-ACM.
the IEE of
Japan, the IECE
of
Japan, and the
IPS of
Japan.
from
Hiroshima
Universitv.
289
