Modeling of Sparsely Sampled Tubular Surfaces Using Coupled Curves

Abstract

We present a variational approach to simultaneously trace the axis and determine the thickness of 3-D (or 2-D) tubular structures defined by sparsely and unevenly sampled noisy surface points. Many existing approaches try to solve the axis-tracing and the precise fitting in two subsequent steps. In contrast to this our model is initialized with a small cylinder segment and converges to the final tubular structure in a single energy minimization using a gradient descent scheme. The energy is based on the error of fit and simultaneously penalizes strong curvature and thickness variations. We demonstrate the performance of this closed formulation on volumetric microscopic data sets of the Arabidopsis root tip, where only the nuclei of the cells are visible.

Full text

Modeling of Sparsely Sampled Tubular Surfaces
Using Coupled Curves
Thorsten Schmidt1,2, Margret Keuper1,2, Taras Pasternak3, Klaus Palme1,3,4
and Olaf Ronneberger1,2
1Lehrstuhl f¨ur Mustererkennung und Bildverabeitung, Institut f¨ur Informatik,
2Centre of Biological Signalling Studies (BIOSS),
3Institut f¨ur Biologie II, 4Freiburg Inst. for Advanced Studies (FRIAS),
Albert-Ludwigs-Universit¨at Freiburg
tschmidt@informatik.uni-freiburg.de
Abstract. We present a variational approach to simultaneously trace
the axis and determine the thickness of 3-D (or 2-D) tubular structures
defined by sparsely and unevenly sampled noisy surface points. Many
existing approaches try to solve the axis-tracing and the precise fitting
in two subsequent steps. In contrast to this our model is initialized with
a small cylinder segment and converges to the final tubular structure in a
single energy minimization using a gradient descent scheme. The energy
is based on the error of fit and simultaneously penalizes strong curvature
and thickness variations. We demonstrate the performance of this closed
formulation on volumetric microscopic data sets of the Arabidopsis root
tip, where only the nuclei of the cells are visible.
1 Introduction
The accurate tracing and segmentation of parallel-line (2D) or tubular structures
(3D) is an active research topic to solve problems coming from medicine, biol-
ogy, robotics, or aerial and satellite image analysis. Especially for biological and
medical applications, with their wide spectrum of imaging methods this mod-
eling is an important step towards data abstraction and quantification. With
the discovery of the green fluorescent protein, and the isolation of its gene, the
possibility of imaging organism development in-vivo has led to a revolution in
biological research and a tremendous increase in data volume to process.
One popular model organism in plant science is Arabidopsis thaliana, due
to its simple architecture and comparably small and fully sequenced genome.
Especially the stem cell niches in the root and shoot tips (root/shoot apical
meristems) are of high interest to understand plant development and signaling
within complex organs. The root apical meristem (RAM) consists of a set of
tissue layers around the root’s axis. Each of these layers can be described using
an axis-thickness model, leading to the continuous anatomical model needed to
localize events within the root. The simplest and most flexible way of imaging
the plant root with cellular resolution is to mark the cell nuclei and relate events
within the organs to a derived overall anatomical model. However, this practical

2 T. Schmidt, M. Keuper, T. Pasternak, K. Palme, O. Ronneberger
imaging advantage leads to a very sparse representation of the coaxial tubular
structures each cell layer represents, posing high demands onto the modeling.
Especially in microscopic data the light is attenuated by the specimen density
and scattered at interfaces between regions with different refractive indices lead-
ing to a significant loss of signal when imaging through thick tissues. Therefore
simple gray-value based approaches like thresholding and/or kernel smoothing to
estimate the root axis are biased towards the part of the root close to the objec-
tive and tend to fail. We therefore avoid direct use of the image gray values, and
instead extract the positions of the nuclei of a selected layer using the detection
scheme described in [10]. The detection quality is also biased due to the described
effects, and a direct fit to the resulting point cloud using kernel smoothing still
shows a systematic fitting error as we will show in the experiments section.
To avoid the described bias we employed a model consisting of a vector-valued
function describing the tube’s axis and a scalar function describing the variable
tube thickness. Both functions are coupled by a common curve parametrization
into a combined tubular model which is fit to the data in a robust variational
energy minimization scheme. The model is designed to work solely on the sparse
point positions, without the need for surface normal estimation. We will show
that it leads to very accurate fits even in the case of high noise and missing
surface points.
A key benefit of the proposed model is that it “grows” into a large and arbi-
trarily complex tubular structure from a small local initialization, i.e. it solves
the tracing and accurate fitting problem within a single energy minimization.
1.1 Related Work
In medical applications various approaches exist to analyze images of vascular
and neuronal networks based on different imaging methods ranging from low
resolution CT and MRT, through light microscopy down to electron microscopy
[7, 3]. All approaches have in common that they rely on densely imaged inter-
faces between the structures of interest and mainly depend on the gray values
and their derivatives to guide the model fitting. One possibility of robustly find-
ing the axis of a tubular structure is a symmetry analysis around the potential
axis [8]. Morphology-driven approaches try to find the axis by structure thin-
ning leading to a skeletonization. Filter based approaches first try to emphasize
the structures using filter banks or steerable filters and apply thresholding and
thinning afterwards. See [4] for an overview comparing the different approaches.
In the field of robotics approaches to fit parametric tubular structures to
point cloud data recorded using laser range scanners are of high interest [1].
Most existing approaches exploit the scanned dense mesh structure to estimate
local surface normals guiding the model fitting process. These approaches have
to cope with noisy data and therefore estimate the normals for each surface
position from relatively large neighborhoods. Others try to detect shapes using
Hough-like voting based approaches [9]. These are especially suited to detect
man-made rigid objects, but don’t perform well on deformable objects as they
are common in biological and medical applications.

Modeling of Sparsely Sampled Tubular Surfaces Using Coupled Curves 3
In [5] the coupling of two evolving splines describing the center-lines and
thicknesses of roads and rivers in aerial and satellite images was introduced.
Although the noise level in images of that kind is very high, the gradients are
still a valuable piece of information to guide the snake evolution. A different
approach using two coupled splines to describe the outlines of the biologically
highly interesting model organism C-Elegans was introduced in [11].
In [6] a non-self-intersecting 1-D line from unstructured and noisy 3D point
data was reconstructed using moving least-squares interpolation. For homoge-
neously distributed tube-surface data around its circumference this approach is
also applicable to solve the tube axis fitting task, although it does not determine
the tube thickness.
Our setting is different from the above-mentioned, since our approach has to
perform the task of simultaneously estimating the axis and variable thickness of a
tubular structure based on sparse surface points only. The low point density and
high data noise preclude the extraction of reliable surface normals. We formulate
the task of fitting the model to a point cloud as one closed energy minimization
problem, which incorporates all available points and a set of tubular models to
which on demand new tubes can be added.
2 Variational Coupled Curve Fitting
Fig. 1. A 2-D sketch of the tube model fit to a point set depicted as black circles.
The axis is shown as blue line, while the dashed lines indicate the estimated tube
incorporating the tube thickness. The distance shown in green is minimized during the
optimization.
We define a tube as a function mapping a curve parameter u∈Rto the
(D+ 1)-dimensional vector a>(u), t (u)>, where a:R→RDis the tube axis
function and t:R→Ris the corresponding tube thickness function. Fig. 1
sketches the tube model. To optimally map the model to a set of tube surface

4 T. Schmidt, M. Keuper, T. Pasternak, K. Palme, O. Ronneberger
points X={x1,...,xn},xi∈RDwe minimize the energy
Edata (a, t) :=
n
X
i=1
ψ(ka(ui)−xik − t(ui))2(1)
where ui:= arg minukxi−a(u)kis the curve parameter projection of xiand
ψρ2is a robust distance measure.
To cope with sparse surface points and high data noise, we additionally intro-
duce smoothness terms penalizing axis curvature and tube thickness variations
Ea(a) = Z∞
−∞ 



d2
du2a(u)



2
duand Et(t) = Z∞
−∞ d
dut(u)2
du . (2)
For shorter notation we define ρi(u) := (ka(u)−xik − t(u)) and get the overall
energy functional to minimize
E(a, t) :=
n
X
i=1
ψρ2+λZ∞
−∞ 



d2
du2a(u)



2
du+µZ∞
−∞ d
dut(u)2
du(3)
where λ, µ ∈R+weigh the influence of the smoothness terms.
3 Parametrization Using B-Splines
We approximate the curves with open B-splines of degree p, therefore the nodes
at the spline endpoints are repeated p+ 1 times. W.l.o.g. we will restrict the
spline parameter uto the [0,1]-range. We obtain the B-spline approximation of
the general functions a(u) and t(u) as follows:
a(u) :=
m−1
X
j=0
ca
jbj,p,s(u) and t(u) :=
m−1
X
j=0
ct
jbj,p,s(u)
where Ca=ca
0,...,ca
m−1and Ct=ct
0, . . . , ct
m−1are the control points,
and bj,p,sare the basis functions with node-vector s= (s0, . . . , sm+p)>.
Lemma 1 (B-spline derivative). Let f(u) := Pm−1
j=0 cjbj,p,s(u)be a B-spline
of degree p∈N0, with control points cj, j = 0, . . . , m −1defined over the knot
vector s= (s0, . . . , sm+p)>. Then the derivative
f0(u) = d
duf(u) =
m−2
X
j=0
c0
jbj,p−1,s0(u)
is another B-spline of degree p−1defined over the knot vector s0= (s1, . . . , sm+p−1)
with control points c0
j=p
sj+p+1−sj+1 (cj+1 −cj).

Modeling of Sparsely Sampled Tubular Surfaces Using Coupled Curves 5
More details to splines as well as this Lemma and its proof are detailed in [2].
The general energy from (3) changes to
Edata (a, t) =
n
X
i=1
ψ(ka(ui)−xik − t(ui))2
+λ·
D
X
d=1 Z1
0

m−3
X
j=0
c0a
j,dbj,p−2,s0(u)

2
du
+µ·Z1
0

m−2
X
j=0
c0t
jbj,p−1,s0(u)

2
du . (4)
The primed variables are obtained applying Lemma 1 (for the axis twice) to the
original splines.
Using the spline parameterization the partial derivatives with respect to the
control points ca
jand ct
jare needed
∂
∂ca
j,d
E(a, t)=2
n
X
i=1
ψ0ρ21−t(ui)
ka(ui)−xik(ad(ui)−xi,d)bj,p,s(ui)
+2λ
m−1
X
j0=0
ca
j0,d Z1
0
d2
du2bj0,p,s(u)d2
du2bj,p,s(u) du(5)
∂
∂ct
j
E(a, t) = −2
n
X
i=1
ψ0ρ2(ka(ui)−xik − t(ui)) bj,p,s(ui)
+2µ
m−1
X
j0=0
ct
j0Z1
0
d
dubj0,p,s(u)d
dubj,p,s(u) du , (6)
finally leading to the following update rules for moving the control points in a
gradient descent manner when introducing an artificial discrete evolution time
kwith step τ∈R+:
ca
j,d
k+1 =ca
j,d
k−τ∂
∂ca
j,d
E(a, t) and ct
j
k+1 =ct
j,d
k−τ∂
∂ct
j
E(a, t).(7)
Since all dimensions come into play during the control point updates in each
iteration, first the derivatives are computed for each control point, then the
update is applied and finally the uifor each point are recomputed.
We define the outlier-robust distance measure
ψρ2:= (ρ2ρ<η
η2ρ≥ηwith derivative ψ0ρ2=(1ρ<η
0ρ≥η
and some user-defined threshold η∈R(which should be chosen in the range of
the structure radius).

6 T. Schmidt, M. Keuper, T. Pasternak, K. Palme, O. Ronneberger
Algorithm 1 The Coupled B-spline fitting algorithm
Require: Point set X, initial cylinder, parameters λ,µ,τ
1: Initialize each model (a, t) with two knots fitting the initial cylinder
2: Compute the initial model energy E(a, t) using (4)
3: while not converged do
4: Minimize E(a, t) using (7)
5: Insert knot and re-parametrize the model
6: end while
7: return The coupled B-spline model (a, t)
Only points within a certain distance range defined by ηwill contribute to the
derivatives which allows to adapt the model fitting to the surface point density
and the data noise. We additionally linearly decrease λand µwith increasing
arc length of the current axis estimate to avoid a bias towards short curves and
update the thickness function only with points mapping orthogonally onto the
axis to avoid a thickness over-estimation at the tube end points.
We initialize the fit with a manually chosen short cylinder segment repre-
sented by a straight B-spline with two knots at the ends with the thickness
intialized to the cylinder radius. During the described optimization the number
of control points remains constant. Therefore the model will evolve until no more
data points can be described by one single degree ppolynomial. To allow more
complex tube shapes, we alternate between fitting and re-gridding step in which
an additional knot is inserted and distribute the knots equidistantly along the
curve leading to an intermediate curve length parametrization. The whole fitting
process is described in Alg. 1.
3.1 Extension to Multiple Tubes
To simultaneously trace multiple tubular structures, for each a seeding cylinder
can be placed. In each iteration step the point set is partitioned into subsets, so
that the points in subset Xmare best explained by the mth tubular model ac-
cording to the data energy term. The evolution of tube mis computed on subset
Xmonly. The Energy then becomes the sum over all single model Energies.
4 Experiments
4.1 Synthetic Data
We compared the proposed model (using cubic splines) to the axis estimates
obtained through Gaussian point cloud kernel smoothing (PKS), which resembles
the drawbacks of averaging techniques for curve fitting. For this we synthetically
generated data sets consisting of point clouds highlighting specific cases. We used
trigonometric functions to model the axis and thickness functions and generated
1000 equally distributed tube surface points around the axis. The point positions
were then moved in an arbitrary direction following a Gaussian distribution with

Modeling of Sparsely Sampled Tubular Surfaces Using Coupled Curves 7
standard deviation σleading to the synthetic ground truth (Fig. 2 left panels).
The PKS kernel width was empirically chosen to minimize the fitting error. The
error comparison between PKS and the proposed coupled curve model (CCM)
is shown in the right panels. For constant tube thickness Fig. 2(a) the axis error
of CCM in each direction stays below 20% of the tube thickness whereas PKS
already over-smooths the curve leading to undershoots. The thickness is a little
over-estimated by on average 5%. Pure thickness variations as in Fig. 2(b) do not
influence the axis localization accuracy, but they are reflected in the thickness
error, because the model is designed to favor a constant thickness. However,
the error stays below 10% for low noise and small µ(here µ= 0). Moderate
thickness variations on a bent model as shown in Fig. 2(c) affect the quality of
fit only marginally. Finally one of the big strengths of the model is highlighted in
−50050
−50
0
50
0
50
100
150
200
250
300
350
400
450
500
x
y
z
0 50 100 150 200 250 300 350 400 450 500
−10
−5
0
5
10
z
x
0 50 100 150 200 250 300 350 400 450 500
−10
−5
0
5
10
z
y
0 50 100 150 200 250 300 350 400 450 500
−2
−1
0
1
2
z
t
0
0
0
50
100
150
200
250
300
350
400
450
500
x
y
z
0 50 100 150 200 250 300 350 400 450 500
−10
−5
0
5
10
z
x
0 50 100 150 200 250 300 350 400 450 500
−10
−5
0
5
10
z
y
0 50 100 150 200 250 300 350 400 450 500
−2
−1
0
1
2
z
t
(a) curved (µ= 0.1) (b) variable thickness (µ= 0.0)
−50050
−50
0
50
0
50
100
150
200
250
300
350
400
450
500
x
y
z
0 50 100 150 200 250 300 350 400 450 500
−10
−5
0
5
10
z
x
0 50 100 150 200 250 300 350 400 450 500
−10
−5
0
5
10
z
y
0 50 100 150 200 250 300 350 400 450 500
−2
−1
0
1
2
z
t
−50050
−50
0
50
0
50
100
150
200
250
300
350
400
450
500
x
y
z
0 50 100 150 200 250 300 350 400 450 500
−10
−5
0
5
10
z
x
0 50 100 150 200 250 300 350 400 450 500
−10
−5
0
5
10
z
y
0 50 100 150 200 250 300 350 400 450 500
−2
−1
0
1
2
z
t
(c) curved + var. thickness (µ= 0.1) (d) “self-occlusion” (µ= 0.1)
Fig. 2. Sample fits to synthetically generated tubes and the fitting errors for each
model dimension (u= [0,500]). The left plots show the generated noisy point clouds
(blue crosses) and the axis estimate using the proposed coupled curves model (CCM).
The right plots show the axis mismatch in x−and y−direction for point based
kernel smoothing (PKS) in green and for the CCM in red. The third plot shows
the CCM thickness mismatch. For all experiments we set λ= 0.1. (a) a(u) =
(50 sin(2πu/300),70 sin(2πu/800), u)>,t(u) = 10, σ= 4; (b) a(u) = (0,0, u)>,
t(u) = 20 + 10 sin(2πu/200), σ= 1; (c) a(u) = (50 sin(2πu/300),70 sin(2πu/800), u)>,
t(u) = 10 +5 sin(2πu/1000), σ= 4; (d) a(u) = (50 sin(2πu/300),70 sin(2πu/800), u)>,
t(u) = 10 + 5 sin(2πu/1000), σ= 4, with “self-occlusion”

8 T. Schmidt, M. Keuper, T. Pasternak, K. Palme, O. Ronneberger
Fig. 2(d), the robustness to biased point cloud distributions on the tube surface.
For this all sample points from Fig. 2(c) which are occluded when assuming a
solid tube and a fixed view angle were removed from the set. This resulted in an
axis position bias for PKS, whereas CCM still reliably estimates tube localization
and thickness.
4.2 Microscopic 3D Volumes of the Arabidopsis Root Tip
To highlight the practical applicability and robustness of our approach we es-
timated axis and thickness of Arabidopsis root tips using the cell nuclei. The
root tips were fixated and DAPI stained to mark the cellular DNA content. Af-
ter preparation they were recorded using a confocal laser scanning microscope
(CLSM) with a 63×water immersion objective. The data volume was recon-
structed from a sequence of images using optical sectioning, leading to a final
anisotropic voxel-size of 0.2µm in lateral (x-y) and 1µm in axial (z) direction.
Two orthogonal views of a sample root with superimposed axis fits are shown in
Fig. 3. The Coupled Curves model fit to sample root tip data sets. In gray the gamma
corrected DAPI signal is shown, the red line (left panels) depicts the estimated root axis,
the yellow mesh the estimated center of the epidermal cell layer and the cyan spheres
the noisy epidermis nucleus positions. The right panels show orthogonal cuts through
the data sets and the axis fits using gray-value-based kernel smoothing (GKS), point-
based kernel smoothing (PKS) and the proposed Coupled Curve Model (CCM). The
blue line indicates the cut shown in the different views. One expert annotation is shown
as white crosses. The parameters for the CCM model were set to λ= 0.1, µ = 0.1.

Modeling of Sparsely Sampled Tubular Surfaces Using Coupled Curves 9
Table 1. Minimum/Maximum/Average root mean squared axis fitting errors between
the expert annotations and the fitting approaches on ten sample roots. (GKS = gray-
value based kernel smoothing, PKS = surface point based kernel smoothing, CCM =
the proposed coupled curve model)
Expert 2 GKS PKS CCM
min/max/avg [µm] min/max/avg [µm] min/max/avg [µm] min/max/avg [µm]
Expert 1 1.78/5.32/3.09 5.52/17.30/10.68 3.69/16.94/8.46 3.18/11.22/6.07
Expert 2 N/A 6.38/14.66/11.38 3.37/17.36/8.65 4.55/12.65/7.32
Fig. 3(a) (panels 2 and 3). Although the images are gamma corrected, the signal
attenuation in z direction is still visible.
To evaluate the axis fits, two experts manually annotated axis points of ten
root tips. For this the data sets were first rotated to roughly align the root
axis with the Euclidean x-axis. This avoids elliptic distortions of the visible root
sections during annotation. Both experts picked the root center at every 100th
x-section of the data set guided by a circle of appropriate diameter. The average
annotation difference between the experts is 3µm, which is in the order of an
average nucleus radius.
We detect the nuclear center positions of a selected tissue layer based on
rotationally invariant volumetric gray value features, namely the magnitudes of
voxelwise solid harmonic spectra [10]. Based on these features a probabilistic
SVM model which was trained on two separate datasets is applied and local
probability maxima are used as nucleus candidate positions.
We again compared the CCM to Gaussian kernel smoothing approaches, this
time incorporating either the gray values directly (GKS) or the positions of the
nuclei (PKS). We chose a kernel width of 40µm to reach a smooth curve, that
still shows good localization properties. The estimated axes on sample roots are
shown in Fig. 3. Especially in Fig. 3(a) the bias of GKS towards regions with
higher gray values is clearly visible. As already seen in the synthetic results
PKS relies on homogeneously distributed points, and therefore on the detector
quality. In all samples (Fig. 3(a-c)) the detector reported many false positives
in low signal parts of the recording, leading to deformations of the PKS axis
estimate towards these points. Also CCM was affected by the false positives in
the root volume (Fig. 3(b) (xz panel)) resulting in a slight shift of the model
axis. In contrast to PKS, in which the points “attracted” the axis, for CCM false
detections in the root interior were explained by an erroneous model shift in the
opposite direction.
5 Conclusion
We presented a variational approach to robustly model tubular structures defined
by their axis and thickness functions based on sparse and noisy surface point

10 T. Schmidt, M. Keuper, T. Pasternak, K. Palme, O. Ronneberger
samples. The approach is able to follow tubes of very complex bending patterns
and also allows for moderate thickness variations. When exchanging the tube
thickness constancy penalizer by a penalizer on a higher derivative degree, the
approach can be adapted to find the axis of arbitrary objects of revolution.
The possibility to introduce arbitrarily many tube seeds into the model al-
lows to simultaneously match all tubes within a data set. Although branching
structures are not yet introduced in the model, its capability to simultaneously
model multiple tubes in a data set in one energy minimization allows to trace
the single branches up to the branching point.
Acknowledgements
We thank the members of our team for helpful comments on the manuscript. We
also gratefully acknowledge the excellent technical support from Roland Nitschke
(ZBSA). This work was supported by the DFG, the Excellence Initiative of the
German Federal and State Governments (EXC 294), European Space Agency,
Bundesministerium f¨ur Bildung und Forschung (BMBF), Deutsches Zentrum f¨ur
Luft und Raumfahrt, and the Freiburg Initiative for Systems Biology (FRISYS).
References
1. Bauer, U., Polthier, K.: Generating parametric models of tubes from laser scans.
Computer-Aided Design 41(10), 719–729 (2009)
2. de Boor, C.: A practical guide to splines, Applied Mathematical Sciences, vol. 27.
Springer-Verlag, 3rd edn. (1978)
3. Friman, O., Hindennach, M., K¨uhnel, C., Peitgen, H.O.: Multiple hypothesis tem-
plate tracking of small 3d vessel structures. Med. Image Anal. 14, 160–171 (2010)
4. Kirbas, C., Quek, F.: A review of vessel extraction techniques and algorithms.
ACM Comput. Surv. 36(2), 81–121 (2004)
5. Laptev, I., Mayer, H., Lindeberg, T., Eckstein, W., Steger, C., Baumgartner, A.:
Automatic extraction of roads from aerial images based on scale space and snakes.
In: Machine Vision and Applications. vol. 12, pp. 23–31 (2000)
6. Lee, I.K.: Curve reconstruction from unorganized points. Computer Aided Geo-
metric Design 17(2), 161–177 (2000)
7. Meijering, E., Jacob, M., Sarria, J.C., Steiner, P., Hirling, H., Unser, M.: Design
and validation of a tool for neurite tracing and analysis in fluorescence microscopy
images. Cytometry Part A 58A(2), 167–176 (2004)
8. Pock, T., Beichel, R., Bischof, H.: A novel robust tube detection filter for 3d cen-
terline extraction. In: Proc. SCIA. vol. 3540, pp. 481–490 (2005)
9. Schnabel, R., Wahl, R., Klein, R.: Efficient ransac for point-cloud shape detection.
Computer Graphics Forum 26(2), 214–226 (2007)
10. Skibbe, H., Reisert, M., Ronneberger, O., Burkhardt, H.: Increasing the dimension
of creativity in rotation invariant feature design using 3d tensorial harmonics.
LNCS 5748/2009, 141–150 (2009)
11. Wang, Q., Ronneberger, O., Burkhardt, H.: Using lateral coupled snakes for mod-
eling the contours of worms. LNCS 5748/2009, 542–551 (2009)