Content uploaded by Zuzana Kukelova

Author content

All content in this area was uploaded by Zuzana Kukelova

Content may be subject to copyright.

SKETCH-BASED MODELING SYSTEM WITH CONVOLUTION AND

VARIATIONAL IMPLICIT SURFACES

Zuzana K´

ukelov´

a∗

Center for Machine Perception, Department of Cybernetics

Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic

Roman ˇ

Durikoviˇ

c†

Faculty of Natural Sciences, University of Saint Cyril and Metod

Trnava, Slovakia Nam. J. Herdu 2, 917 01 Trnava, Slovak Republic

ABSTRACT

In this work we focus on function representation suitable for creating 3D freeform shapes with sketched silhouette

curves. We aim at comparison of two approaches for sketch-based modeling, namely the skeleton-based convolu-

tion surfaces and variational implicit surfaces. Both methods are extended to handle smooth set theoretic operations

between components deﬁned by sketching silhouette curves on different projection planes. Our approach includes

additional extensions to sketch-based systems like automatic skeleton generation that can be directly used for

object animation, carving operation, creating surfaces with handles and surface texturing.

1 INTRODUCTION

Sketching is a very natural process for most people, it makes the design faster and intuitive. In this work we focus

on designing a 3D freeform shape from sketched silhouette curves supporting natural human abilities and offering

the rich expression afforded by pen movements.

First freeform sketched-based modeling system Teddy [4] used polygonal mesh as geometric representation. In

recent years it has been, in most of the freeform user interfaces, replaced by implicit surface representation because

polygonal representation has limitations on surface smoothness and fairness. Implicit functions allow to introduce

new interesting operations on freeform models like blending union or carving that are very difﬁcult to implement

with polygonal meshes.

Owada et al. [6] proposed a similar interface as the Teddy system but instead of polygonal representation they

used volumetric representation. Volumetric representation can achieve smooth result but at the cost of higher

storage and also higher computation time. Turk et al. [11] ﬁrst adopted the variational implicit surfaces for object

modeling. The main problem of their approach is the difﬁculty of reconstruction of the 3D shape thickness. The

method works well for ellipse-like shapes other shapes hard to create. One of recent freeform user interfaces that

use skeletons is system ConvMo [10] which works based on cylindrical convolution surfaces. A weakness of their

approach is that it produces unwanted bulges. Above disadvantages of ConvMo system were partially eliminated

∗e-mail: kukelova@cmp.felk.cvut.cz

†e-mail: roman.durikovic@fmph.uniba.sk Also with Faculty of Mathematics, Physics and Informatics, Comenius University, Mlynska

dolina, 842 48 Bratislava, Slovakia

in [1] by representing the surface as blended spheres, placed along the skeleton similar to [12]. Spheres have

smaller regions of inﬂuence as convolution cylinders, so they follow more faithfully the shape of silhouette curve.

Inspired by the above approaches we have developed a system consisting of two approaches the skeleton based

convolution surfaces and variational surfaces to reconstruct a 3D shape from user deﬁned silhouette. By comparing

of two approaches we conclude that the mixture of both methods for different parts of the shape should work best

of all. In our ﬁrst approach we reduce unwanted bulging effect by simplifying the skeleton and diminishing of

ﬁeld contribution at skeletal joints. Since this approach is not enough to remove the effect, we proposed the global

interpolation with initial solution found by local interpolation step. Moreover, we propose the method for reduction

of constraints thus reducing the computation costs of variational implicit surface and unwanted oscillations on

surfaces. At the end we propose several operation for shape manipulation that are robust and very simple to

implement.

The paper is organized as follows. In Section 2 we discuss the previous work on function shape representation.

The proposed methods are discussed in Section 3 We demonstrate the results obtained using our techniques in

Section 4and we conclude this paper in Section 5

2. SHAPE REPRESENTATION: F-REP

In the process of sketching the objects the algorithms must perform a variety of geometrical calculations, therefore

the general mathematical representation is crucial shape representation. We have focus on function representation

based on two deﬁning functions, namely convolution surfaces generated by skeletons and variational implicit

functions. The convolution surfaces generated by skeletons are well suited for animatable shapes, since they can

be stored it in a very compact way. On the other hand, variational functions have the beneﬁt of locally deﬁned ﬁeld

functions to optimize the modeling process; unfortunately, such shapes have no skeleton and thus are not suited

for animation.

Let us consider closed subsets of n-dimensional Euclidian space Enwith the deﬁnition:

f(x1, x2, ..., xn)≥0,

where fis a real continuous function deﬁned on En. The above inequality is called a function representation

(F-rep) of a geometric object and function fis called the deﬁning function [7].

2.1 Shape from convolution cylinders

Let us consider from now on the deﬁning function given by the convolution operator between a kernel and all the

points of a skeleton. A skeleton-based implicit surface generated by a set of skeleton primitives ri, i = 1, ..., n

with associated energy function Fiis commonly deﬁned as an iso-surface satisfying the following equation:

S=(p∈R3:

n

X

i=1

Fi(p) = T).

Parameter Tis the iso-potential threshold value and energy functions Fiare monotonically decreasing functions

of distance rifrom the skeleton primitive: The shape of a convolution surface can be varied in several ways: by

changing the skeleton, the modifying the convolution kernel, and by spatial deformation of convolution surface.

Jin at al. proposed a polynomial weighted kernel along line segment, arc, and quadric curve

F(p,c(·), w(·)) = Zl

0

w(t)

(1 + s2r2(t))2dt,

where c(t),0≤t≤lis a curve, w(t)is a weight and r2(t) = kr−c(t)k. Authors also derived the analytical

solution of the integral.

2.2 Shape from variational functions

Variational implicit surface is an interpolative function, f, satisfying:

f(ci) =

0for each known surface point ci,

1for ciinside the surface,

−1for cioutside the surface.

Thinking about points cias constraints we can formulate the following scattered-data interpolation problem:

Problem 1 Given a set of kconstraint points c1,..., ckscattered over xy-plane, with associated scalar height

values h1,...,hk, construct a surface that interpolates the constraints (ci, hi), e.g. f(ci) = hifor i= 1, . . . , k.

The thin-plate solution to this problem is the function f(x)that satisﬁes the constraints and minimizes the energy

function E(f) = RΩf2

xx(x) + 2f2

xy(x) + f2

yy (x)dx, where fxx,fxy , and fyy are second partial derivatives. We can

thing of fas a linear combination of radial basis functions (RBF) centered at constraint points ci. General RBF

can be written in form fi(x) = φ(kx−cik)that depends only on the distance between argument point xand RBF

center ci. Taking into account the linear and constant part of the solution we can write the resulting interpolated

function as

f(x) =

k

X

i=1

wiφ(kx−cik) + P(x),

where P(x) = a·x+b,a∈R3and b∈R. The problem is to determine weights wiand coefﬁcients of P(x)

which yields to the system on linear equations:

φ11 φ12 . φ12 1cx

1cy

1cz

1

φ21 φ22 . φ22 1cx

2cy

2cz

2

. . . . . . . .

φk1φk2. φk21cx

kcy

kcz

k

1 1 .1 0 0 0 0

cx

1cx

2. cx

k0 0 0 0

cy

1cy

2. cy

k0 0 0 0

cz

1cz

2. cz

k0 0 0 0

w1

w2

.

wk

b

ax

ay

az

=

h1

h2

.

hk

0

0

0

0

where ci= (cx

i, cy

i, cz

i)and φij =φ(kci−cjk). Last four rows in the system correspond to orthogonality

conditions: k

X

j=1

wj=

k

X

j=1

wjcx

j=

k

X

j=1

wjcy

j=

k

X

j=1

wjcz

j= 0.

The associated linear system will always have a solution, therefore it can be solved by iterative methods or SVD

decomposition.

The last thing that should be done is the deﬁnition of constraints. To simplify the process Turk [11] proposed

four different constraints:

1. Boundary constraints (ci, hi)verify that f(ci) = hiwith hi= 0 and the surface must pass through points

ci.

2. Interior constraints (ci, hi)verify that f(ci) = hiwith hi>0. Points ciare located arbitrary inside the

surface.

3. Exterior constraints (ci, hi)verify that f(ci) = hiwith hi<0. Points ciare located arbitrary outside the

surface.

4. Normal constraints (ci, hi)specify the surface normal and are located outside the surface at a small distance

from a boundary in direction ~

n. They verify that f(ci+ε~

n) = hiwith hi<0.

3. RECONSTRUCTION OF SURFACE FROM SILHOUETTE CURVE

In this section we describe the main steps for creating convolution and variational implicit surfaces from sketched

silhouette curves. Both methods have some common steps but they differ in technique used for extracting the data

needed for shape reconstruction from the silhouette curve.

3.1 Skeleton based surface

The following method summarizes the main steps of skeleton based convolution surface reconstruction from

sketch:

1. Process the input stroke and extract the skeleton from its simpliﬁcation.

2. Convolution surface construction by local ﬁtting of weighted kernel along the skeleton segments to the sil-

houette curve.

3. Global ﬁtting of convolution surface to the silhouette by least-square method.

4. Modify the generated surface components by merging, carving, and other editing operations, refer to Fig. 1.

3

4

2

Constructing

convolution surface

Visualization of

convolution surface

1

Silhouette

curve

Filtering of the

input stroke

Projecting

filtered stroke

CDT & Skeleton

extraction

Further component-based

editing operations

New projection

plane

Global fitting of

convolution surface

End of

creating

3

2

Constructing VIS

Visualization of VIS

1

Silhouette

curve

Filtering of the

input stroke

Projecting

filtered stroke

CDT & Skeleton

extraction

Further editing operations

New projection

plane

End of

creating

Figure 1. Basic steps. Left: Skeleton based convolution surface generation from sketch.

Right: Implicit surface construction from sketch.

Figure 2. Process of generating convolution surface from silhouette. From left: Filtered

silhouette curve; result of CDT; terminal (blue), sleeve (green), and junction (red) triangles,

inﬂuence circles; simpliﬁed skeleton; reconstructed convolution surface.

3.1.1 Processing the input stroke

Filtering the input stroke. At this step the silhouette curve drawn by the user is converted into a simple polygon by

sampling the input device movement. Several different algorithms can be found for point reduction in a polygon.

In general they can be calciﬁed into two groups those dependent on screen resolution using the edges with uniform

length [4] and those using adaptive edge length [3, 10]. We use a modiﬁed Douglas and Peucker algorithm [3].

Original nonuniform distribution of points along the silhouette is re-sampled to space the points uniformly on

simpliﬁed polygon, see Fig. 2. All simpliﬁed strokes with self-intersections are rejected at this stage and user is

requested to draw a new stroke.

Projecting ﬁltered stroke. At this point we transform the stroke from screen coordinates to 3D coordinates by

projecting it to a projection plane. Projection plane is parallel to screen plane passing through 3D position of a ﬁrst

silhouette point obtained from camera depth, see Fig. 3.

Projection plane

First point of the silhouette

Figure 3. Projection of ﬁltered stroke.

Skeleton extraction. At this step a good skeleton must be found to get the best guess of surface close to the

silhouette. Similar to chordal axis transform [8], ﬁrst, the constrained Delaunay triangulation (CDT) is applied,

where constrains are the edges of ﬁltered polygon. Second, triangles are clacifﬁed into three categories, see Fig. 2

top raw:

•Terminal triangle (T-triangle) is a triangle with two external edges (marked in blue),

•Sleeve triangle (S-triangle) is a triangle with one external edge (marked in green),

•Junction triangle (J-triangle) is a triangle that has no external edges (marked in red).

To ﬁnd a good initial guess an algorithm based on medial axis comprising of line segments should be used as a

skeleton. Unfortunately this approach will produce unstable results for so called ”folding” strokes [10].

We propose the skeleton extraction algorithm which unlike [10] uses not only circumscribed circles of triangles

but rather for T-triangle it uses inscribed circles with their incenters; for J-triangles it uses circumscribed with

circumcenters; and for S-triangles it uses either circumscribed circles or circles with external edge of polygon as

tangent and third vertex of triangle as point of circle.

3.1.2 Constructing a convolution surface.

To ﬁnd the convolution that ﬁts given silhouette, we need to determine parameters of its ﬁeld function F(r).

The ﬁrst approximation to F(r)is computed by considering each skeleton segment being isolated, neglecting the

inﬂuence on other elements. In such a case for a single line segment with end points p,qand length lwe assign

weights w0and w1at the end points by setting the ﬁeld contribution at pto rpat distance Rpand at qto rqat

distance Rq, see Fig. ??.Rxdenotes the distance from center xto the point where ﬁeld function is equal to the

threshold T, e.g. F(rp) = F(rq) = T. Last equation can be written as

lT =w0(lFl(rp)−Ft(rp))+w1Ft(rp),

lT =w0(lFl(rq)−Ft(rq)) + w1Ft(rq).

Because Ft(rp)and lFl(rq)−Ft(rq)are small we get the estimation of weights as follows:

w0=lT

(lFl(rp)−Ft(rp)) ,

w1=lT

Ft(rq).

Unfortunately, the resulting surface has a bulge in the middle. We use similar idea as in [10] to ensure that the

surface passes through the point rmid, which is at the distance (Rp+Rq)/2we scale the weight w0and w1by a

correction factor of κ=T /F (rmid).

Finally, the ﬁeld function of resulting convolution surface consisting of several linear skeletal elements can be

obtained by summing the ﬁeld function for all line segments Li,i= 1,2, . . . , n:F(r) = Pn

i=1 λiF(r, Li), where

lambdaiis control scale factor. In the ﬁrst approximation step lambdai= 1.

3.1.3 Global ﬁtting of the convolution surface.

With lambdai= 1 and weights w0, w1computed as described above, one could notice that the surface does not

interpolate the silhouette curve, see Fig. 4 top. This is caused by the fact that when ﬁeld contribution of all segments

are summed the resulting surface begins to inﬂate. This inﬂating effect is fully suppressed in global interpolation

step. We solve the constrained least-squares problem for unknowns lambdai= 1 while minimizing the distance

between the convolution surface and the silhouette polygon points.

Figure 4. Surface without and with global interpolation.

3.1.4 Operations

Merging of two components is for free because two ﬁeld functions are summed up to form a smooth surface. In or-

der to have larger control over the blending of two components the F-rep blending union operation is implemented.

Carving operation is also implemented as F-rep blending substraction operation.

Surfaces with handles can be created by substraction operation between two components. Hoverer, there is a

way create such surfaces directly form two silhouettes one inside the other one. Our algorithm can handle such

cases and the result will be a torus.

3.2 Variational implicit surface

The main steps of a method for reconstruction of variational implicit surfaces (VIS) are as follows:

1. Process the input stroke and extract the skeleton from its simpliﬁcation.

2. Construction if variational implicit surface by deﬁnition of boundary and normal constraints. Final VIS then

matches the silhouette curve.

3. Global ﬁtting of convolution surface to the silhouette by least-square method.

4. Modify the generated surface components by merging, carving, and other editing operations, refer to Fig. 1.

3.2.1 Processing the input stroke

The ﬁrst step of VIS reconstruction method from sketched silhouette curve, processing of the input stroke, is

similar to the steps from Section 3.1.1. First two step Filtering the input stroke and Projecting ﬁltered stroke are

exactly the same steps as in Section 3.1.1.

Skeleton extraction. At this step a good skeleton must be found with the height value for each skeleton vertex.

The third step is a bit different. To ﬁnd the skeleton we use our proposed method from above, this guaranties us

non-redundant constraints for skeleton vertices. This step is different from commonly used approaches based on

VIS where they had many constraints and big matrices [2]. Finally, we assign the height value to each skeleton

vertex equal to the average distance from the sketched polygon vertices in the neighborhood. By this approach we

have eliminated unwanted surface oscillations.

3.2.2 Constructing a variational surface

Construction of variational implicit surface requires the speciﬁcation of constraints. For each silhouette point we

compute the outside normal to the silhouette polygon. Then a normal constraint is deﬁned by shifting the boundary

constraint placed at the silhouette point by εin normal direction. Such normal constraints all lie in the silhouette

plane. For each skeleton vertex we generate two normal constraints, one above and one below the silhouette plane.

After deﬁning these constraints we have all necessary information to create the linear system from Sec. 2.2. The

resulting function interpolates the constraints and no additional step are needed to correct the solution.

3.2.3 Operations

Carving operation wasn’t proposed in previous sketch-based systems based on VIS, even though the mathematical

concept is very easy. Similar, to merging algorithm, we eliminate the constraints of the carved component which

lie in the intersection of two input components and of eliminating the constraints of carving component which lie

outside the carved component. The constraints of carving object that remained are reversed, see Fig. 5.

In order to have better control over the blending of two components the F-rep blending substraction operation

can also be implemented.

Figure 5. VIS carving. Left: The constraints of carving object that remained in use are re-

versed. Center: Carving example. Right: Examples of various objects created in SketchCo.

4 COMPARISON OF PROPOSED METHODS

In this chapter we want to compare two methods for creating 3D freeform shapes from 2D sketched silhouette

curves proposed above. Namely, possibilities and limits of skeleton based convolution surface and variational

implicit surface for sketch-based modeling.

In the case of VIS, the resulting 3D freeform surface can be generated directly from silhouette points because

VIS are speciﬁed by a set of boundary or normal constraints. Therefore it is sufﬁcient to sample input stroke and

sample its points for constrain deﬁnition. On the other hand skeleton based convolution surface needs to ﬁnd the

skeletal elements for their deﬁnition, however such objects are directly suitable for animation.

Moreover, robustness of resulting linear system strongly depends on the set of constraints, the change of a

single constraint can change the result signiﬁcantly. Considering the shape manipulation, VIS allows very easy to

implement basic operations, however, when more control is needed over the operation (such as blending of shape

) skeleton based representation suites better the purpose.

5 CONCLUSION

Proposed methods were implemented as SketchCo modeling system. Models in Figure 5 were created in 6

minutes by children from 6 to 15 years. We have considered several geometric representations for sketch-based

modeling. Finally, we have proposed several modiﬁcations of two implicit surface representations, the skeleton

based convolution surfaces with linearly weighted distribution and variational implicit surfaces with few editing

operations. In our ﬁrst approach with skeleton based convolution surfaces we reduced unwanted bulging effect that

appears around junctions of blend line segments by simplifying the skeleton and diminishing of ﬁeld contribution

at skeletal joints. Since this approach is not enough to remove the effect, we proposed the global interpolation with

initial solution found by local interpolation step. In our approach using VIS we proposed the method for reduction

of constraints thus reducing the computation costs and unwanted oscillations on surfaces.

References

[1] A. Alexe, V. Gaildrat, and L. Barthe. Interactive modelling from sketches using spherical implicit functions. In AFRI-

GRAPH ’04: Proceedings of the 3rd international conference on Computer graphics, virtual reality, visualisation and

interaction in Africa, pages 25–34, New York, NY, USA, 2004. ACM Press.

[2] B. D. Araujo and J. Jorge. Blobmaker: Free-form modelling with variational implicit surfaces. In Proceedings of 12th

Encontro Portuguess de Computacao Graﬁca, 12 EPCC.

[3] D. Douglas and T. Peucker. Algorithms for the reduction of the number of points required to represent a digitized line or

its caricature. Canadian Cartographer, 10(2):112–122, 1973.

[4] T. Igarashi, S. Matsuoka, and H. Tanaka. Teddy: A sketching interface for 3d freeform design. Proceedings of SIGGRAPH

99, pages 409–416, August 1999. ISBN 0-20148-560-5. Held in Los Angeles, California.

[5] J. McCormack and A. Sherstyuk. Creating and rendering convolution surfaces. Computer Graphics Forum, 17(2):113–

120, 1998.

[6] S. Owada, F. Nielsen, F. Nakazawa, and T. Igarashi. A sketching interface for modeling the internal structures of 3d

shapes. In Proceedings of 3rd International Symposium on Smart Graphics, Lecture Notes in Computer Science, pages

49–57, Heidelberg,Germany, 1999. Springer.

[7] A. Pasko, V. Adzhiev, A. Sourin, and S. V. Function representation in geometric modelling: concepts, implementation

and applications. The Visual Computer, 11(8):429–444, 1995.

[8] L. Prasad. orphological analysis of shapes. CNLS Newsletter, (139):1–18, 1997.

[9] A. Sherstyuk. Interactive shape design with convolution surface. In Proceeding of Shape Modeling and Applications,

Aizu-Wakamatsu, Japan, pages 56–65, Los Alamitos, California, 1999. IEEE CS.

[10] C. L. Tai, H. Zhang, and C. K. Fong. Prototype modeling from sketched silhouettes based on convolution surfaces.

Computer Graphics, 23(1):71–83, 2004.

[11] G. Turk and J. F. O’Brian. Modeling with implicit surfaces that interpolate. ACM Transactions on Graphics, 21(4):855–

873, 2002.

[12] R. ˇ

Durikoviˇ

c, K. Kaneda, and H. Yamashita. Animation of biological organ growth based on l-systems. Computer

Graphics Forum (EUROGRAPHICS’98), 17(3):1–13, Aug. 1998.