Content uploaded by Kunio Kondo
Author content
All content in this area was uploaded by Kunio Kondo on Oct 31, 2014
Content may be subject to copyright.
Freehand Sketch System
for
Koichi
MATSUDA, Satoru SUGISHITA,
Zheng
XU,
Kunio
KONDO,
Hisashi
SAT
and
Shizuo
SHIMADA
Dept.
of
Information and Computer Sciences, SAITAMA University
Shimo-okubo
255, URAWA, SAITAMR
338,
JAPAN
{matsuda,kondo}@ke.ics.saitama-u.ac.jp
Abstract
This paper introduces a new method to deal with
sketches for inputting geometric models at a worksta-
tion. The sketches are drawn on a
CRT
screen through
a stylus pencil and a tablet by a designer at an early
stage of design procedures. He can keep their drawing
stylus at a workstation as the same manner as using a
pen on paper. The system ‘Sketch Interpreter’ can cor-
rect geometric models
in
a computer even though input
sketches are geometrically distorted. This system can
create
30
geometric models
in
a computer even though
input
data of
2D
sketches are drawn by freehand. The
system
is
constructed
in
terms of three characteristic
procedures on a workstation with a pointing device
;
(1)
to recognize hand sketches on the screen. Free-
hand bines are drawn by pen operations
from
which a
computer can construct a
30
geometric model. The
first operation computes mathematical parameters for
the projective transformation;
(2)
to consiruct addi-
tional geometric models by
inputting
more sketches
drawn perspectively; and
(3)
to
redraw modified geo-
metric models replacing the sketch lines. Our interac-
tive methods suit to realistically construct any geomet-
ric shapes a designer imagines. Data created are trans-
ferred to an advanced
3D-CAD
system. Our system is
applied as a front-end-processor of design practice such
as for ofice equapments.
1.
Introduction
At early stages of shape design,
a
designer always
tries to make his new ideas into sketches as soon as
possible. Ullman
[l]
argued that sketching is an essen-
tial activity for creative design because it allows
a
de-
signer to think aloud and to evaluate new ideas quickly.
It also assists the designer’s short term memory and
communication with other people. But current
CAD
systems do not
suit
for trying out
a
variety
of
ideas
by
thumbnail sketching. Binh Pham
[2]
mentioned
CA
still falls short of designer’s expectation, they stili re-
main to be seen more as
a
drafting tool than
a
desi
tool. Designers cannot u5e
or
at
best feel very
uficom-
fortable to use these systems
for
the early stages of
design. In current
CAD
system, specifications for geo-
metric input and constraints need to be very specific,
and unconventient to make free-form surfaces, hence
they are only useful
in
the final stages
of
design when
a designer knows exactly what the object should look
like.
In engineering design, training
is
required
for
a
de-
signer to imagine
a
3D
object that does not exist yet,
and
tG
transform it into
a
2D
sketch before making en-
gineering drawings. At the stage to imagine
a
shape,
a
designer makes brain storming by himself
for
drawing
sketches. However, at the design of engineering shapes,
sketches are drawn at an early stage of design proce-
dures, then
3D
models must be constructed
in
the
fol-
lowing many stages. Most advanced
JD-CAD
systems
demand accurate dimensions of shape to
a
designer,
and will not accept vague information as shown
in
the
sketches. The shape of
a
target object
is,
therefore, to
be modified frequently during stages
of
engineerin
sign,
so
that it is desirable to shorten the times
for
such
repeated stages. The aims of our study are to build
a
front-end processor which creates
3D
mode!s from de-
signer’s idea and to transfer the data for an advanced
3D-CAD
system.
Typical techniques to recognize
a
3D
object
from
a
2D drawing have been using the perspective theory
[3][4][5][6]
or
the orthographic view theory [7][8].
‘In
the
perspective theory,
3D
models can be created from
2D
drawings which are taken into
a
computer by
a,
scanner.
For
an instance,
a
lot of efforts have been paid to
Skq-
plify the processes to generate
3D
models
by
scanning
sketches. Akeo[6] allows users to scan sketches vhich
includes perspective vanishing lines and
3D
cross
sec-
55
0-8186-7867-4/96 $10.00
0
1997 IEEE
Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.
tions into the computer. The scanned data is projected
onto the 3D mark-up to complete the process with var-
ious editing.
In general,
a
2D view shows less information about
sizes of
a
3D object,
so
that designers have to fulfill ad-
ditional information such as depth sizes. We do not use
the orthographic view theory in this paper because our
aims deal with sketches mostly drawn perspectively.
Our Sketch Interpreter is an interactive system based
on the perspective theory. Our system is unique, since
a
sketch drawn on
a
CRT
screen is directly recognized
by
a
computer as
a
3D object. Designers can draw
sketches interactively by Sketch Interpreter as the same
manner as drawing sketches on paper. Sketch Inter-
preter records histories of designer's operation,
so
that
the lack of information can be recovered by the addi-
tional data with historical information.
Sketch Interpreter helps designers of shape design
especially for small industrial parts, office equipments,
wooden furniture in living house, etc. Textile and color
design to 3D models are not discussed in this paper.
2.
Overview
of
Sketch Interpreter
2.1.
Systems Concept
Engineering design is generally composed
of
six
stages of procedures (Fig.
1).
At every stage, the shape
of
a
target object is inspected from various aspects such
as artistic design, functional requirements, and
so
on.
When these are not satisfied,
a
modified shape is tested
from the preceding steps. Trial and correction proce-
dures take usually
a
lot of hours,
so
that it is desirable
to shorten the repeated steps. At the stage to imag-
ine
a
shape,
a
designer draws any possible ideas onto
sketches.
lil
Figure
1.
Design processes
Figure 2 exemplifies procedures of design ideas
which are usually drawn by
a
designer on paper. Figure
2-a
shows
a
target object by outlines of parallelepiped
at
the beginning of sketches drawings. More details are
then added over the rectangular object (Fig. 2-b), and
finally
a
clean copy is completed(Fig. 2-c).
Figure
2.
Hand
sketch procedures
on
paper
A designer draws rough sketches on the tablet as the
same manners as on paper, but the figures are drawn on
the graphics screen of
a
computer. Our system adopts
nearly the same manners as pencil drawings on paper,
since
a
designers is not always acquainted with com-
puter handling.
A
designer draws
a
perspective shape
of some possible rectangular parallelepipeds onto the
screen with freehand lines. A computer adjusts the
freehand lines into straight edges. Vertices are decided
as starting positions of relevant edges. Two
or
three
vanishing points in the perspective theory are reason-
ably proposed by the computer, and
a
geometrically
correct perspective view is redrawn on the screen. At
this moment,
a
view position is decided as representing
an eye of the designer. We call this perspective shape
as 'Basic Shape' which plays an important role in
our
system.
A designer then draws additional freehand lines over
the basic shape to make more details
as
though he
complete
a
picture on
a
canvas. The freehand lines
may indicate cutting lines to the model and the cut-
ting operation is carried out by pointing the part to
be eliminated. Models on the screen can be inspected
perspectively changing view positions(Fig.
3).
2.2.
Steps
of
Drawing Stage
Based on the perspective theory the system can rec-
ognize
a
2D sketch and change it into
a
3D geometric
model even though the input sketch data are uncer-
tain. The designer can also manage adding and cut-
ting operations on the basic shape, and make rounding
operation to the edges and corners. Figure
4
shows the
steps of procedures of
our
system.
At the first step,
a
designer draws freehand lines on
the screen through the tablet(Fig. 4-a). The freehand
lines are
a
series of positions which become
a
chain
56
Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.
a
j
draw an outline add details
(dr
Lch)
I
acomputer
I
t
(calculate ID-data)
I
restore
to
a
3D
objmt
*
modify
3D
object
Figure
3.
Computer
assisted
sketch proce-
dures
of pixels, but are not yet straight lines on the screen.
Five procedures are designed to correct the shape: line
recognition, unifying nodes, numbering on vertices, ad-
justing edge direction, and restoring to
a
basic shape
in
a
3D
geometric model. A computer calculates these
freehand lines to become edges of
a
3D
object and com-
pletes
a
‘Basic Shape’ (Fig. 4-b, c, d, e). As the same
manners, the designer draws more additional freehand
lines over the basic shape to add
or
to cut
off
parts
from the basic shape and makes rounding operation on
edges and corners(Fig. 4f,
g).
a.
Ptxel drawmg Line drawing Basic Shape Rectifying
twists
Restoring
to
3D-object Detailed drawings Rounding operation
Figure
4.
Steps
in
Sketch
Interpreter
3.
Thinning Algorithm for Interactive
Line Drawing
This section introduces the methods to fix an edge
line from freehand sketch lines. Practically
a
proposed
edge line is drawn by many overlapped lines by
a
de-
signer. He repeatedly draws such lines roughly the first
and more concretely the later.
A
freehand line drawn
later and later becomes preferable to be the proposed
edge line. We edit therefore an edge line from freehand
lines considering these time sequential input. The algo-
rithm falls into three categories
:
segment composing,
clipping, and segment joining.
3.1.
Segment Composing
When
a
designer proposes an edge line with
a
pencil
on paper, he composes the line from several stroke of
freehand lines. A segment is
a
straight line bundled
from
a
or
more freehand lines which are drawn during
a
short time
of
period. Another segment can be overlaid
to the preceding segment, and then two segments are
composed into
a
renewal segment. A segment has two
end points, vertices, which are the beginning and end
points of
a
freehand line. The freehand line is tested in
its shape whether it is nearly
a
straight line. When it
shows
a
curved line within
a
strip, it is assumed that
two
or
more arcs are connected to become
a
folded
line composed of two
or
more segments. The ratio
of
width to length of the strip is decided
as
about
0.1
experimentally. In order to test two segments, each
segment is identified by the coordinates
of
its middle
point and an angle
of
inclination from the horizontal
line. Four sorts of tests are carried out to compare
two segments whether they are independent
or
they
must be edited into one, that is, angles, distance, time
period, and length(Fig.
5).
Figure
5.
Composing
stroke
segments
(1)
test about angles
In order to compare two segments by angle of in-
clination, the relative angles
cr
and
/3
are calculated
as
shown in Fig.
6.
Accounting the magnitude of angles
cr
and
/3,
two segments will be unified into one segment
or
prolonged into
a
longer segment.
Figure
6.
Angle
difference
of
two
stroke
segments
57
Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.
(2)
test about distance
Distance of the middle point of
a
segment to an-
other segment line is calculated(Fig.
7).
When the
distance is bigger than
a
threshold, two segments are
independent.
Figure
7.
Distance between two segments
(3)
test about time sequence
We observed experiments how
a
designer builds his
sketches practically, and found that the later drawn
lines are more adopted then the former lines.
In
order
to synthesize two segments into one, the new segment
is
calculated as an algebraical mean with weight ratio
X
and
py
where the ratio is experimentally decided as
X:p=l:l
(4)
test about length
When two segments are edited into
a
new segment,
they are compared by their length, and the overall
!ongest length
is
adopted for the new segment.
.2.
Clipping and Joining
After many segments have been drawn indepen-
dently on the screen, every possible vertex has two
or
more segments which shall be edges joining
at
a
po-
sition. These segments may not meet
or
cross
at
the
position(Fig. 8-a). Intersected points are calculated
among the segments. When two
or
more intersected
points exist
on
a
segment, the far lying point is adopted
as
a
vertex to this segment as well as to the opponent
segment(Fig. 8-b). There may be still two
or
more
intersected points around
a
proposed vertex which is
then decided
as
a
gravity center of such points(Fig
.8-
c). After these procedures against all the segments,
vertices and edges are then decided.
As
described in the article
2.2,
a
basic shape
is
drawn
as
a
perspective view of
a
parallelepiped
for
a container
to
a
3D
model. The figure is
a
2D
image look like
Figure
8.
Clipping and Joining
(a)
sketches
(b)
Line composing
(e) Clipping
(d)
Result
Figure
9.
Thinning example
a
map. The data of the image must be then recon-
structed to have
3D
properties. Sketch Interpreter has
functions for this purpose by numbering to vertices,
adjusting edge direction, and decision
of
a
view point
for
perspective transformation.
4.1.
Numbering
to
vertices
A
3D
parallelepiped has 8 vertices, however the basic
shape drawn perspectively shows
at
most seven vertices
on the screen. Labels to these vertices are decided be-
forehand as shown in
Fig.
10.
A
designer has composed
edges by segments for the basic shape in an arbitrary
order onto the screen. We prepare
a
fixed memory area
to hold the data of edges and vertices. At most three
surfaces are visible
for
a
view
of
a
parallelepiped.
A
loop is considered to trace along edges around
a
sur-
face in reversal clockwise turn. A
or
more common
edges are found to bound two surfaces.
A
vertical edge
of such common edges gives the key for numbering to
vertices.
58
Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.
4.2.
Adjusting edge direction
A designer at first makes his sketch regardless
of
ge-
ometric accuracy to keep the perspective theory. This
causes that extended edges of
a
parallelepiped may not
meet at
a
fixed vanishing point.
A
proposed vanishing
point
is
decided along one of the common edges as the
mean position
of
other intersected points (Fig
.lo).
In
addition, two vanishing points are set on the horizontal
line. After a
or
two vanishing points are decided, all
edges except the common edges are slightly rotated to
aim such vanishing points.
decide vanislung points rotate edges
Figure
10.
Adjusting edge direction
4.3.
Decision
of
a
view point
A perspective view
is
graphically displayed on the
theoretical projective plane through
a
point which
stands with
a
distance
f
from the plane. Practically
the perspective view
is
obtained by
a
camera,
so
that
we
can suppose an imaginary camera with focus length
f,
the problem to decide
a
view points is to deduce the
focus length
f
and the center of photograph. The view
point is the position
of
lens and stands spatially with
the distance
f
from the perspective view. The view
point is obtained by graphics geometry
as
shown in
Fig.
11
under the conditions that every surface is the
right angle rectangle in
3D
space[4].
4.4.
Restoring
3-D
coordinate
system
We can extract the geometric solid model from the
2D
sketch by the method introduced by Kondo
[4][9].
A
transformation matrix is deduced from the relations.
The world coordinates
of
all the vertices can be then
induced from the data of graphic image.
\is
Figlare
11.
Decision
of
a
view
point
lar parallelepiped(Fig.
12-2).
The system calculates
these freehand lines to edges
as
2D
object(Fig. 12-b,
c,
d),
so
the shapes are incorrect perspectively. Therefore
it
is
corrected using
a
view point with vanishing points
automatically.
Figure
12.
Creating
a
basic
shape
5.
Modeling Functions
5.2.
Cutting
5.1.
Thinning
At
first,
a
designer draws freehand lines onto
a
tablet
to become
a
perspective view for
a
proposed rectangu-
A basic shape is operated either adding parts
or
cut-
ting
off
pieces from the basic shape.
A
cutting plane is
decided from three points which are intersected points
59
Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.
of additional cutting lines on
a
basic shape. The most
probable plane is calculated if four or more points are
obtained (Fig.
13).
Basic shape Rough sketches
1
Cutting Rough sketches
2
Cutting Delete Erase cutting line Another view
Figure
13.
Cutting a basic shape
5
3.
Rounding
This section briefly introduces
a
method of rounding
operation from the polyhedra generated by the Sketch
Interpreter System. Rounding operation
is
carried out
by another program. The surfaces are represented by
the four-sided BCzier patches of the second degree. The
corners of polyhedra are represented by the patches
connected in
C1
continuity and the edges of polyhedra
are rounded by the patches coenected in
G1
continuity.
We can deal with the various degrees of rounding(Fig.
14).
6.
Examples
of
Product
Design
Figure 15 shows drawing procedures for an exam-
ple.
(l)and(2)
A
basic shape is at first constructed from
freehand lines. The size
of
a
model is automatically
evaluated in
a
computer from the sketch.
A
designer
need not take care of its practical size.
(3)
Another parallelepipeds are added. Edges are par-
allel to those
of
the basic shape.
(4)and(5) Cutting procedures are carried out by using
additional lines.
(6)
Expanding of
a
basic shape is carried out.
(7)
Modification of the adding parallelepipeds is carried
(8)
Models themselves can be rotated in the screen.
out.
7.
Conclusion
Figure
14.
Rounding Example
rithm of interactive line drawing, and reconstruction
of geometric models from sketches without scanning
sketches.
A
designer makes
3D
models like sculptur-
ing, to draw cutting lines on
3D
models directly.
Freehand Sketch Interpreter System is
a
new type
of
CAD
system which increases flexibility of data in-
put for
3D
models and will become the future trend
of
CAD
system. The system helps
a
designer on de-
signing industrial parts, whose shape is composed of
straight edges. Freehand lines are directly input to
a
computer through
a
tablet and instantly redrawn on
a
CRT screen with geometrically correct sketches. The
procedures work
so
intelligent that
a
designer feels free
of worrying about dimensioning of
a
spatial shape. The
designer can work at his workstation as the same man-
ner
as
drawing sketches with straight
lines
on
pqper.
Furthermore Sketch Interpreter transfers to another
program which can generate curved surfaces for pro-
viding high quality images of
3D
objects.
This paper introduces
a
user-friendly interactive
graphic system ”Sketch Interpreter”, thinning algo-
60
Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.
1.
Draw freehand lines
3.
Add another palallelepiped
2.
A
Basic Shape
4.
Modify a Basic Shape
5.
Many times modified drawing
6.
Expand palallelepipeds
7.
Complete a drawing
8.
Change view point
Figure
15.
Example
Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.
seaaces
[I]
Ullman D.G., Wood
S.,
Craig D.
:
The Impor-
tance
of
Drawing in the Mechanical Design Process,
Comp. and
Graph.14(2),1990,263-274.
[2]
Binh Pham
:
From CAD to Automation of Cre-
ative Design Process, CAGD94,Penang,Malaysia,
July. 1994.
[3]
B.
J. Hale,
R.
P.
Burton,
D.
R.
Olsen, and
W.
D.
Stout: A three-dimensional sketching environment
using two-dimensional perspective input, Journal
of
Imaging Science And Technology, pp.188-196,
March 1992.
[4]
K.
Kondo, F. Kimura, and
T.
Tajima: Estimation
of
a
Point of View with Perspective Drawing and
the Application, Journal of Information Processing
Society
of
Japan,Vol.29,No.7,July 1988.
[5]
K.
Kondo: Representation
of
3-D shapes with the
Interactive Rendering System,
a
doctor’s thesis,
University of TOKYO Japan, October 1987.
.
Akeo
et al.:
Computer-Assisted Product Design,
NICOGRAPM’93, pp.55-65, 1993.
[7]
Z
Chen and D-B Perng: Automatic reconstruction
of 3D solid objects from 2D orthographic views,
Pattern Recognition, pp.439-449, 1988.
[8]
T.
Fujiu
et aE.:
Usage
of
Heuristic Knowledge
for Solid Construction from Engineering Draft in
Orthographic View, NICOGRAPH’86, pp.49-58,
1986.
[9]
S.
Sugishita,
K.
Kondo, H.Sato, S.Shimada
:Sketch inteqweter system for geometric modelling,
CAGD’94, Penang, Malaysia,July. 1994.
[lo]
H.
Chiyokura: Solid Modelling (Japanese), Kogyo
Chosakai Publishing Co.,pp.l-31,1985.
[Ill
N.
Toriya: Invitation to 3D modelling (Japanese),
Myoritsu Syuppan Co., July 1991.
[
121
Masuda, Yamaguchi: Automatic Reconstruction
of
Polyhedron from Orthographic Views, Proc. of
Graphics and CAD Symposium, IPSJ, pp.105-114,
1992.
[13]
M.
Ono: Performance modelling
for
pen input,
Information Processing Society of Japan’93-HI-47-
L.
[14]
S.
Furushima
et
al.:
Generation
of
3-Dimensional
geometric model from rough-sketch,
NICOGRAPW’SO, pp.11-20, 1990.
[15] Robert
C.
Zeleznik
et
al.
:
SKETCH: An Interface
for Sketching 3D Scenes, SIGGRAPH ’96 Proceed-
ings, pp.163-170, 1996
[16]
S.
Sugishita, K. Kondo
et al.:
Interactive Free-
hand Sketch Interpreter for Geometric Modelling,
Symbiosis of Human and Artifact, pp.561-566, 1995
[17] K. Matsuda
et al.
:
User Interface for Sketch
In-
terpreter, Proceedings
of
the Eleventh Symposium
on Human Interface, pp.69-76, 1995
[IS]
XU
Zheng,
et al.
:
Free-form Surfaces Generation
in Freehand Sketch Interpreter, CADEM’95, Xi’an,
China, pp.261-266, 1995
62
Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.
