Content uploaded by Kunio Kondo

Author content

All content in this area was uploaded by Kunio Kondo on Oct 31, 2014

Content may be subject to copyright.

Freehand Sketch System

for

Koichi

MATSUDA, Satoru SUGISHITA,

Zheng

XU,

Kunio

KONDO,

Hisashi

SAT

and

Shizuo

SHIMADA

Dept.

of

Information and Computer Sciences, SAITAMA University

Shimo-okubo

255, URAWA, SAITAMR

338,

JAPAN

{matsuda,kondo}@ke.ics.saitama-u.ac.jp

Abstract

This paper introduces a new method to deal with

sketches for inputting geometric models at a worksta-

tion. The sketches are drawn on a

CRT

screen through

a stylus pencil and a tablet by a designer at an early

stage of design procedures. He can keep their drawing

stylus at a workstation as the same manner as using a

pen on paper. The system ‘Sketch Interpreter’ can cor-

rect geometric models

in

a computer even though input

sketches are geometrically distorted. This system can

create

30

geometric models

in

a computer even though

input

data of

2D

sketches are drawn by freehand. The

system

is

constructed

in

terms of three characteristic

procedures on a workstation with a pointing device

;

(1)

to recognize hand sketches on the screen. Free-

hand bines are drawn by pen operations

from

which a

computer can construct a

30

geometric model. The

first operation computes mathematical parameters for

the projective transformation;

(2)

to consiruct addi-

tional geometric models by

inputting

more sketches

drawn perspectively; and

(3)

to

redraw modified geo-

metric models replacing the sketch lines. Our interac-

tive methods suit to realistically construct any geomet-

ric shapes a designer imagines. Data created are trans-

ferred to an advanced

3D-CAD

system. Our system is

applied as a front-end-processor of design practice such

as for ofice equapments.

1.

Introduction

At early stages of shape design,

a

designer always

tries to make his new ideas into sketches as soon as

possible. Ullman

[l]

argued that sketching is an essen-

tial activity for creative design because it allows

a

de-

signer to think aloud and to evaluate new ideas quickly.

It also assists the designer’s short term memory and

communication with other people. But current

CAD

systems do not

suit

for trying out

a

variety

of

ideas

by

thumbnail sketching. Binh Pham

[2]

mentioned

CA

still falls short of designer’s expectation, they stili re-

main to be seen more as

a

drafting tool than

a

desi

tool. Designers cannot u5e

or

at

best feel very

uficom-

fortable to use these systems

for

the early stages of

design. In current

CAD

system, specifications for geo-

metric input and constraints need to be very specific,

and unconventient to make free-form surfaces, hence

they are only useful

in

the final stages

of

design when

a designer knows exactly what the object should look

like.

In engineering design, training

is

required

for

a

de-

signer to imagine

a

3D

object that does not exist yet,

and

tG

transform it into

a

2D

sketch before making en-

gineering drawings. At the stage to imagine

a

shape,

a

designer makes brain storming by himself

for

drawing

sketches. However, at the design of engineering shapes,

sketches are drawn at an early stage of design proce-

dures, then

3D

models must be constructed

in

the

fol-

lowing many stages. Most advanced

JD-CAD

systems

demand accurate dimensions of shape to

a

designer,

and will not accept vague information as shown

in

the

sketches. The shape of

a

target object

is,

therefore, to

be modified frequently during stages

of

engineerin

sign,

so

that it is desirable to shorten the times

for

such

repeated stages. The aims of our study are to build

a

front-end processor which creates

3D

mode!s from de-

signer’s idea and to transfer the data for an advanced

3D-CAD

system.

Typical techniques to recognize

a

3D

object

from

a

2D drawing have been using the perspective theory

[3][4][5][6]

or

the orthographic view theory [7][8].

‘In

the

perspective theory,

3D

models can be created from

2D

drawings which are taken into

a

computer by

a,

scanner.

For

an instance,

a

lot of efforts have been paid to

Skq-

plify the processes to generate

3D

models

by

scanning

sketches. Akeo[6] allows users to scan sketches vhich

includes perspective vanishing lines and

3D

cross

sec-

55

0-8186-7867-4/96 $10.00

0

1997 IEEE

Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.

tions into the computer. The scanned data is projected

onto the 3D mark-up to complete the process with var-

ious editing.

In general,

a

2D view shows less information about

sizes of

a

3D object,

so

that designers have to fulfill ad-

ditional information such as depth sizes. We do not use

the orthographic view theory in this paper because our

aims deal with sketches mostly drawn perspectively.

Our Sketch Interpreter is an interactive system based

on the perspective theory. Our system is unique, since

a

sketch drawn on

a

CRT

screen is directly recognized

by

a

computer as

a

3D object. Designers can draw

sketches interactively by Sketch Interpreter as the same

manner as drawing sketches on paper. Sketch Inter-

preter records histories of designer's operation,

so

that

the lack of information can be recovered by the addi-

tional data with historical information.

Sketch Interpreter helps designers of shape design

especially for small industrial parts, office equipments,

wooden furniture in living house, etc. Textile and color

design to 3D models are not discussed in this paper.

2.

Overview

of

Sketch Interpreter

2.1.

Systems Concept

Engineering design is generally composed

of

six

stages of procedures (Fig.

1).

At every stage, the shape

of

a

target object is inspected from various aspects such

as artistic design, functional requirements, and

so

on.

When these are not satisfied,

a

modified shape is tested

from the preceding steps. Trial and correction proce-

dures take usually

a

lot of hours,

so

that it is desirable

to shorten the repeated steps. At the stage to imag-

ine

a

shape,

a

designer draws any possible ideas onto

sketches.

lil

Figure

1.

Design processes

Figure 2 exemplifies procedures of design ideas

which are usually drawn by

a

designer on paper. Figure

2-a

shows

a

target object by outlines of parallelepiped

at

the beginning of sketches drawings. More details are

then added over the rectangular object (Fig. 2-b), and

finally

a

clean copy is completed(Fig. 2-c).

Figure

2.

Hand

sketch procedures

on

paper

A designer draws rough sketches on the tablet as the

same manners as on paper, but the figures are drawn on

the graphics screen of

a

computer. Our system adopts

nearly the same manners as pencil drawings on paper,

since

a

designers is not always acquainted with com-

puter handling.

A

designer draws

a

perspective shape

of some possible rectangular parallelepipeds onto the

screen with freehand lines. A computer adjusts the

freehand lines into straight edges. Vertices are decided

as starting positions of relevant edges. Two

or

three

vanishing points in the perspective theory are reason-

ably proposed by the computer, and

a

geometrically

correct perspective view is redrawn on the screen. At

this moment,

a

view position is decided as representing

an eye of the designer. We call this perspective shape

as 'Basic Shape' which plays an important role in

our

system.

A designer then draws additional freehand lines over

the basic shape to make more details

as

though he

complete

a

picture on

a

canvas. The freehand lines

may indicate cutting lines to the model and the cut-

ting operation is carried out by pointing the part to

be eliminated. Models on the screen can be inspected

perspectively changing view positions(Fig.

3).

2.2.

Steps

of

Drawing Stage

Based on the perspective theory the system can rec-

ognize

a

2D sketch and change it into

a

3D geometric

model even though the input sketch data are uncer-

tain. The designer can also manage adding and cut-

ting operations on the basic shape, and make rounding

operation to the edges and corners. Figure

4

shows the

steps of procedures of

our

system.

At the first step,

a

designer draws freehand lines on

the screen through the tablet(Fig. 4-a). The freehand

lines are

a

series of positions which become

a

chain

56

Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.

a

j

draw an outline add details

(dr

Lch)

I

acomputer

I

t

(calculate ID-data)

I

restore

to

a

3D

objmt

*

modify

3D

object

Figure

3.

Computer

assisted

sketch proce-

dures

of pixels, but are not yet straight lines on the screen.

Five procedures are designed to correct the shape: line

recognition, unifying nodes, numbering on vertices, ad-

justing edge direction, and restoring to

a

basic shape

in

a

3D

geometric model. A computer calculates these

freehand lines to become edges of

a

3D

object and com-

pletes

a

‘Basic Shape’ (Fig. 4-b, c, d, e). As the same

manners, the designer draws more additional freehand

lines over the basic shape to add

or

to cut

off

parts

from the basic shape and makes rounding operation on

edges and corners(Fig. 4f,

g).

a.

Ptxel drawmg Line drawing Basic Shape Rectifying

twists

Restoring

to

3D-object Detailed drawings Rounding operation

Figure

4.

Steps

in

Sketch

Interpreter

3.

Thinning Algorithm for Interactive

Line Drawing

This section introduces the methods to fix an edge

line from freehand sketch lines. Practically

a

proposed

edge line is drawn by many overlapped lines by

a

de-

signer. He repeatedly draws such lines roughly the first

and more concretely the later.

A

freehand line drawn

later and later becomes preferable to be the proposed

edge line. We edit therefore an edge line from freehand

lines considering these time sequential input. The algo-

rithm falls into three categories

:

segment composing,

clipping, and segment joining.

3.1.

Segment Composing

When

a

designer proposes an edge line with

a

pencil

on paper, he composes the line from several stroke of

freehand lines. A segment is

a

straight line bundled

from

a

or

more freehand lines which are drawn during

a

short time

of

period. Another segment can be overlaid

to the preceding segment, and then two segments are

composed into

a

renewal segment. A segment has two

end points, vertices, which are the beginning and end

points of

a

freehand line. The freehand line is tested in

its shape whether it is nearly

a

straight line. When it

shows

a

curved line within

a

strip, it is assumed that

two

or

more arcs are connected to become

a

folded

line composed of two

or

more segments. The ratio

of

width to length of the strip is decided

as

about

0.1

experimentally. In order to test two segments, each

segment is identified by the coordinates

of

its middle

point and an angle

of

inclination from the horizontal

line. Four sorts of tests are carried out to compare

two segments whether they are independent

or

they

must be edited into one, that is, angles, distance, time

period, and length(Fig.

5).

Figure

5.

Composing

stroke

segments

(1)

test about angles

In order to compare two segments by angle of in-

clination, the relative angles

cr

and

/3

are calculated

as

shown in Fig.

6.

Accounting the magnitude of angles

cr

and

/3,

two segments will be unified into one segment

or

prolonged into

a

longer segment.

Figure

6.

Angle

difference

of

two

stroke

segments

57

Authorized licensed use limited to: Communication University of China. Downloaded on December 10, 2009 at 07:09 from IEEE Xplore. Restrictions apply.

(2)

test about distance

Distance of the middle point of

a

segment to an-

other segment line is calculated(Fig.

7).

When the

distance is bigger than

a

threshold, two segments are

independent.

Figure

7.

Distance between two segments

(3)

test about time sequence

We observed experiments how

a

designer builds his

sketches practically, and found that the later drawn

lines are more adopted then the former lines.

In

order

to synthesize two segments into one, the new segment

is

calculated as an algebraical mean with weight ratio

X

and

py

where the ratio is experimentally decided as

X:p=l:l

(4)

test about length

When two segments are edited into

a

new segment,

they are compared by their length, and the overall

!ongest length

is

adopted for the new segment.

.2.

Clipping and Joining

After many segments have been drawn indepen-

dently on the screen, every possible vertex has two

or

more segments which shall be edges joining

at

a

po-

sition. These segments may not meet

or

cross

at

the

position(Fig. 8-a). Intersected points are calculated

among the segments. When two

or

more intersected

points exist

on

a

segment, the far lying point is adopted

as

a

vertex to this segment as well as to the opponent

segment(Fig. 8-b). There may be still two

or

more

intersected points around

a

proposed vertex which is

then decided

as

a

gravity center of such points(Fig

.8-

c). After these procedures against all the segments,

vertices and edges are then decided.

As

described in the article

2.2,

a

basic shape

is

drawn

as

a

perspective view of

a

parallelepiped

for

a container

to

a

3D

model. The figure is

a

2D

image look like

Figure

8.

Clipping and Joining

(a)

sketches

(b)

Line composing

(e) Clipping

(d)

Result

Figure

9.

Thinning example

a

map. The data of the image must be then recon-

structed to have

3D

properties. Sketch Interpreter has

functions for this purpose by numbering to vertices,

adjusting edge direction, and decision

of

a

view point

for

perspective transformation.

4.1.

Numbering

to

vertices

A

3D

parallelepiped has 8 vertices, however the basic

shape drawn perspectively shows

at

most seven vertices

on the screen. Labels to these vertices are decided be-

forehand as shown in

Fig.

10.

A

designer has composed

edges by segments for the basic shape in an arbitrary

order onto the screen. We prepare

a

fixed memory area

to hold the data of edges and vertices. At most three

surfaces are visible

for

a

view

of

a

parallelepiped.

A

loop is considered to trace along edges around

a

sur-

face in reversal clockwise turn. A

or

more common

edges are found to bound two surfaces.

A

vertical edge

of such common edges gives the key for numbering to

vertices.

58

4.2.

Adjusting edge direction

A designer at first makes his sketch regardless

of

ge-

ometric accuracy to keep the perspective theory. This

causes that extended edges of

a

parallelepiped may not

meet at

a

fixed vanishing point.

A

proposed vanishing

point

is

decided along one of the common edges as the

mean position

of

other intersected points (Fig

.lo).

In

addition, two vanishing points are set on the horizontal

line. After a

or

two vanishing points are decided, all

edges except the common edges are slightly rotated to

aim such vanishing points.

decide vanislung points rotate edges

Figure

10.

Adjusting edge direction

4.3.

Decision

of

a

view point

A perspective view

is

graphically displayed on the

theoretical projective plane through

a

point which

stands with

a

distance

f

from the plane. Practically

the perspective view

is

obtained by

a

camera,

so

that

we

can suppose an imaginary camera with focus length

f,

the problem to decide

a

view points is to deduce the

focus length

f

and the center of photograph. The view

point is the position

of

lens and stands spatially with

the distance

f

from the perspective view. The view

point is obtained by graphics geometry

as

shown in

Fig.

11

under the conditions that every surface is the

right angle rectangle in

3D

space[4].

4.4.

Restoring

3-D

coordinate

system

We can extract the geometric solid model from the

2D

sketch by the method introduced by Kondo

[4][9].

A

transformation matrix is deduced from the relations.

The world coordinates

of

all the vertices can be then

induced from the data of graphic image.

\is

Figlare

11.

Decision

of

a

view

point

lar parallelepiped(Fig.

12-2).

The system calculates

these freehand lines to edges

as

2D

object(Fig. 12-b,

c,

d),

so

the shapes are incorrect perspectively. Therefore

it

is

corrected using

a

view point with vanishing points

automatically.

Figure

12.

Creating

a

basic

shape

5.

Modeling Functions

5.2.

Cutting

5.1.

Thinning

At

first,

a

designer draws freehand lines onto

a

tablet

to become

a

perspective view for

a

proposed rectangu-

A basic shape is operated either adding parts

or

cut-

ting

off

pieces from the basic shape.

A

cutting plane is

decided from three points which are intersected points

59

of additional cutting lines on

a

basic shape. The most

probable plane is calculated if four or more points are

obtained (Fig.

13).

Basic shape Rough sketches

1

Cutting Rough sketches

2

Cutting Delete Erase cutting line Another view

Figure

13.

Cutting a basic shape

5

3.

Rounding

This section briefly introduces

a

method of rounding

operation from the polyhedra generated by the Sketch

Interpreter System. Rounding operation

is

carried out

by another program. The surfaces are represented by

the four-sided BCzier patches of the second degree. The

corners of polyhedra are represented by the patches

connected in

C1

continuity and the edges of polyhedra

are rounded by the patches coenected in

G1

continuity.

We can deal with the various degrees of rounding(Fig.

14).

6.

Examples

of

Product

Design

Figure 15 shows drawing procedures for an exam-

ple.

(l)and(2)

A

basic shape is at first constructed from

freehand lines. The size

of

a

model is automatically

evaluated in

a

computer from the sketch.

A

designer

need not take care of its practical size.

(3)

Another parallelepipeds are added. Edges are par-

allel to those

of

the basic shape.

(4)and(5) Cutting procedures are carried out by using

additional lines.

(6)

Expanding of

a

basic shape is carried out.

(7)

Modification of the adding parallelepipeds is carried

(8)

Models themselves can be rotated in the screen.

out.

7.

Conclusion

Figure

14.

Rounding Example

rithm of interactive line drawing, and reconstruction

of geometric models from sketches without scanning

sketches.

A

designer makes

3D

models like sculptur-

ing, to draw cutting lines on

3D

models directly.

Freehand Sketch Interpreter System is

a

new type

of

CAD

system which increases flexibility of data in-

put for

3D

models and will become the future trend

of

CAD

system. The system helps

a

designer on de-

signing industrial parts, whose shape is composed of

straight edges. Freehand lines are directly input to

a

computer through

a

tablet and instantly redrawn on

a

CRT screen with geometrically correct sketches. The

procedures work

so

intelligent that

a

designer feels free

of worrying about dimensioning of

a

spatial shape. The

designer can work at his workstation as the same man-

ner

as

drawing sketches with straight

lines

on

pqper.

Furthermore Sketch Interpreter transfers to another

program which can generate curved surfaces for pro-

viding high quality images of

3D

objects.

This paper introduces

a

user-friendly interactive

graphic system ”Sketch Interpreter”, thinning algo-

60

1.

Draw freehand lines

3.

Add another palallelepiped

2.

A

Basic Shape

4.

Modify a Basic Shape

5.

Many times modified drawing

6.

Expand palallelepipeds

7.

Complete a drawing

8.

Change view point

Figure

15.

Example

seaaces

[I]

Ullman D.G., Wood

S.,

Craig D.

:

The Impor-

tance

of

Drawing in the Mechanical Design Process,

Comp. and

Graph.14(2),1990,263-274.

[2]

Binh Pham

:

From CAD to Automation of Cre-

ative Design Process, CAGD94,Penang,Malaysia,

July. 1994.

[3]

B.

J. Hale,

R.

P.

Burton,

D.

R.

Olsen, and

W.

D.

Stout: A three-dimensional sketching environment

using two-dimensional perspective input, Journal

of

Imaging Science And Technology, pp.188-196,

March 1992.

[4]

K.

Kondo, F. Kimura, and

T.

Tajima: Estimation

of

a

Point of View with Perspective Drawing and

the Application, Journal of Information Processing

Society

of

Japan,Vol.29,No.7,July 1988.

[5]

K.

Kondo: Representation

of

3-D shapes with the

Interactive Rendering System,

a

doctor’s thesis,

University of TOKYO Japan, October 1987.

.

Akeo

et al.:

Computer-Assisted Product Design,

NICOGRAPM’93, pp.55-65, 1993.

[7]

Z

Chen and D-B Perng: Automatic reconstruction

of 3D solid objects from 2D orthographic views,

Pattern Recognition, pp.439-449, 1988.

[8]

T.

Fujiu

et aE.:

Usage

of

Heuristic Knowledge

for Solid Construction from Engineering Draft in

Orthographic View, NICOGRAPH’86, pp.49-58,

1986.

[9]

S.

Sugishita,

K.

Kondo, H.Sato, S.Shimada

:Sketch inteqweter system for geometric modelling,

CAGD’94, Penang, Malaysia,July. 1994.

[lo]

H.

Chiyokura: Solid Modelling (Japanese), Kogyo

Chosakai Publishing Co.,pp.l-31,1985.

[Ill

N.

Toriya: Invitation to 3D modelling (Japanese),

Myoritsu Syuppan Co., July 1991.

[

121

Masuda, Yamaguchi: Automatic Reconstruction

of

Polyhedron from Orthographic Views, Proc. of

Graphics and CAD Symposium, IPSJ, pp.105-114,

1992.

[13]

M.

Ono: Performance modelling

for

pen input,

Information Processing Society of Japan’93-HI-47-

L.

[14]

S.

Furushima

et

al.:

Generation

of

3-Dimensional

geometric model from rough-sketch,

NICOGRAPW’SO, pp.11-20, 1990.

[15] Robert

C.

Zeleznik

et

al.

:

SKETCH: An Interface

for Sketching 3D Scenes, SIGGRAPH ’96 Proceed-

ings, pp.163-170, 1996

[16]

S.

Sugishita, K. Kondo

et al.:

Interactive Free-

hand Sketch Interpreter for Geometric Modelling,

Symbiosis of Human and Artifact, pp.561-566, 1995

[17] K. Matsuda

et al.

:

User Interface for Sketch

In-

terpreter, Proceedings

of

the Eleventh Symposium

on Human Interface, pp.69-76, 1995

[IS]

XU

Zheng,

et al.

:

Free-form Surfaces Generation

in Freehand Sketch Interpreter, CADEM’95, Xi’an,

China, pp.261-266, 1995

62