Page 1

A New Feature Integration Approach and Its Application to 3D Model

Retrieval

Jau-Ling Shih, Chang-Hsing Lee, Chih-Hsun Chou, Yu-Cheng Chang

Department of Computers Science and Information Engineering, Chung Hua University,

Hsinchu, Taiwan

E-mail: {sjl, chlee, chc, m09602057}@chu.edu.tw

Abstract

In recent years, advanced techniques on digitization

and visualization of 3D models have made 3D models

as plentiful as images and video. The rapid generation

of 3D models has made the development of efficient 3D

model retrieval systems become urgently. In this paper,

we will propose a feature integration approach in

which a weighted distance method is developed to

combine the distance evaluated by each individual one

of the descriptors. The weight associated with each

feature descriptor can be automatically determined

according to the retrieval result using each individual

feature descriptor. Experiments conducted on the

Princeton Shape Benchmark (PSB) database have

shown that the proposed feature integration approach

provides a promising retrieval result.

1. Introduction

The rapid generation of multimedia information has

made the development of efficient multimedia retrieval

systems become urgently. Traditional multimedia

retrieval systems employ keywords to search the

interested multimedia data. In general, the managers

must manually annotate well-chosen keywords for

every multimedia data. If the database is very large, the

task is laborious and time consuming. Moreover,

different manages will reveal different interpretations

on the same multimedia data, which makes the

keywords annotated on the same multimedia data

differ from person to person. To overcome the

inefficiency and difficulties

retrieval, an alternative approach, content-based

multimedia retrieval, has become a popular research

topic.

__________________________

This research was supported in part by the National Science

Council of R.O.C. under contract NSC-96-2221-E-216-041-

MY2.

of keyword-based

In recent years, advanced techniques on digitization

and visualization of 3D models have made 3D models

as plentiful as images and video. The primary

challenge to a content-based 3D model retrieval system

[1] is to extract proper features to discriminate between

the diverse shapes of 3D models. Various features have

been proposed to describe the diverse characteristic of

3D models.

Osada et al. [2] proposed five features, A3, D1, D2,

D3, and D4, to represent each 3D model by the

probability distributions of geometric properties

computed from a set of randomly selected points

located on the surface of the model. These geometric

properties, including distance, angle, area, and volume,

are employed to describe the shape distribution.

Among these distributions, the most effective one is

D2, which measures the distribution of distances

between any two randomly selected points.

In typical mesh-based representation of 3D models,

many polygonal meshes are required to finely represent

the complex components of a 3D model. As a result, an

area weighted defect will occur since the random

sampling of surface points is greatly affected by the

complex components. To deal with this problem, Shih

et al. [3] proposed a descriptor called grid D2 (GD2) to

alleviate this problem. In GD2, a 3D model is first

decomposed into a voxel grid. Instead of performing

sampling on random points, they performed the

random sampling operation on voxels within which

some polygonal surfaces are located.

Vranic et al. [4] applied Fourier transform to the

sphere with spherical harmonics to generate embedded

multi-resolution 3D shape features. To be rotation

invariant, pose normalization must be conducted prior

to feature extraction. Therefore, Funkhouser et al. [5]

Proposed a modified rotation invariant shape descriptor

based on the spherical harmonics in which no pose

normalization is need [6, 7]. First, a 3D model is

decomposed into a collection of spherical functions

which are derived by interesting the model with a set

2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing

978-0-7695-3762-7/09 $26.00 © 2009 IEEE

DOI 10.1109/IIH-MSP.2009.255DOI 10.1109/IIH-MSP.2009.255

10261026 978-0-7695-3762-7/09 $26.00 © 2009 IEEE

Page 2

of concentric spheres of different radii. Each spherical

function is decomposed into a number of harmonics of

different spatial frequencies. The sum of norms of all

spatial frequency components at the same radius is

regarded as the shape descriptor of a spherical

function. The descriptors of all spherical functions will

constitute the shape descriptor of a 3D model. The

reason for the descriptor to be rotation invariant is that

rotating a spherical function does not change the

energies in each frequency component.

The shape spectrum descriptor (SSD) [8] is adopted

in the MPEG-7 standard for 3D model retrieval. SSD

represents the histogram of curvatures of all points on

the 3D surface. The advantages of SSD are that it can

match two 3D models without first aligning the 3D

objects, and that it is robust to the tessellation of the

3D polygonal model.

Reisert and Burkhardt have compared the retrieval

performance of different geometric properties and

developed some methods which combine a number of

features to improve the performance [9], including

distance histograms (D), SHT distance histograms

(SD), Extended SHT distance histograms (SDE),

Alpha/distance histograms

histograms (BD), Alpha/Beta histograms (AB),

Alpha/Beta/distance

Alpha/beta/distance SHT histograms (ABSD) and

Alpha/beta/distance extended

(ABSDE).

Shih et al. proposed the elevation descriptor (ED)

derived from six elevations for 3D model retrieval

[10]. Each elevation is represented by a 2D gray-level

image in which the gray level of each pixel denotes the

elevation information. For each gray-level image, the

features that describe the energies at different radii

were extracted to form the ED. The experimental

results have shown that the ED outperforms other

descriptors such as spherical harmonics (SH), MEPG-7

SSD [8], and D2 [2].

Shih et al. [11] also proposed the principal plane

descriptor (PPD) in which a 3D model is first

transformed into a 2D binary image by projecting it on

the principal plane, the symmetric surface of a 3D

model [12]. Moreover, for exactly representing a 3D

model, the second and third planes were also calculated

to obtain the other two binary projecting images.

Ricard et al. [13] presented the generalization of the

2D ART descriptor to describe 3D shape (3D ART).

The 3D ART descriptor is robust to translations,

scaling, multi-representation

distortions), noises and 3D rotations.

In fact, a single feature is insufficient to represent

all kinds of 3D models appropriately. Therefore, the

retrieval accuracy can be improved if different features

can be appropriately combined. In this paper, we will

(AD), Beta/distance

histograms (ABD),

SHT histograms

(remeshing, weak

propose a feature integration method to effectively

combine different features in order to improve the

retrieval accuracy. Six feature descriptors, including

A3 [2], grid D2 (GD2) [3], beta/distance histogram

(BD) [9], elevation descriptor (ED) [10], principal

plane descriptor (PPD) [11] and 3D-ART [13] are

combined for 3D model retrieval. We will develop a

weighted distance approach in which the weight

associated with each feature descriptor can be

automatically determined depending on the retrieval

result using each individual feature descriptor.

2. Proposed feature integration approach

Typical, no single one feature will always achieve

the best retrieval performance for every class of 3D

models. Therefore, a weighted distance approach

which combines the distances computed from different

descriptors will be developed in an attempt to get a

better retrieval result. In fact, the weighted distance

approach was widely used in many image retrieval

systems [14]. However, the weight associated with

each feature descriptor is usually determined according

to the relevance feedback of user interactions. In this

paper, we will exploit the retrieval result of each

individual feature descriptor

determine the weight associated with each feature

descriptor. Six features, A3 [2], GD2 [3], BD [9], ED

[10], PPD [11], 3D-ART [13], will be combined to find

similar 3D models. Each individual one of these

descriptors is employed to find its corresponding set of

similar 3D models. Given a query model, let Sm =

{sm,1, sm,2, …, sm,k} denote the set containing the indices

of the top k models retrieved by using only the m-th

feature descriptor, sm,i denotes the index of the model

with rank i when the m-th feature descriptor is used for

3D model retrieval. In this paper, k is set as 20. For the

i-th model in the database, its grade associated with the

m-th feature descriptor is defined as follows:

if, 1

)(

⎪⎩

Then, the overall grade of the i-th model is defined as

follows:

6

1∑

=

m

Therefore, if a model appears frequently among all

Sm’s, this model will get a high grade value. Finally,

we can calculate the weight associated with each

individual feature descriptor as follows:

∑

∈

m

Si

to automatically

,

otherwise, 0

⎪

⎨

⎧

∈

=

m

m

Si

ig

(1)

, )()(

=

migiG

(2)

, )i(

=

m

G

ω

(3)

10271027

Page 3

That is, the weight associated with the m-th feature

descriptor will be large if every corresponding

retrieved model yields a high grade value.

Let {v1, v2,…, v6} and {u1, u2, …, u6} denote

respectively the sets of feature descriptors of the query

model and a matching model, vm and um (1≦m≦6)

denote respectively the m-th feature descriptors of the

query model and the matching model. The distance

between vm and um is defined as follows:

)()(

)(

,,

∑

=

j

minm maxm

where lm is the length of the m-th descriptor, vm(j) and

um(j) denote the j-th feature values in vm and um, um,max

and um,min are the maximum and minimum values

among the feature values in the m-th descriptor of all

matching models. Finally, the weighted distance is

defined as follows:

6

1∑

=

m

The similarity measure between the query model

and a matching model is defined as the inverse of the

weighted distance:

1

w

D

The larger the similarity value, the more similar a

matching model is to the query one.

3. Experimental results

To demonstrate the effectiveness of the proposed

method for various 3D models, some experiments have

been conducted on the Princeton Shape Benchmark

(PSB) database [15]. The PSB database contains 1814

models (161 classes) which contains 907 training

models (90 classes) and 907 test models (92 classes).

Note that in this database the number of models is

different for each class. Fig. 1 shows some example

models in the PSB database.

The retrieval performance is measured by the recall

value. Because the number of models in each class is

different, the recall value for the j-th query model in

the i-th class is defined as follows:

,/ )()(

iii

TjNjRe

=

(7)

where Ni(j) is the number of relevant models in the

retrieval set, Ti is the total number of relevant models

in the database. In our experiments, each model in the

database is presented as a query one. The average

recall is defined by the following equation:

1

∑∑

==

ij

S

T

where Ts=T1+T2+…+T92. Table 1 compares the

retrieval results in terms of the average recall values.

Form this Table, we can see that our proposed feature

,

1

−

−

=

m l

mm

uu

jujv

mD

(4)

, )(

=

m

w

mDD

ω

(5)

,Sim =

(6)

, )j(

92

11

=

T

i

i

ReRe

(8)

integration approach is better than each individual

feature descriptor as well as the simple feature

combination approach which assign equal weight to

each feature descriptor. The simple equal weight

approach can improve the recall value by 4%

compared to the best feature descriptor, whereas the

proposed feature descriptor integration approach can

further raise the recall value by 2%. Fig. 2 gives an

example to illustrate the query result of the motorcycle

model by using each feature descriptor, equal weight

combination approach, and our proposed feature

integration method.

(a)

(b)

Fig. 1 The Princeton Shape Benchmark (PSB)

database [15]. (a) Some classes in PSB database. (b)

All models belong to the biplane class.

Table 1 Comparison of the recall value. (P is the

total number of retrieval models)

Method Recall

(P=Ti) (P=2Ti)

ED [10] 0.3679 0.4570

PPD [11] 0.3274 0.4257

3D-ART [13] 0.2945 0.3720

GD2 [3] 0.2770 0.3691

BD [9] 0.2516 0.3183

A3 [2] 0.2043 0.2630

Equal weight

Combination

Our method 0.4205 0.5248

4. Conclusions

In this paper, we propose a feature integration

approach in which a weighted distance method is

developed to combine the distance evaluated by each

individual one of the descriptors. The weight

associated with each feature descriptor can be

automatically determined according to the retrieval

result using each individual feature descriptor.

Experiments conducted on the Princeton Shape

Benchmark (PSB) database have shown that the

proposed feature integration approach provides a

promising retrieval result.

Recall Recall

(P=3Ti)

0.5140

0.4851

0.4179

0.4249

0.3665

0.3047

0.5580

Recall

(P=4Ti)

0.5582

0.5295

0.4537

0.4669

0.4041

0.3385

0.5941 0.4050 0.5035

0.5788 0.6173

1028 1028

Page 4

References

[1] D. Zarpalas, P. Daras, A. Axenopoulos, D. Tzovaras and

M.G. Strintzis, “3D Model Search and Retrieval Using the

Spherical Trace Transform”, EURASIP Journal on

Advances in Signal Processing, Vol. 2007, Article ID 23912,

pp. 207-225, 2007.

[2] R. Osada, T. Funkhouser, B. Chazelle and D. Dobkin,

“Shape distributions”, ACM Trans. on Graphics, Vol. 21,

No. 4, pp. 807-832, 2002.

[3] J.L. Shih, C. H. Lee, and J. T. Wang, “3D object retrieval

system based on grid D2”, Electronics Letters, Vol. 41, No.

4, pp. 23-24, 2005.

[4] D.V. Vranic, D. Saupe, and J. Richter, “Tools for 3D-

object retrieval: Karhunen-Loeve transform and spherical

harmonics”, IEEE 2001 Workshop Multimedia Signal

Processing, pp. 293–298,2001.

[5] T. Funkhouser, P. Min, M. Kazhdan, J. Chen, A.

Halderman, D. Dobkin, and D. Jacobs, “A Search Engine for

3D Models”, ACM Trans. on Graphics, vol. 22, no. 1, pp. 83-

105, 2003.

[6] M. Ankerst, G.K. ¨Uller, H.P. Kriegel, and T. Seidl, “3D

shape histograms for similarity search and classification in

spatial databases”, in Proc. SSD, 1999.

[7] M. Ankerst, G.K. ¨Uller, H.P. Kriegel, and T. Seidl,

“Nearest neighbor classification, 3D protein databases”, in

Proc. ISMB, T. 1999.

[8] S. Manjunath, P. Salembier, and T. Sikora, “Introduction

to MPEG-7 Multimedia Content Descriptor Interface”, John

Wiley & Sons Ltd., 2002.

[9] M. Reisert and H. Burkhardt, “Second order 3D shape

features: An exhaustive study”, Computers & Graphics, Vol.

30, No. 2, pp. 197-206, 2006.

[10] J.L. Shih and H.Y. Chen, “A new 3D model retrieval

approach based on the Shell Grid and Modified Elevation

Descriptor”, Pattern Recognition, Vol. 40, No. 1, pp. 283-

295, 2007.

[11] J.L. Shih and W.C. Wang, “A 3D Model Retrieval

Approach based on The Principal Plane Descriptor”,

Proceedings of The Second International Conference on

Innovative Computing, Information and Control (ICICIC),

pp. 59-62, 2007.

[12] C.T. Kuo and S.C. Cheng, “3D model retrieval using

principal plane analysis and dynamic programming”, Pattern

Recognition, 2007, vol. 40, no. 2, pp. 742-755.

[13] J. Ricard, D. Coeurjolly

“Generalizations of angular radial transform for 2D and 3D

shape retrieval”, Pattern Recognition Letters, Vol. 26, No.

14, pp. 2174-2186, 2005.

[14] Y.Wu and A. Zhang, “A feature re-weighting approach

for relevance feedback in image retrieval”, In IEEE

International Conference on Image Processing (ICIP’02),

Rochester, NewYork, pp. 581–584, September 2002.

[15] P. Shilane, P. Min, M. Kazhdan and T. Funkhouser,

“The Princeton shape benchmark”, Proceedings of the Shape

Modeling Applications, pp. 167-178, 2004.

and A. Baskurt,

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

Fig. 2 Query result of the motorcycle model by

using each feature descriptor, equal weight

combination approach, and our method. (a) Query

model (b) ED (Re=0.5) (c) PPD (Re=0.3333) (d) 3D-

ART (Re=0.5) (e) GD2 (Re=0.16667) (f) BD

(Re=0.1667) (g) A3 (Re=0.1667) (h) Equal weight

combination approach (Re=0.6667) (i) Our method

(Re=0.8333)

10291029