Cover PagePDF Available

New Graph Distance based on Stable Marriage formulation for Deformable 3D Objects Recognition

Cover Page

New Graph Distance based on Stable Marriage formulation for Deformable 3D Objects Recognition

Abstract and Figures

We propose a novel fast graph matching approach based on a new formulation of the stable marriage problem, to measure the distance between graphs. The proposed approach is optimal in terms of execution time, i.e. quadratic time complexity O(n 2). Our technique is based on the decomposition of graphs into a set of substructures which are subsequently matched with the stable marriage algorithm. In this paper, we address the problem of comparing deformable 3D objects represented by graphs, we use a triangle-stars decomposition for triangular tessellations (graphs of 3D shapes). The proposed approach is based on computing an approximation of Graph Edit Distance which is fault-tolerant to noise and distortion which makes our method especially relevant for deformable 3D shapes comparison. We analyze and determine its time complexity. The proposed method is evaluated against benchmark databases under different evaluation criteria. Our experimental results consistently demonstrate the effectiveness and the high performances of our approach.
Content may be subject to copyright.
New Graph Distance based on Stable Marriage
formulation for Deformable 3D Objects Recognition
Kamel Madi
Umanis, Research & Innovation, Levallois-Perret, 92300, France
kmadi@umanis.com
Eric Paquet
National Research Council Canada, Ottawa, Canada
Eric.Paquet@nrc-cnrc.gc.ca
Hamamache Kheddouci
Universit´
e de Lyon, CNRS, LIRIS, UMR5205, France
hamamache.kheddouci@univ-lyon1.fr
Abstract—We propose a novel fast graph matching approach
based on a new formulation of the stable marriage problem, to
measure the distance between graphs. The proposed approach is
optimal in terms of execution time, i.e. quadratic time complexity
O(n2). Our technique is based on the decomposition of graphs
into a set of substructures which are subsequently matched with
the stable marriage algorithm. In this paper, we address the
problem of comparing deformable 3D objects represented by
graphs, we use a triangle-stars decomposition for triangular
tessellations (graphs of 3D shapes). The proposed approach is
based on computing an approximation of Graph Edit Distance
which is fault-tolerant to noise and distortion which makes our
method especially relevant for deformable 3D shapes comparison.
We analyze and determine its time complexity. The proposed
method is evaluated against benchmark databases under dif-
ferent evaluation criteria. Our experimental results consistently
demonstrate the effectiveness and the high performances of our
approach.
Index Terms—Graph matching, Graph edit distance, Graph
decomposition, Stable Marriage, Pattern recognition, 3D object
recognition, Deformable object recognition.
I. INTRODUCTION
3D object recognition and classification are one of the
fundamental challenges in computer vision, which have been
studied during many decades [1]. In the last years, there has a
growing interest on the 3D objects analysis. The high advances
in different fields of technology (especially in the field of
3D) and the increased availability of 3D data, engender a
high growing need of automated approaches for 3D shapes
recognition and classification. Shapes represented by graphs,
may be compared using graph matching techniques. Graph
matching is the process of measuring a similarity (or a
dissimilarity) between two graphs by finding a correspondence
between vertices and edges of two graphs that satisfies a
set of constraints, ensuring that substructures in one graph
are mapped to similar substructures in the other. In order
to solve the graph matching problem, various methods have
been proposed [2], [3], [4]. Graph Edit Distance (GED) is one
of the most famous measures to determine such a distance
[5], [6], [7] and [8]. GED is defined as the minimum-cost
sequence of edit operations needed to transform one graph into
another. The tolerance to noise and distortion represent one
of the most important advantages of graph edit distance. Un-
fortunately, GED has a high computational complexity which
grows exponentially with the number of vertices [9]. In this
paper, we address the problem of comparing deformable 3D
objects represented by graphs. We propose a novel fast graph
matching approach based on a new formulation of the stable
marriage problem [10]. The proposed approach is optimal
in terms of execution time, i.e., quadratic time complexity
O(n2). This approach is based on graph decomposition into
a set of substructures called triangle-stars [11], [12] which
are subsequently matched with the help of the stable marriage
algorithm.
The remainder of the paper is organized as follows. In
Section II, we briefly review some related works. Our approach
is described in Section III. The time complexity is determined
in Section IV. In Section V, we present and discuss our exper-
imental results while comparing them with some benchmark
shape-matching algorithms. Finally, Section VI concludes the
paper.
II. RE LATE D WO RK S
The state of the art includes a wide range of 3D objects
recognition approaches, we present in this section a brief
review of 3D shapes recognition methods, with a particular
focusing on graph-based approaches. We can generally di-
vide existing methods, for 3D object comparison techniques,
on three main categories [13]: feature-based methods, graph
based methods and others. In the first category, feature based
methods, a set of features associated with the geometrical and
topological properties of objects are used for their comparison.
These features may be global, local or spatial maps. The entire
object is represented by a unique descriptor, in the case of
global features and spatial maps, while, in the case of local
features, a descriptor is either associated with the vertices, the
triangles or any substructure of interest. Various feature-based
approaches have been proposed in the state of the art [14],
[15], [16], [17]. In the second category, graph representations
and related approaches are considered. Graphs constitute a
powerful and flexible modeling tool allowing both the descrip-
tion of properties of a shape and the relationships between
a set of shapes. In a graph, the properties are associated
with the vertices while the relationships are represented by
the edges. Vertices, edges and their attributes are specified
according to the underlying application; for instance, the
vertices may represent points, regions of interest or any other
substructure obtained by applying a data reduction process
such as segmentation. Edges are associated with the vertice’s
connectivity, defining a topological relationship between them,
such as proximity, adjacency, etc. Depending to the kind of
graph considered, several graph-based techniques have been
proposed for 3D objects comparison [13]. For instance, some
approaches reduce shapes to skeletons through a thinning
process [18], [19]. The resulting skeletons are compared using
graph matching techniques. Other approaches rely on Reeb
graphs which are constructed from mapping functions defined
on the shape manifolds [20], [21]. Various segmentation-based
techniques have been proposed in the literature, in which the
shapes are segmented into a finite set of components from
which a graph is constructed: the segments corresponding to
the vertices while their topological relationships are repre-
sented by the edges. These graphs may be compared with
graph matching techniques as well [22], [23]. In the last
category, others approaches have been proposed [13]: view
based similarity, volumetric error based similarity, weighted
point set based similarity, etc. In the case of view based sim-
ilarity, two shapes with similar projections, from all possible
viewpoints, are considered similar [24], [25]. For volumetric
error based similarity, the distance between two objects is
measured by estimating the volumetric error [26], [27]. In the
case of weighted point set based similarity, a distance between
two objects is measured using a set of descriptors which
consist of weighted 3D points. The shapes are decomposed
into substructures represented by weighted points which are
subsequently matched. The matching process depends on the
weight considered. The volume of a component is considered
as a weight in [28] and a measure for the curvature represent
a weight in [29]. The authors of [30] consider a hierarchy of
weighted point sets, representing spherical shape approxima-
tions. For a more exhaustive review, we refer the reader to [13]
and [31] for 3D object recognition methods and to [9], [32],
[33], [34] and [35] for pattern recognition and graph matching
techniques.
III. ALGORITHM OVERVIEW
We propose a novel fast graph matching approach based
on a new formulation of the stable marriage problem. The
proposed approach is based on graph decomposition into a set
of substructures which are matched with the stable marriage
algorithm [10]. We address the problem of 3D deformable
object recognition using triangle-stars decomposition [11],
[12]. In this section, we firstly introduce the stable marriage
problem [10], then we present the triangle-stars decomposition
as well the distance between triangle-stars [11], [12]. Finally,
we describe the proposed approach.
A. Stable marriage problem
The stable marriage problem is the problem of finding a
stable matching between two sets of elements of equal size,
based on an order of preferences; the latter being associated
to each pair of elements. Here, stability means that every
matched couple of elements prefers to stay together rather
than to be ”married” to another one [10]. Originally, the stable
marriage problem was introduced by Gale and Shapley in
1962 in order to find a matching between men and women,
based on preference lists in which each person (man and
woman) presents his or her preferences regarding the other
candidates. The principle is summarized in the following
algorithm (Algo. 1). The problem has then adapted in several
research areas, such as in mathematics, economics, game
theory, computer science, etc. We refer the readers to [36]
for further details.
Algorithm 1 Stable marriage algorithm
1: Begin
2: Initialize all men mMand women wWto free
3: while (mMfree) do
4: w=W[0] first woman on the list of mto whom m
has not yet proposed;
5: if (wis free)
6: wand mare engaged ;
7: else
8: if (wprefer mto her engaged m0)
9: wand mare engaged and m0is free;
10: else
11: wrejects mand mis free ;
12: end if
13: end if
14: end while
15: return the stable marriage which is the ncouples ;
16: End
B. Triangle-stars decomposition
The triangle-stars decomposition [11], [12] is a decomposi-
tion method of a triangular tessellations (of a 3d shape) into
a set of connected components called triangle-stars. This de-
composition aimed to reduce the number of components while
covering larger neighborhoods depending on the considered
neighborhood order Nk. From a triangle-star representation, a
description is defined which is invariant or at least oblivious
under most common deformations. Prior to the decomposition,
a strict total order on the triangles is established, in order
to reduce the number of triangle-stars and guaranteeing the
uniqueness of the decomposition. The authors of [11], [12]
introduce the triangle-stars using the following definitions:
a) Definition 1 (Nk-neighborhood of a triangle):two
triangles are neighbors (k= 1), if they share at least a common
vertex. Two triangles t0and tkare Nk-neighbors, if between
t0and tkthere is a chain of at most (k1) distinct triangles,
which are pairwise consecutive neighbors. Formally, t0and tk
are Nk-neighbors ⇔ ∃ ti=1..k1where :
i1...(k1), tiand ti+1 are neighbors [11], [12].
b) Definition 2 (Nk-triangle-star):A triangle-star ts
(k= 1) is a labeled subgraph, defined by a triangle and the
set formed by its neighbors. Formally, a triangle-star ts is a
three tuple ts = (tr, T 0, θ), where: tris the root triangle, T0
is the set of adjacent triangles and θ:TLTis the triangle
labelling function while LTis a set of labels. A Nk-triangle-
star Nk-ts is triangle-star defined by a triangle and the set of
its Nk-neighbors [11], [12].
c) Triangle-star vector representation:Each triangle-
star ts is characterized by a set of descriptors represented
by a vector consisting of the global area AG and the
global perimeter P G of ts, the area Aand the perime-
ter Pof each triangles belonging to ts, the weights
associated with their edges Was well as the degrees
deg associated with their vertices. This vector is given
by: {AG(ts), P G(ts),{A(ti), P (ti), W (ti, j=1...3),
deg(ti, j=1...3)}i=kT(ts)k
i=1 }. The variables are described in Ta-
ble I. The triangles belonging to ts are ranked according to
their areas in descending order. The weights and the degrees
are ranked by descending order as well. All T S vectors have
the same size: size = 2 + (8 Γ). If a ts has a number of
triangles inferior to Γ, the unassigned entries are completed
with zeros [11], [12].
d) Triangle-stars decomposition process:A descend-
ing strict total order, based on the number of neighbors
kneighborskand the vertex’s coordinates, is applied on the
set of triangles prior to their decomposition into triangle-stars;
in order to ensure the uniqueness of the decomposition and
to further reduce the number of triangle-stars. The decom-
position process is performed as follows: according to the
order established for the triangles (strict descending order),
the first Nk-triangle-star is constructed from the first triangle
and its corresponding Nk-neighbors; that is, the triangles not
belonging to any other Nk-triangle-stars. Then, the set of
triangles and the set of the resulting Nk-triangle-stars are
updated. The process is repeated until each triangle of the
graph is associated with a Nk-triangle-star [11], [12].
C. Distance between triangle-stars
In this section, we present the graph distance between two
triangle-stars [11], [12]. The similarity measure dbetween two
triangle-stars tsiand tsjis defined as:
d(tsi, tsj) = 1 Pk=6
k=1 simk(tsi, tsj)
Pk=6
k=1 αk
(1)
The similarity measure drequires six auxiliary functions
simk:
dsimk(tsi, tsj) = αk
|AG(tsi)AG(tsj)|
AGMAX if k= 1
|P G(tsi)P G(tsj)|
P GMAX if k= 2
Pl
l=1 |A(T(tsi)l)A(T(tsj)l)|
AMAX Γif k= 3
Pl
l=1 |P(ti,l)P(tj,l )|
PMAX Γif k= 4
Pl
l=1 Pk=3
k=1 |Wi,l,k Wj,l,k |
3WMAX Γif k= 5
Pl
l=1 Pk=3
k=1 |Degi,l,k Degj,l,k |
3DegM AX Γif k= 6
(2)
The associated symbols are defined in Table I.
Symbol Description
ti,l Triangle tlbelonging to the triangle-star tsi:tltsi
Wi,l,k Weight (Euclidean distance) of edge ekbelonging to triangle tltsi
Degi,l,k Degree of vertex vkbelonging to triangle tltsi
ΓMax number of triangles in the triangle-stars
αk=1...6Parameters associated with the descriptors αkNand
Pk=6
k=1 αk>0
A(ti)Area of triangle i.
P(ti)Perimeter of triangle i.
AG(tsi)Area of triangle-star i.AG(tsi) = Pj=kT(tsj)k
j=1 A(tj)
P G(tsi)Perimeter of triangle-star i.P G(tsi) = Pj=kT(tsj)k
j=1 P(tj)
TABLE I
SYM BOL S AS SOC IATE D WIT H TH E SIM IL ARI TY M EAS URE A ND T HEI R
DESCRIPTION.
D. Graph Distance based on the Stable Marriage Algorithm
(GDStM)
The first step of our approach is to decompose two graphs
G1and G2into two sets of substructures S1and S2. Since
3D deformable object recognition is addressed in this paper, a
triangle-stars decomposition is employed for the graphs (see
Section III-B). The next step consists of associating to each
substructure s1,i S1, a vector of preferences regarding the
other substructures s2,j S2. The vectors associated to s1,i
S1(respectively s2,j S2) contain the set of substructures
S2(respectively S1) ordered following the preferences with
a descending order. The preference P(s1,i, s2,j )between s1,i
S1and s2,j S2is measured using the distance defined
in Eq. 1. All the vectors have the same size t, which corre-
sponds to the maximum size, i.e.,t= max(||S1||,||S2||).
If ||S1|| 6=||S2||, the smaller vector is completed by
(max(||S1||,||S2||)min(||S1||,||S2||)) empty substructures
(ε), with their corresponding preferences P(sk,i, ε). From
these vectors, we construct a square preference matrix Dt,t
in which each entry Di,j contains a substructure (or an
empty substructure ε) as well as the corresponding preferences
Pi,j . In the third step, the algorithm of stable marriage is
employed in order to find the best match between the different
substructures S1and S2based on their preferences vectors,
in quadratic time O(n2). Finally, the normalized distance is
estimated based on the preference values, using the following
formula:
GDStM(S1, S2)=1Pi,j=t
i,j=1 P(s1,i , s2,j )
max(||S1||,||S2||)(3)
Algorithm 2 Graph Distance based on Stable Marriage Algo-
rithm (GDStM)
1: Inputs: Two graphs g1and g2.
2: Outputs: The distance d(g1, g2)between g1and g2.
3: Begin
4: S1=triangle-stars of g1;
5: S2=triangle-stars of g2;
6: For each siS1and sjS2do
7: sia vector of preferences regarding sjS2
with descending order using (Eq. 1);
8: sja vector of preferences regarding siS1
with descending order using (Eq. 1);
9: end For each
10: Use stable marriage algorithm (Algo. 1);
11: return GDStM (S1, S2)and (si, sj)(S1×S2);
12: End
The algorithm of stable marriage is asymmetrical and the
considered version favors men. In order to ensure the symme-
try in our approach (GDStM(S1, S2) = GDStM(S2, S1)),
we consider the set of substructures (S1or S2) with higher
cardinality as the first set (men) and the other set as second
(women). In the case of equality (||S2|| =||S1||), we consider
the average of the two distances. The symmetrized distance
between S1and S2is computed as follows:
GDStM (S1, S2) =
GDStM (S1,S2)+GDStM (S2,S1)
2if ||S2|| =||S1||
GDStM (S1, S2)if ||S1|| >||S2||
GDStM (S2, S1)if ||S2|| >||S1||
(4)
Our approach is summarized in Algorithm 2. An illustration
of our approach is proposed in Example 2.
a) Example 2.:Let S1and S2be two set of substruc-
tures (triangle-Stars) associated with two graphs G1and G2.
kS1k=kS2k= 4. Let Dbe the matrix of similarities
between S1and S2calculated using (Eq. 1).
s2,0s2,1s2,2s2,3
s1,00.11 0.90 0.25 0.21
s1,10.10 0.15 0.65 0.89
s1,20.67 0.03 0.51 0.17
s1,30.66 0.88 0.33 0.99
Firstly, we associate to each substructure s1,i=0..3S1
and s2,j=0..3S2, a vector of preferences with respect to
the other set of substructures s2,j=0..3S2and s1,i=0..3
S1respectively, with a descending order.
Figure 1(a) shows the four vectors representing the preferences
of s1,i=0..3S1with respect to s2,j=0..3S2. Figure 1(b)
shows the four vectors representing the preferences of
s2,j=0..3S2with respect to s1,i=0..3S1. Secondly, we
use the stable marriage algorithm in order to find the best
match between the different substructures S1and S2based
on their preferences vectors. We obtain the following couples:
{(s1,0, s2,1); (s1,1, s2,2); (s1,2, s2,0); (s1,3, s2,3)}. Finally, we
calculate the distance corresponding to the couples selected.
The score obtained is 3.21 and the corresponding normalized
distance is GDStM(S1, S2)=13.21
4= 0.1975.
S2,1
S2,2
S2,3
S2,0
0,90
0,25
0,21
0,11
S2,3
S2,2
S2,1
S2,0
0,89
0,65
0,15
0,10
S2,0
S2,2
S2,3
S2,1
0,67
0,51
0,17
0,03
S2,3
S2,1
S2,0
S2,2
0,99
0,88
0,66
0,33
S1,0
S1,1
S1,2
S1,3
S1,2
S1,3
S1,0
S1,1
0,67
0,66
0,11
0,10
S1,0
S1,3
S1,1
S1,2
0,90
0,88
0,15
0,03
S1,1
S1,2
S1,3
S1,0
0,65
0,51
0,33
0,25
S1,3
S1,1
S1,0
S1,2
0,99
0,89
0,21
0,17
S2,0
S2,1
S2,2
S2,3
(a) S1preferences (b) S2preferences
Fig. 1. Vectors of preferences of s1,i=0..3S1regarding to s2,j=0..3
S2and vectors of preferences of s2,j=0..3S2regarding to s1,i=0..3
S1, in descending order.
IV. COM PL EX IT Y OF T HE P ROPOSED APPROACH
In terms of time complexity, the most demanding part of
the algorithm, is the one solving the assignment problem. The
Stable Marriage algorithm [10] is employed in order to find
the best assignment in O(n2)time, where nis the maximum
number of components in the two graphs. For triangle-stars
decomposition (3D), we consider n= max(kV1k,kV2k)and
n0= max(kT S1k,kT S2k), where Viis the set of vertices
and T Siis the set of triangle-stars in gtr i. Any triangle-
star has at least one triangle. Consequently, in the worst case,
we have n0=n
3, which means that the complexity is in the
order of O(0.037 n2). However, the number of triangle-stars
depends on the structure of the underlying graph as well as
the neighborhood order Nkconsidered. Indeed, the number
of triangle-stars decreases when the neighborhood order Nk
increases.
Table II summarizes the relation between the computational
complexity and the neighborhood order Nk=1...6, for the
TOSCA database [37], [38]. The complexity is in the order
of O(α[n
log(n)]2), where α[1.80 107,0.74] for
Nk=1...6. Table II shows that the complexity decreases when
the neighborhood order Nkincreases. The neighborhood order
Nk=1 has the highest complexity while the neighborhood order
Nk=6 has the lowest.
NkComplexity
N1O(0.74 [ n
log(n)]2)
N2O(0.0001 [ n
log(n)]2)
N3O(5.6115 106[n
log(n)]2)
N4O(9.84 107[n
log(n)]2)
N5O(3.59 107[n
log(n)]2)
N6O(1.80 107[n
log(n)]2)
TABLE II
TIME COMPLEXITY ACCORDING TO THE NEIGHBORHOOD ORDER Nk,FOR
TH E TOSCA DATABASE .
We propose a novel fast graph distance, which is an approx-
imation of Graph Edit Distance based on a new formulation
Class Pose 1 Pose 2 Pose 3
Centaur
Gorilla
TABLE III
SMA LL SA MP LE FRO M TH E TOSCA DATABASE .
of the stable marriage problem using triangle-star (TS) de-
composition. The considered dissimilarity measure d(tsi, tsj)
(Eq. 1) is equivalent to a substitution edit operation of two
triangle-stars tsiand tsj. The insertion and deletion edit
operations are equivalent to replace an empty triangle-star ε
(tsid(ε, tsi)or ε d(tsi, ε)). Consequently, the dimensions of
the cost matrix Dare reduced to n×n, where nis given
by n= max(kT S1k,kT S2k)(kT SkkVk). Consequently,
our approach has a better time complexity than the state of
the art. Indeed, in [39] and [40], the complexity achieved is
O((n+m)3), where nand mare the number of vertices in
the two graphs. In [41] and [42], the obtained time complexity
is O((max(n, m))3). However, the Jonker-Volgenant linear
solver [43] presents some convergence problems for some
kind of cost matrices. The proposed algorithm in [44] has
a quadratic time complexity O(nm). However, the obtained
correspondence between the vertices of the two graphs is not
bijective. In [45], the complexity achieved is O((min(n, m))2
max(n, m)). In [11], [12], the computational complexity
achieved, in general, is O(0.037 n3)and for the TOSCA
database, is of the order of O(α[n
log(n)]3), where α
[1.80 107,0.74] for Nk=1...6.
V. EXPERIMENTS
In order to evaluate our approach, we undertook a set of
experiments in which our approach is compared with some
benchmark algorithms for shape-matching for the TOSCA
database [37], [38].
A. Description of the TOSCA database
The TOSCA database [37], [38] consists of 148 three-
dimensional shapes. Each shape is represented by a triangular
tessellation. The database is organized on 12 classes. Each
class is composed of the same shape submitted to isometric
or quasi-isometric deformations. The database is unbalanced
(from 1to 24 objects per class). On average, the number of
vertices is 3154 while the number of triangles is 6220. Table III
shows a small sample of 3D shapes belonging to the TOSCA
database.
B. Benchmark for shape-matching algorithms
For the evaluation of our approach, we performed a set
of experiments and we compare the proposed method with
some state-of-the-art shape-matching algorithms associated
with TOSCA database [37], [38]. Consequently, our method
GDStM has been compared with the following algorithms:
TSM [11], [12]: An approach based on an approximation
of Graph Edit Distance using triangle-stars decomposition,
then followed by a matching of these substructures using the
Hungarian algorithm [46].
CAM [47]: An approach in which surfaces are represented by
3D-curves extracted around feature points.
GeodesicD2 [48]: A global description which consists of the
distribution of the geodesic distances associated with a given
3D shape.
DSR [49]: The Hybrid Feature Vector is a combination of two
view-based descriptors: a depth buffer for the silhouette and a
radialized extent function descriptor.
RSH [50]: The Ray-based approach with Spherical Harmonic
Representation is a method which aligns the models into a
canonical position, extracts the maximal extents and applies a
spherical harmonic decomposition.
TD [51]: The Temperature Distribution descriptor is a shape
descriptor based on the heat kernel. The L2 norm is used in
order to evaluate the distance between the descriptors.
Shape-DNA [52]: The Shape-DNA is a numerical fingerprint
obtained by evaluating the eigenvalues of the Laplace-Beltrami
operator associated with the manifold. The matching in be-
tween two objects is obtained by comparing their respective
eigenvalues.
SRCP-TD [53]: The SRCP-TD is a method based on a sparse
representation of a scale-invariant heat kernel. The authors
use Laplace-Beltrami eigenfunctions in order to detect critical
points on the manifold. The descriptor is constructed from the
values of heat kernel at these points. A sparse representation
is employed in order to reduce the dimensionality of the
descriptor.
C. Experimental results
In this section, we compare our approach with the tech-
niques previously described. We consider the following met-
rics: Precision and Recall, Accuracy, True Positive Rate TPR,
True Negative Rate TNR, F-measure and Runtime [54]. Our
approach GDStM is parameterized through a set of variables
αkwhich determine the weights attributed to the various at-
tributes. The default value for these parameters is: αk= 1,k.
In addition, a threshold is determined in order to perform
classification. The weights and the threshold may be specified
by inspection or by using machine learning techniques.
The distance between each pair of triangular tessellations
was evaluated using the similarity measure GDStM , for
various neighborhood orders Nk=1...6. For each neighborhood
order, a n×nconfusion matrix was calculated. Two triangular
tessellations were considered similar if their GDStM distance
was below a certain threshold. The results differ according
to the values of the parameters αk, the level of the threshold
and the neighborhood order Nk. Table IV shows the Accuracy,
TPR and T N R obtained by our approach GDS tM and
TSM for the TOSCA database, for six neighborhood order
Nk=1...6. For our approach GDStM , a direct neighborhood
Nk=1 with a threshold = 0.066, has the highest accuracy
(78%), followed by Nk=4 (68%), Nk=3,Nk=5 (66%), Nk=6
and Nk=2 (62%). Table IV shows that GDStM and TSM have
approximatively the same performances with TSM performing
slightly better mainly for Nk=1 where TSM has an accuracy
of 83% as opposed to 78% for GDStM.
Methods NkThreshold Accuracy T P R T NR
GDStM 1 0.066 78.44 % 78.80 % 78.40 %
GDStM 2 0.135 62.09 % 62.74 % 62.02 %
GDStM 3 0.192 66.50 % 66.06 % 66.55 %
GDStM 4 0.231 68.28 % 69.80 % 68.11 %
GDStM 5 0.265 65.27 % 64.40 % 65.38 %
GDStM 6 0.273 62.44 % 61.43 % 62.57 %
TSM 1 0.06 83.24 % 82.29 % 83.35 %
TSM 2 0.13 60.56 % 61.34 % 60.47 %
TSM 3 0.19 65.78 % 66.93 % 65.65 %
TSM 4 0.22 68.65 % 67.89 % 68.74 %
TSM 5 0.264 65.01 % 64.31 % 65.09 %
TSM 6 0.273 62.11 % 61.52 % 62.18 %
TABLE IV
GDStM A ND TSM AC CUR ACY,T P R A ND T N R FOR T HE TOSCA
DATABAS E FO R VARIO US NE IG HBO RH OOD O RD ERS .
Some results are displayed in Figure 2. The first column
represents the query objects, while the other columns are asso-
ciated with the first four objects retrieved. All the objects were
properly classified. Table V shows the resulting correspon-
dences in terms of triangle-stars, using our approach GDStM
with different neighborhood orders, between two poses of four
objects (dog, horse, lioness and michael) belonging to the
TOSCA Database.
dog2 ×dog10 -N8horse9 ×horse0 -N8
michael19 ×michael15 -N6lioness13 ×lioness15 -N8
TABLE V
THE CORRESPONDENCE,US ING O UR AP PRO ACH GDS tM WITH
DIFFERENT NEIGHBORHOOD ORDERS,BETW EE N TWO P OS ES OF F OU R
OBJECTS BELONGING TO THE TOS CA DATABA SE.
Figure 3 shows the precision-recall curves for the six queries
appearing in Figure 2 when a neighborhood order of Nk=1
is employed. A threshold of 0.066 was selected for these
experiments. This implies in turn that only the objects for
which dissimilarity threshold were considered when
evaluating the recall and precision curves. Indeed, when the
dissimilarity is greater than the threshold, the objects are
automatically classified as dissimilar by our algorithm. As
Query
Result 1
Result 3
Result 4
Fig. 2. Some retrieval results for the TOSCA database with a neighborhood
order of Nk=1.
showed in Figure 3, we have obtained excellent precision-
recall curves which are consistent with the results shown in
Figure 2. For instance, centaur0,gorilla0,horse0,lioness0
and seahorse0have a precision and recall of 100%.
Fig. 3. Precision-recall curves for six distinct 3D-objects of the TOSCA
database with a neighborhood order of Nk=1.
We also compared our method GDStM with four
other SHREC benchmark methods namely TSM, CAM,
GeodesicD2, DSR and RSH. Figure 4 shows the precision and
recall curves for our approach with a neighborhood order of
Nk=1, as well as for the methods previously mentioned. The
precision of our approach is essentially always superior than
the one associated with the other methods, except TSM: we
conclude that our algorithm outperforms the others except for
TSM. Nonetheless, our method GDStM is faster as shown in
Figure 5 (logarithmic scale). Figure 5 shows the total runtime
of our approach GDStM and the TSM method according to
the neighborhood order Nkconsidered. Figure 5 further shows
that better performances, in terms of the runtime, are obtained
when using a larger neighborhood. The runtime performances,
as reported in Figure 5, confirm the theoretical quadratic time
complexity O(n2)for our approach GDStM and cubic O(n3)
for TSM; nbeing the number of nodes associated with the
graphs. A comparison in terms of F-measure was performed in
Fig. 4. Comparison of our approach GDStM with the TSM, CAM,
GeodesicD2, DSR and RSH methods, in terms of precision and recall curves,
for the TOSCA database.
Fig. 5. Comparison of our approach GDStM with the TSM method, in
terms of total runtime, for the TOSCA database.
between our method and four SHREC benchmark algorithms
namely TSM, TD, Shape-DNA and SRCP-TD. Our results
are reported in Table VI. Table VI clearly indicates that our
GDStM method has excellent results in terms of F-measure.
Table VI shows that our method GDStM outperforms the
other methods except for TSM. Nonetheless, our method
GDStM is faster as shown in Figure 5.
Methods GDStM TSM TD Shape-DNA SRCP-TD
F-measure 0.74 0.90 0.67 0.45 0.44
TABLE VI
F-MEASURE RESULTS WITH OUR GDStM METHOD FOR Nk=1
NEIGHBORHOOD AS COMPARED WITH FOUR OTHER BENCHMARK
METHODS FOR THE TOSCA DATABAS E.
D. Discussion
Our experimentation against the TOSCA, shows the perfor-
mances and the robustness of our approach GDStM . Indeed,
we obtained excellent results in terms of Accuracy, TPR
and T N R for the TOSCA database. Our predicted quadratic
time complexity has been systematically confirmed by our
experiments. GDStM with quadratic time complexity O(n2)
is faster than TSM by one order of magnitude. Our method
GDStM outperforms CAM, GeodesicD2, DSR and RSH
while retaining the same level of performance as TSM in terms
of precision-recall and F-measure.
VI. CONCLUSIONS
In this paper, we presented a novel graph matching approach
based on a new formulation of the stable marriage problem,
to measure the distance between graphs. The proposed ap-
proach is optimal in terms of execution time, i.e. quadratic
time complexity O(n2). Our approach is based on graph
decomposition into a set of substructures which are matched
with the stable marriage algorithm. A triangle-stars decompo-
sition is employed for triangular tessellations (graphs of 3D
shapes). The present approach is based on an approximation
of the Graph Edit Distance which is fault-tolerant to noise
and distortion: making our technique particularly suitable for
the comparison of deformable 3D objects. Our experimental
results with deformable shapes (TOSCA database) and the
time complexity analysis confirm the performances and the
accuracy of our algorithm.
REFERENCES
[1] S. Ullman, High-Level Vision: Object Recognition and Visual Cognition.
illustrated edition ed. The MIT Press, July 1996.
[2] H. Bunke, A. M ¨
unger, and X. Jiang, “Combinatorial search versus
genetic algorithms: A case study based on the generalized median graph
problem,” PR Letters, vol. 20, no. 11-13, pp. 1271–1277, 1999.
[3] R. Myers, R. C. Wison, and E. R. Hancock, “Bayesian graph edit
distance,” IEEE TPAMI, vol. 22, no. 6, pp. 628–635, 2000.
[4] M. Gori, M. Maggini, and L. Sarti, “Exact and approximate graph
matching using random walks,” IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 27, no. 7, pp. 1100–1111, Jul. 2005.
[5] A. Sanfeliu and K.-S. Fu, “A distance measure between attributed
relational graphs for pattern recognition,” Systems, Man and Cybernetics,
IEEE Transactions on, vol. SMC-13, no. 3, pp. 353–362, May 1983.
[6] H. Bunke and K. Shearer, “A graph distance metric based on the maximal
common subgraph,” PR Letters, vol. 19, no. 3-4, pp. 255–259, 1998.
[7] A. Papadopoulos and Y. Manolopoulos, “Structure-based similarity
search with graph histograms,” in 10th International Workshop on
Database & Expert Systems Applications, Florence, Italy, September
1-3, 1999, Proceedings., 1999, pp. 174–178.
[8] S. Sorlin, C. Solnon, and J. Jolion, “A generic graph distance measure
based on multivalent matchings,” in Applied Graph Theory in Computer
Vision and Pattern Recognition, 2007, pp. 151–181.
[9] D. Conte, P. Foggia, C. Sansone, and M. Vento, “Thirty years of graph
matching in pattern recognition,” IJPRAI, vol. 18, no. 3, pp. 265–298,
2004.
[10] D. Gusfield and R. W. Irving, The stable marriage problem: structure
and algorithms. MIT press, 1989.
[11] K. Madi, E. Paquet, and H. Kheddouci, “New graph distance for de-
formable 3d objects recognition based on triangle-stars decomposition,”
Pattern Recognition, vol. 90, pp. 297–307, 2019.
[12] K. Madi, E. Paquet, H. Seba, and H. Kheddouci, “Graph edit distance
based on triangle-stars decomposition for deformable 3d objects recogni-
tion,” in 2015 International Conference on 3D Vision, 3DV 2015, Lyon,
France, October 19-22, 2015, 2015, pp. 55–63.
[13] J. W. H. Tangelder and R. C. Veltkamp, “A survey of content based
3d shape retrieval methods,Multimedia Tools Appl., vol. 39, no. 3, pp.
441–471, 2008.
[14] E. Paquet, M. Rioux, A. Murching, T. Naveen, and A. Tabatabai,
“Description of shape information for 2-d and 3-d objects,” Signal
processing: Image communication, vol. 16, no. 1, pp. 103–122, 2000.
[15] M. K¨
ortgen, G.-J. Park, M. Novotni, and R. Klein, “3d shape matching
with 3d shape contexts,” in The 7th central European seminar on
computer graphics, Budmerice, Slovakia., vol. 3. Budmerice, 2003,
pp. 5–17.
[16] B. Drost, M. Ulrich, N. Navab, and S. Ilic, “Model globally, match
locally: Efficient and robust 3d object recognition,” in The Twenty-Third
IEEE Conference on Computer Vision and Pattern Recognition, CVPR
2010, San Francisco, CA, USA, 13-18 June 2010, 2010, pp. 998–1005.
[17] J. Xie, Y. Fang, F. Zhu, and E. Wong, “Deepshape: Deep learned shape
descriptor for 3d shape matching and retrieval,” in 2015 CVPR, June
2015, pp. 1275–1283.
[18] H. Sundar, D. Silver, N. Gagvani, and S. J. Dickinson, “Skeleton based
shape matching and retrieval,” in 2003 International Conference on
Shape Modeling and Applications (SMI 2003), 12-16 May 2003, Seoul,
Korea, 2003, pp. 130–142, 290.
[19] “Skeleton graph matching vs. maximum weight cliques aorta registration
techniques,” Computerized Medical Imaging and Graphics, vol. 46, Part
2, pp. 142 – 152, 2015, information Technologies in Biomedicine.
[20] M. Hilaga, Y. Shinagawa, T. Komura, and T. L. Kunii, “Topology
matching for fully automatic similarity estimation of 3d shapes,” in
SIGGRAPH 2001, Los Angeles, California, USA, August 12-17, 2001,
2001, pp. 203–212.
[21] V. Barra and S. Biasotti, “3d shape retrieval using kernels on extended
reeb graphs,” Pattern Recognition, vol. 46, no. 11, pp. 2985–2999, 2013.
[22] H. Laga, M. Mortara, and M. Spagnuolo, “Geometry and context for
semantic correspondences and functionality recognition in man-made
3d shapes,” ACM TOG, vol. 32, no. 5, p. 150, 2013.
[23] Y. Kleiman, O. van Kaick, O. Sorkine-Hornung, and D. Cohen-Or,
“SHED: shape edit distance for fine-grained shape similarity,ACM
Trans. Graph., vol. 34, no. 6, p. 235, 2015.
[24] Y. Gao and Q. Dai, “View-based 3d object retrieval: Challenges and
approaches,” IEEE MultiMedia, vol. 21, no. 3, pp. 52–57, 2014.
[25] S. Zhao, H. Yao, Y. Zhang, Y. Wang, and S. Liu, “View-based 3d object
retrieval via multi-modal graph learning,Signal Processing, vol. 112,
pp. 110–118, 2015.
[26] M. Novotni and R. Klein, “A geometric approach to 3d object compar-
ison,” in Proceedings International Conference on Shape Modeling and
Applications, 2001, pp. 167–175.
[27] H. S´
anchez-Cruz and E. Bribiesca, “A method of optimum transforma-
tion of 3d objects used as a measure of shape dissimilarity,Image and
Vision Computing, vol. 21, no. 12, pp. 1027–1036, 2003.
[28] T. K. Dey, J. Giesen, and S. Goswami, “Shape segmentation and
matching with flow discretization,” in Algorithms and Data Structures,
8th International Workshop, 2003, Ottawa, Ontario, Canada, July 30 -
August 1, 2003, Proceedings, 2003, pp. 25–36.
[29] J. W. Tangelder and R. C. Veltkamp, “Polyhedral model retrieval using
weighted point sets,” International journal of image and graphics, vol. 3,
no. 01, pp. 209–229, 2003.
[30] A. Shamir, A. Sharf, and D. Cohen-Or, “Enhanced hierarchical shape
matching for shape transformation,” International Journal of Shape
Modeling, vol. 9, no. 2, pp. 203–222, 2003.
[31] O. van Kaick, H. Zhang, G. Hamarneh, and D. Cohen-Or, “A survey
on shape correspondence,” Comput. Graph. Forum, vol. 30, no. 6, pp.
1681–1707, 2011.
[32] H. Bunke and K. Riesen, “Recent advances in graph-based pattern recog-
nition with applications in document analysis,” Pattern Recognition,
vol. 44, no. 5, pp. 1057–1067, 2011.
[33] P. Foggia, G. Percannella, and M. Vento, “Graph matching and learning
in pattern recognition in the last 10 years,” IJPRAI, vol. 28, no. 1, 2014.
[34] M. Vento, “A long trip in the charming world of graphs for pattern
recognition,” Pattern Recognition, vol. 48, no. 2, pp. 291–301, 2015.
[35] J. Yan, X.-C. Yin, W. Lin, C. Deng, H. Zha, and X. Yang, “A short
survey of recent advances in graph matching,” in Proceedings of the
2016 ACM on International Conference on Multimedia Retrieval, ser.
ICMR ’16, 2016, pp. 167–174.
[36] K. Iwama and S. Miyazaki, “A survey of the stable marriage problem
and its variants,” in Informatics Education and Research for Knowledge-
Circulating Society, 2008. ICKS 2008. International Conference on.
IEEE, 2008, pp. 131–136.
[37] A. M. Bronstein, M. M. Bronstein, and R. Kimmel, “Efficient computa-
tion of isometry-invariant distances between surfaces,SIAM J. Scientific
Computing, vol. 28, no. 5, pp. 1812–1836, 2006.
[38] A. M. Bronstein, M. M. Bronstein, and Kimmel, “Calculus of nonrigid
surfaces for geometry and texture manipulation,IEEE Trans. Vis.
Comput. Graph., vol. 13, no. 5, pp. 902–913, 2007.
[39] K. Riesen and H. Bunke, “Approximate graph edit distance computation
by means of bipartite graph matching,” Image Vision Comput., vol. 27,
no. 7, pp. 950–959, 2009.
[40] Z. Zeng, A. K. H. Tung, J. Wang, J. Feng, and L. Zhou, “Comparing
stars: On approximating graph edit distance,” PVLDB, vol. 2, no. 1, pp.
25–36, 2009.
[41] F. Serratosa, “Fast computation of bipartite graph matching,Pattern
Recognition Letters, vol. 45, pp. 244–250, 2014.
[42] F.Serratosa, “Speeding up fast bipartite graph matching through a new
cost matrix,” IJPRAI, vol. 29, no. 2, 2015.
[43] R. Jonker and A. Volgenant, “A shortest augmenting path algorithm for
dense and sparse linear assignment problems,” Computing, vol. 38, no. 4,
pp. 325–340, 1987.
[44] A. Fischer, C. Y. Suen, V. Frinken, K. Riesen, and H. Bunke, “Approx-
imation of graph edit distance based on hausdorff matching,” Pattern
Recognition, vol. 48, no. 2, pp. 331–343, 2015.
[45] S. Bougleux, B. Gauz‘ere, D. B. Blumenthal, and L. Brun, “Fast linear
sum assignment with error-correction and no cost constraints,” Pattern
Recognition Letters, 2018.
[46] H. W. Kuhn, “The hungarian method for the assignment problem,” Naval
Research Logistics Quarterly, vol. 2, no. 1-2.
[47] H. Tabia, M. Daoudi, J. Vandeborre, and O. Colot, “A new 3d-matching
method of nonrigid and partially similar models using curve analysis,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 4, pp. 852–858,
2011.
[48] R. Osada, T. A. Funkhouser, B. Chazelle, and D. P. Dobkin, “Shape
distributions,” ACM Trans. Graph., vol. 21, no. 4, pp. 807–832, 2002.
[49] D. Vranic, “3d model retrieval,” Ph.D. dissertation, University of of
Leipzig, may 2004.
[50] D. Saupe and D. V. Vranic, “3d model retrieval with spherical har-
monics and moments,” in Pattern Recognition, 23rd DAGM-Symposium,
Munich, Germany, September 12-14, 2001, Proceedings, 2001, pp. 392–
397.
[51] Y. Fang, M. Sun, and K. Ramani, “Temperature distribution descriptor
for robust 3d shape retrieval,” in CVPR Workshops 2011, Colorado
Springs, CO, USA, 20-25 June, 2011, 2011, pp. 9–16.
[52] M. Reuter, F. Wolter, and N. Peinecke, “Laplace-beltrami spectra as
’shape-dna’ of surfaces and solids,Computer-Aided Design, vol. 38,
no. 4, pp. 342–366, 2006.
[53] M. Abdelrahman, M. T. El-Melegy, and A. A. Farag, “Heat kernels for
non-rigid shape retrieval: Sparse representation and efficient classifica-
tion,” in Ninth Conference on Computer and Robot Vision, CRV 2012,
Toronto, Ontario, Canada, May 28-30, 2012, 2012, pp. 153–160.
[54] D. M. W. Powers, “Evaluation: from precision, recall and f-factor to roc,
informedness, markedness and correlation,” School of Informatics and
Engineering Technical Reports, no. SIE-07-001, 2007.
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
Graph matching, which refers to a class of computational problems of finding an optimal correspondence between the vertices of graphs to minimize (maximize) their node and edge disagreements (affinities), is a fundamental problem in computer science and relates to many areas such as combinatorics, pattern recognition, multimedia and computer vision. Compared with the exact graph (sub)isomorphism often considered in a theoretical setting, inexact weighted graph matching receives more attentions due to its flexibility and practical utility. A short review of the recent research activity concerning (inexact) weighted graph matching is presented, detailing the methodologies, formulations, and algorithms. It highlights the methods under several key bullets, e.g. how many graphs are involved, how the affinity is modeled, how the problem order is explored, and how the matching procedure is conducted etc. Moreover, the research activity at the forefront of graph matching applications especially in computer vision, multimedia and machine learning is reported. The aim is to provide a systematic and compact framework regarding the recent development and the current state-of-the-arts in graph matching.
Conference Paper
Full-text available
We consider the problem of comparing deformable 3D objects represented by graphs, i.e., triangular tessellations. We propose a new algorithm to measure the distance between triangular tessellations using a new decomposition of triangular tessellations into triangle-Stars. The proposed algorithm assures a minimum number of disjoint triangle-Stars, offers a better measure by covering a larger neighborhood and uses a set of descriptors which are invariant or at least oblivious under most common deformations. We prove that the proposed distance is a pseudo-metric. We analyse its time complexity and we present a set of experimental results which confirm the high performance and accuracy of our algorithm.
Article
Full-text available
Content-based 3D object retrieval has wide applications in various domains, ranging from virtual reality to computer aided design and entertainment. With the rapid development of digitizing technologies, different views of 3D objects are captured, which requires for effective and efficient view-based 3D object retrieval (V3DOR) techniques. As each object is represented by a set of multiple views, V3DOR becomes a group matching problem. Most of state-of-the-art V3DOR methods use one single feature to describe a 3D object, which is often insufficient. In this paper, we propose a feature fusion method via multi-modal graph learning for view-based 3D object retrieval. Firstly, different visual features, including 2D Zernike moments, 2D Fourier descriptor and 2D Krawtchouk moments, are extracted to describe each view of a 3D object. Then the Hausdorff distance is computed to measure the similarity between two 3D objects with multiple views. Finally we construct multiple graphs based on different features and learn the optimized weights of each graph automatically for feature fusion task. Extensive experiments are conducted on the ETH-80 dataset and the National Taiwan University 3D model dataset. The results demonstrate the superior performance of the proposed method, as compared to the state-of-the-art approaches.
Article
Full-text available
Bipartite (BP) has been seen to be a fast and accurate suboptimal algorithm to solve the Error-Tolerant Graph Matching problem. Recently, Fast Bipartite (FBP) has been presented that obtains the same distance value and node labelings but in a reduced time. Both algorithms approximate the quadratic problem in a linear problem and they do it through a specific cost matrix. FBP imposes the Edit costs to be defined such as the Edit distance is a distance function. Originally, the Hungarian method was used but it has been seen the Jonker-Volgenant linear solver obtains similar results than the Hungarian method but with an important run time reduction. Nevertheless, this second solver has some convergence problems on some specific cost matrices. The aim of this paper is to define a new cost matrix such that the Jonker-Volgenant solver converges and the matching algorithm obtains the same distance value than the BP algorithm.
Article
We address the problem of comparing deformable 3D objects represented by graphs such as triangular tessellations. We propose a new graph matching technique to measure the distance between these graphs. The proposed approach is based on a new decomposition of triangular tessellations into triangle-stars. The algorithm ensures a minimum number of disjoint triangle-stars, provides improved dissimilarity by covering larger neighbors and allows the creation of descriptors that are invariant or at least oblivious under the most common deformations. The present approach is based on an approximation of the Graph Edit Distance, which is fault-tolerant to noise and distortion, thus making our technique particularly suitable for the comparison of deformable objects. Classification is performed with supervised machine learning techniques. Our approach defines a metric space using graph embedding and graph kernel techniques. It is proved that the proposed distance is a pseudo-metric. Its time complexity is determined and the method is evaluated against benchmark databases. Our experimental results confirm the performances and the accuracy of our system.
Article
We propose an algorithm that efficiently solves the linear sum assignment problem with error-correction and no cost constraints. This problem is encountered for instance in the approximation of the graph edit distance. The fastest currently available solvers for the linear sum assignment problem require the pairwise costs to respect the triangle inequality. Our algorithm is as fast as these algorithms, but manages to drop the cost constraint. The main technical ingredient of our algorithm is a cost-dependent factorization of the node substitutions.
Article
n this work, we present a method which transforms an object into another. The computation of this transformation is used as a measure of shape-of-object dissimilarity. The considered objects are composed of voxels. Thus, the shape difference of two objects can be ascertained by counting how many voxels we have to move and how far to change one object into another. This work is based on the method presented in [Pattern Recognition 29 (1996) 1117], and our contributions to such a work are a method of optimum transformation of objects and a proposed method of principal axes, which is used to orientate objects. The proposed method is applied to global data. Finally, we present some results using objects of the real world.