ChapterPDF Available

SPLX-Perm: A Novel Permutation-Based Representation for Approximate Metric Search



Many approaches for approximate metric search rely on a permutation-based representation of the original data objects. The main advantage of transforming metric objects into permutations is that the latter can be efficiently indexed and searched using data structures such as inverted-files and prefix trees. Typically, the permutation is obtained by ordering the identifiers of a set of pivots according to their distances to the object to be represented. In this paper, we present a novel approach to transform metric objects into permutations. It uses the object-pivot distances in combination with a metric transformation, called n-Simplex projection. The resulting permutation-based representation, named SPLX-Perm, is suitable only for the large class of metric space satisfying the n-point property. We tested the proposed approach on two benchmarks for similarity search. Our preliminary results are encouraging and open new perspectives for further investigations on the use of the n-Simplex projection for supporting permutation-based indexing.
SPLX-Perm: A Novel Permutation-Based
Representation for Approximate Metric Search
Lucia Vadicamo1, Richard Connor2, Fabrizio Falchi1,
Claudio Gennaro1, and Fausto Rabitti1
1Institute of Information Science and Technologies (ISTI), CNR, Pisa, Italy
2Division of Mathematics and Computing Science, University of Stirling, Scotland
Abstract. Many approaches for approximate metric search rely on a
permutation-based representation of the original data objects. The main
advantage of transforming metric objects into permutations is that the
latter can be efficiently indexed and searched using data structures such
as inverted-files and prefix trees. Typically, the permutation is obtained
by ordering the identifiers of a set of pivots according to their dis-
tances to the object to be represented. In this paper, we present a
novel approach to transform metric objects into permutations. It uses
the object-pivot distances in combination with a metric transformation,
called n-Simplex projection. The resulting permutation-based represen-
tation, named SPLX-Perm, is suitable only for the large class of metric
space satisfying the n-point property. We tested the proposed approach
on two benchmarks for similarity search. Our preliminary results are en-
couraging and open new perspectives for further investigations on the use
of the n-Simplex projection for supporting permutation-based indexing.
Keywords: approximate metric search ·permutation-based indexing ·
metric embedding ·n-point property ·n-Simplex projection
1 Introduction
Searching a data set for the most similar objects to a given query is a fun-
damental task in computer science. Over the years several methods for exact
similarity search were proposed in the literature. These approaches guarantee
to find the true result set. However, they scale poorly with the dimensionality
of the data (a phenomenon known as “curse of dimensionality”) and mostly
they are not convenient to deal with very large data sets. To overcome these
issues, the research community has developed a wide spectrum of techniques for
approximate similarity search, which have higher efficiency though at the price
of some imprecision in the results (e.g. some relevant results might be missing
or some ranking errors might occur). Among them, we can distinguish between
Springer Nature Switzerland AG 2019
G. Amato et al. (Eds.): SISAP 2019, LNCS 11807, pp. 4048, 2019.
Final authenticated publication:
2 L. Vadicamo et al.
1) approaches specialised for a particular kind of data (e.g. Euclidean vectors),
and 2) techniques applicable to generic metric data objects. assessing the dis-
similarity of any two objects. The advantage of the former class of approaches,
like the Product Quantization [13] and the Inverted Multi-Index [5], is that they
have very high efficiency and effectiveness. However, the engineering effort to de-
sign a method specialised for any particular data or application is typically too
high. The metric approaches, instead, overcome this issue since they are applica-
ble to generic metric objects without assuming a prior knowledge of the nature
of the data. Successful examples of metric approximate indexing and searching
techniques are the Permutation-based Indexing (PBI) ones, such as [4,7,14].
PBI techniques leverage the idea of transforming each metric object into a
permutation of a finite set of integers in such a way that similar objects have
similar permutations. The main advantage is that the permutations can be ef-
ficiently indexed and searched, e.g., using inverted files. The similarity queries
are then performed in the permutation space by selecting objects whose permu-
tations are the most similar to the query permutation. The common approach
to generate a permutation-based representation of a data object is based on se-
lecting a finite set of pivots (reference objects) and measuring the distances of
each pivot to the object to be represented: the permutation is obtained as the
list of the pivot identifiers ordered according to their distance to the object.
The main contribution of this paper is describing a novel approach to gener-
ate permutations associated with metric data objects. The proposed technique
is applicable only to the large class of metric spaces satisfying the so-called n-
point property [8,6]. This class encompasses many commonly used metric spaces,
such as Cartesian spaces of any dimensionality regarded with the Euclidean,
Cosine, Jensen-Shannon or Quadratic Form distances, and more generally any
Hilbert-embeddable space [6]. Our technique exploits the n-Simplex projection
[10], which is a metric transformation that allows projecting the data objects
into a finite-dimensional Euclidean space. Starting from the idea that this space
transformation maps similar objects into similar Euclidean vectors, we propose
to process each projected vector to further generate a permutation-based repre-
sentation. We show that, in most of the tested cases, our permutations are more
effective than traditional permutations. Therefore, we believe that our technique
may be relevant for many permutation-based indexing and searching techniques,
even though we are aware that it may require more work to mature.
2 Background
We are interested in searching a (large) finite subset of a metric space (D, d),
where Dis a domain of objects and d:D×DR+is a metric function
[15]. Many methods for approximate metric search rely on transforming the
original data objects into a more tractable space, e.g. by exploiting the distances
to a set of pivots. In the following, we summarise key concepts of two pivot-
based approaches that transform metric objects into permutations and Euclidean
vectors, respectively.
SPLX-Perm 3
Permutation-based representation. For a given metric space (D, d) and a
set of pivots {p1, . . . , pn} ⊂ D, the traditional permutation-based representa-
tion Πo(briefly permutation) of an object o∈ D is the sequence of the pivots
identifiers {1, . . . , n}ordered by their distance to o. Formally, the permutation
Πo= [Πo(1), Πo(2), ..., Πo(n)] lists the pivot identifiers in an order such that
i∈ {1, . . . , n 1},d(o, pΠo(i))d(o, pΠo(i+1)). An equivalent representation
is the inverted permutation Π1
owhose i-th element denotes the position of the
pivot piin the permutation Πo.
Most of the PBI methods, e.g. [4,11,14], use only a fixed-length prefix of
the permutations to represent and compare objects. It means that only the
positions of the nearest lout of npivots are used for the data encoding. In
this work, we do the same since often the prefix-permutations have better or
similar effectiveness than the full-length permutations [4], resulting also in a
more compact data encoding. The prefix permutations are compared using top-l
distances [12]. We use the Spearman Rho with location parameter l, defined as
Sρ,l(Πo1, Πo2) = `2(Π1
o1,l, Π 1
o2,l), where Π1
o,l is the inverted prefix permutation:
o,l (i) = (Π1
o(i) if Π1
l+ 1 otherwise .(1)
n-Simplex Projection. Recently, Connor et al. [8,9,10] investigated how to
enhance the metric search on a class of spaces meeting the so-called n-point
property, which is a geometrical property stronger than the triangle inequality.
A metric space has the n-point property if for any finite set of nobjects there
exists an isometric embedding of those objects into a (n1)-dimensional Eu-
clidean space. They exploited this property to define a space transformation,
called n-Simplex projection, that allows a metric space to be trasformed into
a finite-dimensional Euclidean space. It uses the distances to a set of pivots
Pn={p1, . . . , pn}for mapping metric objects to Euclidean vectors. Formally,
the n-Simplex projection associated with the pivot set Pnis the transformation
φPn: (D, d)(Rn, `2)
o7→ vo
where vois the only vector with a positive last component that preserves the dis-
tances of the data object to the pivots, i.e. `2(vo, vpi) = d(o, p), i∈ {1, . . . , n}.
The algorithm to compute the n-Simplex projected vectors is described in [10].
We recall that one interesting outcome of this space transformation is that
the Euclidean distance between any two projected vectors is a lower-bound of the
actual distance, and that the lower-bound converges to the actual distance for
increasing number of pivots n. Thus, the larger the nthe better the preservation
of the similarities between the data objects.
3 SPLX-Perm Representation
As recalled above, the traditional approach to associate a permutation to a data
object is sorting a set of pivot identifiers in ascending order with respect to
4 L. Vadicamo et al.
Algorithm 1: SPLX-Perm computation
Input : Pn={p1,...,pn} ⊂ D, o D
Output: The SPLX-Perm Πoassociated to the the object o
1voφPn(o); // n-Simplex projection into Rn
2voR vo;// Rotate the vector using a random rotation matrix R
3[vsorted, vindex ] = sort(vo,ascending) ; // sorts the vector elements of vo
in ascending order; vsorted is the sorted array, vindex is the sort
index vector describing the rearrangement of each element of vo
the distances of those pivots to the object to be represented. This approach is
justified by the observation that objects very close to each other should have
similar relative distances to the pivots, and thus, similar permutations.
The main goal of such kind of metric transformation is that the similar-
ity between the permutations reflects as much as possible the similarity of the
original data objects. Starting from this concept we observe that, on one hand,
the traditional permutation representation takes in consideration only the rel-
ative distances to the pivots, i.e. which is the closest pivot, the second closest
pivot, etc. On the other hand, the recently proposed n-Simplex projection maps
the data objects to Euclidean vectors by taking into consideration both object-
pivot and pivot-pivot distances. Moreover, the Euclidean distance between those
projected vectors well approximates the actual distance, especially when using
a large number of pivots. Therefore, our idea is to start from these good ap-
proximations of the data objects and further transform them into permutations.
Since we are now working in a Euclidean space, and the Euclidean distance does
not mix the contribution of values in different dimensions of the vectors, it is
reasonable to think that two vectors are very close to each other if they have
similar components in each dimension. By exploiting this idea, we propose to
generate the permutations by ordering the dimensional indexes of the n-Simplex
projected vectors in ascending order with respect to their corresponding values.
For example, the Euclidean vector [0.4,1.6,0.3,0.5] is transformed into the per-
mutation [3,1,4,2], since the third element of the vector is the smallest one, the
first element is the second smallest one, and so on.
The idea of generating a permutation from a Euclidean vector by ordering
its dimensional indexes was investigated also in [2], where only the case of fea-
tures extracted from images using a deep Convolutional Neural Network was
analysed. Moreover, in [2] the intuition was that individual dimensions of the
deep feature vectors represent some sort of visual concepts and that the value
of each dimension specifies the importance of that visual concept in the image.
Here we observe that a similar approach can be applied to Euclidean data in
general, and thanks to the use of the n-Simplex projection it can be extended
to a large class of metric objects as well. The only problem on applying this
approach on general Euclidean vectors is that the variance of the values in a
given dimensional position might be very different when varying the considered
SPLX-Perm 5
position. This happens, for example, in the case of vectors obtained using the
Principal Component Analysis where elements in the first dimensional positions
have higher variance than elements in the other dimensions. Other examples are
the vectors obtained with the n-Simplex projection that, by construction, have
higher values in top position and values that decrease to zero in the last compo-
nents. To overcome this issue we propose to randomly rotate the vectors before
transforming them into permutations. In facts, the random rotation distributes
the information equally along all the dimensions of the vectors while preserving
the Euclidean distance.
In summary, given a set of pivots Pn, the proposed approach to associate a
permutation to an object oDis 1) compute the n-Simplex projected vector
φPn(o); 2) randomly rotate the obtained vector (the same rotation matrix is used
for all the data objects); 3) generate the permutation by ordering the values of the
rotated vectors. We use the term SPLX-Perms for referring to the so obtained
permutations (a pseudo-code is reported in Algorithm 1).
4 Experiments
We compared our permutation representations (SPLX-Perms) with the tradi-
tional permutation-based representations (Perms) in an approximate similarity
search scenario. The experiments were conducted on two publicly available data
SISAP colors is a benchmark for metric indexing. It contains about 113K color
histograms of medical images, each represented as 112-dimensional vector.
YFCC100M is a collection of almost 100M images from Flickr. We used a sub-
set of 1M deep Convolutional Neural Network features extracted by Amato
et al. [1] and available at Specifically, we
used the activations of the fc6 layer of the HybridNet [16] after ReLu and
`2normalization. The resulting features are 4,096-dimensional vectors.
The metrics used in the experiments are the Jensen-Shannon distance for the
SISAP colors data, and the Euclidean distance for the YFCC100M deep features.
For each data set, we considered 1,000 randomly selected queries and we built
the ground-truth for the exact k-NN query search. The approximate results set
for a given query is selected by performing the k-NN search in the permutation
space. The quality of the approximate results was evaluated using the recall@k,
that is |R ∩ RA|/k where Ris the result set of the exact k-NN search, and RA
is the set of the kapproximate results.
To have a better overview of the tested approaches, we also consider the case
in which the permutations are used to select a candidate result set to be re-
ranked using the original distance d. In such cases, a k0-NN search (with k0> k)
is performed in the permutation space in order to select the candidate result set.
Then the candidate results are re-ranked according to the actual distance d, and
the top-kobjects are selected to form the final approximate result set RA. In
the experiments, we used k0= 100 and k= 10, if not specified otherwise.
6 L. Vadicamo et al.
110 100 1000
location parameter l
Perms, re-rank(d)
SPLX-Perms, re-rank(d)
(a) SISAP colors, Jensen-Shannon
dist., n= 1,000
10 100 1000
location parameter l
Perms, re-rank(d)
SPLX-Perms, re-rank(d)
(b) YFCC100M(1M), Euclidean
dist.,n= 4,000
Fig. 1: Recall@10 varying the location parameter l(i.e. the prefix length).
To generate the permutation-based representations we used n= 1,000 pivots
for the SISAP Colors data set, and n= 4,000 pivots for the YFCC100M data
set. We tested the quality of the results obtained using either the full-length
permutations or a fixed-length prefix of the permutations. The metric used in
the permutation space is the Spearman’s rho with location parameter l, where
the location parameter lis the length of the prefix permutation.
4.1 Results
Figures 1a and 1b show the recall@10 for the SISAP Colors and YFCC100M
data sets, respectively. Lines “Perms” and “SPLX-Perms” refer to the cases in
which the permutations are used to select the approximate result set by per-
forming a 10-NN search in the permutation space. Lines “Perms, re-rank(d)
and “SPLX-Perms, re-rank(d)” refer to the cases in which the permutation-
representation are used to select a candidate result set (obtained by performing
a 100-NN search) that is then re-ranked using the actual distance d. It is inter-
esting to note that on YFCC100M data, our full-length SPLX-Perms represen-
tation allowed us to achieve a recall that not only is better than that achieved
using the traditional full-length permutation, but it is even better than that ob-
tained by the re-ranked approach. However, we also observe that for very short
prefix-lengths the traditional permutations shown better performance than our
technique. Another interesting aspect is that when considering the traditional
permutation-based representation there is usually an optimal prefix length l < n
for which the best recall is achieved or for which the recall curve shows a plateau.
This is evident in Fig. 1, where the recall lightly decrease as the location param-
eter lgrows. This is a phenomenon experimentally observed also in other data
SPLX-Perm 7
110 100
Perms (n=1000, l=200)
SPLX-Perms (n=1000, l=200)
(a) SISAP Colors, Jensen-Shannon dist.
110 100
Perms (n=4000, l=800)
SPLX-Perms (n=4000, l=800)
(b) YFCC100M(1M), Euclidean dist.
Fig. 2: Recall@kvarying k(fixed location parameter l)
sets as shown in several works (see e.g., [3,4]). Our SPLX-Perms seems to be not
affected by this phenomenon since its recall increases when considering larger l.
Moreover, the re-ranking of candidate results selected using our permutations
achieved a recall very close to one for large prefix-lengths.
In Figure 2, we also report the recall@k with kranging from 1 to 100 for the
baselines approaches (i.e. without considering the re-ranking phase) using a fixed
prefix-length l. We can see that the improvement of the proposed approach over
the traditional permutation-based representation holds for all ks. Our SPLX-
Perm representation seems also to be more stable and provides recall values that
are up to 1.6 times higher than that obtained using traditional permutations.
5 Conclusions
In this paper, we presented a novel permutation-based representation for metric
objects, called SPLX-Perm. It exploits the n-Simplex projection to map the data
object to Euclidean vectors, which are in turn transformed into permutations.
The approach used to transform the Euclidean vectors into permutations has
some analogies with the Deep Permutation approach that was proposed in [2]
for associating permutations to visual deep features. To some extent, our work
can be viewed as a generalisation of this technique to the large class of met-
ric space meeting the n-point property. Our preliminary results show that our
SPLX-Perms are more effective than the traditional permutations, even if there
are some drawbacks with respect to the traditional permutations: 1) worse per-
formance for very small prefix permutation; 2) higher cost for generating the
SPLX-Perm since for each object we need to compute both the object-pivot dis-
tances and the n-Simplex projection. Nevertheless, we believe that our technique
8 L. Vadicamo et al.
as a lot of potentialities and deserves further investigations. In this perspective,
we plan to extend our experimental evaluation on more data sets and metrics,
using a different prefix length for the query object (to reduce the search cost)
and using a pivot selection specifically designed for the n-simplex projection.
This work was partially supported by VISECH ARCO-CNR, CUP B56J17001330004,
the AI4EU project, funded by the EC (H2020 - Contract n. 825619), and the
Short-Term-Mobility (STM) program of the CNR.
1. Amato, G., Falchi, F., Gennaro, C., Rabitti, F.: YFCC100M-HNfc6: a large-scale
deep features benchmark for similarity search. In: Proceedings of SISAP 2016. pp.
196–209. Springer International Publishing (2016)
2. Amato, G., Falchi, F., Gennaro, C., Vadicamo, L.: Deep Permutations: Deep con-
volutional neural networks and permutation-based indexing. In: Proceedings of
SISAP 2016. pp. 93–106. Springer International Publishing (2016)
3. Amato, G., Falchi, F., Rabitti, F., Vadicamo, L.: Some theoretical and experimental
observations on permutation spaces and similarity search. In: Proceedings of SISAP
2014. pp. 37–49. Springer (2014)
4. Amato, G., Gennaro, C., Savino, P.: MI-File: Using inverted files for scalable ap-
proximate similarity search. Multimed. Tools and Appl. 71(3), 1333–1362 (2014)
5. Babenko, A., Lempitsky, V.: The inverted multi-index. IEEE Trans. Pattern Anal.
Mach. Intell. 37(6), 1247–1260 (June 2015)
6. Blumenthal, L.M.: Theory and applications of distance geometry. Clarendon Press
7. Chavez, E., Figueroa, K., Navarro, G.: Effective proximity retrieval by ordering
permutations. IEEE Trans. Pattern. Anal. Mach. Intell. 30(9), 1647–1658 (2008)
8. Connor, R., Cardillo, F.A., Vadicamo, L., Rabitti, F.: Hilbert Exclusion: Improved
metric search through finite isometric embeddings. ACM Trans. Inf. Syst. 35(3),
17:1–17:27 (Dec 2016)
9. Connor, R., Vadicamo, L., Cardillo, F.A., Rabitti, F.: Supermetric search. Infor-
mation Systems (2018)
10. Connor, R., Vadicamo, L., Rabitti, F.: High-dimensional simplexes for supermetric
search. In: Proceedings of SISAP 2017. pp. 96–109. Springer (2017)
11. Esuli, A.: Use of permutation prefixes for efficient and scalable approximate simi-
larity search. Information Processing & Management 48(5), 889–902 (2012)
12. Fagin, R., Kumar, R., Sivakumar, D.: Comparing top k lists. In: Proceedings of
SODA 2003. pp. 28–36. Society for Industrial and Applied Mathematics (2003)
13. J´egou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor
search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (Jan 2011)
14. Novak, D., Zezula, P.: PPP-codes for large-scale similarity searching. In: TLDKS
XXIV. pp. 61–87. Springer (2016)
15. Zezula, P., Amato, G., Dohnal, V., Batko, M.: Similarity search: the metric space
approach, vol. 32. Springer Science & Business Media (2006)
16. Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features
for scene recognition using places database. In: Proceedings of NIPS 2014, pp.
487–495. Curran Associates, Inc. (2014)
... In previous work, we have developed a technique for mapping metric spaces into Euclidean vector spaces, called nSimplex projection [44], which can be applied on spaces meeting the the n-point property. Recently, by exploiting this technique we defined a novel approach to generate permutations associated with metric data objects [50]. Our SPLX-Perms (i.e., the permutations obtained using nSimplex projection) resulted, in most of the tested cases, more effective than traditional permutations, which makes them particularly suitable for permutation-based indexing techniques (see Section 3.2.13). ...
... Specifically, we encode all the visual and textual descriptors extracted from the videos into (surrogate) textual representations that are then efficiently indexed and searched using an off-the-shelf text search engine using similarity functions." [50]. Abstract: "Many approaches for approximate metric search rely on a permutation-based representation of the original data objects. ...
Technical Report
Full-text available
The Artificial Intelligence for Multimedia Information Retrieval (AIMIR) research group is part of the NeMIS laboratory of the Information Science and Technologies Institute ``A. Faedo'' (ISTI) of the Italian National Research Council (CNR). The AIMIR group has a long experience in topics related to: Artificial Intelligence, Multimedia Information Retrieval, Computer Vision and Similarity search on a large scale. We aim at investigating the use of Artificial Intelligence and Deep Learning, for Multimedia Information Retrieval, addressing both effectiveness and efficiency. Multimedia information retrieval techniques should be able to provide users with pertinent results, fast, on huge amount of multimedia data. Application areas of our research results range from cultural heritage to smart tourism, from security to smart cities, from mobile visual search to augmented reality. This report summarize the 2019 activities of the research group.
... To further improve the metric similarity query efficiency, high quality approximate answers instead of exact results can be returned, resulting in approximate metric similarity search. To answer approximate metric similarity search, many metric indexes are developed, including approximate M-tree variants [40,70,109,125,126], hash based methods [8,9,78], permeationbased indexes [3,29,53,81,83,116], and kNN graph based methods [74,95,106,107], to name but a few. P-Shere tree [59] is built by using the sample of query objects. ...
With the continued digitalization of societal processes, we are seeing an explosion in available data. This is referred to as big data. In a research setting, three aspects of the data are often viewed as the main sources of challenges when attempting to enable value creation from big data: volume, velocity and variety. Many studies address volume or velocity, while much fewer studies concern the variety. Metric space is ideal for addressing variety because it can accommodate any type of data as long as its associated distance notion satisfies the triangle inequality. To accelerate search in metric space, a collection of indexing techniques for metric data have been proposed. However, existing surveys each offers only a narrow coverage, and no comprehensive empirical study of those techniques exists. We offer a survey of all the existing metric indexes that can support exact similarity search, by i) summarizing all the existing partitioning, pruning and validation techniques used for metric indexes, ii) providing the time and storage complexity analysis on the index construction, and iii) report on a comprehensive empirical comparison of their similarity query processing performance. Here, empirical comparisons are used to evaluate the index performance during search as it is hard to see the complexity analysis differences on the similarity query processing and the query performance depends on the pruning and validation abilities related to the data distribution. This article aims at revealing different strengths and weaknesses of different indexing techniques in order to offer guidance on selecting an appropriate indexing technique for a given setting, and directing the future research for metric indexes.
In the domain of approximate metric search, the Permutation-based Indexing (PBI) approaches have been proved to be particularly suitable for dealing with large data collections. These methods employ a permutation-based representation of the data, which can be efficiently indexed using data structures such as inverted files. In the literature, the definition of the permutation of a metric object was derived by reordering the distances of the object to a set of pivots. In this paper, we aim at generalizing this definition in order to enlarge the class of permutations that can be used by PBI approaches. As a practical outcome, we defined a new type of permutation that is calculated using distances from pairs of pivots. The proposed technique permits us to produce longer permutations than traditional ones for the same number of object-pivot distance calculations. The advantage is that the use of inverted files built on permutation prefixes leads to greater efficiency in the search phase when longer permutations are used.
Full-text available
Metric search is concerned with the efficient evaluation of queries in metric spaces. In general,a large space of objects is arranged in such a way that, when a further object is presented as a query, those objects most similar to the query can be efficiently found. Most mechanisms rely upon the triangle inequality property of the metric governing the space. The triangle inequality property is equivalent to a finite embedding property, which states that any three points of the space can be isometrically embedded in two-dimensional Euclidean space. In this paper, we examine a class of semimetric space which is finitely four-embeddable in three-dimensional Euclidean space. In mathematics this property has been extensively studied and is generally known as the four-point property. All spaces with the four-point property are metric spaces, but they also have some stronger geometric guarantees. We coin the term supermetric space as, in terms of metric search, they are significantly more tractable. Supermetric spaces include all those governed by Euclidean, Cosine, Jensen-Shannon and Triangular distances, and are thus commonly used within many domains. In previous work we have given a generic mathematical basis for the supermetric property and shown how it can improve indexing performance for a given exact search structure. Here we present a full investigation into its use within a variety of different hyperplane partition indexing structures, and go on to show some more of its flexibility by examining a search structure whose partition and exclusion conditions are tailored, at each node, to suit the individual reference points and data set present there. Among the results given, we show a new best performance for exact search using a well-known benchmark.
Conference Paper
Full-text available
Permutation based approaches represent data objects as ordered lists of predefined reference objects. Similarity queries are executed by searching for data objects whose permutation representation is similar to the query one. Various permutation-based indexes have been recently proposed. They typically allow high efficiency with acceptable effectiveness. Moreover, various parameters can be set in order to find an optimal trade-off between quality of results and costs. In this paper we studied the permutation space without referring to any particular index structure focusing on both theoretical and experimental aspects. We used both synthetic and real-word datasets for our experiments. The results of this work are relevant in both developing and setting parameters of permutation-based similarity searching approaches.
Conference Paper
Full-text available
In this paper, we present YFCC100M-HNfc6, a benchmark consisting of 97M deep features extracted from the Yahoo Creative Commons 100M (YFCC100M) dataset. Three type of features were extracted using a state-of-the-art Convolutional Neural Network trained on the ImageNet and Places datasets. Together with the features, we made publicly available a set of 1,000 queries and k-NN results obtained by sequential scan. We first report detailed statistical information on both the features and search results. Then, we show an example of performance evaluation, performed using this benchmark, on the MI-File approximate similarity access method.
Conference Paper
Full-text available
The activation of the Deep Convolutional Neural Networks hidden layers can be successfully used as features, often referred as Deep Features, in generic visual similarity search tasks. Recently scientists have shown that permutation-based methods offer very good performance in indexing and supporting approximate similarity search on large database of objects. Permutation-based approaches represent metric objects as sequences (permutations) of reference objects, chosen from a predefined set of data. However, associating objects with permutations might have a high cost due to the distance calculation between the data objects and the reference objects. In this work, we propose a new approach to generate permutations at a very low computational cost, when objects to be indexed are Deep Features. We show that the permutations generated using the proposed method are more effective than those obtained using pivot selection criteria specifically developed for permutation-based methods.
Full-text available
Scene recognition is one of the hallmark tasks of computer vision, allowing definition of a context for object recognition. Whereas the tremendous recent progress in object recognition tasks is due to the availability of large datasets like ImageNet and the rise of Convolutional Neural Networks (CNNs) for learning high-level features, performance at scene recognition has not attained the same level of success. This may be because current deep features trained from ImageNet are not competitive enough for such tasks. Here, we introduce a new scene-centric database called Places with over 7 million labeled pictures of scenes. We propose new methods to compare the density and diversity of image datasets and show that Places is as dense as other scene datasets and has more diversity. Using CNN, we learn deep features for scene recognition tasks, and establish new state-of-the-art results on several scene-centric datasets. A visualization of the CNN layers' responses allows us to show differences in the internal representations of object-centric and scene-centric networks.
Conference Paper
In a metric space, triangle inequality implies that, for any three objects, a triangle with edge lengths corresponding to their pairwise distances can be formed. The n-point property is a generalisation of this where, for any \((n+1)\) objects in the space, there exists an n-dimensional simplex whose edge lengths correspond to the distances among the objects. In general, metric spaces do not have this property; however in 1953, Blumenthal showed that any semi-metric space which is isometrically embeddable in a Hilbert space also has the n-point property.
Many current applications need to organize data with respect to mutual similarity between data objects. A typical general strategy to retrieve objects similar to a given sample is to access and then refine a candidate set of objects. We propose an indexing and search technique that can significantly reduce the candidate set size by combination of several space partitionings. Specifically, we propose a mapping of objects from a generic metric space onto main memory codes using several pivot spaces; our search algorithm first ranks objects within each pivot space and then aggregates these rankings producing a candidate set reduced by two orders of magnitude while keeping the same answer quality. Our approach is designed to well exploit contemporary HW: (1) larger main memories allow us to use rich and fast index, (2) multi-core CPUs well suit our parallel search algorithm, and (3) SSD disks without mechanical seeks enable efficient selective retrieval of candidate objects. The gain of the significant candidate set reduction is paid by the overhead of the candidate ranking algorithm and thus our approach is more advantageous for datasets with expensive candidate set refinement, i.e. large data objects or expensive similarity function. On real-life datasets, the search time speedup achieved by our approach is by factor of two to five.
Most research into similarity search in metric spaces relies upon the triangle inequality property. This property allows the space to be arranged according to relative distances to avoid searching some subspaces. We show that many common metric spaces, notably including those using Euclidean and Jensen-Shannon distances, also have a stronger property, sometimes called the four-point property: in essence, these spaces allow an isometric embedding of any four points in three-dimensional Euclidean space, as well as any three points in two-dimensional Euclidean space. In fact, we show that any space which is isometrically embeddable in Hilbert space has the stronger property. This property gives stronger geometric guarantees, and one in particular, which we name the Hilbert Exclusion property, allows any indexing mechanism which uses hyperplane partitioning to perform better. One outcome of this observation is that a number of state-of-the-art indexing mechanisms over high dimensional spaces can be easily extended to give a significant increase in performance; furthermore, the improvement given is greater in higher dimensions. This therefore leads to a significant improvement in the cost of metric search in these spaces.
We present the Permutation Prefix Index (this work is a revised and extended version of Esuli (2009b), presented at the 2009 LSDS-IR Workshop, held in Boston) (PP-Index), an index data structure that supports efficient approximate similarity search.The PP-Index belongs to the family of the permutation-based indexes, which are based on representing any indexed object with “its view of the surrounding world”, i.e., a list of the elements of a set of reference objects sorted by their distance order with respect to the indexed object.In its basic formulation, the PP-Index is strongly biased toward efficiency. We show how the effectiveness can easily reach optimal levels just by adopting two “boosting” strategies: multiple index search and multiple query search, which both have nice parallelization properties.We study both the efficiency and the effectiveness properties of the PP-Index, experimenting with collections of sizes up to one hundred million objects, represented in a very high-dimensional similarity space.
We propose a new efficient and accurate technique for generic approximate similarity searching, based on the use of inverted files. We represent each object of a dataset by the ordering of a number of reference objects according to their distance from the object itself. In order to compare two objects in the dataset, we compare the two corresponding orderings of the reference objects.We show that this representation enables us to use inverted files to obtain very efficiently a very small set of good candidates for the query result. The candidate set is then reordered using the original similarity function to obtain the approximate similarity search result. The proposed technique performs several orders of magnitude better than exact similarity searches, still guaranteeing high accuracy. To also demonstrate the scalability of the proposed approach, tests were executed with various dataset sizes, ranging from 200,000 to 100 million objects.