Understanding geometric manipulations of images through bovw-based hashing

Abstract

The increasing use of low cost imaging devices and the innovations in terms of media distribution technologies induce a growing interest on technologies able to protect digital visual media against malicious manipulations of the visual contents. One of the main problems addressed in this research area is the blind detection of traces of forgery on an image obtained through the internet. Specifically, in this paper we consider the context of communications, where malicious image manipulations should be detected by a receiver. In the proposed method, an image hash based on the Bag of Visual Words paradigm is attached as signature to the image before trans mission. The forensic hash is then analyzed at destination to detect the geometric transformations which have been applied to the received image. This task is fundamental for further processing which usually assumes that the received image is aligned with the original one, as in the case of tampering detection systems. Experiments show that the proposed approach outperforms state-of-the art methods by obtaining a good margin in terms of performances.

Full text

Page 1

UNDERSTANDING GEOMETRIC MANIPULATIONS OF IMAGES
THROUGH BOVW-BASED HASHING
S. Battiato, G. M. Farinella, E. Messina, G. Puglisi
Image Processing Laboratory
University of Catania, Italy
{battiato, gfarinella, emessina, puglisi}@dmi.unict.it
ABSTRACT
The increasing use of low cost imaging devices and the in-
novationsintermsofmediadistributiontechnologiesinducea
growing interest on technologies able to protect digital visual
media against malicious manipulations of the visual contents.
One of the main problems addressed in this research area is
the blind detection of traces of forgery on an image obtained
through the internet. Specifically, in this paper we consider
the context of communications, where malicious image ma-
nipulations should be detected by a receiver. In the proposed
method, an image hash based on the Bag of Visual Words
paradigm is attached as signature to the image before trans-
mission. The forensic hash is then analyzed at destination
to detect the geometric transformations which have been ap-
plied to the received image. This task is fundamental for fur-
ther processing which usually assumes that the received im-
age is aligned with the original one, as in the case of tamper-
ing detection systems. Experiments show that the proposed
approach outperforms state-of-the art methods by obtaining a
good margin in terms of performances.
Index Terms— Image forensics, Forensic hash, Bag of
Visual Word, Tampering, Geometric transformations, Image
validation and authentication
1. INTRODUCTION AND MOTIVATIONS
The increasing use of digital visual media in communications
has a significant impact in nowadays society. Recent esti-
mates about popular websites [1] report about 4 billion pho-
tographs on Flickr (≈ 4000 uploads per minute), 120 million
videos on YouTube (≈ 20 hours of video are uploaded per
minute) and 15 billion images on Facebook (≈ 22,000 up-
loads per minute). Moreover, among the other archives used
inimageforensics, ahugeamountofdatacomesfromsurveil-
lance systems and from digital archives of probationary ma-
terial used in courts of law.
The need of methods useful to establish the validity and
authenticity of an image received through an Internet commu-
nication is confirmed by many episodes that make question-
able the use of digital images as evidence material [2]. To
deal with this problem different solutions have been recently
proposed in literature [3, 4, 5, 6]. Most of them share the
same basic scheme: i) a hash code based on the content of
the image to be sent is attached to the image; ii) the hash is
analyzed at destination to verify the reliability of the received
image.
An image hash is a distinctive signature which represents
the visual content of the image in a compact way (usually just
few bytes). The image hash should be robust against allowed
operations and at the same time it should differ from the
one computed on a different/tampered image. Image hashing
techniques are considered extremely useful to validate the au-
thenticity of an image received through the Internet. Despite
the importance of the binary decision task related to the im-
age authentication, in the application context of Forensic Sci-
ence is fundamental to provide scientific evidences through
the history of the possible manipulations applied to the origi-
nal image to obtain the one under analysis. In many cases, the
source image is typically unknown, and, as in the application
context of this paper, all the information about the history of
the image should be recovered through the short image hash
signature, making more challenging the task (blind estimation
of manipulations). The list of manipulation provides to the
end user the information needed to decide whether the image
can be trusted or not.
In order to perform tampering localization1, the receiver
should be able to filter out all the geometric transformations
(e.g., rotation, scaling) added to the tampered image, in order
to align the received image with the one at the sender. The
alignment should be done in a blind way: at destination one
can use only the received image and the image hash to deal
with the alignment problem since the reference image does
not exist. The challenging task of blindly understanding the
geometric transformations occurred on a received image mo-
tivates this paper.
Despite different robust alignment techniques have been
proposed by computer vision researchers [7, 8, 9, 10], these
1Tampering localization is the process of localizing the regions of the
image that have been manipulated for malicious purposes to change the se-
mantic meaning of the visual message.

Page 2

Training ImagesFeatures ExtractionClustering Codebook
UntrustedUntrusted
Network
Manipulation/Tampering
Selected Features Selected FeaturesSelected FeaturesSelected Features
11
2
3
4
5



2
3
22
3
3
4
5


11
33
11
2
3
3
4
5



2
33
32
1
sr

ˆ

Trusted
Network
4
5
4
5
4
5
r


2
1
ˆ
11

n

nn
n
Image hash
11

n
hr

nn
n
11

n
hs

nn
n
Alignment
s

 ˆ
ˆ,
 ˆ
Fig. 1. Schema of the proposed approach.
techniques are unsuitable in the context of forensic hashing,
since the most important requirement is that the image signa-
ture should be as “compact” as possible to reduce the over-
head of the network communications. To fit the underlying
requirements, authors of [5] have proposed to exploit infor-
mation extracted through Radon transform and scale space
theory to estimate the parameters of the geometric transfor-
mations. To make more robust the alignment phase under ma-
nipulations such as cropping and tampering, an image hash
based on robust invariant features has been proposed in [6].
The latter technique extended the idea previously proposed in
[4] by employing the Bag of Visual Words (BOVW) model
to represent the features to be used as image hash. The ex-
ploitation of the BOVW representation is useful to reduce the
space needed for the image signature, by maintaining the per-
formances of the alignment component.
Building on the technique described in [6], we propose a
new method to detect the geometric manipulations occurred
on an image starting from the hash computed on the origi-
nal one. We exploit replicated visual words and a cascade
of estimators to establish the geometric parameters (rotation
and scale). As pointed out by the experimental results, our
approach obtains the best results with a significant margin in
terms of estimation accuracy with respect to state-of-the-art
methods.
The remainder of the paper is organized as follows: Sec-
tion 2 presents the proposed framework. Section 3 reports
the experiments and discusses the results. Finally, Section 4
concludes the paper with avenues for further research.
2. PROPOSED METHOD
As stated in the previous section, one of the common steps of
tampering detection systems is the alignment of the received
image. Image registration is crucial since all the other tasks
(e.g., tampering localization) usually assume that the received
image is aligned with the original one, and hence could fail if
the registration is not properly done. Classical registration
approaches [7, 8, 9, 10] cannot be directly employed in the
context considered in this work due the limited information
that can be used (i.e., original image is unknown and the im-
age hash should be as short as possible).
The schema of the system is shown in Fig. 1. As in [6],
we adopt a Bag of Visual Words based representations [11]
to reduce the dimensionality of the feature descriptors to be
used as hash for the alignment component. A codebook is
generated by clustering a set of SIFT [12] extracted on train-
ing images. The pre-computed codebook is shared between
sender and receiver. It should be noted that the codebook is
built only once, and used for all the communications between
a sender and a receiver (i.e., no extra overhead for each com-
munication). Sender extracts SIFT features and sorts them
in descending order with respect to the contrast values. Af-
terward, the top n SIFT features are selected and associated
to the id label corresponding to the closest visual word be-
longing to the pre-computed codebook. Hence, the final hash
signature for the alignment component is created: for each
selected SIFT, the id label, the dominant direction θ, and the
scale σ are considered. The hash signature computed at the
source (hs) together with the corresponding image are sent
to the destination. Our system assumes that the image is
sent over a network consisting of possibly untrusted nodes,

Page 3

0 50100150 200250300350
0
2
4
6
8
10
12
14
θs(degrees)
(a)
050 100150200250 300350
0
5
10
15
20
θr(degrees)
(b)
−150−100 −50
θr−θs (degrees)
0 50100150
0
1000
2000
3000
4000
5000
6000
(c)
0246810121416
0
20
40
60
80
100
120
σs
(d)
0510
σr
1520
0
50
100
150
200
250
(e)
0510 1520 25
0
2
4
6
8
10
12x 104
σr/σs
(f)
Fig. 2. (a) and (b): histograms of the dominant directions of Isand Ir; (c): histogram of the difference of the directions of
wrong matching; (d) and (e): histograms of the scales of Isand Ir; (f): histogram of the ratios of the scales of wrong matching.
whereas the signature is sent upon request from a trusted au-
thentication server which encrypts the hash in order to guar-
antee its integrity [3]. During the untrusted communication
the image could be manipulated for malicious purposes.
Once the image reaches the destination, the receiver gen-
erates the hash signature (hr) on the received image by
using the same procedure employed by the sender.
terward, the entries of the hashes hs and hr are matched
considering the id values (see Fig. 1). The alignment is
hence performed by estimating two transformation param-
eters: the rotation angle?θ and the scaling factor ? σ.
directions of the signature entries with corresponding id,
Θ = {θr,i− θs,j| ids,i= idr,j; i,j = 1...n}, and the ra-
tio of scales of the matchings between hs and hr, Σ =
?σr,i
geometric parameters proceeds with a cascade of simple es-
timators. A distribution of orientations is obtained by quan-
tizing the values of Θ. Specifically, a histogram with bins of
regular size in the range [−180,180] degrees is built. The bin
with the highest frequency is selected, and the estimation of
therotation?θ isperformedbyconsideringthemeanoftheval-
to filter out the wrong matchings between the hash signatures
hsand hrto further estimate the scale parameter. Indeed, the
scale parameter ? σ is estimated by building a histogram with
erageoftheratioofscalescorrespondingtofilteredmatchings
which fall into the bin with highest frequency.
Af-
To
this aim, we consider the differences between dominant
σs,j| ids,i= idr,j; i,j = 1...n
?
. The estimation of the
ues of Θ which fall into the selected bin. This step is used also
quantized bins in the range [0,maxσ], and considering the av-
It should be noted that some id values may appear more
than once in hsand/or in hr. Even if a small number of SIFT
are selected during the image hash generation process, the
conflict due the replicated id can arise. For example, it is pos-
sible that a selected SIFT has no unique dominant direction
[12]; in this case the different directions are coupled with the
same descriptor, and hence will be considered more than once
by the selection process which generates many instance of the
same id with different dominant directions.
Asexperimentallydemonstratedinthenextsection, byre-
taining the replicated visual words the accuracy of geometric
transformation estimation increases, and the number of “un-
matched” images decreases (i.e., image pairs that the algo-
rithm is not able to process because there are no matchings
between hsand hr). Our approach considers all the possible
matchings in order to preserve the useful information. The
correct matchings are hence retained but other wrong pairs
could be generated. Since the noise introduced by consider-
ing correct and incorrect pairs can badly influence the final
estimation results, the presence of possible noising matchings
should be taken into account during the estimation process.
Our approach deals with the problem of wrong matchings by
employing the estimators in cascade. The rationale behind
of the usefulness of the cascade is reported in the following
through an example.
In Fig. 2(a) and Fig. 2(d) are reported the distribution
of the dominant directions and scales of SIFT extracted on
an image Is, whereas in Fig. 2(b) and Fig. 2(e) are shown
the ones related to a transformed version Irobtained through

Page 4

scaling and rotation.
In order to study the distributions related to the wrong
matchings, all the matchings between SIFT points of the im-
age Isand the ones of the image Irhave been generated, and
the correct matchings have been removed. The distribution of
the subsets obtained from Θ and Σ by considering only the
wrong matchings are reported in Fig. 2(c) and Fig. 2(f). Note
that we are interested to understand the distributions of the
directions and scales coming from wrong matchings, since
these can introduce errors during the estimation process and
our aim is to filter them out.
As can be seen from Fig. 2(c) the histogram of the differ-
ences of dominant directions (θid,r− θid,s) over the wrong
matchings shows a distribution which is quite uniform. This
happens especially when no preferred directions exist in the
considered image. So, the uniform bias introduced by the
wrong matchings into the distribution of the values belong-
ing to Θ does not influence too much the rotational estima-
tion performed through the selection of the bin of the his-
togram with the highest frequency. On the contrary a peak is
clearly visible in the scale ratio histogram
in Fig. 2(f). Hence, a direct estimation by considering the
distribution of the values within Σ fails since the estimator
is strongly induced to privilege values around one. Starting
from the above considerations, we first extract rotational in-
formationbymakinguseofthedistributionofthevaluesinΘ.
This estimate can be considered reliable even in presence of
replicated matchings (they typically distribute uniformly on
the allowed range). Then, the scale estimation is performed
by means of distribution on the scale ratio which is built by
considering only the reliable matchings which have been con-
sidered to estimate the rotational parameters (see Fig. 1). In
the proposed approach, the rotation estimation is hence used
to estimate the rotational parameter and to filter out outlier
matchings in order to better estimate the scale. This cas-
cade estimation allows us to avoid discarding useful infor-
mation coming from the replicate matchings without being
influenced by the introduced wrong pairs. As pointed out by
the results reported in the next section, the proposed cascade
approach obtains a significant margin in terms of scale esti-
mation accuracy with respect to the approach proposed in [6].
?σid,r
σid,s
?
reported
3. EXPERIMENTAL RESULTS
This section reports a number of experiments on which the
proposedapproachhasbeentestedandcomparedwithrespect
to [6]. The tests have been performed considering a subset of
the benchmark dataset used in [13] for scene classification
purposes. This dataset is made up of 4485 images (average
size of 244 × 272 pixels) belonging to fifteen different scene
categories at basic level of description: bedroom, suburb, in-
dustrial, kitchen, living room, coast, forest, highway, inside
city, mountain, opencountry, street, tallbuilding, office, store.
Thecompositionoftheconsidereddatasetallowstocopewith
the high scene variability needed to properly test methods in
the context of application of this paper.
The training set used in our experiments is built through
a random selection of 150 images from the aforementioned
dataset. Specifically, ten images have been randomly sampled
from each scene category. The test set consists of 5250 im-
ages generated through the application of different transfor-
mations on the training images2. The considered transforma-
tions are typically available on image manipulation software
(Tab. 1). Accordingly with [6], the following image manip-
ulations have been applied: cropping, rotation, scaling, tam-
pering, JPEG compression. Tampering has been performed
through the swapping of blocks (50 × 50) between two im-
ages randomly selected from the training set. Various com-
binations of the basic transformations have been also used to
make more challenging the task to be addressed. Considering
the different parameter settings, for each training image there
are 35 corresponding manipulated images into the test set.
Table 1. Image transformations.
Operations
Rotation (θ)
3, 5, 10, 30, 45 degrees
Scaling (σ)factor= 0.5, 0.7, 0.9, 1.2, 1.5
Cropping19%, 28%, 36%, of entire image
Tampering
Compression
Various combinations of above operations
Parameters
block size 50x50
JPEG Q=10
To demonstrate the usefulness of considering the rotation
and scaling estimators in cascade, and to underline the con-
tribution of the replicated visual word during the estimation,
thecomparativetestsontheaforementioneddatasethavebeen
performed by considering our method, the approach proposed
in [6]. Although Lu et al. [6] affirm that further refinements
are performed considering the points that occur more than
once, actually they do not provide any implementation detail.
In our test we have hence considered two versions of [6] with
and without replicated matchings.
The approach proposed in [6] has been reimplemented.
The Ransac [14] thresholds needed to perform the geometric
parameter estimation as proposed in [6] have been set to 3.5
degrees for the rotational model and to 0.025 for the scaling
one. These thresholds have been obtained through data anal-
ysis (inliers and outliers distributions). In order to perform
a fair comparison, the bin size of the histograms involved in
our approach has been derived taking into account the thresh-
old values used by Ransac. Specifically, the bin size used by
our approach has been obtained doubling the corresponding
Ransac thresholds: a histogram with bin size of 7 degrees
ranging from −180 to 180 degrees has been used for the rota-
tion estimation step, whereas a histogram with bin size equal
to 0.05 ranging from 0 to maxσ= 10 was employed to esti-
mate the scale. Finally, a codebook with 1000 visual words
2Training and test sets used for the experiments are available at
http://iplab.dmi.unict.it/download/CPAF_ICME_2011/
Dataset.zip.

Page 5

Table 2. Comparison with respect to unmatched images.
Number of SIFT
Lu et al. [6]
Proposed Approach
15
6.74%
0.35%
30
1.99%
0.05%
45
0.88%
0,01%
60
0.49%
0.01%
Unmatched Images
Table 3. Average rotational and scaling error.
# SIFT UnmatchedLu et al. [6]
1511.73%
303.60%
451.64%
600.91%
1511.73%
303.60%
451.64%
600.91%
Proposed Approach
9.5464
5.1619
3.9650
3.4326
14.5572
14.3468
13.5139
13.7529
0.1264
0.1136
0.1100
0.1150
0.0631
0.0438
0.0330
0.0292
Mean
Error θ
Mean
Error σ
0 50 100150200250
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
REC CURVE − THETA
Absolute deviation
REC CURVE − SIGMA
Accuracy


Lu et al. [6]
Proposed approach
012345678910
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Absolute deviation
Accuracy


Lu et al. [6]
Proposed approach
Fig. 3. REC curves comparison.
has been used to compare the approaches. The codebook has
been learned through k-means clustering on the overall SIFT
extracted on the training set.
First, let us examine the typically cases in which the con-
sidered approaches are not able to work. Two cases can be
distinguished: i) no matchings are found between the hash
built at the sender (hs) and the one computed by the re-
ceiver (hr); ii) all the matchings are replicated. The first prob-
lem can be mitigated considering a higher number of features
3 3563
3,3563
3,5
Lu et al. [6]Proposed approach
3,0
2 0
2,0
2,5
ror
Mean ErrM
1,5229
1,5184
1,5
0,947
0,7978
0,82
0,8531
0,9444
1,0
0,2414
0,3314
0 00,0
0,5
3510
θ
3045
(a) Average rotation error at varying of the rotation angle.
0,221
0,1072
0,2281
0,100, 0
0,15
0,20
0,25
Mean Error
M
Lu et al. [6]Proposed approach
0,0119
0,0696
0,0455
0,0262
0,0116
0,0172
0,0805
0,00
0,05
0,50,70,9
σ
1,21,5
(b) Average scaling error at varying of the scale factor.
Fig. 4. Comparison on single transformation (60 SIFT).
(SIFT). The second one is solved only by allowing the repli-
cated matchings (see Section 2). As reported in Tab. 2, by
increasing the the number of SIFT points there is a decreasing
of the number of unmatched images for both approaches, but
the percentage of images on which our algorithm is not able
to work is much lower than the one of [6] without replicated
matchings.
Tab. 3 shows the results obtained in terms of rotational
and scale estimation through mean error. In order to properly
compare the methods, the results have been computed taking
into account only the images on which both approaches are
able to work. The proposed approach outperforms [6] ob-
taining a considerable gain both in terms of rotational and
scaling accuracy (Tab. 3). Moreover, the performance of our
approach significantly improves with the increasing of the ex-
tracted feature points (SIFT). On the contrary, the technique
in [6] is not able to properly exploit the additional information
coming from the higher number of extracted points.
To better compare the methods, the Regression Error
Characteristic Curves (REC) has been employed [15]. The
REC curve is essentially the cumulative distribution function
of the error. The area over the curve is a biased estimation
of the expected error of an employed estimation model. In
Fig. 3 the comparison through REC curves is shown. The re-
sults have been obtained considering sixty SIFT to build the
signature of training and test images.
Additional experiments have been performed in order to

Page 6

Table 4. Results of [6] by considering replicated matchings.
Lu et al. [6] modified to consider replicated matchings
Mean Error θ
15 30 4560 15304560
8.6085 4.8084 3.73383.0924 0.16020.1815 0.19880.2176
Mean Error σ
examine the dependence of the average rotational and scal-
ing error with respect to the rotation and scale transformation
parameters respectively. Results in Fig. 4(a) show that the
rotational estimation error slightly increases with the rotation
angle. For the scale transformation, the error has lower val-
ues in the proximity of one (no scale change) and increases
considering scale factors higher or lower than one (Fig. 4(b)).
It should be noted that the proposed approach obtains in all
cases the best performances.
Finally, to further study the contribution of the replicated
matchings we performed some tests by considering the mod-
ified version of the approach proposed in [6] in which repli-
cated visual words have been introduced (Tab. 4). Although
the modified approach obtains better performance with re-
spect to the original one in terms of rotational accuracy, it
is not able to obtain satisfactory results in terms of scaling
estimation. The wrong pairs introduced by the replication of
matchings are not properly handled. On the contrary our ap-
proach works properly both in terms of rotational and scaling
estimation. As already stated in Section 2, the noise statistics
introduced by the replication of the matchings can be better
handled in the rotational histogram rather than in the scale
one. Hence, it is better to retrieve the scale information only
at the end on more reliable information filtered through the
rotational estimation step.
4. CONCLUSION AND FUTURE WORKS
The assessment of the reliability of an image received through
the Internet is an important issue in nowadays society. In this
paper the task of building a robust image signature to be used
for the alignment component of distributed forensic systems
has been addressed. The image registration step is crucial to
perform further analysis (e.g., tampering localization) which
arecommonlyusedtoestablishifareceivedimageistrustable
or not. The proposed solution exploits the Bag of Visual
Words representation, and a cascade of estimators to estab-
lish the rotational and scaling factors. The application of the
estimators in cascade allows to retain the useful information
carried by replicated matchings by avoiding the noisy infor-
mation introduced by wrong pairs. Experiments have been
performed on a representative dataset of scenes. The pro-
posed framework outperforms a recently appeared technique
by obtaining a significant margin in terms of estimation accu-
racy. Future works should concern a more in depth analysis
to establish the minimal number of SIFT needed to guaran-
tee an accurate estimation of the geometric transformations,
and a study in terms of space complexity needed to represent
the alignment component of the image signature. Moreover,
ad-hoc attacks [16] should be considered by performing tests
of performances of the proposed framework with respect to
other methods.
5. REFERENCES
[1] J. F. Gantz,
an updated forecast of worldwide information growth through
2011,” International Data Corporation White Paper, 2008.
[2] H. Farid, “Digital doctoring: how to tell the real from the fake,”
Significance, vol. 3, no. 4, pp. 162–166, 2006.
[3] Y.-C. Lin, D. Varodayan, and B. Girod, “Image authentica-
tion based on distributed source coding,” in Proceedings of the
IEEE International Conference on Image Processing, 2007.
[4] S. Roy and Q. Sun, “Robust hash for detecting and localizing
image tampering,” in Proceedings of the IEEE International
Conference on Image Processing, 2007, pp. 117–120.
[5] W. Lu, A. L. Varna, and M. Wu, “Forensic hash for multime-
dia information,” in Proceedings of the IS&T-SPIE Electronic
ImagingSymposium-Media ForensicsandSecurity, 2010, vol.
7541.
[6] W.J.Lu andM. Wu, “Multimedia forensic hashbased onvisual
words,” in Proceedings of the IEEE International Conference
on Image Processing, 2010, pp. 989–992.
[7] R. Szeliski and S. Kang, “Direct methods for visual scene re-
construction,” in Proceedings of the IEEE Workshop on Rep-
resentations of Visual Scenes, 2000, pp. 26–33.
[8] H. Shum and R. Szeliski, “Construction of panoramic mo-
saics with global and local alignment,” International Journal
of Computer Vision, vol. 36, no. 2, pp. 101–130, 2000.
[9] P. McLauchlan and A. Jaenicke, “Image mosaicing using se-
quential bundle adjustment,” Image and Vision Computing,
vol. 20, pp. 751–759, 2002.
[10] M. Brown and D. Lowe, “Automatic panoramic image stitch-
ing using invariant features,” International Journal of Com-
puter Vision, vol. 47, no. 1, pp. 59–73, 2007.
[11] R. Szeliski, Computer Vision: Algorithms and Applications,
Springer, 2010.
[12] D. Lowe, “Distinctive image features from scale-invariant key-
points,” International Journal of Computer Vision, vol. 60, no.
2, pp. 91–110, 2004.
[13] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of fea-
tures: Spatial pyramid matching for recognizing natural scene
categories,” in Proceedings of the IEEE Conference on Com-
puter Vision and Pattern Recognition, 2006, pp. 2169–2178.
[14] M.A.FischlerandR.C.Bolles, “Randomsampleconsensus: a
paradigm model fitting with applications to image analysis and
automated cartography,” Communications of the ACM, vol.
24(6), pp. 381–395, 1981.
[15] J. Bi and K. P. Bennett,
curves,” in Proceedings of the International Conference on
Machine Learning, 2003.
[16] T.-T. Do, E. Kijak, T. Furon, and L. Amsaleg, “Deluding image
recognition in SIFT-based CBIR systems,” in Proceedings of
the ACM workshop on Multimedia in forensics, security and
intelligence, 2010, pp. 7–12.
“The diverse and exploding digital universe -
“Regression error characteristic