Searching the EAGLE Epigraphic Material through
Image Recognition via a Mobile Device
Paolo Bolettieri1, Vittore Casarosa1, Fabrizio Falchi1, Lucia Vadicamo1,
Philippe Martineau2, Silvia Orlandi3, Raffaella Santucci3
1CNR-ISTI, 2Eureva, 3Università di Roma La Sapienza
Abstract. This demonstration paper describes the mobile application developed
by the EAGLE project to increase the use and visibility of its epigraphic material.
The EAGLE project (European network of Ancient Greek and Latin Epigraphy)
is gathering a comprehensive collection of inscriptions (about 80% of the surviv-
ing material) and making it accessible through a user-friendly portal, which sup-
ports searching and browsing of the epigraphic material. In order to increase the
usefulness and visibility of its content, EAGLE has developed also a mobile ap-
plication to enable tourists and scholars to obtain detailed information about the
inscriptions they are looking at by taking pictures with their smartphones and
sending them to the EAGLE portal for recognition. In this demonstration paper
we describe the EAGLE mobile application and give an outline of its features
and its architecture.
Keywords: mobile application, image recognition, similarity search, epigraphy,
Latin and Greek inscriptions
1 The EAGLE Project
One of the main motivations of the project EAGLE (Europeana network of Ancient
Greek and Latin Epigraphy , a Best Practice Network partially funded by the Euro-
pean Commission) was to collect in a single repository information about the thousands
of Greek and Latin inscriptions presently scattered in a number of different institutions
(museums and universities) across all Europe. The collected information, about 1,5 mil-
lion digital objects (texts and images), representing approximately 80% of the total
amount of classified inscriptions in the Mediterranean area, is being ingested into Eu-
ropeana and is also made available to the scholarly community and to the general pub-
lic, for research and cultural dissemination, through a user-friendly portal supporting
advanced query and search capabilities.
In addition to the query capabilities (full text search a la Google, fielded search,
faceted search and filtering), the EAGLE portal supports two applications intended to
make the fruition of the epigraphic material easier and more useful. A Story Telling
application provides tools to assemble epigraphy-based narratives to be made available
at the EAGLE portal, intended for the fruition of the epigraphic material by less knowl-
edgeable users or young students. A Flagship Mobile Application (FMA) enables a user
to get information about one visible inscription by taking a picture with a mobile device,
and sending it to the EAGLE portal for recognition. This demo will show the EAGLE
Flagship Mobile Application (presently implemented on Android) and the next sections
will briefly describe the functionality and the architecture of the FMA.
2 The Flagship Mobile Application
The FMA enables a user to get information about one visible inscription by taking a
picture with a mobile device, and sending it to the EAGLE portal, specifying the recog-
nition mode. In “Similarity Search Mode” the result is a list of inscriptions (just thumb-
nails and some summary information) ranked in order of similarity to the image sent to
the EAGLE server; by clicking on one of the thumbnails the user will receive all the
information associated with that inscription. In “Exact Match Mode” the result is all the
information associated with the image, if recognized, or a message saying that the im-
age was not recognized.
The Graphical User Interface (GUI) of the FMA, available on the touch screen of
the mobile device gives access to the functions listed below. The user can navigate
through the different functions with tabs, and at any moment has access to the initial
Search EAGLE content using image recognition in Similarity Search mode
Search EAGLE content using image recognition in Exact Match mode
Search EAGLE content using text search
Login to the mobile application using an account already existing at the EAGLE
For logged-in users, annotate and save queries and their results
For logged-in users, annotate and save pictures taken with the mobile device
For logged-in user, access and review the navigation history.
The mobile application communicates (through the Internet) with the Flagship Mo-
bile Application (FMA) server, which in turn communicates with the EAGLE server
using the specific APIs supporting the mobile application. Figure 1 shows the main
functionality blocks of the EAGLE portal and the communication APIs between the
FMA server and the EAGLE server. Complete details of the architecture and the mobile
application can be found in .
The Image Recognizer (middle block on the right in the EAGLE server) has three
main functions: (i) Image Feature Extractor, (ii) Image Indexer and support of Similar-
ity Search Mode, (iii) Support of Exact Match Mode.
Figure 1 – Basic Architecture of the EAGLE server and the FMA server
2.1 Image Feature Extractor
The Image Feature Extractor analyses the visual content of the EAGLE images and
captures certain local visual properties of an image (features). Local features are low
level descriptions of Keypoints (or salient points), which are interest points in an image,
whose description is invariant to scale and orientation. The result of extraction of visual
features is a mathematical description of the image visual content that can be used to
compare different images, judge their similarity, and identify common content. The
Image Recognizer in EAGLE has a multi-threaded architecture for fast extraction of
features and for taking advantage of multicore processors. It has a plug-in architecture,
so that it is easy to add or delete the mathematical libraries supporting the many differ-
ent algorithms for the extraction of local visual features, such as SIFT, SURF, ORB,
etc. and their aggregations, such as BoF and VLAD .
2.2 Indexer and support of Similarity Search and Exact Match Modes
The Image Indexer leverages the functionality of the Melampo CBIR System.
Melampo stands for Multimedia Enhancement for Lucene to Advanced Metric PivOt-
ing . It is an open source Content Based Image Retrieval (CBIR) library developed
at CNR-ISTI that allows efficient comparison of images by visual similarity through
the use of local features.
After the visual feature extraction, the local features are encoded using an approach
called “Bag of Features”, where a vocabulary of visual words is created starting from
all the local descriptors of the whole dataset. The set of all the local descriptors of all
the images is divided into a number of clusters (depending on the algorithms used, this
number can go from a few hundreds to tens of thousands) and a textual tag is assigned
to each cluster (usually in a random fashion). The set of all the textual tags becomes the
“vocabulary” of visual words related to the whole set of images. At this point each
image can be described by a set of “words” in this vocabulary, corresponding to the
clusters containing the visual features of the image.
The support of Similarity Search Mode is based on the use of the Lucene search
engine. Each image is represented by a set of words (the textual tags of the visual vo-
cabulary), and Lucene builds the index of those words. At query time, the query image
is transformed into a set of words, and then Lucene performs a similarity search, re-
turning a list of images ranked according to the similarity with the query image.
The support of Exact Match Mode is based on a set of classifiers, each one recog-
nizing a specific epigraph. The construction of the classifiers is done off-line, selecting
from the complete database those epigraphies for which several images are available.
The set of images representing the same epigraph is the training set used for building
the classifier of that epigraph. At query time, the recognizer performs a similarity search
for the image to be recognized and then takes from the result list the first k results for
which there is also a classifier. The recognizer uses the RANSAC algorithm to perform
geometry consistency checks  and assign a score to each class. We decided to assign
to each class the highest matching score (i.e., percentage of inliers after the RANSAC)
between the query image and all the image in the classifier. If the score is above a given
threshold, the image is recognized.
The Flagship Mobile Application has been tested on a preliminary database of about
17 thousand images for Similarity Search and 70 training sets for Exact Match, using
different vocabulary size and visual features representation. Presently, the best results
have been obtained using VLAD for visual features aggregations, with a codebook size
1. The EAGLE Project http://www.eagle-network.eu/
2. The EAGLE Project, Deliverable D4.1 – Aggregation and Image Retrieval system (AIM)
3. Gennaro C., Amato G., Bolettieri P., Savino P.. An approach to content-based image re-
trieval based on the Lucene search engine library. In: ECDL 2010 - Research and Advanced
Technology for Digital Libraries. 14th European Conference (Glasgow (UK), , pp. 55 - 66.
4. Jégou, H., Perronnin, F., Douze, M., Sanchez, J., Perez, P., & Schmid, C. (2012). Aggregat-
ing local image descriptors into compact codes. Pattern Analysis and Machine Intelligence,
IEEE Transactions on, 34(9), 1704-1716.
5. Amato, Giuseppe, Fabrizio Falchi, and Claudio Gennaro. "Geometric consistency checks
for kNN based image classification relying on local features." In Proceedings of the Fourth
International Conference on SImilarity Search and APplications, pp. 81-88. ACM, 2011.