Fausto Rabitti

Fausto Rabitti
Italian National Research Council | CNR · Institute of Information Science and Technology "Alessandro Faedo" ISTI

Laurea in Computer Science

About

137
Publications
15,829
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
2,241
Citations
Introduction
Machine learning: applying deep learning techniques to vide/image content retieval
Skills and Expertise

Publications

Publications (137)
Technical Report
Full-text available
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2021 activiti...
Technical Report
Full-text available
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2020 activiti...
Technical Report
Full-text available
The Artificial Intelligence for Multimedia Information Retrieval (AIMIR) research group is part of the NeMIS laboratory of the Information Science and Technologies Institute ``A. Faedo'' (ISTI) of the Italian National Research Council (CNR). The AIMIR group has a long experience in topics related to: Artificial Intelligence, Multimedia Information...
Chapter
Full-text available
Many approaches for approximate metric search rely on a permutation-based representation of the original data objects. The main advantage of transforming metric objects into permutations is that the latter can be efficiently indexed and searched using data structures such as inverted-files and prefix trees. Typically, the permutation is obtained by...
Conference Paper
Full-text available
La visita a musei o a luoghi di interesse di città d'ar-te può essere completamente reinventata attraverso modalità di fruizione moderne e dinamiche, basa-te su tecnologie di riconoscimento e localizzazione visuale, ricerca per immagini e visualizzazioni in realtà aumentata. Da anni il gruppo di ricerca AI-MIR porta avanti attività di ricerca su qu...
Conference Paper
Full-text available
This paper aims to develop a method that can accurately count vehicles from images of parking areas captured by smart cameras. To this end, we have proposed a deep learning-based approach for car detection that permits the input images to be of arbitrary perspectives, illumination, and occlusions. No other information about the scenes is needed, su...
Conference Paper
In a metric space, triangle inequality implies that, for any three objects, a triangle with edge lengths corresponding to their pairwise distances can be formed. The n-point property is a generalisation of this where, for any \((n+1)\) objects in the space, there exists an n-dimensional simplex whose edge lengths correspond to the distances among t...
Article
Full-text available
In 1953, Blumenthal showed that every semi-metric space that is isometrically embeddable in a Hilbert space has the n-point property; we have previously called such spaces supermetric spaces. Although this is a strictly stronger property than triangle inequality, it is nonetheless closely related and many useful metric spaces possess it. These incl...
Article
Full-text available
Metric search is concerned with the efficient evaluation of queries in metric spaces. In general,a large space of objects is arranged in such a way that, when a further object is presented as a query, those objects most similar to the query can be efficiently found. Most mechanisms rely upon the triangle inequality property of the metric governing...
Conference Paper
Full-text available
We present an image search engine that allows searching by similarity about 100M images included in the YFCC100M dataset, and annotate query images. Image similarity search is performed using YFCC100M-HNfc6, the set of deep features we extracted from the YFCC100M dataset, which was indexed using the MI-File index for efficient similarity searching....
Chapter
During the last 35 years, data management principles such as physical and logical independence, declarative querying and cost-based optimization have led to profound pervasiveness of relational databases in any kind of organization. More importantly, these technical advances have enabled the first round of business intelligence applications and...
Conference Paper
Metric indexing research is concerned with the efficient evaluation of queries in metric spaces. In general, a large space of objects is arranged in such a way that, when a further object is presented as a query, those objects most similar to the query can be efficiently found. Most such mechanisms rely upon the triangle inequality property of the...
Conference Paper
Full-text available
In this paper, we present YFCC100M-HNfc6, a benchmark consisting of 97M deep features extracted from the Yahoo Creative Commons 100M (YFCC100M) dataset. Three type of features were extracted using a state-of-the-art Convolutional Neural Network trained on the ImageNet and Places datasets. Together with the features, we made publicly available a set...
Conference Paper
This paper presents a corpus of deep features extracted from the YFCC100M images considering the fc6 hidden layer activation of the HybridNet deep convolutional neural network. For a set of random selected queries we made available k-NN results obtained sequentially scanning the entire set features comparing both using the Euclidean and Hamming Dis...
Conference Paper
Full-text available
Content-based image retrieval using Deep Learning has become very popular during the last few years. In this work, we propose an approach to index Deep Convolutional Neural Network Features to support efficient retrieval on very large image databases. The idea is to provide a text encoding for these features enabling the use of a text retrieval eng...
Article
Most research into similarity search in metric spaces relies upon the triangle inequality property. This property allows the space to be arranged according to relative distances to avoid searching some subspaces. We show that many common metric spaces, notably including those using Euclidean and Jensen-Shannon distances, also have a stronger proper...
Article
Full-text available
In this paper, we present a system for visually retrieving an- cient inscriptions, developed in the context of the ongoing Europeana network of Ancient Greek and Latin Epigraphy (EAGLE) EU Project. The system allows the user in front of an inscription (e.g, in a museum, street, archaeological site) or watching a reproduction (e.g., in a book, from...
Conference Paper
Full-text available
Permutation based approaches represent data objects as ordered lists of predefined reference objects. Similarity queries are executed by searching for data objects whose permutation representation is similar to the query one. Various permutation-based indexes have been recently proposed. They typically allow high efficiency with acceptable effectiv...
Article
In this paper, we consider the task of recognizing epigraphs in images such as photos taken using mobile devices. Given a set of 17,155 photos related to 14,560 epigraphs, we used a k-NearestNeighbor approach in order to perform the recognition. The contribution of this work is in evaluating state-ofthe-art visual object recognition techniques in t...
Conference Paper
Jensen-Shannon divergence is a symmetrised, smoothed version of Küllback-Leibler. It has been shown to be the square of a proper distance metric, and has other properties which make it an excellent choice for many high-dimensional spaces in ℝ*. The metric as defined is however expensive to evaluate. In sparse spaces over many dimensions the Intrin...
Article
Full-text available
In the last few years, aerial and satellite photographs have become more an more important for historical records. The availability of Geographical Information Systems and the increasing number of photos made per year allows very advanced fruition of large number of contents. In this paper we illustrate the GeoMemories approach and we focus on its...
Article
Full-text available
The state-of-The-art algorithms for large visual content recognition and content based similarity search today use the "Bag of Features" (BoF) or "Bag of Words" (BoW) approach. The idea, borrowed from text retrieval, enables the use of inverted files. A very well known issue with this approach is that the query images, as well as the stored data, a...
Conference Paper
Similarity search technique has been proved to be an effective way for retrieving multimedia content. However, as the amount of available multimedia data increases, the cost of developing from scratch a robust and scalable system with content-based image retrieval facilities is quite prohibitive. In this paper, we propose to exploit an approach tha...
Article
Full-text available
Feature-rich data, such as audio-video recordings, digital images, and results of scientific experiments, nowadays constitute the largest fraction of the massive data sets produced daily in the e-society. Content-based similarity search systems working on such data collections are rapidly growing in importance. Unfortunately, similarity search is i...
Article
Full-text available
This paper discusses and compares various approach to automatic landmark recognition in pictures, based upon image content analysis and classification. The paper first compares various visual features and image similarity functions based on local features. Finally it discusses and compares a new classification technique to decide the landmark conta...
Conference Paper
Full-text available
In this paper we propose a novel approach that allows processing image content based queries expressed as arbitrary combinations of local and global visual features, by using a single index realized as an inverted file. The index was implemented on top of the Lucene retrieval engine. This is particularly useful to allow people to efficiently and in...
Conference Paper
In this paper, we propose a technique for automatic element detection from Laser Induced Breakdown Spectroscopy (LIBS) spectra. The presented approach uses a technique derived from information retrieval and, more specifically, from the Vector Space Model, to compute the similarity between spectra of elements and samples. These spectra, obtained by...
Article
Full-text available
In many applications, the information required by the user cannot be found in just one source, but has to be retrieved from many varying sources. This is true not only of formatted data in database management systems, but also of textual documents and multimedia data, such as images and videos. We propose a mediator system that provides the end-use...
Article
Full-text available
As the number of digital images is growing fast and Content-based Image Retrieval (CBIR) is gaining in popularity, CBIR systems should leap towards Web-scale datasets. In this paper, we report on our experience in building an experimental similarity search system on a test collection of more than 50 million images. The first big challenge we have b...
Conference Paper
Efficient processing of similarity joins is important for a large class of data analysis and data-mining applications. This primitive finds all pairs of records within a predefined distance threshold of each other. However, most of the existing approaches have been based on spatial join techniques designed primarily for data in a vector space. Trea...
Article
Full-text available
The scalability, as well as the effectiveness, of the different Content-based Image Retrieval (CBIR) approaches proposed in literature, is today an important research issue. Given the wealth of images on the Web, CBIR systems must in fact leap towards Web-scale datasets. In this paper, we report on our experience in building a test collection of 10...
Article
The state of the art of searching for non-text data (e.g., images) is to use extracted metadata annotations or text, which might be available as a related information. However, supporting real content-based audio-visual search, based on similarity search on features, is signicantly more expensive than searching for text. Moreover, such search exhib...
Conference Paper
Full-text available
In order to become an effective complement to traditional Web-scale text-based image retrieval solutions, content-based image retrieval must address scalability and efficiency issues. In this paper we investigate the possibility of caching the answers to content-based image retrieval queries in metric space, with the aim of reducing the average cos...
Conference Paper
Full-text available
In this paper we present the web user interface of a scalable and distributed system for image retrieval based on visual features and annotated text, developed in the context of the SAPIR project. Its ar- chitecture makes use of Peer-to-Peer networks to achieve scalability and eciency allowing the management of huge amount of data and simulta- neou...
Conference Paper
Full-text available
In this paper, we report on our experience in building an experimental similarity search system on a test collection of more than 50 million images, to show the possibility to scale Content-based Image Retrieval (CBIR) systems towards the Web size. First, we had to tackle the non-trivial process of image crawling and descriptive feature extraction,...
Conference Paper
Similarity search for content-based retrieval (where content can be any combination of text, image, audio/video, etc.) has gained importance in recent years, also because of the advantage of ranking the retrieved results according to their proximity to a query. However, to use similarity search in real world applications, we need to tackle the prob...
Conference Paper
Full-text available
In this paper we present a prototype system to enrich au- diovisual contents with annotations, which exploits exist- ing technologies for automatic extraction of metadata (such as OCR, speech recognition, cut detection, visual descrip- tors, etc.). The prototype relies on a metadata model that unifies MPEG-7 and LOM descriptions to edit and enrich...
Conference Paper
Full-text available
Similarity search in metric spaces is a general paradigm that can be used in several application elds. It can also be ef- fectively exploited in content-based image retrieval systems, which are shifting their target towards the Web-scale dimen- sion. In this context, an important issue becomes the design of scalable solutions, which combine paralle...
Conference Paper
Full-text available
The objective of this paper is to demonstrate the reuse of digital content, as video documents or PowerPoint presentations, by exploiting existing technologies for automatic extraction of metadata (OCR, speech recognition, cut detection, MPEG-7 visual descriptors, etc.). The multimedia documents and the extracted metadata are then indexed and manag...
Conference Paper
Full-text available
In this paper we present a scalable and distributed system for image retrieval based on visual features and annotated text. This system is the core of the SAPIR project. Its architecture makes use of Peer-to-Peer networks to achieve scalability and efficiency allowing the management of huge amount of data. For the presented demo we use 10 million i...
Conference Paper
Full-text available
Searching for non-text data (e.g., images) is mostly done by means of metadata annotations or by extracting the text close to the data. However, supporting real content-based audio-visual search, based on similarity search on features, is significantly more expensive than searching for text. More- over, the search exhibits linear scalability with r...
Conference Paper
Full-text available
In this paper we present the architecture of a Digital Library for enabling the reusing of audiovisual documents in an e-Learning context. The reuse of Learning Objects is based on automatically extracted descriptors carrying a semantic meaning for the professional that uses these Learning Objects to prepare new interactive multimedia lectures. The...
Conference Paper
In this paper we present the MILOS1 Multimedia Content Management System. MILOS supports the storage and content based retrieval of any multimedia documents whose descriptions are provided by using arbitrary metadata models represented in XML. It provides developers of digital library applications with functionalities for dealing with heterogeneous...
Conference Paper
Full-text available
Similarity search for content-based retrieval (where content can be any combination of text, image, audio/video, etc.) has gained importance in recent years, also because of the advantage of ranking the retrieved results according to their proximity to a query. However, to use similarity search in real world applications, we need to tackle the prob...
Conference Paper
Full-text available
The digital library field is recently broadening its scope of applicability and it is also continuously adapting to the frequent changes occurring in the internet society. Accordingly, digital libraries are slightly moving from a controlled environment accessible only to professionals and domain-experts, to environments accessible to casual users t...
Chapter
The retrieval process in multimedia document systems is inherently different from the retrieval process in traditional (record oriented) database systems. While the latter can be considered an exact process (records either satisfy the query or not), the former is not an exact process and the system must take into account the uncertainty factor (i.e...
Conference Paper
Full-text available
Building new digital library applications requires a developement plat- form that offers standard and powerful building blocks to support application de- velopers. In this paper we discuss our experience of using MILOS, a multimedia content management system oriented to the construction of digital libraries, to build a demanding application dedicat...
Conference Paper
Full-text available
Given the lack of standard building component, in several cases digital library applications are built from scratch using ad-hoc ap-proaches to implement all required components. On the other hand, our claim is that the development of ad-hoc software modules for each new digital library is not convenient. It is necessary to define and design stan-d...
Article
Full-text available
This paper charts a research agenda on systems-oriented issues in digital libraries. It focuses on the most central and generic system issues, including system architecture, user-level functionality, and the overall operational environment. With respect to user-level functionality, in particular, it abstracts the overall information lifecycle in di...
Conference Paper
Full-text available
(Extended abstract) Abstract. This paper describes the architecture of the MILOS Content Management System. MILOS supports the storage and content based retrieval of any XML document, as well as multimedia documents whose descriptions are provided by using heterogenous metadata models represented in XML. MILOS is flexible in the management of docum...
Conference Paper
Full-text available
We study the problem of finding relevant relationships among user defined nodes of XML documents. We define a language that determines the nodes as results of XPath expressions. The expressions are structured in a conjunctive normal form and the relationships among nodes qualifying in different conjuncts are determined as tree twigs of the searched...
Conference Paper
Full-text available
In this paper a technique for evaluating the effectiveness of MPEG-7 image features on specific image data sets is proposed. It is based on well defined statistical characteristics. The aim is to improve the effectiveness of the image retrieval process, based on the similarity computed on these features. This technique is validated with extensive e...
Article
While pages on the Web contain more and more multimedia information, such as images, videos and audio, today search engines are mostly based on textual information. There is an emerging need of a new generation of search engines that try to exploit the full multimedia information present on the Web. The approach presented in this paper is based on...
Chapter
Full-text available
Video can be considered today as a primarily mean of communication, due to its richness in informative content and to its appeal. Indeed, the combination of audio and video is an extremely important communication channel: it is considered that approximately 50% of what is seen and heard simultaneously is retained. Due to all these considerations, a...
Conference Paper
Full-text available
In order to accelerate execution of various matching and navigation operations on collections of XML documents, new indexing structure, based on tree signatures, is proposed. We show that XML tree structures can be efficiently represented as ordered sequences of preorder and postorder ranks, on which extended string matching techniques can easily s...
Conference Paper
Full-text available
As many metadata are encoded in XML, and many digital libraries need to manage XML documents, efficient techniques for search- ing in such formatted data are required. In order to efficiently process path expressions with wildcards on XML data, a new path index is pro- posed. Extensive evaluation confirms better performance with respect to other te...
Article
Full-text available
Similarity search structures for metric data typically bound object partitions by ball regions. Since regions can overlap, a relevant issue is to estimate the proximity of regions in order to predict the number of objects in the regions’ intersection. This paper analyzes the problem using a probabilistic approach and provides a solution that effect...