Conference PaperPDF Available

Using MILOS to build a Multimedia Digital Library Application: The PhotoBook experience

Authors:

Abstract and Figures

The digital library field is recently broadening its scope of applicability and it is also continuously adapting to the frequent changes occurring in the internet society. Accordingly, digital libraries are slightly moving from a controlled environment accessible only to professionals and domain-experts, to environments accessible to casual users that want to exploit the potentialities offered by the digital library technology. These new trends require, for instance, new search paradigms to be offered, new media content to be managed, and new description extraction techniques to be used. Building digital library applications, and effectively adapting them to new emerging trends, requires to develop a platform that offers standard and powerful building blocks to support application developers. In this paper we discuss our experience of using MILOS, a multimedia content management system oriented to the construction of digital libraries, to build a demanding application dedicated to non-professional users. Specifically, we discuss the design and implementation of an on-line photo album (PhotoBook), which is a digital library application that allows people to manage their own photos, to share them with friends, and to make them publicly available and searchable. PhotoBook, uses a complex internal metadata schema (MPEG-7) and allows users to simply express complex queries (combining similarity search and fielded search), enabling them to retrieve material of interest even if metadata are imprecise or missing.
Content may be subject to copyright.
Using MILOS to build a Multimedia Digital Library
Application: The PhotoBook experience ?
Giuseppe Amato, Paolo Bolettieri, Franca Debole, Fabrizio Falchi, Fausto Rabitti, and
Pasquale Savino
ISTI-CNR, Pisa, Italy
firstname.lastname@isti.cnr.it
Abstract. The digital library field is recently broadening its scope of applica-
bility and it is also continuously adapting to the frequent changes occurring in
the internet society. Accordingly, digital libraries are slightly moving from a con-
trolled environment accessible only to professionals and domain-experts, to en-
vironments accessible to casual users that want to exploit the potentialities of-
fered by the digital library technology. These new trends require, for instance,
new search paradigms to be offered, new media content to be managed, and new
description extraction techniques to be used.
Building digital library applications, and effectively adapting them to new emerg-
ing trends, requires to develop a platform that offers standard and powerful build-
ing blocks to support application developers. In this paper we discuss our expe-
rience of using MILOS, a multimedia content management system oriented to
the construction of digital libraries, to build a demanding application dedicated
to non-professional users. Specifically, we discuss the design and implementation
of an on-line photo album (PhotoBook), which is a digital library application that
allows people to manage their own photos, to share them with friends, and to
make them publicly available and searchable.
PhotoBook, uses a complex internal metadata schema (MPEG-7) and allows
users to simply express complex queries (combining similarity search and fielded
search), enabling them to retrieve material of interest even if metadata are impre-
cise or missing.
1 Introduction
Many Digital Library Applications (DLA) supporting an effective and efficient archiv-
ing and retrieval of documents with different types of data and content, and that are used
for many diverse purposes, have been developed. However, Digital Library technology
will arrive at a definitive maturity when powerful and simple software development
tools that enable the development of DLAs, will be available. Regrettably, nowadays
several Digital Library Applications are monolithic software modules where the ap-
plication itself, the content management software, and the digital library are merged
?This work was partially supported by the DELOS NoE, funded by the European Commission
under FP6 (Sixth Framework Programme), by the ECD and VICE projects, both funded by the
Italian Ministry of Research and Education. The PhotoBook Application can be accessed from
http://milos.isti.cnr.it/
2 Authors Suppressed Due to Excessive Length
together. Many systems were built having in mind a specific application and, in many
cases, a specific document collection, thus resulting in an ad-hoc solution: all compo-
nents of the DLA – the data repository, the metadata manager, the search and retrieval
components, etc. – are specific to a given application and cannot be easily used in other
environments. In these cases the development of the DLA is expensive and it is not
possible to personalize or specialize the DL services and to adapt them to new user
requirements.
An important building block of any DLA is the module responsible for the archiv-
ing, access, and retrieval of data objects. In many cases, as types of data managed
change, or when different metadata format are used, or if different search functionality
is required, a new component is build. Standard building components to help digital
library application developers are needed.
The aim of this work is to illustrate the use of MILOS, a Multimedia Content Man-
agement System (MCMS) for digital libraries, to develop a demanding application: the
PhotoBook Digital Library (http://milos.isti.cnr.it).
The advent of digital photography, combined to the wide access to internet re-
sources, made popular the creation of personal and publicly accessible distributed photo
archives. Typical examples are the Flickr service (www.flickr.com), the snapfish service
offered by HP (www.snapfish.com), and Picasa (picasa.google.com). Users of these
archives may create and manage their own photo albums, decide who and how can
access their photos, provide a description of the photos to simplify their access. This
simple yet powerful application, poses several complex requirements to the MCMS
component, which must support: (i) the distributed storage and classification of photos,
(ii) the description of photo content through an appropriate metadata model, (iii) the
search based on photo description and photo content, (iv) the management of personal
folders (photo albums), (v) the controlled access management. Some of the mentioned
functionality, do require the development of an appropriate Web-based user interface. A
final, but significant requirement is that the development of the application must be sim-
ple and fast. The tools previously mentioned (flickr, snapfish, and Picasa) offer most of
these functionality but with search capabilities limited to manually associated metadata
tags. Furthermore, as mentioned at the beginning of this paper, they are ad-hoc applica-
tions, whose development required significant investments and whose extensions would
also require considerable efforts. Our aim is to show how, by using the general purpose
MCMS system MILOS, is possible to build an application with similar functionality –
and even with more powerful search capabilities – with a limited effort.
The development of the Photobook application has a twofold purpose: i) testing
how the most recent digital library technology is able to deal with these new emerging
trends, and ii) to verify if the MILOS multimedia content management system is capable
of providing effective tools to rapidly develop a multimedia digital library application
with high demanding requirements.
In the following we will briefly resume the functionality of the MILOS MCMS. We
will discuss the functionality of the PhotoBook Digital Library Application. Then, we
will present the architecture of PhotoBook and the specific functionality offered and
the solutions adopted for picture management, metadata management – with particular
attention to automatic metadata extraction – and photo search.
Title Suppressed Due to Excessive Length 3
2 MILOS Overview
The MILOS system [4, 1] is a Multimedia Content Management System with a num-
ber of characteristics that make it particularly suitable for the development of Digital
Library applications. In particular, as we will underline in this section, MILOS func-
tionality are particularly appealing for the development of the Photobook application,
which requires the archiving of a large number of images from users distributed over the
internet, efficient content-based retrieval of photos, according to their metadata values
and based on their physical content, creation of a Web interface for the storage, query
formulation and presentation of results.
Key characteristics of the MILOS system are the flexibility in managing different
types of data and metadata and the independence from the specific format used to rep-
resent them. This implies that the application developer is not required to specify the
details of the storage strategies used and the details of the access methods to be adopted;
he/she only needs to specify the characteristics of the data and metadata and the func-
tionality that are required, such as the requirement of storing high resolution photos,
and supporting their efficient access based on the combination of metadata attributes
and physical characteristics of photos. The flexibility of the MILOS system is also re-
lated to the possibility of developing end-user applications which are independent from
the modality used to store data and search them. In particular, it is possible to store data
described with a specific metadata model and to search them by using a different model.
Another key characteristic of MILOS is the efficiency in storing and searching mul-
timedia objects. This requires a system which is (a) scalable when the size of the archive
and the number of users accessing the application varies, and (b) efficient when process-
ing complex queries on metadata values and data object’s content.
The MILOS system has three main components: the Metadata Storage and Retrieval
(MSR) component – dedicated to support the storage of metadata and the content-based
retrieval of multimedia objects – the Multi Media Server (MMS) component – dedicated
to support the storage and access of multimedia content – and the Repository Meta-
data Integrator (RMI) component – dedicated to provide the independence between the
metadata format used by end-user applications and the internal metadata formats.
Let us provide some more details on the MSR, which is of particular importance in
the PhotBook application. Details on the MMS and RMI are provided elsewhere [4].
2.1 Metadata Storage and Retrieval
The MILOS MSR, despite the approaches adopted by other similar systems, uses an
enhanced native XML database/repository system with special features for DL applica-
tions. Indeed, XML represented metadata may have arbitrary complex structures, which
allows one to deal with complex metadata schemas, and can be easily exported and im-
ported.
The MILOS XML database [2] stores and retrieves any valid XML document. No
metadata schema or XML schema definition is needed before inserting an XML docu-
ment, except optional index definition for performance improvement. Once an arbitrary
XML document has been inserted in the database it can be immediately retrieved. This
allows DL applications to use arbitrary (XML encoded) metadata schemas and to deal
4 Authors Suppressed Due to Excessive Length
with heterogeneous metadata, without schema design constraints and/or overhead due
to metadata translation.
This special purpose native XML database/repository system is much simpler than a
general purpose commercial XML database system, but with much better performances
where needed. It supports standard XML query languages (XPath [6] and XQuery [7])
and it offers advanced search and indexing functionality on XML documents. The MI-
LOS XML database supports high performance search and retrieval on heavily struc-
tured XML documents, relying on specific index structures [3, 14], as well as full text
search [13], automatic classification [8], and feature similarity search [15, 5]. This is
particularly relevant in the Photobook application, where metadata photos are repre-
sented in the MPEG-7 format: each photo includes in the description also features au-
tomatically extracted from visual properties, such as color histograms, textures, shapes,
etc.
To deal easily and transparently with these advanced search and indexing function-
alities, the syntax of the basic XQuery language was extended to deal with approximate
match and ranking. Specifically the operator is used for approximate match (that is,
feature similarity and text search). More details can be found in [2].
3 The PhotoBook digital library application
The use of digital cameras is becoming very popular in the society. People can take
picture and immediately see the result by downloading the images on their personal
computers, avoiding the cost of printing them. Digital technology applied to the per-
sonal cameras has changed the way people use and manage their pictures. People take
much more pictures, given that they can immediately see them at no cost, print a sig-
nificant minority of them, and especially share them with friends by sending them by
email, or more frequently by storing and publishing them on on-line photo albums.
Several sites that offer popular on-line photo albums services are nowadays avail-
able on the internet and they are continuously enriched by new services to help people
on one hand to publish and share their material, on the other to access and search for
published material.
Even if on-line photo albums are mainly addressed to non-professional users, they
can be reasonably considered as digital library applications. In fact, such systems typi-
cally offer important functionality also offered by professional digital library systems.
As an example, on-line photo albums offer storage and preservation services, privacy
and rights management, personalization, metadata editing and annotation functionality,
advanced search functionality.
Two distinct types of users can be broadly identified for such systems. Users can act
as publishers when they publish, annotate, classify, and manage digital photos. Users
can act as searchers when they search for published digital material. Both publishers,
and searchers are non-professional users of these on-line services. The two roles some-
time can be merged. Many people are in fact at the same time publishers and searchers.
On-line photo albums nowadays represent an emergent phenomenon of the internet
society. Several stand-alone on-line photo album services are in fact available on the
Title Suppressed Due to Excessive Length 5
internet. In addition, almost all internet providers, portals, search engines, photo cam-
era producer, and photo printing services have added an on-line photo albums to the
services that they already provide.
In the following we discuss the requirements of the PhotoBook (http://milos.isti.cnr.it/)
application , its architecture and the various components.
3.1 PhotoBook Application Functionality
The PhotoBook application is intended to be accessible on-line by everybody. It offers
services to two categories of users: non-registered users and registered users.
Non-registered users can use the PhotoBook application to search for and access
photos that were uploaded in the system and made publicly available.
Registered users in addition to the capabilities of the non-registered users can also
1. upload photos in the system
2. manage their own photos
3. share their photos with friends
4. make their photo publicly available
Each photo managed by the PhotoBook has one single owner, which is the user that
uploaded it (we suppose that a person that uploads a picture owns it or he/she has the
right to do it). The owner can set a photo to be private or public.
All users (registered and non-registered) should be allowed to search and access
digital pictures that were uploaded in the system and that were marked by the owner as
public.
Registered users can organize their own photos in albums. An album is basically a
collection of photos. For instance, a user can decide to create an album containing the
photos that he/she took during last summer. A photo may belong to several albums at
the same time. Users can share an album (containing both private and public photos)
with friends. A private photo can be accessed just by the owner of the photo and by
his friends, when they are given an handler to an album that contains it. A user’s friend
does not need to be registered in the system. Access to an album is obtained by using a
system-generated URL containing the handler to the album. Every non-registered user
can access an album using the album handler.
The owner of an album can remove an album at any time. An album deletion does
not delete the photos that it contains, which must be explicitly removed from the system
if needed.
When a registered user uploads a new photo, he/she can associate it with some
descriptions related to the picture’s content. Descriptions can be changed and refined
afterward. A bulk upload functionality is also supported to easily insert photos having
common descriptions. For instance, a user can insert, with a single action, all photos
taken during last weekend in the mountains. In this case, all inserted photos are associ-
ated with the same common description.
In addition to descriptions created by the users, the system automatically analyzes
photos to extract additional metadata. Specifically, feature descriptors that enable sim-
ilarity search are automatically extracted. Similarity search [15] allows users to search
6 Authors Suppressed Due to Excessive Length
for pictures similar to pictures chosen as queries. This possibility can be particularly
useful to retrieve poorly described pictures. Consider that, as previously stated, the Pho-
toBook application is addressed to a non professional target. Therefore, imprecise, erro-
neous, incomplete, or completely missing descriptions may frequently occur. Similarity
search is an option for searching for photos of interest, which is really useful especially
in this non-professional context. With similarity search, a user can be able to retrieve,
for instance, pictures of the tour Eiffel by using another picture of the tour Eiffel as a
query, even if the retrieved pictures were not correctly annotated by their owner.
Metadata manually and automatically generated are represented by using standard
metadata schema. Specifically we have used MPEG-7 [9] metadata schema, given that
it is able to represent both low level feature descriptors, for similarity search, and more
conceptual descriptions. However, given that the PhotoBook application is intended to
be used by non professional users, the metadata complexity is hidden to them. This
implies that information is presented to the user in a natural and intuitive way and that
users are asked to insert a minimal amount of descriptions when uploading photos. A
relevant part of information is obtained automatically, and the system is able to satisfy
most of the user requirements even with incomplete, erroneous, or partial information.
For instance, in addition to image analysis to support similarity search, a lot of more
conceptual information can be obtained exploiting file names and folders names.
During photo management and photo searching activities, users see on the screen
the photo thumbnails. However, a user can download full size pictures, if needed.
3.2 PhotoBook Architecture
The architecture of the PhotoBook application is sketched in Figure 1. It is a classical
n-tier application, where the data layer, the application layer, the presentation layer, and
the client layer are responsible of different classes of activity of the application.
At the data layer we have MILOS, which is responsible for the management, access,
and retrieval, of all types of data of the application. Data managed by MILOS include
pictures uploaded by users, metadata, and user data.
At the application layer we have a set of components responsible of mediating the
interaction of the modules that implement the web user interface with MILOS. These
components actually implement the application’s logic, separately and independently
from the user interface logic and the data management logic. These components are
responsible of checking the credentials of users, organizing retrieved result on behalf
of the application logic, translating the user queries into correct queries to MILOS,
building valid metadata on the basis of the user input, implementing storage strategies
for the uploaded material (relying on the MILOS features), and generating additional
metadata by analyzing uploaded data. Interaction with MILOS is obtained using the
SOAP protocol, being MILOS a Web service. These components are also implemented
as web services.
The presentation layer contains the software modules in charge of drawing the user
interface on behalf of the client layer. PhotoBook is a web application: accordingly,
the interface layer is composed of Java Server Page (JSP) modules which dynamically
generate the web user interface. These modules interact with the application layer by
using SOAP.
Title Suppressed Due to Excessive Length 7
MILOS
Data Layer
Application Layer
Presentat. Layer
Home page
management
Search
management
User home
management
Upload
Management
Automatic
metadata
generation
Home page
Interface
Search
interface
User home
interface
Upload
Interface
Client Layer
Web browser
Automatic
metadata
generation
Bulk upload
tool
HTTP
SOAP
SOAP
Fig. 1. Architecture of the PhotoBook Application
Clients access PhotoBook by using a normal web browser. In addition at the client
side, users may optionally use a Java bulk-load component that allows them to upload
several pictures in the system, and to locally analyze them to automatically generate
additional metadata. The use of the bulk-load tool, moves part of the logic for metadata
generation to the client side, distributing the burden of image analysis among several
clients and improving upload performance of the system.
3.3 The PhotoBook user interface
A sketch of the main web interfaces is shown in Figure 2. The home page of the Pho-
toBook application is shown in Figure 2a. From the home page, every user (registered
and non-registered) can search for public material on the system, login for managing the
owned material, registering into the system. From the home page users can search for
pictures by using a fielded search or similarity search. In case of fielded search users can
search for pictures by expressing restrictions on the owner of the pictures, the location
where they were taken, their title, and on the textual description of the pictures. In case
of similarity search, the user can search by choosing a picture among those randomly
proposed by the system. Random pictures can be renewed on demand by the user.
Results are shown in the search page (Figure 2b). From there, users can refine their
queries by choosing a picture in the result to submit a new similarity search or to submit
a complex search query, which combines similarity and fielded search. For instance, a
user can search for images similar to the chosen one, whose location is Pisa.
Registered users that enter the system (Figure 2c), can also manage their own collec-
tion of pictures. Registered users can update metadata associated with pictures, organize
pictures in thematic albums, share albums with friends, and upload new pictures.
8 Authors Suppressed Due to Excessive Length
a) b)
c) d)
Fig. 2. The PhotoBook interface (http://milos.isti.cnr.it). a) The home page of the PhotoBook
application, b) The search interface of the PhotoBook application, c) The registered user page of
the PhotoBook application, d) The upload page of the PhotoBook application
There are two options for uploading new pictures. Registered users can use the
web interface (Figure 2d) or a bulk-load tool running on their computers. By using
the web interface users can upload up to two pictures at time, by specifying common
descriptions for both. In this case, automatic metadata extraction for similarity search
is executed on the PhotoBook server. By using the bulk-load tool, users can submit an
arbitrary amount of pictures from their hard disk. In this case, users can assign common
descriptions to the uploaded picture. In addition the bulk-load tool can refine these
description by using information extracted from the folder names and file names. Image
analysis for similarity search is also performed on the user computer. By using the
upload tool, the PhotoBook server has just to register the insertions, store the pictures,
and update the internal indexes.
Title Suppressed Due to Excessive Length 9
4 Picture management
MILOS allows different storage strategies to be used when dealing with different types
of media. The use of specific storage strategies is transparent to the application. To
obtain that, MILOS (the MMS component) identifies all documents with an URN and
maintains a mapping table to associate URN with actual storage locations and access
protocols. When an application requires to retrieve a document, it uses the URN to make
the request to the MMS, which returns the URL to be used to access the document.
In the PhotoBook application we decided to store pictures in a normal file system,
and to access them via a web server using the HTTP protocol. Pictures belonging to a
user are stored in a personal folder. Two different versions of every pictures are stored:
a thumbnail version and a full size version. Thumbnails and full version pictures are
stored in two different subfolder of the user’s folder.
An example of URN used in the PhotoBook is the following:
urn:milos:album:Paolo:image jpeg:7897056ecc55cd6f7cffa78413e4e2ac
which refers to a jpeg image of the user Paolo. When the PhotoBook application
wants to retrieve a picture, it uses the getURL(URN,version) method of MILOS to
obtain the URL to be used to retrieve it. When a request for the full size version of
previous URN is receioved, MILOS returns the following URL:
http://milos.isti.cnr.it/milos-MMS/MMS/album/ Paolo/big/image jpeg/
/image jpeg7897056ecc55cd6f7cffa78413e4e2ac.jpg
5 Metadata Management
MILOS is able to deal with any metadata schema. The only requirement is that meta-
data should be encoded in XML. No schema definition is needed to instruct MILOS to
deal with a specific metadata schema. Once an XML document is inserted in MILOS, it
is immediately available to be searched and accessed, using the advanced search func-
tionality of the native XML search engine embedded in MILOS.
This feature gave us a lot of freedom and flexibility in the choice of the metadata
to be used for the application and freed us from the difficulties of instructing a specific
search engine or database to deal with them.
We decided to use MPEG-7 metadata [9] for the image description. Figure 3a gives
an example of the MPEG-7 description of a picture. An MPEG-7 description contains
low level features to be used for similarity search, conceptual content descriptions, us-
age rights, creation time information, etc. Specifically, the <VisualDescriptor>
tags, in the figure, contain scalable color, color layout, color structure, edge histogram,
homogeneous texture information to be used for image similarity search. MILOS in-
dexes this tag with a special index to offer efficient similarity search. Conceptual infor-
mation is contained in the <Title>,<Abstract>, and <Location> tags. Right
management information is contained in tags <UsageInformation>. Creation time
information is maintained in <CreationTool> tag. Specifically, eXtended Image
File Format (XIFF) information is automatically extracted from pictures. This informa-
tion includes, for instance, parameters set in the camera when the photo was taken, the
10 Authors Suppressed Due to Excessive Length
<Mpeg7 schemaLocation="urn :mpeg:mpeg7:schema:2001 Mpeg7-2001.xsd" >
<Description type="ContentEntityType" >
<MultimediaContent type="Imag eType" >
<Image>
<MediaLocator>
<MediaUri>
urn:milos:album:paolo:imag e_jpeg:7897056ecc55cd6f7cffa78413e4e2ac
</MediaUri>
</MediaLocator>
<VisualDescriptor type="...." .... >....</VisualDescriptor>
<UsageInformation>
<Rights> <RightsID>paolo:public</Rig htsID> </Rights>
</UsageInformation>
<CreationInformation>
<Creation>
<Title>Tour in Umbria</Title>
<Abstract><FreeTextAnnotation>…</FreeTextAnnotation></Abstract>
<Creator><Role href="creatorCS" ><Name>creator</Name></Role>
<Agent type="PersonType" >
<Name><GivenNa me>paolo</GivenName></Name>
</Agent>
</Creator>
<CreationCoordinates>
<Location><Name>Perugia</Name><Region></Region>< /Location>
</CreationCoordinates>
<CreationTool>
….eXtended Image File Format (Xiff) data….
<Tool><Name>Canon Canon PowerShot A85</Name></Tool>
</CreationTool>
</Creation>
</CreationInformation>
</Image>
</MultimediaContent>
<DescriptionMetadata>
<CreationTime>Aug 5, 2005</CreationTime>
</DescriptionMetadata>
</Description>
</Mpeg7>
a)
<album>
<owner>gamato</owner>
<name>Roccella Jonica</name>
<id>fead7ff5333fa069401abe8bf2521d3d</id>
<urn>…..</urn>
<urn>…..</urn>
…..
</album>
b)
<login>
<username>gamato</username>
<password>sjadfe</password>
<email>gamato@interfree.it</email>
</login>
c)
Fig. 3. Metadata used in the PhotoBook application: a) Mpeg7 metadata to describe the pictures,
b) representation of albums (thematic collection pictures), c) user data
type of camera, etc. The MPEG-7 standard prescribes that metadata must be encoded
in XML. Therefore, its use was straightforward in MILOS.
Users can create thematic albums containing pictures related for instance to an
event. Albums are also encoded in XML as shown in Figure 3b. An album has basi-
cally a name, an owner, and a set of pictures included in the album.
User data are also very simple. Figure 3c shows how they are represented in Photo-
Book. A user has a username, a password, and an email address.
5.1 Automatic image processing
Feature extraction was performed employing an application we built upon the MPEG-7
experimentation model (XM, [11]) of MPEG-7 Part 6: Reference Software. The soft-
ware can extract all MPEG-7 image Visual Descriptors defined in [10]. For the Pho-
toBook we extract 5 MPEG-7 descriptors: ScalableColor (a color Histogram in the
HSV Color Space), ColorStructure (captures both color content and information about
the spatial arrangement of the colors), ColorLayout (represents the spatial layout of
color images), EdgeHistogram (spatial distribution of five types of edges), and Homo-
geneousTexture (characterizes the properties of texture in an image). For all thoose
descriptors the suggested distance functions [12] are metric. The result of the extrac-
Title Suppressed Due to Excessive Length 11
tion process is an XML document like the one in Figure 3 without usage and creation
information. The values inside the <VisualDescriptor> tags are integer vectors
(ScalavbleColor, ColorSTructure and EdgeHistogram) or more complicated XML sub-
trees with integers as values (ColorLayout and HomogeneousTexture).
6 Search capabilities
The MILOS native XML database/repository supports high performance search and re-
trieval on heavily structured XML documents, relying on specific index structures [3,
14], as well as full text search [13], automatic classification [8], and feature similarity
search [5]. This is compatible with current trends of the new generation of XML en-
coded metadata standards, such as MPEG-7, which include in their description also fea-
tures automatically extracted from visual documents, such as color histograms, textures,
shapes, etc. Specifically, the MILOS XML database allows the system administrator to
associate specific XML element names with special indexes. Therefore, for instance,
the tag name <abstract> can be associated with a full text index. On the other hand,
the MPEG-7 <VisualDescriptor> tag can be associated with a similarity search
index structure.
These features of the MILOS system were very useful to provide users of Photo-
Book with advanced search functionalities and to provide developers with all needed
functionality for picture description, album, and user management.
Specifically the PhotoBook application allows users to use similarity search, by
exploiting the <VisualDescriptor> tags included in the MPEG-7 metadata. Users
can also submit full-text queries, by using the full-text descriptions included in the
<abstract> tags. In addition, users can perform a search by expressing queries that
use any tag content of the MPEG-7 metadata.
Users can also express complex queries, where full-text, fielded, and similarity
search is conveniently combined.
7 Conclusions
This paper illustrates the main characteristics, architecture and design choices adopted
in the PhotoBook Digital Library application, which supports archiving, indexing, shar-
ing and content-based search of photos. PhotoBook was built by using a general purpose
Multimedia Content Management System, MILOS, which is specifically designed to
create high performance Digital Library applications. By using MILOS we had several
advantages: (a) the development of the entire application was realized with a limited ef-
fort – approximately one month of work of an experienced programmer, that developed
the user interface of the application, (b) powerful and efficient content-based search
capabilities have been included, (c) flexible storage management is possible – for ex-
ample, if the size of the archive will increase in the future, the storage strategies can
change without any modification to the application and transparently for end-users, (d)
integration with other similar archives, based on different metadata formats, or archiv-
ing of photos represented in metadata formats different from that used in PhotoBook,
can be easily obtained.
12 Authors Suppressed Due to Excessive Length
Future research efforts will be spent in extending some MILOS functionality: the
development of the PhotoBook application has shown that tools to be used for the devel-
opment of the application user interface would greatly reduce application development
costs; on the other side, by improving the automatic semantic analysis of multimedia
content, it should be possible to support more powerful content-based searches.
The extended MILOS functionality will then be used to provide a new version of the
PhotoBook application, with the extended classification, indexing and search capabili-
ties. Finally, it is our intention to continue the test of the MILOS system by developing
other Digital Library applications.
References
1. MILOS: Multimedia dIgital Library for On-line Search. http://milos.isti.cnr.it/.
2. G. Amato and F. Debole. A native xml database supporting approximate match search. In
Europeean Conference on Digital Libraries, ECDL 2005, Vienna, AT, September 18-23 2005,
2005.
3. G. Amato, F. Debole, F. Rabitti, and P. Zezula. YAPI: Yet another path index for XML
searching. In ECDL 2003, 7th European Conference on Research and Advanced Technology
for Digital Libraries, Trondheim, Norway, August 17-22, 2003, 2003.
4. G. Amato, C. Gennaro, F. Rabitti, and P. Savino. Milos: A multimedia content management
system for digital library applications. In Europeean Conference on Digital Libraries, ECDL
2004, Bath, UK, September 12-17 2004, 2004.
5. C. B¨
ohm, S. Berchtold, and D. Keim. Searching in high-dimensional spaces: Index struc-
tures for improving the performance of multimedia databases. ACM Computing Surveys,
33(3):322–373, September 2001.
6. W. W. W. Consortium. XML path language (XPath), version 1.0, W3C. Recommendation,
November 1999.
7. W. W. W. Consortium. XQuery 1.0: An XML query language. W3C Working Draft, Novem-
ber 2002. http://www.w3.org/TR/xquery.
8. N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambidge
University Press, 2000.
9. ISO/IEC. Information technology - Multimedia content description interfaces. 15938.
10. ISO/IEC. Information technology - Multimedia content description interfaces. Part 3: Visual.
15938-3:2002.
11. ISO/IEC. Information technology - Multimedia content description interfaces. Part 6: Ref-
erence Software. 15938-6:2003.
12. P. Salembier and T. Sikora. Introduction to MPEG-7: Multimedia Content Description In-
terface. John Wiley & Sons, Inc., New York, NY, USA, 2002.
13. G. Salton and M. J. McGill. Introduction to Modern Information Retrieval. McGraw-Hill
Book Company, 1983.
14. P. Zezula, G. Amato, F. Debole, and F. Rabitti. Tree signatures for xml querying and navi-
gation. In Database and XML Technologies, First International XML Database Symposium,
XSym 2003, volume 2824 of LNCS, pages 149–163. Springer, 2003.
15. P. Zezula, G. Amato, V. Dohnal, and M. Batko. Similarity Search - The Metric Space Ap-
proach, volume 32 of Advances in Database Systems. Springer, 2006.
... DL-MEDIA supports concrete domains with specific predicates on it. The concrete predicates that DL-MEDIA allows are not only relational predicates such as ([i] ≤ 1500) (e.g. the value of the i-th column of an nary relation is less or equal than 1500), but also similarity predicates such as ([i] simT xt logic, image, retrieval ), which given a piece of text x appearing in the i-th column of a tuple returns the system's degree (in [0, 1]) of being x about the keywords 'logic, image, retrieval'. Formally, a concrete domain in DL-MEDIA is a pair ∆ D , Φ D , where ∆ D is an interpretation domain and Φ D is the set of domain predicates d with a predefined arity n and an interpretation d D : ∆ n D → [0, 1] (see also [10]). ...
... The concrete predicates that DL-MEDIA allows are not only relational predicates such as ([i] ≤ 1500) (e.g. the value of the i-th column of an nary relation is less or equal than 1500), but also similarity predicates such as ([i] simT xt logic, image, retrieval ), which given a piece of text x appearing in the i-th column of a tuple returns the system's degree (in [0, 1]) of being x about the keywords 'logic, image, retrieval'. Formally, a concrete domain in DL-MEDIA is a pair ∆ D , Φ D , where ∆ D is an interpretation domain and Φ D is the set of domain predicates d with a predefined arity n and an interpretation d D : ∆ n D → [0, 1] (see also [10]). The list of the specific domain predicates is presented below. ...
... Hence, e.g. ∃[1, 3]MyMetadata is the binary relation that is the projection on the first and third column of the MyMetadata relation. As a consequence, the axioms ...
Conference Paper
We outline DL-Media, an ontology mediated multimedia information retrieval system, which combines logic-based retrieval with multimedia feature-based similarity retrieval. An ontology layer is used to define (in terms of a fuzzy description logic) the relevant abstract concepts and relations of the application domain, while a content-based multimedia retrieval system is used for feature-based retrieval.
... For multifeature indexing SAPIR makes uses of the MUFIN (Multifeature Indexing Network) which is built over the MESSIF 2 (Metric Similarity Search Implementation Framework) archi- tecture [9]. The web user interface used for this demo is derived from the one we developed for a photo album application built upon MILOS [10] (a centralized Multimedia Content Management System). ...
Conference Paper
Full-text available
In this paper we present a scalable and distributed system for image retrieval based on visual features and annotated text. This system is the core of the SAPIR project. Its architecture makes use of Peer-to-Peer networks to achieve scalability and efficiency allowing the management of huge amount of data. For the presented demo we use 10 million images and accom-panying text (tags, comments, etc.) taken from Flickr. Through the web interface it is possible to efficient perform content-based similarity search, as well as traditional text search on the metadata annotated by the Flickr community. Fast complex query processing is also possible combining visual features and text. We show that the combination of content-based and text search on a large scale can dramatically improve the capability of a multimedia search system to answer the users needs and that the Peer-to-Peer based architecture can cope with the scalability issues (response time obtained for this demo over 10 million images is always below 500 milliseconds).
Article
Purpose – The purpose of this paper is to provide a tutorial and survey on recent advances in multimedia networking from an integrated perspective of both video networking and building digital video libraries. The nature of video networking, coupled with various recent developments in standards, proposals and applications, poses great challenges to the research and industrial communities working in this area. Design/methodology/approach – This paper presents an insightful analysis for recent and emerging multimedia applications in digital video libraries and on video coding standards and their applications in digital libraries. Emphasis is given on those standards and mechanisms that enable multimedia content adaptation fully interoperable according to the vision of Universal Multimedia Access vision. Findings – The tutorial helps elucidate the similarities and differences among the considered standards and networking applications. A number of research trends and challenges are identified, and selected promising solutions are discussed. This practice would needle further thoughts on the development of this area and open-up more research and application opportunities. Research limitations/implications – The paper does not provide methodical studies of networking application scenarios for all the discussed video coding standards and Quality of Service (QoS) management mechanisms. Practical implications – The paper provides an overview of which technologies/mechanisms are being used broadly in networking scenarios of digital video libraries. The discussed networking scenarios bring together video coding standards and various emerging wireless networking paradigms toward innovative application scenarios. Originality/value – QoS mechanisms and video coding standards that support multimedia applications for digital video libraries need to become well-known by library managers and professional associations in the fields of libraries and archives. The comprehensive overview and critiques on existing standards and application approaches offer a valuable reference for researchers and system developers in related research and industrial communities.
Article
Personal digital library(PDL) plays an important role in learning, working and daily life. The paper proposes a model of PDL according to its functions. The model includes data layer, service layer, assistant layer. The PDL can be interactive with public library or digital repository. After analyzing current tools suitable for building personal digital library and their deficiency in details, the author explores some work that should be done in the future to make personal digital library work well.
Conference Paper
We outline DL-Media, an ontology mediated multimedia information retrieval system, which combines logic-based retrieval with multimedia feature- based similarity retrieval. An ontology layer is used to define (in terms of a fuzzy DLR-Lite like description logic) the relevant abstract concepts and relations of the application domain, while a content-based multimedia retrieval system is used for feature-based retrieval. We will illustrate its logical model, its architecture, its representation and query language and the preliminary experiments we conducted.
Conference Paper
Full-text available
We outline DLMedia, an ontology mediated multimedia information retrieval system, which combines logic-based retrieval with multimedia featurebased similarity retrieval. An ontology layer may be used to define (in terms of a DLR-Lite like description logic) the relevant abstract concepts and relations of the application domain, while a content-based multimedia retrieval system is used for feature-based retrieval.
Conference Paper
Full-text available
In this paper we present a scalable and distributed access structure for similarity search in metric spaces. The approach is based on the Content- addressable Network (CAN) paradigm, which provides a Distributed Hash Table (DHT) abstraction over a Cartesian space. We have extended the CAN structure to support storage and retrieval of more generic metric space objects. We use pivots for projecting objects of the metric space in an N -dimensional vector space, and exploit the CAN organization for distributing the objects among computer nodes of the structure. We obtain a Peer-to-Peer network, called the MCAN, which is able to search metric space objects by means of the similarity range queries. Experiments conducted on our prototype system confirm full scalability of the approach.
Conference Paper
Full-text available
XML is becoming the standard representation format for metadata. Metadata for multimedia documents, as for instance MPEG-7, require approximate match search functionalities to be supported in addition to exact match search. As an example, consider image search performed by using MPEG-7 visual descriptors. It does not make sense to search for images that are exactly equal to a query image. Rather, images similar to a query image are more likely to be searched. We present the architecture of an XML search engine where special techniques are used to integrate approximate and exact match search functionalities.
Conference Paper
Full-text available
In order to accelerate execution of various matching and navigation operations on collections of XML documents, new indexing structure, based on tree signatures, is proposed. We show that XML tree structures can be efficiently represented as ordered sequences of preorder and postorder ranks, on which extended string matching techniques can easily solve the tree matching problem. We also show how to apply tree signatures in query processing and demonstrate that a speedup of up to one order of magnitude can be achieved over the containment join strat- egy. Other alternatives of using the tree signatures in intelligent XML searching are outlined in the conclusions.
Conference Paper
Full-text available
As many metadata are encoded in XML, and many digital libraries need to manage XML documents, efficient techniques for search- ing in such formatted data are required. In order to efficiently process path expressions with wildcards on XML data, a new path index is pro- posed. Extensive evaluation confirms better performance with respect to other techniques proposed in the literature. An extension of the pro- posed technique to deal with the content of XML documents in addition to their structure is also discussed.
Conference Paper
Full-text available
(Extended abstract) Abstract. This paper describes the architecture of the MILOS Content Management System. MILOS supports the storage and content based retrieval of any XML document, as well as multimedia documents whose descriptions are provided by using heterogenous metadata models represented in XML. MILOS is flexible in the management of documents containing different types of data and content descriptions; it is efficient and scalable in the storage and content based retrieval of these documents. The paper illustrates the solutions adopted to support the management of different metadata descriptions of multimedia documents in the same repository, and it illustrates the experiments performed by using the MILOS system to archive documents belonging to three different and heterogenous collections which contain news agencies, scientific papers, and audiovideo documentaries. 1
Book
Full-text available
In the Information Society, information holds the master key to economic influence. Similarity Search: The Metric Space Approach will focus on efficient ways to locate user-relevant information in collections of objects, the similarity of which is quantified using a pairwise distance measure. This book is a direct response to recent advances in computing, communications and storage which have led to the current flood of digital libraries, data warehouses and the limitless heterogeneity of internet resources. Similarity Search: The Metric Space Approach will introduce state-of-the-art in developing index structures for searching complex data modeled as instances of a metric space. This book consists of two parts. Part 1 presents the metric search approach in a nutshell by defining the problem, describes major theoretical principals, and provides an extensive survey of specific techniques for a large range of applications. Part 2 concentrates on approaches particularly designed for searching in very large collections of data. Similarity Search: The Metric Space Approach is designed for a professional audience, composed of academic researchers as well as practitioners in industry. This book is also suitable as introductory material for graduate-level students in computer science.
Article
Full-text available
During the last decade, multimedia databases have become increasingly important in many application areas such as medicine, CAD, geography, or molecular biology. An important research issue in the field of multimedia databases is the content based retrieval of similar multimedia objects such as images, text, and videos. However, in contrast to searching data in a relational database, a content based retrieval requires the search of similar objects as a basic functionality of the database system. Most of the approaches addressing similarity search use a so-called feature transformation which transforms important properties of the multimedia objects into high-dimensional points (feature vectors). Thus, the similarity search is transformed into a search of points in the feature space which are close to a given query point in the high-dimensional feature space. Query Processing in high-dimensional spaces has therefore been a very active research area over the last few years. A number of new index structures and algorithms have been proposed. It has been shown that the new index structures considerably improve the performance in querying large multimedia databases. Based on recent tutorials [BK 98, BK 00], in this survey we provide an overview of the current state-of-the-art in querying multimedia databases, describing the index structures and algorithms for an efficient query processing in high-dimensional spaces. We identify the problems of processing queries in high-dimensional space, and we provide an overview of the proposed approaches to overcome these problems.
Article
This book is the first comprehensive introduction to Support Vector Machines (SVMs), a new generation learning system based on recent advances in statistical learning theory. The book also introduces Bayesian analysis of learning and relates SVMs to Gaussian Processes and other kernel based learning methods. SVMs deliver state-of-the-art performance in real-world applications such as text categorisation, hand-written character recognition, image classification, biosequences analysis, etc. Their first introduction in the early 1990s lead to a recent explosion of applications and deepening theoretical analysis, that has now established Support Vector Machines along with neural networks as one of the standard tools for machine learning and data mining. Students will find the book both stimulating and accessible, while practitioners will be guided smoothly through the material required for a good grasp of the theory and application of these techniques. The concepts are introduced gradually in accessible and self-contained stages, though in each stage the presentation is rigorous and thorough. Pointers to relevant literature and web sites containing software ensure that it forms an ideal starting point for further study. Equally the book will equip the practitioner to apply the techniques and an associated web site will provide pointers to updated literature, new applications, and on-line software.