Conference Paper

A Unified Model for Socially Interconnected Multimedia-Enriched Objects

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Enabling effective multimedia information processing, analysis, and access applications in online social multimedia settings requires data representation models that capture a broad range of the characteristics of such environments and ensure interoperability. We propose a flexible model for describing Socially Interconnected MultiMedia-enriched Objects (SIMMO) that integrates in a unified manner the representation of multimedia and social features in online environments. Its specification is based on a set of identified requirements and its expressive power is illustrated using several diverse examples. Finally, a comparison of SIMMO with existing approaches demonstrates its unique features.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... The design of the TENSOR ontology is based to some extent on established existing models. We rely on SIMMO, a model for describing socially interconnected multimedia-enriched objects that integrates in a unified manner the representation of multimedia and social features in online environments [27]. Since SIMMO is not available as an ontology, we simply adapted its key constructs and properties into the TENSOR ontology. ...
Conference Paper
The Web and social media nowadays play an increasingly significant role in spreading terrorism-related propaganda and content. In order to deploy counterterrorism measures, authorities rely on automated systems for analysing text, multimedia, and social media content on the Web. However, since each of these systems is an isolated solution, investigators often face the challenge of having to cope with a diverse array of heterogeneous sources and formats that generate vast volumes of data. Semantic Web technologies can alleviate this problem by delivering a toolset of mechanisms for knowledge representation, information fusion, semantic search, and sophisticated analyses of terrorist networks and spatiotemporal information. In the Semantic Web environment, ontologies play a key role by offering a shared, uniform model for semantically integrating information from multimodal heterogeneous sources. An additional benefit is that ontologies can be augmented with powerful tools for semantic enrichment and reasoning. This paper presents such a unified semantic infrastructure for information fusion of terrorism-related content and threat detection on the Web. The framework is deployed within the TENSOR EU-funded project, and consists of an ontology and an adaptable semantic reasoning mechanism. We strongly believe that, in the short- and long-term, these techniques can greatly assist Law Enforcement Agencies in their investigational operations.
... time stamp, tags, source, position in a social graph, etc.) [3]. All these sources of information formulate a multimedia item (multimodal object), such as the example webpage in Fig. 1, and access to several modalities is possible through efficient multimedia representation and indexing techniques, such as [26]. ...
Article
Full-text available
Heterogeneous sources of information, such as images, videos, text and metadata are often used to describe different or complementary views of the same multimedia object, especially in the online news domain and in large annotated image collections. The retrieval of multimedia objects, given a multimodal query, requires the combination of several sources of information in an efficient and scalable way. Towards this direction, we provide a novel unsupervised framework for multimodal fusion of visual and textual similarities, which are based on visual features, visual concepts and textual metadata, integrating non-linear graph-based fusion and Partial Least Squares Regression. The fusion strategy is based on the construction of a multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. Our framework can employ more than two modalities and high-level information, without increase in memory complexity, when compared to state-of-the-art baseline methods. The experimental comparison is done in three public multimedia collections in the multimedia retrieval task. The results have shown that the proposed method outperforms the baseline methods, in terms of Mean Average Precision and Precision@20.
... The objective of this component is to achieve integration and retrieval of content along different dimensions. 1) Multimodal indexing and retrieval: In this module, a multimedia data representation framework that allows for the efficient storage and retrieval of socially connected multimedia objects is developed. The representation model is called SIMMO (Socially Interconnected MultiMediaenriched Objects) [3] and has the ability to fully capture all the content information of interconnected multimedia objects, while at the same time avoiding the complexity of previously proposed models. ...
Conference Paper
Full-text available
This paper presents an overview and the first results of the FP7 MULTISENSOR project, which deals with multidimensional content integration of multimedia content for intelligent sentiment enriched and context oriented interpretation. MULTISENSOR aims at providing unified access to multilingual, multimedia and multicultural economic, news story material across borders in order to support journalism and media monitoring tasks and provide decision support for internationalisation of companies.
... The module includes the following submodules: (i) crawling and scraping to extract relevant content from web resources, for which we use Apache Nutch 2 and boilerpipe 3 , and social media; (ii) content extraction that distills content elements from web resources and social media (using MetaMap, 4 Babelfy, 5 and Unitex 6 ); (iii) social media topic detection that detects health-related topics based on clustering; (iv) indexing, search, and retrieval of multimodal information from web [42]; (v) domain-specific query formulation for querying the indexed repository and triggering the ontology population mechanism; and (vi) user interaction query formulation that copes with the derivation of knowledgedriven interpretations from users' input in order to compile the appropriate reactions. ...
Conference Paper
Full-text available
We present work in progress on an intelligent embodied conversation agent in the basic care and healthcare domain. In contrast to most of the existing agents, the presented agent is aimed to have linguistic cultural, social and emotional competence needed to interact with elderly and migrants. It is composed of an ontology-based and reasoning-driven dialogue manager, multimodal communication analysis and generation modules and a search engine for the retrieval of multimedia background content from the web needed for conducting a conversation on a given topic.
... All sources of information need to be indexed smartly, in order to ensure fast access to the modalities of a multimedia object. Recently an efficient representation and indexing technique for multimodal objects [2], named Socially Interconnected MultiMedia-enriched Objects (SIMMO) has been proposed. For the multimedia retrieval task, the key problem is to combine all available information in order to retrieve multimodal documents, relevant to a given multimodal query. ...
... The module includes the following submodules: (i) crawling and scraping to extract relevant content from web resources, for which we use Apache Nutch 2 and boilerpipe 3 , and social media; (ii) content extraction that distills content elements from web resources and social media (using MetaMap, 4 Babelfy, 5 and Unitex 6 ); (iii) social media topic detection that detects health-related topics based on clustering; (iv) indexing, search, and retrieval of multimodal information from web [42]; (v) domain-specific query formulation for querying the indexed repository and triggering the ontology population mechanism; and (vi) user interaction query formulation that copes with the derivation of knowledgedriven interpretations from users' input in order to compile the appropriate reactions. ...
Conference Paper
Full-text available
We present work in progress on an intelligent embodied conversation agent in the basic care and healthcare domain. In contrast to most of the existing agents, the presented agent is aimed to have linguistic cultural, social and emotional competence needed to interact with elderly and migrants. It is composed of an ontology-based and reasoning-driven dialogue manager, multimodal communication analysis and generation modules and a search engine for the retrieval of multimedia background content from the web needed for conducting a conversation on a given topic.
Article
The backbone of the semantic web is ontology, dealing with the context of details associated with a specific domain. Domain ontology (DO) is an important source of information for knowledge-based systems. Nonetheless, developing DO is a time-consuming procedure that is heavily reliant on the developer's expertise. Here, a novel semi-automated technique for creating Ontologies in the terrorism domain is suggested. Terrorism actions provide critical information that can be used to improve a country's security system. To obtain the most up-to-date knowledge of the domain, online social network (OSN) data, specifically Twitter text data, is retrieved, and then concepts and associated relationships are recognized and mapped through formal concept analysis (FCA). The fluent editor tool (FET) displays a number of user-defined associations. Knowledge is also extracted using a query-based approach and a reasoner window in the FET. The created DO is broadcast on the web using an ontology web language (OWL) that may be used in a variety of other applications. The suggested work is notable because it creates broad-coverage DO for the terrorist domain using a tool called Fluent Editor (FE) instead of the typical tool (protégé), and semantic information is retrieved with 100% correctness, similar to a query-based search system (QBS).
Article
Full-text available
Social multimedia can be leveraged for a wide range of applications, but mining and search systems require innovative crawling solutions to meet both technical and policy-related obstacles.
Article
Full-text available
MPEG-21 is an open standards-based framework for multimedia delivery and consumption. It aims to enable the use of multimedia resources across a wide range of networks and devices. We discuss MPEG-21's parts, achievements, ongoing activities, and opportunities for new technologies.
Article
Full-text available
In this paper, a novel framework for description of rich media content is introduced. Firstly, the concept of 'content objects' is provided. Content objects are rich media presentations, enclosing different types of media, along with real-world information and user-related information. These highly complex presentations require a suitable description scheme in order to be searched and retrieved by end users. Therefore, a novel rich unified content description is analysed, which provides a uniform descriptor for all types of content objects irrespective of the underlying media and accompanying information.
Article
Full-text available
In this paper, we introduce the EADS' WebLab platform (http://weblab-project.org) that aims at providing an integration infrastructure for multimedia information processing components. In the following, we explain the motivations that have led to the realisation of this project within EADS and the requirements that have led our choices. After a quick review of existing information processing platforms, we present the chosen service oriented architecture, and the three layers of the WebLab project (infrastructure, services and applications). Then, we detail the chosen exchange model and normalised services interfaces that enable semantic interoperability between information processing components. We present the technical choices made to guarantee technical interoperability between the components by the use of an Enterprise Service Bus (ESB). Moreover, we present the orchestration and portal mechanisms that we have added to the WebLab to enable architects to quickly build multimedia processing applications. In the following, we illustrate the integration process by describing three applications that have been developed on top of this architecture on three R&D projects (Vitalas, WebContent and eWok-Hub). Finally, we propose some perspectives such as the realisation of an information processing services directory, or a toolkit following MDA (Model Driven Architecture) approach to ease the integration process.
Article
Full-text available
The development of Social Websites that help users in creating and gathering knowledge, are discussed. These Social Websites help users in creating and gathering knowledge, by simplifying user contributions through blogs, tagging, folksonomies, wikis, podcasts, and online social networks. These Social Websites have also enabled community-based knowledge acquisition, with efforts, such as Wikipedia, which demonstrate the idea of users in creating the world's largest online encyclopedia. The Semantic Web aims to provide the tools that are needed to define extensible and flexible standards for information exchange and interoperability on the Social Websites. The application of Semantic Web frameworks, such as Semantically Interlinked Online Communities (SIOC) and Friend-of-a-friend (FOAF) to the Semantic Web can lead to a Social Semantic Web, creating a network of interlinked and semantically significant knowledge.
Book
Social media is now ubiquitous on the internet, generating both new possibilities and new challenges in information analysis and retrieval. This comprehensive text/reference examines in depth the synergy between multimedia content analysis, personalization, and next-generation networking. The book demonstrates how this integration can result in robust, personalized services that provide users with an improved multimedia-centric quality of experience. Each chapter offers a practical step-by-step walkthrough for a variety of concepts, components and technologies relating to the development of applications and services. Topics and features: • Provides contributions from an international and interdisciplinary selection of experts in their fields • Introduces the fundamentals of social media retrieval, presenting the most important areas of research in this domain • Examines the important topic of multimedia tagging in social environments, including geo-tagging • Discusses issues of personalization and privacy in social media • Reviews advances in encoding, compression and network architectures for the exchange of social media information • Describes a range of applications related to social media Researchers and students interested in social media retrieval will find this book a valuable resource, covering a broad overview of state-of-the-art research and emerging trends in this area. The text will also be of use to practicing engineers involved in envisioning and building innovative social media applications and services.
Article
MPEG-7, formally known as the Multimedia Content Description Interface, includes standardized tools (descriptors, description schemes, and language) enabling structural, detailed descriptions of audio-visual information at different granularity levels (region, image, video segment, collection) and in different areas (content description, management, organization, navigation, and user interaction). It aims to support and facilitate a wide range of applications, such as media portals, content broadcasting, and ubiquitous multimedia. We present a high-level overview of the MPEG-7 standard. We first discuss the scope, basic terminology, and potential applications. Next, we discuss the constituent components. Then, we compare the relationship with other standards to highlight its capabilities