A rule-based video annotation system

Dept. of Electron. Eng., Univ. of London, UK
IEEE Transactions on Circuits and Systems for Video Technology (Impact Factor: 2.26). 06/2004; DOI: 10.1109/TCSVT.2004.826764
Source: IEEE Xplore

ABSTRACT A generic system for automatic annotation of videos is introduced. The proposed approach is based on the premise that the rules needed to infer a set of high-level concepts from low-level descriptors cannot be defined a priori. Rather, knowledge embedded in the database and interaction with an expert user is exploited to enable system learning. Underpinning the system at the implementation level is preannotated data that dynamically creates signification links between a set of low-level features extracted directly from the video dataset and high-level semantic concepts defined in the lexicon. The lexicon may consist of words, icons, or any set of symbols that convey the meaning to the user. Thus, the lexicon is contingent on the user, application, time, and the entire context of the annotation process. The main system modules use fuzzy logic and rule mining techniques to approximate human-like reasoning. A rule-knowledge base is created on a small sample selected by the expert user during the learning phase. Using this rule-knowledge base, the system automatically assigns keywords from the lexicon to nonannotated video clips in the database. Using common low-level video representations, the system performance was assessed on a database containing hundreds of broadcasting videos. The experimental evaluation showed robust and high annotation accuracy. The system architecture offers straightforward expansion to relevance feedback and autonomous learning capabilities.

  • [Show abstract] [Hide abstract]
    ABSTRACT: The amount of the internet video has been growing rapidly in recent years. Efficient video indexing and retrieval, therefore, is becoming an important research and system-design issue. Reliably extracting metadata from video as indexes is one major step toward efficient video management. There are numerous video types, and everyone can define new video types of his own. We believe an open video analysis framework should help when one needs to automatically process various types of videos. More, the nature of video can be so different that we may end up having a dedicated video analysis module for each video type. It is infeasible to design a system to automatically process every type of video. In the paper, we propose a scene-based video analytic studio that comes with (1) an open video analysis framework where the video analysis modules are developed and deployed as plug-ins, (2) an authoring tool where videos can be manually tagged, and (3) an HTML5-based video player the play backs videos using the metadata we generate. In addition, it provides a runtime environment with standard libraries and proprietary rule-based automaton modules to facilitate the plug-in development. At the end, we will show its application to click able (shoppable) videos, which we plan to apply to our e-learning projects.
    2014 IEEE International Conference on Semantic Computing (ICSC); 06/2014
  • [Show abstract] [Hide abstract]
    ABSTRACT: The dramatic growth of video content over modern media channels (such as the Internet and mobile phone platforms) directs the interest of media broadcasters towards the topics of video retrieval and content browsing. Several video retrieval systems benefit from the use of semantic indexing based on content, since it allows an intuitive categorization of videos. However, indexing is usually performed through manual annotation, thus introducing potential problems such as ambiguity, lack of information, and non-relevance of index terms. In this paper, we present SHIATSU, a complete system for video retrieval which is based on the (semi-)automatic hierarchical semantic annotation of videos exploiting the analysis of visual content; videos can then be searched by means of attached tags and/or visual features. We experimentally evaluate the performance of SHIATSU on two different real video benchmarks, proving its accuracy and efficiency.
    Multimedia Tools and Applications 03/2013; 63(2). · 1.06 Impact Factor
  • Source

Preview (2 Sources)

Available from