A rule-based video annotation system

Dept. of Electron. Eng., Univ. of London, UK
IEEE Transactions on Circuits and Systems for Video Technology (Impact Factor: 1.82). 06/2004; DOI: 10.1109/TCSVT.2004.826764
Source: IEEE Xplore

ABSTRACT A generic system for automatic annotation of videos is introduced. The proposed approach is based on the premise that the rules needed to infer a set of high-level concepts from low-level descriptors cannot be defined a priori. Rather, knowledge embedded in the database and interaction with an expert user is exploited to enable system learning. Underpinning the system at the implementation level is preannotated data that dynamically creates signification links between a set of low-level features extracted directly from the video dataset and high-level semantic concepts defined in the lexicon. The lexicon may consist of words, icons, or any set of symbols that convey the meaning to the user. Thus, the lexicon is contingent on the user, application, time, and the entire context of the annotation process. The main system modules use fuzzy logic and rule mining techniques to approximate human-like reasoning. A rule-knowledge base is created on a small sample selected by the expert user during the learning phase. Using this rule-knowledge base, the system automatically assigns keywords from the lexicon to nonannotated video clips in the database. Using common low-level video representations, the system performance was assessed on a database containing hundreds of broadcasting videos. The experimental evaluation showed robust and high annotation accuracy. The system architecture offers straightforward expansion to relevance feedback and autonomous learning capabilities.

  • [Show abstract] [Hide abstract]
    ABSTRACT: As surveillance cameras become widespread, filters are needed to process large streams of image data and spot interesting events. Programmed image filters generally result in low to medium performing solutions. Data-derived filters perform better in that they tap on selected image features, but require a per-sensor effort by an analyst or a machine learning expert. This contribution addresses filter shaping as a data-driven process that is ‘placed in the hands of many end-users’ with extensive domain knowledge but no expertise in machine learning. The focus is on interactive machine learning technologies as a means to achieve self programming and specialization of image filters that learn to search images by their content, sequential order, and temporal attributes. We describe and assess the performance of two interactive algorithms designed and implemented for a real case study in process monitoring for nuclear safeguards. Experiments show that interactive machine learning helps detect safeguards relevant event while significantly reducing the number of false positives. KeywordsInteractive machine learning–Hidden Markov models–Decision trees
    04/2011: pages 289-325;
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Information summarization and retrieval are significant research topics associated with recent advancements in sensor devices, data compression and storage techniques, and high-speed internet. As a result of these advances, it is possible for people to collect huge life-logs. Video is one of the most important life information sources. This paper describes a method of summarizing video life-logs in an office environment with a multi-camera system. Previously, multi-camera systems have been used to track moving objects or to cover a wide area. This paper focuses on capturing diverse views of each office event using a multi-camera system with several cameras observing the same area. The summarization process includes camera view selection and event sequence summarization. View selection produces a single event sequence from multiple event sequences by selecting an optimal view at each time, for which domain knowledge based on the elements of the office environment and rules from questionnaire surveys have been used. Summarization creates a summary sequence from whole sequences by using a fuzzy rule-based system to approximate human decision making. The user-entered degrees of interest in objects, persons, and events are used for a personalized summarization. We confirmed experimentally that the proposed method provides promising results.
    Inf. Syst. 01/2011; 36:1124-1134.
  • [Show abstract] [Hide abstract]
    ABSTRACT: Rheumatoid arthritis (RA) is a complex polygenic inflammatory disease associated with accelerated atherosclerosis. IL-6 is a key mediator of inflammation in RA. A recent study showed an association between IL6-174 G/C gene polymorphism and cardiovascular (CV) disease in UK individuals with RA. To confirm this association we assessed the influence of three IL6 gene polymorphisms in the risk of CV disease in a large series of patients with RA. We studied 1250 Spanish patients with RA. Besides genotyping the traditional single nucleotide polymorphism (SNP) promoter -174G/C (rs1800795), we assessed another two SNPs (rs2069827 and rs2069840) located in the IL6 gene that were selected by SNP-tagging. Two-hundred and twenty (17.6%) of the 1250 patients experienced CV events. No significant differences in the genotype, allele and haplotype frequencies between RA patients with and without CV events were observed. Our results do not confirm in a Spanish population the association of IL6 gene with CV disease in RA previously reported in the UK.
    Atherosclerosis 08/2011; 219(2):655-8. · 3.71 Impact Factor

Full-text (2 Sources)

Available from