Conference Paper

Real-Time Foreground Segmentation for the Moving Camera Based on H.264 Video Coding Information

Nat. Central Univ., Taoyuan
DOI: 10.1109/FGCN.2007.191 Conference: Future generation communication and networking (fgcn 2007), Volume: 1
Source: IEEE Xplore


Foreground segmentation for video frames has played an important role in many video applications, such as video surveillance, video indexing, etc. Due to most videos are compressed, foreground segmentation can benefit from utilizing such coding information and save much processing time. In this paper, we propose a real-time foreground segmentation algorithm for the moving camera based on the H.264 video coding information. In the proposed algorithm, we first utilize the relative global motion model to calculate the approximate global motion vector and get the motion vector difference of each block. Then, according to the block partition modes, we assign different weightings and apply spatio-temporal refinement to these motion vector differences for further improving the accuracy of segmentation results. Finally, we segment out the foreground blocks by an adaptive threshold. With the aid of H.264 video coding information, the proposed segmentation algorithm is more practical than many other methods based on spatial domain information in computational complexity.

Full-text preview

Available from:
    • "The algorithm suits only for segmenting videos captured from the fixed cameras. Hong et al.[33]processed the MVs to estimate the global motion. Due to this, their approach was capable of handling the case of moving camera as well. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Image and video analysis requires rich features that can characterize various aspects of visual information. These rich features are typically extracted from the pixel values of the images and videos, which require huge amount of computation and seldom useful for real-time analysis. On the contrary, the compressed domain analysis offers relevant information pertaining to the visual content in the form of transform coefficients, motion vectors, quantization steps, coded block patterns with minimal computational burden. The quantum of work done in compressed domain is relatively much less compared to pixel domain. This paper aims to survey various video analysis efforts published during the last decade across the spectrum of video compression standards. In this survey, we have included only the analysis part, excluding the processing aspect of compressed domain. This analysis spans through various computer vision applications such as moving object segmentation, human action recognition, indexing, retrieval, face detection, video classification and object tracking in compressed videos.
    No preview · Article · Nov 2014 · Multimedia Tools and Applications
  • [Show abstract] [Hide abstract]
    ABSTRACT: A new algorithm to distinguish the foreground from compressed videos is proposed in this paper. Local motion, which is estimated from the residual between the original motion and the global motion, is one of the strongest influences on visual attention. Global motion is modeled by four parameters related to camera pan, tilt, zoom and rotation. The initial parameters are obtained from the least-squares method and updated iteratively using the Levenberg-Marquardt algorithm. Temporal and spatial filters are also introduced to revise the final global motion. In addition, DC coefficients are employed to refine the result based on the local motion. Experiments show that the proposed algorithm can segment foreground effectively with a largely reduced computational complexity, as DC coefficients and motion vectors can easily be extracted from compressed videos.
    No preview · Article · Jul 2008
  • [Show abstract] [Hide abstract]
    ABSTRACT: Moving object segmentation directly from compressed video stream is a challenging task in video signal processing community. A novel moving object detection algorithm for H.264/AVC stream is presented in this paper. Firstly, a series of operations, including spatial-temporal normalization, bidirectional accumulation, median filtering and thresholding magnitudes, are conducted on the raw motion vector (MV) field so as to gain a robust MV field, which can indicate the moving region reliably. Then global motion compensation in compressed domain is implemented if there exist global motion caused by camera. Finally, the macroblock coded mode with variable size is combined with the temporal information to locate the foreground object. Its effectiveness of the proposed algorithm has been demonstrated in our experimental results. The main advantage of the algorithm lies in its robustness and real-time performance.
    No preview · Article · Aug 2009
Show more