Conference Paper

Real-Time Foreground Segmentation for the Moving Camera Based on H.264 Video Coding Information

Nat. Central Univ., Taoyuan
DOI: 10.1109/FGCN.2007.191 Conference: Future generation communication and networking (fgcn 2007), Volume: 1
Source: IEEE Xplore


Foreground segmentation for video frames has played an important role in many video applications, such as video surveillance, video indexing, etc. Due to most videos are compressed, foreground segmentation can benefit from utilizing such coding information and save much processing time. In this paper, we propose a real-time foreground segmentation algorithm for the moving camera based on the H.264 video coding information. In the proposed algorithm, we first utilize the relative global motion model to calculate the approximate global motion vector and get the motion vector difference of each block. Then, according to the block partition modes, we assign different weightings and apply spatio-temporal refinement to these motion vector differences for further improving the accuracy of segmentation results. Finally, we segment out the foreground blocks by an adaptive threshold. With the aid of H.264 video coding information, the proposed segmentation algorithm is more practical than many other methods based on spatial domain information in computational complexity.

Full-text preview

Available from:
  • [Show abstract] [Hide abstract]
    ABSTRACT: A new algorithm to distinguish the foreground from compressed videos is proposed in this paper. Local motion, which is estimated from the residual between the original motion and the global motion, is one of the strongest influences on visual attention. Global motion is modeled by four parameters related to camera pan, tilt, zoom and rotation. The initial parameters are obtained from the least-squares method and updated iteratively using the Levenberg-Marquardt algorithm. Temporal and spatial filters are also introduced to revise the final global motion. In addition, DC coefficients are employed to refine the result based on the local motion. Experiments show that the proposed algorithm can segment foreground effectively with a largely reduced computational complexity, as DC coefficients and motion vectors can easily be extracted from compressed videos.
  • [Show abstract] [Hide abstract]
    ABSTRACT: Moving object segmentation directly from compressed video stream is a challenging task in video signal processing community. A novel moving object detection algorithm for H.264/AVC stream is presented in this paper. Firstly, a series of operations, including spatial-temporal normalization, bidirectional accumulation, median filtering and thresholding magnitudes, are conducted on the raw motion vector (MV) field so as to gain a robust MV field, which can indicate the moving region reliably. Then global motion compensation in compressed domain is implemented if there exist global motion caused by camera. Finally, the macroblock coded mode with variable size is combined with the temporal information to locate the foreground object. Its effectiveness of the proposed algorithm has been demonstrated in our experimental results. The main advantage of the algorithm lies in its robustness and real-time performance.
  • [Show abstract] [Hide abstract]
    ABSTRACT: This paper presents a new method to distinguish the foreground from compressed videos. It demonstrates that the foreground detection can be effectively achieved by the local motion combine luminance and chromaticity factors. In order to calculate the local motion, which is estimated from the original motion and the global motion, we built a four –parameter model to compute global motion. Levenberg-Marquardt algorithm is introduced to revise those parameters. For luminance and chromaticity factors, DC coefficients are employed to segment the objects, which are considered as the index of the image. Some implement results show that our algorithm provides a practical way for foreground extraction and more researches of high level video processing will be carried out based on it.
Show more