Motion Detail Preserving Optical Flow Estimation

The Chinese University of Hong Kong, Hong Kong.
IEEE Transactions on Software Engineering (Impact Factor: 5.78). 12/2011; 34(9). DOI: 10.1109/TPAMI.2011.236
Source: PubMed


A common problem of optical flow estimation in the multi-scale variational framework is that fine motion structures cannot always be correctly estimated, especially for regions with significant and abrupt displacement variation. A novel extended coarse-to-fine (EC2F) refinement framework is introduced in this paper to address this issue, which reduces the reliance of flow estimates on their initial values propagated from the coarse level and enables recovering many motion details in each scale. The contribution of this paper also includes adaption of the objective function to handle outliers and development of a new optimization procedure. The effectiveness of our algorithm is borne out by the Middlebury optical flow benchmark and by experiments on challenging examples that involve large-displacement motion.

16 Reads
  • Source
    • "Typical methods include similarity, affine, or projective transformations for rigid scenes [20]. For non-rigid and dynamic scenes, optic flow can be used to correct misaligned camera geometries [24] [29]. Our approach extends SIFT flow [14] by considering multi-lighting constraints. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Detecting fine-grained subtle changes among a scene is critically important in practice. Previous change detection methods, focusing on detecting large-scale significant changes, cannot do this well. This paper proposes a feasible end-to-end approach to this challenging problem. We start from active camera relocation that quickly relocates camera to nearly the same pose and position of the last time observation. To guarantee detection sensitivity and accuracy of minute changes, in an observation, we capture a group of images under multiple illuminations, which need only to be roughly aligned to the last time lighting conditions. Given two times observations, we formulate fine-grained change detection as a joint optimization problem of three related factors, i.e., normal-aware lighting difference, camera geometry correction flow, and real scene change mask. We solve the three factors in a coarse-to-fine manner and achieve reliable change decision by rank minimization. We build three real-world datasets to benchmark fine-grained change detection of misaligned scenes under varied multiple lighting conditions. Extensive experiments show the superior performance of our approach over state-of-the-art change detection methods and its ability to distinguish real scene changes from false ones caused by lighting variations.
    ICCV; 12/2015
  • Source
    • "The objective here is to eliminate artificial block boundaries, while efficiently describing true discontinuities in the motion flow. There is a body of research on estimating piecewisesmooth motion fields with sharp transitions at object boundaries [6], [7], but this remains a challenging task. As we will see, we require the discontinuities to be aligned across multiple motion fields originating from a given frame, which is a topic of ongoing research. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Existing video coders anchor motion fields at frames that are to be predicted. In this paper, we demonstrate how changing the anchoring of motion fields to reference frames has some important advantages over conventional anchoring. We work with piecewise-smooth motion fields, and use breakpoints to signal discontinuities at moving object boundaries. We show how discontinuity information can be used to resolve double mappings arising when motion is warped from reference to target frames. We present an analytical model that allows to determine weights for texture, motion, and breakpoints to guide the rate-allocation for scalable encoding. Compared to the conventional way of anchoring motion fields, the proposed scheme requires fewer bits for the coding of motion; furthermore, the reconstructed video frames contain fewer ghosting artefacts. Experimental results show superior performance compared to the traditional anchoring, and demonstrate the high scalability attributes of the proposed method.
    IEEE Transactions on Image Processing 11/2015; DOI:10.1109/TIP.2015.2496332 · 3.63 Impact Factor
    • "The classic-NL algorithm [36] uses a non-local median filtering along the edges found with a Sobel edge detector in order to dilate them for obtaining accurate flow boundaries. The MDP-flow approach proposed by [37] first uses a discrete optimization based on candidate matches providing an initial OF estimate. In the second stage, a TV-L 1 optimization scheme with parallelizable primal-dual formulation is used for obtaining the final OF field. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Image mosaicing is a technique widely used for extending the field of view of industrial, medical, outdoor or indoor scenes. However, image registration can be very challenging ,e.g. due to large texture variability, illumination changes, image blur and camera perspective changes. In this paper, a total variational optical flow approach is investigated to estimate dense point correspondences between image pairs. An edge preserving Riesz wavelet scale-space combined with an novel TV regularizer is proposed for preserving motion discontinuities along the edges of weak textures and for handling strong in-plane rotations present in image sequences. An anisotropic weighted median filtering is implemented for minimizing outliers. Quantitative evaluation of the method on the Middlebury image database and simulated sequences with known ground truth demonstrates high accuracy of the proposed method in comparison with other state-of-the-art methods,including a robust graph-cut method and a patch matching approach. Qualitative results on video-sequences of difficult real scenes demonstrate the robustness of the proposed method.
    Pattern Recognition 10/2015; DOI:10.1016/j.patcog.2015.09.021 · 3.10 Impact Factor
Show more


16 Reads
Available from