Real-time joint disparity and disparity flow estimation on programmable graphics hardware

Department of Computer Science, Memorial University of Newfoundland, St. John’s, Nfld, Canada A1A 3X5
Computer Vision and Image Understanding (Impact Factor: 1.36). 01/2009; DOI: 10.1016/j.cviu.2008.07.007
Source: DBLP

ABSTRACT Disparity flow depicts the 3D motion of a scene in the disparity space of a given view and can be considered as view-dependent scene flow. A novel algorithm is presented to compute disparity maps and disparity flow maps in an integrated process. Consequently, the disparity flow maps obtained helps to enforce the temporal consistency between disparity maps of adjacent frames. The disparity maps found also provides the spatial correspondence information that can be used to cross-validate disparity flow maps of different views. Two different optimization approaches are integrated in the presented algorithm for searching optimal disparity values and disparity flows. The local winner-take-all approach runs faster, whereas the global dynamic programming based approach produces better results. All major computations are performed in the image space of the given view, leading to an efficient implementation on programmable graphics hardware. Experimental results on captured stereo sequences demonstrate the algorithm’s capability of estimating both 3D depth and 3D motion in real-time. Quantitative performance evaluation using synthetic data with ground truth is also provided.

  • [Show abstract] [Hide abstract]
    ABSTRACT: The authors present a novel approach to integrate the optical flow estimation into a block stereo matching algorithm using dynamic programming (DP). It is well known that reliable ground control points can significantly improve the performance of stereo matching, but false matches can also significantly degrade the performance of stereo matching. In order to extract as many reliable ground control points as possible, the authors use the Lucas-Kanade method in pyramids to estimate the optical flow of every pixel and use a bidirectional matching process to remove false matches. To further reduce the impact of false matches, a matching cost is assigned to each point to form a set of -soft- ground control points. Experimental results show that the proposed algorithm, which uses -soft- ground control points and DP, can achieve the performance close to the best performance of algorithms using DP.
    IET Image Processing 01/2012; 6(3):205-212. · 0.68 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Real-time stereo matching in image sequences is important in video monitoring, robotic navigation and intelligent vehicle, etc. Spatiotemporal stereo and scene flow can be used to produce temporally coherent disparity of dynamic scenes. However, most methods do not use the previous disparity map sufficiently to compute the current one. Thus, the disparity range limits the speed of disparity computation for each stereo pair. This paper integrates the temporal information into the stereo computation, and presents the relationship between consecutive disparity maps, which makes the disparity prediction reasonable. The scheme can produce a sequence of temporally coherent disparity maps rapidly. The tests performed on simulated and real stereo sequences confirm the validity of our approach.
    Neurocomputing 10/2014; 142:335–342. · 2.01 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: This paper surveys the state of the art in evaluating the performance of scene flow estimation and points out the difficulties in generating benchmarks with ground truth which have not allowed the development of general, reliable solutions. Hopefully, the renewed interest in dynamic 3D content, which has led to increased research in this area, will also lead to more rigorous evaluation and more effective algorithms. We begin by classifying methods that estimate depth, motion or both from multi-view sequences according to their parameterization of shape and motion. Then, we present several criteria for their evaluation, discuss their strengths and weaknesses and conclude with recommendations.
    Proceedings of the 12th international conference on Computer Vision - Volume 2; 10/2012