Real-time joint disparity and disparity flow estimation on programmable graphics hardware

Department of Computer Science, Memorial University of Newfoundland, St. John’s, Nfld, Canada A1A 3X5
Computer Vision and Image Understanding (Impact Factor: 1.54). 01/2009; 113(1):90-100. DOI: 10.1016/j.cviu.2008.07.007
Source: DBLP


Disparity flow depicts the 3D motion of a scene in the disparity space of a given view and can be considered as view-dependent scene flow. A novel algorithm is presented to compute disparity maps and disparity flow maps in an integrated process. Consequently, the disparity flow maps obtained helps to enforce the temporal consistency between disparity maps of adjacent frames. The disparity maps found also provides the spatial correspondence information that can be used to cross-validate disparity flow maps of different views. Two different optimization approaches are integrated in the presented algorithm for searching optimal disparity values and disparity flows. The local winner-take-all approach runs faster, whereas the global dynamic programming based approach produces better results. All major computations are performed in the image space of the given view, leading to an efficient implementation on programmable graphics hardware. Experimental results on captured stereo sequences demonstrate the algorithm’s capability of estimating both 3D depth and 3D motion in real-time. Quantitative performance evaluation using synthetic data with ground truth is also provided.

18 Reads
  • Source
    • "However, a few methods address dynamic programming for 2D motion estimation, e.g. [23] [28], but the first method that was able to deal with reasonable large 2D displacement fields was proposed in [12] and was then extended for scene flow estimation in [11]. Still, both methods are restricted to deal with displacement vectors of only 25 and 10 pixels, respectively. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Scan-line optimization via cost accumulation has be-come very popular for stereo estimation in computer vision applications and is often combined with a semi-global cost integration strategy, known as SGM. This paper introduces this combination as a general and effective optimization technique. It is the first time that this concept is applied to 3D medical image registration. The presented algorithm, SGM-3D, employs a coarse-to-fine strategy and reduces the search space dimension for consecutive pyramid levels by a fixed linear rate. This allows it to handle large displacements to an extent that is required for clinical applications in high dimensional data. SGM-3D is evaluated in context of pulmonary motion analysis on the recently extended DIR-lab benchmark that provides ten 4D computed tomography (CT) image data sets, as well as ten challenging 3D CT scan pairs from the COPDgene study archive. Results show that both registra-tion errors as well as run-time performance are very com-petitive with current state-of-the-art methods.
    Computer Vision and Pattern Recognition, Columbus, Ohio; 06/2014
  • Source
    • "Some algorithms are designed to take advantage of joint calculation of disparity maps and disparity (scene) flow. For example , in [9] disparities are computed either using WTA or DP strategy, then the disparity flow is calculated using previous frame. Disparity prediction is done for the next frame and matching costs are updated to ensure temporal smoothness . "
    [Show abstract] [Hide abstract]
    ABSTRACT: We present a method for calculation of disparity maps from stereo sequences. Disparity map from previous frame is first transferred to the new frame using estimated motion of the calibrated stereo rig. The predicted disparities are validated for the new frame and areas where prediction failed are matched with a traditional stereo matching algorithm. This method produces very fast and temporally stable stereo matching suitable for real-time applications even on non-parallel hardware. 1. Intro Last decade marked an increasing interest of researchers in stereo matching of image sequences. This task comes up mainly in automotive industry, 3D TV technologies and robotics.
    IEEE International Conference on Computer Vision Workshops, ICCV 2011 Workshops, Barcelona, Spain, November 6-13, 2011; 11/2011
  • Source
    • "Real-time sub-pixel accurate scene flow algorithms, such as the one presented in Rabe et al. (2007), provide only sparse results both for the disparity and the displacement es- timates. The only real-time scene flow algorithm presented in the literature so far is the disparity flow algorithm in Gong (2009), which is an extension of Gong and Yang (2006). This method is a discrete, combinatorial method and requires , a-priori, the allowed range (and discretisation) of values. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Building upon recent developments in optical flow and stereo matching estimation, we propose a variational framework for the estimation of stereoscopic scene flow, i.e., the motion of points in the three-dimensional world from stereo image sequences. The proposed algorithm takes into account image pairs from two consecutive times and computes both depth and a 3D motion vector associated with each point in the image. In contrast to previous works, we partially decouple the depth estimation from the motion estimation, which has many practical advantages. The variational formulation is quite flexible and can handle both sparse or dense disparity maps. The proposed method is very efficient; with the depth map being computed on an FPGA, and the scene flow computed on the GPU, the proposed algorithm runs at frame rates of 20 frames per second on QVGA images (320×240 pixels). Furthermore, we present solutions to two important problems in scene flow estimation: violations of intensity consistency between input images, and the uncertainty measures for the scene flow result.
    International Journal of Computer Vision 10/2011; 95(1):29-51. DOI:10.1007/s11263-010-0404-0 · 3.81 Impact Factor
Show more