On Learning Conditional Random Fields for Stereo

International Journal of Computer Vision (Impact Factor: 3.81). 09/2012; 99(3):1-19. DOI: 10.1007/s11263-010-0385-z


Until recently, the lack of ground truth data has hindered the application of discriminative structured prediction techniques
to the stereo problem. In this paper we use ground truth data sets that we have recently constructed to explore different
model structures and parameter learning techniques. To estimate parameters in Markov random fields (MRFs) via maximum likelihood
one usually needs to perform approximate probabilistic inference. Conditional random fields (CRFs) are discriminative versions
of traditional MRFs. We explore a number of novel CRF model structures including a CRF for stereo matching with an explicit
occlusion model. CRFs require expensive inference steps for each iteration of optimization and inference is particularly slow
when there are many discrete states. We explore belief propagation, variational message passing and graph cuts as inference
methods during learning and compare with learning via pseudolikelihood. To accelerate approximate inference we have developed
a new method called sparse variational message passing which can reduce inference time by an order of magnitude with negligible
loss in quality. Learning using sparse variational message passing improves upon previous approaches using graph cuts and
allows efficient learning over large data sets when energy functions violate the constraints imposed by graph cuts.

KeywordsStereo-Learning-Structured prediction-Approximate inference

  • Source
    • "The exponential model [1], [2], [3] assumes costs to be exponentially distributed and is given by "
    [Show abstract] [Hide abstract]
    ABSTRACT: Stereo confidence measures are important functions for global reconstruction methods and some applications of stereo. In this article we evaluate and compare several models of confidence which are defined at the whole disparity range. We propose a new stereo confidence measure to which we call the Histogram Sensor Model (HSM), and show how it is one of the best performing functions overall. We also introduce, for parametric models, a systematic method for estimating their parameters which is shown to lead to better performance when compared to parameters as computed in previous literature. All models were evaluated when applied to two different cost functions at different window sizes and model parameters. Contrary to previous stereo confidence measure benchmark literature, we evaluate the models with criteria important not only to winnertake- all stereo, but also to global applications. To this end, we evaluate the models on a real-world application using a recent formulation of 3D reconstruction through occupancy grids which integrates stereo confidence at all disparities. We obtain and discuss our results on both indoors’ and outdoors’ publicly available datasets.
    Full-text · Article · Dec 2015 · IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper we present a novel stereo view synthesis algorithm that is highly accurate with respect to inter-view consistency, thus to enabling stereo contents to be viewed on the autostereoscopic displays. The algorithm finds identical occluded regions within each virtual view and aligns them together to extract a surrounding background layer. The background layer for each occluded region is then used with an exemplar based inpainting method to synthesize all virtual views simultaneously. Our algorithm requires the alignment and extraction of background layers for each occluded region; however, these two steps are done efficiently with lower computational complexity in comparison to previous approaches using the exemplar based inpainting algorithms. Thus, it is more efficient than existing algorithms that synthesize one virtual view at a time. This paper also describes the implementation of a simplified GPU accelerated version of the approach and its implementation in CUDA. Our CUDA method has sublinear complexity in terms of the number of views that need to be generated, which makes it especially useful for generating content for autostereoscopic displays that require many views to operate. An objective of our work is to allow the user to change depth and viewing perspective on the fly. Therefore, to further accelerate the CUDA variant of our approach, we present a modified version of our method to warp the background pixels from reference views to a middle view to recover background pixels. We then use an exemplar based inpainting method to fill in the occluded regions. We use warping of the foreground from the reference images and background from the filled regions to synthesize new virtual views on the fly. Our experimental results indicate that the simplified CUDA implementation decreases running time by orders of magnitude with negligible loss in quality.
    Full-text · Article · Mar 2012 · 3D Research
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: When stereo images are shown in three-dimensional (3D) display devices of different aspect ratios, the resizing algorithm for single image could lead to shape and depth distortion of the stereo image’s main content. This paper aims to propose a novel method for retargeting stereo image pairs without distorting important objects in the scene while still maintaining the consistency between the left and right images. We extended seam carving algorithm to stereo images. The novelty of our method is that important objects are determined by jointly considering the intensities of gradients and visual fusion area. The retargeted stereo pair has a feasible 3D interpretation that is similar to the original one. Our method protected the important content and reduced the visual distortion in each of the images as well as the depth distortion. Experimental results are presented to demonstrate that the proposed method effectively guaranteed the geometric consistency of resized stereo images.
    Preview · Article · Jan 2013 · EURASIP Journal on Wireless Communications and Networking
Show more