Multiview Video Coding Using View Interpolation and Color Correction

Universal Media Res. Center, Tokyo
IEEE Transactions on Circuits and Systems for Video Technology (Impact Factor: 2.62). 12/2007; 17(11):1436 - 1449. DOI: 10.1109/TCSVT.2007.903802
Source: IEEE Xplore


Neighboring views must be highly correlated in multiview video systems. We should therefore use various neighboring views to efficiently compress videos. There are many approaches to doing this. However, most of these treat pictures of other views in the same way as they treat pictures of the current view, i.e., pictures of other views are used as reference pictures (inter-view prediction). We introduce two approaches to improving compression efficiency in this paper. The first is by synthesizing pictures at a given time and a given position by using view interpolation and using them as reference pictures (view-interpolation prediction). In other words, we tried to compensate for geometry to obtain precise predictions. The second approach is to correct the luminance and chrominance of other views by using lookup tables to compensate for photoelectric variations in individual cameras. We implemented these ideas in H.264/AVC with inter-view prediction and confirmed that they worked well. The experimental results revealed that these ideas can reduce the number of generated bits by approximately 15% without loss of PSNR.

13 Reads
  • Source
    • "Similarly, Lee et al. [25] propose a DC coefficient modification scheme and integrate it into the MPEG Joint Multiview Video Model (JMVM) reference software. Yamamoto et al. [26] also use correction lookup tables during the compression process for interview prediction. "
    [Show abstract] [Hide abstract]
    ABSTRACT: When compared to conventional 2-D video, multiview video can significantly enhance the visual 3-D experience in 3-D applications by offering horizontal parallax. However, when processing images originating from different views, it is common that the colors between the different cameras are not well- calibrated . To solve this problem, a novel energy function -based color correction method for multiview camera setups is proposed to enforce that colors are as close as possible to those in the reference image but also that the overall structural information is well-preserved. The proposed system introduces a spatio-temporal correspondence matching method to ensure that each pixel in the input image gets bijectively mapped to a reference pixel. By combining this mapping with the original structural information, we construct a global optimization algorithm in a Laplacian matrix formulation and solve it using a sparse matrix solver. We further introduce a novel forward-reverse objective evaluation model to overcome the problem of lack of ground truth in this field. The visual comparisons are shown to outperform state-of-the-art multiview color correction methods, while the objective evaluation reports PSNR gains of up to 1.34 dB and SSIM gains of up to 3.2%, respectively.
    IEEE Transactions on Multimedia 05/2015; 17(5):577-590. DOI:10.1109/TMM.2015.2412879 · 2.30 Impact Factor
  • Source
    • "However, we need to put more efforts to control a huge number of cameras at the same time. Moreover, since the multi-view camera system usually requires complicated coding and transmission schemes [7] in proportional to the number of cameras, it is hard to send its data to the receiver side within limited bandwidth channel environments. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In recent years, various multimedia services have become available and the demand for three-dimensional television (3DTV) is growing rapidly. Since 3DTV is considered as the next generation broadcasting service that can deliver realistic and immersive experiences, a number of advanced 3D video technologies have been studied. In this tutorial lecture, we are going to discuss the current activities for 3DTV research and development. After reviewing the main components of the 3DTV system, we are going to cover several challenging technical issues: representation of 3D scenes, acquisition of 3D video contents, illumination compensation and color correction, camera calibration and image rectification, depth map modeling and enhancement, 3-D warping and depth map refinement, coding of multi-view video and depth map, hole filling for occluded objects, and virtual view synthesis.
    Systems, Signals and Image Processing (IWSSIP), 2013 20th International Conference on; 01/2013
  • Source
    • "In this paper we focus on depth image based rendering (DIBR) [2] [3], where a depth image is encoded and transmitted, along with natural video captured in different view positions, in order to provide 3-D geometry information. Our previous research [4] [5] studies the relationship between the rendered view quality and the depth map distortion. "
    [Show abstract] [Hide abstract]
    ABSTRACT: To improve rendered view quality in a 3-D video system, we propose to encode and transmit depth transition data, which represents, for each pixel in a frame, the location in between two existing views where the depth corresponding to that pixel changes. Given the highly localized and non-linear characteristics of rendered view distortion, it is possible to achieve better coding performance by providing this depth transition data only for subjectively important regions. In this paper, a method to apply the depth transition data to the view rendering procedure is proposed. Experimental results verify that improvements in subjective quality can be achieved by the proposed method.
    Multimedia and Expo (ICME), 2011 IEEE International Conference on; 08/2011
Show more