-
[show abstract]
[hide abstract]
ABSTRACT: This paper presents a fast algorithm for restoring video sequences. The proposed algorithm, as opposed to existing methods, does not consider video restoration as a sequence of image restoration problems. Rather, it treats a video sequence as a space-time volume and poses a space-time total variation regularization to enhance the smoothness of the solution. The optimization problem is solved by transforming the original unconstrained minimization problem to an equivalent constrained minimization problem. An augmented Lagrangian method is used to handle the constraints, and an alternating direction method is used to iteratively find solutions to the subproblems. The proposed algorithm has a wide range of applications, including video deblurring and denoising, video disparity refinement, and hot-air turbulence effect reduction.
IEEE Transactions on Image Processing 12/2011; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We propose an adaptive loop filter to remove the redundancy between current and motion compensated frames so that the residual signal is minimized, thus coding efficiency increases. The loop filter coefficients and offset are optimized for each frame or a set of blocks to minimize the total energy of the residual signal resulting from motion estimation and compensation. The optimized loop filter with offset is applied for the set of blocks where the filtering process gives coding gain based upon rate-distortion cost. The proposed loop filter is used for the motion compensated frame whereas the conventional adaptive interpolation filter (AIF) is applied to the reference frames to interpolate the subpixel values. Another conventional scheme adaptive loop filter (ALF), is used after deblocking filtering to enhance quality of reconstructed frames, not to minimize energy of residual signal. The proposed loop filter can be used in combination with the AIF and ALF. Experimental results show that proposed algorithm provides the averaged bit reduction of 8% compared to conventional H.264/AVC scheme. When the proposed scheme is combined with AIF and ALF, the coding gain increases even further.
IEEE Transactions on Image Processing 09/2011; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Liquid crystal display (LCD) devices are well known for their slow responses due to the physical limitations of liquid crystals. Therefore, fast moving objects in a scene are often perceived as blurred. This effect is known as the LCD motion blur. In order to reduce LCD motion blur, an accurate LCD model and an efficient deblurring algorithm are needed. However, existing LCD motion blur models are insufficient to reflect the limitation of human-eye-tracking system. Also, the spatiotemporal equivalence in LCD motion blur models has not been proven directly in the discrete 2-D spatial domain, although it is widely used. There are three main contributions of this paper: modeling, analysis, and algorithm. First, a comprehensive LCD motion blur model is presented, in which human-eye-tracking limits are taken into consideration. Second, a complete analysis of spatiotemporal equivalence is provided and verified using real video sequences. Third, an LCD motion blur reduction algorithm is proposed. The proposed algorithm solves an l <sub>1</sub>-norm regularized least-squares minimization problem using a subgradient projection method. Numerical results show that the proposed algorithm gives higher peak SNR, lower temporal error, and lower spatial error than motion-compensated inverse filtering and Lucy-Richardson deconvolution algorithm, which are two state-of-the-art LCD deblurring algorithms.
IEEE Transactions on Image Processing 09/2011; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Multimedia data delivered to mobile devices over wireless channels or the Internet are complicated by bandwidth fluctuation and the variety of mobile devices. Scalable video coding has been developed as an extension of H.264/AVC to solve this problem. Since scalable video codec provides various scalabilities to adapt the bitstream for the channel conditions and terminal types, scalable codec is one of the useful codecs for wired or wireless multimedia communication systems, such as IPTV and streaming services. In such scalable multimedia communication systems, video quality fluctuation degrades the visual perception significantly. It is important to efficiently use the target bits in order to maintain a consistent video quality or achieve a small distortion variation throughout the whole video sequence. The scheme proposed in this paper provides a useful function to control video quality in applications supporting scalability, whereas conventional schemes have been proposed to control video quality in the H.264 and MPEG-4 systems. The proposed algorithm decides the quantization parameter of the enhancement layer to maintain a consistent video quality throughout the entire sequence. The video quality of the enhancement layer is controlled based on a closed-form formula which utilizes the residual data and quantization error of the base layer. The simulation results show that the proposed algorithm controls the frame quality of the enhancement layer in a simple operation, where the parameter decision algorithm is applied to each frame.
IEEE Transactions on Image Processing 09/2011; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In this paper, we propose bit allocation between luma samples and chroma samples using chroma quantization parameter (QP) offsets for Cb and Cr. For this work, we propose an efficient adaptive Lagrange multiplier selection method using classification-maximization, and then apply the proposed adaptive Lagrange multiplier selection to decide chroma QP offsets for Cb and Cr. To our knowledge, this is the first proposal to adaptively decide chroma QP offsets. Because the default mapping function between a chroma QP and a luma QP in H.264 is unbalanced at especially low QPs, the proposed chroma QP offset decision achieves improvement up to 0.8 dB from the experimental results.
IEEE Transactions on Circuits and Systems for Video Technology 07/2011; · 1.65 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In this work, a novel occlusion detection algorithm using online learning is proposed for video applications. Each frame of a video is considered as a time-step for which pixels are classified as being either occluded or non-occluded. The Hedge algorithm is employed to determine weights for a set of experts, each of which is tuned to detect a specific type of occlusion boundary. In contrast to previous training-based methods, the proposed algorithm does not require any training, and has a runtime linear with respect to the number of experts considered. Detection performance is excellent on novel video sequences for which training data does not exist. In addition, the proposed algorithm is easily extended to provide classification results supplementary to detection. We demonstrate results on a series of challenging video sequences including a dataset of hand-labelled occlusion boundaries.
Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: There is an increasing number of methods for removing haze and fog from a single image. One of such methods is Dark Channel Prior (DCP). The goal of this paper is to develop a mathematical explanation on why DCP works well by using principal component analysis, and minimum volume ellipsoid approximations.
Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The paper proposes a novel method for coding artifact reduction in compressed images. For removing blocking artifacts, a localized DCT-based filter with condition on the similarity between surrounding blocks is considered. To reduce ringing, a localized fuzzy filter is utilized to avoid the blurry effect of linear filter and painting-like effect of conventional fuzzy filter. To enhance chroma components and reduce the color bleeding, the localized filter for luma component are implemented for the chroma components. Simulations on a wide range of compressed images are performed to verify the effectiveness of the algorithm.
Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: A new method for scale-aware saliency detection is introduced in this work. Scale determination is realized through a fast scale-space algorithm using color and texture. Scale information is fed back to a Discriminant Saliency engine by automatically tuning center-surround parameters. Excellent results are demonstrated for predicted fixations using a public database of measured human fixations. Further evidence of the proposed algorithm's performance is exhibited through an application to Frame Rate Up-Conversion (FRUC). The ability of the algorithm to detect salient objects at multiple scales allows for class-leading performance both objectively in terms of PSNR/SSIM as well as subjectively. Finally, the need for operator tuning of saliency parameters is dramatically reduced by the inclusion of scale information. The proposed method is well-suited for any application requiring automatic saliency determination for images or video.
Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Binocular luster is an extremely salient effect seen in 3D when an object in each stereo image exhibits a different contrast polarity relative to the background. The object appears to shimmer, a phenomenon seen in nature and on 3D displays, which the Human Visual System rapidly detects. This binocular luster is also induced by compression of stereo imagery where corresponding blocks are quantized to different values. In this paper, we discuss the psychovisual background of binocular luster, introduce the "shine" artifact induced by compression, and present an algorithm for detection and removal of shine from JPEG compressed stereo images without introducing additional bitrate.
Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In accordance with the recent improvement in the quality of image displays for consumer electronics, digital image compression artifacts are visible than ever. Noise removable methods are key issues not only in research areas such as image sensing and satellite imaging but also in consumer electronics such as television signal conversion, DVD and Blu-ray encodings. Among various compression artifact removable methods, the Total Variation (TV) regularization method is one of the most promising solutions. However, there are two problems: one is loss of image minuteness in the texture components, and the other is insufficient compression artifact removal in low bit rates. In this paper, we propose a new noise removable method utilizing the TV regularization method for moving pictures compressed by MPEG-2. We obtain fine experimental results in image quality improvement, and our method is possible to be applied to the other DCT based-image compression standards such as MPEG-4 and H.264.
IEEE Transactions on Consumer Electronics 03/2011; · 0.94 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: This correspondence proposes an efficient semidefinite programming (SDP) method for the design of a class of linear phase finite impulse response triplet halfband filter banks whose filters have optimal frequency selectivity for a prescribed regularity order. The design problem is formulated as the minimization of the least square error subject to peak error constraints and regularity constraints. By using the linear matrix inequality characterization of the trigonometric semi-infinite constraints, it can then be exactly cast as a SDP problem with a small number of variables and, hence, can be solved efficiently. Several design examples of the triplet halfband filter bank are provided for illustration and comparison with previous works. Finally, the image coding performance of the filter bank is presented.
IEEE Transactions on Image Processing 03/2011; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Motion information scalability is an important requirement for a fully scalable video codec, especially in low bit rate or small resolution decoding scenarios, for which the fully scalable motion model (SMM) has been proposed. SMM can collaborate flawlessly with other scalabilities, such as spatial, temporal and quality, in a scalable video codec. It performs better than the nonscalable motion model. To further improve the SMM, this correspondence extends the algorithm to support the hierarchical B frame structure and bidirectional or multidirectional motion estimation. Furthermore, the corresponding rate distortion optimized estimation for improved efficiency in several scenarios is discussed. Several simulation results based upon the updated framework are presented to verify the advantage of this extension.
IEEE Transactions on Image Processing 12/2010; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Motion-compensated frame interpolation (MCFI) is a technique used extensively for increasing the temporal frequency of a video sequence. In order to obtain a high quality interpolation, the motion field between frames must be well-estimated. However, many current techniques for determining the motion are prone to errors in occlusion regions, as well as regions with repetitive structure. We propose an algorithm for improving both the objective and subjective quality of MCFI by refining the motion vector field. We first utilize a discriminant saliency classifier to determine which regions of the motion field are most important to a human observer. These regions are refined using a multistage motion vector refinement (MVR), which promotes motion vector candidates based upon their likelihood given a local neighborhood. For regions which fall below the saliency-threshold, a frame segmentation is used to locate regions of homogeneous color and texture via normalized cuts. Motion vectors are promoted such that each homogeneous region has a consistent motion. Experimental results demonstrate an improvement over previous frame rate up-conversion (FRUC) methods in both objective and subjective picture quality.
IEEE Transactions on Image Processing 12/2010; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The paper proposes a novel method for optimal weight prediction in coding inter-frame of H.264/AVC compressed video sequences. The weights are optimized so that the motion compensated frames is closest to the current frame. This minimizes the residual frame and thus helps reducing the number of bits required to code the residual frame. We first consider the general optimal spatial-temporal weight prediction and then specify the special solution in case of temporal weight prediction using two reference frames (bi-predictive coding). Simulations on a wide range of video sequences are performed to verify the effectiveness of the algorithm on both visual quality and rate-distortion relation.
Advanced Technologies for Communications (ATC), 2010 International Conference on; 11/2010
-
[show abstract]
[hide abstract]
ABSTRACT: Due to the novel fluid optics, unique image processing challenges are presented by the fluidic lens camera system. Developed for surgical applications, unique properties, such as no moving parts while zooming and better miniaturization than traditional glass optics, are advantages of the fluid lens. Despite these abilities, sharp color planes and blurred color planes are created by the nonuniform reaction of the liquid lens to different color wavelengths. Severe axial color aberrations are caused by this reaction. In order to deblur color images without estimating a point spread function, a contourlet filter bank system is proposed. Information from sharp color planes is used by this multiband deblurring method to improve blurred color planes. Compared to traditional Lucy-Richardson and Wiener deconvolution algorithms, significantly improved sharpness and reduced ghosting artifacts are produced by a previous wavelet-based method. Directional filtering is used by the proposed contourlet-based system to adjust to the contours of the image. An image is produced by the proposed method which has a similar level of sharpness to the previous wavelet-based method and has fewer ghosting artifacts. Conditions for when this algorithm will reduce the mean squared error are analyzed. While improving the blue color plane by using information from the green color plane is the primary focus of this paper, these methods could be adjusted to improve the red color plane. Many multiband systems such as global mapping, infrared imaging, and computer assisted surgery are natural extensions of this work. This information sharing algorithm is beneficial to any image set with high edge correlation. Improved results in the areas of deblurring, noise reduction, and resolution enhancement can be produced by the proposed algorithm.
IEEE Transactions on Image Processing 11/2010; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We propose a novel method to synthesize intermediate views from two stereo images and disparity maps that is robust to errors in disparity maps. The proposed method computes a placement matrix from each disparity map that can be used to correct errors when warping pixels from reference view to virtual view. The second contribution is a new hole filling method that uses depth, edge, and segmentation information to aid the process of filling disoccluded pixels. The proposed method selects pixels from segmented regions that are connected to the disoccluded region as candidates to fill the disoccluded pixels. We also provide an explicit probabilistic model to select the best candidate for each disoccluded pixel efficiently with Conditional Random Fields (CRFs) and graph-cuts.
Image Processing (ICIP), 2010 17th IEEE International Conference on; 10/2010
-
[show abstract]
[hide abstract]
ABSTRACT: We propose total subset variation (TSV), a convexity preserving generalization of the total variation (TV) prior, for higher order clique MRF. A proposed differentiable approximation of the TSV prior makes it amenable for use in large images (e.g. 1080p). A convex relaxation of sub-exponential distribution is proposed as a criterion to determine the parameters of the optimization problem resulting from the TSV prior. For the super-resolution application, experiments show reconstruction error improvement with respect to the TV and other methods.
Image Processing (ICIP), 2010 17th IEEE International Conference on; 10/2010
-
[show abstract]
[hide abstract]
ABSTRACT: Liquid crystal display (LCD) is known to have motion blur due to the slow response and sample-hold characteristics of the liquid crystal (LC). To alleviate the LCD motion blur, improving the LC response is the most fundamental solution. However, if the response time is shortened, then more frames are needed and hence frame rate up conversion (FRUC) should be used. In this paper, we study two FRUC methods. We compare the output signal qualities by studying the temporal and spatial profile of the two methods. We use the solution of Erickson-Leslie equation to derive the step response, in contrast to the existing literature where the resistor-capacitor (RC) approximation and uniform averaging function are used. The step response we derived is able to model not only the general trend of the rising and falling edges, but also the effects of different gray level transitions. Based on the step response, we analyze the two methods by comparing the observed signal in both the spatial and temporal domain.
IEEE Signal Processing Letters 10/2010; · 1.39 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Hierarchical structure is a useful tool for providing the necessary scalability in adapting to the variety of channel environments. For schemes involving hierarchical picture structures, bit allocation, and rate control algorithms are vital components for improving video codec performance. Since conventional bit allocation schemes do not efficiently consider the hierarchical structure characteristics, it is difficult to optimize the video quality at an arbitrary bitrate. Similarly, conventional quantization parameter decision methods are not appropriate for controlling the bitrate generated by a codec using a hierarchical encoding structure. In this paper, we propose an effective bit allocation scheme that assigns the target number of bits to pictures or macroblocks (MBs) and improves the overall quality of images encoded by a hierarchical-based encoder. A rate control scheme is also proposed to ensure that the generated bitrate is equal to the assigned target bitrate. From the simulation results, the proposed schemes outperformed conventional methods from a rate-distortion perspective, by efficiently controlling the bitrate of the MB unit. The algorithms regulated the generated bits to achieve the target bits by using the proposed linear R-Q model.
IEEE Transactions on Circuits and Systems for Video Technology 10/2010; · 1.65 Impact Factor
