Generalizing the Nonlocal-Means to Super-Resolution Reconstruction

Department of Computer Science, The Technion-Israel Institute of Technology, Haifa, Israel.
IEEE Transactions on Image Processing (Impact Factor: 3.63). 02/2009; 18(1):36-51. DOI: 10.1109/TIP.2008.2008067
Source: PubMed

ABSTRACT Super-resolution reconstruction proposes a fusion of several low-quality images into one higher quality result with better optical resolution. Classic super-resolution techniques strongly rely on the availability of accurate motion estimation for this fusion task. When the motion is estimated inaccurately, as often happens for nonglobal motion fields, annoying artifacts appear in the super-resolved outcome. Encouraged by recent developments on the video denoising problem, where state-of-the-art algorithms are formed with no explicit motion estimation, we seek a super-resolution algorithm of similar nature that will allow processing sequences with general motion patterns. In this paper, we base our solution on the Nonlocal-Means (NLM) algorithm. We show how this denoising method is generalized to become a relatively simple super-resolution algorithm with no explicit motion estimation. Results on several test movies show that the proposed method is very successful in providing super-resolution on general sequences.

74 Reads
  • Source
    • "For a given unknown image patch, many similar patches which are either close or far to this patch might be found. This type of nonlocal similarity has been effectively used in image de-noising [20] [21] [22] [23], de-blurring [24] [25] and super resolution [26] [27]. NARM aims to model a given pixel as the linear combination of its nonlocal neighbouring pixels. "
    02/2015; 7(3):38-44. DOI:10.5815/ijigsp.2015.03.06
  • Source
    • "We use a non-local data-fidelity term instead of a non-local regularization. This type of approach has been used in a super-resolution context by Protter et al. [12] and d'Angelo and Vandergheynst [13]. They use the normalized weights issued from the NL-means to define a non-local datafidelity term. "
    [Show abstract] [Hide abstract]
    ABSTRACT: We derive a denoising method based on an adaptive regularization of the non-local means. The NL-means reduce noise by using the redundancy in natural images. They compute a weighted average of pixels whose surroundings are close. This method performs well but it suffers from residual noise on singular structures. We use the weights computed in the NL-means as a measure of performance of the denoising process. These weights balance the data-fidelity term in an adapted ROF model, in order to locally perform adaptive TV regularization. Besides, this model can be adapted to different noise statistics and a fast resolution can be computed in the general case of the exponential family. We adapt this model to video denoising by using spatio-temporal patches. Compared to spatial patches, they offer better temporal stability, while the adaptive TV regularization corrects the residual noise observed around moving structures.
  • Source
    • "An exemplar-based video SR based on the codebooks derived from key-frames was also proposed in [19]. Moreover, the property of nonlocal-means was adopted for video SR in [20], which upscales each input LR patch by linearly fusing multiple similar LR patches based on self-similarity with no explicit motion estimation. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper addresses the problem of hallucinating the missing high-resolution (HR) details of a low-resolution (LR) video while maintaining the temporal coherence of the reconstructed HR details by using dynamic texture synthesis (DTS). Most existing multi-frame-based video super-resolution (SR) methods suffer from the problem of limited reconstructed visual quality due to inaccurate sub-pixel motion estimation between frames in a LR video. To achieve high-quality reconstruction of HR details for a LR video, we propose a texture-synthesis-based video SR method, in which a novel DTS scheme is proposed to render the reconstructed HR details in a temporally coherent way, which effectively addresses the temporal incoherence problem caused by traditional texture synthesis based image SR methods. To further reduce the complexity of the proposed method, our method only performs the texture synthesis-based SR (TS-SR) on a set of key-frames, while the HR details of the remaining non-key-frames are simply predicted using the bi-directional overlapped block motion compensation. After all frames are upscaled, the proposed DTS-SR is applied to maintain the temporal coherence in the HR video. Experimental results demonstrate that the proposed method achieves significant subjective and objective visual quality improvement over state-of-the-art video SR methods.
    IEEE Transactions on Image Processing 01/2015; 24(3). DOI:10.1109/TIP.2014.2387416 · 3.63 Impact Factor
Show more


74 Reads
Available from