Generalizing the Nonlocal-Means to Super-Resolution Reconstruction

Department of Computer Science, The Technion-Israel Institute of Technology, Haifa, Israel.
IEEE Transactions on Image Processing (Impact Factor: 3.11). 02/2009; 18(1):36-51. DOI: 10.1109/TIP.2008.2008067
Source: PubMed

ABSTRACT Super-resolution reconstruction proposes a fusion of several low-quality images into one higher quality result with better optical resolution. Classic super-resolution techniques strongly rely on the availability of accurate motion estimation for this fusion task. When the motion is estimated inaccurately, as often happens for nonglobal motion fields, annoying artifacts appear in the super-resolved outcome. Encouraged by recent developments on the video denoising problem, where state-of-the-art algorithms are formed with no explicit motion estimation, we seek a super-resolution algorithm of similar nature that will allow processing sequences with general motion patterns. In this paper, we base our solution on the Nonlocal-Means (NLM) algorithm. We show how this denoising method is generalized to become a relatively simple super-resolution algorithm with no explicit motion estimation. Results on several test movies show that the proposed method is very successful in providing super-resolution on general sequences.

  • Source
    • "For a given unknown image patch, many similar patches which are either close or far to this patch might be found. This type of nonlocal similarity has been effectively used in image de-noising [20] [21] [22] [23], de-blurring [24] [25] and super resolution [26] [27]. NARM aims to model a given pixel as the linear combination of its nonlocal neighbouring pixels. "
    02/2015; 7(3):38-44. DOI:10.5815/ijigsp.2015.03.06
  • Source
    • "An exemplar-based video SR based on the codebooks derived from key-frames was also proposed in [19]. Moreover, the property of nonlocal-means was adopted for video SR in [20], which upscales each input LR patch by linearly fusing multiple similar LR patches based on self-similarity with no explicit motion estimation. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper addresses the problem of hallucinating the missing high-resolution (HR) details of a low-resolution (LR) video while maintaining the temporal coherence of the reconstructed HR details by using dynamic texture synthesis (DTS). Most existing multi-frame-based video super-resolution (SR) methods suffer from the problem of limited reconstructed visual quality due to inaccurate sub-pixel motion estimation between frames in a LR video. To achieve high-quality reconstruction of HR details for a LR video, we propose a texture-synthesis-based video SR method, in which a novel DTS scheme is proposed to render the reconstructed HR details in a temporally coherent way, which effectively addresses the temporal incoherence problem caused by traditional texture synthesis based image SR methods. To further reduce the complexity of the proposed method, our method only performs the texture synthesis-based SR (TS-SR) on a set of key-frames, while the HR details of the remaining non-key-frames are simply predicted using the bi-directional overlapped block motion compensation. After all frames are upscaled, the proposed DTS-SR is applied to maintain the temporal coherence in the HR video. Experimental results demonstrate that the proposed method achieves significant subjective and objective visual quality improvement over state-of-the-art video SR methods.
    IEEE Transactions on Image Processing 01/2015; 24(3). DOI:10.1109/TIP.2014.2387416 · 3.11 Impact Factor
  • Source
    • "Numerous super-resolution (SR) methods combining several low-resolution (LR) images to compute one high-resolution (HR) image have been developed and applied in microscopy, astronomy or camera photography, see [2] for a review. However, most precise methods require long computational time. "
    [Show abstract] [Hide abstract]
    ABSTRACT: It is well known that the registration process is a key step for super-resolution reconstruction. In this work, we propose to use a piezoelectric system that is easily adaptable on all microscopes and telescopes for controlling accurately their motion (down to nanometers) and therefore acquiring multiple images of the same scene at different controlled positions. Then a fast super-resolution algorithm \cite{eh01} can be used for efficient super-resolution reconstruction. In this case, the optimal use of $r^2$ images for a resolution enhancement factor $r$ is generally not enough to obtain satisfying results due to the random inaccuracy of the positioning system. Thus we propose to take several images around each reference position. We study the error produced by the super-resolution algorithm due to spatial uncertainty as a function of the number of images per position. We obtain a lower bound on the number of images that is necessary to ensure a given error upper bound with probability higher than some desired confidence level.
Show more


Available from