Page 1

FAST EDGE-FILTERED IMAGE UPSAMPLING

Shantanu H. Joshi1, Antonio L. Marquina2, Stanley J. Osher3, Ivo Dinov1, Arthur W. Toga1,

and John D. Van Horn1

1Laboratory of Neuro Imaging, University of California, Los Angeles, CA 90095, USA

2Departamento de Matematica Aplicada, Universidad de Valencia, C/ Dr Moliner, 50, 46100

Burjassot, Spain

3Department of Mathematics, University of California, Los Angeles, CA 90095, USA

Abstract

We present a novel edge preserved interpolation scheme for fast upsampling of natural images.

The proposed piecewise hyperbolic operator uses a slope-limiter function that conveniently lends

itself to higher-order approximations and is responsible for restricting spatial oscillations arising

due to the edges and sharp details in the image. As a consequence the upsampled image not only

exhibits enhanced edges, and discontinuities across boundaries, but also preserves smoothly

varying features in images. Experimental results show an improvement in the PSNR compared to

typical cubic, and spline-based interpolation approaches.

Keywords

interpolation; edge-preserving; slope-limiter

1 Introduction

Interpolation is a frequently needed tool for many imaging applications ranging from image

zooming, resizing, retouching, formatting, manipulation, and compositing. Aside from

applications aiming to aesthetically manipulate images, one routinely needs image

resampling for the purpose of image matching and registration for computer vision

applications. Often times, high throughput machine vision tasks such as scene reconstruction

and warping utilize simple but fast techniques such as bilinear, bicubic, and occasionally b-

spline interpolation for single images. These methods implicitly assume a smoothness prior

in the image resampling process. For example, bilinear interpolation assumes that the

resampled intensity values arise from first order local averaging of neighboring intensities of

the image, whereas higher order methods such as bicubic, and b-spline assume that local

intensities are estimated by imposing smoothness constraints by fitting high-order

polynomials to the intensity function of the image. While bilinear interpolation restricts

signal overshoots at discontinuities, bicubic, b-spline and other higher order methods

introduce ringing, and haloing artifacts in images. Furthermore, the performance of many

vision applications rely on accurate preservation and detection of edges from images. Thus a

compromise needs to be achieved between edge fidelity and processing latency.

There are several approaches for image upsampling, especially for preserving edges [1, 2, 3]

during the interpolation process. Our approach focuses on single image upsampling, and is

different from image super-resolution approaches [4] that typically involve either fusion of

multiple images, or integration of example-based constraints along with sub-pixel

homologies. Conventionally, the problem of image upsampling is approached by

formulating a degradation model specified by a convolution kernel as well as a

NIH Public Access

Author Manuscript

Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.

Published in final edited form as:

Proc Int Conf Image Proc. 2011 September 14; : 1165–1168.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 2

downsampling operator. The high resolution image is then reconstructed by solving the

inverse problem of image reconstruction that assigns intensity values to the desired image.

In this paper, instead of explicitly optimizing over the degradation model, we directly focus

on the upsampling operator that yields a one-step interpolation of the observed image. Our

resampling approach borrows from methods in fluid dynamics [5, 6] that attempt to

construct total variation diminishing (TVD) solutions from high resolution numerical

schemes for modeling shocks and discontinuities in fluid flows. Our interpolation method

does not require iterative optimization; instead it provides a one-step up/downsampling of

the original image. It relies on higher order derivatives of the image, and limits spatial

oscillations at edges and discontinuities, and at the same time, satisfies the dual requirement

of speed and edge accuracy.

2 Edge-Preserved Sampling

This section formulates the upsampling problem, and introduces the edge-preserving

operator based on a slope-limiter function. Before using the operator for the purpose of

image upsampling, we make the following observation first. The reconstruction error for a

single frame image degradation model is usually expressed as a convolution of the high

resolution image u with a blur kernel operator, followed by a downsampling process, and

can be given by

(1)

where S is an upsampling operator, and D ο S = I, an identity matrix. The degraded image is

confounded by both the down sampling process, and the blur operator. Assuming that the

operators D and h, are fixed, and independent of the image u in Eq. 1, there are a variety of

upsampling operators S, that seek to minimize the cost in Eq. 1. For example, it can be

shown that for piecewise smooth images, a bilinear interpolant operator yields a lower

estimate for the error E in equation 1 when compared with a nearest neighbor interpolant.

Slope-limiter functions have previously been introduced [7, 5, 6] in fluid dynamics for

clamping spurious oscillations, and improving the resolution at edge discontinuities. The

idea here is to choose an appropriate slope-limiter form such that the interpolant ensures

accurate spatial approximation, and prevents excessive increase in the total variation in the

neighborhood of a pixel. In order to fix notation, we consider a two-dimensional m × n

image, and set up a pixel ujk : 1 ≤ j ≤ m, 1 ≤ k ≤ n, where ujk is an average of the true signal

intensity g(x, y) in the pixel, and is written as

(2)

where hx and hy are the pixel step sizes along the X and Y dimension of the image.

Furthermore, we assume that the domain of the pixel function ujk, centered at (xj, yk) is given

by , and denote the divided differences in the image by

, and , and .

2.1 Higher-order Piecewise Hyperbolic Operator

Our goal is to approximate the true intensity function of the image g(x, y) in each pixel by

means of an elementary function Hjk(x, y), such that Eq. 2 is satisfied. While there are

several different edge-preserving forms for the function Hjk, following Marquina [6] we

restrict our focus to special type of functions also known as slope-limiters. There is a wide

Joshi et al. Page 2

Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 3

variety of such nonlinear functions [8] that preprocess divided differences and enforce the

order of accuracy. In this paper, we propose a third order approximation to the function H(x,

y) using a piecewise hyperbolic form. Additionally we will use the harmod limiter function

[6] that uses the notion of a harmonic mean instead of an absolute mean of the divided

differences. The harmod limiter operator is given by

We assume that the general class of the nonlinear interpolant functions has a general form

given by

.

(3)

Our goal is to find a specific form of the above nonlinear function. We consider the

following ansatz [6] in order to represent the functional form given in Eq. 3,

(4)

where the parameters ajk, bjk are given by , and

. In order to define αx and αy, we first define

, and . Then the parameters αx and

βy are defined as

(5)

We now define the upsampling operator S at the center of half-size pixels in each dimension

as

(6)

where xj(θx) = xj + θxhx, and yk(θy) = yk + θyhy, and θx, θy ∈ [−1/4, 1/4]. Similarly, the

downsampling operator D is defined as

higher-order approximation, it is noted that the upsampling criteria is given by D ο S(u) = u

+ O(((hxhy)), and no longer identity. However, from a practical standpoint, the errors are

negligible, and the signal is not degraded in a significant way. It is also noted that the

operation of downsampling followed by upsampling is not reversible, and thus D ο S ≠ S ο

D.

. On account of the

3 Results

In this section, we present results of upsampled images by applying the operator specified in

Eq. 6. Figure 1 shows upsampled images obtained using bilinear, b-spline, and the edge-

Joshi et al. Page 3

Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 4

preserved upsampling operators. The respective PSNR values are quantified in Table 1. In

order to calculate the PSNR, the original images were downsampled using bilinear

interpolation and then upsampled using each of the upsampling operators. The edge-

preserving piecewise hyperbolic operator not only outperforms bicubic, and b-spline

interpolation, but the resulting upsampled images appear sharper, and show better edges.

Finally, the edge-preserving operator is also used to upsample color images as shown in Fig.

2. Here, the color image was first separated into luminance (Y), and the two chrominance

channels (UV), and each interpolation operator was applied separately to each channel, and

then converted back to the RGB colorspace. It is observed that the edge-preserved

upsampled images resolve sharpness and detail better compared to the bilinear, and spline

based approaches.

4 Discussion

The proposed piecewise hyperbolic operator ensures that the upsampled reconstructions are

smooth inside each pixel, and limits oscillations and jump discontinuities located at the pixel

interfaces. From the experimental results obtained after applying our edge-preserving

operator for interpolation, we observe that this reconstruction error is further lowered

compared with both bilinear, bicubic, as well as b-spline interpolates. The operator is simple

to implement, and executes in 0.2 s for an average image size of 256×256 pixels in

MATLAB on an Intel 2.4 GHz platform. We anticipate the usefulness of this operator in

routine image processing as well as computer vision tasks, where the preservation of edge

quality is of primary importance.

Acknowledgments

This work is supported in part by NIH grants RC1MH088194, and P41 RR013642. Additionally, Dr. Antonio

Marquina gratefully acknowledges the support from the NSF grants DMS-0312222, ACI-0321917, the NIH grant

G54 RR021813, as well as DGICYT MTM2008-03597 from the Spanish Government Agency.

5 References

[1]. Fattal R. Image upsampling via imposed edge statistics. ACM Transactions on Graphics. Aug;

2007 26(3):95.1–95.7.

[2]. Mishiba, K.; Suzuki, T.; Ikehara, M. Edge-adaptive image interpolation using constrained least

squares; 17th IEEE International Conference on Image Processing (ICIP), 2010; 2010; p.

2837-2840.

[3]. Casciola G, Montefusco LB, Morigi S. Edge-driven image interpolation using adaptive anisotropic

radial basis functions. J Math Imaging Vis. Jan; 2010 36(2):125–139.

[4]. Glasner, D.; Bagon, S.; Irani, M. Super-resolution from a single image; 12th IEEE International

Conference on Computer Vision; 2009; p. 349-356.

[5]. Harten A. High resolution schemes for hyperbolic conservation laws. Journal of Computational

Physics. 1983; 135(2):357–393.

[6]. Marquina A. Local piecewise hyperbolic reconstruction of numerical fluxes for nonlinear scalar

conservation-laws. SIAM J Sci Comput. Jan; 1994 15(4):892–915.

[7]. Van Leer B. Towards the ultimate conservative difference scheme II. monotonicity and

conservation combined in a second order scheme. Journal of Computational Physics. 1974;

14(4):361–370.

[8]. Serna S, Marquina A. Power ENO methods: a fifth-order accurate weighted power ENO method. J

Comput Phys. Jan; 2004 194(2):632–658.

Joshi et al. Page 4

Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 5

Fig. 1.

Upsampled images (×2) resulting from bilinear, b-spline, and edge-preserving interpolation

applied to the downsampled image.

Joshi et al.Page 5

Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 6

Fig. 2.

From top: Upsampled images (×2) resulting from bilinear, b-spline, and edge-preserved

interpolation applied to the downsampled original image. The same interpolation method is

applied separately to each of the luminance and the two chrominance channels.

Joshi et al.Page 6

Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 7

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Joshi et al.Page 7

Table 1

Comparison of PNSR (

the original image for each of the interpolation operators.

) calculated from the norm between the upsampled and

Image BilinearBicubic B-SplineEdge-preserved

Moon

49.59 dB51.24 dB 51.37 dB

51.77 dB

Math

73.85 dB 77.73 dB79.16 dB

82.44 dB

Text

39.18 dB40.83 dB 41.38 dB

43.61 dB

House

47.02 dB 48.58 dB 48.84 dB

49.50 dB

Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.