Page 1

POISSON IMAGE RECONSTRUCTION WITH TOTAL VARIATION REGULARIZATION

Rebecca M. Willett1, Zachary T. Harmany1, and Roummel F. Marcia2

1Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708 USA

2School of Natural Sciences, University of California, Merced, Merced, CA 95348 USA

ABSTRACT

This paper describes an optimization framework for reconstructing

nonnegative image intensities from linear projections contaminated

with Poisson noise. Such Poisson inverse problems arise in a variety

of applications, ranging from medical imaging to astronomy. A to-

tal variation regularization term is used to counter the ill-posedness

of the inverse problem and results in reconstructions that are piece-

wise smooth. The proposed algorithm sequentially approximates the

objective function with a regularized quadratic surrogate which can

easily be minimized. Unlike alternative methods, this approach en-

sures that the natural nonnegativity constraints are satisfied without

placing prohibitive restrictions on the nature of the linear projec-

tions to ensure computational tractability. The resulting algorithm

is computationally efficient and outperforms similar methods using

wavelet-sparsity or partition-based regularization.

Index Terms— Photon-limited imaging, Poisson noise, total

variation, convex optimization, sparse approximation

1. INTRODUCTION

In applications such as nuclear medicine imaging, night vision, as-

tronomy, and hyperspectral imaging, data are collected by counting

a series of discrete events, namely photons hitting a detector within

a certain time period. These counts reflect an underlying rate whose

features we wish to reconstruct as accurately as possible. However

the measurements are often inherently noisy when the count levels

are low. In such contexts, the inhomogeneous Poisson process model

[1, 2] has been used to effectively describe such phenomenon.

In many of these settings, the reconstruction problem from Pois-

son counts is complicated by the indirect nature of the measure-

ments. Specifically, instead of observing the image of interest di-

rectly, we collect Poisson measurements of linear projections of the

image [2]. The number of linear projection measurements may be

much smaller than the number of pixel or voxel intensities to be esti-

mated, resulting in a very ill-posed inverse problem. Regularization

techniquesareoftenemployedtocompensatefortheill-posednessof

the estimation problem. Outside the Poisson context, for example in

the presence of additive white Gaussian noise, regularization meth-

ods based on wavelet or curvelet sparsity [3], models of wavelets’

clusteringandpersistenceproperties[4], andavarietyofotherpenal-

ties (cf. [5]) have proven successful.

Regularization based on a total variation (TV) seminorm has

also garnered significant recent attention (cf, [6, 7]). This seminorm

is described in detail below; in general, it measures how much an

image varies across pixels, so that a highly textured or noisy im-

age will have a large TV seminorm, while a smooth or piecewise

This work was supported by NSF CAREER Award No. CCF-06-43947,

DARPA Grant No. HR0011-07-1-003, and NSF Grant DMS-08-11062.

constant image would have a relatively small TV seminorm. This

is often a useful alternative to wavelet-based regularizers, which are

also designed to be small for piecewise smooth images but can re-

sult in spurious large, isolated wavelet coefficients and related image

artifacts.

In the context of Poisson inverse problems, however, adaptation

of these regularization methods can be challenging for two main

reasons. First, the negative Poisson log likelihood used in the for-

mulation of an objective function often requires the application of

relatively sophisticated optimization theory principles. Second, be-

cause Poisson intensities are inherently nonnegative, the resulting

optimization problem must be solved over a constrained feasible set,

increasing the complexity of most algorithms. Some recent head-

way has been made using multiscale or smoothness-based penalties

[1,8,9]. Onerecentwork[10]bypasses thePoissonstatisticalmodel

in favor of an additive Gaussian noise model through the use of the

Anscombe variance stabilizing transform. This statistical simplifica-

tion is not without cost, as the linear projections of the scene must

now be characterized as nonlinear observations. Another recent ef-

fort solves Poisson image reconstruction problems with TV semi-

norm regularization, but the method involves a matrix inverse opera-

tion which can be extremely difficult to compute for large problems

outside of deconvolution settings [11].

In this paper, we show that recent research into computation-

ally efficient TV denoising can be effectively leveraged for TV-

regularized Poisson image reconstruction, even in inverse problem

settings. The resulting algorithm has several compelling features:

• thesolutionisguaranteedtosatisfythenonnegativityconstraints,

even if stopped early (before convergence);

• thesolution doesnotexhibitspurious waveletartifactsassociated

with wavelet sparsity regularization; and

• the method exhibits important convergence properties.

2. PROBLEM FORMULATION

Under the inhomogeneous Poisson process model, the true scene in-

tensity f?∈ Rm

imaging system described by the matrix A ∈ RN×m

tector photon intensity Af?. We then observe a Poisson realization

y ∈ ZN

y ∼ Poisson(Af?),

Specifically, under the model in (1), the negative log likelihood of

observing a particular vector of counts y is given by

+is passed through an intensity preserving passive

+

, yielding a de-

+= {0,1,2,...}N, that is

(1)

−logp(y|Af?) =

N

?

j=1

(Af?)j− yjlog(Af?)j− logyj!,

4177978-1-4244-7993-1/10/$26.00 ©2010 IEEEICIP 2010

Proceedings of 2010 IEEE 17th International Conference on Image ProcessingSeptember 26-29, 2010, Hong Kong

Page 2

where (Af?)j is the jthcomponent of Af?.

We propose estimating f?from y using a TV-regularized Pois-

son log-likelihood objective function. Our algorithm reconstructs

an estimate?f of the true intensity f?by solving the following con-

?f =

where

strained minimization problem:

argmin

f

subject to

Φ(f) ? φ(f) + τ?f?TV

f ? 0,

(2)

φ(f)

?

N

?

√m−1

?

√m

?

j=1

[(Af)j− yjlog(Af)j],

(3)

?f?TV

?

k=1

√m

?

√m−1

?

l=1

|fk,l− fk+1,l|

+

k=1l=1

|fk,l− fk,l+1|,

(4)

τ > 0 is a regularization parameter, and the standard notation f ? 0

means that the components of f are nonnegative. In (4), we have

used 2-d pixel indices instead of vector indices in a small abuse of

notation. We have also assumed that f ∈ Rmis a vector corre-

sponding to a square√m ×√m image for simplicity of presenta-

tion, but this assumption is not necessary for the algorithm. Here we

assume the regularization parameter τ, which balances between the

two terms in (2), is specified by the user. However in practice this

parameter can be chosen via a cross-validation procedure.

3. TOTAL VARIATION REGULARIZATION

Recent work by Beck and Teboulle [7] presents a fast computational

method for solving the TV-regularized problem

?f = argmin

f

subject to

?? Af − b?2

2+ λ?f?TV

f ∈ C,

(5)

where λ > 0 is a tuning parameter, C is a closed convex set and

? A is a linear, spatially invariant blur operator. This method utilizes

erative shrinkage and thresholding algorithm. When? A = I (the

with a total variation regularizer. While (5) is certainly different

from the objective upon which this paper is focused, namely (2), we

will see in the sequel that methods for solving (5) can be leveraged

effectively within an optimization framework for solving (2).

a gradient-based optimization approach founded on a monotone it-

identity matrix), then (5) reduces to least-squares “denoising” of b

4. SPIRAL ALGORITHM

In previous work [9], we proposed optimization methods called

Sparse Poisson Intensity Reconstruction Algorithms (SPIRAL) for

solving (2) for various types of penalties. Specifically, we con-

sidered sparsity-based (?1) and structure-based (recursive dyadic

partition [8]) penalties. In this section, we describe the application

of SPIRAL for solving the TV-penalized optimization problem (2).

The SPIRAL approach sequentially approximates the objective

function in (2) by regularized quadratic functions that are easier to

minimize. In particular, if we denote the current estimate of f?at the

kthiteration by fkand consider the second-order Taylor expansion

ofφ(f)aroundfkwheretheHessian∇2φ(fk)isapproximatedbya

positive scalar multiple of the identity matrix (i.e., ∇2φ(fk) ≈ ηkI

for ηk> 0), we obtain the following quadratic approximation φk(f)

to φ(f) near fk:

φk(f) ? φ(fk) + (f−fk)T∇φ(fk) +ηk

2?f − fk?2

2.

Replacing φ(f) in (2) by this quadratic function φk(f) leads to a

separable quadratic minimization problem, which can be reformu-

lated as

fk+1=argmin

f

subject to

?f − sk?2

f ? 0,

2+2τ

ηk?f?TV

(6)

wheresk= fk−1

Borwein (spectral) methods (see [9] for details).

The minimization (6) can be viewed as a denoising subproblem

applied to sk, the next gradient descent iterate. In fact, this denoising

subproblem is of the form (5), where

ηk∇φ(fk). WechoosethescalarηkusingBarzilai-

? A = I,

With this identification, we can solve this optimization subproblem

with the method described in [7]. The process of approximating

(2) by the denoising subproblem (6) is iterated until suitable con-

vergence criteria are satisfied. We call this sequential quadratic pro-

gramming approach with a total variation penalty SPIRAL-TV.

b = sk,λ =2τ

ηk, and C = {f ∈ Rm: f ? 0}.

5. CONVERGENCE

In [9], we proved the following convergence result for the SPIRAL

algorithm when applied to a generalized version of the objective

function in (2).

Theorem 1. Given ? > 0, there exists some K? > 0 such that for all

k > K?, the primal-dual pair (fk,λk) nearly satisfies the Karush-

Kuhn-Tucker (KKT) optimality conditions associated with (2); that

is, for all k sufficiently large, we have

?∇Φ(fk) − λk?

<?

λk

fk

≥

≥

=

0

0

0.

(λk)Tfk

This theorem implies that for any arbitrarily small tolerance

level ?, we can satisfy the KKT conditions to within that tolerance if

the number of SPIRAL iterations, k, is sufficiently large. Note, as

claimed in the introduction, that the iterates are always feasible with

respect to the nonnegativity constraint fk≥ 0.

6. SIMULATIONS

Here we consider the application of our reconstruction method to

emission computed tomography (ECT). In medical ECT, a human

subject is injected with a radioactive pharmaceutical used to tag cer-

tain tissues or tumors. To obtain a mapping of the radiopharmaceu-

tical uptake, we detect photons emitted as the radiopharmaceutical

decays. From these projection data (the indirect projections y in our

problem), we wish to estimate the underlying radiopharmaceutical

distribution (the intensity f?). The probability transition matrix A is

4178

Page 3

(a)(b) (c)

Fig. 1. Simulated emission tomography experimental setup: (a) true

emission image, (b)noisy projection data, (c) estimate used as ini-

tialization.

derived from the physics and geometry of the detection device and

data collection process, and also incorporates the attenuation cor-

rection needed to account for the attenuation effects introduced by

bodily tissue [12].

The underlying 2-d intensity in our simulation is the 128 × 128

square image shown in Figure 1(a)); it is a standard test image in-

cluded in the Image Reconstruction Toolbox (IRT) by Fessler [13].

We consider the case where the tomographic projection corresponds

to a limited-angle parallel strip-integral geometry with 128 radial

samples and 128 angular samples spaced uniformly over 135 de-

grees. With these parameters, the resulting transition probability

matrix A was computed using the IRT software [13]. Ten realiza-

tions of the noisy tomographic data y are then simulated according

to the inhomogeneous Poisson process (1). We only show images

reconstructed using a particular realization of the data shown in Fig-

ure 1(b), the other nine realizations are used to examine the ten-trial

average performance of the reconstruction methods considered. In

the particular case shown, the noisy sinogram observations have a

total photon count of 2.0 × 105, a mean count over the support of

the tomographic projections of 18.08, and a maximum count of 44.

6.1. Comparison with other methods

To show the effectiveness of the TV regularization, we compare

the results of our SPIRAL-TV approach with the SPIRAL frame-

work applied to objectives resulting from two different regulariza-

tion methods. The first, a wavelet ?1-norm regularizer, reconstructs

an estimate?fwaveletaccording to the following objective:

?fwavelet= argmin

f

subject to

φ(f) + τ?WTf?1

f ? 0,

(7)

where WTf is the discrete wavelet transform of f. Methods for

solvingthisoptimizationaredescribedin[9]. Wedenotethismethod

as SPIRAL-?1. As seen below, this regularization can result in sig-

nificant artifacts corresponding to spurious non-zero wavelet coeffi-

cients.

The second regularization scheme is built on the framework of

recursive dyadic partitions (RDPs), which we summarize here and

are described in detail in [9]. It can be shown that partition-based

methodsarecloselyrelatedtoHaarwaveletdenoisingwithanimpor-

tant hereditary constraint placed on the thresholded coefficients—if

a parent coefficient is thresholded, then its children coefficients must

also be thresholded [8]. This constraint is akin to wavelet-tree ideas

which exploit persistence of significant wavelet coefficients across

scales and have recently been shown highly useful in compressed

sensing settings [4]. The hereditary constraint is also useful compu-

tationally since it admits an efficient dynamic programming imple-

mentation. Within this approach, it is straightforward to introduce

an efficient cycle-spinning strategy that yields a translationally in-

variant algorithm, boosting performance in practice. For brevity, we

only show results for this translationally-invariant method, denoted

SPIRAL-RDP-TI. While partition-based regularization yields accu-

rate results in many settings, it also results in a non-convex optimiza-

tion problem. The regularizer penalizes the number of cells in the

partition-based estimate, essentially an ?0 measure of sparsity. Al-

though RDP-based regularization is computationally tractable (un-

like ?0regularization), the non-convexity of the problem causes the

solution to depend heavily on the initialization and convergence to a

global minimizer cannot be guaranteed.

We also compare our SPIRAL approaches with two compet-

ing Poisson reconstruction methods. The first, denoted SPS-OS,

uses a separable paraboloidal surrogate with ordered subsets algo-

rithm [1]. The second, denoted EPL-INC-3, employs an incremental

penalized Poisson likelihood EM algorithm and was suggested by

Prof. Fessler as representative of the current state-of-the-art in emis-

sion tomographic reconstruction. In our experiments a roughness

penalty based on the Huber potential function yielded the best re-

sults for both of these methods and are shown below. Both of these

methods are available as part of the IRT [13]; specifically, we used

the pwls sps os and epl inc functions from the toolbox. In ad-

dition to these Poisson methods, we also compare to the SpaRSA

algorithm [5] which solves the traditional CS ?1-regularized least-

squares (?2-?1) problem. Both wavelet-based approaches (SPIRAL-

?1and SpaRSA) use the Daubechies-6 wavelet basis for W. As the

solution provided by SpaRSA is not constrained to be nonnegative,

we threshold the result to obtain a feasible – and therefore more ac-

curate – solution. Including this result allows us to demonstrate the

effectiveness of solving the optimization formulation that utilizes the

Poisson likelihood.

6.2. Algorithm setup

All of the methods considered here were initialized with the estimate

shown in Fig. 1(c). This initialization results from two iterations of

a non-convergent version of the EPL-INC-3 algorithm, itself initial-

ized by a filtered back-projection estimate. All algorithms executed

for a minimum of 50 iterations, and global convergence was declared

when the relative change in the iterates, ?fk+1− fk?2/?fk?2, fell

below a tolerance tolP = 5 × 10−4. The advantage of this crite-

ria is that it applies to general penalty functions. The disadvantage,

however, is that it is possible that the change between two consecu-

tive iterates may be small even though the iterates are still far from

the true minimizer. However, forcing the algorithms to perform a

suitable minimum number of iterations typically avoids any issues

with premature termination.

Lastly, in all of the experiments presented in this paper, we

chose any parameters associated with each algorithm (such as τ)

to minimize the root-mean-square error (RMSE (%) = 100 · ??f −

sible in practice, it does allow us to compare the best-case perfor-

mance of various algorithms and penalization methods. In prac-

tical settings, regularization parameters can be chosen via cross-

validation. This is particularly well-suited to many photon-limited

imagingapplicationsinwhicheachdetectedphotonhasatimestamp

associated with it; this timing information can be used to construct

multiple independent and identically distributed realizations of the

underlying Poisson process in software. The details of this proce-

dure are a significant component of our ongoing research.

f??2/?f??2) of the reconstruction. While this would not be pos-

4179

Page 4

(a) SPIRAL-TV

RMSE = 24.404%

(b) SPIRAL-?1

RMSE = 28.626%

(c) SpaRSA

RMSE = 31.172%

(d) SPIRAL-RDP-TI

RMSE = 27.557%

(e) EPL-INC-3

RMSE = 24.748%

(f) SPS-OS

RMSE = 27.555%

Fig. 2. Single-trial reconstructed images with RMSE values.

6.3. Results

The result of the ECT simulation described above are presented in

Figure 2. The TV-regularized result is highly accurate and has the

lowest RMSE among all the methods we considered. The wavelet-

regularized methods (SPIRAL-?1and SpaRSA) exhibit significantly

more spurious artifacts and have higher RMSE. The performance of

SpaRSA is particularly poor since it solves the conventional ?2-?1

minimization problem, which is unsuitable for Poisson image recon-

struction. The partition-based result fares better than the ?1-based

approaches; however it is oversmoothed and looses edge detail cap-

tured in the SPIRAL-TV result. The EPL-INC-3 method offers the

toughest competition to the RMSE achieved by the SPIRAL-TV

method, but there is a rough noise-like texture throughout the image,

causing poor reconstruction in homogeneous regions. The SPS-OS

method is ultimately limited by many streaking artifacts that cross

through the image. Both the single-trial RMSE, and a ten-trial av-

erage RMSE are reported in Table 1, showing the claims above are

robust with respect to different realizations of the data.

Single-Trial

RMSE (%)

Ten-Trial Average

RMSE (%) Method

SPS-OS

EPL-INC-3

SpaRSA

SPIRAL-?1

SPIRAL-TV

SPIRAL-RDP-TI

27.555

24.748

31.172

28.626

24.404

27.557

27.057

24.462

29.987

28.050

24.270

27.669

Table 1. Reconstruction RMSE for single-trial results, and results

averaged over ten trials. RMSE (%) = 100 · ??f − f??2/?f??2.

7. CONCLUSIONS

Using total variation to regularize solutions to Poisson inverse prob-

lems yields highly accurate estimates. The proposed approach out-

performs the current state-of-the-art approaches developed specifi-

cally for emission tomography. Particularly, it results in fewer spuri-

ous artifacts than wavelet-regularized methods, and unlike partition-

regularized methods, is theresult ofa convex optimizationprocedure

with provable convergence properties.

8. REFERENCES

[1] S. Ahn and J. Fessler, “Globally convergent image reconstruc-

tion for emission tomography using relaxed ordered subsets

algorithms,” IEEE Trans. Med. Imaging, vol. 22, no. 5, pp.

613–626, May 2003.

[2] R. Nowak and E. Kolaczyk, “A multiscale statistical frame-

work for Poisson inverse problems,” IEEE Trans. Info. Theory,

vol. 46, pp. 1811–1825, 2000.

[3] E. J. Cand` es and T. Tao, “Near-optimal signal recovery from

random projections: universal encoding strategies?,”

Trans. Inform. Theory, vol. 52, no. 12, pp. 5406–5425, 2006.

[4] R.G.Baraniuk, V.Cevher, M.F.Duarte, andC.Hegde, “Model-

based compressive sensing,”

2008, Submitted.

[5] S. Wright, R. Nowak, and M. Figueiredo, “Sparse reconstruc-

tion by separable approximation,” IEEE Trans. Signal Pro-

cess., vol. 57, pp. 2479–2493, 2009.

[6] T. Chan and J. Shen, Image Processing And Analysis: Vari-

ational, PDE, Wavelet, And Stochastic Methods, Society for

Industrial and Applied Mathematics, 2005.

[7] A. Beck and M. Teboulle,

for constrained total variation image denoising and deblurring

problems,” IEEE Trans. Image Processing, vol. 18, no. 11, pp.

2419–34, 2009.

[8] R. Willett and R. Nowak, “Multiscale Poisson Intensity and

Density Estimation,” IEEE Trans. Inform. Theory, vol. 53, no.

9, pp. 3171–3187, 2007.

[9] Z. T. Harmany, R. F. Marcia, and R. M. Willett, “SPIRAL

out of convexity: Sparsity-regularized algorithms for photon-

limited imaging,” in SPIE Computational Imaging, 2010.

[10] F.-X.Dupe, J.M.Fadili, andJ.-L.Starck, “Aproximaliteration

for deconvolving poisson noisy images using sparse represen-

tations,” IEEE Trans. Image Processing, vol. 18, no. 2, pp.

310–321, 2009.

[11] M. A. T. Figueiredo and J. M. Bioucas-Dias,

of Poissonian images using alternating direction optimization,”

IEEE Trans. Image Process., 2010, Accepted.

[12] Y. Vardi, L. A. Shepp, and L. Kaufman, “A statistical model

for positron emission tomography,” J. Amer. Statist. Assoc.,

vol. 80, pp. 8–37, 1985.

[13] J. Fessler et al., “Image reconstruction toolbox,” http://

www.eecs.umich.edu/˜fessler/code.

IEEE

IEEE Trans. Inform. Theory,

“Fast gradient-based algorithms

“Restoration

4180