Poisson image reconstruction with total variation regularization

Abstract

This paper describes an optimization framework for reconstructing nonnegative image intensities from linear projections contaminated with Poisson noise. Such Poisson inverse problems arise in a variety of applications, ranging from medical imaging to astronomy. A total variation regularization term is used to counter the ill-posedness of the inverse problem and results in reconstructions that are piecewise smooth. The proposed algorithm sequentially approximates the objective function with a regularized quadratic surrogate which can easily be minimized. Unlike alternative methods, this approach ensures that the natural nonnegativity constraints are satisfied without placing prohibitive restrictions on the nature of the linear projections to ensure computational tractability. The resulting algorithm is computationally efficient and outperforms similar methods using wavelet-sparsity or partition-based regularization.

Full text

Page 1

POISSON IMAGE RECONSTRUCTION WITH TOTAL VARIATION REGULARIZATION
Rebecca M. Willett1, Zachary T. Harmany1, and Roummel F. Marcia2
1Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708 USA
2School of Natural Sciences, University of California, Merced, Merced, CA 95348 USA
ABSTRACT
This paper describes an optimization framework for reconstructing
nonnegative image intensities from linear projections contaminated
with Poisson noise. Such Poisson inverse problems arise in a variety
of applications, ranging from medical imaging to astronomy. A to-
tal variation regularization term is used to counter the ill-posedness
of the inverse problem and results in reconstructions that are piece-
wise smooth. The proposed algorithm sequentially approximates the
objective function with a regularized quadratic surrogate which can
easily be minimized. Unlike alternative methods, this approach en-
sures that the natural nonnegativity constraints are satisfied without
placing prohibitive restrictions on the nature of the linear projec-
tions to ensure computational tractability. The resulting algorithm
is computationally efficient and outperforms similar methods using
wavelet-sparsity or partition-based regularization.
Index Terms— Photon-limited imaging, Poisson noise, total
variation, convex optimization, sparse approximation
1. INTRODUCTION
In applications such as nuclear medicine imaging, night vision, as-
tronomy, and hyperspectral imaging, data are collected by counting
a series of discrete events, namely photons hitting a detector within
a certain time period. These counts reflect an underlying rate whose
features we wish to reconstruct as accurately as possible. However
the measurements are often inherently noisy when the count levels
are low. In such contexts, the inhomogeneous Poisson process model
[1, 2] has been used to effectively describe such phenomenon.
In many of these settings, the reconstruction problem from Pois-
son counts is complicated by the indirect nature of the measure-
ments. Specifically, instead of observing the image of interest di-
rectly, we collect Poisson measurements of linear projections of the
image [2]. The number of linear projection measurements may be
much smaller than the number of pixel or voxel intensities to be esti-
mated, resulting in a very ill-posed inverse problem. Regularization
techniquesareoftenemployedtocompensatefortheill-posednessof
the estimation problem. Outside the Poisson context, for example in
the presence of additive white Gaussian noise, regularization meth-
ods based on wavelet or curvelet sparsity [3], models of wavelets’
clusteringandpersistenceproperties[4], andavarietyofotherpenal-
ties (cf. [5]) have proven successful.
Regularization based on a total variation (TV) seminorm has
also garnered significant recent attention (cf, [6, 7]). This seminorm
is described in detail below; in general, it measures how much an
image varies across pixels, so that a highly textured or noisy im-
age will have a large TV seminorm, while a smooth or piecewise
This work was supported by NSF CAREER Award No. CCF-06-43947,
DARPA Grant No. HR0011-07-1-003, and NSF Grant DMS-08-11062.
constant image would have a relatively small TV seminorm. This
is often a useful alternative to wavelet-based regularizers, which are
also designed to be small for piecewise smooth images but can re-
sult in spurious large, isolated wavelet coefficients and related image
artifacts.
In the context of Poisson inverse problems, however, adaptation
of these regularization methods can be challenging for two main
reasons. First, the negative Poisson log likelihood used in the for-
mulation of an objective function often requires the application of
relatively sophisticated optimization theory principles. Second, be-
cause Poisson intensities are inherently nonnegative, the resulting
optimization problem must be solved over a constrained feasible set,
increasing the complexity of most algorithms. Some recent head-
way has been made using multiscale or smoothness-based penalties
[1,8,9]. Onerecentwork[10]bypasses thePoissonstatisticalmodel
in favor of an additive Gaussian noise model through the use of the
Anscombe variance stabilizing transform. This statistical simplifica-
tion is not without cost, as the linear projections of the scene must
now be characterized as nonlinear observations. Another recent ef-
fort solves Poisson image reconstruction problems with TV semi-
norm regularization, but the method involves a matrix inverse opera-
tion which can be extremely difficult to compute for large problems
outside of deconvolution settings [11].
In this paper, we show that recent research into computation-
ally efficient TV denoising can be effectively leveraged for TV-
regularized Poisson image reconstruction, even in inverse problem
settings. The resulting algorithm has several compelling features:
• thesolutionisguaranteedtosatisfythenonnegativityconstraints,
even if stopped early (before convergence);
• thesolution doesnotexhibitspurious waveletartifactsassociated
with wavelet sparsity regularization; and
• the method exhibits important convergence properties.
2. PROBLEM FORMULATION
Under the inhomogeneous Poisson process model, the true scene in-
tensity f?∈ Rm
imaging system described by the matrix A ∈ RN×m
tector photon intensity Af?. We then observe a Poisson realization
y ∈ ZN
y ∼ Poisson(Af?),
Specifically, under the model in (1), the negative log likelihood of
observing a particular vector of counts y is given by
+is passed through an intensity preserving passive
+
, yielding a de-
+= {0,1,2,...}N, that is
(1)
−logp(y|Af?) =
N
?
j=1
(Af?)j− yjlog(Af?)j− logyj!,
4177 978-1-4244-7993-1/10/$26.00 ©2010 IEEEICIP 2010
Proceedings of 2010 IEEE 17th International Conference on Image Processing September 26-29, 2010, Hong Kong

Page 2

where (Af?)j is the jthcomponent of Af?.
We propose estimating f?from y using a TV-regularized Pois-
son log-likelihood objective function. Our algorithm reconstructs
an estimate?f of the true intensity f?by solving the following con-
?f =
where
strained minimization problem:
argmin
f
subject to
Φ(f) ? φ(f) + τ?f?TV
f ? 0,
(2)
φ(f)
?
N
?
√m−1
?
√m
?
j=1
[(Af)j− yjlog(Af)j],
(3)
?f?TV
?
k=1
√m
?
√m−1
?
l=1
|fk,l− fk+1,l|
+
k=1l=1
|fk,l− fk,l+1|,
(4)
τ > 0 is a regularization parameter, and the standard notation f ? 0
means that the components of f are nonnegative. In (4), we have
used 2-d pixel indices instead of vector indices in a small abuse of
notation. We have also assumed that f ∈ Rmis a vector corre-
sponding to a square√m ×√m image for simplicity of presenta-
tion, but this assumption is not necessary for the algorithm. Here we
assume the regularization parameter τ, which balances between the
two terms in (2), is specified by the user. However in practice this
parameter can be chosen via a cross-validation procedure.
3. TOTAL VARIATION REGULARIZATION
Recent work by Beck and Teboulle [7] presents a fast computational
method for solving the TV-regularized problem
?f =argmin
f
subject to
?? Af − b?2
2+ λ?f?TV
f ∈ C,
(5)
where λ > 0 is a tuning parameter, C is a closed convex set and
? A is a linear, spatially invariant blur operator. This method utilizes
erative shrinkage and thresholding algorithm. When? A = I (the
with a total variation regularizer. While (5) is certainly different
from the objective upon which this paper is focused, namely (2), we
will see in the sequel that methods for solving (5) can be leveraged
effectively within an optimization framework for solving (2).
a gradient-based optimization approach founded on a monotone it-
identity matrix), then (5) reduces to least-squares “denoising” of b
4. SPIRAL ALGORITHM
In previous work [9], we proposed optimization methods called
Sparse Poisson Intensity Reconstruction Algorithms (SPIRAL) for
solving (2) for various types of penalties. Specifically, we con-
sidered sparsity-based (?1) and structure-based (recursive dyadic
partition [8]) penalties. In this section, we describe the application
of SPIRAL for solving the TV-penalized optimization problem (2).
The SPIRAL approach sequentially approximates the objective
function in (2) by regularized quadratic functions that are easier to
minimize. In particular, if we denote the current estimate of f?at the
kthiteration by fkand consider the second-order Taylor expansion
ofφ(f)aroundfkwheretheHessian∇2φ(fk)isapproximatedbya
positive scalar multiple of the identity matrix (i.e., ∇2φ(fk) ≈ ηkI
for ηk> 0), we obtain the following quadratic approximation φk(f)
to φ(f) near fk:
φk(f) ? φ(fk) + (f−fk)T∇φ(fk) +ηk
2?f − fk?2
2.
Replacing φ(f) in (2) by this quadratic function φk(f) leads to a
separable quadratic minimization problem, which can be reformu-
lated as
fk+1= argmin
f
subject to
?f − sk?2
f ? 0,
2+2τ
ηk?f?TV
(6)
wheresk= fk−1
Borwein (spectral) methods (see [9] for details).
The minimization (6) can be viewed as a denoising subproblem
applied to sk, the next gradient descent iterate. In fact, this denoising
subproblem is of the form (5), where
ηk∇φ(fk). WechoosethescalarηkusingBarzilai-
? A = I,
With this identification, we can solve this optimization subproblem
with the method described in [7]. The process of approximating
(2) by the denoising subproblem (6) is iterated until suitable con-
vergence criteria are satisfied. We call this sequential quadratic pro-
gramming approach with a total variation penalty SPIRAL-TV.
b = sk,λ =2τ
ηk, and C = {f ∈ Rm: f ? 0}.
5. CONVERGENCE
In [9], we proved the following convergence result for the SPIRAL
algorithm when applied to a generalized version of the objective
function in (2).
Theorem 1. Given ? > 0, there exists some K? > 0 such that for all
k > K?, the primal-dual pair (fk,λk) nearly satisfies the Karush-
Kuhn-Tucker (KKT) optimality conditions associated with (2); that
is, for all k sufficiently large, we have
?∇Φ(fk) − λk?
<?
λk
fk
≥
≥
=
0
0
0.
(λk)Tfk
This theorem implies that for any arbitrarily small tolerance
level ?, we can satisfy the KKT conditions to within that tolerance if
the number of SPIRAL iterations, k, is sufficiently large. Note, as
claimed in the introduction, that the iterates are always feasible with
respect to the nonnegativity constraint fk≥ 0.
6. SIMULATIONS
Here we consider the application of our reconstruction method to
emission computed tomography (ECT). In medical ECT, a human
subject is injected with a radioactive pharmaceutical used to tag cer-
tain tissues or tumors. To obtain a mapping of the radiopharmaceu-
tical uptake, we detect photons emitted as the radiopharmaceutical
decays. From these projection data (the indirect projections y in our
problem), we wish to estimate the underlying radiopharmaceutical
distribution (the intensity f?). The probability transition matrix A is
4178

Page 3

(a)(b)(c)
Fig. 1. Simulated emission tomography experimental setup: (a) true
emission image, (b)noisy projection data, (c) estimate used as ini-
tialization.
derived from the physics and geometry of the detection device and
data collection process, and also incorporates the attenuation cor-
rection needed to account for the attenuation effects introduced by
bodily tissue [12].
The underlying 2-d intensity in our simulation is the 128 × 128
square image shown in Figure 1(a)); it is a standard test image in-
cluded in the Image Reconstruction Toolbox (IRT) by Fessler [13].
We consider the case where the tomographic projection corresponds
to a limited-angle parallel strip-integral geometry with 128 radial
samples and 128 angular samples spaced uniformly over 135 de-
grees. With these parameters, the resulting transition probability
matrix A was computed using the IRT software [13]. Ten realiza-
tions of the noisy tomographic data y are then simulated according
to the inhomogeneous Poisson process (1). We only show images
reconstructed using a particular realization of the data shown in Fig-
ure 1(b), the other nine realizations are used to examine the ten-trial
average performance of the reconstruction methods considered. In
the particular case shown, the noisy sinogram observations have a
total photon count of 2.0 × 105, a mean count over the support of
the tomographic projections of 18.08, and a maximum count of 44.
6.1. Comparison with other methods
To show the effectiveness of the TV regularization, we compare
the results of our SPIRAL-TV approach with the SPIRAL frame-
work applied to objectives resulting from two different regulariza-
tion methods. The first, a wavelet ?1-norm regularizer, reconstructs
an estimate?fwaveletaccording to the following objective:
?fwavelet= argmin
f
subject to
φ(f) + τ?WTf?1
f ? 0,
(7)
where WTf is the discrete wavelet transform of f. Methods for
solvingthisoptimizationaredescribedin[9]. Wedenotethismethod
as SPIRAL-?1. As seen below, this regularization can result in sig-
nificant artifacts corresponding to spurious non-zero wavelet coeffi-
cients.
The second regularization scheme is built on the framework of
recursive dyadic partitions (RDPs), which we summarize here and
are described in detail in [9]. It can be shown that partition-based
methodsarecloselyrelatedtoHaarwaveletdenoisingwithanimpor-
tant hereditary constraint placed on the thresholded coefficients—if
a parent coefficient is thresholded, then its children coefficients must
also be thresholded [8]. This constraint is akin to wavelet-tree ideas
which exploit persistence of significant wavelet coefficients across
scales and have recently been shown highly useful in compressed
sensing settings [4]. The hereditary constraint is also useful compu-
tationally since it admits an efficient dynamic programming imple-
mentation. Within this approach, it is straightforward to introduce
an efficient cycle-spinning strategy that yields a translationally in-
variant algorithm, boosting performance in practice. For brevity, we
only show results for this translationally-invariant method, denoted
SPIRAL-RDP-TI. While partition-based regularization yields accu-
rate results in many settings, it also results in a non-convex optimiza-
tion problem. The regularizer penalizes the number of cells in the
partition-based estimate, essentially an ?0 measure of sparsity. Al-
though RDP-based regularization is computationally tractable (un-
like ?0regularization), the non-convexity of the problem causes the
solution to depend heavily on the initialization and convergence to a
global minimizer cannot be guaranteed.
We also compare our SPIRAL approaches with two compet-
ing Poisson reconstruction methods. The first, denoted SPS-OS,
uses a separable paraboloidal surrogate with ordered subsets algo-
rithm [1]. The second, denoted EPL-INC-3, employs an incremental
penalized Poisson likelihood EM algorithm and was suggested by
Prof. Fessler as representative of the current state-of-the-art in emis-
sion tomographic reconstruction. In our experiments a roughness
penalty based on the Huber potential function yielded the best re-
sults for both of these methods and are shown below. Both of these
methods are available as part of the IRT [13]; specifically, we used
the pwls sps os and epl inc functions from the toolbox. In ad-
dition to these Poisson methods, we also compare to the SpaRSA
algorithm [5] which solves the traditional CS ?1-regularized least-
squares (?2-?1) problem. Both wavelet-based approaches (SPIRAL-
?1and SpaRSA) use the Daubechies-6 wavelet basis for W. As the
solution provided by SpaRSA is not constrained to be nonnegative,
we threshold the result to obtain a feasible – and therefore more ac-
curate – solution. Including this result allows us to demonstrate the
effectiveness of solving the optimization formulation that utilizes the
Poisson likelihood.
6.2. Algorithm setup
All of the methods considered here were initialized with the estimate
shown in Fig. 1(c). This initialization results from two iterations of
a non-convergent version of the EPL-INC-3 algorithm, itself initial-
ized by a filtered back-projection estimate. All algorithms executed
for a minimum of 50 iterations, and global convergence was declared
when the relative change in the iterates, ?fk+1− fk?2/?fk?2, fell
below a tolerance tolP = 5 × 10−4. The advantage of this crite-
ria is that it applies to general penalty functions. The disadvantage,
however, is that it is possible that the change between two consecu-
tive iterates may be small even though the iterates are still far from
the true minimizer. However, forcing the algorithms to perform a
suitable minimum number of iterations typically avoids any issues
with premature termination.
Lastly, in all of the experiments presented in this paper, we
chose any parameters associated with each algorithm (such as τ)
to minimize the root-mean-square error (RMSE (%) = 100 · ??f −
sible in practice, it does allow us to compare the best-case perfor-
mance of various algorithms and penalization methods. In prac-
tical settings, regularization parameters can be chosen via cross-
validation. This is particularly well-suited to many photon-limited
imagingapplicationsinwhicheachdetectedphotonhasatimestamp
associated with it; this timing information can be used to construct
multiple independent and identically distributed realizations of the
underlying Poisson process in software. The details of this proce-
dure are a significant component of our ongoing research.
f??2/?f??2) of the reconstruction. While this would not be pos-
4179

Page 4

(a) SPIRAL-TV
RMSE = 24.404%
(b) SPIRAL-?1
RMSE = 28.626%
(c) SpaRSA
RMSE = 31.172%
(d) SPIRAL-RDP-TI
RMSE = 27.557%
(e) EPL-INC-3
RMSE = 24.748%
(f) SPS-OS
RMSE = 27.555%
Fig. 2. Single-trial reconstructed images with RMSE values.
6.3. Results
The result of the ECT simulation described above are presented in
Figure 2. The TV-regularized result is highly accurate and has the
lowest RMSE among all the methods we considered. The wavelet-
regularized methods (SPIRAL-?1and SpaRSA) exhibit significantly
more spurious artifacts and have higher RMSE. The performance of
SpaRSA is particularly poor since it solves the conventional ?2-?1
minimization problem, which is unsuitable for Poisson image recon-
struction. The partition-based result fares better than the ?1-based
approaches; however it is oversmoothed and looses edge detail cap-
tured in the SPIRAL-TV result. The EPL-INC-3 method offers the
toughest competition to the RMSE achieved by the SPIRAL-TV
method, but there is a rough noise-like texture throughout the image,
causing poor reconstruction in homogeneous regions. The SPS-OS
method is ultimately limited by many streaking artifacts that cross
through the image. Both the single-trial RMSE, and a ten-trial av-
erage RMSE are reported in Table 1, showing the claims above are
robust with respect to different realizations of the data.
Single-Trial
RMSE (%)
Ten-Trial Average
RMSE (%) Method
SPS-OS
EPL-INC-3
SpaRSA
SPIRAL-?1
SPIRAL-TV
SPIRAL-RDP-TI
27.555
24.748
31.172
28.626
24.404
27.557
27.057
24.462
29.987
28.050
24.270
27.669
Table 1. Reconstruction RMSE for single-trial results, and results
averaged over ten trials. RMSE (%) = 100 · ??f − f??2/?f??2.
7. CONCLUSIONS
Using total variation to regularize solutions to Poisson inverse prob-
lems yields highly accurate estimates. The proposed approach out-
performs the current state-of-the-art approaches developed specifi-
cally for emission tomography. Particularly, it results in fewer spuri-
ous artifacts than wavelet-regularized methods, and unlike partition-
regularized methods, is theresult ofa convex optimizationprocedure
with provable convergence properties.
8. REFERENCES
[1] S. Ahn and J. Fessler, “Globally convergent image reconstruc-
tion for emission tomography using relaxed ordered subsets
algorithms,” IEEE Trans. Med. Imaging, vol. 22, no. 5, pp.
613–626, May 2003.
[2] R. Nowak and E. Kolaczyk, “A multiscale statistical frame-
work for Poisson inverse problems,” IEEE Trans. Info. Theory,
vol. 46, pp. 1811–1825, 2000.
[3] E. J. Cand` es and T. Tao, “Near-optimal signal recovery from
random projections: universal encoding strategies?,”
Trans. Inform. Theory, vol. 52, no. 12, pp. 5406–5425, 2006.
[4] R.G.Baraniuk, V.Cevher, M.F.Duarte, andC.Hegde, “Model-
based compressive sensing,”
2008, Submitted.
[5] S. Wright, R. Nowak, and M. Figueiredo, “Sparse reconstruc-
tion by separable approximation,” IEEE Trans. Signal Pro-
cess., vol. 57, pp. 2479–2493, 2009.
[6] T. Chan and J. Shen, Image Processing And Analysis: Vari-
ational, PDE, Wavelet, And Stochastic Methods, Society for
Industrial and Applied Mathematics, 2005.
[7] A. Beck and M. Teboulle,
for constrained total variation image denoising and deblurring
problems,” IEEE Trans. Image Processing, vol. 18, no. 11, pp.
2419–34, 2009.
[8] R. Willett and R. Nowak, “Multiscale Poisson Intensity and
Density Estimation,” IEEE Trans. Inform. Theory, vol. 53, no.
9, pp. 3171–3187, 2007.
[9] Z. T. Harmany, R. F. Marcia, and R. M. Willett, “SPIRAL
out of convexity: Sparsity-regularized algorithms for photon-
limited imaging,” in SPIE Computational Imaging, 2010.
[10] F.-X.Dupe, J.M.Fadili, andJ.-L.Starck, “Aproximaliteration
for deconvolving poisson noisy images using sparse represen-
tations,” IEEE Trans. Image Processing, vol. 18, no. 2, pp.
310–321, 2009.
[11] M. A. T. Figueiredo and J. M. Bioucas-Dias,
of Poissonian images using alternating direction optimization,”
IEEE Trans. Image Process., 2010, Accepted.
[12] Y. Vardi, L. A. Shepp, and L. Kaufman, “A statistical model
for positron emission tomography,” J. Amer. Statist. Assoc.,
vol. 80, pp. 8–37, 1985.
[13] J. Fessler et al., “Image reconstruction toolbox,” http://
www.eecs.umich.edu/˜fessler/code.
IEEE
IEEE Trans. Inform. Theory,
“Fast gradient-based algorithms
“Restoration
4180