Page 1

Journal of Microscopy, Vol. 234, Pt 1 2009, pp. 47–61

Received 21 April 2008; accepted 10 November 2008

Application of the split-gradient method to 3D image

deconvolution in fluorescence microscopy

G. VICIDOMINI∗,†,‡,§, P. BOCCACCI†,‡, A. DIASPRO‡,§

& M. BERTERO†

∗Department of NanoBiophotonics, Max Planck Institute for Biophysical Chemistry,

Am Fassberg 11, Goettingen, Germany

†DISI, University of Genoa, Via Dodecaneso 35, Genoa, Italy

‡LAMBS-MicroSCoBiO, University of Genoa, Via Dodecaneso 33, Genoa, Italy

§IFOM, Foundation FIRC Institute for Molecular Oncology, Via Adamelo 16, Milan, Italy

Key words. Deconvolution, edge-preserving, fluorescence microscopy,

Markov random field, split-gradient method.

Summary

The methods of image deconvolution are important for

improving the quality of the detected images in the

different modalities of fluorescence microscopy such as wide-

field, confocal, two-photon excitation and 4Pi. Because

deconvolution is an ill-posed problem, it is, in general,

reformulated in a statistical framework such as maximum

likelihood or Bayes and reduced to the minimization of

a suitable functional, more precisely, to a constrained

minimization, because non-negativity of the solution is an

important requirement. Next, iterative methods are designed

for approximating such a solution.

In this paper, we consider the Bayesian approach based

on the assumption that the noise is dominated by photon

counting, so the likelihood is of the Poisson-type, and that

the prior is edge-preserving, as derived from a simple Markov

random field model. By considering the negative logarithm

of the a posteriori probability distribution, the computation of

the maximum a posteriori (MAP) estimate is reduced to the

constrained minimization of a functional that is the sum of

the Csisz´ ar I-divergence and a regularization term. For the

solution of this problem, we propose an iterative algorithm

derived from a general approach known as split-gradient

method (SGM) and based on a suitable decomposition of the

gradient of the functional into a negative and positive part.

TheresultisasimplemodificationofthestandardRichardson–

Lucy algorithm, very easily implementable and assuring

automatically the non-negativity of the iterates. Next, we

applythismethodtotheparticularcaseofconfocalmicroscopy

for investigating the effect of several edge-preserving priors

proposed in the literature using both synthetic and real

Correspondence to: G. Vicidomini. Tel: +49 (0) 551 201 2615; fax: +49 (0) 551

201 2505; e-mail: gvicido@gwdg.de

confocal images. The quality of the restoration is estimated

bothbycomputationoftheKullback–Leiblerdivergenceofthe

restoredimagefromthedetectedoneandbyvisualinspection.

Itisobservedthatthenoiseartefactsareconsiderablyreduced

anddesiredcharacteristics(edgesandminutefeaturesasislets)

are retained in the restored images. The algorithm is stable,

robust and tolerant at various noise (Poisson) levels. Finally,

by remarking that the proposed method is essentially a scaled

gradient method, a possible modification of the algorithm is

briefly discussed in view of obtaining fast convergence and

reduction in computational time.

Introduction

Several approaches and algorithms have been proposed for

image deconvolution, an important topic in imaging science.

The purpose is to improve the quality of images degraded

by blurring and noise. The ill-posedness of the problem is

the basic reason of the profusion of methods: many different

formulations are introduced, based on different statistical

modelsofthenoiseaffectingthedata,andmanydifferentpriors

are considered for regularizing the problem, for instance, in

a Bayesian approach. As a result, deconvolution is usually

formulated in one of several possible variational forms,

and for a given formulation, several different minimization

algorithms, in general iterative, are proposed.

For the specific application to microscopy, we mention a

few methods, without pretending to be exhaustive. Under

the assumption of a Gaussian white noise, the approach

based on the Tikhonov regularization theory (Tikhonov &

Arsenin, 1977; Engl et al., 1996) and the methods proposed

for computing non-negative minimizers of the corresponding

functional, such as the method of Carrington (1990) or the

iterativemethodproposedinvanderVoort&Strasters(1995),

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society

Page 2

48 G. VICIDOMINI ET AL.

are worth mentioning. On the other hand, in the case of

Poisson statistics (noise dominated by photon counting), the

classicalRichardson–Lucy(RL)algorithm(Richardson,1972;

Lucy, 1974), derived from a maximum likelihood approach

(Shepp & Vardi, 1982), can be routinely used. A quantitative

comparison of these methods is given in van Kempen et al.

(1997) and van Kempen & van Vliet (2000b). They were

applied mainly to wide-field and confocal microscopy, but

the cases of the 4Pi and two-photon excitation microscopy

were also investigated (Schrader et al., 1998; Mondal et al.,

2008). We also remark that in some applications, the images

are affected by both Poisson and Gaussian read-out noise.

For treating this case, a more refined model was proposed by

Snyder et al. (1993), with a related expectation maximization

(EM) algorithm. However, as confirmed by a recent analysis

(Benvenuto et al., 2008), this model improvement does not

produceasignificantimprovementintheimagerestoration,so

theassumptionofPoissonstatisticsis,ingeneral,satisfactory.

As is known, early stopping of the RL method (RLM)

provides a regularization effect (Bertero & Boccacci, 1998),

even if in some cases, the restoration is not satisfactory. To

this purpose, a regularization of the Poisson likelihood by

means of the Tikhonov functional is proposed in Conchello &

McNally(1996).ThisapproachisknownastheRL–Conchello

algorithm(vanKempen&vanVliet,2000a).Inamorerecent

paper,Deyetal.(2006)investigatetheregularizationbasedon

the total variation (TV) functional, introduced by Rudin et al.

(1992) for image denoising. They also propose an iterative

algorithm derived from the one-step-late (OSL) method of

Green (1990). OSL is introduced as a modified EM algorithm,

but its structure is that of a scaled gradient method, with

a scaling that is not automatically positive. For this reason,

convergence can be proved only in the case of a sufficiently

small value of the regularization parameter (Lange, 1990).

In this paper, we consider the regularization of the Poisson

likelihood in the framework of a Bayesian approach. In

general, regularization can be obtained by imposing a priori

information about properties of the object to be restored, and

the Bayesian approach enables integration of this available

prior knowledge with the likelihood using Bayes’ law; then,

the estimate of the desired object is obtained by maximizing

the resulting posterior function. This approach is called the

maximum a posteriori (MAP) method and can be reduced to

thesolutionofaminimizationproblembytakingthenegative

logarithm of the posterior function.

In general, the prior information can be introduced

by regarding the object as a realization of a Markov

random field (MRF) (Geman & Geman, 1984); then, the

probability distribution of the object is obtained using the

equivalence of MRF and Gibbs random fields (GRF) (Besag,

1974). In particular, the potential function of the Gibbs

distribution can be appropriately chosen to bring out desired

statistical properties of the object. The combination of

MRF and MAP estimation is extensively studied in single

photonemissioncomputedtomography(SPECT)andpositron

emission tomography (PET) image reconstruction (Geman &

McClure, 1985; Green, 1990). Recently, it was demonstrated

that such a combination provides a powerful framework also

for three-dimensional (3D) image restoration in fluorescence

microscopy (Vicidomini et al., 2006; Mondal et al., 2007).

A simple and well-known regularization is based on the

assumptionthatobjectsaremadeofsmoothregions,separated

by sharp edges. This is called edge-preserving regularization

and requires non-quadratic potential functions. Therefore,

in this paper, we consider the regularization of the negative

logarithmofthePoissonlikelihoodbymeansofdifferentedge-

preserving potential functions proposed by different authors

(see, for instance, Geman & Geman, 1984; Charbonnier et al.,

1997).

In view of the application to fluorescence microscopy, we

must consider the minimization of this functional on the

convex set of the non-negative images (the non-negative

orthant) and, in order to overcome the difficulties of the OSL

algorithm,weinvestigatetheapplicabilityofthesplit-gradient

method (SGM) to this problem. SGM, proposed by Lant´ eri

etal.(2001,2002),isageneralapproachthatallowsdesigning

of iterative algorithms for the constrained minimization of

regularizedfunctionalsbothinthecaseofGaussianandinthe

caseofPoissonnoiseaswellasinthecaseinwhichbothnoises

are present (Lant´ eri & Theys, 2005). The general structure is

thatofascaledgradient method,withascaling thatisalways

strictly positive. Therefore, from this point of view, SGM is

superior to the OSL method.

Finally, we point out that, thanks to SGM, the edge-

preserving regularizations investigated in this paper can also

be easily applied to the case of the least-square problem (i.e.

additive Gaussian noise); hence, we provide an approach

to image deconvolution both for microscopy techniques in

whichPoissonnoiseisdominant,suchasconfocalmicroscopy,

and for techniques in which a Gaussian noise can be more

appropriate, such as wide-field microscopy.

The paper is organized as follows. In ‘The edge-preserving

approach’,webrieflyrecallthemaximumlikelihoodapproach

inthecaseofPoissonnoise,itsreductiontotheminimizationof

Csisz´ ar I-divergence (also called Kullback–Leibler divergence)

and the RL algorithm; moreover, we introduce the edge-

preserving potentials considered in this paper. In ‘Split-

gradient method (SGM)’, we give SGM in the simple case

of step length 1. This form can be easily obtained (Bertero

et al., 2008) as an application of the method of successive

approximations to a fixed-point equation derived from the

Karush–Kuhn–Tucker (KKT) conditions for the constrained

minimizersofthefunctionalandbasedonasuitablesplittingof

thegradientintoapositiveandnegativepart.Theconvergence

of this simplified version of SGM is not proved, even if it

has always been verified in numerical experiments. However,

convergence can be obtained by a suitable search of the step

length(Lant´ erietal.,2002)orbyapplyingarecentlyproposed

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 3

SPLIT-GRADIENT METHOD IN FLUORESCENCE MICROSCOPY DECONVOLUTION 49

scaled gradient projection (SGP) method (Bonettini et al.,

2007).TherelationshipbetweenthesimplifiedversionofSGM

and OSL is also shown. Moreover, we determine the splitting

of the gradient for the edge-preserving potentials introduced

in the previous section. In ‘Numerical results’, we present the

results of our numerical experiments in the case of confocal

microscopyinwhichphotoncountingnoiseisdominant,and

therefore, a Poisson noise assumption is more appropriate.

We compare the effect of the different potential functions

usingasimple3Dphantomconsistingofsphereswithdifferent

intensitiesandconcludethathigh-qualityrestorationscanbe

obtained with the hyper-surface potential (Charbonnier et al.,

1994)andtheGeman–McClurepotential(Geman&McClure,

1985), both being superior to the quadratic potential (the

Tikhonov regularization). The estimates of the parameters

derived from these simulations are used for a deconvolution

of real images. In particular, the improvement provided by

the deconvolution method is proved by comparing an image

obtainedwithahighnumericalaperture(NA)andtherestored

image obtained from a low NA image of the same object. In

‘Concluding remarks’, we give conclusions.

The edge-preserving approach

Inthissection,webrieflysummarizethemaximumlikelihood

approachinthePoissonnoisecaseandtheBayesianapproach

and introduce the edge-preserving regularization functionals

investigated in this paper.

Maximum likelihood approach

We use bold letters for denoting N1× N2× N3cubes, whose

voxels are labelled by a multi-index n = (n1, n2, n3). If g (n) is

the value of the image at voxel n, then we can write

g(n) = gobj(n) + gback(n),

(1)

wheregobj(n)isthenumberofphotoelectronsduetoradiation

from the object (specimen), and gback(n) is the number of

photoelectrons due to background.

If we assume that noise is dominated by photon counting,

these two terms are realizations of independent Poisson

processes Gobj(n) and Gback(n) so that their sum G(n) =

Gobj(n) + Gback(n) is also a Poisson process; its expected value

is given by

E{G(n)} = E{Gobj(n) + Gback(n)} = (Af)(n) + b(n), (2)

where f is the object array, b is the expected value of the

background (usually constant over the image domain) and

A is the imaging matrix, and under the linear and space-

invariantassumption,istheconvolutionmatrixrelatedtothe

point spread function (PSF) k by (Bertero & Boccacci, 1998)

Af = k ∗ f.

(3)

Because the Poisson processes related to different voxels

are statistically independent, the probability distribution of

the multi-valued random variable G = {G(n)}, PG(g|f) is the

productoftheprobabilitydistributionsoftherandomvariables

G(n), and therefore, the likelihood function is given by

?

LG

g(f) = PG(g|f) =

n

e−[(Af)(n)+b(n)]((Af)(n) + b(n))g(n)

g(n)!

,

(4)

where g is the detected image.

An estimate of the unknown object f corresponding to the

image g is defined as any maximum point f∗of the likelihood

function.Itiseasytodemonstratethatthemaximizationofthe

likelihood (ML) function is equivalent to the minimization of

theCsisz´ arI-divergence(Csisz´ ar,1991)(alsocalledKullback–

Leiblerdivergence(Kullback&Leibler,1951))ofthecomputed

image Af +b (associated with f) from the detected image g,

J0(f;g) = DK L(g,Af + b)

?

+[(Af)(n) + b(n) − g(n)]

which is a convex and non-negative function.

An iterative method for the minimization of J0(f; g) was

proposed by several authors, in particular, by Richardson

(1972)andLucy(1974)forspectraandimagedeconvolution

and by Shepp & Vardi (1982) for emission tomography. As

shown in (Shepp & Vardi, 1982), this method is related to a

general approach for the solution of ML problems, known as

EM.Forthesereasons,thealgorithmisknownasbothRLand

EM method. The very same method was adopted by Holmes &

Liu(1989)andHolmes(1988)for3Dflorescencemicroscopy.

Inthispaper,werefertoitastheRLM.Fortheconvenienceof

the reader, we recall that it is as follows:

- choose an initial guess f(0)such that

=

n

?

g(n)ln

g(n)

(Af)(n) + b(n)

?

,

(5)

f(0)(n) > 0 and(6)

- given f(i), compute

f(i+1)= f(i)

?

AT

g

Af(i)+ b

?

.

(7)

It is evident that each approximation f(i)is non-negative. For

b=0,severalproofsoftheconvergenceofthealgorithmtoan

MLsolutionareavailable.Moreover,ifthePSFkisnormalized

in such a way that the sum of all its voxel values is 1, then for

each iteration, the following property is true:

?

This property is also called flux conservation because

it guarantees that the total number of counts of the

reconstructedobjectcoincideswiththetotalnumberofcounts

of the detected image. It is not satisfied in the case b ?= 0.

Moreover, the convergence of the algorithm seems not to be

proved in such a case.

n

f(i)(n) =

?

n

g(n).

(8)

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 4

50 G. VICIDOMINI ET AL.

Edge-preserving Bayesian approach

TheMLestimateisaffectedbylargenoiseartefactsthatbecome

evident at large number of RLM iterations. In general, early

stopping is used for regularizing the solution. However, ML

does not incorporate prior knowledge about the unknown

object. This can be done in a Bayesian approach.

Indeed, it is assumed that for each n, f(n) is a realization

of a random variable F(n). We denote by F the set of the F(n)

and by PF(f) its probability distribution, the so-called prior. If

PG(g | f) is interpreted as the conditional probability of G for

a given value f of F, then Bayes’ formula allows computing of

PF(f | g), that is, the conditional probability of F for a given

value g of G. Finally, the posterior probability distribution of F

is obtained from PF(f | g) when G is the detected image. The

final result is given by

PF

g(f) = LG

g(f)PF(f)

PG(g),

(9)

where PG(g) istheprobability distribution of G.The posterior

function PF

the observed data is g.

Maximizationoftheposterior

minimizations of functionals of the following form:

g(f) gives the probability for an object f, given that

functionleads to

Jμ(f;g) = J0(f;g) + μJR(f),

(10)

where, in the case of Poison noise, J0(f; g) is given by Eq. (5),

JR(f) is a regularization functional (called also regularization

term) coming from the negative logarithm of the prior and μ

is a hyper-parameter that usually is called the regularization

parameter. A minimum of this functional is generally termed

as the MAP estimate of the unknown object. Note that in the

case μ = 0, the MAP problem becomes the ML problem.

We conclude by remarking that it is not obvious that a

minimum point f∗of Jμ(f; g) is a sensible estimate of the

unknown object f. In fact, in this formulation, we have

a free parameter μ. In the classical regularization theory

(Tikhonov & Arsenin, 1977), a wide literature exists on the

problem of the optimal choice of this parameter (Engl et al.,

1996; van Kempen & van Vliet, 2000b), but as far as we

know, this problem has not yet been thoroughly investigated

in the more general framework provided by the Bayesian

regularization.

A wide class of priors is provided by MRF, first introduced

into image processing by Geman & Geman (1984). The key

result concerning MRF is the Hammersley–Clifford theorem,

stating that F is an MRF if and only if it is a suitable GRF so

that its probability distribution is given by (Besag, 1974)

PF(f) =1

Zexp

?

−1

β

?

c∈C

ϕc(f)

?

,

(11)

where Z is a normalization constant, β is the Gibbs hyper-

parameter, C is the collection of all possible cliques associated

to the MRF (we recall that a clique is either a single site or a

set of sites such that each one of them is a neighbour of all the

other sites in the set.) and ϕcis a clique potential that depends

only on the values of f in the sites of the clique c. With such a

choice oftheprior, theregularization termin Eq. (10)is given

by

JR(f) =

?

c∈C

ϕc(f),

(12)

andtheregularizationparameterμbecomesequalto1/β.

In general, clique potentials are functions of a derivative

of the object (Geman & Reynolds, 1992). The order of

the derivative and the neighbourhood system are chosen

depending on the kind of object that is sought. In this work,

we assume that the object consists of piecewise smooth

regionsseparatedbysharpedgesandusefirst-orderdifferences

between voxels belonging to the double-site 3D clique shown

in Fig. 1 so that Eq. (12) becomes

JR(f) =1

4

?

n

?

m∈Nn

ϕ

?f(n) − f(m)

d(n,m)δ

?

,

(13)

where Nnis the set of the first neighbours of the voxel n as

derivedfromFig.1;δ andd(n,m)aretwoscalingparameters,

the first tuning the value of the gradient above which a

discontinuity is detected, the second taking into account the

differentdistancesbetweenthevoxelsofaclique.Forexample,

inarealcaseinwhichsamplinginthelateraldirection(deltaxy)

isdifferentfromsamplingintheaxialdirection(deltaz),d(n,m)

Fig. 1. (A) The set of neighbours of a voxel n = (n1, n2, n3) used in this paper and (B) the corresponding double-site cliques.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 5

SPLIT-GRADIENT METHOD IN FLUORESCENCE MICROSCOPY DECONVOLUTION 51

Table1. Potentialfunctionsconsideredinthispaper:ϕQ=quadraticpotential;ϕGM=Geman–

McClurepotential;ϕHL=Hebert–Leahypotential;ϕHB=Huberpotential;ϕHS=hyper-surface

potential.

Potential

function

Expression of ϕ(t) Expression of

ψ(t) = ϕ?(t)/2t

Reference

ϕQ

t2

1(Charbonnier et al., 1997)

ϕGM

t2

1 + t2

log (1 + t2)

?

2|t| − 1,

2√1 + t2− 2

1

(1 + t2)2

1

1 + t2

?

1/|t|,

1

?

(Geman & McClure, 1985)

ϕHL

(Hebert & Leahy, 1989)

ϕHB

t2,

|t| ≤ 1

|t| > 1

1,

|t| ≤ 1

|t| > 1

(Schultz & Stevenson, 1995)

ϕHS

1 + t2

(Charbonnier et al., 1994)

are defined as follows:

d(n,m) =

⎧

⎪⎪⎩

Higher-orderneighbourhoodsystems,combinedwithsecond-

order derivatives, promote the formation of piecewise linear

areas in the solution; combined with third-order derivatives,

they promote piecewise planar areas (Geman & Reynolds,

1992).

In this work, we refer to ϕ as the potential function. Its form

takes a key part in the estimation of the object to be restored.

Table 1 shows different potential functions proposed in

the literature and their corresponding weighting functions

ψ(t)=ϕ?(t)/2t,whereϕ?standsforthefirstderivativeofϕ.In

the standard Tikhonov regularization (Tikhonov & Arsenin,

1977), the potential function takes the pure quadratic form

ϕQ.

It is well known that quadratic regularization imposes

smoothness constraint everywhere; because large gradients

in the solution are penalized, the result is that edges are

completelylost.Toovercomethisproblem,aseriesofpotential

functions were proposed. In recent years, the problem such

as ‘What properties must a potential function satisfy to

ensure the preservation of edges?’ has provided abundant

literature (see, for instance, Geman & McClure, 1985; Hebert

&Leahy,1989;Schultz&Stevenson,1995).Tounifyallthese

approaches,Charbonnieretal.(1997)proposedasetofgeneral

conditions that the edge-preserving regularization potential

function should satisfy, independently of the algorithm used

to minimize Eq. (10). For the convenience of the reader, these

conditions are summarized below:

(a) ϕ(t) ≥ 0 ∀t and ϕ(0) = 0;

(b) ϕ(t) = ϕ(− t);

⎪⎪⎨

1,

√2,

m ∈?(n1± 1,n2,n3),(n1,n2± 1,n3)?

m ∈?(n1,n2,n3± 1)?.

m ∈?(n1± 1,n2± 1,n3)?

deltaz/deltaxy,

(14)

(c) ϕ continuously differentiable;

(d) ϕ?(t) ≥ 0 ∀t ≥ 0;

(e) ψ(t) =ϕ?(t)

∞);

(f) limt→+∞ψ(t) = 0 and

(g) limt→0ψ(t) = M, 0 < M < +∞.

Allthepotentialfunctionstestedinthiswork(seeTable1)are

in complete agreement with these conditions, except ϕHB. In

fact, the weighting function of ϕHBis not strictly decreasing

on [0, +∞), as required by condition (e), but only for t >

1. However, tests on both synthetic and real data show that

ϕHBalso leads to well-preserved edge restorations. Therefore,

we believe that condition (e) can be weakened and must be

satisfied only for sufficiently large values of t.

2tcontinuous and strictly decreasing on [0, +

Split-gradient method (SGM)

An iterative method for the minimization of Eqs. (10), (5) and

(13), with the additional constraints of non-negativity and

flux conservation,

f(n) ≥ 0,

?

n

f(n) =

?

n

{g(n) − b(n)}.= c,

(15)

can be obtained by means of a general approach known as

theSGM,proposedbyLant´ erietal.(2001,2002).Themethod

provides iterative algorithms that, in general, are not very

efficient(theyrequirealargenumberofiterations)buthavea

verynicefeature:forverybroadclassesofregularizationterms,

theycanbeobtainedbymeansofaverysimplemodificationof

RLMandthereforecanbeveryeasilyimplemented.Therefore,

itispossibletocompareinasimplewaytheresultsobtainable

by means of different regularization terms. In particular, we

will apply the method to all the regularization terms derived

from the potential functions of Table 1.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 6

52 G. VICIDOMINI ET AL.

The SGM is based on the following decomposition of the

gradient of the functional of Eq. (10):

−∇fJ0(f;g) = U0(f;g) − V0(f;g),

−∇fJR(f) = UR(f) − VR(f),

whereU0(f ;g)andV0(f ;g)arepositivearraysdependingonf

andg,andUR(f)andVR(f)arenon-negativearraysdepending

on f. It is obvious that such a decomposition always exists but

isnotunique.Thegeneralstructureoftheiterativealgorithm,

as described in (Lant´ eri et al., 2001), is as follows:

- chooseaninitialguessf(0)>0,satisfyingtheconstraintsof

Eq. (15);

- given f(i), compute

?U0(f(i);g) + μUR(f(i))

˜ c(i+1)=

n

(16)

˜f(i+1)= f(i)

V0(f(i);g) + μVR(f(i))

˜f(i+1)(n) and

?

,

?

(17)

- set

f(i+1)=

c

˜ c(i+1)˜f(i+1).

Itisimportanttoremarksomeinterestingfeatures.Thefirstis

thatalltheiteratesareautomaticallynon-negativeiftheinitial

guessf(0)isnon-negative,asonecaneasilyverify.Thesecond

is that the algorithm is a scaled gradient method, with step

size 1, because the update step can be written in the following

form:

˜f(i+1)= f(i)− Si?

where

?

∇fJ0(f(i);g) + μ∇fJR(f(i))

?

,

(18)

Si= diag

f(i)(n)

?V0(f(i);g) + μVR(f(i))?(n)

?

.

(19)

The convergence of the algorithm has not been proved in

general, but only in some particular cases (Lant´ eri et al.,

2001); nevertheless, it has been verified experimentally in

all the applications that we have considered. Moreover, it is

not difficult to prove that if the sequence of the iterates is

convergent, then the limit is a constrained minimum point of

the functional in Eq. (10) (Bertero et al., 2008).

In the case with PSF k normalized in such a way that the

sum all its voxel values is 1, a possible SGM decomposition of

∇fJ0(f; g), both for the Poisson and the additive Gaussian

noise cases, is given by Bertero et al. (2008). Because we

are mainly interested in the Poisson case, we report only the

corresponding decomposition

g

Af + b,

where1isthearraywhoseentriesareallequalto1.Therefore,

ifweinserttheseexpressionsinEq.(17)withμ=0,weobtain

RLM, with the flux conservation constraint (Eq. (7)).

U0(f;g) = AT

V0(f;g) = 1,

(20)

The gradient of the regularization term JR(f) of Eq. (13) is

given by

1

δ2

m∈Nn

(∇fJR(f))(n) =

?

f(n) − f(m)

d(n,m)2ψ

?f(n) − f(m)

d(n,m)δ

?

.

(21)

Using conditions (b) and (d), it is easy to demonstrate that

ψ(t) ≥ 0 for all t. Thus, by taking into account that we are

considering the restriction of the regularization functional

JR(f) to the closed and convex set of the non-negative arrays,

we can choose the following decomposition of the gradient:

1

δ2

m∈Nn

1

δ2

m∈Nn

In conclusion, the iterative step of Eq. (17) becomes

f(i)

1 + μVR(f(i))

BylookingatEq.(7),weseethatthisregularizedalgorithmisa

simplemodificationoftheRLM.Aswehavealreadyremarked,

even if we do not have a proof of convergence, convergence

has always been observed in our numerical experiments. In

any case, convergence can be obtained by a suitable line

search (Lant´ eri et al., 2002) because, as shown in Eq. (18),

this algorithm is a scaled gradient method with step length 1.

ItmaybeinterestingtocompareouralgorithmwiththeOSL

algorithm proposed by Green (1990) that, in our notations, is

given by

f(i)

1 + μ∇fJR(f(i))AT

We note that in our algorithm, the denominator is strictly

positive (in fact ≥1) for any value of μ, whereas this is not

true for OSL. Indeed, it is known that OSL may not converge

(Pierro & Yamagishi, 2001) and may not produce images

with non-negative intensities. This is mainly due to possible

oscillations of the gradient of the regularization terms, which

can be amplified by the regularization parameter. Because

OSL is also a scaled gradient method with step length 1,

in Lange (1990), a line search was added to the algorithm,

proving convergence and non-negativity of the iterates in

the case of sufficiently small values of μ. To solve this

problem,theoriginalalgorithmwasfurthermodified(Gravier

& Yang, 2005; Mair & Zahnen, 2006) to ensure positivity of

the estimates and (numerical) convergence of the negative

logarithm of the posterior function; however, these methods

lead to cumbersome steps in the algorithm.

(UR(f))(n) =

?

?

f(m)

d(n,m)2ψ

?f(n) − f(m)

?f(n) − f(m)

d(n,m)δ

?

?

,

(VR(f))(n) =

f(n)

d(n,m)2ψ

d(n,m)δ

.

(22)

˜f(i+1)=

?

AT

g

Af + b+ μUR(f(i))

?

.

(23)

˜f(i+1)=

g

Af + b.

(24)

Numerical results

Weapplythepreviousmethodtotheparticularcaseofconfocal

scanning laser microscopy (CSLM) in which Poisson noise

assumption is more realistic.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 7

SPLIT-GRADIENT METHOD IN FLUORESCENCE MICROSCOPY DECONVOLUTION 53

Point spread function model

For CLSM, the 3D PSF is defined by

kCLSM(x, y,z) =?R(x, y) ∗ |− →

where R describes the geometry of the pinhole, ∗ is the

convolution operator,− →

hλexis the time-independent part of

the electric field generated by the excitation laser light (at

wavelengthλex)inthefocusoftheobjectiveand− →

to− →

hλex, but for the emission light (at wavelength λem).

The electric field distribution− →

aberration-free lens is given in Richards & Wolf (1959). This

distributiondependsonthewavelengthofthelight,theNAof

the lens, and the refractive index of the medium in which the

lens works. Note that in this work, we use the well-developed

vectorialtheoryoflight,becauseforrealdataexperiments,we

use high NA objectives.

hλem(x, y,z)|2?

×|− →

hλex(x, y,z)|2,

(25)

hλemissimilar

h in the focal region for an

Simulated data

A first experiment is the restoration of a synthetic image

with simulated degradation. Figure 2(A) represents a sphere

containing five ellipsoids with different sizes and intensities

(128 × 128 × 64 voxels; intensity of the larger sphere: 100

units,largerellipsoid:200units,otherstructures:255units).

To simulate CLSM image formation, the object is convolved

withaconfocalPSF,aconstantbackgroundof5unitsisadded

and finally the result is corrupted by Poisson noise (Fig. 2).

Confocal PSF is computed using Eq. (25) with the following

acquisitionparameters:excitationwavelengthλex=488nm;

emissionwavelengthλem=520nm;NAoftheobjective=1.4;

refractiveindexoftheoilinwhichtheobjectiveisembedded=

1.518; diameter of the circular pinhole = 1 Airy unit, which

corresponds to a backprojected radius on the sample of

∼220 nm.

The Nyquist samplings for a confocal system under the

acquisition parameters described above are ∼150 nm along

the optical axis and ∼50 nm along the lateral axis. In this

simulation, we oversample the object with respect to the

Nyquist criteria, assuming deltaey = 35 nm in the lateral

direction and deltaz = 105 nm in the axial direction. To

changethesignal-to-noiseratio(SNR)ofthesimulatedimage,

we change τ, the reciprocal of the photon conversion factor.

We assume that the noise is described by a Poisson process,

implyingthattheSNRinfluorescenceimagingdependssolely

on the total number of detected photons. For this reason,

we choose the mean of the Poisson process equal to τ(Af +

b), with fixed intensities of Af + b. Thus, by increasing τ,

the average number of detected photons increases and hence

the noise level decreases. In the real life, photon conversion

factor is determined by several multiplicative factors such as

integrationtimeandquantumefficiencyofthedetector(Jovin

Fig. 2. Slice view (slice no. 33) and two axial views along the dotted lines of the original object (A), simulated confocal image (B) and restored objects

using RLM (C), quadratic potential (D), Geman–McClure potential (E), Hebert–Leahy potential (F), Huber potential (G) and hyper-surface potential (H).

The simulated confocal image is corrupted by Poisson noise, with a value τ = 0.46 for the reciprocal of the photon conversion factor, corresponding to

SNR = 20; this means that the data are normalized to a maximum of 100 counts before the Poisson noise filter is applied.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 8

54 G. VICIDOMINI ET AL.

&Arndt-Jovin,1989).TherelationbetweenSNRandτ isgiven

by

SNR = 10log?τ max

For a numerical comparison of the quality of the restored

images obtained with the proposed SGM method, coupled

with the different potential functions, we use the Kullback–

Leibler divergence of the restored object from the true one,

which is the best measure in the presence of a non-negativity

constraint(Csisz´ ar,1991).WerecallthattheKullback–Leibler

divergence(KLD)ofa3Dimageqfroma3Dimagerisdefined

by

K LD(r,q) =1

N

n

where N = N1× N2× N3. Note that KLDis non-symmetric,

that is, KLD(r, q) ?= KLD(q, r).

InFig.3,foreachproposedalgorithm,weplotthebehaviour

of KLD(f, f(i)) as a function of the number of the iterations i

(SNR=20).NumericalinstabilityofRLMisevident.Moreover,

itisimportanttonotethatthereisaminimumalsointhecase

of GM and HL (semiconvergence of the method), whereas HB,

HS and Q are convergent.

TheKullback–Leiblerdivergencecanalsobeusedforfinding

asuitablecriterionforstoppingtheiterations,especiallyinthe

case of algorithms with a convergent behaviour. Indeed, we

stop the iterations when the difference between the values

of KLD(f, f(i)) corresponding to two consecutive iterations is

smaller that a certain threshold T

n

(Af + b)(n)?.

(26)

?

?

r(n)lnr(n)

q(n)+ [q(n) − r(n)]

?

, (27)

KLD(f,f(i)) − K LD(f,f(i+1)) ≤ T.

A simple rule for choosing T does not exist, and its choice

follows from our numerical experience. In this work, we

use T = 10−6as a reliable value for obtaining satisfactory

(28)

Fig. 3. Behaviour of the KLDof the iterates f(i)from the original object f

as a function of the number of iterations for RLM and for the algorithms

regularized with the potential functions of Table 1 (SNR = 20).

reconstructions. In the case of convergence (Q, HB and HS),

thismeansthatthereconstructedsolutionˆf andtheassociated

K LD(f,ˆf) value do not change much after the threshold has

been reached.

Asalreadydiscussed, all

functions contain a scaling parameter δ. Thus, for an

accurate quantitative evaluation of the proposed regularized

algorithms, it is necessary to compare their results using

appropriate values of the parameters β and δ. Again, the KLD

criterion is used for obtaining these values that we define as

those providing the minimum of K LD(f,ˆf). Figure 4 shows,

foreachpotentialfunction,thebehaviourofthe K LD(f,ˆf)asa

functionoftheregularizationparameterβ fordifferentvalues

ofthescalingparameterδinthecaseoftheobjectofFig.2with

SNR = 20. In order to reduce the computational burden, the

iterations,initializedwithauniformimage,arestoppedwitha

threshold T =10−4.InTable2,wereportthevaluesobtained

with this procedure.

The reconstructed imagesˆf of the object of Fig. 2, with

different SNR values, are computed using the optimal values

of Table 2 and a threshold T = 10−6. In order to measure the

quality of the reconstructions, we introduce an improvement

factor (IF) defined by

edge-preserving potential

I F =K LD(f,g) − K LD(f,ˆf)

K LD(f,g)

.

(29)

IF measures the relative global improvement provided by the

reconstructed imageˆf with respect to the initial raw image

g in terms of the KLD divergence from the true object. It is

equalto1inthecaseofaperfectreconstruction.Itsvaluesare

reported in Table 3. As expected, they increase when the SNR

increases.Theyindicatethataconsiderableimprovementcan

be provided by the use of a reconstruction method; however,

they do not discriminate significantly between the different

methods. In Table 4, we also report the number of iterations

and the computational cost per iteration in C++ software

environment (Version 6.0, Microsoft Corporation, USA). An

HP workstation XW4200 (Hewlet-Packard Company, Palo

Alto, CA, USA) Pentium(R)4 CPU 3.40 GHz equipped with

2.00 GB of RAM is used for reconstruction. All computations,

including the simulation of the image formation process (i.e.

blurringandnoise),areperformedwithdoubleprecision.The

asterisk indicates the cases in which the threshold is not

reached within the maximum number of iterations allowed

bythecode.Weseethatthecomputationalcostsperiteration

forthedifferentalgorithmsarecomparable.

The parameter IF may be a good choice as a global

measure of the quality of a restoration, but it may not

be able to detect small artefacts that may be important

for deciding the reliability of the reconstruction. Thus, the

previous comparisons are only the first step in determining

which algorithm is the most suitable for a given application.

In order to complete our analysis, we also considered

visual inspection as direct evidence for quantifying the

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 9

SPLIT-GRADIENT METHOD IN FLUORESCENCE MICROSCOPY DECONVOLUTION 55

Fig. 4. Relationship between the parameters β and δ (SNR = 20).

Table 2. Optimal values of the parameters δ and β for the different

regularization potentials and different values of SNR.

NoiseQ GMHL HBHS

SNR

τδβδβδβδβδβ

15

20

25

0.145

0.461

1.457

2

2

2

2250

5000

15000

4

4

5

20

25

20

1.5

1.5

2

125

150

125

0.5

1

1.5

500

700

600

0.5

0.5

0.5

600

1200

1800

Table 3. Values of the improvement factor IF corresponding to the

reconstructions provided by the different methods and for different SNR

values. For each SNR, the table reports both the mean value (upper

lines) and the standard deviation (lower lines) obtained with 10 different

realizations of noise.

SNR RLMQ GMHLHB HS

15 0.7187

±0.0014 ±0.0012 ±0.0017 ±0.0031 ±0.0021 ±0.0012

0.7409 0.78780.8381

±0.0012 ±0.0008 ±0.0031 ±0.0021 ±0.0020 ±0.0029

0.77450.81400.8667

±0.0011 ±0.0011 ±0.0014 ±0.0022 ±0.0009 ±0.0012

0.77270.80860.80170.8321 0.8315

200.83480.85720.8581

250.86220.88480.8857

quality of the 3D reconstructions. It is evident that RLM-

restored images (Fig. 2C) suffer severely from noise artefacts,

known as the checkerboard effect (many components of the

solution are too small, whereas others grow up), whereas

the smoothing nature of the quadratic potential is evident

inQ-restoredimages(Fig.2D).Alltherestorationsusingedge-

preserving potential functions (Figs. 2E–H) exhibit excellent

performance in both suppressing noise effect and preserving

edges: the corresponding restored images are free from the

unfavourable oversmoothing effect and the checkerboard

effect. It is important to note that the GM- (Fig. 2E) and HL-

restored(Fig.2F)imagesareverysimilar.Asimilarbehaviour

can be observed by comparing HB- and HS-restored images

(Fig. G and H, respectively). These results are in complete

agreement with the values of IF. It is also interesting to

remarkthatGMandHLrestorationspresentverysharpedges

in comparison to HB and HS restorations. This behaviour

becomes more evident from line plot investigations. Figure

5 shows the intensity profiles along pre-defined lines through

the original object and the restored images, both for lateral

(Figs. 5A–C) and for axial views (Figs. 5D and E). It is evident

that the profiles of the edge-preserving reconstructions have

a much better agreement with the profiles of the synthetic

object. The insets of Figs. 5(E) and (F) show in detail the edge-

preserving capability of the different potential functions and

revealclearlytheverysharpresultsobtainedwithGMandHL

in comparison to HB and HS.

However, the capability of GM and HL to obtain very sharp

edges represents also the limit of these potential functions. As

it is possible to observe in the restored simulated images, GM

(Fig. 2E) and HL (Fig. 2F) tend to produce blocky piecewise

regions that are not present in the original object. This effect,

also called the staircase effect, is thereby reduced in HB- (Fig.

2G)andHS-(Fig.2H)restoredimages.Inotherwords,HLand

GM produce very sharp edges but also introduce ‘false’ edges

in the restoration.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 10

56 G. VICIDOMINI ET AL.

Table 4. Number of iterations used for the optimal reconstructions provided by the different

methods. The asterisk indicates the cases in which the convergence was not reached before

the maximum allowed number of iterations (1000). The table reports the mean value and the

standard deviation obtained by means of ten different realizations of noise. For each method,

we also give the computational time per iteration.

SNRRLMQ GM HLHBHS

15

20

25

37 ± 1

76 ± 1

165 ± 3

1.82

1000∗

1000∗

1000∗

2.35

192 ± 6

355 ± 15

404 ± 13

2.57

178 ± 5

307 ± 10

388 ± 10

2.61

751 ± 48

1000∗

1000∗

2.65

783 ± 45

1000∗

1000∗

3.19Iteration time (s)

To better demonstrate and quantify such a drawback, we

apply an edge detection algorithm to the restored images, in

particular, a Laplacian- and second derivative of the gradient

direction(SDGD)-based zero-crossing procedure (Verbeek &

van Vliet, 1994). A threshold step on the image resulting

from this algorithm gives us a binary image representing

the edges in the restored image. The very same algorithm is

appliedtotheoriginalobjectandtheresultsarecompared.The

difference between the binary map associated to the original

object and that associated to the restored image gives the

distribution ofwhat wecall ‘badvoxels’. Avoxel ofthebinary

map associated to the restored image is defined ‘bad’ in two

cases:(1)thevoxelcontainsanedgeinthemapoftheoriginal

object but not in the map of the restored image (negative bad

voxel); (2) the voxel does not contain an edge in the map of

the original object but contains an edge in the map of the

restored image (false-positive bad voxel). Figure 6 compares

thenumberof‘badvoxels’forthedifferentpotentialfunctions

atdifferentnoiselevels(forsimplicity,weconsideronlyQ,GM

andHS).Asexpected,theedge-preservingcapabilityofHSand

GM potential functions is better than that of the Q potential.

Moreover, it is important to note that for all noise levels, the

numberof‘badvoxels’ofGMisalwaysgreaterthanthatofHS;

in particular, although the number of ‘negative bad voxels’ is

comparableforthetwomethods,thenumberof‘false-positive

bad voxel’ is larger for GM. This is in agreement with the

drawback of GM due to the generation of ‘false’ edges during

the restoration process. No appreciable difference is observed

between GM and HL and between HS and HB.

In conclusion, the convex potentials HB and HS provide

better values, both in terms of the parameter IF and in terms

of the edge-preserving capability, than the non-convex ones,

GM and HL. However, we believe that the difference between

the edge-preserving capabilities of these methods tends to

disappear for higher SNR, as can be observed for SNR = 25

(Fig. 6).

Real data

To test the validity of our methods, we compare the

different algorithms on real confocal images. To this purpose,

we use both a well-known pattern such as fluorescence

beads and a more intricate biological specimen such as

bovine pulmonary artery endothelial (BPAE) cells. The 3D

image of green TetraspeckTMbeads (Molecular Probes) of

∼500-nm diameter is shown in Fig. 7(A). BPAE cells are

stained with a combination of fluorescent dyes. F-actin

was stained using green fluorescent BODYPY FL phallacidin

(Molecular Probes, Invitrogen, Milan, Italy), as shown in Fig.

8(A), and mitochondria were labelled with red fluorescent

MitoTracker Red CMXRos (Molecular Probes), as shown

in Fig. 8(B). The fluorescence excitation wavelength for

MitoTracker is λex= 543 nm; the emission wavelength is

λem= 600 nm; the fluorescence excitation wavelength for

F-actin and green beads is λex = 488 nm; the emission

wavelength is λem= 520 nm; the pinhole size was set to 1

Airy unit (backprojected radius of 208 nm); sampling is 45

nm in the lateral direction and 135 nm in the axial direction

for BPAE images, whereas for beads image, it is 30 nm in the

lateral direction and 90 nm in the axial direction.

The imaging device is a Leica TCS (true confocal scanning

system) SP5 (Leica Microsystems, Heidelberg, Germany)

equipped with a variable NA objective (immersion oil HCX

PL APO 63×/NA = 0.6–1.4) (Leica Microsystems).

For real-data experiment, it is not easy to carry out

an unambiguous comparison of the different algorithms,

particularlybecausetheregularizationparameter,thescaling

parameter or the optimal number of iterations can not

be determined in the same way as is used for synthetic

images. To partially overcome this problem, we try several

values of the regularization and scaling parameters and

inspect the resulting image visually to obtain a restoration

with optimal sharpness of the edges without introducing

artefacts due to noise amplification. In particular, in the

case of beads restoration, where we know the exact size,

we pay attention to use parameters that do not lead to the

underestimate dimension of the beads. Moreover, we use the

following criterion for stopping the iterations of regularized

and unregularized algorithms, based on the Kullback–Leibler

divergence of Af(i)+ b from g

K LD(g,Af(i)+ b) − K LD(g,Af(i+1)+ b) ≤ T.

(30)

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 11

SPLIT-GRADIENT METHOD IN FLUORESCENCE MICROSCOPY DECONVOLUTION 57

Fig. 5. Intensity profiles along a lateral line (A–C) and an axial line (D–F) in the original object and in the restored object. Insets of (A) and (D) represent,

respectively, the regions where the intensity profile are obtained (SNR = 20).

Fig. 6. Plot bar representing the distribution of the numbers of ‘bad voxels’ (BVs), ‘negative bad voxels’ and ‘false-positive bad voxel’ for different

restoration methods and for different SNRs. Each value represents the mean value obtained with ten different realizations of noise. The variance of these

data is so small that it is not visible.

A threshold of T = 10−4is used for image restoration of both

beadsandBPAEcells.FormitochondriaandF-actinimages,a

backgroundof7unitsisestimatedusingthehistogram-based

methods (van Kempen & van Vliet, 2000a), whereas a null

background is estimated for beads image.

Inagreementwiththenumericalresults,wedonotobserve

particulardifferencesbetweenGM-andHL-restoredimagesas

well as between HS and HB restorations; therefore, we report

only the results obtained by GM and HS regularization.

Appreciable restoration of the localized beads can

be seen using edge-preserving regularization, whereas

oversmoothing is evident in Q-restored images and noisy

artefact in RLM-restored images. The drawbacks and the

ability of the different restoration methods can be much more

appreciated when the previously introduced edge detection

algorithm is applied on the respective restored images. The

insets of Fig. 7 show how GM and HS thereby improve the

performance of the edge detection algorithm. Similar skills

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 12

58 G. VICIDOMINI ET AL.

Fig. 7. Restoration of confocal images of fluorescent beads. (A) Raw image (B–E). Restored objects: RLM restoration (B). Regularized restoration using

quadraticpotential(C),Geman–McClurepotential(D)andhyper-surfacepotential(E).Foreachrawimageandrestoredobject,twoaxialviewsalongthe

dotted lines and the result of the edge-detection algorithm applied on the image are reported.

Fig. 8. Restoration of confocal images of BPAE cells. (A) Raw image of F-actin sub-cellular structures. (B) Raw image of mitochondria sub-cellular

structures. (C–N) Restored objects associated to F-actin and mitochondria images: RLM restorations (C, D). Regularized restoration using quadratic

potential (E, F), Geman–McClure potential (G, H) and hyper-surface potential (I, J). For each raw image and restored object, two axial views along the

dottedlinesarereported.Tobetterappreciatethedifferencesbetweenthedifferentrestorations,theinsetsshowthemagnificationoftheareainthewhite

box.

and drawbacks can be observed from BPAE cell experiment.

TheinsetsofFig.8representingamagnificationofaparticular

regionofthesampleareintroducedintheoriginalimagesand

restored images in order to better appreciate the difference

between the methods. Checkerboard noise artefacts are

clearlyobservableinRLMrestoration;blockypiecewiseregion

presentedintheGMrestorationarecompletelyavoided inthe

case of HS.

The improvement obtained by means of deconvolution

becomes essential for high-quality 3D volume renderings.

Figure 9 shows the 3D volume renderings for the raw and

HS-restored stacks of Fig. 8.

Finally, a nice trick to qualitatively compare the

performance of the proposed algorithms is to acquire the

same sample at different resolution levels using a variable

NAobjective.Figure10(A)showsF-actinstructuresacquired

Fig. 9. Three-dimensional volume rendering of raw and deconvolved images of BPAE cells. The volume rendering represents the same structures of

Fig. 8. (A) Raw image. (B) Quadratic potential restoration. (C) Hyper-surface potential restoration.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 13

SPLIT-GRADIENT METHOD IN FLUORESCENCE MICROSCOPY DECONVOLUTION 59

Fig. 10. Comparison between restored low numerical aperture images and raw high numerical aperture images. (A, B) Raw F-actin images obtained

using a low numerical aperture objective (NA = 0.6) and a high numerical aperture objective (NA = 1.4), respectively. Note that both images represent

the same structure. Restored objects obtained from low numerical aperture image using RLM (C) and regularized methods with quadratic potential (D),

Geman–McClurepotential(E)andhyper-surfacepotential(F).Foreachrawimageandrestoredobject,twoaxialviewsalongthedottedlinesarereported.

using the objective under the minimal NA (NA = 0.6). The

resolution improvement is evident using the objective under

the maximal NA (NA = 1.4) (Fig. 10B). The restored images

in Figs. 10(C–F) show thereby the increase of information

provided by the reconstruction; the previously unresolved

F-actin filament structure are fully resolved after image

deconvolution (see insets) and artefacts are not generated.

This remark and that at the end of the previous sub-section

shouldbetakenintoaccountwhendecidingwhatkindofedge-

preserving potential can be conveniently used in a practical

application.

Concluding remarks

A Bayesian regularization is essential to obtain artefact-

free image restoration in 3D confocal microscopy. Quadratic

potential function is the most used approach in the Bayesian

regularization,butitleadstooverstatingrestoration,withthe

consequence that edge structures are completely lost. In this

paper,wetestandcomparedifferentedge-preservingpotential

functions in the MRF framework. Our results show that HS

and HB potential functions provide the best reconstructions,

both in terms of quantitative KLD analysis and in terms

of qualitative visual inspection. Moreover, we propose the

general SGM to devise effective iteration algorithms for

maximization of a posteriori probability under non-negativity

and flux conservation constraints. We believe that such

a method method thereby helps to test different kinds of

regularized potentialfunctions. Slowconvergence oftheSGM

is observed.

However,amodificationofthismethod,knownastheSGP,

has been recently proposed (Bertero et al., 2008). In this way,

onecanobtainbothatheoreticalandapracticalimprovement

of the method. Indeed, it is possible to prove convergence of

the modified algorithm, and preliminary numerical results in

the case of RLM applied to the deconvolution of 2D images

indicate a reduction in the computational time by at least an

orderofmagnitude(Bonettinietal.,2007).Theapplication of

thisapproachtothedeconvolutionof3Dimagesinmicroscopy

is in progress.

In this work, we assume that object is piecewise constant.

However, we believe that a quadratic piecewise assumption

can further improve the results, but this assumption requires

higher-orderneighboursystemsandthusmultiplevoxelsites,

thereby increasing the computational cost of each iteration.

For this reason, future work will be addressed not only to

test different object models but also to increase the rate of

convergence of the SGM along the lines indicated above.

Acknowledgements

We thanks members of the Department of NanoBiophotonics

forthesupporttothiswork.Thisworkwaspartiallysupported

by MUR (Italian Ministry for University and Research), grant

2006018748.

References

Benvenuto, F., Camera, A.L., Theys, C., Ferrari, A., Lanteri, H. & Bertero,

M. (2008) The study of an iterative method for the reconstruction of

images corrupted by Poisson and Gaussian noise. Inverse Probl. 24,

035016.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 14

60 G. VICIDOMINI ET AL.

Bertero,M.&Boccacci,P.(1998)IntroductiontoInverseProbleminImaging.

IoP Publishing, Bristol.

Bertero,M.,Lant´ eri,H.&Zanni,L.(2008)Iterativeimagereconstruction:

apointofview.MathematicalMethodsinBiomedicalImagingandIntensity-

Modulated Radiation Therapy (IMRT), CRM Series, Vol. 7 (ed. by Y.

Censor, M. Jiang & A. Louis), pp. 37–63. Edizioni della Normale, Pisa.

Besag, J. (1974) Spatial interaction and the statistical analysis of lattice

systems. J. Roy. Stat. Soc. B 36, 192–236.

Bonettini, S., Zanella, R. & Zanni, L. (2009) A scaled gradient projection

for constrained image deblurring. Inverse Problems, 25, 015002,

Department of Mathematics, University of Modena and Reggio Emilia.

Carrington,W.A.(1990)Imagerestorationin3Dmicroscopywithlimited

data. Bioimaging and Two-Dimensional Spectroscopy, Vol. 1205 (ed. by

L. C. Smith), pp. 72–83. Society of Photo-Optical Instrumentation

Engineers.

Charbonnier, P., Blanc-Feraud, L., Aubert, G. & Barlaud, M. (1994) Two

deterministic half-quadratic regularization algorithms for computed

imaging.ImageProcessing,1994.Proceedings.ICIP-94,Vol.2,pp.168–

172. Institute of Electrical and Electronics Engineers, Austin, TX, USA.

Charbonnier, P., Blanc-Feraud, L., Aubert, G. & Barlaud, M. (1997)

Deterministic edge-preserving regularization in computed imaging.

IEEE Trans. Image Process. 6, 298–311.

Conchello, J.A. & McNally, J.G. (1996) Fast regularization technique for

expectationmaximizationalgorithmforopticalsectioningmicroscopy.

Three-Dimensional Microscopy: Image Acquisition and Processing III, Vol.

2655 (ed. C.J. Cogswell, G.S. Kino, G.S. & T. Wilson), pp. 199–208.

Society of Photo-Optical Instrumentation Engineers, Bellingham, WA,

USA.

Csisz´ ar,I.(1991)Whyleastsquaresandmaximumentropy?Anaxiomatic

approachtoinferenceforlinearinverseproblems.Ann.Stat.19,2032–

2066.

Dey,N.,Blanc-Feraud,L.,Zimmer,C.,Roux,P.,Kam,Z.,Olivo-Marin,J.C.

& Zerubia, J. (2006) Richardson-Lucy algorithm with total variation

regularizationfor3Dconfocalmicroscopedeconvolution.Microsc.Res.

Tech. 69, 260–266.

Engl, H.W., Hanke, M. & Neubauer, A. (1996) Regularization of Inverse

Problems. Kluwer Academic Publishers, Dordrecht.

Geman,D.&Reynolds,C.(1992)Constraintrestorationandtherecovery

of discontinuities. IEEE Trans. Pattern Anal. 14, 367–383.

Geman, S. & Geman, D. (1984) Stochastic relaxation, Gibbs distribution

and the Bayesian restoration of images. IEEE Trans. Pattern Anal. 6,

721–741.

Geman,S.&McClure,D.E.(1985)Bayesianimageanalysis:anapplication

to single photon emission tomography. Proceedings of the Statistical

Computation Section, pp. 12–18. American Statistical Association,

Washington, DC.

Gravier, E. & Yang, Y. (2005) Motion-compensated reconstruction of

tomographic image sequences. IEEE Trans. Nucl. Sci. 52, 51–56.

Green, P.J. (1990) Bayesian reconstructions from emission tomography

data using a modified EM algorithm. IEEE Trans. Med. Imag. 9, 84–93.

Hebert,T.&Leahy,R.(1989)AgeneralizedEMalgorithmfor3DBayesian

reconstruction from Poisson data using Gibbs priors. IEEE Trans. Med.

Imag. 8, 194–202.

Holmes, T.J. (1988) Maximum-likelihood image restoration adapted for

noncoherent optical imaging. J. Opt. Soc. Am. A 5, 666–673.

Holmes, T.J. & Liu, Y.-H. (1989) Richardson-Lucy/maximum likelihood

image restoration algorithm for fluorescence microscopy: further

testing. Appl. Optics 28, 4930–4938.

Jovin, T.M. & Arndt-Jovin, D.J. (1989) Luminescence digital imaging

microscopy. Annu. Rev. Bioph. Biom. 18, 271–308.

van Kempen, G.M.P. & van Vliet, L.J. (2000a) Background estimation in

nonlinear image restoration. J. Opt. Soc. Am. A 17, 425–433.

van Kempen, G.M.P. & van Vliet, L.J. (2000b). The influence of the

regularization parameter and the first estimate on the performance

of Tikhonov regularized non-linear image restoration algorithm. J.

Microsc. 198, 63–75.

vanKempen,G.M.P.,vanVliet,L.J.,Verveer,P.J.&vanderVoort,H.T.M.

(1997) A quantitative comparison of image restoration methods for

confocal microscopy. J. Microsc. 185, 165–181.

Kullback, S. & Leibler., R.A. (1951) On information and sufficiency. Ann.

Math. Stat. 22, 79–86.

Lange, K. (1990) Convergence of EM image reconstruction algorithms

with Gibbs smoothing. IEEE Trans. Med. Imag. 9, 439–446.

Lant´ eri, H. & Theys, C. (2005) Restoration of astrophysical images—the

case of Poisson data with additive Gaussian noise. EURASIP J. Appl.

Signal Proc. 15, 2500–2513.

Lant´ eri, H., Roche, M., Cuevas, O. & Aime, C. (2001) A general

methodtodevisemaximum-likelihoodsignalrestorationmultiplicative

algorithms with non-negativity constraints. Signal Process. 81, 945–

974.

Lant´ eri, H., Roche, M. & Aime, C. (2002) Penalized maximum likelihood

imagerestorationwithpositivityconstraints:multiplicativealgorithms.

Inverse Probl. 18, 1397–1419.

Lucy, L.B. (1974) An iterative technique for the rectification of observed

distributions. Astron. J. 79, 745–754.

Mair, B.A. & Zahnen, J. (2006) A generalization of Green’s one-step-

late algorithm for penalized ML reconstruction of PET images. Nuclear

Science Symposium Conference Record, 2006, Vol. 5, pp. 2775–2777.

Institute of Electrical and Electronics Engineers, San Diego, CA,

USA.

Mondal, P.P., Vicidomini, G. & Diaspro, A. (2007) Markov random

field aided Bayesian approach for image reconstruction in confocal

microscopy. J. Appl. Phys. 102, 044701.

Mondal, P.P., Vicidomini, G. & Diaspro, A. (2008) Image reconstruction

formultiphotonfluorescencemicroscopy.Appl.Phys.Lett.92,103902.

Pierro, A.R.D. & Yamagishi, M.E.B. (2001) Fast EM-like methods for

maximum a posteriori estimates in emission tomography. IEEE Trans.

Med. Imag. 20, 280–288.

Richards, B. & Wolf, E. (1959) Electromagnetic diffraction in optical

systems II. Structure of the image field in an aplanatic system. Proc.

Roy. Soc. Lond. A 253, 358–379.

Richardson, W.H. (1972) Bayesian-based iterative method of image

restoration. J. Opt. Soc. Am. 62, 55–59.

Rudin, L.I., Osher, S. & Fatemi, E. (1992) Nonlinear total variation based

noise removal algorithms. Physica D 60, 259–268.

Schrader, M., Hell, S.W. & van der Voort., H.T.M. (1998) Three-

dimensional super-resolution with a 4Pi-confocal microscope using

image restoration. J. Appl. Phys. 84, 4033–4042.

Schultz, R.R. & Stevenson, R.L. (1995) Stochastic modeling and

estimation of multispectral image data. IEEE Trans. Image Process. 4,

1109–1119.

Shepp, L.A. & Vardi, Y. (1982) Maximum likelihood reconstruction in

positron emission tomography. IEEE Trans. Med. Imag. 1, 113–122.

Snyder,D.L.,Hammoud,A.M.&White.,R.L.(1993)Imagerecoveryfrom

data acquired with a charge-coupled-device camera. J. Opt. Soc. Am. A

10, 1014–1023.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61

Page 15

SPLIT-GRADIENT METHOD IN FLUORESCENCE MICROSCOPY DECONVOLUTION 61

Tikhonov, A.N. & Arsenin, V.Y. (1977) Solutions of Ill-Posed Problems.

Wiley, New York.

Verbeek, P.W. & van Vliet, L.J. (1994) On the location error of curved

edges in low-pass filtered 2D and 3D images. IEEE Trans. Pattern Anal.

16, 726–733.

Vicidomini, G., Mondal, P.P. & Diaspro, A. (2006) Fuzzy logic and

maximumaposteriori-basedimagerestorationforconfocalmicroscopy.

Opt. Lett. 31, 3582–3584.

van der Voort, H.T.M. & Strasters, K.C. (1995) Restoration of confocal

images for quantitative image analysis. J. Microsc. 178, 165–181.

C ?2009 The Authors

Journal compilationC ?2009 The Royal Microscopical Society, Journal of Microscopy, 234, 47–61