Improved Bayesian image denoising based on wavelets with applications to electron microscopy

Abstract

In this work we discuss an improvement of the image-denoising wavelet-based method presented by Bijaoui [Wavelets, Gaussian mixtures and Wiener filtering, Signal Process. 82 (2002) 709–712]. We show that the parameter estimation step can be replaced by a constrained nonlinear optimization. We propose three different methods to estimate the parameters. As in Bijaoui's original article, two of them deal with white noise. We show that the resulting algorithms improve the one originally proposed. Our third method extends the applicability of the denoising algorithm to colored noise. We test our algorithms with images simulating electron microscopy (EM) conditions as well as experimental EM images.

Full text

Page 1

Pattern Recognition 39 (2006) 1205–1213
www.elsevier.com/locate/patcog
ImprovedBayesianimagedenoisingbasedonwaveletswithapplications
toelectronmicroscopy
C.O.S. Sorzanoa,b,∗, E. Ortiza, M. Lópezc, J. Rodrigod
aDept. Sistemas Electrónicos y de Telecomunicación, Escuela Politécnica Superior, Univ. San Pablo-CEU, Urb. Montepríncipe s/n,
Boadilla del Monte, 28668 Madrid, Spain
bBiocomputing Unit, National Center of Biotechnology (CSIC), Campus Univ. Autónoma s/n, 28049 Madrid, Spain
cDept. Matemática e Informática Aplicadas a la Ingeniería Civil, E.T.S. Ingenieros de Caminos, Univ. Politécnicade Madrid,
Ciudad Universitaria s/n, 28040 Madrid, Spain
dDept. Matemática Aplicada, E.T.S. Ingeniería, Univ. Pontificia Comillas, c/Alberto Aguilera, 23, 28015 Madrid, Spain
Received 14 June 2004; received in revised form 1 December 2005; accepted 1 December 2005
Abstract
In this work we discuss an improvement of the image-denoising wavelet-based method presented by Bijaoui [Wavelets, Gaussian
mixtures and Wiener filtering, Signal Process. 82 (2002) 709–712]. We show that the parameter estimation step can be replaced by a
constrained nonlinear optimization. We propose three different methods to estimate the parameters. As in Bijaoui’s original article, two
of them deal with white noise. We show that the resulting algorithms improve the one originally proposed. Our third method extends
the applicability of the denoising algorithm to colored noise. We test our algorithms with images simulating electron microscopy (EM)
conditions as well as experimental EM images.
? 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
Keywords: Image denoising; Bayesian filtering; Wavelets
1. Introduction
Many are the papers addressing the problem of image de-
noising using wavelets [1]. In brief, all these algorithms first
perform the wavelet transform of the image to denoise, then
apply some filter to the wavelet coefficients, and finally take
the inverse wavelet transform to restore the denoised im-
age. Most popular wavelet-filtering algorithms are based on
thresholding [2], Wiener filtering [3] and Bayesian filtering
[4–7]. In fact, Bayesian filtering in the wavelet space has be-
come the standard in the field and the work of Portilla et al.
[7] represents the current state of the art of the algorithms
employed.
Bijaoui [1] introduced a Bayesian approach to image de-
noising in wavelet space shown to be superior to a num-
ber of previous thresholding or Wiener-filtering algorithms.
∗Corresponding author. Tel.: +34913724033; fax: +34913724049.
E-mail address: coss.eps@ceu.es (C.O.S. Sorzano).
0031-3203/$30.00 ? 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
doi:10.1016/j.patcog.2005.12.009
However, as is shown in this paper, the parameter estimation
step of this algorithm can be substantially improved. As an
alternative, we propose three different methods to estimate
the noise and signal parameters, keeping the Bayesian de-
noising step proposed by Bijaoui [1]. Each method is based
on different assumptions and, ultimately, they solve a linear
equation system in a least-squares sense subject to a number
of reasonable constraints.
We tested our algorithms on a set of simulated as well as
experimental EM images as used in single-particle structural
studies of macromolecular complexes [8]. The final goal
of this kind of studies is to elucidate the three-dimensional
structure of the specimen electron density combining dif-
ferent projection images taken with an electron microscope.
One of the advantages of this technique is the wide range
of specimen sizes that can be studied: from cellular or-
ganelles like the ribosome, to viruses, protein complexes, or
individual proteins. Another advantage, particularly when
dealing with proteins or protein complexes, is that the

Page 2

1206
C.O.S. Sorzano et al. / Pattern Recognition 39 (2006) 1205–1213
technique does not change the structural conformation, be-
cause macromolecules are not crystallized. This makes the
technique suitable for studies where the quaternary structure
of protein complexes has to be elucidated. The resolution
that can be obtained depends on the quality, quantity and
angular coverage of the projection images.
The main drawbacks of EM images are their extremely
low signal-to-noise ratio (below −3dB) and the complicated
nature of the electron microscope transfer function [9]. This
function is referred to as contrast transfer function (CTF)
and it must be estimated directly from the micrographs [10].
The shape of the CTF in Fourier space can be described by a
damped sinusoid. However, in a rough approximation it can
be modelled as a low-pass filter. The filtered nature of these
images make Bijaoui’s approach unapplicable. However, we
show that the method can be generalized to address this kind
of images.
Image denoising techniques can be applied in two ways in
EM: first, if the image signal components are not modified,
then denoising techniques can be used before reconstruct-
ing the three-dimensional structure of the macromolecule to
improve the quality of the reconstruction; second, if the im-
age signal components are modified but the result enhances
most of the main features of the particle, then they can be
used to facilitate intermediate processes like bidimensional
alignments, automatic particle picking, etc.
Thispaperisorganizedasfollows.First,theimagedenois-
ing procedure based on a Bayesian approach in the wavelet
space is introduced in Section 2. Then, we present our pro-
posed modifications to the parameter estimation step in Sec-
tion 3. Results obtained with simulated as well as experi-
mental data are shown in Section 5. Finally, we discuss the
results and conclude.
2. The filtering procedure
The filtering technique used in this article is based on the
Bayesian approach proposed by Bijaoui [1]. Let us assume
that the image formation model is of the form
y = x + n,
where x is the ideal image, n is random independent noise,
and y is the measured image. We assume that the noise is
white and normally distributed. Thus, its probability density
function (PDF) is given by
(1)
p(n) = G(n,N) =
1
√2?Ne−n2/2N, (2)
where N is the noise variance. Solving the image formation
model for n it can be seen that the conditional PDF of y
given x is
p(y|x) = p(n) = G(y − x,N).(3)
It is assumed that the PDF of the ideal image can be ex-
pressed as a mixture of zero-mean Gaussians
?
Under these assumptions it can be proved that the PDF of the
measured image is given by the convolution of both PDFs
?
Then, the denoising problem is stated as the Bayesian prob-
lem of estimating x from y. The a posteriori PDF is given by
?
The filtering is done by taking the a posteriori expectation
of x
?
The relationship of this filter with classical Wiener filtering
is given in the article by Bijaoui [1].
The previous filter is applied independently to each of the
scales of the wavelet transform of the measured image, i.e.,
the variable x is formed by all those wavelet coefficients be-
longing to the same scale. Therefore, each scale has its own
?iand Siparameters while the noise power, N, is common
to all the scales since the noise is assumed to be white.
Theproblemnowistoestimatethe?i,SiandNparameters
from the measured images. Bijaoui [1] assumes that there is
no signal at the finest scale. Therefore, all the energy at that
scale is coming from the noise term. Under this assumption
the noise power N can be estimated from that scale and later
used for the rest of the scales. Bijaoui [1] proposes a robust
estimate of N at the finest scale based on a k − ? clipping
strategy.
For the rest of the scales, the original paper affirms that, in
practical terms, only two Gaussians are necessary to model
the PDF of the measured wavelet coefficients at a given scale
p(x) =
i
?iG(x,Si). (4)
p(y) = p(x) ? p(n) =
i
?iG(x,Si+ N). (5)
p(x|y) =
i?iG(x,Si)G(y − x,N)
?
i?iG(y,Si+ N)
. (6)
E{x|y} = y
i?i(Si/Si+ N)G(y,Si+ N)
?
i?iG(y,Si+ N)
= yw(y). (7)
p(y) = (1 − a)G(y,N) + aG(y,S + N),
where a and S are estimated at each scale through the 2nd
(M2) and fourth order (M4) moments of the measured
wavelet coefficients at that scale
(8)
S =M4/3 − N2
M2− N
a =(M2− N)2
,
M4/3 − N2.
In order to guarantee the convex nature of the combina-
tion given by Eq. (8) a is set to 0 if M2− N <0 or if
(9)

Page 3

C.O.S. Sorzano et al. / Pattern Recognition 39 (2006) 1205–1213
1207
M4/3− N2<0. Accordingly, a is set to 1 if a >1 and, in
this case, S is estimated only from the variance M2.
3. Improvement of the parameter estimation
The main contribution of our study refers to the estima-
tion of the parameters S, a and N. We provide three new
estimation procedures, each one with different hypothesis
and applicable to different fields. In the first one, we si-
multaneously solve for all the parameters using a rationale
similar to the one used by Bijaoui [1]. The first method
assumes that the wavelet coefficients follow a Gaussian dis-
tribution and that the noise is white and independent from
the signal. In the second one, we solve for the same denois-
ing problem (removal of white noise) following a different
approach based on the power decomposition of the mea-
sured image. This second method assumes that the noise is
white and independent from the signal. Finally, we extend
the power decomposition procedure to the case of non-
white noise assuming that the noise is independent from
the signal.
3.1. Method 1
In this method we solve simultaneously for all the noise
and signal parameters in the problem. Our changes also af-
fect the “practical” PDF given by Eq. (8). For the kind of
images addressed in this work a single Gaussian suffices to
model the ideal wavelet coefficients, x, at a given scale (this
point will be explicitly checked in our experiments). In this
way, our “practical” PDF becomes
p(y) = G(y,S + N).
This model has an advantage over the model in Eq. (8)
since the estimates given by Eq. (9) may become highly un-
stable under certain circumstances. Let us assume that there
is no signal energy at the first (finest) and second scales.
Under these circumstances, the noise power is correctly es-
timated from the first scale. However, when estimating the
signal power at the second scale we are using the second and
fourthordermomentsoftheGaussiandefiningthenoise,that
should be the same as for the first scale. Therefore, M2and
M4for the second scale take the value N and 3N2, respec-
tively, (for a Gaussian PDF it can be proved that M4=3M2
Plugging these values into Eq. (9) we have 0 over 0, and
consequently the S and a parameters cannot be determined.
In practical terms, 0 over 0 is never obtained due to the ran-
dom nature of the estimates of the M2and M4moments.
However, the resulting S and a parameters may be highly
unstable.
Using Eq. (10) an alternative equation system can be
posed. As in the original article, we assume that there is no
significant signal at the finest scale. However, we do not es-
timate the noise power only from the first scale (s =1), and
subsequently all the rest of parameters, but we estimate all
(10)
2).
the parameters simultaneously in a least squares sense con-
strained by some reasonable hypothesis.
Concisely, given the PDF model in Eq. (10), the following
equation must hold for all scales s = 1,2,...
M2s= Ss+ N,
where M2s is the variance of the measured wavelet coef-
ficients at the scale s and Ss is the variance of the ideal
wavelet coefficients at that scale. Furthermore, we know that
all variances must be positive (N ?0 and Ss?0), and for the
kind of images employed in this article Ss?Ss+1(i.e., most
of the energy is concentrated at low frequencies). Finally,
if reasonable lower and upper limits for the signal-to-noise
ratio (SNRland SNRh) are provided, then it must hold that
(11)
I
1 + SNRh
where we have made used of the fact that the total image
power, I, must be I = ST + N (ST is an estimate of the
total signal power). In case that no clue is available about
the SNR (SNRl=0 and SNRh=∞) the bounds of the noise
power become 0?N ?I.
To solve simultaneously for all the parameters we propose
to minimize ?Cx − M2?2subject to Ax?b, where
⎛
⎜
⎜
...... .........
?N ?
I
1 + SNRl,(12)
C =
⎜
⎜
⎜
⎜
⎜
⎝
1000
...
1100
...
1010
...
1001
...
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
,
x =
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎝
N
S2
S3
S4
...
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
,
M2=
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎝
M21
M22
M23
M24
...
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
...
,
A =
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎝
⎜
⎜
⎜
⎜
⎜
01
−1
1
0
00
−1
...
...
............
−1
0
000
...
−1
0
00
...
0
−1
0
0
...
00
−1
...
...
...... ......
−1
0
000
...
100
...
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
⎟
⎟
⎟
⎟
⎟
,

Page 4

1208
C.O.S. Sorzano et al. / Pattern Recognition 39 (2006) 1205–1213
and
b =
⎛
⎜
⎜
⎜
⎜
⎝
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
0
0
0
...
0
0
0
...
I
1 + SNRh
I
1 + SNRl
The main advantages of the proposed method with respect
to the original paper are its simultaneous nature (allowing
for a higher stability) and the possibility of constraining the
parameters with a priori information (SNR and the constrain
that for a certain kind of images Ss?Ss+1).
⎞
⎟
⎟
⎟
⎟
⎠
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
.
3.2. Method 2
The second method is based on the power decomposi-
tion of the measured image. If the image formation model
is y =x +n, and x and n are independent variables, then the
measured image power I =Pyis the sum of the correspond-
ing powers Pxand Pn. Since the wavelet decomposition is
orthogonal, this fact is also true at any scale of the wavelet
decomposition. Furthermore, due to the orthonormality of
our wavelet transform I = Pwy, where Pwyis the power of
the wavelet transform of the image y.
In this way, the power of the wavelet transform of y at
each scale s, Ps, can be decomposed as the sum of two
components:onecomingfromnoiseandanotheronecoming
from the signal
Ps= (N + Ss)ns,
where ns is the number of wavelet coefficients at scale s.
Again, we assume that there is no significant signal at the
finest scale (S1= 0). The power decomposition must be
constrained in such a way that the sum of the power at all
scales yields the overall image power
?
Eq. (12) can still be used to incorporate our a priori infor-
mation about noise.
This second procedure also solves simultaneously for
all the noise and signal parameters involved minimizing
?Cx − P?2subject to Ax?b and Aeqx = beq, where
⎛
⎜
... ............
(13)
I =
s
Ps. (14)
C =
⎜
⎜
⎝
n1
n2
n3
n4
0
n2
0
0
0
0
n3
0
0
0
0
n4
...
...
...
...
⎞
⎟
⎟
⎟
⎠,
x =
⎛
⎜
⎜
⎜
⎝
N
S2
S3
S4
...
⎞
⎟
⎟
⎟
⎠,
P =
⎛
⎜
⎜
⎜
⎝
P1
P2
P3
P4
...
⎞
⎟
⎟
⎟
⎠,
A =
⎛
⎜
⎜
⎜
⎝
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
0
0
1
0
−1
1
...
0
0
−1
0
...
0
0
0
...
...
...
...
...
...
...
...
...
...
−1
...
0
0
0
−1
...
0
0
...
−1
0
0
0
...
−1
0
...
0
−1
0
0
...
0
1
⎞
⎟
⎟
⎟
⎠
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
,
b =
⎛
⎜
⎜
⎜
⎜
⎝
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
0
0
0
...
0
0
0
...
I
1 + SNRh
I
1 + SNRl
⎞
⎟
⎟
⎟
⎟
⎠
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
,
Aeq=
??
s
ns n2 n3 n4 ...
?
and
beq= (I).
This second method is based on a weaker assumption on
the image formation model: noise is independent from sig-
nal. A drawback of the two procedures presented above is
that they assume that noise is white (i.e., it has the same
power at all frequencies) and that there is no significant sig-
nal at the finest wavelet scale. This precludes the application
of this denoising method to filtered images. Next procedure
will partially overcome this problem.
3.3. Method 3
If images are low-pass filtered, for instance, then the noise
power measured at the finest scale does not correspond to
the true noise power present at the rest of the scales. This
is a basic assumption in Methods 1 and 2. Furthermore, in
those methods it was assumed that there was no significant
signal at the finest scale, which may not always be the case.
In this last procedure, we provide a framework in which
both situations can be handled easily. It is an extension of
Method 2 and, therefore, it is based on the decomposition
of the image power in different components due to different
sources. The main departure from the previous method is
that we allow for different noise powers Ns at each scale
and that S1 is not necessarily null. Therefore, the power
decomposition at each scale is now
Ps= (Ns+ Ss)ns.(15)
Eq. (14) still must hold because signal and noise are again
independent. The SNR constraints adopt a different expres-
sion
?
SNRl?
snsSs
snsNs
?
?SNRh.(16)

Page 5

C.O.S. Sorzano et al. / Pattern Recognition 39 (2006) 1205–1213
1209
Additionally, we impose the constraint Ns?Ns+1. This con-
straint assumes that the imaging system attenuates high fre-
quencies more than low frequencies.
Summarizing, the cost function to minimize is still
?Cx − P?2subject to Ax?b and Aeqx = beq, where now
C =
⎛
⎜
⎜
⎜
⎝
n1
0
0
0
...
0
n2
0
0
...
0
0
n3
0
...
0
0
0
n4
...
...
...
...
...
...
n1
0
0
0
...
0
n2
0
0
...
0
0
n3
0
...
0
0
0
n4
...
...
...
...
...
...
⎞
⎟
⎟
⎟
⎠, x =
⎛
⎜
⎜
⎜
⎝
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
N1
N2
N3
N4
...
S1
S2
S3
S4
...
⎞
⎟
⎟
⎟
⎠
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
,
P =
⎛
⎜
⎜
⎜
⎝
P1
P2
P3
P4
...
⎞
⎟
⎟
⎟
⎠,
A =
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎝
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
1
0
0
−1
1
0
...
0
0
0
0
0
−1
0
0
...
0
0
0
0
0
00
0
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
−1
1
...
0
0
0
0
0
0
−1
0
...
0
0
0
0
0
−1
...
0
0
0
0
0
0
0
−1
...
0
0
0
0
0
...
0
0
0
0
−1
0
0
0
...
0
0
0
0
0
−1
1
0
...
0
0
0
0
0
0
−1
0
0
...
n2
−n2
−1
1
...
0
0
0
0
0
0
0
−1
0
...
n3
−n3
−1
...
0
0
0
0
0
0
0
0
−1
...
n4
−n4
...
0
0
0
0
0
−1
0
0
0
...
n1
−n1
−SNRhn1
SNRln1
??
−SNRhn2
SNRln2
−SNRhn3
SNRln3
?
−SNRhn4
SNRln4
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
,
b =
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎝
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
0
0
0
...
0
0
0
...
0
0
0
0
...
0
0
0
0
...
0
0
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
,
Aeq=
s
ns n2 n3 n4 ...
and
beq= (Pwy).
4. Relationship to Wiener filtering
In our model, it has been assumed that the PDF of the
signal coefficients can be represented with a single Gaus-
sian. In this case, the filter applied falls back to a classical
Wiener filter. The hypothesis of the Wiener filter is that the
relationship between the original image x (x is a vector with
all pixel values ordered lexicographically) and the acquired
image y is
y = Hx + n,
whereHisablurringmatrixandnisarandomnoisevector.n
is assumed to follow a multivariate normal distribution with
zero mean and covariance matrix ?2I, i.e., p(n)=G(0,?2I)
(17)
and the input vector x is assumed to follow a multivari-
ate normal distribution with zero mean and covariance ma-
trix ?x, i.e., p(x) = G(0,?x). Under these assumptions it
has been proven [11, Chapter 46] that the filter W to opti-
mally recover ˜ x = Wy from y in a maximum a posteriori
sense is
W = ?xHt(?2I + H?xHt)−1,(18)
where Htdenotes the transpose of H.
Taking into account that we are considering one-
dimensional vectors, for us H = I, and that in our notation
?x= S and ?2= N this filter boils down to
S
N + S=
Inotherwords,ourBayesianfilteringisthesameasaWiener
filter and our contribution lays solely on how to estimate the
filter parameters N and S at each wavelet scale.
W =
1
1 + (1/SNR). (19)