SSIM image quality metric for denoised images

Abstract

The mean square error (MSE) and its related metrics such as peak signal to noise ratio (PSNR), root mean square error (RMSE), mean absolute error (MAE), and signal to noise ratio (SNR) have been the basis for mathematically defined image quality measurement for a long time. These methods are all based on the MSE. Denoisng quality has also been traditionally measured in terms of the MSE or its derivatives. But none of these metrics takes the structural fidelity of the image into account. Here, we investigate the structural changes that occur during the denoising process. In particular, we ascertain the structural fidelity of TV-denoised images.

Full text

SSIM Image Quality Metric for Denoised Images
PETER NDAJAH, HISAKAZU KIKUCHI, MASAHIRO YUKAWA,
HIDENORI WATANABE and SHOGO MURAMATSU
Department of Electrical and Electronics Engineering,
Niigata University,
JAPAN
email: ndajah@telecom0.eng.niigata-u.ac.jp
September 3, 2010
Abstract
The mean square error (MSE) and its related metrics such as peak
signal to noise ratio (PSNR), root mean square error (RMSE), mean
absolute error (MAE), and signal to noise ratio (SNR) have been the
basis for mathematically defined image quality measurement for a long
time. These methods are all based on the MSE. Denoisng quality has
also been traditionally measured in terms of the MSE or its derivatives.
But none of these metrics takes the structural fidelity of the image into
account. Here, we investigate the structural changes that occur during
the denoising process. In particular, we ascertain the structural fidelity
of TV-denoised images.
Keywords: SSIM, MSE, TV, PSNR, NOISE, DENOISE, METRIC
1MSE-based Im-
age Quality Mea-
sure
The MSE has been the basis for
image quality measure. Usu-
ally, one of the images (the orig-
inal) is assumed to contain no
distortions while the other im-
age is contaminated by noise or
some other kind of error. Sup-
pose x={xi|i=1,2, ..., N }and
y={yi|i=1,2, ..., N }where xi
and yiare the ith samples in x
and yand Nis the number of
signal samples. Then the MSE
between the signals is
MSE(x,y)= 1
N
N

i=1
(xi−yi)2
ei=(xi−yi) is referred to as error
signal. An image is a two dimen-
sional signal so the MSE is given
as
d(x,y)=N

i=1
|ei|2
Generally, in image processing,
the MSE is often used in the form
of the peak signal to noise ra-
tio(PSNR) measure
PSNR =10log
10
L2
MSE
1
ADVANCES in VISUALIZATION, IMAGING and SIMULATION
ISSN: 1792-6130
53
ISBN: 978-960-474-246-2

The PSNR is more useful than the
MSE only when images of differ-
ent dynamic ranges are being com-
pared otherwise it is equivalent to
the MSE [7].
2Drawbacks of
MSE-based Im-
age Quality Mea-
sure
In imaging, the true aim of any
denoising method is to improve
the visual quality and fidelity of a
noisy image but the MSE does not
take into account image depen-
dencies such as textures, order-
ings, patterns, etc. all of which af-
fect image perception quality. Im-
age pixel order transmit vital in-
formation about the structure of
a visual scene. Unfortunately the
MSE does not measure this. The
correlation between the error sig-
nal and the underlying image sig-
nificantly affects perceptual image
distortion but this is also ignored
by the MSE. The MSE does not
take into account the signs of the
error (since its square is used) sig-
nal added to an image. However,
the visual fidelity of the resulting
image has been proved to be dras-
tically different. Since all images
are treated equally in the formu-
lation of the MSE, image content-
dependent variations in image fi-
delity cannot be accouted for.
3 Structural Simi-
larity (SSIM)
The SSIM is a recently proposed
image fidelity measure which has
proved highly effective in mea-
suring the fidelity of signals. The
SSIM approach was originally mo-
tivated by the observation that
natural images have highly struc-
tured signals with strong neigh-
borhood dependencies. These de-
pendencies carry useful informa-
tion about the structures of the
objects in the visual scene.
The human visual system is highly
adapted to extract structural in-
formation from visual scenes. For
this reason, image fidelity mea-
surement should retain the signal
structure as an important content.
A distinction has to be made be-
tween non-structural distortions
such as variations in luminance,
contrast, Gamma distortions, and
spatial shift(these do not change
the structure of the image in any
way) and the structural distor-
tions such as additive Gaussian
noise, blur and lossy compres-
sion(e.g. JPEG). These distort
the structure of the image signifi-
cantly.
The human visual system is highly
sensitive to structural distortions
and easily compensates for non-
structural distortions. The main
function of the SSIM is to simu-
late this functionality.
Let x={xi|i=1,2, ..., N}and
y={yi|i=1,2, ..., N }be the
original and the test image signals
respectively. Then, the SSIM
Q=4σxy ¯x¯y
(σ2
x+σ2
y)[(¯x)2+(¯y)2](1)
The above equation can be
rewritten as
Q=σxy
σxσy
·2¯x¯y
(¯x)2+(¯y)2·2σxσy
σ2
x+σ2
y
(2)
The SSIM measures distor-
tionsasacombinationofthree
2
ADVANCES in VISUALIZATION, IMAGING and SIMULATION
ISSN: 1792-6130
54
ISBN: 978-960-474-246-2

factors: loss of correlation, lumi-
nance distortion and contrast dis-
tortion. The first component in
(2) is the correlation coefficient
between xand y.Itmeasures
the degree of correlation between
xand y. Its dynamic range is
[−1,1] and the best value 1 is ob-
tained when yiis linear with re-
spect to xifor all i=1,2, ..., N i.e.
yi=axi+b. The second compo-
nent has a value range of [0,1]. It
measures the mean luminance be-
tween x. It equals 1 if and only if
¯x=¯y. The third compnent meau-
res the similarity of the contrast
between xand y. Its range is also
[0,1], where the best value is 1.
This occurs only when σx=σy.
4 A Comparison of the MSE and the SSIM
Reference Image
MSE = 0, SSIM =1 Contrast Stretch
MSE = 255, SSIM = 0.9172 Negative Image
MSE = 255, SSIM = −0.1632
Gaussian White Noise
MSE = 255, SSIM = 0.5927 Lossy compression
MSE = 255, SSIM = 0.6947 Blurred Image
MSE = 255, SSIM = 0.7722
Figure 1: Images with different structural distortions but the same MSE
values
Figure 1 illustrates the shortcom-
ings of the MSE. In all the images
shown, the MSE = 255 even when
the visual structures are greatly
distorted. The SSIM on the other
hand seems to reflects the struc-
tural changes in the images more
faithfully. This is the advantage of
the SSIM over the MSE. The hu-
man visual system (HVS) is very
sensitive to structural changes,
therefore any metric that will be
well correlated to the HVS must
take into account the structural
dependencies of the signal samples
in order to provide effective pre-
3
ADVANCES in VISUALIZATION, IMAGING and SIMULATION
ISSN: 1792-6130
55
ISBN: 978-960-474-246-2

Reference Image: MSE = 255, SSIM = 1
(a) Denoised Image: λ = 60,τ = 0.01,
MSE = 255, SSIM = 0.653430
(b)
Denoised Image: λ = 12, τ = 0.01,
MSE = 255, SSIM = 0.892388
(c)
Denoised Image: λ = 2, τ = 0.01,
MSE = 255, SSIM = 0.748494
(d)
Denoised Image: λ = 1, τ = 0.01,
MSE = 255, SSIM = 0.712412
(e)
Denoised Image: λ = 0.5, τ = 0.01,
MSE = 255, SSIM = 0.685501
(f)
Figure 2: Denoised images showing values MSE values and SSIM index
values
dictions of image quality. As of-
ten happens during denoising of
images, structural changes such
as blurring can happen. Most
denoising algorithms do not ac-
tually ’remove’ the noise. It is
more a process of noise minimiza-
tion rather than removal. The
amount of noise still left in the im-
age sample after the denoising op-
eration depends on the amount of
noise originally in the image be-
fore the denoising operation. But
the MSE-based metrics may not
be able to capture this reality be-
cause they are not designed to to
measure the structural distortions
that may occur.
5 Denoised Image
Structural Fi-
delity
So why use the SSIM index to
measure the quality of denoised
images? Because the MSE-based
metrics do not tell the whole story.
The ultimate objective of denois-
ing is to produce an image that
is judged to be a good representa-
tion of the reference image (known
or unknown). The HVS is the ul-
timate judge of what a good qual-
ity image is. This means that the
structural fidelity of the denoised
image is of utmost importance be-
cause the HVS uses the structural
fidelity to measure the quality of
an image. The MSE-based met-
rics fail to measure the structural
improvement or degradation in an
image after denoising. This is be-
4
ADVANCES in VISUALIZATION, IMAGING and SIMULATION
ISSN: 1792-6130
56
ISBN: 978-960-474-246-2

cause in the MSE-based metrics,
the signal samples are considered
to be independent of each other.
As we can see in Figure 2, the de-
noised images have different SSIM
values (as judged by the HVS) but
they have practically the same
MSE values.
The total variation denoising al-
gorithm was used to denoise the
images because of its effectiveness
and also because it has tunable
parameters λand τthat control
the effectiveness of the denoising
process. We have varied the val-
ues of λand kept τconstant in the
experiments.
6Conclusion
We used the lena image as the test
image in our experiments. As Fig-
ure 2 shows, the changes in struc-
tural similarity indices of the im-
ages correlate somewhat with hu-
man visual system. For example,
when λ≤2, ((d)-(f)), the algo-
rithm causes blurring in the im-
ages. The SSIM index reflects this
fact as the SSIM values become
progressively smaller with reduc-
ing visual quality of the images,
However, the MSE remained the
same throughout our experiments.
for this reason, it may be use-
ful to use the SSIM as an alter-
native metric of denoised image
quality since it is a good measure
of the structural degradation or
improvement in a denoised image.
References
[1] Z. Wang, L. Lu and A.C.
Bovik. Video Quality Assess-
ment based on Structural Dis-
tortion Analysis. IEEE Inter-
national Conference on Image
Processing, Genoa, Italy, Sept.
11-14, 2005.
[2] Z. Wang, A.C. Bovik and
E.P. Simoncelli. Structural
Approaches to Image Quality
Assessment in Handbook of
Image and Video Processing
(Al Bovik, ed.), 2nd Edition,
Academic Press, 2005.
[3] Z. Wang, A.C. Bovik and L.
Lu. Why is Image Quality
Assessment Difficult? IEEE
International Conference on
Acoustics, Speech and Signal
Processing, May, 2002
[4] Z. Wang and A.C. Bovik. A
Universal Image Quality In-
dex. IEEE Signal Processing
Letters, vol. 9. no.3, pp. 81-84,
March 2002
[5] Z. Wang, A.C. Bovik, H.R.
Sheikh and E. P. Simon-
celli. Image Quality Assess-
ment: From error Visibility
to Structural Similarity. IEEE
Transactions on Image Pro-
cessing, vol. 13, no. 4, pp. 600-
612, 2004
[6] Zhou Wang and Alan C.
Bovik. Mean Squared Error:
Love it ot Leave it? IEEE Sig-
nal Processing Magazine, Jan-
uary 2009.
[7] Zhou Wang and Alan C.
Bovik. A Universal Image
Quality Index. IEEE Signal
Processing Letters, vol. 9 no.
3, pp. 81-84, 2002.
5
ADVANCES in VISUALIZATION, IMAGING and SIMULATION
ISSN: 1792-6130
57
ISBN: 978-960-474-246-2