Content uploaded by Nurdiani Zamhari
Author content
All content in this area was uploaded by Nurdiani Zamhari on Sep 01, 2016
Content may be subject to copyright.
Abstract—Image quality analysis is to study the quality of
images and develop methods to efficiently and swiftly
determine the quality of images. It is an important process
especially in this digital age whereby transmission,
compression and conversion are compulsory. Therefore, this
paper proposed a hybrid method to determine the image
quality by using Levenberg-Marquardt Back-Propagation
Neural Network (LMBNN). Three known quality metrics
were combined as the input element to the network. A proper
set of network properties was chosen to represent this element
and was trained using Levenberg-Marquardt algorithm
(trainlm) in MATLAB. From the preliminary simulation, a
promising output result was obtained indicated by good
performance metrics results and good regression fitting.
Index Terms—Image Quality Metrics, Levenberg-Marquardt,
Neural Network, hybrid
I. INTRODUCTION
Image quality analysis is the science of analyzing and
comparing the characteristics and features of an image with
reference to the original image of predetermined/preset
standards. Image quality analysis measures should be
employed to determine the usability of images after they
have undergone any kind of manipulation, for example,
compression, transmission or conversion. Therefore,
studying the various approaches to image quality analysis
will provide information on method of image quality
assessment that can be efficiently employed under any
circumstances.
Kuryati Kipli is with Department of Electronic Engineering, Faculty of
Engineering, Universiti Malaysia Sarawak, 94300 Kota Samarahan,
Sarawak, Malaysia (+6082-583354; fax:+6082-583410; email:
kkuryati@feng.unimas.my).
Mohd Saufee Muhammad, Sh. Masniah Wan Masra, Nurdiani Zamhari,
Kasumawati Lias and Dayang Azra Awang Mat are with Department of
Electronic Engineering, Faculty of Engineering, Universiti Malaysia
Sarawak, 94300 Kota Samarahan, Sarawak, Malaysia.
(email: msaufee@feng.unimas.my, wmmasnia@feng.unimas.my,
znurdiani@feng.unimas.my, lkasumawati@feng.unimas.my,
dmazra@feng.unimas.my).
Method of image quality assessment can be classified
into subjective and objective methods. The subjective
method requires the use of human discretion to decide the
level of the image’s quality [1-2]. This method is subject to
bias in the form of the tester’s taste and preferences.
However, the result of the subjective analysis is very often
a trusted method as it is only natural for people to judge
with their own eyes. The demerit of subjective assessment
is that it is time and labour consuming. The objective
method is unbiased and automated therefore it provides a
result that is faithful to all assigned parameters [1-2]. The
demerit of objective assessment is that it may not be
reliable.
Most of digital image analysis processes trying to
simulate the human visual cortex as the human eye remains
a very superior judge of image quality. For example, if the
computer saying the image is of a good quality but a
human saying it is of a bad quality, the image will most
likely be scrapped. Therefore, the computer’s reliability
and accuracy will be considered low if there is a poor
correlation between its results and the human eye’s
judgment.
Depending on the existence of reference images, there
are three categories of objective image quality metrics
(IQMs); full-reference (FR), reduced reference (RR) and
non reference (NR) [3-5]. These IQMs are developed based
on color appearance, blur assessment, wavelet, pixels
comparison, hue saturation and many others. Among the
available image quality metrics, the widely known metrics
are Peak Signal-to-Noise Ratio (PSNR), Mean Squared
Error (MSE) and Structural Similarity (SSIM) [6-7].
The objective of this paper is to investigate the potential
of combining multiple metrics with artificial neural
network (ANN) in order to achieve image quality score
that similar to human visual measure. Another objective is
to evaluate the performance of LMBP as a hybrid IQM. To
achieve these objectives, a number of objective
assessments was conducted and compared to a
corresponding subjective assessment. Afterward, this
measurement data were combined and used as the input
vectors to Levenberg-Marquardt Back-propagation
network.
Performance of Levenberg-Marquardt
Backpropagation for
Full Reference Hybrid Image Quality Metrics
Kuryati Kipli, Mohd Saufee Muhammad, Sh. Masniah Wan Masra,
Nurdiani Zamhari, Kasumawati Lias, Dayang Azra Awang Mat
Proceedings of the International MultiConference of Engineers and Computer Scientists 2012 Vol I,
IMECS 2012, March 14 - 16, 2012, Hong Kong
ISBN: 978-988-19251-1-4
ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)
IMECS 2012
The paper aims to combine PSNR, MSE and SSIM
metrics with the assumption each of the metrics could
overcome each other weaknesses. Each of this metrics has
its own weakness in term of detecting certain types of
image degradation. Other limitations of the existing metrics
are consistency, accuracy and computational cost. By
combining these metrics values, and trained using LMBP
ANN, it is hopefully will become a new image quality
metrics which is more intelligent and accurate.
II. METHODOLOGY
Four assessments were modeled consist of three
objectives and one subjective assessments. A subjective
assessment was conducted using a sample set of 34 images
and 80 participants. The sample set consists of a reference
image and 33 digitally altered images using four categories
of operations namely, morphological operations, noise-
adding, format conversion and filtered images. The results
of this subjective assessment were tabulated as Mean
Opinion Score (MOS) values ranging from 1 to 5, with 5
being the highest quality. These values will be further used
as the target for the network.
Three objective assessments were conducted with the
aim of determining the Mean Squared Error (MSE), the
Peak Signal-to-Noise Ratio (PSNR), and the Structural
Similarity (SSIM) between the reference image and the
sample images. The results of all four image quality
assessments were then tabulated and analyzed to determine
correlation characteristics using Regression, R correlation
between each objective and subjective assessment.
The measured data of PSNR, MSE and SSIM were used
as input of the network and MOS as target for the network.
Experimental was setup using MATLAB Neural Network
Toolbox. This network was trained with Levenberg-
Marquardt backpropagation algorithm (trainlm). This
network was chosen due to its good characteristic for
solving fitting problems. The neural network must map
well between a data set of numeric inputs and a set of
numeric targets.
The network used is a two-layer feed-forward network
as illustrated in Fig. 1. The two-layer feed-forward network
with sigmoid hidden neurons and linear output neurons
(newfit), can fit multi-dimensional mapping problems
arbitrarily well, given consistent data and enough neurons
in its hidden layer. After a few experimental run, the
number of neurons in the hidden layers was set to 20.
Fig.1 Two-layer feed-forward network
A. Levenberg-Marquardt BP
The application of Levenberg-Marquardt to neural
network training is described in [8-9]. This algorithm has
been shown to be the fastest method for training moderate-
sized feed-forward neural networks (up to several hundred
weights). It also has an efficient implementation in
MATLAB software, since the solution of the matrix
equation is a built-in function, so its attributes become even
more pronounced in a MATLAB environment [10].
The network trainlm can train any network as long as its
weight, net input, and transfer functions have derivative
functions. Backpropagation is used to calculate the
Jacobian jX of performance with respect to the weight and
bias variables X. Each variable is adjusted according to
Levenberg-Marquardt equation,
jj = jX * jX
je = jX * E
dX = -(jj+I*mu) \ je (1)
where E is all errors and I is the identity matrix. The
adaptive value mu is increased until the change results in a
reduced performance value [10].
III. RESULTS AND DISCUSSIONS
For the purpose of analysis, the regression of existing
IQMs with MOS will be discussed. Detailed correlation of
these IQMs based on categories of degradation is also
shown. Results of training and testing using hybrid metric
using LMBP is also presented.
A. Existing IQMs (PSNR, MSE and SSIM)
Results of objective measurements for PSNR, SSIM and
MSE are shown in Table I and Table II respectively. Table
I shows the correlation of objective measure with
subjective measure as overall while Table II shows the
correlation between the three objective measurements and
MOS for specific categories. From these results, the three
metrics did show good correlation with MOS when
measured based on categories. However, the correlation
dropped when an overall score was considered.
TABLE I
CORRELATION COEFFICIENT
Metrics SSIM PSNR MSE
Regression, R 0.5180 0.5007 0.5735
TABLE II
CORRELATION COEFFICIENT BASED ON CATEGORIES
Objective Assessment/
Category of Images SSIM PSNR MSE
Morphologically Altered 0.7144 -0.1360 0.7326
Noise-Added 0.9121 -0.8290 0.3851
Format Converted 0.0000 0.9000 -0.9449
Filtered 0.6844 -0.0459 0.7901
Table II indicates that the SSIM algorithm achieved a
satisfactory degree of correlation to the subjective
assessment for every category except the format converted
images. The MSE algorithm had satisfactory correlations
Proceedings of the International MultiConference of Engineers and Computer Scientists 2012 Vol I,
IMECS 2012, March 14 - 16, 2012, Hong Kong
ISBN: 978-988-19251-1-4
ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)
IMECS 2012
for 2 out of 4 of the categories and the PSNR performed
the worst with negative correlations for 3 out of 4
categories. The results revealed that even though objective
assessment could not achieved a satisfactory level of
correlation to the subjective assessment in overall, each of
the assessments did performed well in analyzing the image
quality for certain categories.
The following results show that by combining the three
unique metrics, an intelligent hybrid metric was obtained.
This is also supported by discussion of Table II that shown
each metrics has its own merits over another depending on
categories.
B. Network Predictive Ability and General Performance
Mean Squared Error (MSE) is performance metric
adopted to determine the network performance, while
regressions, R is used to measure the correlation between
outputs and targets. The fitting curve between targets with
inputs is shown in Fig. 2 and the MSE and Regression is
tabulated in Table III. MSE is the average squared
difference between outputs and targets. Lower values of
MSE are better as zero indicate no error while R value of 1
indicated closed relationship while 0 is a random
relationship.
(a)
(b)
Fig. 2 The fitting curve between output/target with input (a)
training only (b)training and testing
TABLE III
THE MSE and REGRESSION OF THE NETWORK
Model Training
Set Test
Set Whole
Set
MSE 0.2435 0.3995 0.2634
Regression, R 0.9236 0.854 0.8907
From the results in Table III and Fig. 2, it is proven that
Levenberg-Marquardt backpropagation algorithm has good
ability to predict the MOS with high regression correlation
for both training and testing. The training error is 0.244
and testing error is 0.399, indicative of about 70%
accuracy. Comparing the new hybrid LMBP metric with
the objective scores, Table IV shows the correlation of
PSNR, SSIM, MSE and new hybrid LMBP metric with
MOS in percentage (data is extracted from Table I and
Table III).
TABLE IV
COMPARISON OF REGRESSION (%) FOR THE FOUR IQMs
Metrics Regression, R
SSIM 51.8%
PSNR 50.1%
MSE 57.4%
LMBP 89.1%
It is shown that correlation with MOS is below 60% for
the existing metrics but the new LMBP metric is more than
80% correlated with MOS. Performance of the new metric
shown a promising result. Further investigation with larger
sample and proper selection of network properties will
definitely improve predictive ability and general
performance of ANN.
IV. CONCLUSIONS
From the study, new method for constructing image
quality metrics has been proposed. This paper is an attempt
to investigate the potential of combining existing metrics
with ANN to predict the quality of images. It was proven
that Levenberg-Marquardt backpropagation algorithm has
good ability to predict the MOS with high correlation for
training and testing with training error and testing error of
0.244 and 0.399 respectively. The regression, R showed
that it is highly correlated with mean opinion score (MOS)
compared to individual metrics (PSNR, MSE or SSIM).
In future work, more distortion measures and feature
domains will be used as the image samples. Also, the
relationship between the metrics adopted for the
combination will be further investigated to find the best
combination among them. More experiments are needed to
validate properties of the network such as it optimum
number of neurons in hidden layers, validation etc.
Performance comparison of LMBP with other networks
should also be discussed.
ACKNOWLEDGMENT
This research work is supported by the Universiti
Malaysia Sarawak (UNIMAS), under grant scheme
NF (F02)/144/2010(47). The research work was carried out
at Department of Electronic Engineering, Faculty of
Engineering, UNIMAS.
Proceedings of the International MultiConference of Engineers and Computer Scientists 2012 Vol I,
IMECS 2012, March 14 - 16, 2012, Hong Kong
ISBN: 978-988-19251-1-4
ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)
IMECS 2012
REFERENCES
[1] Wang, Z., Sheikh, H.R., & Bovik, A.C. (2003). Objective
Video Quality Assessment. Handbook of Video Databases:
Design and Applications (Chapter 41 pp. 1041-1078). CRC
Press.
[2] Saffor, A., Ramli, A., Ng., K., & Dowsett, D. (2002).
Objective and Subjective Evaluation of Compressed
Computed Tomography (CT) Images. Internet Journal of
Medical Simulation, Vol.1(1).
[3] Eskicioglu, A.M., & Fisher, P.S. (1995). Image Quality
Measures and their performance. IEEE Transactions on
Communication Vol. 43 (12).
[4] Ahmet, M., & Paul, S. (1995). Image quality measures and
their performance. IEEE Transactions on Communications,
Vol. 43, pp. 2959-2965.
[5] Choi, M.G., Jung, J.H., & Jeon, J.W. (2009). No-Reference
Image Quality Assessment using Blur Noise. International
Journal of Computer Science and Engineering. Vol. 3(2),
pp. 76-80.
[6] Cui, L., & Allen, A.R. (2008). An Image Quality Metric
Based on a Colour Appearance Model. Proceedings of the
10th International Conference on Advanced Concepts for
Intelligent Vision Systems, France, pp.696-707.
[7] Wang, Z., Bovik, A.C., Sheikh, H.R., & Simoncelli, E.P
(2004) Image quality Assessment: from error visibility to
structural similarity. IEEE Tran. Image Processing 13(4),
pp. 600-612.
[8] Hagan, M.T., & Menhaj, M. (1994). Training feed-forward
networks with the Marquardt algorithm. IEEE Transactions
on Neural Networks, Vol. 5, No. 6, 1999, pp. 989-99.
[9] Kermani, B.G., Schiffman, S.S., & Nagle, H.G. (2005).
Performance of the Levenberg–Marquardt neural network
training method in electronic nose applications. Science
Direct, Sensors and Actuators B: Chemical, Volume 110,
Issue 1, pp. 13-22.
[10] Charrier, C., Lebrun, G., & Lezoray, O. (2007). Selection of
features by a machine learning expert to design a color
image quality metrics. 3rd Int. Workshop on Video
Processing and Quality Metrics for Consumer Electronics
(VPQM), pp.113-119, Scottsdale, Arizona.
[11] Zampolo, R. & Seara, R. (2003). A measure for perceptual
image quality assessment. Proceeding IEEE International
Conference Image Processing.
[12] Cadik, M., & Slavik, P. (2004). Evaluation of two principal
approaches to objective image quality assessment. 8th
International Conference on Information Visualization:
IEEE Computer Society Press, pp. 513-518.
[13] Demuth, H., & Beale, M., Matlab Neural Network Toolbox
Help, Version R2008a, The MathWorks, Inc.
Proceedings of the International MultiConference of Engineers and Computer Scientists 2012 Vol I,
IMECS 2012, March 14 - 16, 2012, Hong Kong
ISBN: 978-988-19251-1-4
ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)
IMECS 2012