Content uploaded by Mohammad A. N. Al-Azawi
Author content
All content in this area was uploaded by Mohammad A. N. Al-Azawi on Jan 17, 2018
Content may be subject to copyright.
International Journal of Computer Applications (0975 – 8887)
Volume 180 – No.10, January 2018
27
Computational Intelligence-based Semantic Image
Background Identification using Colour-Texture Feature
Mohammad Al-Azawi
Oman College of Management & Technology
Barka, Oman
ABSTRACT
Most of the studies in image contents recognition and
identification focus on identifying the objects that the image
contains. Although, background identification is an important
part of any image contents identification and description, it
has received less attention. Background identification is as
important as the objects in the image in describing the image
contents. This paper presents a new algorithm to identify the
background in an image which utilises the principles of
human image understanding and the way they describe the
contents of the image. Because backgrounds are with less
variation as compared to objects variations in different
images, neural network can be used in recognising them.
Usually backgrounds can be described based on their texture
and colour; thus, these two features will be used in the
identification process. Texture is described using measures
extracted from Grey Level Co-occurrence Matrices (GLCMs
and singular GLCM). Since different textures may produce
similar GLCMs, colour features can be used to discriminate
among them. Fuzzy Standard Colours are presented in this
research which is a simulation to the way the human describe
the colours. Additionally, Fuzzy intelligence is used to
combine the two features to decide the type of the
background.
General Terms
Image Processing, Computational Intelligence, Neural
Network, Fuzzy Logic, Image Contents Identification, Image
understanding, Fuzzy Standard Colours.
Keywords
Image, Fuzzy, Background, Identification.
1. INTRODUCTION
Image contents recognition and identification is still a
challenging topic for researchers. Although, numerous
researches are published every year, still the obtained results
are not quite satisfactory. Studying the principles of Human
Vision System (HVS) has inspired many researchers to design
images identification systems that utilise these principles.
In order to give the machine the ability to recognise and
understand the contents of the image, one needs to convert the
image from the visual, or spatial domain, into a form that can
be processed and measured by the machine which is the
features domain. After this conversion, the features in the
features domain are converted into descriptors in the
description domain. Features domain includes many types of
features such as colour and texture which are widely used to
describe the contents of the image.
Texture feature has been employed widely in image
identification since colour is not sufficient to describe them.
Several measures have been proposed to describe the texture
such as wavelet, Gabor filters and GLCMs. GLCMs provide
description for the relation of the pixel with the surrounding
pixels, which can be considered as a description of the texture.
Since GLCMs are extracted from the grey level image, which
means, only the intensity values are used in determining
GLCMs, then the colour should be used as well to give a
complete description to the background.
Computational Intelligence (CI) techniques, such as Neural
Networks (NNs and singular NN) and Fuzzy Intelligence (FI)
are used to mimic the human abilities in identifying the
images contents. Various NNs have been adopted due to their
ability of learning and being used in the recognition phase.
Since both the inputs and the corresponding outputs can be
provided to the NNs, then supervised learning can be used and
it works efficiently. Different learning techniques such as
Back Propagation (BP), Evolutionary Learning (EL) and
Delta Rule (DR) can be used for training the NNs. Fuzzy
Intelligence is used to combine the properties that are
obtained from NN’s and to specify the type of the
background.
Numerous literatures have been published in the field of
image content recognition, texture analysis, and
computational intelligence. Most of the proposed algorithms
are domain-oriented; i.e. they are not general and cannot be
applied on general image datasets. In other words, they deal
with a specific field of applications such as medical, GSI,
weather, geological applications, etc. Medical applications
such as tumour identification and MRI recognition were the
focus of numerous researches such as Ref. [1], [2], [3] and
many others. Atmosphere and weather applications are
another fields that utilise GLCMs and NNs in their
identification process [4] [5]. Similar techniques have been
used in applications such as in fabric defection identification
[6], coal/rock identification [7], fruit identification [8] and
others.
In all of the aforementioned applications, in additional to
other application, it is noticed that the images used in the
dataset are related to each other, e.g. skin tissues, MRI,
clouds, rocks, etc. In each images dataset, the images contain
very similar texture and usually the algorithm role is to
identify the abnormal cases like in tumour identification. NNs
work effectively in such applications since they do not need to
be retrained frequently when the data is changed.
In this research, GLCMs, NNs and Fuzzy intelligence, in
additional to the colour features, are used to design a new
algorithm that can identify different backgrounds in an image.
The proposed technique overcomes the problem of retrain the
NNs frequently when a new texture is added, especially when
it is applied on images from different domains. Instead of
retraining the NNs frequently, Fuzzy Interface System (FIS) is
used, which made the proposed algorithm more flexible by
International Journal of Computer Applications (0975 – 8887)
Volume 180 – No.10, January 2018
28
defining new backgrounds in the rulebase based on the results
obtained from the NNs.
2. BACKGROUND AND THEORY
This section focuses on the background and the needed theory
upon which the rest of the paper will be built. Image
background identification is an important part of image
contents identification as it gives information about the nature
of the image. Most of the image contents identification
algorithms focus on the objects in the image without giving
sufficient information about the background. When a person
describes a scene, he, usually, gives description for the object
and the surrounding environment such as a ball in a field or
playing yard. Based on this, background identification is as
important as the object identification.
Colour feature is used to describe the contents and the nature
of the image in many applications. The most well-known
representation for colour feature is the histogram, which gives
good description for colour distribution in the image but it
suffers from a certain weakness, which is that it losses the
space information of the colour [9]. In addition, different
image may have similar histograms such as the sky and the
sea or tree leaves and grass; therefore, texture is used to solve
such problems. When it refers to the description of the
image’s texture, texture’s statistical feature and structural
feature, as well as, frequency domain (spectral) features can
be used [10]. Although texture is not well defined, as in the
case of colour feature, still it can give good description for the
contents in images that contain objects such as cloud, trees,
bricks, and fabric. Texture features can be extracted using
various approaches such as Gabor filter, wavelet transform
and local statistics measures. The paper’s main interest is in
the statistical approach which includes many techniques such
as the Moments of Intensity and Grey Level Co-occurrence
Matrices.
2.1 Grey Level Co-occurrence Matrices
(GLCMs)
GLCMs, which were presented by Haralick [11], are used to
describe the relation of each pixel’s grey intensity level with
its neighbours. They are based on the joint probability
distributions of pairs of pixels, each GLCM can be specified
in a matrix of relative frequencies with which two
neighbouring pixels separated by a distance and angle
occur on the image, one with grey level and the other with
grey level [12].
The relative frequencies of grey level pairs of pixels separated
by a distance in the direction are combined to form a
relative displacement vector = (,). This vector is
computed and stored in the GLCMs (). These matrices are
used to extract second-order statistical texture features.
GLCMs are used to describe the texture since they measure
the relation between the adjacent pixels displaced by a certain
distance. GLCMs are, usually, described in terms of
displacement and angle , i.e. = (,,,). The
angle often takes standard values such as, 0°, 90°, 180°, etc.
Instead of using the angle , two different displacements
and can be used. The GLCM is then can be defined as
follows:
(,) = 1
(,)
=0
=0
1
,= 1 ,= +,+=
0
where is a GLCM with size of ×, and is the number
of grey levels (intensity levels) in the image.
Several measures can be extracted from the GLCMs which
can be used to describe the texture. In our application, two
important features for the texture will be defined; these
features are coarseness and regularity.
2.1.1 Coarseness
It was noticed that, the shape of the GLCMs is dependent
upon the coarseness of the texture; this is because in case of
fine texture, GLCMs extraction process produces GLCMs
with a clear spot. For instance, assume that ,,, 0 is
calculated, which means only the horizontal relationship
between the adjacent pixels is considered. For coarse texture,
if the width of the textures is , then if is less than , the
value of at some grey level with respect to itself will be
higher, i.e. g, will be higher than the case of fine texture
in which the ratio of / is higher than that for coarse
texture. In other words, the number of pixels at which
,= +,+ is higher in coarse texture than
in fine texture. Thus, the width of the GLCMs shall be
considered as a measure of the coarseness of the texture.
From Table 1, it is clear that for fine texture, the GLCM is
concentrated in one region (spot) for fine texture, while its
size is larger and there are some values corresponding to the
same intensity values for coarse texture. The location of the
spot in the 2D GLCM or the centre in 1D GLCM has been
affected by the brightness of the image, which may affect the
result slightly. This problem can be solved by shifting the
centre of the spot to a fixed position.
Table 1. The effect of coarseness on GLCM
Texture
2D GLCM
1D GLCM
Fine texture
Coarse Texture
2.1.2 Regularity
Regularity is another important property of the texture that
can affect the shape of the GLCM. Regularity measures the
repetition of the texture in the image, e.g. the grass is regular
since the same texture is repeated continuously, while the
cloudy sky is irregular. The irregularity effect on the GLCMs
is characterised by having more than one spot in the GLCMs,
one spot for each part, e.g. in the case of cloudy sky there are
two spots one for the blue part and one for the clouds. Thus,
the GLCMs shall be divided into more than one part i.e.
G = G1+ G2++ Gm; where for = 1,2, . . are the
GLCMs corresponding to different parts of the texture, and
each one produces its own spot since they are calculated for a
International Journal of Computer Applications (0975 – 8887)
Volume 180 – No.10, January 2018
29
specific range of grey levels. Figure 1 shows the effect of
irregularity on GLCMs; from the figure, it is obvious that
there are two major peaks for the 1D GLCMs: one for the
clouds and one for the blue background and two spots in the
2D GLCM for the same reason.
(a)
(b)
(c)
Fig 1: The effect of irregularity on the GLCM, (a) original
image, (b) 2D GLCM 16 grey levels, (c) 1D GLCM 16 grey
levels
To measure the texture features we shall use different
measures that can be extracted from 1D GLCMs, such as the
mean , standard deviation , range , number of peaks
(tops) , and average of the tops . Each one of the measures
is affected by the properties of the texture, e.g. the coarseness
affects the range, and the regularity increases the number of
peaks.
2.2 Neural Networks
Neural Computing is widely used in image identification
algorithms, since NNs have the ability of adaptation and
learning. In such algorithms, the input to the NN is a vector of
features that are extracted from images. A set of input vectors
is used to train the NN so it can recognise other vectors that
are extracted from other images. Although both supervised
and unsupervised learning techniques can be used in image
classification, in most of the proposed systems, supervised
learning is used because it gives the possibility to improve the
identification performance. Such retrieval systems utilise user
feedback as one of the inputs to the supervised learning NN.
NNs suffer from a certain constraint, which is they can be
used effectively to identify images that are related to each
other, such as MRI, therefore, they are widely used in medical
applications; but it is not feasible to use them in identifying
image contents with different natures since they need to be
retrained with every new type. Thus, most of the applications
of NNs in image retrieval systems are in relevance feedback
[13], general classification [14] and human computer
interaction [15].
2.3 Fuzzy Intelligence
Fuzzy intelligence provides excellent way to convert
quantitative numerical values into qualitative values by using
linguistic variables, which can store linguistic values instead
of numerical values. These linguistic values are specified by
the selected membership function. To process the linguistic
variables, we need to use what is called Fuzzy rules, which is
a set of (if-then) relationships. The set of linguistic variables
is known as the database and the set of rules is known as
rulebase, these two together form important part of the Fuzzy
interface system FIS. The use of Fuzzy intelligence in the
proposed algorithm is to simulate the uncertainty in the
human visual system.
3. PROPOSED ALGORITHM
The proposed algorithm has been built upon the following
assumptions: firstly, no frequent retraining for the NNs should
be performed; secondly, identifying the texture coarseness and
regularity in additional to the colour using NNs; and thirdly,
using Fuzzy intelligence to combine the results obtained from
the NNs to decide the texture type. The role of the NNs in the
proposed algorithm is to identify the background properties
such as coarseness, regularity and colour.
A bank of NNs, which consists of three NNs, was used. The
purpose of the first NN is to identify the coarseness, the
second one is used to identify the regularity and the last one is
dedicated to identifying the colour. The inputs to the first two
NNs are the measures of the reduced 1D GLCMs. The idea of
using the reduced GLCMs and not the original ones is to
reduce the complexity of calculating the measures of the input
vectors. The reduction in GLCM does not affect the result
much since the GLCMs maintain the correspondence with the
texture. Fig 2 shows the effect of reducing the size of GLCMs
from 256 grey level (GL) to 16 GL.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
Fig 2: 16 and 256 GL GLCM (a) original image, (b) 16 GL
GLCM, (c) 256 GL GLCM, (d) 16 GL 2D GLCM
histogram, (e) 256 GL 2D GLCM histogram, (f) 16 GL 3D
GLCM contour, (g) 16 GL 3D GLCM contour, (h) 16 GL
1D GLCM histogram, (i) 256 GL 1D GLCM histogram
The input to the third NN is the colour components of the
image. Hue, Saturation and Value (HSV) colour system is
used in identifying the colour since it is analogy to Human
Vision Systems and Munsell Well. It is possible to obtain
most of the Fuzzy Standard Colours (FSC) using HSV. In
FSC, we aim at finding a way to describe the colour in a way
similar to human description, which means, to give a
possessive values for the colour, which might be represented
by a Fuzzy membership function. For example, we may
describe the colour as Reddish Brown, which means the value
for the brown membership value is higher than that for red,
but still there is a value for the red.
Figure 3 shows the block diagram of the proposed algorithm.
NN1 is used to identify the coarseness, NN2 is used to
identify the regularity and NN3 is used to identify the colours.
The FIS consists of the linguistic database which contains
linguistic variable such as Colour= {Green, Brown, Blue,
etc.}, Coarseness= {very fine, fine, course, very Coarse},
Regularity= {very regular, regular, irregular, very regular}
International Journal of Computer Applications (0975 – 8887)
Volume 180 – No.10, January 2018
30
and finally texture = {field grass, beach sand, desert, sky,
cloudy sky, etc.}. In additional to the linguistic database,
rulebase is defined as well. Rules such as “IF the coarseness
IS smooth and the Regularity IS very regular and the Colour
IS light brown THEN the texture IS Desert Sand” can be
used to identify the texture.
4. EXPERIMENTAL RESULTS
The above algorithm has been applied on different sets of
backgrounds with different textures. The standard colours
used to train the neural network were obtained from different
users feedbacks with different backgrounds. The standard
colours set contained colours like red, brown, green, blue,
yellow, white, black, purple, orange, and grey.
Fig 3: The proposed algorithm block diagram
The colour experimental results are shown in Figure 4, in
which different colours have been labelled based on the Hue
value with constant value for the saturation and value.
Fig 4: FSC vs. Hue, Saturation=0.5 and Value = 0.5
Some other FSC can be obtained by changing the values of
the Saturation and value. The names of the FSC along with
the HSV values are used to train the neural network NN3, and
then it is used to identify other colours. NN3 has number of
outputs equals to the number of FSC used. Thus, the output
corresponding to a specific colour is considered as the
membership value for that colour, e.g. if the output of the red
colour is 0.7 and that of the brown is 0.3, that means the
colour is brownish red.
In the same way, the output of NN1 gives how coarse the
texture is, e.g. if the output is 0.9 means very coarse, 0.7
means coarse, and 0.2 means fine. Finally, NN2 gives how
regular the texture is. The outputs of the neural networks are
then used to define the type of the texture in FIS. The
following table shows examples of the results obtained from
applying the proposed algorithm.
From Table 2 one can notice that in case 1,
is smaller than that of case 2, which means that the first
textures is with higher coarseness than the second one. The
value of in (1) is higher than that in (2) for the same reason.
The number of peaks in (1) is 1 while in (2) is 3 which means
that the first image is regular and the second one is with less
regularity.
Table 2. Measures obtained from applying the proposed
algorithm
#
Image
Colour
1
104
23
151
1
1
0.6:
Green
0.1:
Brown
2
137
56
187
3
0.64
0.9:
Blue
0.002:
Green
By using the trained neural networks and the FIS, the results
given in Table 3 have been obtained.
Table 3. Results obtained from applying the proposed
algorithm
#
Image
Coarseness
Regularity
Colour
1
0.87
Coarse
0.78
Regular
0.6:
Green
0.1:
Brown
2
0.73
Fine
0.68
Irregular
0.9:
Blue
0.002:
Green
Based on the results in the above table, the description of the
image in (1) will be a medium coarseness, pale green grass,
and the image in (2) is Day, blue, partially cloudy sky.
Table 4 gives the details of NNs and the training data.
Table 4. Neural Networks details
NN
Number of Neurons
in
Training
Images
Epochs
Elapsed
Time (sec)
Min Error
input
hidden
output
NN1
5
5
1
20
1300
0.2
0.09
9
NN2
5
5
1
20
25000
2.3
0.09
9
NN3
3
3
10
20
11300
2.5
0.09
9
The number of input neurons for each neural network is equal
to the number of measures used in identifying the image
properties. The algorithm was tested on 250 different
background images and the results are shown in Table 5.
.
.
NN1
.
.
NN2
.
.
NN3
Fuzzy
Interface
System
Identified
background
GLCM
Colour
International Journal of Computer Applications (0975 – 8887)
Volume 180 – No.10, January 2018
31
Table 5. The percentage of correct results
Colour
Regularity
Coarseness
Class
88%
84%
73%
77%
It is noticed from the obtained results that the algorithm
worked perfectly on backgrounds with textures such as grass,
sand, tree leaves, sea, clouds, stones, and many others. The
percentage of wrongly classified backgrounds were in fine
clouds and in bricks. In the first case the fine clouds were
similar to sea and in the case of bricks, the texture of the brick
itself has been considered and not the texture of the wall.
Another important merit of the proposed algorithm is that, due
to the use of Fuzzy intelligence, it is possible to have more
than one class for the same image, for example the image
shown in Table 6.
Table 6. A texture may belong to more than one class
Image
Colour
Coars
eness
Reg
ulari
ty
Class
0.9999
6
Brown
0.12
0.9
Sand
0.99
Brown
0.981
5
0.76
Gravel
0.9933
Brown
0.44
0.78
0.4 Coarse
Sand OR 0.6
Fine Gravel
5. CONCLUSION
In this paper, an integrated identification system has been
proposed. This system can be used to identify the texture
image in general, and as an application on the proposed
algorithm, we applied it on background identification. The
system has some semantic properties since it utilises CI and
FIS and inspired by the human visual system. The system uses
two main features, which are the colour and texture; these
features are used by HVS to identify the contents of the
image. Uncertainty is one of the important features of HVS,
which can be utilised to have more possibilities or definition
of the image. The uncertainty has been represented by using
the FIS in terms of colour, coarseness, and regularity
membership function.
In addition, NNs play important role in the identification
process as they give the possessive value of a specific input to
some class. These values can be used, in their turn, to identify
the texture properties. The second important advantage of the
proposed algorithm is that it overcomes the main limitation of
NNs in identification, which is retraining them whenever
there is a new class. This limitation has been overcome by
using the FIS. Instead of retrain the NN, one can define a new
rule in the FIS.
The algorithm can be used in many applications which uses
texture-colour features in identifying the contents, such as
medical, atmosphere, geological and industrial applications.
6. REFERENCES
[1] S. Maniar, and J. S. Shah, “Classification of Content
Based Medical Image Retrieval Using Texture and Shape
Feature with Neural Network,” Int. J. Adv. Appl. Sci.,
vol. 6, no. 3, pp. 378–384, 2017.
[2] S. Anand, “Segmentation coupled textural feature
classification for lung tumor prediction,” in IEEE
International Conference on Communication Control and
Computing Technologies (ICCCCT), 2010.
[3] D. Kadam, S. Gade, M. Uplane and R. Prasad, “An
Artificial Neural Network Approach for Brain Tumor
Detection Based on Characteristics of GLCM Texture
Features,” Int. J. Innov. Eng. Technol., vol. 2, no. 1, pp.
193–199, 2013.
[4] B. Tian, et al., “Neural network-based cloud
classification on satellite imagery using textural
features,” in International Conference on Image
Processing, 1997.
[5] B. Tian, et al., “A study of cloud classification with
neural networks using spectral and textural features,”
IEEE Trans. Neural Networks, vol. 10, no. 1, pp. 138–
151, 1999.
[6] J. Raheja, et al., “Fabric defect detection based on
GLCM and Gabor filter: A comparison,” Opt. - Int. J.
Light Electron Opt., vol. 124, no. 23, pp. 6469–6474,
2013.
[7] L. Haonan, et al., “Research on identification of coal and
waste rock based on GLCM and BP neural network,” in
2nd International Conference on Signal Processing
Systems (ICSPS), 2010.
[8] G. Capizzi, et al., “A novel neural networks-based
texture image processing algorithm for orange defects
classification,” Int. J. Comput. Sci. Appl., vol. 13, no. 2,
pp. 45–60, 2016.
[9] H. Yang, and Z, Xuemei, “Research of Content Based
Image Retrieval Technology,” in Proceedings of the
Third International Symposium on Electronic Commerce
and Security Workshops(ISECS ’10), 2010.
[10] N. Singhai, and K. Shishir, “A Survey On: Content
Based Image Retrieval Systems,” Int. J. Comput. Appl.,
vol. 4, no. 2, pp. 22–26, 2010.
[11] L. Haralick, and R. Watson, “A facet model for image
data,” Comput. Vision, Graph. Image Process, no. 15, pp.
113–129, 1981.
[12] S. Elberink, and M. Hans-Gerd, “The use of Anisotropic
height texture measures for the segmentation of airborne
laser scanner data,” IAPRS, vol. XXXIII, 2000.
[13] S. Nematipour, J. Shanbehzadeh, and R. Moghadam,
“Relevance Feedback Optimization in Content Based
Image Retrieval Via Enhanced Radial Basis Function
Network,” in Proceeding of the International
Multiconference of Engineers and Computer Scientists
IMECS, 2011.
[14] N. Qazi, and B. Wong, “Semantic based image retrieval
through combined classifiers of deep neural network and
wavelet decomposition of image signal,” in 9th IEEE
EUROSIM Congress on Modelling and Simulation
(EUROSIM), 2016.
[15] H. Lee, and S. Yoo, “Intelligent image retrieval using
neural network,” IEICE Trans. Inf. Syst., vol. E84–D, no.
12, pp. 1810–1819, 2001.
IJCATM : www.ijcaonline.org
