10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010)
Finding the Number of Hidden Neurons for an MLP
Neural Network Using Coarse to Fine Search
Chelsia Amy Doukim, Jamal Ahmed Dargham, Ali Chekima
Computer Engineering Program, School of Engineering and Information Technology
Universiti Malaysia Sabah, Locked Bag No. 2073, 88999 Kota Kinabalu
email@example.com, firstname.lastname@example.org, email@example.com
Abstract- Skin detection is an important preliminary process
for subsequent feature extraction in image processing techniques.
There are several techniques that are used for skin detection. In
this work, the multi-layer perceptron (MLP) neural network is
used. One of the important aspects of MLP is how to determine
the network topology. The number of neurons in the inputs and
output layers are determined by the number of available inputs
and required outputs respectively.
remaining is how to determine the number of neurons in the
hidden layer. Therefore, we employed the coarse to fine search
method to find the number of neurons. First, the number of
hidden neurons is initially set using the binary search mode,
HN=l, 2, 4, 8, 16, 32, 64 and 128, where HN indicates the number
of hidden neurons. The 30 networks with these HN values are
trained and their Mean Squared Error (MSE) is calculated. Then
a sequential search, fine search,
neighbourhood of the HN that gave the lowest MSE. The selected
number of neurons in the hidden layer is the lowest HN that gave
the lowest MSE. The YCbCr colour space is used in this work due
to its capability to separate the luminance and chrominance
components explicitly. Several chrominance components are
Thus, the only thing
will be used in the
Index Terms-Skin detection, Multi-layer perceptron,
Skin detection is an important preliminary process for
subsequent feature extraction in a wide range of image
processing techniques ranging from face detection, face
tracking, gesture analysis, content-based image retrieval
systems and various computer vision applications. There are
several techniques for skin detection. In this work, the multi
layer perceptron (MLP) neural network is used for skin
detection. A multi-layer perceptron is a feedforward artificial
neural network model that maps sets of input data onto a set of
appropriate output. A multi-layer perceptron has three typical
characteristics, first is the model of each neuron in the network
includes a nonlinear activation function, second, the network
consists of one or more layers of hidden neurons that are not
part of the input or output of the network and third, the network
reveals a high degrees of connectivity which determined by the
synapses of the network. One of the important aspects of MLP
is how to determine the network topology. The inputs are
dictated by the number of features of available inputs and the
978-1-4244-7167-6/101$26.00 ©2010 IEEE
outputs are dictated by the number of classes. Hence, the only
thing remaining is how to determine the number of neurons in
the hidden layer. Currently there is no official way of
determining the size of a neural network, according to
complexity of the problem to solve, the solutions being
searched by means of empirical testing or trial and error. This
approach, although straightforward, is rather inefficient since
many networks may have to be trained before an acceptable
one is found . There are two existing techniques for
determining the number of hidden neurons namely network
growing and network pruning. Growing algorithms have been
used in neural networks to allow adaptation of the network
structure to optimal values . Growing algorithm starts with
an initially undersized number of hidden neurons and adds
neurons to the hidden layer until the design specifications are
met . This approach is time consuming and there is no
guarantee that the selected number of hidden neurons is the
best. On the other hand, the pruning algorithm starts with an
initially oversized number of hidden neurons and then the
network is pruned by eliminating certain synaptic weights in a
selective and orderly approach . The problem with this
approach is that we need to determine the size of the initial
oversized number of hidden neurons. Recently, the growing
and pruning algorithms were combined to determine the size of
the hidden layer for control of linear plants in , for
identification of time varying non-linear systems in , and for
use on a delta rule neural network in . In this work, the
number of hidden neurons is determined in two approaches. In
the first approach, the number of neurons is determined using
the binary search. Secondly, the sequential search around a
fixed range is performed in order to find the best number of
hidden neurons. The paper is organised as follows. Section 2
briefly describes the data preparation. In Section 3, the neural
network properties of this work are explained. Section 4
explains the method for finding the number of neurons in the
hidden layer. Section 5 gives the experiments results and
Section 6 concludes the paper.
The database used in this work is the famous training and
test database for skin detection namely Compaq database .
This database consists of 13,640 images with its corresponding
masked images. Figure 1 shows the example of images and
their corresponding masked images. There are three sets of data
which are the training dataset, validation dataset and test
dataset. Training data and validation data consist of 420,000
randomly selected pixels from the images of the Compaq
database respectively, which makes a total of 840,000 pixels.
Both training data and validation data are split into 30 data files
correspondingly where each data file consists of 14,000 pixels.
Each data file will be treated as a network, which means there
are a total of 30 networks. The testing data is used to evaluate
the performance of the skin detection algorithm. The test
dataset comprises of 100 randomly picked images from the
Figure I. Example of images from the Compaq database with their
corresponding masked images.
NEURAL NETWORK PROPERTIES
The network topology used in this work is one hidden layer
multi-layer perceptron. The input layer for the MLP will have
one neuron for network using single chrominance from the
YCbCr colour space. The output layer will have one neuron
decoded as 1 for skin and 0 for non-skin. Therefore, the neural
network topology will be I-HN-1 where HN indicates the
number of neurons in the hidden layer. The training algorithm
used is the Levenberg-Marquardt because of its rapid
convergence time compared to other faster training algorithm
such as Conjugate gradient and Quasi-Newton. The transfer
function applied is of sigmoid function. The maximum number
of epochs is set to 500 and the training goal is set to O.
FINDING THE NUMBER OF NEURONS IN THE HIDDEN
The technique used in finding the number of neurons in the
hidden layer is known as coarse to fine search and it is done in
two approaches. First, the number of hidden neurons is
initially set using the binary search mode, HN=I, 2, 4, 8, 16,
32, 64 and 128, where HN indicates the number of hidden
neurons. A total of 30 networks with these HN values are
trained and their Mean Squared Error (MSE) is calculated.
Figure 2 gives the results for the Cb-Cr chrominance
component. From the figure, the lowest MSE is given by
HN= 128 with MSE of 0.041l. The first approach is stopped at
HN=128 because larger number of hidden neurons requires
very long training times.
For the second approach, a sequential search, or fine search
around fixed range of the chosen HN is done in order to obtain
the smallest HN value that gives the lowest MSE. In this case,
we searched from HN = 96 to HN = 128. As can be seen from
Figure 3, the lowest MSE obtained is 0.0411 given by HN =
112, which is similar to the MSE value given by HN = 128.
Thus, the number of hidden neurons for Cb-Cr chrominance
network is HN = 112. Table I provides the fixed network
structures for each chrominance component studied in the
YCbCr colour space.
Sequential search range
1-1-1 1-2-1 1-4-1 1-8-1 1-16-1 1-32-1 1-64-1 1-128-1
Figure 2. Coarse binary search for the Cb-C, chrominance.
NEURAL NETWORK FOR SKIN DETECTION
As for the evaluation of the skin detection performance,
three performance metrics are calculated. The first metric is
the correct detection rate, CDR and is given in Equation l.
The false acceptance rate
FAR is the percentage of
identification instances in which false acceptance occurs. For
example, an unauthorized person is identified as an authorized
one. The false rejection rate FRR is the percentage of
identification instances in which false rejection occurs. This is
the case when the system fails to recognize an authorized
person and rejects that person as an impostor. The FAR and
FRR are expressed in Equation (2) and (3) respectively.
CDR = Number of pixels correctly classified
Total pixels in the test dataset
FAR = Number of non-skin pixels classified as skin pixels
Total pixels in the test dataset
FRR = Number of skin pixels classified as non-skin pixels
Total pixels in the test dataset
.... .... .. .... .... .... .... .... .... .... .,. .,. ....
d-, 0 .!! N A
0'1 q-. 0'1 q-. 0
.... .... .. .... .... .,. .,. .,. .,. .,. .,. .,. .,. "
.... .... .... .... .... .... .... ....
0 0 0 0 :=; 0 c
Figure 3. Sequential search for the Cr chrominance.
The fixed network structure from each chrominance
component of the YCbCr colour space is tested on 100
images from the test dataset in order to evaluate the skin
detection performance. Table II shows the skin detection
results for several chrominance combinations. As can be
seen from Table II, the combination of Cb-Cr chrominance
component gave the highest correct detection rate followed
by CJCr• The best and the worst segmented images are given
in Figure 4 and Figure 5 respectively. The white colour
indicates the skin and the black colour indicates the non-skin.
FIXED NETWORK STRUCTURES FOR EACH CHROMINANCE
COMPONENT IN THE YCBCR COLOUR SPACE
.,. " .... '7 '7 '7 '7 '7 '7 '7 .,. '7 .,. '7 " '7 " '7 '7 '7
.... .... .... .. .... .... .... .... .... .... .... .... .... ....
.... .... .. .,. .... .... .... .... .... .,. .... .... ....
oJ, J, r-!. cO
.... .... .. .... .... .... .... .... .... .... N
, .... .... .. ....
J, r-!. cO
0 .!! N
N " N " N
.... .... .. ....
A COMPREHENSIVE SKIN DETECTION PERFORMANCE
RESULTS IN THE YCllCR COLOUR SPACE
78.63 20.50 0.86
Figure 4. The best segmented image segmented using Cb-Cr chrominance
component. Original image (left), segmented image (right).
Figure 5. The worst segmented image segmented using Cb-C, chrominance Download full-text
component. Original image (left), segmented image (right).
In this work, a modus operandi of finding the number of
hidden neurons in the hidden layer of a neural network is
proposed. The technique is separated into two parts; the first
part used binary search approach where the HN is set to
HN=l, 2, 4, 8, 6, 32, 64 and 128. Secondly, a sequential
search around a fixed range is used in the neighbourhood of
the HN that gave the lowest MSE. The lowest HN value that
gave the lowest MSE is then selected to construct the neural
network structure. The fixed network structures were then
tested on 100 images from the test dataset which comprises
images from the Compaq database. The skin detection result
obtained is almost the same as the detection rate of 80%
given by Jones and Rehg  albeit using different colour
R. Reed, "Pruning algorithm - A survey," IEEE Trans. Neural
Networks, vol. 4, pp. 740-747, Sep. 1993.
S. Haykin, Neural Networks: A Comprehensive Foundation. Upper
Saddle River, NJ: Prentice-Hall, 1993, pp. 218-226.
S. E. Fahlman and C. Lebiere, "The cascade-correlation learning
architecture," in Proc. Advances in neural information processing
systems 2, San Francisco, 1990, pp. 524-532.
P. N. Suganthan, E. K. Teoh, and D. P. Mital, "'Multilayer
backpropagation network for flexible circuit recognition," in Proc.
Industrial Electronics, Control, and Instrumentation, Maui, 1993, pp.
 Q. Y Zhu, G. B. Huang, and C. K. Siew, "A real-time learning
algorithm for two-hidden-Iayer feed forward networks," in Proc.
Control and Automation, Montreal, Canada, 2003, pp. 496-500.
time-varying non-linear systems using minimal radial basis function
neural networks," in Proc. Control Theory Applications, vol. 144, No.
2, 1997, pp. 202-208.
neural networks," in Proc. World Congress on Computational
Intelligence, Orlando, 1994, pp. 1403-1408.
applications to skin detection," International Journal of Computer
Vision, vol. 46, pp. 81-96, Jan. 2002.
L. Yingwei, N. Sundararajan, and P.Saratchandran, "Identification of
K. Khorasani and W. Weng, "Structure adaptation in feed-forward
M. 1. Jones and J. M. Rehg, "Statistical color models with