Content uploaded by Khaled Elleithy
Author content
All content in this area was uploaded by Khaled Elleithy
Content may be subject to copyright.
Content uploaded by Varun Pande
Author content
All content in this area was uploaded by Varun Pande
Content may be subject to copyright.
Parallel Processing for Multi Face Detection and
Recognition
Varun Pande, Khaled Elleithy, Laiali Almazaydeh
Department of Computer Science and Engineering
University of Bridgeport
Bridgeport, CT 06604, USA
{vpande, elleithy, lalmazay}@bridgeport.edu
Abstract— In this paper, a robust approach for real time face
recognition where the images come from live video is proposed.
To improve the algorithmic efficiency of face detection, we
combine the eigenface method using Haar-like features to detect
both of eyes and face, and Robert cross edge detector to locate the
human face position. Robert Cross uses the integral image
representation and simple rectangular features to eliminate the
need of expensive calculation of multi-scale image pyramid.
Moreover, In order to provide fast response in our system, we use
Principal Component Analysis (PCA) to reduce the
dimensionality of the training set, leaving only those features that
are critical for face recognition.
Eigendistance is used in face recognition to match the new face
while it is projected on the face space. The matching is done when
the variation difference between the new image and the stored
image is below the threshold value.
The experimental results demonstrate that the proposed scheme
significantly improves the recognition performance. Overall, we
find the system outperforms other techniques. Moreover, the
proposed system can be used in different vision-based human
computer interaction such as ATM, cell phone, intelligent
buildings, etc.
Keywords: Face Detection, Eyes detection, Face Recognition,
Haar- like features, PCA, Eigenfaces, Roberts Cross Edge Detector,
Human Computer Interaction, Real-time System.
I. INTRODUCTION
As the famous proverb says, “Face is the index of the
mind”. A face to face interaction between human beings is
considered most important and natural way to communicate.
Recognition of faces and processing the data is a challenging
task with very large database using low-cost desktop embedded
computing.
The face recognizing systems available so far maintain a
database that has each pre-processed human faces and their
corresponding unique features as determined from each face.
These features are stored with the respective individual’s face.
Therefore, when a query is raised, the unique features are
extracted from the face and then compared with the features in
the database and results in a perfect match [1].
There are many areas of applications that use face
recognition technology ranging from security applications for
recognizing criminals in public spaces such as airports, and
shopping centers, verifying access to private property, and
casting votes, to intelligent vision-based human computer
interaction such as ATM, cell phones and intelligent buildings
[2].
Identifying the images and processing the data is a
challenging task because of the various factors involved in this
sophisticated process such as illumination, angle of pose,
accessories, facial expression, and aging effects [2].
There are two sets of data involved in face recognition
system. The first is the training set of data that is used in the
learning stage. The second is testing set which is used during
recognition [3].
There are numerous technologies and algorithms used in
face recognition systems and the most popular among them is
the Eigenfaces algorithm which we have implement in our
system. Often real time response is understood to be in order of
milliseconds and sometimes microseconds, which is the most
crucial criteria in the system design.
The rest of this paper is organized as follows. In Section II,
we glance at a variety of face detection and recognition
methods. Section III contains an overview of the system,
including a description of the Eigenface with Haar-like
features, Robert cross edge detector, and details on the
analysis methodology of the paper. We describe the steps to
determine face detection and recognition. In Section IV, we
provide details of the experiment and the results of our system.
Finally, Section V concludes this paper regarding the potential
usefulness of our system, and highlights some directions for
future research.
II. R
ELATED WORK
Several methods have been suggested for face recognition
over the past few years and a recent survey could be found in
[4]. The most common techniques used in face recognition are
Principal Component Analysis (PCA), Partitioned Iterated
Function System (PIFS), Local Feature Analysis (LFA),
Wavelets and Discrete Cosine Transform (DCT), Neural
Network, Template Matching, and Model Matching. The
choice of using a particular method is specified by its
suitability for a specific application [5].
In face recognition by Template Matching [6], salient
regions of the facial image are extracted, and then these regions
are compared on a pixel-by-pixel basis with an image in the
database. The advantage of this method is that the image
preprocessing is simple, but the database search and
comparison are computationally expensive.
In face recognition by Neural Network [7] based on
learning of the faces in the training phase, the learning set of
faces should be large enough in number to realize the variations
in real life situations. Neural Network solutions model the face
recognition problem very well, but they take significant
training time.
In Local Feature Analysis (LFA) [8] technique, dozens of
features from different regions of the face and the relative
location of these features are utilized and incorporated to
identify and verify the face image. Although LFA method
offers robustness in carrying out a match with local variations
on the facial image, the technique is not robust against global
facial attributes.
In [9] Hidden Markov Models (HMM) and Wavelets for
face recognition are used, and during the learning model the
best matching model, offers a query image. Success of Model
Matching methods depends mainly on building realistic
representative model.
Eigenface [2] method is one of the well-known face
detection and face recognition algorithms. In Eigen
presentation phase, every face in the database can be
represented as a vector of weights, and then PCA is used to
encode face images and capture face features. The face
recognition is done by locating the images in the database
whose weights are the closest in Euclidean distance to the
weights of the test images. Automatic learning and later face
recognition is practical within Eigenface scheme, and it has
advantages over other face recognition algorithms in making
the application practical for its simplicity, speed, learning
capacity, and insensitivity even to progressive change in the
face e images.
In our proposed methodology we use a multi algorithm
combining PCA with eigenface method using Haar-like
features and Robert cross edge detector applied on the same
facial data to decide the identity of a subject. In this paper, we
present a prototype system implementing our technique of face
recognition.
III. N
EW APPROACH
The main target of this research is to build a real-time
system that could be used in real-world environment, where
many technical systems require natural human-computer
interfaces using different kind of cameras installed into
everyday living and working environment.
As facial recognition is not possible if the face is not
detected and isolated from the background as the first step to be
processed for face recognition, our approach consists of two
stages. The first stage is to locate the faces in the image, using
the face detector to examine the image location which comes
from a live video. The second stage after analysis of facial
image is to declare a possible match face which is close enough
from the faces previously stored in the database for the new
face. The whole algorithmic architecture is shown in Figure 1.
A. Detection Process
Since face recognition algorithms are very sensitive to
different parameters such as lighting conditions, facial emotion
(angry, smiling, etc.), hair and makeup, it is extremely
important to pre-process detected faces before applying face
recognition.
Two face detection systems were trained: one with Haar-
like feature set of Viola and Jones [10] in which the basic Haar
features for the eyes and the face are added to Eigenface
method , where is the detection is very fast and one with the
Robert cross edge detector in which face edges are detected
and added to the database.
The Eigenface detection technique is based on PCA [1],
which considers as a best dimensionality reduction tool that
helps to reduce the data set to a smaller one. In this approach,
we preserve as much information as possible in the mean
square sense. Suppose that we have the set of faces , and these
faces is a subspace of the set of images, and it is K
dimensional, it is possible to find the best subspace using PCA
when you fit a hyper plane to the set of the faces.
PCA with computing Eigen faces is considered as first step
to process the image database, i.e. store the set of images with
labels.
After given a collection of n labeled training images, the
Eigenfaces modeling works as follows [11]:
Step 1: Each image Ii in the training set is transformed into a
column vector
.i
, and placed into the set.
},....,,,{ 321 MS
(1)
Step 2: Compute the mean image
, which equal to the
average image face Vector.
M
i
i
M
1
1
(2)
Where
M
is the number of the face images and
is the face
images vector.
Step 3: Find the difference
between the input image and the
mean image.
ii
(3)
Step 4: Find the Covariance Matrix by
M
i
i
M
C
1
T
i
1
(4)
Step 5: Compute the eigenvectors and eigenvalues of
C
.
Figure 1: The Framework for Multi face detection and recognition
Step 6: The
'
M
significant eigenvectors are chosen as those
with the largest corresponding eigenvalues.
Step 7: Project all the face images into these eigenvectors and
form the feature vectors of each face image.
Since our design target is to build a real time system, we
emphasize achieving very robust face detection rates.
According to [10], Haar-like features are simple and
inexpensive image features. A special representation of the
image called integral image makes feature extraction faster.
Using the integral image representation, it is possible to
compute the value of any rectangular sum of rectangular
features instead of pixels. Therefore, image scaling is not
necessary, and it is replaced by scale the rectangular features.
Since some parts of human face are more important than
other parts to successful face recognition, in addition to face
detection, we use Haar-like features to detect the eyes too, in
order to recognize the occluded face. Figure 2 shows the
detected eyes and detected face in the image, and display the
image with detected face outlined in green, and detected eyes
outlined in blue.
Once the Eignface process strats, Robert Cross edge
detection [12] is working simultaneously as a parallel
processing.
There are many methods to perform edge detection, one of
the main categories is gradient method, which detects the edges
by looking for the maximum and minimum values in the first
derivative of the image. In order to detect the edges in Robert
Cross technique, 2-D spatial gradient measurement on an
image is performed, where the regions of high spatial
frequency which greatly correspond to edges is highlighted.
Figure 2 shows the input image after edge detection using
Robert Cross, and the results in grayscale edge segmented
image is shown in Figure 3.
B. Face Recognition
Once the Eigenfaces are created in the first phase, we can
project any of a test face images into Eigenspace [13]. Then
Figure 2. Edge detected image using Robert Cross operator
Figure 3. Gray Scale image using Robert Cross operator
the classification can be made by a simple Euclidean distance
measure between the projected vector and each face image
feature vector. Euclidean distance formula can be found in
[14]. After applying Euclidean distance measure, the closest
face in Eigenspace is selected to recognize the test face image
as the match if the distance to that face is above athreshold,
otherwise, classify the face as an “unknown” face. Following
this methodology we can classify the image as known face
image, and unknown face image.
IV. EXPERIMENT AND RESULTS
The complete face recognition system is implemented in
Visual C# in MS-Windows platform. A standard web-camera
with 60/sec is used to capture video frames. The experimental
results were conducted on a Pentium 2.4GHz computer
system.
Figure 4 shows the interface used in our system. First the
faces in an input image must be located and registered to the
database, meaning enter the names of the people in the image
manually. Then, once we have detected a face, later if the
same face is detected again, the system will show how many
faces in the image and for whom those faces are, since face
recognition means figuring out whose face it is.
We conducted our experiments for people in different
places using live video. The video includes different
backgrounds. It is found that the system successfully detects
almost all the faces and the eyes in the images.
Figures 4, 5 respectively show recognized occluded face
with hand, and multiple recognized faces.
To evaluate the performance of our proposed face recognition
system, a number of self-captured face images were used in
the experiment. We measure the recognition rate for various
face expressions and poses. The recognition rate is the ratio of
successful attempts (cases where the best match is the correct
match) to the total attempts [5].
The results show that the correction rate is 92%. This
makes our multi-algorithmic technique very much suitable for
many applications.
Figure 4. Recognized occluded face with the hand
Figure 5. Multiple recognized faces
V. CONCLUSIONS AND FUTURE WORKS
Face recognition can be used in many applications such as
ATM, surveillance, video conferencing, etc. In this work, we
propose a new technique that relies on an effective combination
of multi algorithms. The approach is fast, easy, and it provides
a practical solution to the recognition problem. The system has
produced a success rate of 92% over a large set of database of
faces.
As a future work, we plan to extend our work by
implementing our technique using wireless multimedia sensor
network boards using IMB400 multimedia boards.
R
EFERENCES
[1] K. Lin, K. Lam, X. Xie and W. Siu,“ An Efficient Human Face Indexing
Scheme Using Eigenfaces,” in
Proceedings of the 2003 IEEE
International Conference on Neural Networks and Signal Processing,
pp. 920-923
, Dec. 2003.
[2] M. Turk and A. Pentland,“Eigenfaces for Recognition,”
Journal of
Cognitive Neuroscience, vol. 3, no. 1,pp. 71-86,
1991.
[3] F. Chichizola, L. Giusti, A. Giusti and M. Naiouf,”Face Recognition:
Reduced Image Eigenfaces Method,” in
Proceedings of the 47
th
International Symposium ELMAR, pp. 159-162,
Jun. 2005.
[4] R. Jafri and H. Arabnia, ”A Survey of Face Recognition Techniques,”
Journal of Information Processing Systems, vol. 5, no. 2,pp. 41-68, June
2009.
[5] S. Kar, S. Hiremath, D. Joshi, V. Chadda and A. Bajpai, “ A Multi-
Algorithmic Face Recognition System,” in
Proceedings of the
International Conference on Advanced Computing and Communications
(ADCOM), pp. 321-326,
Dec 2006.
[6] R. Brunelli and T. Poggio, “Face Recognition: Features versus
Templates,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 15, no. 10, pp. 1042-1052, Oct 1993.
[7] D. Bryliuk and V. Starovoitov, “Access Control by Face Recognition
using Neural Networks and Negative Examples,” in
Proceedings of 2
nd
International Conference on Artificial Intelligence, Crimea, Ukraine, pp.
428-436,
Sept 2002.
[8] P. Penev and J. Atick, “Local Feature Analysis : A general statistical
theory for object representation,”
Computational Neuroscience
Laboratory, The Rockefeller University, USA.
[9] M. Bieego, U. Castellani and V. Murino,”Using Hidden Markov Models
and Wavalets for face recognition,”
in Proceedings of the 12
th
IEEE
International Conference on Image Analysis and Processing (ICIAP’03),
pp. 52-56,
Sept 2003.
[10] P. Viola and M. Jones,” Robust Real-time Object Detection,”
in Second
International Workshop On Statistical And Computational Theories Of
Vision-Modeling, Learning, Computing, And Sampling, Vancouver,
Canada
, July 2001.
[11] A. Khan and L. Alizai,”Introduction to Face Detection Using
Eigenfaces,”
in Proceedings of the 2
nd
IEEE International Conference on
Emerging Technologies (ICET 2006), pp. 128-132
, Nov 2006.
[12] H. Lakshmi and S. Patilkulakarni,”Segmentation Algorithm for Multiple
Face Detection in Color Images with Skin Tone Regions using Color
Spaces and Edge Detection Techniques,”
International Journal of
Computer Theory and Engineering, vol. 2, no. 4,pp. 552-558,
Aug 2010.
[13] Md. Monwar, P. Polash, Md. Islam and S. Rezaei,”A real-Time Face
Recognition Approach From Video Sequence Using Skin Color Model
And Eigenface Method,”
in Candian Conference on Electrical and
Computer Engineering (CCECE 2006), pp. 2181-2185
, May 2006.
[14] D. Pissarenko,”Eigenface-based facial Recognition,” Dec 12002.
