Conference PaperPDF Available

Assistance for the Paralyzed Using Eye Blink Detection

Authors:
Conference Paper

Assistance for the Paralyzed Using Eye Blink Detection

Abstract

Paralysis is defined as the complete loss of muscle function in any part of the body. It occurs when there is a problem with the passage of messages between the muscles and the brain. Some paralyzed people cannot move even a single part of the body other than their eyes. Hence, the main aim of this paper is to design a real time interactive system that can assist the paralyzed to control appliances such as lights, fans etc. or by playing pre-recorded audio messages, through a predefined number of eye blinks. Image processing techniques have been implemented in order to detect the eye blinks. In our system, the face tracking is accomplished by using a set of trained Haar cascade classifier, and a template matching technique is employed to track the eye. Initially, the involuntary blinks of the paralyzed person are used to locate the patient's eyes by finding the number of connected components in a frame. Once the eyes are detected, an online template is created which is then used to track the patient's eye.
ASSISTANCE FOR THE PARALYZED USING EYE BLINK
DETECTION
Atish Udayashankar, Amit R Kowshik,
Chandramouli S
Dept. of Telecommunication Engg.
PES Institute of Technology
Bangalore, India
atishudayashankar@gmail.com,
amithamithu.kowshik@gmail.com,
chandru.m16@gmail.com
Abstract - Paralysis is defined as the complete loss of muscle
function in any part of the body. It occurs when there is a
problem with the passage of messages between the muscles and
the brain. Some paralyzed people cannot move even a single
part of the body other than their eyes. Hence, the main aim of
this paper is to design a real time interactive system that can
assist the paralyzed to control appliances such as lights, fans
etc. or by playing pre-recorded audio messages, through a
predefined number of eye blinks. Image processing techniques
have been implemented in order to detect the eye blinks. In our
system, the face tracking is accomplished by using a set of
trained Haar cascade classifier, and a template matching
technique is employed to track the eye. Initially, the
involuntary blinks of the paralyzed person are used to locate
the patient’s eyes by finding the number of connected
components in a frame. Once the eyes are detected, an online
template is created which is then used to track the patient’s
eye.
Keywords - Paralyzed patients, template matching, Haar
cascade classifier, eye blinks, face tracking
I. INTRODUCTION
Paralysis is the complete loss of muscle function for one
or more muscle groups. Paralysis can cause loss of feeling
or mobility in the affected areas. Paralysis can be localized,
or generalized, or it may follow a specific pattern. Most
paralyses caused by nervous system damage (i.e. spinal cord
injuries) are constant in nature; however, there are forms of
periodic paralysis, including sleep paralysis, which are
caused by other factors. Paralysis is most often caused by
damage to the nervous system, especially the spinal cord.
Major causes are stroke, trauma with nerve injury,
poliomyelitis, amyotrophic lateral sclerosis (ALS),
botulism, spina bifida, multiple sclerosis, and Guillain-Barré
syndrome. According to a survey by the Christopher and
Dana Reeve foundation, nearly 1 in every 50 people are
paralyzed [2].
Fully paralyzed patients require 24 hour support. But in
the present day and age, it is not possible for anyone to be
available at all times. So in those situations where the
patient is alone in a room, he/she could use this application
H S Prashanth
Dept. of Telecommunication Engg.
PES Institute of Technology
Bangalore, India
prashanth@pes.edu
to call for help if required or switch on/off a light, a fan or
any other appliance. Hence, our application will help the
patient to be self-sufficient to a certain extent.
There are innumerable applications which can be
derived from eye blink detection and these are not limited
for usage by only paralyzed people. An efficient, real time
blink detection algorithm can be used for almost any
purpose. It can be used for switching on/off appliances such
as a television or a microwave oven. It can also be used to
send an email or call someone on Skype. All this can be
accomplished with just a few eye blinks.
Innumerable techniques have been devised for face
tracking. CamShift face tracking algorithm, Haar face
tracking algorithm and face tracking using Eigenfaces are
some of these. There are some techniques devised for blink
detection as well. Some of them are software-oriented i.e.
using image processing, and a few are hardware-oriented
using sensors. Some of the image processing techniques
include blink detection using Gabor filters, contour
extraction, and eye blink detection using Median blur
filtering. The hardware-based approaches are primarily
using infrared and magnetic sensors [4, 7]. The advantage of
using sensors is that the entire system would be more
compact. However, risks involving the safety of the eye are
too high and outweigh the advantages of a sensor system. If
the frequency of the infrared light emitted by the sensor is
outside the visible spectrum, it could cause permanent
damage to the eyes.
The primary purpose of this paper is to propose a system
that can assist the paralyzed. It does so by tracking the
person’s eye and counting the blinks, and employs this
count to control various appliances and play pre-recorded
audio messages.
Although a number of techniques have been
implemented for eye blink detection, there is no application
that has been developed to actually put the blink detection to
practical use. The principal contribution of this paper is the
conceptualization of a system which will go a long way in
helping the paralyzed and disabled to achieve some level of
independence. Moreover, the algorithm provided by [1] has
been improved upon, by incorporating face tracking, so as to
reduce the effect caused by movements in the background.
2012 Fourth International Conference on Digital Home
978-0-7695-4899-9/12 $26.00 © 2012 IEEE
DOI 10.1109/ICDH.2012.9
104
II. BLINK DETECTION
There are a number of image processing algorithms for
eye blink detection. A brief overview of three of these
algorithms is provided.
A. Contour Extraction
In this technique, a set of 16 landmarks are created at
regular intervals to outline the contour of the eye. Eight
points are used to represent each eye. The distance between
the highest and lowest landmark is denoted by d1, and the
distance between the centroids of the two eyes is denoted by
d2. Now d1/d2 is computed, and assigned to a variable D.
Now the value of D is used to distinguish between open eye
and closed eye. Generally, a value of D equal to 0.158
implies open eye and a value equal to 0.016 implies closed
eye. These values have been experimentally derived by [3].
B. Gabor Filter
The Gabor filter is used to extract arcs of the eye. Here,
the eye region is first extracted and then the filter is applied
to obtain the arcs of the eye. Then connected component
labeling method is used to detect the top-bottom arcs. The
distance between the arcs is measured to determine the
blinking [5].
C. Median Blur Filtering
In this method, the image of the eye is first thresholded
and then a median blur filter is applied to it. The resultant
image obtained after applying the filtering shows a clear
difference between the open and the closed eye, and hence
helps in identifying eye blinks [6].
III. ALGORITHM USED IN THE PAPER
The algorithm used for eye blink detection is carried out
in five phases as shown in Figure 1. The first phase involves
face tracking in order to reduce the search window size. The
second phase involves identifying the location of the eye by
detecting involuntary eye blinks. The third phase involves
creation of an online template of the open eye. The fourth
phase is eye tracking and the final phase is the detection of
eye blinks. The algorithm is initialized automatically i.e. if
the location of the eye is lost, then the algorithm re-
initializes on its own to locate the eye. The system has a
timer programmed to run for a period of ten seconds during
which the patient will need to blink a desired number of
times. Finally after the timer runs out, the number of eye
blinks is used to perform the desired action, if it corresponds
to a pre-set value.
A. Face Tracking
Initially, the Haar face detector is loaded into the
program. The input image is then converted from
multichannel to single channel for faster processing. The
face region is then detected. A rectangular box is drawn to
outline the face region, and the coordinates of the rectangle
is provided to the next stage.
B. Initialization
In the first step, the difference between the current frame
and previous frame is found out. This difference image is
then thresholded. A threshold value of 5 was chosen. This
value was derived after some experimental trials.
Next, a structuring element is passed over the binary
image in order to remove the noise. The structuring element
used here is a 3x3 star shaped convolution kernel. This is
done in an open morphological operation. This operation
first performs erosion of the image followed by dilation
using the same structuring element for both the operations.
The primary effect of the opening operation is to preserve
the foreground regions that have a shape similar to the
structuring element or which can completely contain the
element. The other regions of foreground pixels are
eliminated.
The next step is to find the number of connected
components in the binary image. In order to do this, a
recursive labeling procedure is used. The lesser the number
of connected components detected indicates that the
algorithm is working as intended. In case, the number of
connected components detected is large in number, then the
binary image being used is discarded and the system re-
initializes. This step helps in preserving the efficiency of the
system. For an eye pair, the number of connected
components must be two.
If the image from the previous steps results in a small
number of connected components, the next step is to
determine which of these connected components constitute
an eye pair. This filtering to determine possible eye pairs is
done by applying experimentally derived heuristics. The
parameters used include: the vertical and horizontal distance
between the centroids of the components and the width and
height of each of the components. If any of these parameters
result in values which would normally not be associated
with an eye pair, then it can be concluded that the connected
components found do not constitute an eye pair. On the
connected components having met the conditions as
imposed by the heuristics, the system proceeds to open eye
template creation.
C. Open Eye Template Creation and Matching
For the creation of an open eye template, the larger of
the two connected components is chosen. This is done
primarily because the larger connected component contains
more brightness information and this helps in maintaining
the efficiency of the system. The open eye templates of
various sizes and orientations are shown in Figure 2.
105
Figure1. Block diagra
m
Figure 2. Open eye templates - Open Eye templates
o
for template matching
The next step is to locate the eye using
eye template. The method used to com
p
image of the eye and the
p
reviously create
normalized value of the square of the differ
e
b
rightness of the current frame and th
e
formula used is given below.
ܴሺݔǡݕሻ ൌ σሺܶݔҧǡݕെܫݔ൅ݔҧǡ
ҧǡ௬
σܶሺݔҧǡݕതሻ
Ǥσܫሺݔ ൅
ҧǡ௬
ҧǡ௬
m
of the system- Block diagram representing the model of the entire sys
o
f various sizes used
the created open
p
are the current
d template is the
e
nce between the
e
template. The
ݕ
൅ݕതሻሻ
ݔҧǡݕ൅ݕതሻ
In the above formula [8],
b
rightness of the video frame at t
h
brightness of the template at the p
o
computes the values of ܴݔǡݕ,i.
e
difference, for various images, the
b
global minimums. Here, the minim
u
performing experimental trials, ha
s
Hence, if the value of ܴሺݔǡݕሻexce
e
that it is not an eye and the re-initia
l
D. Blink Detection
The final step is the detection
o
to accomplish this, the concept of
This step is very similar to the step
located, except here the search wi
n
order to decide if there is an eye b
l
applied. The first one is that t
h
connected component. The seco
n
component must be located at the c
e
tem
ܶݔҧǡݕ represents the
h
e poin
t
ݔҧǡݕ, ܫሺݔҧǡݕതሻ the
o
int ݔҧǡݕ. As the function
e
. the normalized square
b
est matches are found as
u
m threshold value, after
s
been chosen to be 0.4.
e
ds 0.4, it is then realized
l
ization starts.
o
f the eye blin
k
. In order
motion analysis is used.
where the user’s eyes are
n
dow size is reduced. In
l
ink or not, two rules are
h
ere must be only one
n
d rule being that the
e
ntroid of the user’s eye.
106
IV. RESULTS
The system was primarily developed and tested on a
Windows 7 PC with an Intel Core i7-2670QM CPU 2.20
GHz processor and an 8 GB RAM. The video was captured
with a Creative camera built in the laptop. The video was
captured at 30 frames per second and was processed as
grayscale images. These images were of a size of 320 x 240
pixels. Many functions from the Intel OpenCV library were
utilized. The code was written in Microsoft Visual Studio
2008 and the GUI was developed using the Visual Studio
MFC application.
The user is required to assign the number of blinks
which correspond to the different appliances and other
functions. This has to be set every time the eye blink
detection system is switched on. The number of blinks
corresponding to any application can be changed at any
given time. For this purpose, a simple user interface was
created where the user can enter values corresponding to
different appliances or functions before starting the blink
detection system. The following figure is a screenshot of the
graphical user interface developed.
Figure 3. The graphical user interface - The graphical user interface used to
set the number of blinks for a particular action
Figure 4. Blink detection without face tracking
Figure 5. Blink detection with face tracking and movement in the
background
A number of trials were performed to find out the level
of efficiency of the algorithm under various circumstances.
The efficiency of the algorithm was computed, taking
several cases into consideration. One case was with the user
wearing glasses and another was with the user not wearing
glasses. Efficiency was also found based on the distance of
the user from the camera. Yet another basis which was
employed to obtain the efficiency was to remove the face
tracking algorithm from the main algorithm. As a result, the
efficiency of the system improved considerably. Our
observations are tabulated as follows.
Table I. Without face tracking algorithm
Total number of blinks Number of blinks detected
20 19
20 14
20 14
20 17
20 20
Efficiency: 84%
Table II. Without face tracking algorithm and user is wearing glasses
Total number of blinks Number of blinks detected
20 13
20 14
20 15
20 14
20 17
Efficiency: 73%
Table III. Without face tracking algorithm and user is far from the screen
(90cm)
Total number of blinks Number of blinks detected
20 18
20 17
20 15
20 15
20 17
Efficiency: 82%
107
Table IV. With face tracking algorithm and people moving in the
background
Total number of blinks Number of blinks detected
20 8
20 7
20 12
20 8
20 10
Efficiency: 45%
From the above tabulations, it can be concluded that the
algorithm works more efficiently when the user is not
wearing spectacles. Moreover, there is a very minute
difference in the efficiency of the program with a change in
distance of the eyes from the screen. It is also observed that
there is a remarkable fall in the efficiency of the algorithm
when face tracking is also implemented. However, if the
face tracking is not included, the efficiency of the algorithm
with people moving in the background is nearly zero.
V. CONCLUSION
Although blink detection systems exist for other
purposes, an implementation of a blink detection system
with the end use of controlling appliances has not been
previously accomplished. While the system is intended to
assist the paralyzed and physically challenged, it can
definitely be used by all types of individuals. The main
challenge involved in the implementation of the system is
the development of a real time robust blink detection
algorithm. Many algorithms have been developed to serve
the purpose, with some being more accurate than the others.
This paper presented a blink detection system based on
online template matching.
The first phase involved the blink detection phase; the
second phase involved the counting of blinks and
subsequent control of appliances through a micro controller.
By enabling the paralyzed to gain control of albeit a small
part of their lives, the system can offer some level of
independence to them. The helpers who are assigned the
task of tending to paralyzed persons through the day can
then be afforded a break. The system needs moderate
processing power, making it suitable for practical use. For
continuous video input, laptops with built in webcams or
USB cameras will suffice.
The system is limited by the efficiency of the blink
detection algorithm and efficiency falls further under
limited lighting conditions. Since the initialization phase of
the algorithm is based on differencing between consecutive
frames, background movement in the frame may lead to
inaccurate operation. Typically, background movement
causes non eye pairs to be detected as eye pairs. This is
overcome to some extent by limiting the search region to the
face of an individual, by implementing a face tracking
algorithm prior to blink detection. However, this in turn can
lead to reduced efficiency in blink detection. By giving an
option to the user to choose between the system with and
without face tracking, a level of flexibility can be reached.
The application of the blink detection system is not
limited to the control of appliances but can also be used for
a variety of other functions. Playback of audio distress
messages over an intercom system is one of the other
applications of the system. Future applications of the system
may include playback of video or audio files by eye blinks
and making a VOIP call to play a distress message. In the
future, the system can be implemented on a Digital Signal
Processor, making it a truly embedded system which could
be used as a standalone device without the need for a laptop
or desktop PC.
VI. REFERENCES
[1] Michael Chau and Margrit Betke,2005. Real Time Eye Tracking and
Blink Detection with USB Cameras. Boston University Computer
Science Technical Report No. 2005-12. Boston, USA
[2] Christopher and Dana Reeve
Foundationhttp://www.christopherreeve.org/site/c.mtKZKgMWKwG
/b.4453145/k.564C/Paralysis_and_Its_Impact.htm
[3] Liting Wang , Xiaoqing Ding , Chi Fang , Changsong Liu , Kongqiao
Wang ,2009. Eye Blink Detection Based on Eye Contour Extraction.
Proceedings of SPIE-IS&T Electronic Imaging. San Jose, CA, USA.
[4] M. Takagi, K. Mohri, M. Katoh and S. Yoshino,1994.Magnet-
Displacement Sensor Using Magneto-Inductive Elements for Sensing
Eyelid Movement. IEEE Translation Journal On Magnetics In Japan,
Vol. 9,No.2, pp 78-83.
[5] Kohei Aai and Ronny Mardiyanto, 2011. Comparative Study on
Blink Detection and Gaze Estimation Methods for HCI, in Particular,
Gabor Filter Utilized Blink Detection Method. Proceedings of Eighth
International Conference on Information Technology: New
Generations. Las Vegas, USA, pp. 441-446.
[6] Chinnawat Devahasdin Na Ayudhya, Thitiwan Srinark, A Method for
Real-Time Eye Blink Detection and Its Application
[7] Cihan Topal, Ömer Nezih Gerek and Atakan Doan,2008. A Head-
Mounted Sensor-Based Eye Tracking Device: Eye Touch System.
Proeedings of the 2008 Symposium on Eye tracking research &
applications. Savannah,GA,USA, pp. 87-90.
[8] OpenCV documentation
http://opencv.willowgarage.com/documentation/
[9] Lae-Kyoung Lee, Su-Yong An , Se-Young Oh, Aug 2011. Efficient
Face Detection and Tracking with extended camshift and haar-like
features. International Conference on Mechatronics and Automation
(ICMA), 2011. Pohang, South Korea, pp. 507-513.
[10] Duan-Sheng Chen , Zheng-Kai Liu, Aug 2011. Generalized Haar-like
features for Fast Face Detection. Conference on Machine Learning
and Cybernetics, 2007. Hong Kong, pp. 2131-2135.
[11] Bhaskar, T.N., Foo Tun Keat , Ranganath, S., Venkatesh, Y.V., Oct
2003. Blink Detection and Eye Tracking for Eye Localization.
Conference on Convergent Technologies for Asia-Pacific Region,
2003,Singapore, pp. 821-824 Vol.2.
108
... Zangana worked face detection algorithm by combining three color model algorithms [15]. Udayashankar and others detected blinks using template matching and assigned different blinks to different appliances [2], but has drawbacks such as the detection efficiency falls when there is background movement and under low light conditions. The proposed paper intends on improving in such areas. ...
Article
Full-text available
Mobility is defined as the ability to move freely and easily. The ability to move independently is vital for all life functions and plays a major role in maintaining mental health. Paralysis is a loss of muscle function in a part or full body of the patient. In every patient suffering from immobility, eye is commonly found to be the only organ in working condition. Hence, the proposed paper presents an automation system to detect and count the number of blinks of the patient using image processing to detect blinks to trigger different appliances of the house along with a health monitor that alerts the caretaker or neighbors in times of emergencies. Blink detection is a concept that has undergone multiple algorithms and approaches, the previous ones made use of artificial neural network (template matching) which involved subjective training of every patient's eyes. Image processing approach sets the proposed paper apart from the rest. Although this paper aimed at developing an automation system for the aid of paralysis patients, practically it can be used for the aid of any patient suffering from movement disability.
... Previous eye-blink detection systems evaluate ocular status as open or closed [40] or eye-closure pathways [41]. For each image, other approaches employ template matching, which lists open and/or closed eye templates and computes a standard cross-reference coefficient for each eye region [42].In order to begin with a blink detection, the eye must first be detected. This is done by employing a pretrained model that offers us sixty-eight face points.Then map the landmarks on the face found.The process is in three sections implemented.Load the pretrained model's contents into an object first.Next, use this object to map the signs on the face seen in the frame and then, as shown in the figure (6), to extract the Cartesian coordinates of these landmarks . ...
Article
Full-text available
Abstract: 3D mask face spoofing attack is an important challenge in recent years and draws further study. Because of the number of deficiencies and small differences in the database, however, a few methods can be proposed to aim at it meanwhile, most developed databases are focused on countering various types of threats and neglect environmental developments in implementations in the real world. The 3D mask against spoofing is used in this paper to simulate the real-world scenario with other options. The database used in the proposed method includes 10 different subject masks (7 subject 3D latex masks and 2 subjects for 2D paper masks and 1 for half mask from below the eye is using to testing the result). Therefore, the total size is 440 videos 400 is fake videos, and 40 is a real video. The directions for future study are shown in the benchmarking experiments. We intend to release the database platform to evaluate various methods this system has been used for a deep Convolution neural network. The result is robust for an eye-blink recognition technique. There are three basic steps in the proposed system: Firstly, video pre-processing, facial recognition, and finally, the output step whether the video is true or falsified. The method used is stronger than most techniques. The suggested approach in this study was used for the MLFP dataset and was very reliable and accurate as a result of the whole experiment and the accuracy obtained is (99.88).
... Zangana worked face detection algorithm by combining three color model algorithms [15]. Udayashankar and others detected blinks using template matching and assigned different blinks to different appliances [2], but has drawbacks such as the detection efficiency falls when there is background movement and under low light conditions. The proposed paper intends on improving in such areas. ...
Article
Full-text available
Mobility is defined as the ability to move freely and easily. The ability to move independently is vital for all life functions and plays a major role in maintaining mental health. Paralysis is a loss of muscle function in a part or full body of the patient. In every patient suffering from immobility, eye is commonly found to be the only organ in working condition. Hence, the proposed paper presents an automation system to detect and count the number of blinks of the patient using image processing to detect blinks to trigger different appliances of the house that alerts the caretaker or neighbors in times of emergencies. Blink detection is a concept that has undergone multiple algorithms and approaches, the previous ones made use of artificial neural network (template matching) which involved subjective training of every patient's eyes. Image processing approach sets the proposed paper apart from the rest. Although this paper aimed at developing an automation system for the aid of paralysis patients, practically it can be used for the aid of any pat ient suffering from movement disability.
Article
Full-text available
The patient who are bedridden due of weakening of muscles of hands, feet or had lost voice, finds difficult to communicate their needs. These conditions occur when there is an issue in the passage of messages between the brain and the muscles. Some of them are affected with quadriplegia cannot move their limbs and some could move only their eyes. Therefore, the aim of this study is to provide an interactive system which would assist them to communicate with caretakers. Communication was done using pre- defined messages and an alert buzzer to grab attention, for corresponding delay in eye blinks. Once the eye blinks, the eye sensor senses it and compares the duration with the defined delays. If the condition is true then corresponding defined message appears on the display, if the condition is false, then it checks for next condition and so on. The user needs to be trained to communicate through this device. The time delays set for each message can be adjusted as per user’s need. More messages can be displayed by changing the time of closure of the eye. The system is employed to control, manage and communicate with the other people through eye blink action.
Article
The constant demand to improve daily living standards for paralysis patients serves as a motivation to develop newer technology. The taskonce performed by big traditional computers is now solved with smaller smart devices. Paralysis isdefined as the complete loss of muscle function in any part of the body. It occurs when there is a problem with the passage of messages between the muscles and the brain. The main objective is todesign a real time interactive system that can assistthe paralysis patients to control appliances such aslights, fans etc. The eye blink sensor is able to detect an intentional blink from a normal blink, which is useful for the paralysis patients especially Tetraplegic patients to regulate their home deviceseasily without any help.
Article
The status of mental health and mood of human beings are well comprehensible by careful observation of movements of different body parts. Eye being the most prominent body part, analysis of different eye parameters such as blink, gaze, opening and closing rate provides important clues on mood status as well as mental health conditions. The present work can be viewed from a statistical and machine learning perspective that utilizes eye blink information to study the mental health status of a person. By using appropriate image processing techniques eye blinks of different subjects were collected through an experimental setup. The setup contained a recording environment where each participant was required to watch two videos of opposite emotions, i.e., joy and sad during different time settings. From the recorded videos of each participant, eye blinks were extracted and investigated. On analyzing the blink rates thoroughly, using statistical and machine learning means we observed; 1) an increase in number of eye blinks when the mood of a participant swings from sad to joy and 2) a significantly smaller number of blinks in depressed participants than the normal participants while in sad mood.
Conference Paper
Full-text available
Blink detection is used for a variety of applications such as Human-Computer Interaction, wearable computing, etc. Blink detection method with Gabor filter is proposed to realize high accurate blink detection. The blink detection accuracy is evaluated by several people and compare to the other existing conventional methods. Through the comparison, it is found that the proposed blink detection method with Gabor filter is superior to the other methods.
Conference Paper
Full-text available
Eye blink detection is one of the important problems in computer vision. It has many applications such as face live detection and driver fatigue analysis. The existing methods towards eye blink detection can be roughly divided into two categories: contour template based and appearance based methods. The former one usually can extract eye contours accurately. However, different templates should be involved for the closed and open eyes separately. These methods are also sensitive to illumination changes. In the appearance based methods, image patches of open-eyes and closed-eyes are collected as positive and negative samples to learn a classifier, but eye contours can not be accurately extracted. To overcome drawbacks of the existing methods, this paper proposes an effective eye blink detection method based on an improved eye contour extraction technique. In our method, eye contour model is represented by 16 landmarks therefore it can describe both open and closed eyes. Each landmark is accurately recognized by fast classifier which is trained from the appearance around this landmark. Experiments have been conducted on YALE and another large data set consisting of frontal face images to extract the eye contour. The experimental results show that the proposed method is capable of affording accurate eye location and robust in closed eye condition. It also performs well in the case of illumination variants. The average time cost of our method is about 140ms on Pentium IV 2.8GHz PC 1G RAM, which satisfies the real-time requirement for face video sequences. This method is also applied in a face live detection system and the results are promising.
Article
This paper presents a new approach to solve the problem of real-time robust face detection and tracking in complex environment. We propose a two level approach to detect faces and tracking. The lower level of the approach implements the extraction of face candidates using the combination of skin color model and haar-like features based adaboost learning algorithm. With this method, multiple-view faces are able to be detected in real-time with high recognition accuracy. The higher level of approach implements the robust face tracking with extended CAMSHIFT (Continuous Adaptive Mean SHIFT). The experimental results show that the proposed algorithm is robust and efficient to detect and track the face-of-interest in the cases of clutter background and the occurrence of occlusion.
Article
Various human behaviors can be indicated by eye blink patterns. In this paper, we present a method based on image processing techniques for detecting human eye blinks and generating inter-eye-blink intervals. We applied Haar Cascade Classifier and Camshift algorithms for face tracking and consequently getting facial axis information. In addition, we applied an Adaptive Haar Cascade Classifier from a cascade of boosted classifiers based on Haar-like features using the relationship between the eyes and the facial axis for positioning the eyes. We proposed a new algorithm and a new measurement for eye blinking detection called "the eyelid's state detecting (ESD) value." The ESD value can then be used for examining the open and close states of eyelids. Our algorithm provides a 99.6% overall accuracy detection for eye blink detection. We generated inter-eye-blink interval graphs by differencing between two consecutive eye blink states. The graphs show that the common blinks of human presents short and long durations alternatively.
Article
A human-computer interface (HCI) system designed for use by people with severe disabilities is presented. People that are severely paralyzed or afflicted with diseases such as ALS (Lou Gehrig's disease) or multiple sclerosis are un-able to move or control any parts of their bodies except for their eyes. The system presented here detects the user's eye blinks and analyzes the pattern and duration of the blinks, using them to provide input to the computer in the form of a mouse click. After the automatic initialization of the sys-tem occurs from the processing of the user's involuntary eye blinks in the first few seconds of use, the eye is tracked in real time using correlation with an online template. If the user's depth changes significantly or rapid head movement occurs, the system is automatically reinitialized. There are no lighting requirements nor offline templates needed for the proper functioning of the system. The system works with inexpensive USB cameras and runs at a frame rate of 30 frames per second. Extensive experiments were conducted to determine both the system's accuracy in classifying vol-untary and involuntary blinks, as well as the system's fitness in varying environment conditions, such as alternative cam-era placements and different lighting conditions. These ex-periments on eight test subjects yielded an overall detection accuracy of 95.3%.
Conference Paper
In this study, a new eye tracking system, namely Eye Touch, is introduced. Eye Touch is based on an eyeglasses-like apparatus on which IrDA sensitive sensors and IrDA light sources are mounted. Using inexpensive sensors and light sources instead of a camera leads to lower system cost and need for the computation power. A prototype of the proposed system is developed and tested to show its capabilities. Based on the test results obtained, Eye Touch is proved to be a promising human-computer interface system.
Conference Paper
In the framework of Viola-Jones' fast object detection, Haar like features are extracted from gray level image. This paper proposes a new concept of gray-like image from which generalized Haar like features can also be exacted, so as to make use of other forms of images in addition to gray level image in Haar+Adaboost schema. As an application of the gray-like images, the generalized Haar-like features are constructed for fast face detection. Experimental results show that the boosted face detector using the generalized Haar-like features outperforms significantly the original using the basic Haar-like features.
Conference Paper
A method of using frame differencing coupled with optical flow computation for eye blink detection is proposed. Frame differencing allows quick determination of possible motion regions. If they are detected, optical flow is computed within these regions. The direction and magnitude of the flow field are then used to determine whether a blink has occurred. The eyes are then tracked using the Kanade Lucas Tomasi (KLT) tracker. We obtained a success rate of 97.0% in blink detection using the proposed method, and localised the eyes automatically at an average rate of 22 frames per second.
We previously constructed an accurate displacement sensor for detection of eyelid movement, using an amorphous magnetic core multivibrator. Improvements in eyelid motion sensors are necessary to achieve independence of the magnetization direction of the cores as well as a broadened visual field. We constructed a second displacement sensor for detection of eyelid movement, this time using amorphous wire magneto-inductive (MI) elements. The elements were formed from tension-annealed zero-magnetostriction amorphous wires. The MI effect and the symmetrical MI characteristics of the elements were used to form thin sensor heads without coils. Inductances change independently of the magnet magnetization directions.
k.564C/Paralysis-and-Its-Impact
  • Dana Reeve Foundationhttp