ArticlePDF Available

Enhancing eye-movement-based biometric identification method by using voting classifiers

Authors:

Abstract and Figures

Eye movements contain a lot of information about human being. The way the eyes are moving is very complicated and eye movement patterns has been subject of studies for over 100 years. However, surprisingly, eye movement based identification is a quite new idea presented for the first time during the Biometrics'2003 Conference in London [17]. The method has several significant advantages: compiles behavioral and physiological properties of human body, it is difficult to forge and it is affordable-with a number of ready-to-use eye registering devices (so called eye trackers). The paper introduces the methodology and presents results of the first eye movement based authorization tests.
Content may be subject to copyright.
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
Enhancing eye movement based biometric identification method by using
voting classifiers
Paweł Kasprowski
1
, Józef Ober
1,2
1
Institute of Computer Science, Silesian University of Technology, 44-100 Gliwice, Poland
2
Institute of Theoretical and Applied Informatics, Polish Academy of Science, Gliwice, Poland
kasprowski@polsl.pl
Abstract. Eye movements contain a lot of information about human being. The way the eyes are moving is
very complicated and eye movement patterns has been subject of studies for over 100 years. However,
surprisingly, eye movement based identification is a quite new idea presented for the first time during the
Biometrics’2003 Conference in London [1]. The method has several significant advantages: compiles behavioral
and physiological properties of human body, it is difficult to forge and it is affordable – with a number of ready-
to-use eye registering devices (so called eye trackers). The paper introduces the methodology and presents test
results.
Introduction
Using eyes to perform biometric human identification has a long tradition including well-
established iris pattern recognition algorithms [2] and retina scanning. However, there are
only few researches concerning identification based on eye movement characteristic [3][4]. It
is a bit surprising because that method has several important advantages.
Firstly, it compiles physiological (muscles) and behavioral (brain) aspects. The most
popular biometric methods like fingerprint verification or iris recognition are based mostly on
physiological properties of human body. Therefore, what is needed for proper identification,
is only a “body” of a person who is to be identified. It makes possible to identify an
unconscious or - in some methods - even a dead person.
Moreover, physiological properties may be forged. Preparing models of a finger or even
retina (using special holograms) is technically possible. As eye movement based identification
uses information which is produced mostly by brain (so far impossible to be imitated), forging
this kind of information seems to be much more difficult.
Although it has not been studied in that paper, it seems possible to perform a covert
identification, i.e. identification of a person unaware of that process (for instance using hidden
cameras).
Last but not least, there are many easy to use eye-tracking devices nowadays, so
performing identification by means of that technique is not very expensive. For instance a
very fast and accurate OBER2 [5] eye tracking system was used in the present work. It
measures eye movements with a very high precision using infrared reflection and the
production costs are comparable to fingerprint scanners.
Experiment
To prove that eye movements may be used for human identification an experiment had to
be provided. The experiment was divided into two stages:
1) Gathering samples of human eye movements from different persons.
2) Processing of samples obtained in the previous step to extract individual features.
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
The process of data gathering consists of series of tests on different subjects (persons).
Each test is a registration of eye movements of the subject for the specified period of time
with the OBER2 system. The result of the single test is a sample, which is then used in the
second stage of the experiment.
There are three possibilities how to perform the single test:
- Registering eye movement only without information about an observed image.
- Registering both eye movements and the observed scene.
- Generating scene and registering eye movements as the answer for it.
The first solution is very simple to conduct even without any cooperation from the person
being identified. Eye movements may be measured during normal activity of that person
without any information about the observed image. However as eye movements are strongly
correlated with the scene, analysis of the data may be difficult.
The second solution gives a lot more data to analyze, yet it also has several serious
drawbacks. First of all, the testing system is more complicated. An additional camera recorder
is needed, which registers the scene the examined person is looking at. Furthermore, we need
to implement special algorithms to synchronize visual data with eye movement signal. A lot
more capacity is also needed for data storing.
In the third solution the testing system consists of the OBER2 eye tracker and PC computer
for both data storing and controlling the monitor, which produces a visual signal (Fig. 1). The
OBER2 system registers answers for that signal produced by the subject’s eyes. However, we
should be aware of the fact that the monitor screen is only a part of the image that eyes see, so
not the whole input is measured. Furthermore, the input may consist of non-visual signals.
Sudden loud sounds may, for instance, cause rapid eye movements.
Fig. 1 Schema of the system generating stimulation on a computer display and registering eye
movements as the answer to that stimulation.
As the last methodology gives influence on ‘input’ of the examined subject it seems to be
the most interesting from researcher point of view. Therefore all tests described in this work
were performed using a stimulation displayed on the monitor with the system architecture
presented on Fig. 1.
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
The stimulation, which has been chosen, was a ‘jumping point’ stimulation with the same
points order for every experiment. There were nine different point placements defined on the
screen, one in the middle and eight on the edges, creating 3 x 3 matrix. The point flashed in
one placement in a given moment. The stimulation began and ended with a point in the
middle of the screen. During the stimulation, point’s placement changed in specified intervals.
The main problem in developing stimulation is to make it short and informative. Those
properties are as if on two opposite poles, so a ‘golden mean’ must be found. It may be
assumed that gathering one sample could not last longer than 10 seconds. Longer stimulations
would be impractical when considering usage in real world. To be informative, experiment
should consist of as many point position changes as possible. However, moving a point too
quickly makes it impossible for eyes to follow it. Experiments and literature [6] confirmed
that the reaction time for change of stimulation is about 100-200 ms. After that time eyes start
a saccade, which moves fovea to the new gaze point. The saccade is very fast and lasts not
longer than 10-20 ms. After a saccade, the brain analyses a new position of the eyes and, if
necessary, tries to correct it. So very often about 50 ms after the first saccade the next saccade
happens. It can be called it a ‘calibration’ saccade. Therefore to register whole reaction for
point change it was necessary to provide an interval between point locations change as more
than 300 ms.
The stimulation, which has been developed and used during all tests, consists of eleven
point position changes giving twelve consecutive point positions. First point appears in the
middle of the screen and the person should look at it with eyes positioned direct ahead. After
1600 ms the point in the middle disappears and for 20 ms a screen is blank. In that time eyes
are in instable state waiting for another point of interest. That moment is uncomfortable for
eyes because there is no point to look at. Then the point appears in the upper right corner. The
flashing point on the blank screen attracts eyes attention even without person’s will. The
‘jumps’ of the point continue until the last point position in the middle of the screen is
reached.
a. 1600 ms
e. 550 ms
i. 550 ms
b. 550 ms
f. 550 ms
j. 550 ms
c. 550 ms
g. 550 ms
k. 550 ms
d. 550 ms
h. 550 ms
l. 1100 ms
Fig. 2 Visual description of stimulation steps (a-l).
Testing
The data obtained in the sample gathering phase has to be transformed to create a vector of
attributes, which will be used for identification process. Each feature should give some
information about a person who was the subject of the experiment. That information may be
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
understandable for instance “his dominant eye is left” or “his eyes are flickering with
frequency 10 Hz”, but the meaning of the feature may be hidden also, giving only the value.
The main problem is how to extract a set of features that have values for different samples of
the same person (inner-class samples) as similar as possible and that have values for different
person’s samples as different as possible.
As it was mentioned earlier, identification based on eye movement analysis is a brand new
technique. The main disadvantage of that is that one cannot use already published algorithms
and just try to improve it with one’s own methods. Therefore, we could only try to use
methods, which have been successfully used with similar problems like:
- Methods used for analyzing eye movement data.
- General methods used in signal processing and classification.
First stage was preprocessing of samples with an normalization algorithm. The
preprocessed samples were included into a dataset containing over 1000 samples obtained
from 47 persons. The next stage was calculation of different samples characteristics like
Fourier transform or average velocity directions and storing it in separate datasets.
Each dataset consisted of vectors of attributes calculated from samples. Most of these
attributes were completely useless for classification. Such irrelevant attributes are not only
making classification more complex and time consuming task but also can disturb it and
increase identification errors. Therefore the next task was extraction of relevant attributes
from the attributes vectors. There was Principal Component Analysis (PCA) technique used
for linear conversion of attributes to create a new attributes explaining most of the dataset
variance.
The data were tested only with authorization tests – that is all samples were always divided
into positive (belonging to the specified person) and negative (belonging to different persons).
The testing process randomly created a training set taking 20 positive and 80 negative
samples from the whole dataset. After preparing PCA for the training set, the classification
models for different numbers of the most significant attributes were created using SVM [7]
and C45 Decision Tree [8] techniques. All samples not chosen to the training set were then
used for data testing. There were separate tests for each conversion, number of the most
significant PCA attributes and classification algorithm performed. Therefore there were 75
results created for each testing phase. These results were than used to create the final result in
a simple voting classifiers algorithm.
Results
There were two kinds of experiments performed. In the first, the test was performed only
once giving positive or negative result. The average, best and worst error rates are presented
in the table below.
Table 1. Average error rates for “one trial” experiment
FAR FRR
worst 3,19 26,94
average 2,31 14,53
best 0,56 3,93
In the second experiment two subsequent trials made on the same user one by one were
used as the one test. If both trials failed the user was rejected, if one or both succeeded the
user was accepted. Such scenario obviously increased FAR error because it was easer for not
honest person to mislead system but FRR error was significantly lower.
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
Table 2. Average error rates for “one of two trials” experiment
FAR FRR
worst 7,13 21,45
average 4,73 10,49
best 1,82 3,44
Summary
The idea presented here seems to be interesting but the results obtained are far from
perfection. However the results are showing that eye movements are indeed giving some
information about human’s identification and they are encouraging for future effort.
Literature
[1] Kasprowski, P., Ober, J.: Eye movement tracking for human identification. 6th World
Conference BIOMETRICS’2003, London (2003)
[2] Daugman, J.G.: High Confidence Visual Recognition of Persons by a Test of Statistical
Independence, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
15, no. 11 (1993)
[3] Kasprowski P., Ober J.: With the flick of an eye. Biometrics Technology Today ISSN
0969-4765, Volume 12, Issue 3, Elsevier Science (March 2004)
[4] Kasprowski, P., Ober, J.: Eye Movement in Biometrics. Proceedings of Biometric
Authentication Workshop, European Conference on Computer Vision in Prague 2004,
LNCS 3087, Springer-Verlag, Berlin (2004)
[5] Ober, J., Hajda, J., Loska, J., Jamnicki, M.: Application of Eye Movement Measuring
System OBER2 to Medicine and Technology. Proceedings of SPIE, Infrared
Technology and applications, Orlando, USA, 3061(1) (1997)
[6] Hung, G. K.: Models of Oculomotor Control, World Scientific Publishing Co. (2001)
[7] Vapnik, V.: Statistical Learning Theory. John Wiley and Sons, Inc. New York (1998)
[8] Quinlan, J. R.: C4.5: Programs for Machine Learning. San Mateo: Morgan Kaufmann
(1993)
... Example of Bio-Ran dataset. The point of light is visible at a different location after a set time interval[68]. Bio-Tex Stimulus. ...
Thesis
Full-text available
This thesis works with the eye-tracking data in the project 'Schau mir in die Augen'. Abstract The study includes working with eye movements for user identification. The main objective of the thesis is to improve the accuracy by modifying a pre-existing pipeline to recognize and take advantage of outliers. For this goal, the reasons for outliers and their particularities are investigated. Various methods are used and compared for detecting the outliers. It is found that removing the outliers from the pipeline decreases the accuracy. Instead, it is studied to include them in the pipeline. For this, a new classifier is added into the pipeline along with fixations and saccades, which deal with outliers. All the experiments are performed on the Bio-Tex dataset and lead to an improvement of accuracy. The final pipeline is tried on the Bio-Ran dataset, but there are no conclusive results, and more experiments need to be performed. Two classifiers are used in the thesis, majorly is Random Forest, and the other is Radial Basis Function Network. The Bio-Tex shows an improvement in accuracy by considering the outliers for the two classifiers. For Bio-Ran, there are no conclusive results obtained. Outliers play a vital role in eye movements and show the potential to improve the accuracy in the user identification field with more research.
... There is a substantial body of research on biometric identification using macromovements of the eye. Most work uses the same stimulus for training and testing-such as a static cross [4], a jumping point [22,23,44,8,47], a face image [43,17,5], or various other kinds of images [10]. Using the same known stimulus for training and testing opens the methods to replay attacks. ...
Chapter
Full-text available
We study involuntary micro-movements of the eye for biometric identification. While prior studies extract lower-frequency macro-movements from the output of video-based eye-tracking systems and engineer explicit features of these macro-movements, we develop a deep convolutional architecture that processes the raw eye-tracking signal. Compared to prior work, the network attains a lower error rate by one order of magnitude and is faster by two orders of magnitude: it identifies users accurately within seconds.
... Here, it is possible to unambiguously identify the person by means of the eye behavior. e rst approaches required a moving point stimulus which was followed by the user [30][31][32] or static images [45]. In 2010, the rst approach that was able to distinguish users with a task independent approach was presented [2]. ...
Preprint
Full-text available
In this paper, we present an approach based on reinforcement learning for eye tracking data manipulation. It is based on two opposing agents, where one tries to classify the data correctly and the second agent looks for patterns in the data, which get manipulated to hide specific information. We show that our approach is successfully applicable to preserve the privacy of a subject. In addition, our approach allows to evaluate the importance of temporal, as well as spatial, information of eye tracking data for specific classification goals. In general, this approach can also be used for stimuli manipulation, making it interesting for gaze guidance. For this purpose, this work provides the theoretical basis, which is why we have also integrated a section on how to apply this method for gaze guidance.
... Eye movement biometrics has emerged as a promising approach to user authentication [Kasprowski and Ober 2003]. While first works required a point stimulus that users were instructed to follow with their eyes [Kasprowski 2004;Kasprowski and Ober 2005], later ones explored static points [Bednarik et al. 2005] [Zhang et al. 2018]. ...
Conference Paper
With eye tracking being increasingly integrated into virtual and augmented reality (VR/AR) head-mounted displays, preserving users' privacy is an ever more important, yet under-explored, topic in the eye tracking community. We report a large-scale online survey (N=124) on privacy aspects of eye tracking that provides the first comprehensive account of with whom, for which services, and to what extent users are willing to share their gaze data. Using these insights, we design a privacy-aware VR interface that uses differential privacy, which we evaluate on a new 20-participant dataset for two privacy sensitive tasks: We show that our method can prevent user re-identification and protect gender information while maintaining high performance for gaze-based document type classification. Our results highlight the privacy challenges particular to gaze data and demonstrate that differential privacy is a potential means to address them. Thus, this paper lays important foundations for future research on privacy-aware gaze interfaces.
... Eye movement biometrics has emerged as a promising approach to user authentication [21]. While first works still required a jumping point stimulus that users were instructed to follow with their eyes [20,22], later ones explored static points [1] or images [32]. Kinnunen et al. presented the first method for "task-independent" person authentication using eye movements [24]. ...
Preprint
With eye tracking being increasingly integrated into virtual and augmented reality (VR/AR) head-mounted displays, preserving users' privacy is an ever more important, yet under-explored, topic in the eye tracking community. We report a large-scale online survey (N=124) on privacy aspects of eye tracking that provides the first comprehensive account of with whom, for which services, and to which extent users are willing to share their gaze data. Using these insights, we design a privacy-aware VR interface that uses differential privacy, which we evaluate on a new 20-participant dataset for two privacy sensitive tasks: We show that our method can prevent user re-identification and protect gender information while maintaining high performance for gaze-based document type classification. Our results highlight the privacy challenges particular to gaze data and demonstrate that differential privacy is a potential means to address them. Thus, this paper lays important foundations for future research on privacy-aware gaze interfaces.
... The eye movement characteristics of eye-movement restricted stimulus mainly represent the properties of the oculomotor plant, because the cognitive strategies from the brain are restricted by the stimulus. In a common free-will visual task, the participant's reaction time will decrease upon increasing the times for which the participant is watching the same stimulus, which is known as the "learning effect" [36]. On the other hand, the recognition time should be as short as possible for a practical biometric identification method. ...
Article
Full-text available
Biometric recognition technology based on eye-movement dynamics has been in development for more than ten years. Different visual tasks, feature extraction and feature recognition methods are proposed to improve the performance of eye movement biometric system. However, the correct identification and verification rates, especially in long-term experiments, as well as the effects of visual tasks and eye trackers' temporal and spatial resolution are still the foremost considerations in eye movement biometrics. With a focus on these issues, we proposed a new visual searching task for eye movement data collection and a new class of eye movement features for biometric recognition. In order to demonstrate the improvement of this visual searching task being used in eye movement biometrics, three other eye movement feature extraction methods were also tested on our eye movement datasets. Compared with the original results, all three methods yielded better results as expected. In addition, the biometric performance of these four feature extraction methods was also compared using the equal error rate (EER) and Rank-1 identification rate (Rank-1 IR), and the texture features introduced in this paper were ultimately shown to offer some advantages with regard to long-term stability and robustness over time and spatial precision. Finally, the results of different combinations of these methods with a score-level fusion method indicated that multi-biometric methods perform better in most cases.
Chapter
In this paper, we present an approach based on reinforcement learning for eye tracking data manipulation. It is based on two opposing agents, where one tries to classify the data correctly and the second agent looks for patterns in the data, which get manipulated to hide specific information. We show that our approach is successfully applicable to preserve the privacy of a subject. For this purpose, we evaluate our approach iterative to showcase the behavior of the reinforcement learning based approach. In addition, we evaluate the importance of temporal, as well as spatial, information of eye tracking data for specific classification goals. In the last part of our evaluation we apply the procedure to further public data sets without re-training the autoencoder nor the data manipulator. The results show that the learned manipulation is generalized and applicable to other data too.
Preprint
Person identification based on eye movements is getting more and more attention, as it is anti-spoofing resistant and can be useful for continuous authentication. Therefore, it is noteworthy for researchers to know who and what is relevant in the field, including authors, journals, conferences, and institutions. This paper presents a comprehensive quantitative overview of the field of eye movement biometrics using a bibliometric approach. All data and analyses are based on documents written in English published between 2004 and 2019. Scopus was used to perform information retrieval. This research focused on temporal evolution, leading authors, most cited papers, leading journals, competitions and collaboration networks.
Article
Person identification based on eye movements is getting more and more attention, as it is anti-spoofing resistant and can be useful for continuous authentication. Therefore, it is noteworthy for researchers to know who and what is relevant in the field, including authors, journals, conferences, and institutions. This paper presents a comprehensive quantitative overview of the field of eye movement biometrics using a bibliometric approach. All data and analyses are based on documents written in English published between 2004 and 2019. Scopus was used to perform information retrieval. This research focused on temporal evolution, leading authors, most cited papers, leading journals, competitions and collaboration networks.
Article
Full-text available
The way a person moves their eyes may not at first seem like the ideal basis for performing identification, but new research shows that this biometric type could offer several advantages. For example, while the biometric is difficult to counterfeit, it is nevertheless easy for a user to use the system. Moreover, it could also be possible to combine this type of biometric with other techniques such as iris or facial recognition.
Conference Paper
Full-text available
The paper presents a brand new technique of performing human identification which is based on eye movements characteristic. Using this method, the system measures human’s eyes reaction for visual stimulation. The eyes of the person who is being identified follow the point on the computer screen and eye movement tracking system is used to collect information about eye movements during the experiment. The first experiments showed that it was possible to identify people by means of that method. The method scrutinized here has several significant advantages. It compiles behavioral and physiological aspects and therefore it is difficult to counterfeit and at the same time it is easy to perform. Moreover, it is possible to combine it with other camera-based techniques like iris or face recognition.
Article
Full-text available
During reading, our eyes perform complicated sequences of fixations on words. Stochastic models of eye movement control suggest that this seemingly erratic behaviour can be attributed to noise in the oculomotor system and random fluctuations in lexical processing. Here, we present a qualitative analysis of a recently published dynamical model [Engbert et al., 2002] and propose that deterministic nonlinear control accounts for much of the observed complexity of eye movement patterns during reading. Based on a symbolic coding technique we analyze robust statistical features of simulated fixation sequences.
Article
Introduction: Basic Anatomy and Physiology of Eye Movements Basic Measurement Terms Basic Control System Concepts Eye Movement Measurement Techniques Static Analysis Techniques: Accommodation System Vergence Systems Linear Analysis of Relationship Between AC and ACG Nonlinear Analysis of AC/A Using the Phoria and Fixation Disparity Methods Proximal Model Sensitivity Analysis of Accommodation and Vergence Interactions Dynamic Analysis Techniques: Accommodation System - Root Locus Analysis Vergence Dual-Mode Dynamic Model Accommodative Dual-Mode Dynamic Characteristics Adaptation Model of Accommodation and Vergence Nearwork-Induced Transient Myopia (NITM) Model Refractive Error Development Model Saccade-Vergence Interactions Dynamic Model.
Article
The OBER 2 is an infrared light eye movement measuring system and it works with IBM PC compatible computers. As one of the safest systems for measuring of eye movement it uses a very short period of infrared light flashing time (80 microsecond for each measure point). System has an advanced analog-digital controller, which includes background suppression and prediction mechanisms guaranteeing elimination of slow changes and fluctuations of external illumination frequency up to 100 Hz, with effectiveness better than 40 dB. Setting from PC the active measure axis, sampling rate (25 - 4000 Hz) and making start and stop the measure, make it possible to control the outside environment in real-time. By proper controlling of gain it is possible to get high time and position resolution of 0.5 minute of arc even for big amplitude of eye movement (plus or minus 20 degree of visual angle). The whole communication system can also be driven directly by eye movement in real time. The possibility of automatic selection of the most essential elements of eye movement, individual for each person and those that take place for each person in determined situations of life independently from personal features, is a key to practical application. Hence one of conducted research topic is a personal identification based on personal features. Another task is a research project of falling asleep detection, which can be applied to warn the drivers before falling asleep while driving. This measuring system with a proper expert system can also be used to detect a dyslexia and other disabilities of the optic system.
Article
Subjects learned and recognized patterns which were marginally visible, requiring them to fixate directly each feature to which they wished to attend. Fixed "scanpaths," specific to subject and pattern, appeared in their saccadic eye movements, both intermittently during learning and in initial eye movements during recognition. A proposed theory of pattern perception explains these results.