ArticlePDF Available

Enhancing eye-movement-based biometric identification method by using voting classifiers

Authors:

Abstract and Figures

Eye movements contain a lot of information about human being. The way the eyes are moving is very complicated and eye movement patterns has been subject of studies for over 100 years. However, surprisingly, eye movement based identification is a quite new idea presented for the first time during the Biometrics'2003 Conference in London [17]. The method has several significant advantages: compiles behavioral and physiological properties of human body, it is difficult to forge and it is affordable-with a number of ready-to-use eye registering devices (so called eye trackers). The paper introduces the methodology and presents results of the first eye movement based authorization tests.
Content may be subject to copyright.
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
Enhancing eye movement based biometric identification method by using
voting classifiers
Paweł Kasprowski
1
, Józef Ober
1,2
1
Institute of Computer Science, Silesian University of Technology, 44-100 Gliwice, Poland
2
Institute of Theoretical and Applied Informatics, Polish Academy of Science, Gliwice, Poland
kasprowski@polsl.pl
Abstract. Eye movements contain a lot of information about human being. The way the eyes are moving is
very complicated and eye movement patterns has been subject of studies for over 100 years. However,
surprisingly, eye movement based identification is a quite new idea presented for the first time during the
Biometrics’2003 Conference in London [1]. The method has several significant advantages: compiles behavioral
and physiological properties of human body, it is difficult to forge and it is affordable – with a number of ready-
to-use eye registering devices (so called eye trackers). The paper introduces the methodology and presents test
results.
Introduction
Using eyes to perform biometric human identification has a long tradition including well-
established iris pattern recognition algorithms [2] and retina scanning. However, there are
only few researches concerning identification based on eye movement characteristic [3][4]. It
is a bit surprising because that method has several important advantages.
Firstly, it compiles physiological (muscles) and behavioral (brain) aspects. The most
popular biometric methods like fingerprint verification or iris recognition are based mostly on
physiological properties of human body. Therefore, what is needed for proper identification,
is only a “body” of a person who is to be identified. It makes possible to identify an
unconscious or - in some methods - even a dead person.
Moreover, physiological properties may be forged. Preparing models of a finger or even
retina (using special holograms) is technically possible. As eye movement based identification
uses information which is produced mostly by brain (so far impossible to be imitated), forging
this kind of information seems to be much more difficult.
Although it has not been studied in that paper, it seems possible to perform a covert
identification, i.e. identification of a person unaware of that process (for instance using hidden
cameras).
Last but not least, there are many easy to use eye-tracking devices nowadays, so
performing identification by means of that technique is not very expensive. For instance a
very fast and accurate OBER2 [5] eye tracking system was used in the present work. It
measures eye movements with a very high precision using infrared reflection and the
production costs are comparable to fingerprint scanners.
Experiment
To prove that eye movements may be used for human identification an experiment had to
be provided. The experiment was divided into two stages:
1) Gathering samples of human eye movements from different persons.
2) Processing of samples obtained in the previous step to extract individual features.
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
The process of data gathering consists of series of tests on different subjects (persons).
Each test is a registration of eye movements of the subject for the specified period of time
with the OBER2 system. The result of the single test is a sample, which is then used in the
second stage of the experiment.
There are three possibilities how to perform the single test:
- Registering eye movement only without information about an observed image.
- Registering both eye movements and the observed scene.
- Generating scene and registering eye movements as the answer for it.
The first solution is very simple to conduct even without any cooperation from the person
being identified. Eye movements may be measured during normal activity of that person
without any information about the observed image. However as eye movements are strongly
correlated with the scene, analysis of the data may be difficult.
The second solution gives a lot more data to analyze, yet it also has several serious
drawbacks. First of all, the testing system is more complicated. An additional camera recorder
is needed, which registers the scene the examined person is looking at. Furthermore, we need
to implement special algorithms to synchronize visual data with eye movement signal. A lot
more capacity is also needed for data storing.
In the third solution the testing system consists of the OBER2 eye tracker and PC computer
for both data storing and controlling the monitor, which produces a visual signal (Fig. 1). The
OBER2 system registers answers for that signal produced by the subject’s eyes. However, we
should be aware of the fact that the monitor screen is only a part of the image that eyes see, so
not the whole input is measured. Furthermore, the input may consist of non-visual signals.
Sudden loud sounds may, for instance, cause rapid eye movements.
Fig. 1 Schema of the system generating stimulation on a computer display and registering eye
movements as the answer to that stimulation.
As the last methodology gives influence on ‘input’ of the examined subject it seems to be
the most interesting from researcher point of view. Therefore all tests described in this work
were performed using a stimulation displayed on the monitor with the system architecture
presented on Fig. 1.
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
The stimulation, which has been chosen, was a ‘jumping point’ stimulation with the same
points order for every experiment. There were nine different point placements defined on the
screen, one in the middle and eight on the edges, creating 3 x 3 matrix. The point flashed in
one placement in a given moment. The stimulation began and ended with a point in the
middle of the screen. During the stimulation, point’s placement changed in specified intervals.
The main problem in developing stimulation is to make it short and informative. Those
properties are as if on two opposite poles, so a ‘golden mean’ must be found. It may be
assumed that gathering one sample could not last longer than 10 seconds. Longer stimulations
would be impractical when considering usage in real world. To be informative, experiment
should consist of as many point position changes as possible. However, moving a point too
quickly makes it impossible for eyes to follow it. Experiments and literature [6] confirmed
that the reaction time for change of stimulation is about 100-200 ms. After that time eyes start
a saccade, which moves fovea to the new gaze point. The saccade is very fast and lasts not
longer than 10-20 ms. After a saccade, the brain analyses a new position of the eyes and, if
necessary, tries to correct it. So very often about 50 ms after the first saccade the next saccade
happens. It can be called it a ‘calibration’ saccade. Therefore to register whole reaction for
point change it was necessary to provide an interval between point locations change as more
than 300 ms.
The stimulation, which has been developed and used during all tests, consists of eleven
point position changes giving twelve consecutive point positions. First point appears in the
middle of the screen and the person should look at it with eyes positioned direct ahead. After
1600 ms the point in the middle disappears and for 20 ms a screen is blank. In that time eyes
are in instable state waiting for another point of interest. That moment is uncomfortable for
eyes because there is no point to look at. Then the point appears in the upper right corner. The
flashing point on the blank screen attracts eyes attention even without person’s will. The
‘jumps’ of the point continue until the last point position in the middle of the screen is
reached.
a. 1600 ms
e. 550 ms
i. 550 ms
b. 550 ms
f. 550 ms
j. 550 ms
c. 550 ms
g. 550 ms
k. 550 ms
d. 550 ms
h. 550 ms
l. 1100 ms
Fig. 2 Visual description of stimulation steps (a-l).
Testing
The data obtained in the sample gathering phase has to be transformed to create a vector of
attributes, which will be used for identification process. Each feature should give some
information about a person who was the subject of the experiment. That information may be
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
understandable for instance “his dominant eye is left” or “his eyes are flickering with
frequency 10 Hz”, but the meaning of the feature may be hidden also, giving only the value.
The main problem is how to extract a set of features that have values for different samples of
the same person (inner-class samples) as similar as possible and that have values for different
person’s samples as different as possible.
As it was mentioned earlier, identification based on eye movement analysis is a brand new
technique. The main disadvantage of that is that one cannot use already published algorithms
and just try to improve it with one’s own methods. Therefore, we could only try to use
methods, which have been successfully used with similar problems like:
- Methods used for analyzing eye movement data.
- General methods used in signal processing and classification.
First stage was preprocessing of samples with an normalization algorithm. The
preprocessed samples were included into a dataset containing over 1000 samples obtained
from 47 persons. The next stage was calculation of different samples characteristics like
Fourier transform or average velocity directions and storing it in separate datasets.
Each dataset consisted of vectors of attributes calculated from samples. Most of these
attributes were completely useless for classification. Such irrelevant attributes are not only
making classification more complex and time consuming task but also can disturb it and
increase identification errors. Therefore the next task was extraction of relevant attributes
from the attributes vectors. There was Principal Component Analysis (PCA) technique used
for linear conversion of attributes to create a new attributes explaining most of the dataset
variance.
The data were tested only with authorization tests – that is all samples were always divided
into positive (belonging to the specified person) and negative (belonging to different persons).
The testing process randomly created a training set taking 20 positive and 80 negative
samples from the whole dataset. After preparing PCA for the training set, the classification
models for different numbers of the most significant attributes were created using SVM [7]
and C45 Decision Tree [8] techniques. All samples not chosen to the training set were then
used for data testing. There were separate tests for each conversion, number of the most
significant PCA attributes and classification algorithm performed. Therefore there were 75
results created for each testing phase. These results were than used to create the final result in
a simple voting classifiers algorithm.
Results
There were two kinds of experiments performed. In the first, the test was performed only
once giving positive or negative result. The average, best and worst error rates are presented
in the table below.
Table 1. Average error rates for “one trial” experiment
FAR FRR
worst 3,19 26,94
average 2,31 14,53
best 0,56 3,93
In the second experiment two subsequent trials made on the same user one by one were
used as the one test. If both trials failed the user was rejected, if one or both succeeded the
user was accepted. Such scenario obviously increased FAR error because it was easer for not
honest person to mislead system but FRR error was significantly lower.
This is a pre-print.
The final version of this paper was published in Proceedings of SPIE, Volume 5779
Table 2. Average error rates for “one of two trials” experiment
FAR FRR
worst 7,13 21,45
average 4,73 10,49
best 1,82 3,44
Summary
The idea presented here seems to be interesting but the results obtained are far from
perfection. However the results are showing that eye movements are indeed giving some
information about human’s identification and they are encouraging for future effort.
Literature
[1] Kasprowski, P., Ober, J.: Eye movement tracking for human identification. 6th World
Conference BIOMETRICS’2003, London (2003)
[2] Daugman, J.G.: High Confidence Visual Recognition of Persons by a Test of Statistical
Independence, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
15, no. 11 (1993)
[3] Kasprowski P., Ober J.: With the flick of an eye. Biometrics Technology Today ISSN
0969-4765, Volume 12, Issue 3, Elsevier Science (March 2004)
[4] Kasprowski, P., Ober, J.: Eye Movement in Biometrics. Proceedings of Biometric
Authentication Workshop, European Conference on Computer Vision in Prague 2004,
LNCS 3087, Springer-Verlag, Berlin (2004)
[5] Ober, J., Hajda, J., Loska, J., Jamnicki, M.: Application of Eye Movement Measuring
System OBER2 to Medicine and Technology. Proceedings of SPIE, Infrared
Technology and applications, Orlando, USA, 3061(1) (1997)
[6] Hung, G. K.: Models of Oculomotor Control, World Scientific Publishing Co. (2001)
[7] Vapnik, V.: Statistical Learning Theory. John Wiley and Sons, Inc. New York (1998)
[8] Quinlan, J. R.: C4.5: Programs for Machine Learning. San Mateo: Morgan Kaufmann
(1993)
... The approaches proposed to extract eye movement features can be split into two categories: traditional approaches [19]- [22] and deep learning (DL)-based approaches [23]- [27]. The traditional approaches include hand-crafted descriptors, e.g., Fixation Density Map (FDM) [28] and traditional machine learning (ML) approaches such as Principal Component Analysis (PCA) [20] and SVM [29]. For handcrafted descriptors, researchers assume a specific probability distribution for the data and propose ad hoc models for feature extraction. ...
... In 2004, Kasprowski et al. [19] proposed using eye-movement features for identity recognition on a self-constructed database. In a later study, [20] used PCA for dimensionality reduction of eye movement features, while Komogortsev et al. [21] considered the bioanatomic properties of the eyeball and established a linear horizontal oculomotor plant mechanical model. [22], [43] expanded on this foundation and proposed the concept of oculomotor plant characteristics (OPC). ...
Preprint
Eye movement biometrics has received increasing attention thanks to its high secure identification. Although deep learning (DL) models have been recently successfully applied for eye movement recognition, the DL architecture still is determined by human prior knowledge. Differentiable Neural Architecture Search (DARTS) automates the manual process of architecture design with high search efficiency. DARTS, however, usually stacks the same multiple learned cells to form a final neural network for evaluation, limiting therefore the diversity of the network. Incidentally, DARTS usually searches the architecture in a shallow network while evaluating it in a deeper one, which results in a large gap between the architecture depths in the search and evaluation scenarios. To address this issue, we propose EM-DARTS, a hierarchical differentiable architecture search algorithm to automatically design the DL architecture for eye movement recognition. First, we define a supernet and propose a global and local alternate Neural Architecture Search method to search the optimal architecture alternately with an differentiable neural architecture search. The local search strategy aims to find an optimal architecture for different cells while the global search strategy is responsible for optimizing the architecture of the target network. To further reduce redundancy, a transfer entropy is proposed to compute the information amount of each layer, so as to further simplify search network. Our experiments on three public databases demonstrate that the proposed EM-DARTS is capable of producing an optimal architecture that leads to state-of-the-art recognition performance.
... Among those behavioural-based approaches, eye movement biometrics (EMB) has proven to have great potential in a variety of contexts. In stimulus-dependent scenarios we can include reading [22,23], static point gazing [24] or jumping point tracking [25][26][27], virtual reality tasks [28][29][30], free viewing of static images [31,32], including faces [33,34]. The more challenging task-independent scenario has been explicitly investigated by Kinnunen et al. [35], whereby no assumptions that the same stimulus appears in training and testing are made. ...
Article
Full-text available
A principled approach to the analysis of eye movements for behavioural biometrics is laid down. The approach grounds in foraging theory, which provides a sound basis to capture the uniqueness of individual eye movement behaviour. We propose a composite Ornstein-Uhlenbeck process for quantifying the exploration/exploitation signature characterising the foraging eye behaviour. The relevant parameters of the composite model, inferred from eye-tracking data via Bayesian analysis, are shown to yield a suitable feature set for biometric identification; the latter is eventually accomplished via a classical classification technique. A proof of concept of the method is provided by measuring its identification performance on a publicly available dataset. Data and code for reproducing the analyses are made available. Overall, we argue that the approach offers a fresh view on either the analyses of eye-tracking data and prospective applications in this field.
... Significant studies were done on identifying people using macro eye movements. The most have used similar stimuli for training and testing, such as static cross [4,7], a jump point [2,4,[8][9], a picture of a face [2,10,11] or different types of images [12]. Using all these stimuli for the train and test, they have devised plans to counter these attacks. ...
Article
Full-text available
In recent years, security concerns, including document fraud and identity theft, terrorism, and cybercrime have led to the growth and development of biometric technologies, and in general, all activities related to biometric research have gained momentum. This research presents a more intelligent methodology for authenticating and verifying one's identity while reading and typing a text. Threats to e-commerce security are wreaking havoc on online trade via a variety of nefarious methods. Thus, this method can be implemented to furtively identify these attacks and impede them, using a variety of texts to study. We refer to the authentication as a behavioral biometric technology based on how people look when typing in which the visual information was implemented to verify the identity of 35 users while typing via using an eye tracker device. Hence, we have employed the two-layer perceptron neural network to model the learning phase in the object recognition process. The fusion of eye tracker device information and two-layer perceptron neural network coefficients was done at the decision level, and we have achieved an average of 98.97%, 98.76% and 98.86% for sensitivity, specificity and accuracy, respectively. By the combination of biometric technology products as well as modern computer technology, it is easy to perform monitoring, management, systems integration, automated management, and security applications.
... Example of Bio-Ran dataset. The point of light is visible at a different location after a set time interval[68]. Bio-Tex Stimulus. ...
Thesis
Full-text available
This thesis works with the eye-tracking data in the project 'Schau mir in die Augen'. Abstract The study includes working with eye movements for user identification. The main objective of the thesis is to improve the accuracy by modifying a pre-existing pipeline to recognize and take advantage of outliers. For this goal, the reasons for outliers and their particularities are investigated. Various methods are used and compared for detecting the outliers. It is found that removing the outliers from the pipeline decreases the accuracy. Instead, it is studied to include them in the pipeline. For this, a new classifier is added into the pipeline along with fixations and saccades, which deal with outliers. All the experiments are performed on the Bio-Tex dataset and lead to an improvement of accuracy. The final pipeline is tried on the Bio-Ran dataset, but there are no conclusive results, and more experiments need to be performed. Two classifiers are used in the thesis, majorly is Random Forest, and the other is Radial Basis Function Network. The Bio-Tex shows an improvement in accuracy by considering the outliers for the two classifiers. For Bio-Ran, there are no conclusive results obtained. Outliers play a vital role in eye movements and show the potential to improve the accuracy in the user identification field with more research.
... The various approaches further differ with respect to the stimulus type they operate on; a static cross [3], various kinds of images [7] including faces [41,14,4], text [17,1,33,19] or dynamical stimuli such as jumping points [21,25,42,5,45] or video sequences [26] have been used. Whereas most of the methods use the same stimulus for training, enrollment and testing-which makes them vulnerable to replay attacks-Kinnunen et al. are the first to identify users on a novel stimulus, and subsequent approaches identify readers on novel text [17,1,33,19]. ...
... There is a substantial body of research on biometric identification using macromovements of the eye. Most work uses the same stimulus for training and testing-such as a static cross [4], a jumping point [22,23,44,8,47], a face image [43,17,5], or various other kinds of images [10]. Using the same known stimulus for training and testing opens the methods to replay attacks. ...
Chapter
Full-text available
We study involuntary micro-movements of the eye for biometric identification. While prior studies extract lower-frequency macro-movements from the output of video-based eye-tracking systems and engineer explicit features of these macro-movements, we develop a deep convolutional architecture that processes the raw eye-tracking signal. Compared to prior work, the network attains a lower error rate by one order of magnitude and is faster by two orders of magnitude: it identifies users accurately within seconds.
... Here, it is possible to unambiguously identify the person by means of the eye behavior. e rst approaches required a moving point stimulus which was followed by the user [30][31][32] or static images [45]. In 2010, the rst approach that was able to distinguish users with a task independent approach was presented [2]. ...
Preprint
Full-text available
In this paper, we present an approach based on reinforcement learning for eye tracking data manipulation. It is based on two opposing agents, where one tries to classify the data correctly and the second agent looks for patterns in the data, which get manipulated to hide specific information. We show that our approach is successfully applicable to preserve the privacy of a subject. In addition, our approach allows to evaluate the importance of temporal, as well as spatial, information of eye tracking data for specific classification goals. In general, this approach can also be used for stimuli manipulation, making it interesting for gaze guidance. For this purpose, this work provides the theoretical basis, which is why we have also integrated a section on how to apply this method for gaze guidance.
Chapter
In this paper, we present an approach based on reinforcement learning for eye tracking data manipulation. It is based on two opposing agents, where one tries to classify the data correctly and the second agent looks for patterns in the data, which get manipulated to hide specific information. We show that our approach is successfully applicable to preserve the privacy of a subject. For this purpose, we evaluate our approach iterative to showcase the behavior of the reinforcement learning based approach. In addition, we evaluate the importance of temporal, as well as spatial, information of eye tracking data for specific classification goals. In the last part of our evaluation we apply the procedure to further public data sets without re-training the autoencoder nor the data manipulator. The results show that the learned manipulation is generalized and applicable to other data too.
Preprint
Person identification based on eye movements is getting more and more attention, as it is anti-spoofing resistant and can be useful for continuous authentication. Therefore, it is noteworthy for researchers to know who and what is relevant in the field, including authors, journals, conferences, and institutions. This paper presents a comprehensive quantitative overview of the field of eye movement biometrics using a bibliometric approach. All data and analyses are based on documents written in English published between 2004 and 2019. Scopus was used to perform information retrieval. This research focused on temporal evolution, leading authors, most cited papers, leading journals, competitions and collaboration networks.
Article
Person identification based on eye movements is getting more and more attention, as it is anti-spoofing resistant and can be useful for continuous authentication. Therefore, it is noteworthy for researchers to know who and what is relevant in the field, including authors, journals, conferences, and institutions. This paper presents a comprehensive quantitative overview of the field of eye movement biometrics using a bibliometric approach. All data and analyses are based on documents written in English published between 2004 and 2019. Scopus was used to perform information retrieval. This research focused on temporal evolution, leading authors, most cited papers, leading journals, competitions and collaboration networks.
Article
Full-text available
The way a person moves their eyes may not at first seem like the ideal basis for performing identification, but new research shows that this biometric type could offer several advantages. For example, while the biometric is difficult to counterfeit, it is nevertheless easy for a user to use the system. Moreover, it could also be possible to combine this type of biometric with other techniques such as iris or facial recognition.
Conference Paper
Full-text available
The paper presents a brand new technique of performing human identification which is based on eye movements characteristic. Using this method, the system measures human’s eyes reaction for visual stimulation. The eyes of the person who is being identified follow the point on the computer screen and eye movement tracking system is used to collect information about eye movements during the experiment. The first experiments showed that it was possible to identify people by means of that method. The method scrutinized here has several significant advantages. It compiles behavioral and physiological aspects and therefore it is difficult to counterfeit and at the same time it is easy to perform. Moreover, it is possible to combine it with other camera-based techniques like iris or face recognition.
Article
Full-text available
During reading, our eyes perform complicated sequences of fixations on words. Stochastic models of eye movement control suggest that this seemingly erratic behavior can be attributed to noise in the oculomotor system and random fluctuations in lexical processing. Here, we present a qualitative analysis of a recently published dynamical model [Engbert et al., 2002] and propose that deterministic nonlinear control accounts for much of the observed complexity of eye movement patterns during reading. Based on a symbolic coding technique we analyze robust statistical features of simulated fixation sequences.
Article
Introduction: Basic Anatomy and Physiology of Eye Movements Basic Measurement Terms Basic Control System Concepts Eye Movement Measurement Techniques Static Analysis Techniques: Accommodation System Vergence Systems Linear Analysis of Relationship Between AC and ACG Nonlinear Analysis of AC/A Using the Phoria and Fixation Disparity Methods Proximal Model Sensitivity Analysis of Accommodation and Vergence Interactions Dynamic Analysis Techniques: Accommodation System - Root Locus Analysis Vergence Dual-Mode Dynamic Model Accommodative Dual-Mode Dynamic Characteristics Adaptation Model of Accommodation and Vergence Nearwork-Induced Transient Myopia (NITM) Model Refractive Error Development Model Saccade-Vergence Interactions Dynamic Model.
Article
The OBER 2 is an infrared light eye movement measuring system and it works with IBM PC compatible computers. As one of the safest systems for measuring of eye movement it uses a very short period of infrared light flashing time (80 microsecond for each measure point). System has an advanced analog-digital controller, which includes background suppression and prediction mechanisms guaranteeing elimination of slow changes and fluctuations of external illumination frequency up to 100 Hz, with effectiveness better than 40 dB. Setting from PC the active measure axis, sampling rate (25 - 4000 Hz) and making start and stop the measure, make it possible to control the outside environment in real-time. By proper controlling of gain it is possible to get high time and position resolution of 0.5 minute of arc even for big amplitude of eye movement (plus or minus 20 degree of visual angle). The whole communication system can also be driven directly by eye movement in real time. The possibility of automatic selection of the most essential elements of eye movement, individual for each person and those that take place for each person in determined situations of life independently from personal features, is a key to practical application. Hence one of conducted research topic is a personal identification based on personal features. Another task is a research project of falling asleep detection, which can be applied to warn the drivers before falling asleep while driving. This measuring system with a proper expert system can also be used to detect a dyslexia and other disabilities of the optic system.
Article
Subjects learned and recognized patterns which were marginally visible, requiring them to fixate directly each feature to which they wished to attend. Fixed "scanpaths," specific to subject and pattern, appeared in their saccadic eye movements, both intermittently during learning and in initial eye movements during recognition. A proposed theory of pattern perception explains these results.