Content uploaded by Pawel Kasprowski
Author content
All content in this area was uploaded by Pawel Kasprowski
Content may be subject to copyright.
A preview of the PDF is not available
The inf uence of dataset quality on the results of behavioral biometric experiments
Abstract and Figures
This paper explores some aspects that are involved during the construction of reliable benchmark sample databases for novel behavioral biometric identification methods, such as the data quality, the recording patterns and the post processing procedures that may be applied on the data. A large collection of eye movement samples was employed as a test case. It was recorded under a variety of settings and processed with a number of different approaches. Our analysis reveals that there are specific features during the construction of a database that may significantly influence the final identification performance. It also leads on the establishment of some guidelines, which can be generalized on other behavioral biometric methods, regarding the factors that should be taken into consideration during the creation, the description and the processing of a database of biometric samples.
Figures - uploaded by Pawel Kasprowski
Author content
All figure content in this area was uploaded by Pawel Kasprowski
Content may be subject to copyright.
... The randomness makes a random forest resilient to overfitting of the data [57]. A random forest was also adopted in previous work, such as [58] and [59]. We implemented the above classification and cross-validation process using the Scikit-Learn Python library [60], a commonly used machine learning open-source library. ...
... The randomness makes a random forest resilient to overfitting of the data [57]. A random forest was also adopted in previous work, such as [58,59]. We implemented the above classification and cross-validation process using the Scikit-Learn Python library [60], a commonly used machine learning open-source library. ...
Eye movement biometrics can enable continuous verification for highly secure environments such as financial transactions and defense establishments, as well as a more personalized and tailored experience in gaze-based human-computer interactions. However, there are numerous challenges to recognizing people in real environments using eye movements, such as implicity and stimulus independence. In the instance of wayfinding, this research intends to investigate implicit and stimulus-independent eye movement biometrics in real-world situations. We collected 39 subjects' eye movement data from real-world wayfinding experiments and derived five sets of eye movement features (the basic statistical, pupillary response, fixation density, fixation semantic and saccade encoding features). We adopted a random forest and performed biometric recognition for both identification and verification scenarios. The best accuracy we obtained in the identification scenario was 78% (equal error rate, EER = 6.3%) with the 10-fold classification and 64% (EER = 12.1%) with the leave-one-route-out classification. The best accuracy we achieved in the verification scenario was 89% (EER = 9.1%). Additionally, we tested performance across the 5 feature sets and 20 time window sizes. The results showed that the verification accuracy was insensitive to the increase in the time window size. These findings are the first indication of the viability of performing implicit and stimulus-independent biometric recognition in real-world settings using wearable eye tracking.
... It is very important to emphasize that the division into training and testing sets was not random. Consecutively collected trials tend to be more similar to each other than trials collected after longer intervals; therefore, due to the shortterm learning effect [47], including them in both training and testing sets may produce improperly obtained better accuracy results. Hence, the general rule was not to use trials of the same user gathered in the same session for both training and testing purpose. ...
... The learning effect is a phenomenon characteristic of biometric modalities that measures changes of human behavior over time [47]. It is sometimes treated as a kind of well-known template aging problem, but its nature is slightly different. ...
This paper presents the first attempt to fuse two different kinds of behavioral biometrics: mouse dynamics and eye movement biometrics. Mouse dynamics were collected without any special equipment, while an affordable The Eye Tribe eye tracker was used to gather eye movement data at a frequency of 30 Hz, which is also potentially possible using a common web camera. We showed that a fusion of these techniques is quite natural and it is easy to prepare an experiment that collects both traits simultaneously. Moreover, the fusion of information from both signals gave 6.8 % equal error rate and 92.9 % accuracy for relatively short registration time (20 s on average). Achieving such results were possible using dissimilarity matrices based on dynamic time warping distance.
... (accuracy: 82%),Kasprowski and Rigas (2013) (86.2%), Schröder et al. (2020) (accuracy: 86.7%), Saeed (2016) (identification rate: 85.72%), George and Routray (2016) (accuracy: 89.54%, EER: 2.59%) and ...
Individuals with different characteristics exhibit different eye movement patterns in map reading and wayfinding tasks. In this study, we aim to explore whether and to what extent map users’ eye movements can be used to detect who created them. Specifically, we focus on the use of gaze data for inferring users’ identities when users are performing map-based spatial tasks. We collected 32 participants’ eye movement data as they utilized maps to complete a series of self-localization and spatial orientation tasks. We extracted five sets of eye movement features and trained a random forest classifier. We used a leave-one-task-out approach to cross-validate the classifier and achieved the best identification rate of 89%, with a 2.7% equal error rate. This result is among the best performances reported in eye movement user identification studies. We evaluated the feature importance and found that basic statistical features (e.g. pupil size, saccade latency and fixation dispersion) yielded better performance than other feature sets (e.g. spatial fixation densities, saccade directions and saccade encodings). The results open the potential to develop personalized and adaptive gaze-based map interactions but also raise concerns about user privacy protection in data sharing and gaze-based geoapplications.
... It shows that short term repeatability of eye movement caused by current attitude or mood of a person may significantly (and artificially) improve classification results. It is in line with conclusions derived in [4] and [9]. ...
The paper presents studies on the application of the dissim-ilarity matrix-based method to the eye movement analysis. This method was utilized in the biometric identification task. To assess its efficiency four different datasets based on similar scenario ('jumping point' type) yet using different eye trackers, recording frequencies and time intervals have been used. It allowed to build the common platform for the research and to draw some interesting comparisons. The dissimilarity matrix, which has never been used for identifying people on the basis of their eye movements, was constructed with usage of different distance measures. Additionally, there were different signal transforms and met-rics checked and their performance on various datasets was compared. It is worth mentioning that the paper presents the algorithm that was used during the BioEye 2015 competition and ranked as one of the top three methods.
... Before the analysis of recorded data it is crucial to improve its quality with the help of off-line digital processing [29]. To remove distortions caused by the electrical line drift and high frequency noise a [30] band-pass filtering was applied to the recorded signals. ...
Performing complex activities that accompany piloting an airplane requires a lot of concentration, coordination and reflex from the pilots. Additionally, the responsibility that lies on a person that controls any kind of flying platform can lead to the increased amount of pressure and stress. Considering all of the aforementioned conditions, it is essential to provide a system capable of evaluating and monitoring of pilot's alertness and mental fatigue during different phases of flight. Such solution can potentially drastically improve the efficiency of flight operating staff, which may lead to the increase in the overall flight safety. In this study we attempt to propose the statistical measures allowing to evaluate mental parameters of airplane pilots during the simulated flight sessions. Thanks to that, a flight aiding solution can be proposed, which would be capable of monitoring and alerting of any potentially dangerous cases of pilot's decreased concentration or increased weariness. Additionally, the specific requirements of different flight phases could be taken into consideration. The electroencephalography (EEG) is one of the most popular technologies used for the recording and monitoring of brain bioelectrical activity. Such activity can then be properly processed and used for the task of statistical description of mental states required for this research. For experiment presented in this article, the Emotiv EPOC+ Scientific Contextual EEG devices were used. Such apparatus provides 14 EEG channels positioned with the accordance to the standard 10–20 electrode placement system. Very important features of the EPOC+ headsets are their portability and ease of montage which are critical in this study. In the research a group of 5 people, with no or little previous experience with flight simulators, participated in an approximately 3 hour long session each. During that time they were asked to continuously perform a piloting sequence which consisted of 3 flight phases: starting (approx. 1 minute), flight (approx. 6 minutes, 4 turns) and landing (approx. 1 minute). As a result about 15 full flight sequences for each subject were obtained and later used for the purpose of alertness and mental fatigue evaluation. All experiments were performed on the Elite S812 FNPT II and Elite S923 FNPT II MCC flight simulators.
Person identification based on eye movements is getting more and more attention, as it is anti-spoofing resistant and can be useful for continuous authentication. Therefore, it is noteworthy for researchers to know who and what is relevant in the field, including authors, journals, conferences, and institutions. This paper presents a comprehensive quantitative overview of the field of eye movement biometrics using a bibliometric approach. All data and analyses are based on documents written in English published between 2004 and 2019. Scopus was used to perform information retrieval. This research focused on temporal evolution, leading authors, most cited papers, leading journals, competitions and collaboration networks.
Person identification based on eye movements is getting more and more attention, as it is anti-spoofing resistant and can be useful for continuous authentication. Therefore, it is noteworthy for researchers to know who and what is relevant in the field, including authors, journals, conferences, and institutions. This paper presents a comprehensive quantitative overview of the field of eye movement biometrics using a bibliometric approach. All data and analyses are based on documents written in English published between 2004 and 2019. Scopus was used to perform information retrieval. This research focused on temporal evolution, leading authors, most cited papers, leading journals, competitions and collaboration networks.
The topic of this paper concerns a way of robotic manipulator control with use of hand gestures. The previously developed hand landmarks detection and localization algorithm is now translated to work with robotic arm manipulator. The proposed algorithm is used for hand gesture recognition and continuous feature points tracking. The points coordinates and their detected paths are transformed into set of orientation and location values in order to achieve proper motion of manipulator. The article presents the way of implementation of hand landmarks tracking algorithm and implementation of robot arm manipulator control algorithm. The main emphasis is laid on the critical problems encountered during the implementation of those two algorithms and the problem of translation of the detection coordinate system into robot task space.
In this paper a simple method of orientational path planning for commonly used 6R manipulators is presented. The chosen criterion is to minimize changes of values of manipulator's joints. For a given set of locations a proper orientation of end-effector has to be computed. The three Euler angles (α, β, γ) have been replaced with one angle (δ). In order to reduce the number of variables describing the orientation an algorithm which derives a plane equation based on three neighboring points is proposed. The angle between normal vector to the plane and one of the vectors describing the coordinate system attached to the end-effector is equal to zero. The choice of right vector is based on end-effector construction and operator's knowledge. If three neighboring points are collinear, algorithm creates the plane based on two points of given path and the origin of base coordinate system. If these four points are collinear, the plane is created with use of the points projection to the base X0Y0 plane or plane parallel to X0Y0. In order to ensure the path between considered points an interpolation is made. A comparison of Hermite polynomials and linear segments with parabolic blends is presented. The angle δ describes the rotation angle of coordinate system at start of the first iteration. The angle change is chosen in order to minimize the joint values changes.
This paper presents the results of the first eye movement verification and identification competition. The work provides background, discusses previous research, and describes the datasets and methods used in the competition. The results highlight the importance of very careful eye positional data capture to ensure meaningfulness of identification outcomes. The discussion about the metrics and scores that can assist in evaluation of the captured data quality is provided. Best identification results varied in the range from 58.6% to 97.7% depending on the dataset and methods employed for the identification. Additionally, this work discusses possible future directions of research in the eye movement-based biometrics domain.
Eye movements’ identification is an interesting alternative to other
biometric identification methods. It compiles both physiological and behavioral
aspects and therefore it is difficult to forge. However, the main obstacle to popularize this methodology is lack of general recommendations considering eye
movement biometrics experiments. Another problem is lack of commonly
available databases of eye movements. Different authors present their methodologies using their own datasets of samples recorded with different devices and
scenarios. It excludes possibility to compare different approaches. It is obvious
that the way the samples were recorded influences the overall results. This work
tries to investigate how one of the elements – temporal proximity between subsequent measurements – influences the identification results. A dataset of 2556
eye movement recordings collected for over 5 months was used as the basis of
analyses. The main purpose of the paper is to identify the impact of sampling
and classification scenarios on the overall identification results and to recommend scenarios for creation of future datasets.
Data quality limits the accuracy of biometrics. Poor data quality is responsible for many or even most matching errors in biometric systems and may be the greatest weakness of some implementations. This paper uses an expansive view of data quality focused on its implications for operational databases and systems. Data quality is defined here not just in regard to samples (images), but also in regard to metadata, which is the information associated with the samples, such as biographic or cross-referencing information. The impact of poor data quality can be reduced in various ways, many of which depend on effective methods of automated data quality measurement. This paper analyzes the causes and implications of poor-quality biometric data, methods of measurement, prevention, and potential remedies. The paper draws on the experience of the authors in engineering and analysis work on a variety of large-scale operational biometric systems and biometric evaluations.
The baseline corpus of a new multimodal database, acquired in the framework of the FP6 EU BioSec Integrated Project, is presented. The corpus consists of fingerprint images acquired with three different sensors, frontal face images from a webcam, iris images from an iris sensor, and voice utterances acquired both with a close-talk headset and a distant webcam microphone. The BioSec baseline corpus includes real multimodal data from 200 individuals in two acquisition sessions. In this contribution, the acquisition setup and protocol are outlined, and the contents of the corpus-including data and population statistics-are described. The database will be publicly available for research purposes by mid-2006.
Human identification is an important task for various activities in society. In this paper, we consider the problem of human identification using eye movement information. This problem, which is usually called the eye movement identification problem, can be solved by training a multiclass classification model to predict a person's identity from his or her eye movements. In this work, we propose using Mel-frequency cepstral coefficients (MFCCs) to encode various features for the classification model. Our experiments show that using MFCCs to represent useful features such as eye position, eye difference, and eye velocity would result in a much better accuracy than using Fourier transform, cepstrum, or raw representations. We also compare various classification models for the task. From our experiments, linear-kernel SVMs achieve the best accuracy with 93.56% and 91.08% accuracy on the small and large datasets respectively. Besides, we conduct experiments to study how the movements of each eye contribute to the final classification accuracy.
This research work proposes an innovative processing scheme for the exploitation of eye movement dynamics on the field of biometrical identification. As the mechanisms that derive eye movements highly depend on each person's idiosyncrasies, cues that reflect at a certain extent individual characteristics may be captured and subsequently deployed for the implementation of a robust identification system. Our methodology involves the employment of a non - parametric statistical test, the multivariate Wald - Wolfowitz test (WW-test), in order to compare the distributions of saccadic velocity and acceleration features, which are extracted while a person fixates on visual stimuli. In the evaluation section we use two publicly available datasets that supply recorded eye movements from a number of subjects during the observation of a moving spot on a computer screen. The resulting identification rates exhibit the efficacy of the suggested scheme to adequately segregate people according to their eye movement traits.
Although the technical evaluation of biometric identification devices has a history spanning over two decades, it is only
now that a general consensus on test and reporting measures and methodologies is developing in the scientific community. By
“technical evaluation”, we mean the measurement of the five parameters generally of interest to engineers and physical scientists:
false match and false non-match rates, binning error rate, penetration coefficient and transaction times. Additional measures,
such as “failure to enroll” or “failure to acquire”, indicative of the percentage of the general population unable to use
any particular biometric method, are also important. We have not included in this chapter measures of more interest to social
scientists, such as user perception and acceptability. Most researchers now accept the “Receiver Operating Characteristic”
(ROC) curve as the appropriate measure of the application-dependent technical performance of any biometric identification
device. Further, we now agree that the error rates illustrated in the ROC must be normalized to be independent of the database
size and other “accept/reject” decision parameters of the test. This chapter discusses the general approach to application-dependent,
decision-policy independent testing and reporting of technical device performance and gives an example of one practical test.
System performance prediction based on test results is also discussed.
Computerised biometric systems are automated methods of verifying or recognising the identity of a user on the basis of some
physiological characteristic, like a fingerprint or some aspects of behaviour such as keystroke patterns. Behaviourally based
biometric systems include signature, speaker and keystroke verification. The investigation of psychological factors which
might impact on the efficiency of a behavioural biometric computer security monitoring system has to our knowledge not been
conducted. Of particular concern in the present paper are the potential effects of state anxiety on individual's physiological
and performance responses. It is suggested that in a behaviourally based biometric computer security monitoring system, state
anxiety may have sufficient effects to alter typical physiological and performance responses, resulting in an increased risk
of security challenges, interruption of work-flow and resultant poor performance. It is also proposed that behaviourally based
biometric systems may have the potential to be used as electronic performance monitoring systems, and typical responses to
such systems need to be examined when developing and implementing any behaviourally based biometric security system.