ChapterPDF Available

The Eye Movement Data Storage – Checking the Possibilities

Authors:

Abstract and Figures

The research currently conducted in various fields of interest base their experiments on the eye movement signal analysis. Among these areas cognitive studies on people’s willingness, intentions or skills and advanced interface design can be enumerated. There are also some studies attempting to apply the eye tracking in security tasks and biometric identification. During these experiments a lot of data is registered, which has to be stored for further analyses. This research purpose is to find efficient structures for managing data gathered in such a way. Four different storages were chosen: Oracle, Excel, MongoDB and Cassandra systems that were examined during various task realizations.
Content may be subject to copyright.
The eye movement data storage checking the
possibilities
Katarzyna Harezlak, Tomasz Wasinski, Pawel Kasprowski
Institute of Informatics
Silesian University of Technology
Gliwice, Poland
katarzyna.harezlak@polsl.pl, pawel.kasprowski@polsl.pl
Abstract. The research currently conducted in various fields of interest base
their experiments on the eye movement signal analysis. Among these areas
cognitive studies on people’s willingness, intentions or skills and advanced in-
terface design can be enumerated. There are also some studies attempting to ap-
ply the eye tracking in security tasks and biometric identification. During these
experiments a lot of data is registered, which has to be stored for further anal-
yses. This research purpose is to find efficient structures for managing data
gathered in such way. Four different storages were chosen: Oracle, Excel,
MongoDB and Cassandra systems that were examined during various tasks re-
alizations.
1 Introduction
The research currently conducted in various fields of interest uses the eye movement
signal to obtain information about people’s willingness, intentions or skills [1, 2] and
for supporting GUIs development [3, 4]. There are some studies attempting to apply
eye tracking in security tasks [5, 6] and biometric identification [7, 8, 9, 10, 11, 12].
Such studies require a preparation of an experimental set-up based on a specialized
device, called eye tracker (ET), responsible for providing eye movement data. There
are several types of eye trackers available, some of them are shown in Fig. 1
a)
The Eye Tribe
b)
SMI
c)
Tobii Glasses
d)
Own developed ET
Fig. 1. Example eye tracker types
It is noteworthy that it is possible to build a simple eye tracker using a single
CMOS camera with IRPass filter. Such kind of own developed video-oculography
(VOG) eye-tracker is presented in Fig. 1(d).
Dependently on recording frequency of an eye tracker used there may exist a ne-
cessity of registering between 25 to even 2000 samples per second. Such data to be
useful should be stored and accessible for further cognitive analysis.
The aim of the research presented in the paper was to find the most efficient struc-
tures to support the aforementioned task. To our knowledge there was not such stud-
ies conducted so far, whereas many research dealing with processing and analyzing of
eye movement data type were carried out. It can be expected that there are sets of data
collected during such experiments, which may be used repeatedly for various purpos-
es. Thus the question is how to store that data to use it efficiently.
Providing an answer for this question was the task of the experiments described in
the subsequent sections. They were devoted to the problems of moving data from
primary to the chosen structures, an efficiency of particular recording search and de-
livering data to the commonly used algorithms for extracting events from an eye
movement signal (like I-DT or I-VT [16, 17]).
2 The test environment
The research was conducted with usage of the previously presented own-developed
VOG eye tracker. The system generated 20 - 25 measurements per second. At the
beginning of the experiments all data obtained from the eye tracker was saved in text
files formatted according to TSV (Tab Separated Values) standard.
The test set consisted of 20 files including all together 30 282 samples. Each sam-
ple was described by a type, its registration time eye gaze coordinates. A content of an
exemplary file is presented in Table 1. Point type represents a position of a stimuli on
the screen (SS) or an eye movement sample (R) containing information about the
current gaze point.
Point type
Registration Time
X coordinate
R
1383154553160
0.5630817
R
1383154553225
0.5637367
R
1383154553281
0.5638777
R
1383154553355
0.5637559
R
1383154553416
0.5633145
SS
1383154553664
0.05
Table 1. Data on simple recordings
It could be expected that using storages dedicated to and supporting data pro-
cessing may improve overall analysis of the eye movement signal. Thus, there were
four type of storages chosen to study in terms of the research purposes:
Oracle Database 11g Express Edition,
MongoDB 2.6 Standard [14, 15] with MongoDB 1.5.2 PHP driver,
Apache Cassandra 2.0.7 [13],
Microsoft Excel - Microsoft Office 2010.
The computer serving as a workstation was equipped with:
Processor: Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz,
RAM: 8GB,
Windows 7 operating system.
Ensuring the same conditions for all chosen data storages was an important aspect
of the carried out studies. To avoid burdening results by tests previously done, only
one database system was operating at a time and RAM memory was cleaned before
each test.
3 Moving data to new storages
As it was discussed above in most cases the eye movement data is at first being
saved in text files (like for example in the previously presented format). To take ad-
vantage of the other storage types it has to be moved there from an original location.
The assessment of this process efficiency was the first research step of the studies.
Additionally, to check its scalability the experiment was repeated tenfold for different
amount of data 50 000, 500 000 and 1000 000 samples.
The obtained results, described in details in subsequent paragraphs, were averaged
and summarized on the chart presented in Fig. 2. To make this presentation possible
there was the logarithmic scale used.
Oracle.
In case of Oracle database, before the migration process started, one additional step
had to be done. Tables in a relational database are expected to have a primary key for
each record. This fact entailed necessity of a sequence object creation to ensure gen-
erating primary key automatically. After that the migration process became possible.
The average results of the tenfold each test execution is presented in Fig. 2. Moving
50 000 samples from the text file to the Oracle database took 26.977 sec, 500 000 -
324.407 sec. and for one million samples it lasted 783.153 sec. These results have
presented almost linear relationship between amount of data and its migration dura-
tion.
Excel.
Test undertaken using MS Excel in case of processing 50 000 samples finished with-
out any problems and the average duration of that task was 55.031 sec. However, the
trial regarding 500 000 samples has revealed a problem with RAM memory usage,
influencing the migrating data duration - averaging on 4 085.048 sec. (Fig. 2). The
analysis of reasons has indicated that the problem was the library used (PHPExcel -
http://www.codeplex.com/PHPExcel). It stored all analysed data in an inner table
being a type of cache, heavily using RAM memory. For this reason experiments with
1 000 000 samples have been omitted.
MongoDB.
The results of migrating the eye movement recordings to MongoDB database turned
out to be interesting, especially in case of the set of 1 000 000 samples (Fig. 2). While
the time of data migration increased linearly for two first amounts of data (9.787 sec.
for 50 000 and 94.042 sec. for 500 000 samples), in case of 1 000 000 the duration
was only about one and a half longer than for 500 000 elements.
Cassandra.
The outcomes obtained for Cassandra database 52.625 sec. for 50 000; 503.986 sec.
for 500 00 and 1 027.915sec. for 1 000 000 samples have shown, similarly to Oracle
system, almost a linear relationship between them. However, in this instance the exe-
cution time was two times longer for each case comparing to Oracle.
Fig. 2. Duration of moving data from text file to different storages.
The analysis of the chart clearly indicates that filling MongoDB database is related
to the most efficient migration process execution. Additionally, its outcome allows to
have an expectation that MongoDB efficiency will increase along with the growth of
the amount of data. On the other hand, exceptionally weak results of Excel incline to
the opinion that this program is not a good solution for managing big sets of data, in
this case, concerning eye movements. Yet, it should be checked using other program-
ing languages environments.
1
10
100
1000
10000
50 000 500 000 1 000 000
Time of data migration [s]
Number of samples
Oracle Excel MongoDB Cassandra
4 The efficiency of data access
Gathering of an eye moment signal makes sense if it is accessible for a further pro-
cessing and analysis for example determining gaze points or finding areas attracting
user’s attention. For this purpose two algorithms I-DT and I-VT [16, 17] - extract-
ing sets of fixations from eye movements have been implemented. During the exper-
iments data structures of the algorithms were filled with data from the previously
described storages. This operation was repeated tenfold for each of the structures and
the results were averaged. Due to the fact that both algorithms data structures were
similar, the analysis regards only one of them.
Once again the tests were conducted using sets with 50 000, 500 00 and 1 000 000
samples. The schema of the test setup is presented in Fig. 3
Fig. 3. The schema of the test setup
Each experiment concerning one data storage has been divided into three parts:
the task of the first one was to provide to the algorithms data from a particular eye
movement recording which means that data was filtered with a given condition;
subsequently, the tests aimed at counting all samples in a particular eye movement
recording were carried out;
finally, during the last part of the experiment, the time required to load all data
available in the storage to the algorithms data structure, were studied.
Examining the efficiency of providing data for further processing, the files in TSV
format were taken into consideration as well. The results obtained for the first task
realization showed that TSV format, with average duration equal 0.048 sec. (inde-
pendently of the number of recordings), and Oracle (Fig. 4) were the most efficient
storages. The stable outcome for text file resulted from the fact that each recording
was saved in a separate file with the known location on the disk. Thus, access to the
recording was tantamount to an access to a single file. The good results for Oracle
system can be easily explained taking well developed functionality of the search en-
gine into account.
The tests were also carried out for Excel storage, however their duration for 50 000
samples (16.789sec.) caused the abandonment of further studies in this environment.
The result is omitted on the chart to make the analysis of other values convenient.
Fig. 4. The duration of filtered data loading
During the second task counting samples in a chosen recording the best results,
for the previously explained reasons, once again were obtained for TSV file (0.012
sec.). However, the second most efficient system was MongoDB with almost the
same average task duration (0.013 sec), whereas Oracle database outcome was twice
worse (0.0251 sec) than both the previously mentioned storages types. Yet, signifi-
cantly the least efficient system was Cassandra one, reaching outcome between two
and five hundred times worse than other systems (5.551 sec).
In the last performance tests (loading all samples into algorithms data structures)
Cassandra surprisingly turned out to be the best one for two bigger sets, realizing its
task in about 7 sec., while the task duration was from two to four times worse in case
of other systems (Fig. 5). The difference for files in TSV format was even bigger
(eight times worse result not presented on the chart).
0
1
2
3
4
5
50 000 500 000 1 000 000
Filtered data loading duration [s]
Number of samples
Oracle MongoDB Cassandra
Fig. 5. All data loading time
5 Conclusion
The variety of the tasks, which were studied during the experiments, have revealed
that performance of a particular storage depend on an operation being executed.
The analysis of the obtained results showed that Excel was not a good solution
when big sets of samples had to be taken into account. It was perceived in both migra-
tion and loading data tasks. MongoDB database, in the former operation, featured by
the good performance, especially in case of big sets of data. The task duration did not
increased proportionally to size of data yet was lower than expected. The Oracle sys-
tem was slightly worse than MongoDB, which may be explained by the necessity of
generating primary key value for each sample. The poorest performance was observed
for Cassandra. Generating the row identifier required while inserting a new data, like
in the case of Oracle database, may be pointed out as one of the reasons. Yet, more
important influence on making migration process more costly had data structures used
in Cassandra database.
Different conclusions may be drawn when studying the outcomes of the rest of the
experiments. The files with TSV format turned out to be the most efficient structure
while working with filtered data counting and providing data from one recording.
However, it has to be once again emphasized that in case of this structure, samples of
a particular recording were stored in one file in the known location. Much worse re-
sults were achieved when all data, from all files, had to be taken into account. In this
case the worst performance from all of the analysed storages, with exception of Excel
one, were observed.
Oracle system was the most efficient in two tasks providing filtered and all re-
cordings to I-DT and I-VT algorithms. However the results were close to outcome of
0
10
20
30
40
50
60
70
50 000 500 000 1 000 000
Time of data loading [s]
Number of samples
TSV format Oracle MongoDB Cassandra
MongoDB database, which in turn was more efficient in checking the number of
samples within one recording.
Cassandra database, for which obtained the longest task execution during (1) the
first tests (loading and counting filtered data) and (2) managing the smallest set of
data, occurred to be the most efficient system in case of providing all data from the
bigger sets with 500 000 and 1 000 000 samples.
Reaching the conclusion, it can be said that Cassandra seems to be a system to be
considered when a lot of samples are supposed to be frequently used. The weak per-
formance during the migration process may not be very important problem if it is
expected to be carried out only once. Yet, if there are plans for selecting a subset of
samples more often than using all of them, MongoDB or Oracle database may be the
better solution.
The studies will be continued to confirm this research findings in regard to data in-
cluding additional information of experiments such as stimuli type and its representa-
tion.
6 Bibliography
1. Bednarik R, Vrzakova H., and Hradis M.: What do you want to do next: a novel approach
for intent prediction in gaze-based interaction. In Proceedings of the Symposium on Eye
Tracking Research and Applications (ETRA '12), Stephen N. Spencer (Ed.). ACM, New
York, NY, USA, 83-90, 2012.
2. Jarodzka, H., Scheiter, K., Gerjets, P., & van Gog, T.: In the eyes of the beholder: How
experts and novices interpret dynamic stimuli. Learning and Instruction, 20, 146-154,
2010..
3. Goldberg J. H., Stimson M. J., Lewenstein M, Scott N., and Wichansky A. M.:. Eye track-
ing in web search tasks: design implications. In Proceedings of the 2002 symposium on
Eye tracking research & applications (ETRA '02). ACM, New York, NY, USA, 51-58.
DOI=10.1145/507072.507082 http://doi.acm.org/10.1145/507072.507082
4. Jacob R. J. K.: Eye tracking in advanced interface design. In Virtual environments and
advanced interface design, Woodrow Barfield and Thomas A. Furness, III (Eds.). Oxford
University Press, Inc., New York, NY, USA 258-288.
5. Kasprowski P., Harężlak K.,: Cheap and easy PIN entering using eye gaze. 3rd Conference
on Cryptography and Security Systems. Annales UMCS Informatica, Volume 14, 1, pp 75-
84, Lublin, 2014.
6. De Luca A., Denzel M., and Hussmann H.: Look into my eyes!: can you guess my pass-
word?. In Proceedings of the 5th Symposium on Usable Privacy and Security (SOUPS
'09). ACM, New York, NY, USA, Article 7 , pp. 1-12, 2009.
7. Kasprowski, P.; Ober. J.: Eye Movement in Biometrics, Proceedings of Biometric Authen-
tication Workshop, European Conference on Computer Vision in Prague 2004, LNCS
3087, Springer-Verlag., 2004.
8. Bednarik, R.; Kinnunen, T.; Mihaila, A.; Fränti, P.: Eye-movements as a biometric, In 14
Scandinavian Conference on Image Analysis, Lecture Notes in Computer Science, Spring-
er-Verlag, vol. 3540, pp. 780-789, 2005.
9. Kinnunen T., Sedlak F., Bednarik R.: Towards task-independent person authentication us-
ing eye movement signals. Proceedings of the 2010 Symposium on Eye-Tracking Re-
search & Applications, ACM New York, NY, USA 2010.
10. Komogortsev O., Jayarathna S., Aragon C.R., Mahmoud M.: Biometric Identification via
an Oculomotor Plant Mathematical Model. Proceedings of the 2010 Symposium on Eye-
Tracking Research & Applications, ACM New York, NY, USA 2010.
11. Rigas, I.; Economou, G.; Fotopoulos, S.: Biometric identification based on the eye move-
ments and graph matching techniques, Pattern Recognition Letters, Volume 33, Issue 6, 15
April 2012, Pages 786-792, ISSN 0167-8655, 2012.
12. Kasprowski, P.; Komogortsev, O. V.; Karpov, A.: First Eye Movement Verification and
Identification Competition at BTAS 2012. IEEE Fifth International Conference on Biomet-
rics: Theory, Applications and Systems (BTAS), 2012.
13. http://cassandra.apache.org/ (dostęp wrzesień 2014) [16]
14. http://www.mongodb.org/ (dostęp wrzesień 2014)[17]
15. http://php.net/manual/en/mongo.tutorial.php (dostęp wrzesień 2014)[18]
16. Salvucci, D. D., Goldberg, J. H.: Identifying fixations and saccades in eye-tracking proto-
cols. In Proceedings of the 2000 Symposiumon Eye Tracking Research and Applications,
7178, NY: ACM Press (2000)
17. Shic, F., Chawarska, K., Scassellati, B.: The incomplete fixation measure. In Proceedings
of the 2008 Symposium on Eye Tracking Research and Applications, 111114, NY: ACM
Press, 2008.
Article
During eye-tracking experiments, a large amount of data is collected. Usually, it is stored in text files, or file formats meant for scientific data. Because these formats provide limited sets of mechanisms for more comprehensive data analysis, various database models were considered as an alternative for them in this study. They included a relational database defined in MS SQL Server and two commonly used NoSQL solutions – MongoDB and Cassandra. The data structures defined for each of them and their performance were compared based on a chosen set of operations. A special multi-database platform was developed, enabling access to all database models without knowing their mechanisms for manipulating data. The studies revealed that Cassandra is the best database to store eye-tracking data when adjusted to the data retrieving operations. MongoDB databases featured with similar performance for various query types, yet with worse execution time. The worst results were obtained for the relational database. Designed to avoid redundancy by assumption, it stores data in many tables, and many join operations are needed while reading data. This fact entails weaker operations efficiency.
Conference Paper
Full-text available
PINs are one of the most popular methods to perform simple and fast user authentication. PIN stands for Personal Identification Number, which may have any number of digits or even letters. Nevertheless, 4-digit PIN is the most common and is used for instance in ATMs or cellular phones. The main ad-vantage of the PIN is that it is easy to remember and fast to enter. There are however some drawbacks. One of them – addressed in this paper – is a possibility to stole the PIN by a technique called 'shoulder surfing'. To avoid such problems a novel method of the PIN entering was proposed. Instead of using a numerical keyboard, the PIN may be entered by eye gazes, which is a hands-free, easy and robust technique.
Conference Paper
Full-text available
This paper presents the results of the first eye movement verification and identification competition. The work provides background, discusses previous research, and describes the datasets and methods used in the competition. The results highlight the importance of very careful eye positional data capture to ensure meaningfulness of identification outcomes. The discussion about the metrics and scores that can assist in evaluation of the captured data quality is provided. Best identification results varied in the range from 58.6% to 97.7% depending on the dataset and methods employed for the identification. Additionally, this work discusses possible future directions of research in the eye movement-based biometrics domain.
Conference Paper
Full-text available
The paper presents a brand new technique of performing human identification which is based on eye movements characteristic. Using this method, the system measures human’s eyes reaction for visual stimulation. The eyes of the person who is being identified follow the point on the computer screen and eye movement tracking system is used to collect information about eye movements during the experiment. The first experiments showed that it was possible to identify people by means of that method. The method scrutinized here has several significant advantages. It compiles behavioral and physiological aspects and therefore it is difficult to counterfeit and at the same time it is easy to perform. Moreover, it is possible to combine it with other camera-based techniques like iris or face recognition.
Conference Paper
Full-text available
Authentication systems for public terminals { and thus pub- lic spaces { have to be fast, easy and secure. Security is of utmost importance since the public setting allows mani- fold attacks from simple shoulder surng to advanced ma- nipulations of the terminals. In this work, we present Eye- PassShapes, an eye tracking authentication method that has been designed to meet these requirements. Instead of using standard eye tracking input methods that require precise and expensive eye trackers, EyePassShapes uses eye ges- tures. This input method works well with data about the rel- ative eye movement, which is much easier to detect than the precise position of the user's gaze and works with cheaper hardware. Dierent evaluations on technical aspects, usabil- ity, security and memorability show that EyePassShapes can signicantly increase security while being easy to use and fast at the same time.
Conference Paper
Full-text available
The process of fixation identification—separating and labeling fixations and saccades in eye-tracking protocols—is an essential part of eye-movement data analysis and can have a dramatic impact on higher-level analyses. However, algorithms for performing fixation identification are often described informally and rarely compared in a meaningful way. In this paper we propose a taxonomy of fixation identification algorithms that classifies algorithms in terms of how they utilize spatial and temporal information in eye-tracking protocols. Using this taxonomy, we describe five algorithms that are representative of different classes in the taxonomy and are based on commonly employed techniques. We then evaluate and compare these algorithms with respect to a number of qualitative characteristics. The results of these comparisons offer interesting implications for the use of the various algorithms in future work.
Conference Paper
Full-text available
An eye tracking study was conducted to evaluate specific design features for a prototype web portal application. This software serves independent web content through separate, rectangular, user-modifiable portlets on a web page. Each of seven participants navigated across multiple web pages while conducting six specific tasks, such as removing a link from a portlet. Specific experimental questions included (1) whether eye tracking-derived parameters were related to page sequence or user actions preceding page visits, (2) whether users were biased to traveling vertically or horizontally while viewing a web page, and (3) whether specific sub-features of portlets were visited in any particular order. Participants required 2-15 screens, and from 7-360+ seconds to complete each task. Based on analysis of screen sequences, there was little evidence that search became more directed as screen sequence increased. Navigation among portlets, when at least two columns exist, was biased towards horizontal search (across columns) as opposed to vertical search (within column). Within a portlet, the header bar was not reliably visited prior to the portlet's body, evidence that header bars are not reliably used for navigation cues. Initial design recommendations emphasized the need to place critical portlets on the left and top of the web portal area, and that related portlets do not need to appear in the same column. Further experimental replications are recommended to generalize these results to other applications.
Conference Paper
Full-text available
We propose a person authentication system using eye movement signals. In security scenarios, eye-tracking has earlier been used for gaze-based password entry. A few authors have also used physical features of eye movement signals for authentication in a task-dependent scenario with matched training and test samples. We propose and implement a task-independent scenario whereby the training and test samples can be arbitrary. We use short-term eye gaze direction to construct feature vectors which are modeled using Gaussian mixtures. The results suggest that there are personspecific features in the eye movements that can be modeled in a task-independent manner. The range of possible applications extends beyond the security-type of authentication to proactive and user-convenience systems.
Article
The last few years a growing research interest has aroused in the field of biometrics, concerning the use of brain dependent characteristics generally known as behavioral features. Human eyes, often referred as the gates to the soul, can possibly comprise a rich source of idiosyncratic information which may be used for the recognition of an individual’s identity. In this paper an innovative experiment and a novel processing approach for the human eye movements is implemented, ultimately aiming at the biometric segregation of individual persons. In our experiment, the subjects observe face images while their eye movements are being monitored, providing information about each participant’s attention spots. The implemented method treats eye trajectories as 2-D distributions of points on the image plane. The efficiency of graph objects in the representation of structural information motivated us on the utilization of a non-parametric multivariate graph-based measure for the comparison of eye movement signals, yielding promising results at the task of identification according to behavioral characteristics of an individual.
Article
Interaction intent prediction and the Midas touch have been a longstanding challenge for eye-tracking researchers and users of gaze-based interaction. Inspired by machine learning approaches in biometric person authentication, we developed and tested an offline framework for task-independent prediction of interaction intents. We describe the principles of the method, the features extracted, normalization methods, and evaluation metrics. We systematically evaluated the proposed approach on an example dataset of gaze-augmented problem-solving sessions. We present results of three normalization methods, different feature sets and fusion of multiple feature types. Our results show that accuracy of up to 76% can be achieved with Area Under Curve around 80%. We discuss the possibility of applying the results for an online system capable of interaction intent prediction.
Article
Tasks with a complex, dynamic visual component require not only the acquisition of conceptual/procedural but also of perceptual/attentional skills. This study examined expertise differences in perceiving and interpreting complex, dynamic visual stimuli on a performance and on a process level, including perceptual and conceptual strategies. Performance, eye movement, and verbal report data were obtained from seven experts and 14 novices. Results show that experts compared to novices attend more to relevant aspects of the stimulus, use more heterogeneous task approaches, and use knowledge-based shortcuts. Implications for instructional design for the acquisition of perceptual skills are discussed.