No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Eye tracker data quality: What it is and how to measure it
Abstract
Data quality is essential to the validity of research results and to the quality of gaze interaction. We argue that the lack of standard measures for eye data quality makes several aspects of manufacturing and using eye trackers, as well as researching eye movements and vision, more difficult than necessary. Uncertainty regarding the comparability of research results is a considerable impediment to progress in the field. In this paper, we illustrate why data quality matters and review previous work on how eye data quality has been measured and reported. The goal is to achieve a common understanding of what data quality is and how it can be defined, measured, evaluated, and reported.
... To study eye movement data, data quality must be ensured (Holmqvist et al., 2012). Holmqvist et al. (2011) recommend a maximum average deviation of the measured gaze points from the actual gaze points of 0.5°, while they describe values above 1.0° as "unacceptable". ...
... Holmqvist et al. (2011) recommend a maximum average deviation of the measured gaze points from the actual gaze points of 0.5°, while they describe values above 1.0° as "unacceptable". Data quality is particularly important for eye movement diagnostics within AOIs, as lower precision and accuracy increases the likelihood that fixations will not be assigned to the correct AOI, especially if there is insufficient distance between AOIs (Holmqvist et al., 2012). According to Orquin et al. (2016), values deviating from the recommended accuracy would have to be compensated for by increasing the size of the AOI, i.e., creating a buffer distance to other sections of the stimulus. ...
... Although this sample size corresponds to a quite common size in relevant national and international eye-tracking research, a larger sample would be desirable for additional analyses of task solving processes, such as a multilevel modeling. Therefore, future research projects need to draw from a larger and more representative sample, and ensure high data quality during the surveys, since the generalizability of findings can be significantly impaired by this (Holmqvist et al., 2012). ...
To successfully learn using freely available (and non-curated) Internet resources, university students need to search for, critically evaluate and select online information, and verify sources (defined as Critical Online Reasoning, COR). Recent research indicates substantial deficits in COR skills among higher education students. To support students in learning how to critically use online information for their learning, it is necessary to better understand the strategies and practices that might elicit less critically-reflective judgments about online information and thus account for such deficits. To this end, using eye tracking data, we investigate how the COR behaviors of students who critically-reflectively evaluate the credibility of online information (‘high performers’) differ from those of students who do not critically-reflectively evaluate it (‘low performers’): 19 students were divided into high and low performers according to their performance in the newly developed Critical Online Reasoning Assessment (CORA). The fixation and dwell times of both groups during CORA task processing were compared regarding time spent on the different processing steps and eye movements on the visited web pages. The results show noticeable differences between the two groups, indicating that low performers indeed approached the task rather heuristically than systematically, and that COR skills require targeted and effective training in higher education.
... Several algorithms have been developed, each with its strengths and limitations. Common methods include the Velocity-Threshold Identification (I-VT), which classifies eye movements based on their speed; the Dispersion-Threshold Identification (I-DT), which uses the spatial dispersion of gaze points, and more sophisticated models like the Hidden Markov Model Identification (I-HMM) and the Kalman Filter Identification (I-KF), which incorporate probabilistic models of eye movement behavior [17,18]. The choice of algorithm can significantly affect the interpretation of eye-tracking data, as different methods may yield varying results for the same raw data. ...
... The I-DT algorithm addresses some of these limitations by incorporating spatial dispersion criteria, which can better distinguish between fixations and saccades in noisier datasets. However, it may miss shorter fixations due to its reliance on spatial thresholds [18]. More advanced algorithms like I-HMM and I-KF provide sophisticated modeling of eye movement dynamics, offering greater accuracy in fixation and saccade classification, but their computational demands can limit their real-time applicability. ...
... These data show that the participants were not looking off-screen; rather, the increase in fixation on the central cross, discussed in later sections, indicates a resolution of sensor-related issues, aligning with participant instructions. This demonstrates that the observed improvements in gaze accuracy are attributable to the filtering process, which effectively addressed sensor errors, thus reflecting true participant behavior [15,18,29]. ...
Determining visual attention during cognitive tasks using activation MRI remains challenging. This study aimed to develop a new eye-tracking (ET) post-processing platform to enhance data accuracy, validate the feasibility of subsequent ET-fMRI applications, and provide tool support. Sixteen volunteers aged 18 to 20 were exposed to a visual temporal paradigm with changing images of objects and faces in various locations while their eye movements were recorded using an MRI-compatible ET system. The results indicate that the accuracy of the data significantly improved after post-processing. Participants generally maintained their visual attention on the screen, with mean gaze positions ranging from 89.1% to 99.9%. In cognitive tasks, the gaze positions showed adherence to instructions, with means ranging from 46.2% to 50%. Temporal consistency assessments indicated prolonged visual tasks can lead to decreased attention during certain tasks. The proposed methodology effectively identified and quantified visual artifacts and losses, providing a precise measure of visual attention. This study offers a robust framework for future work integrating filtered eye-tracking data with fMRI analyses, supporting cognitive neuroscience research.
... The field of reading research (of static, unfamiliar texts) often uses high-speed eye trackers in a setup where a chin and/or a forehead rest is used. This type of setup offers high spatial accuracy and precision as well as high temporal resolution in the gaze data (see Holmqvist et al., 2012). However, as it is not possible to move the head and body during data collection with these systems, they are not well suited for writing research. ...
... However, as it is not possible to move the head and body during data collection with these systems, they are not well suited for writing research. More suitable alternatives are remote systems, which film the eyes from a distance -typically from a camera below the computer display -allowing writers to move their head and body to some extent without sacrificing too much spatial precision and accuracy in the gaze data (see Holmqvist et al., 2012). Since not all remote systems on the market offer sufficient spatial and temporal resolution for writing research, it will be important to carefully evaluate the affordances and limitations of specific eye-tracking systems. ...
... Additionally, mobile head-mounted systems, such as eye-tracking glasses, allow for full mobility of the head and body (see for example Hacker et al., 2017). These systems record a scene video, filming in the line of the writers' sight, onto which the gaze data is later "superimposed" (Holmqvist et al., 2012). The reference frame is therefore in constant movement and gaze data cannot automatically be associated with information within a frame, such as a particular word in a specific location. ...
This volume brings together the perspectives of new and established scholars who have connected with the broad fields of first language (L1) and second language (L2) writing to discuss critically key methodological developments and challenges in the study of L2 writing processes. The focus is on studies of composing and of engagement with feedback on written drafts, with particular attention to methods of process-tracing through data such as concurrent or stimulated verbal reports, interviews, diaries, digital recording, visual screen capture, eye tracking, keystroke logging, questionnaires, and/or ethnographic observation. The chapters in the book illustrate how progress has been made in developing research methods and empirical understandings of writing processes, in introducing methodological innovations, and in pointing to future methodological directions. It will be an essential methodological guide for novice and experienced researchers, senior students, and educators investigating the processes of writing in additional languages.
... This may not be achievable in an actual experimental study. The quality of the eye-tracking signal is influenced by various factors, including the hardware manufacture, deployed platform, and the participants and experimental environment [8,9]. Only a few studies have evaluated the accuracy and usability of VR eye-tracking headsets [14,15], which has revealed discrepancies between official values and their actual performance in specific research use cases. ...
... As the primary measures of eye-tracking data quality, we computed the spatial accuracy and spatial precision based on the definition from Holmqvist et al. [7,8]. ...
... Additionally, the eye-tracking quality of Quest Pro was generally better when targets were positioned at the centre of the field of view given all the targets lie at the same distance. This pattern aligns with the decreasing gaze accuracy for peripheral eye-tracking [8,14,15] for screen-based and HMD devices. ...
... Consequently, there is a strong focus on calibration and validation procedures that measure errors of the system ( Figure 5). In the eye-tracking literature, 'precision' refers specifically to trial-level precision (Glossary in Appendix; Holmqvist et al., 2012) of the time series signal during fixations. Another important index of data quality is the percentage of tracking loss, indicating the robustness of eye-tracking across the temporal domain (Holmqvist et al., 2023). ...
... For instance, two classes of event-detection algorithms are available (Salvucci and Goldberg, 2000) to separate periods of relatively stable eye gaze (i.e., fixations) from abrupt changes in gaze position (i.e., saccades): Velocity-based algorithms have a higher precision and accuracy, but require higher sampling rates (>100 Hz). For lower sampling rates, dispersion-based procedures are recommended (Holmqvist et al., 2012). When relying on manufacturers' software packages, the implemented algorithm and its thresholds are usually not accessible. ...
... Finally, different metrics can be derived from the segmented gaze position data that usually rely on associating gaze shifts or positions to ROIs. A plethora of metrics are used in the literature (Holmqvist et al., 2012) but in general, they describe gaze data in terms of movement (e.g., saccadic direction or amplitude), spatio-temporal distribution (e.g., total dwell time on an ROI), numerosity (e.g., number of initial or recurrent fixations on an ROI), and latency (e.g., latency of first fixation on an ROI). In general, precision is presumably increased for highly aggregated metrics (e.g., dwell time during long periods of exploration) as compared to isolated features (e.g., latency of the first fixation). ...
Human neuroscience has always been pushing the boundary of what is measurable. During the last decade, concerns about statistical power and replicability - in science in general, but also specifically in human neuroscience - have fueled an extensive debate. One important insight from this discourse is the need for larger samples, which naturally increases statistical power. An alternative is to increase the precision of measurements, which is the focus of this review. This option is often overlooked, even though statistical power benefits from increasing precision as much as from increasing sample size. Nonetheless, precision has always been at the heart of good scientific practice in human neuroscience, with researchers relying on lab traditions or rules of thumb to ensure sufficient precision for their studies. In this review, we encourage a more systematic approach to precision. We start by introducing measurement precision and its importance for well-powered studies in human neuroscience. Then, determinants for precision in a range of neuroscientific methods (MRI, M/EEG, EDA, Eye-Tracking, and Endocrinology) are elaborated. We end by discussing how a more systematic evaluation of precision and the application of respective insights can lead to an increase in reproducibility in human neuroscience.
... Recent research has highlighted the importance of eye tracking in various domains, providing insights into cognitive processes and user behavior. Holmqvist et al. (2021) reviewed eye--tracking measures and their applications in visual cognition. They emphasized the importance of microsaccades, small involuntary eye movements that occur several times per second, in maintaining visual perception and attention. ...
... They emphasized the importance of microsaccades, small involuntary eye movements that occur several times per second, in maintaining visual perception and attention. Their study highlighted how these micro-movements contribute to the stability of visual perception and can be used as indicators of cognitive load and attention allocation (Holmqvist et al., 2021). ...
The main objective of this study is to understand how the target’s position influences eye movement in navigational and informative tasks. The study included 20 university students, 13 females and 7 males, aged between 18 and 44. Participants answered a socio-demographic questionnaire and performed two tasks: navigational and informative. Linear combinations of dependent variables were performed through the MANOVA (2x2) test, which maximized the differences between several conditions of the independent variables. The results indicate that the target’s position in the navigational task significantly influences the eye movement of university students. Additionally, the individual’s position influences eye movement in both navigational and informative tasks. It is concluded that both the target’s position and the individual’s position determine eye movement factors during navigational and informative tasks, highlighting the importance of considering these variables in studies of visual perception and human-computer interaction.
... Recent research has highlighted the importance of eye tracking in various domains, providing insights into cognitive processes and user behavior. Holmqvist et al. (2021) reviewed eye--tracking measures and their applications in visual cognition. They emphasized the importance of microsaccades, small involuntary eye movements that occur several times per second, in maintaining visual perception and attention. ...
... They emphasized the importance of microsaccades, small involuntary eye movements that occur several times per second, in maintaining visual perception and attention. Their study highlighted how these micro-movements contribute to the stability of visual perception and can be used as indicators of cognitive load and attention allocation (Holmqvist et al., 2021). ...
This paper is a conceptual and interpretative update of a previously published version. The main objective of this study was to understand how target position influences eye movements in navigational and informative tasks. The sample comprised 20 university students (13 females 7 males, aged 18-44). Participants completed a socio-demographic questionnaire and performed two tasks: navigational and informative. Eye movements were recorded during task performance. A 2x2 MANOVA was conducted to analyze linear combinations of dependent variables (blink duration, blink frequency, and fixation duration) across task types and target positions. Results revealed significant differences in eye movement patterns between tasks. The navigational task showed shorter average blink durations (204.236-1656.397 ms) and fewer blinks (1.987-9.786) compared to the informative task (553.598-1864.440 ms; 9.648-20.040 blinks, respectively). Strong interaction effects were observed between average fixation duration and individual position in both navigational (ηp2 = .216) and informative (ηp2 = .176) tasks. We conclude that target position in the navigational task significantly influences university students’ eye movements, while individual position affects eye movements in both navigational and informative tasks. These findings contribute to understanding how task demands modulate visual attention and potentially affect user interface design and educational technology.
... As in previous studies that examined the accuracy of eye-tracking devices or algorithms (Hessels et al., 2015;Holmqvist et al., 2012;Huang et al., 2024;, a fixation task in which the volunteer looked at visual targets at pre-specified locations was tested. As noted above, two different testing pipelines were used in the present study, leveraging the accessibility features of AVP (the accessibility pipeline) and Unity PolySpatial Input (the unity pipeline). ...
... For convenience, we will refer to the time window in which the visual target appeared at a given position as an "epoch". For each epoch, the gaze coordinates from three continuous frames (~ 100 ms) were selected to estimate the eye-tracking accuracy (Huang et al., 2024; for an in-depth discussion on sample selection and eye-tracking data accuracy, see Holmqvist et al., 2012). With an n-sample recording T, the samples for estimating tracking accuracy are selected using a sliding window with a stride of 1 and a window size of k (k = 3, in the present study). ...
With built-in eye-tracking cameras, the recently released Apple Vision Pro (AVP) mixed reality (MR) headset features gaze-based interaction, eye image rendering on external screens, and iris recognition for device unlocking. One of the technological advancements of the AVP is its heavy reliance on gaze- and gesture-based interaction. However, limited information is available regarding the technological specifications of the eye-tracking capability of the AVP, and raw gaze data is inaccessible to developers. This study evaluates the eye-tracking accuracy of the AVP with two sets of tests spanning both MR and virtual reality (VR) applications. This study also examines how eye-tracking accuracy relates to user-reported usability. The results revealed an overall eye-tracking accuracy of 1.11° and 0.93° in two testing setups, within a field of view (FOV) of approximately 34° x 18°. The usability and learnability scores of the AVP, measured using the standard System Usability Scale (SUS), were 75.24 and 68.26, respectively. Importantly, no statistically reliable correlation was found between eye-tracking accuracy and usability scores. These results suggest that eye-tracking accuracy is critical for gaze-based interaction, but it is not the sole determinant of user experience in VR/AR.
... Following the best practice for ensuring the quality of eye-tracking data [6], validation trials were included before and after each text segment. For this purpose, a dynamic target on a light gray background was presented. ...
... Furthermore, traditional statistical methods are ill-suited for handling high-dimensional data, which are often used to describe human experiences and responses from physiological, psychological, and behavioral perspectives. For example, in smart voice system interactions, various data dimensions, such as user experience quality, comprehension performance, and pupil data, are collected with a 1 kHz eye tracker that gathers 1000 data points per second (Holmqvist et al., 2012). Conventional regression methods are limited by such highdimensional data. ...
Smart voice systems, such as voice assistants and smart speakers, are integral to domains such as smart homes, customer service, healthcare, and smart learning. The effectiveness of these systems relies on user comprehension performance, which is crucial for enhancing user experience. In this study, the primary factors influencing comprehension performance in multilanguage smart voice systems are examined, and the efficacy of various analytical methods, including LASSO regression, SEM, PLS-SEM, CNN, and BiLSTM, are assessed by identifying and improving these factors. Using a diverse dataset from human–computer interaction experiments made publicly available on GitHub, these five methods are applied to discern the impact of environmental and user-specific factors on comprehension. The key findings indicate the following: 1) Noise types and noise sound levels markedly affect comprehension. Noise sound level exhibited an inverted U-shape curve (parameter: 0.088) due to the low and high levels of noise. Certain rhythmic noises, such as those from clocks (parameter: 0.033), enhance comprehension by fostering a conducive auditory environment. 2) Analytical method comparisons reveal that while LASSO regression (MSE = 0.026), SEM, and PLS-SEM effectively map the linear relationships and pathways affecting comprehension, deep learning approaches such as CNN and BiLSTM (MSE = 0.019) excel at handling complex, multidimensional data, offering superior predictive performance.3) In a non-native language environment, the evaluation of user comprehension models is notably different from that in native language settings (native R²: 0.545; non-native R²: 0.347). Specifically, in non-native language environments, the variables and mechanisms influencing user comprehension models are clearer, more controllable, and more susceptible to proficiency levels (parameter: 0.164). This comprehensive study presents a novel comparison of traditional statistical and machine learning methods in analyzing smart voice system interaction across languages. These findings emphasize the significance of tailoring smart voice systems to user diversity in language proficiency, age, and educational background and suggest optimizing these systems under varied environmental conditions to improve comprehension and overall effectiveness. The insights from this study are critical for policymakers and designers aiming to refine the adaptability and user-centric nature of smart voice systems.
... "Fixation" and "saccade" detection depends on the eye-tracking sampling rate, because definitions are often based on the concept of eye-rotation velocity. For the term "saccade", two understandings can be distinguished: physiological -fast rapid eye movement corresponding to an uncorrectable control pattern during execution (Holmqvist et al., 2012;Kruchinina & Yakushev, 2018), and non-physiological, which is equal to "no fixation". In mathematical terms, we should talk about periods of average fast movement and average slow movement. ...
Background
The study of eye-movement strategies of athletes of various disciplines and skill levels is highly significant for sports psychology, since the results can be used in training to improve performance. Such studies are extremely scarce for ice hockey.
Objective
To determine successful eye-movement strategies for ice hockey players compared to wrestlers and controls (non-athletes) during puck-hitting tasks of various degrees of difficulty, using virtual reality.
Design
The study involved 31 participants (male), including 13 ice hockey players (age 20 ± 2.5), 9 wrestlers (age 19 ± 1.9), and 9 controls (age 19 ± 1.3). We used a pre-developed VR-PACE technology that simulates an ice rink in virtual reality (VR). The task was to hit pucks. VR was presented via the HTC Vive Pro Eye with a built-in eye tracker (100 Hz). We analyzed the parameter that reflected the share of puck presence in one of selected retina areas (0–5°, 5–10°, 10–15°, 15–25°, 25–35°) of the left and right eyes and the head.
Results
Ice hockey players exhibited longer puck-tracking using both the near periphery (5–15°) and central retinal area (0–5°). Puck speed had minimal impact on eye-movement strategies, and the visual focus on these areas remained consistent regardless of task type. For both wrestlers and controls, visual fixations in the central retinal area increased when tracking the puck without a motor response, likely leading to higher energy consumption and sensory fatigue.
Conclusion
The optimal eye-movement strategy involves parafoveal tracking in the near periphery (5–15°) and partial foveal tracking (0–5°), allowing for better object information retention and efficient puck trajectory tracking with reduced energy expenditure.
... Note. The eye-movement data quality report (Holmqvist, Nyström, & Mulvey, 2012, Holmqvist et al., 2023: calibration accuracy (°), the proportion of valid data (%), and RMS-S2S precision (°) (SDs in parentheses). Since it is not possible to run a calibration procedure in Tobii Pro Lab with the Tobii 4C, a 6-point calibration was performed for each participant with the Tobii Eye Tracker Manager (v. ...
In research on music conducting, there is a lack of studies concerning conductors’ score-reading. The present investigation explored the reading strategies of four Swedish choral conductors. Two interconnected studies addressed the conductors’ explicit conceptions about score reading and their silent-reading strategies in actual reading situations. All conductors emphasized overviewing and script-like cognitive strategies, read relatively linearly, and tended to combine holistic descriptions with quicker scanning. However, verbal reports also revealed individual cognitive orientations that appeared to influence the conductors’ eye movements. These findings contribute to the understanding of the contrast between shared models of professional practice and individual reading styles.
... The eye tracker having sub-optimal accuracy isn't too much of a concern for other researchers too, as according to Holmqvist et al. (2012), the standardisation of the accuracy or precision of eye-tracking data would likely hold back research in the field. ...
The central research question revolves around measuring the effectiveness of video game onboarding vs full play autonomy. The findings aim to inform game designers, offering practical guidance for creating games that strike a harmonious balance between complexity, simplicity, and intrinsic enjoyment.
Using a custom-built video game as primary research, this paper explores the ideal amount of onboarding a video game should have to avoid high amounts of stress while also remaining enjoyable. The study employed a combination of a pre-test survey, cognitive load assessment during gameplay with eye pupil dilation tracking over time, and post-gameplay feedback analysis to provide insights into the delicate balance required for successful integration of video game onboarding.
Using our three gathered points of data (pre-survey, post-survey and pupil dilation), we used a technique referred to as triangulation to identify a correlation between all of them. By performing this, we were able to build a broad but complete picture to assess how they may be connected.
Key findings include that measuring the rate of change in pupil-diameter is an effective way to measure video game difficulty, having no onboarding creates a better player experience compared to having poor onboarding, and eye-position-tracking is not suitable for data collection in video games due to their nuance and complexity in design, causing difficulties in interpreting generated heat maps compared to other non-video game applications.
... As such, the accuracy of software-only eye-tracking is much lower (closer to 3°) (Krafka et al., 2016;Zhang, Sugano, Fritz, & Bulling, 2015), as is the signal-to-noise ratio (Gómez-Poveda & Gaudioso, 2016). The latter is perhaps even more detrimental, because noisy data can often obscure small eye movements (Holmqvist, Nyström, & Mulvey, 2012;Ko, Snodderly, & Poletti, 2016). Further, the low sampling rate of software-only eye-trackers (15-30 Hz) (Valliappan et al., 2020) makes them unsuitable for capturing rapid eye movements (lasting 20-40 ms). ...
Applications for eye-tracking-particularly in the clinic-are limited by a reliance on dedicated hardware. Here we compare eye-tracking implemented on an Apple iPad Pro 11" (third generation)-using the device's infrared head-tracking and front-facing camera-with a Tobii 4c infrared eye-tracker. We estimated gaze location using both systems while 28 observers performed a variety of tasks. For estimating fixation, gaze position estimates from the iPad were less accurate and precise than the Tobii (mean absolute error of 3.2° ± 2.0° compared with 0.75° ± 0.43°), but fixation stability estimates were correlated across devices (r = 0.44, p < 0.05). For tasks eliciting saccades >1.5°, estimated saccade counts (r = 0.4-0.73, all p < 0.05) were moderately correlated across devices. For tasks eliciting saccades >8° we observed moderate correlations in estimated saccade speed and amplitude (r = 0.4-0.53, all p < 0.05). We did, however, note considerable variation in the vertical component of estimated smooth pursuit speed from the iPad and a catastrophic failure of tracking on the iPad in 5% to 20% of observers (depending on the test). Our findings sound a note of caution to researchers seeking to use iPads for eye-tracking and emphasize the need to properly examine their eye-tracking data to remove artifacts and outliers.
... After the first article in 2008, very few articles on the topic examining NDD using eye-tracking and serious games were published until 2016. This is most likely a result of the emerging popularity of eye-tracking technology, with high-quality data collection being essential when providing connections between gaze data and cognitive metrics [Holmqvist et al., 2012]. ...
... So we need a similar interactive system that uses hardware that is more familiar to many people. In eye movement-based human-computer interaction, three types of eye movements can be used: fixation, saccade, or smooth pursuit [9]. Smooth pursuit is very commonly used in building various interactive applications. ...
Interest in using ebooks by the academic community is very high. Still, there is a problem when readers are reading through screens, tend to read fast, only scan the necessary parts, and don't focus on paying attention to the content they read, so this reduces the quality of reading because readers don't study the overall meaning of the sentence. Hence, this research aims to build an interactive reader system by integrating eye tracker technology with a webcam which is expected to solve the problem of decreasing the quality of reading through the screen by helping readers stay focused on their reading and providing an interactive system that makes it easier for readers to control the computer while reading. This research adopts the waterfall method and is divided into six stages. The system is designed using class diagrams, use case diagrams, and activity diagrams. Also, the system is built using the Python language with the Django framework. Then, the interactive reader system was tested using black box testing and usability testing methods. Based on the test results, it is shown that the interactive reader system that was built can help improve the quality and concentration when reading activities take place.
... A greater fixation count on a specific region or entity could potentially signify its elevated significance or interest to the viewer. In other words, fixation count serves as an invaluable tool to gauge an individual's visual preferences and attentiveness towards various stimuli [31]. Lastly, time to first fixation pertains to the duration required for an individual's eyes to fixate on a particular region or item after the exhibition of a visual impetus [25]. ...
As the marketing landscape continues to evolve, consumer preference remains a key driver of corporate profitability. Extensive research has shown that visual attention is a critical factor in consumer decision-making. However, a comprehensive meta-analysis of online shopping visual presentation has yet to be conducted. This paper applies various eye-tracking dependent variables to investigate consumer visual attention in relation to four common interface design factors: brand, endorser, product, and text. Generally, from the research it shown that product and brand havd positive effect, while text might be negative. It is worthy mention that we identified the subgroup analysis involving total time of fixation (SMD=-0.020, 95%CI: [-0.079,0.039], p=0.507), fixation count (SMD=-0.032, 95%CI: [-0.109,0.045], p=0.421) and time to first fixation (SMD=0.464, 95%CI: [0.346,0.582], p=0.000). In this paper, exposure time obviously impacted FC (Q-value=11.637, p=0.003) and TTFF (Q-value=10.316, p=0.006) in the reanalysis studies. Meanwhile consumer preference highly related to FC (Q=10.953, p=0.001) and TTFF (Q=6.540, p=0.011) were under concern. Studies contained 17 papers with a total of 1071 participants. The publication bias was within the reasonable rang and the heterogeneity mainly resulted in subgroup and moderator differences. Our study on systematic review and meta-analysis show that, to appropriately control the consumer visual attention attributes could be a good solution for increasing consumer preference in online shopping interaction experience. Furthermore, more controllable design factor and moderators related to visual attention should be concerned for neuromarketing progress. In the future, other measurements such as ERPs, FMRI, fINRs could be explored for making better consumer sentiment experience.
... This was used to create a post-hoc X and Y adjustment to better estimate the actual fixation of each participant and to determine an adjustable threshold for saccade detection (described in detail below). These adjustments could also be used as a measure of eye tracker accuracy that would complement other similar measures [16]. This new data structure was then entered into an automated pipeline that initially performs pre-processing (described below) and then conducts task-specific trial-by-trial auto-marking for task performance (Figure 1, bottom row) and behavioral markers (described in Section 2.8). ...
The tremendous increase in the use of video-based eye tracking has made it possible to collect eye tracking data from thousands of participants. The traditional procedures for the manual detection and classification of saccades and for trial categorization (e.g., correct vs. incorrect) are not viable for the large datasets being collected. Additionally, video-based eye trackers allow for the analysis of pupil responses and blink behaviors. Here, we present a detailed description of our pipeline for collecting, storing, and cleaning data, as well as for organizing participant codes, which are fairly lab-specific but nonetheless, are important precursory steps in establishing standardized pipelines. More importantly, we also include descriptions of the automated detection and classification of saccades, blinks, “blincades” (blinks occurring during saccades), and boomerang saccades (two nearly simultaneous saccades in opposite directions where speed-based algorithms fail to split them), This is almost entirely task-agnostic and can be used on a wide variety of data. We additionally describe novel findings regarding post-saccadic oscillations and provide a method to achieve more accurate estimates for saccade end points. Lastly, we describe the automated behavior classification for the interleaved pro/anti-saccade task (IPAST), a task that probes voluntary and inhibitory control. This pipeline was evaluated using data collected from 592 human participants between 5 and 93 years of age, making it robust enough to handle large clinical patient datasets. In summary, this pipeline has been optimized to consistently handle large datasets obtained from diverse study cohorts (i.e., developmental, aging, clinical) and collected across multiple laboratory sites.
... Para ello se propone calcular la repetibilidad y la exactitud del ensayo de 4 dianas mencionado anteriormente, para una muestra reducida de 3 sujetos (N ). La repetibilidad y la exactitud son recursos comúnmente utilizados para la evaluación del error obtenido en sistemas de medición, como se puede apreciar en Holmqvist et al., (2012), con la validación de la calidad de los sistemas de seguimiento visual. La exactitud se define como la cercanía de las medidas obtenidas al valor real, mientras que la repetibilidad es el grado de proximidad entre las mediciones. ...
El factor humano en conducción presenta numerosos desafíos relativos a la seguridad, los cuales podrían ser abordados eficientemente a través de tecnologías relacionadas con la conducción autónoma. En los últimos años ha habido un avance significativo en los sistemas de asistencia al conductor, proporcionando funciones de apoyo y mejorando la seguridad y la comodidad en carretera. En relación con la automatización total, los fabricantes de automóviles han emprendido una carrera tecnológica en busca del vehículo sin conductor invirtiendo recursos significativos en investigación y desarrollo. Sin embargo, existen ciertas barreras que ralentizan la integración de estos vehículos en el parque automovilístico actual.
Uno de los factores más determinantes es la aceptación social, condicionada directamente por la confiabilidad de estos sistemas. A pesar de las múltiples pruebas en entornos cerrados y el aumento de sensores, es difícil abarcar el total de la casuística de accidentes de tráfico que se pueden producir en tráfico real. Muchos de los problemas detectados en el ámbito de la conducción autónoma se relacionan con problemas que un conductor humano podría resolver con relativa sencillez, apuntando a una falta de reglas en el sistema de decisión. En este aspecto, los estudios naturalistas desempeñan un papel fundamental en el desarrollo de algoritmos de toma de decisiones basados en el comportamiento humano, ya que los vehículos carecen de cierta información que los conductores adquieren de forma natural.
Es por ello que el estudio del comportamiento del conductor es crucial para el desarrollo de sistemas que interactúen con vehículos de conducción manual. Comprender los procesos cognitivos seguidos por un conductor y su estado en diversos entornos perfeccionará el diseño de las reglas de decisión ante diferentes maniobras, optimizando la toma de decisiones en conducción autónoma.
El objetivo principal de la tesis es mejorar la caracterización del comportamiento del conductor mediante el análisis de la percepción visual en maniobras complejas realizadas en vías de alta capacidad, como son autovías o autopistas. A lo largo de este estudio, se evalúa la influencia de las variables atencionales del conductor ante diferentes niveles de asistencia a la conducción, observando la repetición de ciertos patrones visuales en función del entorno.
La integración de la información visual del conductor en un modelo de toma de decisiones naturalista permitió una validación exitosa del mismo con ensayos experimentales realizados en tráfico real. Previamente, se realizó una fusión sensorial del sistema de percepción del entorno con el sistema de seguimiento visual, permitiendo la proyección automática de la mirada del conductor en el entorno exterior. Los desarrollos realizados generaron adicionalmente conocimiento destacable en relación con la anticipación de la maniobra de cambio de carril y el hueco aceptable para el desarrollo de modelos de conducción.
Las conclusiones de la Tesis doctoral contribuyen a una mejora de la modelización del comportamiento del conductor y aportarán un enfoque más naturalista al desarrollo de algoritmos de toma de decisiones, con el objetivo de mejorar la integración de la conducción autónoma en el tráfico mixto.
... 131-135). 3 In other words, it has been observed that pupil dilation reflects the cognitive reasoning of the central nervous system in response to a certain external stimulus and that the direction of gaze reflects the activity of the part of the brain responsible for emotional reactions (Ekman, 2004;Ekman & Friesen, 1978;Holmqvist et al., 2012). ...
The Regulation (EU) 2016/679 on the protection of natural persons regarding the processing of personal data (GDPR) is one of the key fundamental pieces of European legislation to protect human rights and freedoms. However, the development of AI systems that are capable of collecting and processing large amounts of data and predicting user habits and emotional states has affected traditional legal categories and tested their resilience. This paper assesses the limits of the current formulation of the GDPR which does not take expressly into account the category of inferred data as a special category of data. Furthermore, it questions whether the toolbox put in place by the GDPR is still effective in protecting data subjects from practices such as neuromarketing and eye-tracking systems. It shows that it is certainly the essential starting point, but that, on the other hand, cannot be spared criticism. For this, in the recent years, the European legislator has adopted further legislations including, in particular, the Digital Services Act (DSA) and the Artificial Intelligence Act (AIA). Although representing a step forward in protection against such technologies, they each have critical aspects that need to be considered.
... First, RMS-S2S precision (Holmqvist et al., 2012;Niehorster et al., 2020a, b) of the CR center signals estimated using the three methods was computed in camera pixels for all the collected gaze data using a moving 200-ms window, after which for each trial the median RMS from all these windows was taken (Niehorster et al., 2020;Hooge et al., 2018Hooge et al., , 2022. The same calculation was performed for the pupil center signal. ...
We present a deep learning method for accurately localizing the center of a single corneal reflection (CR) in an eye image. Unlike previous approaches, we use a convolutional neural network (CNN) that was trained solely using synthetic data. Using only synthetic data has the benefit of completely sidestepping the time-consuming process of manual annotation that is required for supervised training on real eye images. To systematically evaluate the accuracy of our method, we first tested it on images with synthetic CRs placed on different backgrounds and embedded in varying levels of noise. Second, we tested the method on two datasets consisting of high-quality videos captured from real eyes. Our method outperformed state-of-the-art algorithmic methods on real eye images with a 3–41.5% reduction in terms of spatial precision across data sets, and performed on par with state-of-the-art on synthetic images in terms of spatial accuracy. We conclude that our method provides a precise method for CR center localization and provides a solution to the data availability problem, which is one of the important common roadblocks in the development of deep learning models for gaze estimation. Due to the superior CR center localization and ease of application, our method has the potential to improve the accuracy and precision of CR-based eye trackers.
... The middle ring was sized at a radius of 18 pixels, which allows for an inaccuracy of 2.49 • , while the outer ring is sized at a radius of 28 pixels, allowing for an inaccuracy of 3.87 • . Precision was calculated as the median root mean square sample-to-sample deviation for each partici- In regard to data loss, our data showed < 10 % invalid data from the calibrated gaze replay over the total experiment across all ages (4mo and 8mo; see Table 1 and Supplementary Figure 3) and participant groups (EL and LL), with no main effect of age or group, suggesting satisfactory robustness (Holmqvist et al., 2012). We also tested for possible age-and group-specific differences in data loss for younger vs. older infants and for EL vs LL infants. ...
Social communication emerges from dynamic, embodied social interactions during which infants coordinate attention to caregivers and objects. Yet many studies of infant attention are constrained to a laboratory setting, neglecting how attention is nested within social contexts where caregivers dynamically scaffold infant behavior in real time. This study evaluates the feasibility and acceptability of the novel use of head-mounted eye tracking (HMET) in the home with N = 40 infants aged 4 and 8 months who are typically developing and at an elevated genetic liability for autism spectrum disorder (ASD). Results suggest that HMET with young infants with limited independent motor abilities and at an elevated likelihood for atypical development is highly feasible and deemed acceptable by caregivers. Feasibility and acceptability did not differ by age or ASD likelihood. Data quality was also acceptable, albeit with younger infants showing slightly lower accuracy, allowing for preliminary analysis of developmental trends in infant gaze behavior. This study provides new evidence for the feasibility of using in-home HMET with young infants during a critical developmental period when more complex interactions with the environment and social partners are emerging. Future research can apply this technology to illuminate atypical developmental trajectories of embodied social attention in infancy.
... These are ten of the most common eye tracking metrics [18,19]: • Ratio provides information about how many respondents guided their gaze to a specific AOI. This metric shows which areas of an image draw the most or least attention and the areas not attended to. ...
This chapter provides an overview of the following data-gathering methods: online surveys, crowdsourcing, eye tracking, mouse tracking, search logs, triangulation, and social media APIs. This information is essential for anyone interested in understanding the methods and best practices for gathering data in web and social media analytics. With this information, you should better understand each method’s usefulness and be able to make informed decisions about the best method for your business strategy needs.
... In the context of eye movement experiments, data quality refers to the extent to which the collected eye-tracking data accurately and reliably reflect the participants' visual behavior (Holmqvist et al., 2012). To ensure data quality in eye-tracking experiments, researchers typically employ various strategies such as calibration and validation procedures, adherence to standardized guidelines for the setup of the eye-tracker and environment, consistent monitoring during data collection, and preprocessing techniques (e.g., noise reduction, outlier removal; Holmqvist et al., 2011). ...
The many benefits of online research and the recent emergence of open-source eye-tracking libraries have sparked an interest in transferring time-consuming and expensive eye-tracking studies from the lab to the web. In the current study, we validate online webcam-based eye-tracking by conceptually replicating three robust eye-tracking studies (the cascade effect, n = 134, the novelty preference, n = 45, and the visual world paradigm, n = 32) online using the participant’s webcam as eye-tracker with the WebGazer.js library. We successfully replicated all three effects, although the effect sizes of all three studies shrank by 20–27%. The visual world paradigm was conducted both online and in the lab, using the same participants and a standard laboratory eye-tracker. The results showed that replication per se could not fully account for the effect size shrinkage, but that the shrinkage was also due to the use of online webcam-based eye-tracking, which is noisier. In conclusion, we argue that eye-tracking studies with relatively large effects that do not require extremely high precision (e.g., studies with four or fewer large regions of interest) can be done online using the participant’s webcam. We also make recommendations for how the quality of online webcam-based eye-tracking could be improved.
... Subsequently, we stress that overall recording environment comprising setup and geometry, measurement space and monitor size, distance between participant and the eye tracker were all kept as constant as possible, although we could not control for every single possible effect. Future procedures might consider either appropriate corrections of the recorded signal or automatic opt for eliminating such trials in order to preserve data quality that can vastly influence the research results (Holmqvist et al., 2012;Holmqvist et al., 2022). Our results show that despite all potential influences, the TTF parameter was able to discern fit-to-drive group of patients. ...
We present a method to automatically calculate time to fixate (TTF) from the eye-tracker data in subjects with neurological impairment using a driving simulator. TTF presents the time interval for a person to notice the stimulus from its first occurrence. Precisely, we measured the time since the children started to cross the street until the drivers directed their look to the children. From 108 neurological patients recruited for the study, the analysis of TTF was performed in 56 patients to assess fit-, unfit-, and conditionally-fit-to-drive patients. The results showed that the proposed method based on the YOLO (you only look once) object detector is efficient for computing TTFs from the eye-tracker data. We obtained discriminative results for fit-to-drive patients by application of Tukey’s honest significant difference post hoc test ( p < 0.01), while no difference was observed between conditionally-fit and unfit-to-drive groups ( p = 0.542). Moreover, we show that time-to-collision (TTC), initial gaze distance (IGD) from pedestrians, and speed at the hazard onset did not influence the result, while the only significant interaction is among fitness, IGD, and TTC on TTF. Obtained TTFs are also compared with the perception response times (PRT) calculated independently from eye-tracker data and YOLO. Although we reached statistically significant results that speak in favor of possible method application for assessment of fitness to drive, we provide detailed directions for future driving simulation-based evaluation and propose processing workflow to secure reliable TTF calculation and its possible application in for example psychology and neuroscience.
... The higher end of this range could disrupt precise gaze-pointing, but the overall mean is likely sufficient for detecting many attentional and affective states. For context, 0.5° of spatial error is often cited as a good threshold for reliable identification of fixation location [14]. Small eye movements, like tremors, drifts and microsaccades require precisions below 0.03 o [15] so might not be useable metrics from current technologies. ...
This paper examines opportunities and challenges associated with using eye tracking as a sensory system for affective computing in extended reality (XR) environments. Affective computing is a rapidly growing field that aims to develop computing systems capable of recognizing, interpreting, and responding to human emotions. Eye tracking has several potential benefits for improving the detection of emotions, including its ability to unobtrusively monitor affective states in real time, and its sensitivity to a variety of affective states. This paper introduces affective computing, explains eye tracking methodologies, and describes how eye tracking can be used to detect attentional and affective states, realizing concepts such as virtual exposure therapy, and adaptive virtual reality. The paper discusses several potential applications of eye tracking for affective computing in healthcare settings
Accurate eye tracking is crucial for gaze-dependent research, but calibrating eye trackers in subjects who cannot follow instructions, such as human infants and nonhuman primates, presents a challenge. Traditional calibration methods rely on verbal instructions, which are ineffective for these populations. To address this, researchers often use attention-grabbing stimuli in known locations; however, existing software for video-based calibration is often proprietary and inflexible. We introduce an extension to the open-source toolbox Titta—a software package integrating desktop Tobii eye trackers with PsychToolbox experiments—to facilitate custom video-based calibration. This toolbox extension offers a flexible platform for attracting attention, calibrating using flexible point selection, and validating the calibration. The toolbox has been refined through extensive use with chimpanzees, baboons, and macaques, demonstrating its effectiveness across species. Our adaptive calibration and validation procedures provide a standardized method for achieving more accurate gaze tracking, enhancing gaze accuracy across diverse species.
Bottled water has become a popular beverage choice worldwide, with consumers increasingly seeking healthier options. However, label elements can significantly influence consumer perception and purchasing decisions. The research aimed to assess how label elements affect the liking, purchase intention, preference and concept of healthy bottled water. Two stages involved 180 and 100 participants aged between 18 and 40, provided sociodemographic information. The first stage used a hedonic scale and ranking test to perception of nine labels with different elements. The second stage selected a consensus label from prior tests. Four labels were designed, differing in brand color and nutritional information placement. In this last stage, the acceptability, preference ranking and concept of healthy were re-evaluated and eye tracking via the Pupil Lab program. Findings showed varied responses in acceptability and purchase intention among consumers. However, significant differences were observed in preferences and healthiness perceptions based on label characteristics. The label with the highest preference and perceived healthiness featured a sky-blue design with nutritional information on the right side. Combining sensory testing and eye tracking offers valuable insights for designing labels that positively impact consumer perception. The results provide important implications for bottled water manufacturers and marketers in developing effective labeling strategies to meet consumer preferences and promote healthier choices.
Virtual reality (VR) is increasingly used in the study and treatment of paranoia. This is based on the finding that people who mistakenly perceive hostile intent from other people also perceive similar threat from virtual characters. However, there has been no study of the programming characteristics of virtual characters that may influence their interpretation. We set out to investigate how the animation and expressions of virtual humans may affect paranoia. In a two-by-two factor, between-groups, randomized design, 122 individuals with elevated paranoia rated their perceptions of virtual humans, set in an eye-tracking enabled VR lift scenario, that varied in facial animation (static or animated) and expression (neutral or positive). Both facial animation (group difference = 102.328 [51.783, 152.872], p < 0.001, η p 2 = 0.125) and positive expressions (group difference = 53.016 [0.054, 105.979], p = 0.049, η p 2 = 0.033) led to less triggering of paranoid thoughts about the virtual humans. Facial animation (group difference = 2.442 [− 4.161, − 0.724], p = 0.006, η p 2 = 0.063) but not positive expressions (group difference = 0.344 [− 1.429, 2.110], p = 0.681, η p 2 = 0.001) significantly increased the likelihood of neutral thoughts about the characters. Our study shows that the detailed programming of virtual humans can impact the occurrence of paranoid thoughts in VR. The programming of virtual humans needs careful consideration depending on the purpose of their use.
Simulated eye-tracking data are an integral tool in the development of eye-tracking methods. Most of the simulated data used in eye-tracking-related research has been generated using low-complexity eye models that include a single spherical corneal surface. This study investigated the influence of eye model complexity on the ability of simulated eye-tracking data to predict real-world outcomes. The experimental procedures of two pertinent comparative eye-tracking studies were replicated in a simulated environment using various eye model complexities. The simulated outcomes were then evaluated against the findings of the comparative studies that were derived from real-world outcomes. The simulated outcomes of both comparative studies were significantly influenced by the eye model complexity. Eye models that included an aspheric corneal surface best replicated experimental eye-tracking outcomes, while including a posterior corneal surface did not improve the ability of simulated data to replicate real-world outcomes. Using a wide-angle eye model that accurately replicates the peripheral optics of the eye did not improve simulated outcomes relative to a paraxial eye model.
This article presents a noval testing methology and platform of spatial accuracy testing of the eye‐tracking systems. The proposed proprietary eye‐tracking test system designed to faithfully recreate scenarios of human gaze and provide true values for eye gaze directions. We evaluated the spatial accuracy, precision, and standard variance of the aforementioned virtual reality devices under field‐of‐view conditions of 15° and 20° with Product A and Product B. The results indicate that both A and B is capable of high eyetracking spatial accuracy which is less than 3°. The standard deviation values for A and B are small.
Camera‐based Driver Monitoring Systems (DMS) have the potential to exploit eye tracking correlates of alcohol intoxication to detect drunk driving. This study investigates how glance, blink, saccade, and fixation metrics are affected by alcohol, and whether possible effects remain stable across three different camera setups, as well as when the driver is out‐of‐the‐loop during level 4 automated driving (Wizard‐of‐Oz setup). Thirty‐five participants drove on a test track first sober and then with increasing intoxication levels reaching a breath alcohol concentration (BrAC) of 1‰. Linear Mixed‐Effects Regression analyses showed that with increasing intoxication levels, eye blinks became longer and slower, glances and fixations became fewer and longer, and more attention was directed to the road area, at the expense of more peripheral areas. Fixation and blink metrics were more robust to changes in automation mode, whereas glance‐based metrics were highly context dependent. Not all effects of alcohol intoxication could be measured with all eye tracking setups, where one‐camera systems showed lower data availability and higher noise levels compared to a five‐camera system. This means that lab findings based on higher quality eye tracking data might not be directly applied to production settings because of hardware limitations.
This study examines vehicle interiors in terms of display configuration and seat orientation from a user experience viewpoint using a driving simulator. Sixteen volunteers were sat in the driver’s seat to evaluate visibility and mental comfort scores of three display configurations used in the vehicle (i.e., floating, flush, and large display). Another sixteen volunteers were sat in the passenger’s seat to evaluate mental and physical comfort scores of three seat orientations (i.e., forward-facing, 15° inboard, and rear-facing seats). The display configurations were evaluated in the movie-watching, the driving-monitoring, and the control takeover situations, while the seat orientations were evaluated in the movie-watching, the conversation, and the driving-monitoring situations. The large display enhanced for movie-watching. However, it was found to be unsuitable for driving-monitoring. The rear-facing and 15° inboard seats were more suited to the conversation situation from the physical comfort viewpoint. The rear-facing seat was found to be unsuitable from the mental comfort viewpoint in the driving-monitoring situation. The effect on drivers and passengers was different depending on the vehicle interiors and the situations. A thoughtful selection of display configuration and seat orientation, considering the context, is vital to enhance driver and passenger comfort. These findings could aid future user-centric vehicle development.
This paper contributes GazeSwitch, an ML-based technique that optimises the real-time switching between eye and head modes for fast and precise hands-free pointing. GazeSwitch reduces false positives from natural head movements and efficiently detects head gestures for input, resulting in an effective hands-free and adaptive technique for interaction. We conducted two user studies to evaluate its performance and user experience. Comparative analyses with baseline switching techniques, Eye+Head Pinpointing (manual) and BimodalGaze (threshold-based) revealed several trade-offs. We found that GazeSwitch provides a natural and effortless experience but trades off control and stability compared to manual mode switching, and requires less head movement compared to BimodalGaze. This work demonstrates the effectiveness of machine learning approach to learn and adapt to patterns in head movement, allowing us to better leverage the synergistic relation between eye and head input modalities for interaction in mixed and extended reality.
Head-mounted Augmented Reality (AR) displays overlay digital information on physical objects. Through eye tracking, they provide insights into user attention, intentions, and activities, and allow novel interaction methods based on this information. However, in physical environments, the implications of using gaze-enabled AR for human activity recognition have not been explored in detail. In an experimental study with the Microsoft HoloLens 2, we collected gaze data from 20 users while they performed three activities: Reading a text, Inspecting a device, and Searching for an object. We trained machine learning models (SVM, Random Forest, Extremely Randomized Trees) with extracted features and achieved up to 89.6% activity-recognition accuracy. Based on the recognized activity, our system—GEAR—then provides users with relevant AR feedback. Due to the sensitivity of the personal (gaze) data GEAR collects, the system further incorporates a novel solution based on the Solid specification for giving users fine-grained control over the sharing of their data. The provided code and anonymized datasets may be used to reproduce and extend our findings, and as teaching material.
Most commercially available eye-tracking devices rely on video cameras and image processing algorithms to track gaze. Despite this, emerging technologies are entering the field, making high-speed, camera-less eye-tracking more accessible. In this study, a series of tests were conducted to compare the data quality of MEMS-based eye-tracking glasses (AdHawk MindLink) with three widely used camera-based eye-tracking devices (EyeLink Portable Duo, Tobii Pro Glasses 2, and SMI Eye Tracking Glasses 2). The data quality measures assessed in these tests included accuracy, precision, data loss, and system latency. The results suggest that, overall, the data quality of the eye-tracking glasses was lower compared to that of a desktop EyeLink Portable Duo eye-tracker. Among the eye-tracking glasses, the accuracy and precision of the MindLink eye-tracking glasses were either higher or on par with those of Tobii Pro Glasses 2 and SMI Eye Tracking Glasses 2. The system latency of MindLink was approximately 9 ms, significantly lower than that of camera-based eye-tracking devices found in VR goggles. These results suggest that the MindLink eye-tracking glasses show promise for research applications where high sampling rates and low latency are preferred.
Eye gaze trackers are devices designed to identify an individual’s gaze fixation in relation to a screen or another reference point. These tools are widely applied in usability testing as they provide various metrics for studying how people interact with applications. In the past, these tools were expensive and required a controlled environment, as well as trained personnel for proper operation. Although nowadays, new implementations do not require physical hardware to perform these tests, they often rely on license-based models instead of being open source. The objective of this work is to create a standalone system that enables any user to implement a low-cost eye gaze tracker using web technologies. The goal is to facilitate its use in remote and face-to-face studies in a simple way, requiring only a computer and a webcam. We evaluated the impact of three different calibration techniques on the performance of a regression-based prediction algorithm in eye-tracking. In our experiments, the best result of linear regression was obtained with a circular calibration system that uses 160 points. Furthermore, we integrated the system with a web interface and an API, enabling users to record their usability sessions and analyze fixation points through heatmaps.
Eye movements can reveal valuable insights into various aspects of human mental processes, physical well-being, and actions. Recently, several datasets have been made available that simultaneously record EEG activity and eye movements. This has triggered the development of various methods to predict gaze direction based on brain activity. However, most of these methods lack interpretability, which limits their technology acceptance. In this paper, we leverage a large data set of simultaneously measured Electroencephalography (EEG) and Eye tracking, proposing an interpretable model for gaze estimation from EEG data. More specifically, we present a novel attention-based deep learning framework for EEG signal analysis, which allows the network to focus on the most relevant information in the signal and discard problematic channels. Additionally, we provide a comprehensive evaluation of the presented framework, demonstrating its superiority over current methods in terms of accuracy and robustness. Finally, the study presents visualizations that explain the results of the analysis and highlights the potential of attention mechanism for improving the efficiency and effectiveness of EEG data analysis in a variety of applications.
This paper investigates the eye movement sequences of users visiting web pages repeatedly. We are interested in potential habituation due to repeated exposure. The scanpath theory posits that every person learns an idiosyncratic gaze sequence on first exposure to a stimulus and re-applies it on subsequent exposures. Josephson and Holmes (2002) tested the applicability of this hypothesis to web page revisitation but results were inconclusive. With a recurrent temporal pattern detection technique, we examine additional aspects and expose scanpaths. Results do not suggest direct applicability of the scanpath theory. While repetitive scan patterns occurred and were individually distinctive, their occurrence was variable, there were often several different patterns per person, and patterns were not primarily formed on the first exposure. However, extensive patterning occurred for some participants yet not for others which deserves further study into its determinants.
In an effort toward standardization, this paper evaluates the performance of five eye-movement classification algorithms in terms of their assessment of oculomotor fixation and saccadic behavior. The results indicate that performance of these five commonly used algorithms vary dramatically, even in the case of a simple stimulus-evoked task using a single, common threshold value. The important contributions of this paper are: evaluation and comparison of performance of five algorithms to classify specific oculomotor behavior; introduction and comparison of new standardized scores to provide more reliable classification performance; logic for a reasonable threshold-value selection for any eye-movement classification algorithm based on the standardized scores; and logic for establishing a criterion-based baseline for performance comparison between any eye-movement classification algorithms. Proposed techniques enable efficient and objective clinical applications providing means to assure meaningful automated eye-movement classification.
The process of fixation identification—separating and labeling fixations and saccades in eye-tracking protocols—is an essential part of eye-movement data analysis and can have a dramatic impact on higher-level analyses. However, algorithms for performing fixation identification are often described informally and rarely compared in a meaningful way. In this paper we propose a taxonomy of fixation identification algorithms that classifies algorithms in terms of how they utilize spatial and temporal information in eye-tracking protocols. Using this taxonomy, we describe five algorithms that are representative of different classes in the taxonomy and are based on commonly employed techniques. We then evaluate and compare these algorithms with respect to a number of qualitative characteristics. The results of these comparisons offer interesting implications for the use of the various algorithms in future work.
In this paper we evaluate several of the most popular algorithms for segmenting fixations from saccades by testing these algorithms on the scanning patterns of toddlers. We show that by changing the pa- rameters of these algorithms we change the reported fixation dura- tions in a systematic fashion. However, we also show how choices in analysis can lead to very different interpretations of the same eye-tracking data. Methods for reconciling the disparate results of different algorithms as well as suggestions for the use of fixation identification algorithms in analysis, are presented. CR Categories: J.4 (Computer Applications): Social and Behav- ioral Sciences—Psychology
Cognitive effort is reflected in pupil dilation, but the assessment of pupil size is potentially susceptible to changes in gaze position. This study exemplarily used sentence reading as a stand-in for paradigms that assess pupil size in tasks during which changes in gaze position are unavoidable. The influence of gaze position on pupil size was first investigated by an artificial eye model with a fixed pupil size. Despite its fixed pupil size, the systematic measurements of the artificial eye model revealed substantial gaze-position-dependent changes in the measured pupil size. We evaluated two functions and showed that they can accurately capture and correct the gaze-dependent measurement error of pupil size recorded during a sentence-reading and an effortless z-string-scanning task. Implications for previous studies are discussed, and recommendations for future studies are provided.
In the course of running an eye-tracking experiment, one computer system or subsystem typically presents the stimuli to the participant and records manual responses, and another collects the eye movement data, with little interaction between the two during the course of the experiment. This article demonstrates how the two systems can interact with each other to facilitate a richer set of experimental designs and applications and to produce more accurate eye tracking data. In an eye-tracking study, a participant is periodically instructed to look at specific screen locations, or explicit required fixation locations (RFLs), in order to calibrate the eye tracker to the participant. The design of an experimental procedure will also often produce a number of implicit RFIs--screen locations that the participant must look at within a certain window of time or at a certain moment in order to successfully and correctly accomplish a task, but without explicit instructions to fixate those locations. In these windows of time or at these moments, the disparity between the fixations recorded by the eye tracker and the screen locations corresponding to implicit RFLs can be examined, and the results of the comparison can be used for a variety of purposes. This article shows how the disparity can be used to monitor the deterioration in the accuracy of the eye tracker calibration and to automatically invoke a recalibration procedure when necessary. This article also demonstrates how the disparity will vary across screen regions and participants and how each participant's unique error signature can be used to reduce the systematic error in the eye movement data collected for that participant.
In the previous chapters of the book, you will have seen multiple applications for using (and the benefits of using) a gaze tracker. In this chapter, you will be given more insight into how an eye tracker operates. Not only can this aid in understanding the eye tracker better, it also gives important information about how future applications might improve on current ones, by using more of the information available from the eye tracker: as we shall see, an eye tracker can often provide you with more information than just coordinates on a screen. This chapter gives an overview of the components of an eye tracker and introduces basics of gaze modelling. It helps in understanding the following chapters which each provide some details of how to build an eye tracker. This section has technical content, but it is our hope that also readers not particularly interested in the details of eye and gaze trackers will gain some useful insights.
The ACE Centre user trials have involved over a hundred people who have severe and complex physical, cognitive and visual difficulties. Participatory Design methodology underpinned the approach that was adopted, which involved being led by the requirements of the most complex users. Jayne was one of many users through whom we not only developed more effective ways of using the technology, but also more innovative strategies to support its implementation. In this chapter, we describe the process, and outcome of our participatory design approach, through the cases of Jayne and other users.
We use simulations to investigate the effect of sampling frequency on common dependent variables in eye-tracking. We identify two large groups of measures that behave differently, but consistently. The effect of sampling frequency on these two groups of measures are explored and simulations are performed to estimate how much data are required to overcome the uncertainty of a limited sampling frequency. Both simulated and real data are used to estimate the temporal uncertainty of data produced by low sampling frequencies. The aim is to provide easy-to-use heuristics for researchers using eye-tracking. For example, we show how to compensate the uncertainty of a low sampling frequency with more data and post-experiment adjustments of measures. These findings have implications primarily for researchers using naturalistic setups where sampling frequencies typically are low.
Countless aspects of visual processing are reflected in eye movements and analyzing eye movements during visual stimulation has become the methodology of choice for many researchers in vision science and beyond. For decades, the scleral searchcoil technique has been considered the “gold standard” in terms of precision and signal to noise ratio, at the cost of pronounced setup overhead and a certain degree of invasiveness. On the other hand, camera-based eyetrackers are easy to use and non-invasive, yet, despite the dramatic improvement of the last generation systems, they have been known to be more noisy and less precise. Recently, a significant impact of changes in pupil size on the accuracy of camera-based eyetrackers during fixation has been reported (Wyatt, 2010). We compared the accuracy and the pupil-size effect between a scleral searchcoil-based eyetracker (DNI) and an up-to-date infrared camera-based eyetracker (SR Research Eyelink 1000) by simultaneously recording human eye movements with both techniques. Between pupil-constricted (PC) vs. pupil-relaxed (PR) conditions we find a subject-specific shift in reported gaze position of up to >2 degrees with the camera based eyetracker, while the scleral searchcoil system simultaneously reported steady fixation, confirming that the actual point of fixation did not change during pupil constriction/relaxation. Individual repetitions of 25-point calibration grids show the positional accuracy of the searchcoil system to be unaffected by pupil size (PC 0.52 +−0.1 deg, PR 0.54+−0.08 deg), whereas the camera-based system is much less accurate in the PR condition (PC 0.38 ± 0.12 deg, PR 0.98 ± 0.22 deg) due to increased pupil size variability. We show how these pupil-dependent shifts in recorded gaze position can affect the recorded dynamics of fixations (drift), saccades (reduced accuracy), pursuit (altered trajectory) and ocular following (directional bias), and we evaluate a dual-calibration-based method to compensate the pupil-based shift utilizing recorded pupil size.
Holmqvist, K., Nyström, N., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (Eds.) (2011). Eye tracking: a comprehensive guide to methods and measures, Oxford, UK: Oxford University Press.
Bell System Technical Journal, also pp. 623-656 (October)
This paper's focus is on the challenges associated with collecting eye-tracking data. Despite operator training conducted by the manufacturer, one year of experience with eye-tracking and extensive calibration, the data collection success rate in the current investigation was very low; only six out of sixteen participants (37.5%) were successfully eye-tracked. We discuss possible explanations for this low success rate, and why we do not currently believe that eye-tracking is ready to be employed in usability laboratories.
Recording eye movement data with high quality is often a prerequisite for producing valid and replicable results and for drawing well-founded conclusions about the oculomotor system. Today, many aspects of data quality are often informally discussed among researchers but are very seldom measured, quantified, and reported. Here we systematically investigated how the calibration method, aspects of participants' eye physiologies, the influences of recording time and gaze direction, and the experience of operators affect the quality of data recorded with a common tower-mounted, video-based eyetracker. We quantified accuracy, precision, and the amount of valid data, and found an increase in data quality when the participant indicated that he or she was looking at a calibration target, as compared to leaving this decision to the operator or the eyetracker software. Moreover, our results provide statistical evidence of how factors such as glasses, contact lenses, eye color, eyelashes, and mascara influence data quality. This method and the results provide eye movement researchers with an understanding of what is required to record high-quality data, as well as providing manufacturers with the knowledge to build better eyetrackers.
Two studies are reported exploring the usefulness of eye-tracking techniques in illuminating computer users' behaviour in conducting collaborative tasks while supported by multimedia communications. We describe how the technology was deployed and the data we derived to explore the use of visual cues in computer-supported collaborative problem solving. Participants made modest use of the video-link to their remote collaborator, devoting most of their visual attention to other onscreen resources. Varying the quality of this video-link did not influence its use. Eye-tracking was found to be a viable and useful evaluation technique in CSCW
Event detection is used to classify recorded gaze points into periods of fixation, saccade, smooth pursuit, blink, and noise. Although there is an overall consensus that current algorithms for event detection have serious flaws and that a de facto standard for event detection does not exist, surprisingly little work has been done to remedy this problem. We suggest a new velocity-based algorithm that takes several of the previously known limitations into account. Most important, the new algorithm identifies so-called glissades, a wobbling movement at the end of many saccades, as a separate class of eye movements. Part of the solution involves designing an adaptive velocity threshold that makes the event detection less sensitive to variations in noise level and the algorithm settings-free for the user. We demonstrate the performance of the new algorithm on eye movements recorded during reading and scene perception and compare it with two of the most commonly used algorithms today. Results show that, unlike the currently used algorithms, fixations, saccades, and glissades are robustly identified by the new algorithm. Using this algorithm, we found that glissades occur in about half of the saccades, during both reading and scene perception, and that they have an average duration close to 24 msec. Due to the high prevalence and long durations of glissades, we argue that researchers must actively choose whether to assign the glissades to saccades or fixations; the choice affects dependent variables such as fixation and saccade duration significantly. Current algorithms do not offer this choice, and their assignments of each glissade are largely arbitrary.
The rate of blinking is related to certain mental activities. One common feature of states associated with low blink rates is the presence of concentrated cognitive activity. The purpose of the present study was to determine how blinking is affected by variations in mental load; it was hypothesized that, for a given nonvisual task, blinking would decrease as mental load increased. The first study reported here manipulated memory load by requiring Ss to retain a sequence of 4, 6, or 8 digits. The second study involved mental arithmetic under time pressure; half the trials contained zeros in the sequence of numbers to be summed. In both studies the rate of blinking was low when mental load was high and the rate was high when mental load was low. It is speculated that blinking may disrupt certain cognitive processes and may therefore be inhibited when these processes are active. When mental load is increased, the inhibition of blinking may be an adaptive mechanism which protects vulnerable cognitive processes from interference.
Eye track-ing: A comprehensive guide to methods and measures Cleaning up systematic error in eye-tracking data by using required fixation locations
- K Holmqvist
- M Nystr Om
- R Andersson
- R Dewhurst
- H Jarodzka
- J And Van De Weijer
- A Hornof
- T And Halverson
HOLMQVIST, K., NYSTR OM, M., ANDERSSON, R., DEWHURST, R., JARODZKA, H., AND VAN DE WEIJER, J. 2011. Eye track-ing: A comprehensive guide to methods and measures. Oxford: Oxford University Press. HORNOF, A., AND HALVERSON, T. 2002. Cleaning up systematic error in eye-tracking data by using required fixation locations. Behavior Research Methods, Instruments, & Computers 34, 4, 592–604.
Participants know best–influence of cali-bration method and eye physiology on eye-tracking data quality Eyetracking Methodology -How to Conduct and Evaluate Usability Studies Using Eyetrack-ing Identifying fixa-tions and saccades in eyetracking protocols
- Nystr Om
- M Andersson
- R Holmqvist
- K And Van De Weijer
- J Pernice
- K And Nielsen
- D And Goldberg
NYSTR OM, M., ANDERSSON, R., HOLMQVIST, K., AND VAN DE WEIJER, J. submitted. Participants know best–influence of cali-bration method and eye physiology on eye-tracking data quality. Journal of Neuroscience Methods. PERNICE, K., AND NIELSEN, J. 2009. Eyetracking Methodology -How to Conduct and Evaluate Usability Studies Using Eyetrack-ing. Berkeley, CA: New Riders Press. SADEGHNIA, G. R. 2011. SMI Technical Report on Data Quality Measurement. SensoMotoric Instruments. SALVUCCI, D., AND GOLDBERG, J. H. 2000. Identifying fixa-tions and saccades in eyetracking protocols. In Proceedings of the 2002 Symposium on Eye-Tracking Research & Applications, New York: ACM, 71–78.
Effects of pupil size on recorded gaze position: a live comparison of two eyetracking systems. Talk presented at the 2011 Annual Meeting of the Vision Science Society Accuracy and precision test method for remote eye trackers
- Jan Drewes
- Anna Montagnini
- G S M Johnsson
- J And Matos
JAN DREWES, ANNA MONTAGNINI, G. S. M. 2011. Effects of pupil size on recorded gaze position: a live comparison of two eyetracking systems. Talk presented at the 2011 Annual Meeting of the Vision Science Society. JOHNSSON, J., AND MATOS, R. 2011. Accuracy and precision test method for remote eye trackers. Tobii Technology.
Introduction to Eye and Gaze Trackers. In Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies
- D W Hansen
- A Villanueva
- F Mulvey
- D And Mar-Danbegi
HANSEN, D. W., VILLANUEVA, A., MULVEY, F., AND MAR-DANBEGI, D. 2011. Introduction to Eye and Gaze Trackers. In Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies, P. Majaranta, H. Aoki, M. Donegan, D. W. Hansen, J. P. Hansen, A. Hyrskykari, and K.-J. Räihä, Eds., no. 2010. IGI Global: Medical Information Science Refer-ence, Hershey PA, ch. 19, 288–295.
Looking at looking: The amorphous fixation measure. In Eye Movements and Psy-chological Functions: International Views
- R Karsh
- F W And Breitenbach
KARSH, R., AND BREITENBACH, F. W. 1983. Looking at looking: The amorphous fixation measure. In Eye Movements and Psy-chological Functions: International Views, R. Groner, C. Menz, D. F. Fisher, and R. A. Monty, Eds. Mahwah NJ: Lawrence Erl-baum Associates, 53–64.
ch. Participatory De-sign -The Story of Jayne and Other Complex Cases Systematic influence of gaze position on pupil size measurement: analysis and correction
- S Cotmore
- M And Donegan
- B Gagl
- S Hawelka
- F And Hutzler
COTMORE, S., AND DONEGAN, M. 2011. ch. Participatory De-sign -The Story of Jayne and Other Complex Cases. GAGL, B., HAWELKA, S., AND HUTZLER, F. 2011. Systematic influence of gaze position on pupil size measurement: analysis and correction. Behavior Research Methods, 1–11.
McGraw-Hill Yearbook of Science & Technology
- J Tecce
TECCE, J. 1992. McGraw-Hill Yearbook of Science & Technology. New York: McGraw-Hill, ch. Psychology, physiological
and experimental., 375-377.
Accuracy and precision test method for remote eye trackers
- J And Matos
JOHNSSON, J., AND MATOS, R. 2011. Accuracy and precision
test method for remote eye trackers. Tobii Technology.
EyeLink User Manual 1.3.0. Mississauga Ontario Canada. SR Research
- Sr Research
SR RESEARCH. 2007. EyeLink User Manual 1.3.0. Mississauga,
Ontario, Canada.
Eyetracking Methodology - How to Conduct and Evaluate Usability Studies Using Eyetracking
- K And Nielsen
PERNICE, K., AND NIELSEN, J. 2009. Eyetracking Methodology -
How to Conduct and Evaluate Usability Studies Using Eyetracking. Berkeley, CA: New Riders Press.
Trials and tribulations of using an eye-tracking system, CHI '00 Extended Abstracts on Human Factors in Computing Systems The Hague, The Netherlands
- K Susan
- Marc W Schnipke
- Todd
Accuracy and precision test method for remote eye trackers. Tobii Technology. Johnsson J. and Matos R. 2011. Accuracy and precision test method for remote eye trackers
- J Johnsson
- R Matos
JOHNSSON, J., AND MATOS, R. 2011. Accuracy and precision
test method for remote eye trackers. Tobii Technology.
Eyetracking Methodology -How to Conduct and Evaluate Usability Studies Using Eyetracking
- K Pernice
- J Nielsen
PERNICE, K., AND NIELSEN, J. 2009. Eyetracking Methodology -How to Conduct and Evaluate Usability Studies Using Eyetracking. Berkeley, CA: New Riders Press.
- G R Sadeghnia
SADEGHNIA, G. R. 2011. SMI Technical Report on Data Quality
Measurement. SensoMotoric Instruments.
and van de Weijer J. submitted. Participants know best--influence of calibration method and eye physiology on eye-tracking data quality
- M Nyström
- R Andersson
- K Holmqvist
- J Van De Weijer
New York : McGraw-Hill , ch. Psychology, physiological and experimental
- J Tecce
- Tecce J.