ThesisPDF Available

Depicting movement data with animations for embodied and real-time decision-making: a user study with air traffic control displays and real-time movement data

Authors:

Abstract and Figures

How to effectively and efficiently represent the dynamics of spatial phenomena and processes has been a long-standing research question in geographic information science (GIScience). In a digital information age, computer-generated animations that depict movement data have become increasingly popular, as they apparently visualize real-world spatio-temporal movement changes with corresponding changes over time in a moving display. Animation thus seems to be a suitable display method for facilitating the recognition of spatio-temporal movement patterns and the prediction of future spatio-temporal events. However, the manner by which animations are designed may limit the effectiveness and efficiency of visuospatial decision-making. Furthermore, the specific decision-making task or context of use, as well as the viewer’s perceptual, cognitive and affective background might also influence visuospatial decision-making with animations. These factors are not well understood to date. More empirical studies, as well as new methods to evaluate animations, are thus needed. This work proposes a user-centred empirical approach to evaluate animation design characteristics for space-time decision-making with movement data. Two experiments are conducted with the overall aim of answering the following main research question: How should animations of real-time movement data be designed considering the task and/or use contexts, and user characteristics? More specifically, we test the influence of the three main visual analytics (VA) dimensions on viewer spatio-temporal decision-making with animations: (1) the use context and respective task characteristics, (2) the animation display design, and (3) user characteristics. To test each respective dimension, we undertook the following investigations: (1) Using current air traffic control (ATC) scenarios and existing ATC displays we empirically investigated how aircraft movement changes and future aircraft movement patterns can be visualized for effective and efficient decision-making in ATC. (2) We empirically investigated how movement characteristics (i.e., acceleration, heading direction, etc.) can be depicted, and how animation design (i.e., continuous vs. semi-static animations) might influence viewer task performances. (3) We empirically investigated how perceptual, cognitive, and affective characteristics of viewers (i.e., expertise, spatial abilities, stress or motivation) might influence visuospatial decision-making with animations. We approached these questions through novel empirical data triangulation that integrates psychophysical sensing (i.e., electrodermal responses (EDA)), brain activity (i.e., electroencephalography (EEG)), and eye tracking (ET) with standardized questionnaires. The results of the experiments showed that these three factors (i.e., the use context and respective task characteristics, the animation display design, and the user characteristics) indeed influence visuospatial decision-making using animations of aircraft movement data. We found that viewer decision-making was affected by animation design depending on expertise and task type. Unsurprisingly, ATC experts performed typical ATC tasks more accurately compared to novices. However, the task performance of the experts differed between continuous animation and semi-static animation designs depending on the ATC task. Surprisingly, experts responded more accurately with the novel continuous animation designs compared to the semi-static animations that are more familiar to them in critical ATC tasks for predicting future aircraft movements. In apprehension tasks of aircraft movement changes, experts performed in similar ways with both animation designs. Moreover, viewer characteristics, such as spatial abilities and emotional aspects including engagement or motivation, seemed to affect viewer task performances as well. Higher-spatial and more engaged (or more motivated) viewers performed both tasks more effectively than lower-spatial decision makers and less-engaged (or less-motivated) viewers. Overall, our unique empirical results related to the depiction of real-time movement data contribute to GIScience and cartography in two important ways. First, we are beginning to better understand how viewer mental processes, including perception and cognition, as well as their affective states might influence the effectiveness and efficiency of visuospatial decision-making with animations. Second, we are now able to derive empirically validated design guidelines for perceptually salient, affectively engaging, and cognitively inspired animations.
Content may be subject to copyright.
A preview of the PDF is not available

Supplementary resource (1)

... FAA is a commonly used measure for motivation, emotion, and cognitive control e.g., [24,25]. Greater relative left frontal activity is associated with increased memory & attentional performance and more-focused task performance [26]. FAA is the average hemispheric difference in EEG alpha power between the left and right frontal regions of the brain during EEG recording [27][28][29]. ...
... Since EEG power is inversely correlated with the activation, the negative alpha asymmetry scores correspond to greater relative right frontal activation, whereas positive ones indicate greater relative left frontal activity [30][31][32][33]. More activity in the left-frontal hemisphere indicates approach and motivation, whereas greater relative right activation refers to withdrawal and avoidance [26]. ...
Article
Full-text available
The aim of this research is to evaluate the use of ET and EEG for studying the cognitive processes of expert and novice map users and to explore these processes by comparing two types of spatial memory experiments through cognitive load measurements. The first experiment consisted of single trials and participants were instructed to study a map stimulus without any time constraints in order to draw a sketch map afterwards. According to the ET metrics (i.e., average fixation duration and the number of fixations per second), no statistically significant differences emerged between experts and novices. A similar result was also obtained with EEG Frontal Alpha Asymmetry calculations. On the contrary, in terms of alpha power across all electrodes, novices exhibited significantly lower alpha power, indicating a higher cognitive load. In the second experiment, a larger number of stimuli were used to study the effect of task difficulty. The same ET metrics used in the first experiment indicated that the difference between these user groups was not statistically significant. The cognitive load was also extracted using EEG event-related spectral power changes at alpha and theta frequency bands. Preliminary data exploration mostly suggested an increase in theta power and a decrease in alpha power.
... To study the cognitive procedures of individuals during a map learning task, eye tracking and electroencephalogram (EEG) technologies can be combined e.g., [3,6,[16][17][18]. Since eye movements and attentive cognition are linked, it is possible to detect users' cognitive states in situ via eye trackers. ...
Article
Full-text available
The main objective of this research is to explore the cognitive processes of expert and novice map users during the retrieval of map-related information, within varying difficulty levels (i.e., easy, moderate, hard), by using eye tracking and electroencephalogram (EEG). In this context, we present a spatial memory experiment consisting of a large number of stimuli to study the effect of task difficulty on map users' behavior through cognitive load measurements. Next to the reaction time and success rate, we used fixation and saccade related eye tracking metrics (i.e., average fixation duration, the number of fixations per second, saccade amplitude and saccade velocity), and EEG power spectrum (i.e., event-related changes in alpha and theta frequency bands) to identify the cognitive load. While fixation metrics indicated no statistically significant difference between experts and novices, saccade metrics proved the otherwise. EEG power spectral density analysis, on the other side, suggested an increase in theta (i.e., event-related synchronization) and a decrease in alpha (except moderate tasks) (i.e., event-related desynchronization) at all difficulty levels of the task for both experts and novices, which is an indicator of cognitive load. Although no significant difference emerged between two groups, we found a significant difference in their overall performances when the participants were classified as good and relatively bad learners. Triangulating EEG results with the recorded eye tracking data and the qualitative analysis of focus maps indeed provided a detailed insight on the differences of the individuals' cognitive processes during this spatial memory task.
Article
Full-text available
Özet: Bu çalışma, uzman ve uzman olmayan harita kullanıcılarının mekânsal bellek yeteneklerini incelemeye yönelik kullanıcı deneylerinin tasarım süreçlerine odaklanmıştır. Deneylerde, farklı harita kullanıcı gruplarının haritaların içeriğini hatırlamaları istendiğinde geliştirdikleri dikkate bağlı bilişsel stratejilerini açığa çıkarmada kullanılan bilişsel yük; taslak haritalar, göz izleme ve elektroensefalogram (EEG) gibi farklı yöntemlerden yararlanılarak ölçülmüştür. Birinci deneyde kullanıcılardan, basitleştirilmiş bir 2B topografik haritanın ana yollar, yerleşim, su ve yeşil alanlar gibi temel yapı elemanlarını, zaman kısıtlaması olmaksızın, ezberleyerek hatırladıklarını serbest elle çizmeleri istenmiştir. Görev sırasında kullanıcıda açığa çıkan bilişsel yük, çizilen taslak haritaların analizinin yanı sıra, sabitleme ve ilgi alanı (AoI) tabanlı göz izleme ve ortalama alfa gücü ve frontal alfa asimetrisi (FAA) gibi EEG metriklerine bağlı olarak değerlendirilmiştir. Karmaşık yapıda tasarlanan ve farklı zorluk seviyelerini içeren ikinci deneyde, çoktan seçmeli grafikler kullanılmış ve denemelerde zaman kısıtlaması uygulanmıştır. Sonuçların değerlendirilmesi aşamasında, ilk deneyden farklı olarak, bilişsel yük tespiti için EEG güç spektral yoğunluk değişimlerinden yararlanılmış; pariyetal alandaki alfa ve frontal alandaki teta frekanslarının olaya bağlı (de)senkronizasyonları dikkate alınmıştır. Abstract: This study focused on the design processes of user experiments conducted to examine the spatial memory abilities, differences and similarities of expert and novice map users. In the experiments, different map user groups were asked to remember the content of maps with uniform or various levels of detail. The cognitive load of the participants that reveals their cognitive strategies depending on the attention they developed was measured using different methods such as sketch maps, eye tracking and electroencephalogram (EEG). In the first experiment, the users were asked to memorize the main structuring elements of a simplified 2D topographic map such as major roads, settlements, green areas and waterbodies without any time limitation, and in the next step, to draw what they remember with free hand. Next to the digital sketch map evaluation, cognitive load was assessed based on the fixation and AoI-based (area of interest) eye tracking metrics, together with EEG mean alpha power and frontal alpha asymmetry (FAA). The second experiment was designed in a more complex structure and included different difficulty levels and multiple-choice graphical options instead of hand-drawn sketch maps. Different from the first experiment, event-related (de)synchronization of alpha for parietal region and theta for frontal region were considered for evaluating the cognitive load through the calculations of EEG power spectral density changes.
Article
Full-text available
This study focused on the design processes of user experiments conducted to examine the spatial memory abilities, differences and similarities of expert and novice map users. In the experiments, different map user groups were asked to remember the content of maps with uniform or various levels of detail. The cognitive load of the participants that reveals their cognitive strategies depending on the attention they developed was measured using different methods such as sketch maps, eye tracking and electroencephalogram (EEG). In the first experiment, the users were asked to memorize the main structuring elements of a simplified 2D topographic map such as major roads, settlements, green areas and waterbodies without any time limitation, and in the next step, to draw what they remember with free hand. Next to the digital sketch map evaluation, cognitive load was assessed based on the fixation and AoI-based (area of interest) eye tracking metrics, together with EEG mean alpha power and frontal alpha asymmetry (FAA). The second experiment was designed in a more complex structure and included different difficulty levels and multiple-choice graphical options instead of hand-drawn sketch maps. Different from the first experiment, event-related (de)synchronization of alpha for parietal region and theta for frontal region were considered for evaluating the cognitive load through the calculations of EEG power spectral density changes. Keywords: EEG, Eye tracking, Sketch map, Cognitive cartography
Thesis
Full-text available
Understanding how our brain copes with complex visual information is a challenge for both cognitive psychology and cartography. If we, cartographers, pursue to design better and more usable cartographic products, we require building a better knowledge on the spatial abilities and limitations of map users. This thesis aims to contribute to the understanding of the cognitive processes of expert and novice map users in learning, acquiring and remembering information presented via digital 2D static maps. To be able to gain insight into the users’ behaviors while they interact with maps (e.g. attention, cognitive load), eye tracking and electroencephalogram (EEG) are enabled as synchronized data collection methods due to them being non-invasive and capturing direct responses of cognitive activities. This research discovered that expertise is not as influential as we think when it comes to simple spatial memory tasks using maps designed for general audience and high cognitive load is not necessarily linked with low cognitive performance. However, experts and novices develop different strategies to approach the map-learning tasks and this needs to be researched in more depth including other individual characteristics affecting spatial cognition.
Book
The Cambridge Handbook of Applied Perception Research covers core areas of research in perception with an emphasis on its application to real-world environments. Topics include multisensory processing of information, time perception, sustained attention, and signal detection, as well as pedagogical issues surrounding the training of applied perception researchers. In addition to familiar topics, such as perceptual learning, the Handbook focuses on emerging areas of importance, such as human-robot coordination, haptic interfaces, and issues facing societies in the twenty-first century (such as terrorism and threat detection, medical errors, the broader implications of automation). Organized into sections representing major areas of theoretical and practical importance for the application of perception psychology to human performance and the design and operation of human-technology interdependence, it also addresses the challenges to basic research, including the problem of quantifying information, defining cognitive resources, and theoretical advances in the nature of attention and perceptual processes.
Book
Das Buch will zu einem leichten Einstieg in die Thematik der kartographischen Aninmation verhelfen und praktische Hilfe geben. Dafür werden technische und methodische Grundlagen vermittelt, der gegenwärtige Stand der Animationstechnik aufgezeigt und Erfahrungen von Fachleuten anhand einer Reihe von Beipielen aus unterschiedlichen Anwendungsbereichen weitergeben. Darüber hinaus werden Gestaltungsprinzipien kartographischer Animationen aus konzeptioneller und anwendungsbezogener Sicht vorgestellt. In technischer Hinsicht wird auf Herstellungsverfahren, Hard- und Softwareanforderungen und -leistungen sowie Internetanwendungen eingegangen. Das Buch ist in einen allgemein einführenden und einen anwendungs- und bespielbezogenen Teil gegliedert. Die diskutierten Beispiele sind auf CD-ROM dem Buch beigefügt.
Book
This SpringerBrief discusses the characteristics of spatiotemporal movement data, including uncertainty and scale. It investigates three core aspects of Computational Movement Analysis: Conceptual modeling of movement and movement spaces, spatiotemporal analysis methods aiming at a better understanding of movement processes (with a focus on data mining for movement patterns), and using decentralized spatial computing methods in movement analysis. The author presents Computational Movement Analysis as an interdisciplinary umbrella for analyzing movement processes with methods from a range of fields including GIScience, spatiotemporal databases and data mining. Key challenges in Computational Movement Analysis include bridging the semantic gap, privacy issues when movement data involves people, incorporating big and open data, and opportunities for decentralized movement analysis arising from the internet of things. The interdisciplinary concepts of Computational Movement Analysis make this an important book for professionals and students in computer science, geographic information science and its application areas, especially movement ecology and transportation research.