Content uploaded by Sara Mary Rosa Lanini-Maggi
Author content
All content in this area was uploaded by Sara Mary Rosa Lanini-Maggi on Jul 31, 2018
Content may be subject to copyright.
A preview of the PDF is not available
Depicting movement data with animations for embodied and real-time decision-making: a user study with air traffic control displays and real-time movement data
Abstract and Figures
How to effectively and efficiently represent the dynamics of spatial phenomena and processes has been a long-standing research question in geographic information science (GIScience). In a digital information age, computer-generated animations that depict movement data have become increasingly popular, as they apparently visualize real-world spatio-temporal movement changes with corresponding changes over time in a moving display. Animation thus seems to be a suitable display method for facilitating the recognition of spatio-temporal movement patterns and the prediction of future spatio-temporal events.
However, the manner by which animations are designed may limit the effectiveness and efficiency of visuospatial decision-making.
Furthermore, the specific decision-making task or context of use, as well as the viewer’s perceptual, cognitive and affective background might also influence visuospatial decision-making with animations. These factors are not well understood to date. More empirical studies, as well as new methods to evaluate animations, are thus needed.
This work proposes a user-centred empirical approach to evaluate animation design characteristics for space-time decision-making with movement data. Two experiments are conducted with the overall aim of answering the following main research question: How should animations of real-time movement data be designed considering the task and/or use contexts, and user characteristics? More specifically, we test the influence of the three main visual analytics (VA) dimensions on viewer spatio-temporal decision-making with animations: (1) the use context and respective task characteristics, (2) the animation display design, and (3) user characteristics. To test each respective dimension, we undertook the following investigations:
(1) Using current air traffic control (ATC) scenarios and existing ATC displays we empirically investigated how aircraft movement changes and future aircraft movement patterns can be visualized for effective and efficient decision-making in ATC. (2) We empirically investigated how movement characteristics (i.e., acceleration, heading direction, etc.) can be depicted, and how animation design (i.e., continuous vs. semi-static animations) might influence viewer task performances.
(3) We empirically investigated how perceptual, cognitive, and affective characteristics of viewers (i.e., expertise, spatial abilities, stress or motivation) might influence visuospatial decision-making with animations.
We approached these questions through novel empirical data triangulation that integrates psychophysical sensing (i.e., electrodermal responses (EDA)), brain activity (i.e., electroencephalography (EEG)), and eye tracking (ET) with standardized questionnaires.
The results of the experiments showed that these three factors (i.e., the use context and respective task characteristics, the animation display design, and the user characteristics) indeed influence visuospatial decision-making using animations of aircraft movement data. We found that viewer decision-making was affected by animation design depending on expertise and task type. Unsurprisingly, ATC experts performed typical ATC tasks more accurately compared to novices. However, the task performance of the experts differed between continuous animation and semi-static animation designs depending on the ATC task. Surprisingly, experts responded more accurately with the novel continuous animation designs compared to the semi-static animations that are more familiar to them in critical ATC tasks for predicting future aircraft movements. In apprehension tasks of aircraft movement changes, experts performed in similar ways with both animation designs. Moreover, viewer characteristics, such as spatial abilities and emotional aspects including engagement or motivation, seemed to affect viewer task performances as well. Higher-spatial and more engaged (or more motivated) viewers performed both tasks more effectively than lower-spatial decision makers and less-engaged (or less-motivated) viewers.
Overall, our unique empirical results related to the depiction of real-time movement data contribute to GIScience and cartography in two important ways. First, we are beginning to better understand how viewer mental processes, including perception and cognition, as well as their affective states might influence the effectiveness and efficiency of visuospatial decision-making with animations. Second, we are now able to derive empirically validated design guidelines for perceptually salient, affectively engaging, and cognitively inspired animations.
Figures - uploaded by Sara Mary Rosa Lanini-Maggi
Author content
All figure content in this area was uploaded by Sara Mary Rosa Lanini-Maggi
Content may be subject to copyright.
Supplementary resource (1)
... Similarly, Keskin et al. (2020) did not find differences in cognitive load across experts and novices (i.e., based on education background in geo-related fields) or various map designs (i.e., varying in levels of depicted environmental features such as roads, parks, rivers, and lakes) when performing a map recall task. Despite the lack of observed differences in cognitive load among various map design conditions and expertise, these studies (Keskin et al., 2020;Lanini-Maggi, 2017) have made an important methodological contribution to the fields of geography and cartography by incorporating neuroscientific methods of investigating users' cognitive states when using maps (Montello et al., 2018;Schinazi & Thrash, 2018). ...
... Some studies have employed EEG as a neuroscientific method of investigating cognitive load in the geo-domain (i.e., geography, cartography, and GIScience) in recent years (Cheng et al., 2022;Keskin et al., 2020;Lanini-Maggi, 2017;Lobben et al., 2005;Lobben et al., 2014). However, all these studies have been conducted in controlled labbased environments, and only some have investigated a navigation context. ...
... In addition, experts are more efficient at guiding their visual attention to task-relevant features and thus are less distracted by task-irrelevant information (Lanini- Maggi et al., 2021;Ooms et al., 2014). Furthermore, they are trained to endure the cognitive demands of the task, in contrast to non-experts (Lanini- Maggi, 2017). While most empirical navigation research rightly controls for the influence of individual and group differences in spatial ability (Cheng et al., 2022;Credé et al., 2019Credé et al., , 2020Lokka & Çöltekin, 2019), there is a lack of empirical wayfinding studies conducted with expert navigators to investigate the influence of various landmark depiction styles on their visual attention, spatial learning, and cognitive load. ...
Even though they are day-to-day activities, humans find navigation and wayfinding to be cognitively challenging. To facilitate their everyday mobility, humans increasingly rely on ubiquitous mobile maps as navigation aids. However, the over-reliance on and habitual use of omnipresent navigation aids deteriorate humans' short-term ability to learn new information about their surroundings and induces a long-term decline in spatial skills. This deterioration in spatial learning is attributed to the fact that these aids capture users' attention and cause them to enter a passive navigation mode. Another factor that limits spatial learning during map-aided navigation is the lack of salient landmark information on mobile maps.
Prior research has already demonstrated that wayfinders rely on landmarks—geographic features that stand out from their surroundings—to facilitate navigation and build a spatial representation of the environments they traverse. Landmarks serve as anchor points and help wayfinders to visually match the spatial information depicted on the mobile map with the information collected during the active exploration of the environment. Considering the acknowledged significance of landmarks for human wayfinding due to their visibility and saliency, this thesis investigates an open research question: how to graphically communicate landmarks on mobile map aids to cue wayfinders' allocation of attentional resources to these task-relevant environmental features. From a cartographic design perspective, landmarks can be depicted on mobile map aids on a graphical continuum ranging from abstract 2D text labels to realistic 3D buildings with high visual fidelity. Based on the importance of landmarks for human wayfinding and the rich cartographic body of research concerning their depiction on mobile maps, this thesis investigated how various landmark visualization styles affect the navigation process of two user groups (expert and general wayfinders) in different navigation use contexts (emergency and general navigation tasks). Specifically, I conducted two real-world map-aided navigation studies to assess the influence of various landmark visualization styles on wayfinders' navigation performance, spatial learning, allocation of visual attention, and cognitive load.
In Study I, I investigated how depicting landmarks as abstract 2D building footprints or realistic 3D buildings on the mobile map affected expert wayfinders' navigation performance, visual attention, spatial learning, and cognitive load during an emergency navigation task. I asked expert navigators recruited from the Swiss Armed Forces to follow a predefined route using a mobile map depicting landmarks as either abstract 2D building footprints or realistic 3D buildings and to identify the depicted task-relevant landmarks in the environment. I recorded the experts' gaze behavior with a mobile eye-tracer and their cognitive load with EEG during the navigation task, and I captured their incidental spatial learning at the end of the task. The wayfinding experts' exhibited high navigation performance and low cognitive load during the map-aided navigation task regardless of the landmark visualization style. Their gaze behavior revealed that wayfinding experts navigating with realistic 3D landmarks focused more on the visualizations of landmarks on the mobile map than those who navigated with abstract 2D landmarks, while the latter focused more on the depicted route. Furthermore, when the experts focused for longer on the environment and the landmarks, their spatial learning improved regardless of the landmark visualization style. I also found that the spatial learning of experts with self-reported low spatial abilities improved when they navigated with landmarks depicted as realistic 3D buildings.
In Study II, I investigated the influence of abstract and realistic 3D landmark visualization styles on wayfinders sampled from the general population. As in Study I, I investigated wayfinders' navigation performance, visual attention, spatial learning, and cognitive load. In contrast to Study I, the participants in Study II were exposed to both landmark visualization styles in a navigation context that mimics everyday navigation. Furthermore, the participants were informed that their spatial knowledge of the environment would be tested after navigation. As in Study I, the wayfinders in Study II exhibited high navigation performance and low cognitive load regardless of the landmark visualization style. Their visual attention revealed that wayfinders with low spatial abilities and wayfinders familiar with the study area fixated on the environment longer when they navigated with realistic 3D landmarks on the mobile map. Spatial learning improved when wayfinders with low spatial abilities were assisted by realistic 3D landmarks. Also, when wayfinders were assisted by realistic 3D landmarks and paid less attention to the map aid, their spatial learning improved.
Taken together, the present real-world navigation studies provide ecologically valid results on the influence of various landmark visualization styles on wayfinders. In particular, the studies demonstrate how visualization style modulates wayfinders' visual attention and facilitates spatial learning across various user groups and navigation use contexts. Furthermore, the results of both studies highlight the importance of individual differences in spatial abilities as predictors of spatial learning during map-assisted navigation. Based on these findings, the present work provides design recommendations for future mobile maps that go beyond the traditional concept of "one fits all." Indeed, the studies support the cause for landmark depiction that directs individual wayfinders' visual attention to task-relevant landmarks to further enhance spatial learning. This would be especially helpful for users with low spatial skills. In doing so, future mobile maps could dynamically adapt the visualization style of landmarks according to wayfinders' spatial abilities for cued visual attention, thus meeting individuals' spatial learning needs.
... FAA is a commonly used measure for motivation, emotion, and cognitive control e.g., [24,25]. Greater relative left frontal activity is associated with increased memory & attentional performance and more-focused task performance [26]. FAA is the average hemispheric difference in EEG alpha power between the left and right frontal regions of the brain during EEG recording [27][28][29]. ...
... Since EEG power is inversely correlated with the activation, the negative alpha asymmetry scores correspond to greater relative right frontal activation, whereas positive ones indicate greater relative left frontal activity [30][31][32][33]. More activity in the left-frontal hemisphere indicates approach and motivation, whereas greater relative right activation refers to withdrawal and avoidance [26]. ...
The aim of this research is to evaluate the use of ET and EEG for studying the cognitive processes of expert and novice map users and to explore these processes by comparing two types of spatial memory experiments through cognitive load measurements. The first experiment consisted of single trials and participants were instructed to study a map stimulus without any time constraints in order to draw a sketch map afterwards. According to the ET metrics (i.e., average fixation duration and the number of fixations per second), no statistically significant differences emerged between experts and novices. A similar result was also obtained with EEG Frontal Alpha Asymmetry calculations. On the contrary, in terms of alpha power across all electrodes, novices exhibited significantly lower alpha power, indicating a higher cognitive load. In the second experiment, a larger number of stimuli were used to study the effect of task difficulty. The same ET metrics used in the first experiment indicated that the difference between these user groups was not statistically significant. The cognitive load was also extracted using EEG event-related spectral power changes at alpha and theta frequency bands. Preliminary data exploration mostly suggested an increase in theta power and a decrease in alpha power.
... Studies of brain activities on a map are far from visual behavior in the field of cartography. Few studies have explored the location of brain activity and cognitive load in the human brain during map use (Lobben, Olson, and Huang 2005;Lanini-Maggi 2017;Keskin et al. 2020), or during map-aided pedestrian navigation Kapaj et al. 2023), using fMRI or EEG techniques. Note that existing studies exclusively focus on one certain task, such as global search on maps, object identification, map memory, or landmark recognition. ...
... To study the cognitive procedures of individuals during a map learning task, eye tracking and electroencephalogram (EEG) technologies can be combined e.g., [3,6,[16][17][18]. Since eye movements and attentive cognition are linked, it is possible to detect users' cognitive states in situ via eye trackers. ...
The main objective of this research is to explore the cognitive processes of expert and novice map users during the retrieval of map-related information, within varying difficulty levels (i.e., easy, moderate, hard), by using eye tracking and electroencephalogram (EEG). In this context, we present a spatial memory experiment consisting of a large number of stimuli to study the effect of task difficulty on map users' behavior through cognitive load measurements. Next to the reaction time and success rate, we used fixation and saccade related eye tracking metrics (i.e., average fixation duration, the number of fixations per second, saccade amplitude and saccade velocity), and EEG power spectrum (i.e., event-related changes in alpha and theta frequency bands) to identify the cognitive load. While fixation metrics indicated no statistically significant difference between experts and novices, saccade metrics proved the otherwise. EEG power spectral density analysis, on the other side, suggested an increase in theta (i.e., event-related synchronization) and a decrease in alpha (except moderate tasks) (i.e., event-related desynchronization) at all difficulty levels of the task for both experts and novices, which is an indicator of cognitive load. Although no significant difference emerged between two groups, we found a significant difference in their overall performances when the participants were classified as good and relatively bad learners. Triangulating EEG results with the recorded eye tracking data and the qualitative analysis of focus maps indeed provided a detailed insight on the differences of the individuals' cognitive processes during this spatial memory task.
Özet:
Bu çalışma, uzman ve uzman olmayan harita kullanıcılarının mekânsal bellek yeteneklerini incelemeye yönelik kullanıcı deneylerinin tasarım süreçlerine odaklanmıştır. Deneylerde, farklı harita kullanıcı gruplarının haritaların içeriğini hatırlamaları istendiğinde geliştirdikleri dikkate bağlı bilişsel stratejilerini açığa çıkarmada kullanılan bilişsel yük; taslak haritalar, göz izleme ve elektroensefalogram (EEG) gibi farklı yöntemlerden yararlanılarak ölçülmüştür. Birinci deneyde kullanıcılardan, basitleştirilmiş bir 2B topografik haritanın ana yollar, yerleşim, su ve yeşil alanlar gibi temel yapı elemanlarını, zaman kısıtlaması olmaksızın, ezberleyerek hatırladıklarını serbest elle çizmeleri istenmiştir. Görev sırasında kullanıcıda açığa çıkan bilişsel yük, çizilen taslak haritaların analizinin yanı sıra, sabitleme ve ilgi alanı (AoI) tabanlı göz izleme ve ortalama alfa gücü ve frontal alfa asimetrisi (FAA) gibi EEG metriklerine bağlı olarak değerlendirilmiştir. Karmaşık yapıda tasarlanan ve farklı zorluk seviyelerini içeren ikinci deneyde, çoktan seçmeli grafikler kullanılmış ve denemelerde zaman kısıtlaması uygulanmıştır. Sonuçların değerlendirilmesi aşamasında, ilk deneyden farklı olarak, bilişsel yük tespiti için EEG güç spektral yoğunluk değişimlerinden yararlanılmış; pariyetal alandaki alfa ve frontal alandaki teta frekanslarının olaya bağlı (de)senkronizasyonları dikkate alınmıştır.
Abstract:
This study focused on the design processes of user experiments conducted to examine the spatial memory abilities, differences and similarities of expert and novice map users. In the experiments, different map user groups were asked to remember the content of maps with uniform or various levels of detail. The cognitive load of the participants that reveals their cognitive strategies depending on the attention they developed was measured using different methods such as sketch maps, eye tracking and electroencephalogram (EEG). In the first experiment, the users were asked to memorize the main structuring elements of a simplified 2D topographic map such as major roads, settlements, green areas and waterbodies without any time limitation, and in the next step, to draw what they remember with free hand. Next to the digital sketch map evaluation, cognitive load was assessed based on the fixation and AoI-based (area of interest) eye tracking metrics, together with EEG mean alpha power and frontal alpha asymmetry (FAA). The second experiment was designed in a more complex structure and included different difficulty levels and multiple-choice graphical options instead of hand-drawn sketch maps. Different from the first experiment, event-related (de)synchronization of alpha for parietal region and theta for frontal region were considered for evaluating the cognitive load through the calculations of EEG power spectral density changes.
This study focused on the design processes of user experiments conducted to examine the spatial memory abilities, differences and similarities of expert and novice map users. In the experiments, different map user groups were asked to remember the content of maps with uniform or various levels of detail. The cognitive load of the participants that reveals their cognitive strategies depending on the attention they developed was measured using different methods such as sketch maps, eye tracking and electroencephalogram (EEG). In the first experiment, the users were asked to memorize the main structuring elements of a simplified 2D topographic map such as major roads, settlements, green areas and waterbodies without any time limitation, and in the next step, to draw what they remember with free hand. Next to the digital sketch map evaluation, cognitive load was assessed based on the fixation and AoI-based (area of interest) eye tracking metrics, together with EEG mean alpha power and frontal alpha asymmetry (FAA). The second experiment was designed in a more complex structure and included different difficulty levels and multiple-choice graphical options instead of hand-drawn sketch maps. Different from the first experiment, event-related (de)synchronization of alpha for parietal region and theta for frontal region were considered for evaluating the cognitive load through the calculations of EEG power spectral density changes.
Keywords: EEG, Eye tracking, Sketch map, Cognitive cartography
Understanding how our brain copes with complex visual information is a challenge for both cognitive psychology and cartography. If we, cartographers, pursue to design better and more usable cartographic products, we require building a better knowledge on the spatial abilities and limitations of map users. This thesis aims to contribute to the understanding of the cognitive processes of expert and novice map users in learning, acquiring and remembering information presented via digital 2D static maps. To be able to gain insight into the users’ behaviors while they interact with maps (e.g. attention, cognitive load), eye tracking and electroencephalogram (EEG) are enabled as synchronized data collection methods due to them being non-invasive and capturing direct responses of cognitive activities.
This research discovered that expertise is not as influential as we think when it comes to simple spatial memory tasks using maps designed for general audience and high cognitive load is not necessarily linked with low cognitive performance. However, experts and novices develop different strategies to approach the map-learning tasks and this needs to be researched in more depth including other individual characteristics affecting spatial cognition.
Events can be understood in terms of their temporal structure. The authors first draw on several bodies of research to construct an analysis of how people use event structure in perception, understanding, planning, and action. Philosophy provides a grounding for the basic units of events and actions. Perceptual psychology provides an analogy to object perception: Like objects, events belong to categories, and, like objects, events have parts. These relationships generate 2 hierarchical organizations for events: taxonomies and partonomies. Event partonomies have been studied by looking at how people segment activity as it happens. Structured representations of events can relate partonomy to goal relationships and causal structure; such representations have been shown to drive narrative comprehension, memory, and planning. Computational models provide insight into how mental representations might be organized and transformed. These different approaches to event structure converge on an explanation of how multiple sources of information interact in event perception and conception.
The Cambridge Handbook of Applied Perception Research covers core areas of research in perception with an emphasis on its application to real-world environments. Topics include multisensory processing of information, time perception, sustained attention, and signal detection, as well as pedagogical issues surrounding the training of applied perception researchers. In addition to familiar topics, such as perceptual learning, the Handbook focuses on emerging areas of importance, such as human-robot coordination, haptic interfaces, and issues facing societies in the twenty-first century (such as terrorism and threat detection, medical errors, the broader implications of automation). Organized into sections representing major areas of theoretical and practical importance for the application of perception psychology to human performance and the design and operation of human-technology interdependence, it also addresses the challenges to basic research, including the problem of quantifying information, defining cognitive resources, and theoretical advances in the nature of attention and perceptual processes.
Das Buch will zu einem leichten Einstieg in die Thematik der kartographischen Aninmation verhelfen und praktische Hilfe geben. Dafür werden technische und methodische Grundlagen vermittelt, der gegenwärtige Stand der Animationstechnik aufgezeigt und Erfahrungen von Fachleuten anhand einer Reihe von Beipielen aus unterschiedlichen Anwendungsbereichen weitergeben. Darüber hinaus werden Gestaltungsprinzipien kartographischer Animationen aus konzeptioneller und anwendungsbezogener Sicht vorgestellt. In technischer Hinsicht wird auf Herstellungsverfahren, Hard- und Softwareanforderungen und -leistungen sowie Internetanwendungen eingegangen. Das Buch ist in einen allgemein einführenden und einen anwendungs- und bespielbezogenen Teil gegliedert. Die diskutierten Beispiele sind auf CD-ROM dem Buch beigefügt.
This SpringerBrief discusses the characteristics of spatiotemporal movement data, including uncertainty and scale. It investigates three core aspects of Computational Movement Analysis: Conceptual modeling of movement and movement spaces, spatiotemporal analysis methods aiming at a better understanding of movement processes (with a focus on data mining for movement patterns), and using decentralized spatial computing methods in movement analysis. The author presents Computational Movement Analysis as an interdisciplinary umbrella for analyzing movement processes with methods from a range of fields including GIScience, spatiotemporal databases and data mining. Key challenges in Computational Movement Analysis include bridging the semantic gap, privacy issues when movement data involves people, incorporating big and open data, and opportunities for decentralized movement analysis arising from the internet of things. The interdisciplinary concepts of Computational Movement Analysis make this an important book for professionals and students in computer science, geographic information science and its application areas, especially movement ecology and transportation research.