August 2024
·
10 Reads
NeuroImage
This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.
August 2024
·
10 Reads
NeuroImage
May 2024
·
66 Reads
·
1 Citation
U.S. service members maintain constant situational awareness (SA) due to training and experience operating in dynamic and complex environments. Work examining how military experience impacts SA during visual search of a complex naturalistic environment, is limited. Here, we compare Active Duty service members and Civilians’ physiological behavior during a navigational visual search task in an open-world virtual environment (VE) while cognitive load was manipulated. We measured eye-tracking and electroencephalogram (EEG) outcomes from Active Duty (N = 21) and Civilians (N = 15) while they navigated a desktop VE at a self-regulated pace. Participants searched and counted targets (N = 15) presented among distractors, while cognitive load was manipulated with an auditory Math Task. Results showed Active Duty participants reported significantly greater/closer to the correct number of targets compared to Civilians. Overall, Active Duty participants scanned the VE with faster peak saccade velocities and greater average saccade magnitudes compared to Civilians. Convolutional Neural Network (CNN) response (EEG P-300) was significantly weighted more to initial fixations for the Active Duty group, showing reduced attentional resources on object refixations compared to Civilians. There were no group differences in fixation outcomes or overall CNN response when comparing targets versus distractor objects. When cognitive load was manipulated, only Civilians significantly decreased their average dwell time on each object and the Active Duty group had significantly fewer numbers of correct answers on the Math Task. Overall, the Active Duty group explored the VE with increased scanning speed and distance and reduced cognitive re-processing on objects, employing a different, perhaps expert, visual search strategy indicative of increased SA. The Active Duty group maintained SA in the main visual search task and did not appear to shift focus to the secondary Math Task. Future work could compare how a stress inducing environment impacts these groups’ physiological or cognitive markers and performance for these groups.
January 2024
Squads of the future battlefield will include a mixture of technically savvy humans and artificially intelligent teammates. Contextually aware AI teammates will be essential for war fighter overmatch. To understand how multimodal physiology can impact mixed team performance, we looked at how physiological team properties emerge in a naturalistic and collaborative environment. Here, we examined internal states and team outcomes based on these states within the context of a complex bomb defusal task in a simulated and naturalistic environment. This overarching research integrates eye gaze behavior, neural activity, speech, heart rate variability, and facial expressions to unravel the intricate relationship between individual and team performance. Here we focus on the facial expression data. Using a novel testbed, we aimed to uncover how these physiological processes evolve and interact with human interactions to influence team dynamics and task performance. Compared to traditional highly controlled lab tasks, this novel testbed enables peripheral measurement of multimodal physiology during naturalistic team formation and collaboration. We report differences between an individual task and teaming task in global facial expressivity results and correlations between facial expression synchrony scores and team task performance.
January 2024
·
40 Reads
While human-agent interaction is intended to ease daily and critical burdens on human operators, issues such as trust, lack of transparency, and system performance often negatively impacts the process to yield sub-optimal outcomes. Here, we propose a human-in-the-loop approach, in which users train an AI, as a potential avenue to remedy this complex problem. We use Tetris® as a use case and require participants to provide trial-by-trial inputs to train the AI model. Improvements in trust correlated with increased satisfaction levels during the training process but not final AI performance. Users’ preference for their trained AI, compared to a pre-trained AI, demonstrated increased improvements in trust. Personality and AI literacy did not affect these relationships. Results suggest positive perceptions towards AI systems can be elicited through psychological ownership pathways. We discuss how users’ involvement in constructing the system may influence ownership giving rise to positive human-agent interactions.
October 2023
·
6 Reads
·
1 Citation
October 2023
·
4 Reads
·
1 Citation
August 2023
·
60 Reads
·
3 Citations
Objective. Currently, there exists very few ways to isolate cognitive processes, historically defined via highly controlled laboratory studies, in more ecologically valid contexts. Specifically, it remains unclear as to what extent patterns of neural activity observed under such constraints actually manifest outside the laboratory in a manner that can be used to make accurate inferences about latent states, associated cognitive processes, or proximal behavior. Improving our understanding of when and how specific patterns of neural activity manifest in ecologically valid scenarios would provide validation for laboratory-based approaches that study similar neural phenomena in isolation and meaningful insight into the latent states that occur during complex tasks. Approach. Domain generalization methods, borrowed from the work of the brain-computer interface community, have the potential to capture high-dimensional patterns of neural activity in a way that can be reliably applied across experimental datasets in order to address this specific challenge. We previously used such an approach to decode phasic neural responses associated with visual target discrimination. Here, we extend that work to more tonic phenomena such as internal latent states. We use data from two highly controlled laboratory paradigms to train two separate domain-generalized models. We apply the trained models to an ecologically valid paradigm in which participants performed multiple, concurrent driving-related tasks while perched atop a six-degrees-of-freedom ride-motion simulator. Main Results. Using the pretrained models, we estimate latent state and the associated patterns of neural activity. As the patterns of neural activity become more similar to those patterns observed in the training data, we find changes in behavior and task performance that are consistent with the observations from the original, laboratory-based paradigms. Significance. These results lend ecological validity to the original, highly controlled, experimental designs and provide a methodology for understanding the relationship between neural activity and behavior during complex tasks.
April 2023
·
14 Reads
·
2 Citations
April 2023
·
30 Reads
There exist very few ways to isolate cognitive processes, historically defined via highly controlled laboratory studies, in more ecologically valid contexts. Specifically, it remains unclear as to what extent patterns of neural activity observed under such constraints actually manifest outside the laboratory in a manner that can be used to make an accurate inference about the latent state, associated cognitive process, or proximal behavior of the individual. Improving our understanding of when and how specific patterns of neural activity manifest in ecologically valid scenarios would provide validation for laboratory-based approaches that study similar neural phenomena in isolation and meaningful insight into the latent states that occur during complex tasks. We argue that domain generalization methods from the brain-computer interface community have the potential to address this challenge. We previously used such an approach to decode phasic neural responses associated with visual target discrimination. Here, we extend that work to more tonic phenomena such as internal latent states. We use data from two highly controlled laboratory paradigms to train two separate domain-generalized models. We apply the trained models to an ecologically valid paradigm in which participants performed multiple, concurrent driving-related tasks. Using the pretrained models, we derive estimates of the underlying latent state and associated patterns of neural activity. Importantly, as the patterns of neural activity change along the axis defined by the original training data, we find changes in behavior and task performance consistent with the observations from the original, laboratory paradigms. We argue that these results lend ecological validity to those experimental designs and provide a methodology for understanding the relationship between observed neural activity and behavior during complex tasks.
February 2022
·
121 Reads
·
1 Citation
Transfer learning and meta-learning offer some of the most promising avenues to unlock the scalability of healthcare and consumer technologies driven by biosignal data. This is because current methods cannot generalise well across human subjects' data and handle learning from different heterogeneously collected data sets, thus limiting the scale of training data. On the other side, developments in transfer learning would benefit significantly from a real-world benchmark with immediate practical application. Therefore, we pick electroencephalography (EEG) as an exemplar for what makes biosignal machine learning hard. We design two transfer learning challenges around diagnostics and Brain-Computer-Interfacing (BCI), that have to be solved in the face of low signal-to-noise ratios, major variability among subjects, differences in the data recording sessions and techniques, and even between the specific BCI tasks recorded in the dataset. Task 1 is centred on the field of medical diagnostics, addressing automatic sleep stage annotation across subjects. Task 2 is centred on Brain-Computer Interfacing (BCI), addressing motor imagery decoding across both subjects and data sets. The BEETL competition with its over 30 competing teams and its 3 winning entries brought attention to the potential of deep transfer learning and combinations of set theory and conventional machine learning techniques to overcome the challenges. The results set a new state-of-the-art for the real-world BEETL benchmark.
... Huang et al. [9] developed a novel EEG classification network, the separable EEG network (S-EEGNet), based on the Hilbert -Huang transform (HHT) and the separable convolutional neural network (CNN) with bilinear interpolation. Lawhern et al. [19] proposed EEGNet, a compact convolutional neural network for EEG-based brain computer interfaces (BCIs) that can generalise across different BCI paradigms in the presence of limited data and produce interpretable features. A comparative study of the spectral power density (PSD) obtained from normal, epileptic and alcoholic EEG signals was performed by Faust et al. [6]. ...
November 2016
... Вернадського. Серія: Технічні науки навання військової техніки [11,12]; (ii) виявлення та супровід рухомих об'єктів на базі керованої ракети та безпілотного літального пристрою (Unmanned Aerial Vehicle; UAV) з можливістю подальшого масштабування апаратно-програмної платформи для автономного прийняття рішень в бойових умовах [13,14]; (iii) аналіз ситуаційної обстановки і виявлення у режимі контекстуальних ознак, таких як потенційні загрози з боку противника або можливість для здійснення контрнаступу [15,16]. ...
May 2024
... In this case, the task is to decode the EEG record to determine the state of the P300 analogue (present or not). Neural decoding methods have been applied and studied in contexts ranging from single cell recordings (Quian Quiroga and Panzeri, 2009) to participant-specific EEG recordings (Wang et al., 2009;Sajda et al., 2009;Lee et al., 2022;Aellen et al., 2023) to large scale analysis of EEG from multiple disparate experiments (Solon et al., 2019;Gordon et al., 2023b). It is the latter area of neural decoding that will be exploited in the current work and the methods used here are derived from approaches originally developed by the Brain-Computer Interface (BCI) community. ...
August 2023
... This same work also established that the SNR improvements from the decoder allowed the conclusions of statistically significant greater responses to targets versus nontargets with, approximately, 75% fewer trials than traditional approaches based in EEG scalp space for a given test set. Subsequent work in (Gordon et al., 2023a) then showed that in the presence of jitter with respect to stimulus onset (i.e., jittering of the entire evoked event) the response profile increased in width by an amount linear to the jitter (Fig. 2D). The authors also showed that under such jittered conditions, applied equally to both target and nontarget stimuli, that the decoder still enabled accurate dissociation between target and nontarget responses in an open-loop paradigm. ...
April 2023
... Notably, the results of the CNN pipelines show a more pronounced bimodal distribution compared to the Riemannian pipelines. The observation that deep convolutional networks may be particularly useful in transfer learning settings is in line with the results of a recent BCI decoding competition that was also won by a CNN approach [37]. However, we remark that the methods benchmarked here were not explicitly designed for cross-subject decoding. ...
February 2022
... Here, we present an analysis focused on combined eye tracking and EEG data collected during the visual search and navigation task to evaluate FRPs related to scene processing. Other published papers focus on describing aspects of the eye movements and navigation task behavior (Enders et al., 2021) and pupillometry data (Thurman et al., 2021). ...
December 2021
... Specifically, it has been shown that saccades suppress visual processing of both color and luminance in the early visual cortex, but do not completely halt it [93]. Furthermore, presaccadic visual content can modulate postsaccadic processing, further indicating ongoing visual processing during eye movements [83]. These findings suggest that our visual system partially maintains continuous processing during saccades while preserving perceptual stability and preventing disorientation from constant visual motion. ...
September 2021
Journal of Vision
... Here, we present an analysis focused on combined eye tracking and EEG data collected during the visual search and navigation task to evaluate FRPs related to scene processing. Other published papers focus on describing aspects of the eye movements and navigation task behavior (Enders et al., 2021) and pupillometry data (Thurman et al., 2021). ...
August 2021
... Finally, (McDaniel, et al., 2020) used the same domain generalization approach to decode the tonic latent state during a driving task. In that work, the authors used a publicly available Nap EEG study (Mei, et al., 2018) as the training set and a simulated nighttime driving task as the target domain. ...
October 2020
... The soldier is a figure on which a large amount of data are analyzed, Figure 11, and whose work is also facilitated by ML [53], providing an advantage, in many cases a strategic advantage, over the enemy, in addition to creating in their environment different areas of study. Applications in these areas, such as resilience, prevention, diagnosis, depression, AI or ML can help to save; currently, there are already studies focused in this direction, such as clinical decision support systems to focused, detailed assessments of suicide risk in patients considered as being at high risk [54]. ...
April 2020