No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The effects of dynamic workload and experience on commercially available EEG cognitive state metrics in a high-fidelity air traffic control environment
Abstract
The current study evaluated the validity of commercially available electroencephalography (EEG) cognitive state metrics of workload and engagement in differentially experienced air traffic control (ATC) students. EEG and pupil diameter recordings were collected from 47 ATC students (27 more experienced and 20 less experienced) during a high-fidelity, variable workload approach-control scenario. Scenario workload was manipulated by increasing the number of aircraft released and the presence of a divided attention task. Results showed that scenario performance significantly degraded with increased aircraft and the presence of the divided attention task. No scenario performance differences were found between experience groups. The EEG engagement metric significantly differed between experience groups, with less experienced controllers exhibiting higher engagement than more experienced controllers. The EEG workload metric and pupil diameter were sensitive to workload manipulations but did not differentiate experience groups. Commercially available EEG cognitive state metrics may be a viable tool for enhancing ATC training.
... Among all the extracted features, the power spectral density (PSD) is the most signif icant and widely used EEG indicator to describe brain unconscious activities during tas execution [49]. Accordingly, we aimed to identify the PSD pattern associated with MW variations for all participants. ...
... Among all the extracted features, the power spectral density (PSD) is the most significant and widely used EEG indicator to describe brain unconscious activities during task execution [49]. Accordingly, we aimed to identify the PSD pattern associated with MWL variations for all participants. ...
... It should be noted that PSD is an independent input into SFFS. The reason is that PSD is the most significant and effective feature [49], and other features are supplements. ...
Elevated mental workload (MWL) experienced by pilots can result in increased reaction times or incorrect actions, potentially compromising flight safety. This study aims to develop a functional system to assist administrators in identifying and detecting pilots’ real-time MWL and evaluate its effectiveness using designed airfield traffic pattern tasks within a realistic flight simulator. The perceived MWL in various situations was assessed and labeled using NASA Task Load Index (NASA-TLX) scores. Physiological features were then extracted using a fast Fourier transformation with 2-s sliding time windows. Feature selection was conducted by comparing the results of the Kruskal-Wallis (K-W) test and Sequential Forward Floating Selection (SFFS). The results proved that the optimal input was all PSD features. Moreover, the study analyzed the effects of electroencephalography (EEG) features from distinct brain regions and PSD changes across different MWL levels to further assess the proposed system’s performance. A 10-fold cross-validation was performed on six classifiers, and the optimal accuracy of 87.57% was attained using a multi-class K-Nearest Neighbor (KNN) classifier for classifying different MWL levels. The findings indicate that the wireless headset-based system is reliable and feasible. Consequently, numerous wireless EEG device-based systems can be developed for application in diverse real-driving scenarios. Additionally, the current system contributes to future research on actual flight conditions.
... These analyses were carried out utilizing the EEGLAB toolbox. Bernhardt et al. (2019) just mentioned in their pre-processing step that the EEG signals were downsampled at 256 Hz and processed online with notch filters as 50, 60, 100, and 120 Hz via low-pass FIR filters while data gathering process. Makransky et al. (2019) analyzed and cleaned up EEG signals per second by ABM's proprietary software for extra muscle acts, slow and fast eye blinks, and movement artifact declinations. ...
... The mean estimates were thus calculated for the theta and alpha bands (decibel power / Hz). Bernhardt et al. (2019) estimated PSD automatically via (Advanced Brain Monitoring; B-Alert Live, 2009) by utilizing FFT on raw EEG signals and computing the sinusoidal component amplitudes, second by second, for specified frequency bins. FFT analyses were conducted with and without applying a Kaiser window for data smoothing after applying a 50 percent overlay between two sequential epochs. ...
... As a result, it was found that post-testing, perceived difficulty, and eye-tracking were relevant to extraneous CL. The behavior of the EEG-derived alpha band and the perceived difficulty were indicative of intrinsic CL. Bernhardt et al. (2019) assessed the validity of commercially available EEG cognitive workload state metrics and engagement in students with differentially experienced air traffic control (ATC). It was indicated that for less experienced students, the engagement was greater than in more experienced students. ...
This research analyzed neuroimaging techniques for measuring cognitive load in multimedia research using a systematic literature review on all related papers published until April 2020. The most striking observation to emerge from the analysis is that electroencephalography, functional magnetic resonance imaging, functional near-infrared spectroscopy, and transcranial doppler ultrasonography have been the most preferred neuroimaging tools utilized in cognitive load in multimedia learning research. Forty articles were reviewed based on the techniques that should be known in the field of neuroimaging to study cognitive load in multimedia learning, the analysis methods for neuroimaging, the results related to cognitive load in multimedia research. The study's findings were evaluated, and many discrepancies in cognitive load research related to multimedia learning were discovered.
... As an ergonomic concept, MWL (also referred to as cognitive workload) is "the dynamic relationships between the resources that are needed to carry out a task and the ability of the operator to adequately supply those resources" [3]. MWL can also be defined as "the amount of cognitive resources being expended at a given point of time" [4]. ...
... A number of studies have provided certain features of MWL variations in the frontal [13][14][15] and parietal [16,17] regions. Moreover, changes in task demand have been presented to yield changes in spectral band frequencies in diverse domains including ergonomics in general [17][18][19] or ATC in particular [3,20,21]. For example, decreased alpha activity was observed with increasing cognitive demands in simulated ATC task by Brookings et al. [20]. ...
... Circadian rhythms are of extreme significance when addressing on-the-job safety and may be utilized to reduce work-related risks [25]. However, evaluation of human operator's cognitive state in real-world environments during the day has been overlooked in the previous literature [3,26,27]. ...
The aim of this study was to investigate the effects of mental workload (MWL) and time of day on cognitive performance and electroencephalographic (EEG) parameters of air traffic controllers. EEG signals recorded while 20 professional air traffic controllers performed cognitive tasks [A-X Continuous Performance Test (AX-CPT) and 3-back working memory task] after they were exposed to two levels of task difficulty (high and low MWL) in the morning and afternoon. Significant decreases in cognitive performance were found when the levels of task difficulty increased in both tasks. The results confirmed the sensitivity of the theta and beta activities to levels of task difficulty in the 3-back task, while they were not affected in the AX-CPT. Theta and beta activities were influenced by time of day in the AX-CPT. The findings provide guidance for application of changes in EEG parameters when MWL level is manipulated during the day that could be implemented in future for the development of real-time monitoring systems to improve aviation safety.
... Twelve studies did not report the gender of the participants. Two studies investigated the differences between expertise of the participants (Bernhardt et al. 2019;Hyun et al. 2006). ...
... Therefore, he suggested that more research is needed before blink rate could be used to measure mental workload. Sao Paulo, Brazil, April 5 -8, 2021 Pupil diameter is correlated to workload Rodriguez et al.(2015) and found to be larger in higher workload levels (Bernhardt et al. 2019;Ahlstrom and Friedman 2006;Martin et al. 2011;Truschzinski 2017). Pupil diameter differed significantly between scenarios Martin et al. (2011);Truschzinski (2017) but not between expertise (Bernhardt et al. 2019). ...
... Sao Paulo, Brazil, April 5 -8, 2021 Pupil diameter is correlated to workload Rodriguez et al.(2015) and found to be larger in higher workload levels (Bernhardt et al. 2019;Ahlstrom and Friedman 2006;Martin et al. 2011;Truschzinski 2017). Pupil diameter differed significantly between scenarios Martin et al. (2011);Truschzinski (2017) but not between expertise (Bernhardt et al. 2019). Pupil diameter was more sensitive to workload from visual rather than EEG workload metric (Bernhardt et al. 2019). ...
Air Traffic Control (ATC) is a complex cognitive task. The complexity may cause the workload of air traffic controllers (ATCs) to increase. Increased workload could degrade operator performance, which further affect safety and functioning of the system. Numerous studies in evaluating mental workload in various tasks used physiological and biochemical measures, including studies in ATCs. The use of physiological measures in those studies provides unique information of the operator's condition. This systematic review summarizes literatures on the measurement of the mental workload in ATCs using physiological and biochemical measures. The conducted systematic search come up with thirty-four studies to include for analysis. Physiological measures are categorized into cardiovascular, ocular, brain, respiration, and skin measures. Biochemical measures consisted of cortisol and salivary immunoglobulin (sIgA). The literature review covers various studies either in simulation environment or real working environment, all with different conditions and task scenarios. Even though this review specifically focuses on mental workload of ATCs, in general, the result of physiological measures still differs between broad studies. The difference might be influenced by the study task loads, difficulty levels, and the participants' characteristics.
... color or symbol coding influence pupil response. Similarly, Bernhardt et al. (2019) suggest that pupil diameter is likely to be sensitive towards increasing visual load. Influences of polarity due to screen illumination on pupil dilation were found by Dobres et al. (2017). ...
... For this reason, we conducted a second experiment focusing on screen polarity and presentation of information in two trials. For instance, Bernhardt et al. (2019) suggest that pupil diameter is likely to be sensitive towards increasing visual load. We hypothesize that screen polarity influences pupil diameter but not the scaled ICA (H03). ...
... Factor levels of information were: none (1), fixation cross (2), low density (3), high density (4), centred circle (5), and moving circle (6). Density was tested with a small number of nine centred dots (3 × 3 rows) versus a high number of dots (9 × 13 rows) spread over the entire screen, according to findings of Bernhardt et al. (2019). Regarding movement, a stationary filled circle was compared to a circle moving with constant speed. ...
Various researchers have proposed pupillometric indicators to assess a person's cognitive strain. However, to distinguish the variation of pupil light response from psychosensory pupil response in experimental field conditions is a challenge. The Index of Cognitive Activity (ICA) addresses this problem by wavelet separation. This research investigates the ICA's sensitivity for multiple level task-evoked cognitive activity and visual influences concerning informational work tasks. Objective and subjective measures assessed cognitive strain of participants (N = 22) during various tasks. In a first experiment, mental arithmetic tasks were used to induce different levels of cognitive activity. In a second experiment, influences of screen polarity and presentation of information were investigated (N = 18). The results indicate that eye metrics are rarely sensitive to slight variations in task difficulty. Moreover, the ICA is likely to be sensitive towards constant screen illumination and shows tendencies regarding changes in displayed information. Possible ramifications for the objective assessment of cognitive strain are discussed.
... Being a participant in numerous and heterogeneous daily tasks, everyone endeavors to improve their efficiency with high engagement [6][7][8]. In highly professional scenarios in particular, such as clinical operations [9], aircraft piloting [10] and aerial work [11], engagement level monitoring is very important and related to life safety [12]. The disability of being engaged in current tasks and dealing with mental workload may lead to severe accidents [13]. ...
... Booth et al. [29] predicted engagement utilizing power spectral density (PSD) features. The PSD features in δ (1-4 Hz), θ (4-8 Hz), α (8)(9)(10)(11)(12)(13), and β (13-30 Hz) were extracted as features to feed into classifiers to predict engagement level. Similarly, 1 Hz bin PSD features were utilized by Li et al. and fed into deep models for engagement assessment [23]. ...
Engagement ability plays a fundamental role in allocating attentional resources and helps us perform daily tasks efficiently. Therefore, it is of great importance to recognize engagement level. Electroencephalography is frequently employed to recognize engagement for its objective and harmless nature. To fully exploit the information contained in EEG signals, an engagement recognition method integrating multi-domain information is proposed. The proposed method extracts frequency information by a filter bank. In order to utilize spatial information, the correlation-based common spatial patterns method is introduced and extended into three versions by replacing different correlation coefficients. In addition, the Hilbert transform helps to obtain both amplitude and phase information. Finally, features in three domains are combined and fed into a support vector machine to realize engagement recognition. The proposed method is experimentally validated on an open dataset composed of 29 subjects. In the comparison with six existing methods, it achieves the best accuracy of 87.74±5.98% in binary engagement recognition with an improvement of 4.03%, which proves its efficiency in the engagement recognition field.
... Relevant techniques include electroencephalography (EEG) (e.g. Mijovi� c et al. 2017;Bernhardt et al. 2019), eye tracking (e.g. Wright, Chen, and Barnes 2018;Shi and Rothrock 2022) and electrocardiography (ECG) (e.g. ...
... Wright, Chen, and Barnes 2018;Shi and Rothrock 2022) and electrocardiography (ECG) (e.g. Bernhardt et al. 2019). However, the complexity of the equipment required, and the high intrusiveness of the sensors have restricted the scalability of the implementation and the generalisability of these techniques in actual safety-critical contexts. ...
In safety-critical automatic systems, safety can be compromised if operators lack engagement. Effective detection of undesirable engagement states can inform the design of interventions for enhancing engagement. However, the existing engagement measurement methods suffer from several limitations which damage their effectiveness in the work environment. A novel engagement evaluation methodology, which adopts Artificial Intelligence (AI) technologies, has been proposed. It was developed using motorway control room operators as subjects. Openpose and Open Source Computer Vision Library (OpenCV) were used to estimate the body postures of operators, then a Support Vector Machine (SVM) was utilized to build the engagement evaluation model based on discrete states of operator engagement. The average accuracy of the evaluation results reached 0.89 and the weighted average precision, recall and f1-score were all above 0.84. This study emphasizes the importance of specific data labelling when measuring typical engagement states, forming the basis for potential control room improvements.
... This makes it an ideal media to induce, study and understand Affective and Cognitive States (ACS). Affective States (AS) refer to states such as emotions, mood, and feelings [132], and Cognitive States (CS) refer to states like cognitive load or mental workload, which influence how information is processed (e.g., reasoning, deliberation, planning) [22]. While there are some distinctions in their definitions and models, AS and CS are interwoven [78]. ...
... They can be manipulated, for instance, by stories, narrative content, and preferences, and they influence decision making. On the other hand, CS derive from cognitive resources, cognitive skills, and influence how information is processed [22]. They can be manipulated, for instance, by the intrinsic difficulty of a task, the number of distractors, and the instructions presentation format. ...
In Virtual Reality (VR), users can be immersed in emotionally intense and cognitively engaging experiences. Yet, despite strong interest from scholars and a large amount of work associating VR and Affective and Cognitive States (ACS), there is a clear lack of structured and systematic form in which this research can be classified. We define "Affective and Cognitive VR" to relate to works which (1) induce ACS, (2) recognize ACS, or (3) exploit ACS by adapting virtual environments based on ACS measures. This survey clarifies the different models of ACS, presents the methods for measuring them with their respective advantages and drawbacks in VR, and showcases Affective and Cognitive VR studies done in an immersive virtual environment (IVE) in a non-clinical context. Our article covers the main research lines in Affective and Cognitive VR. We provide a comprehensive list of references with the analysis of 63 research articles and summarize future works directions.
... Roles in aviation operation and management, such as pilots, air traffic control, and ground crew operation are also frequently confronted with cognitively demanding tasks related to the precise and timely decision-making demands. Such roles require high levels of information processing, working memory, and response selection (Bernhardt et al., 2019). Again, the consequences of impaired performance due to prior cognitive demands are potentially severe, with these positions responsible for the safety of hundreds of lives and high-cost vehicles. ...
... This requires the careful consideration of various factors, such as the reliability, validity, intrusiveness, and sensitivity of each physiological measure. Bernhardt et al. [18] collected EEG and pupil diameter data from controllers in different regulatory scenarios and used statistical methods to verify the sensitivity of the EEG load index and pupil diameter to changes in cognitive load. Radüntz et al. [20] pioneered the dual-frequency head maps (DFHM) approach to assess workload using an instantaneous self-assessment questionnaire; these authors also constructed the DFHM-workload index based on their EEG data and validated the reliability and stability of the index. ...
Due to increased air traffic flow, air traffic controllers (ATCs) operate in a state of high load or even overload for long periods of time, which can seriously affect the reliability and efficiency of controllers’ commands. Thus, the early identification of ATCs who are overworked is crucial to the maintenance of flight safety while increasing overall flight efficiency. This study uses a comprehensive comparison of existing cognitive load assessment methods combined with the characteristics of the ATC as a basis from which a method for the utilization of speech parameters to assess cognitive load is proposed. This method is ultimately selected due to the minimal interference of the collection equipment and the abundance of speech signals. The speech signal is pre-processed to generate a Mel spectrogram, which contains temporal information in addition to energy, tone, and other spatial information. Therefore, a speech cognitive load evaluation model based on a stacked convolutional neural network (CNN) and the Transformer encoder (SCNN-TransE) is proposed. The use of a CNN and the Transformer encoder allows us to extract spatial features and temporal features, respectively, from contextual information from speech data and facilitates the fusion of spatial features and temporal features into spatio-temporal features, which improves our method’s ability to capture the depth features of speech. We conduct experiments on air traffic control communication data, which show that the detection accuracy and F1 score of SCNN-TransE are better than the results from the support-vector machine (SVM), k-nearest neighbors (KNN), random forest (RF), adaptive boosting (AdaBoost), and stacked CNN parallel long short-term memory with attention (SCNN-LSTM-Attention) models, reaching values of 97.48% and 97.07%, respectively. Thus, our proposed model can realize the effective evaluation of cognitive load levels.
... In addition, eye tracking metrics used to study cognitive load (e.g., pupillometry, blink rate) may be better equipped to address any applicability of the resource depletion explanation and/or the recruitment of mental resources due to these metrics' ability to specifically address amounts of mental resources. Similarly, psychophysiological measures of task engagement and/or fatigue could help further detail the effort regulation explanation (e.g., Bernhardt et al., 2019;Naeeri et al., 2019). More generally, different types of workload transitions should be explored with this current analysis approach, especially when considering the prevalence and impact of unexpected and dramatic changes in workload (Endsley, 2017). ...
Given there is no unifying theory or design guidance for workload transitions, this work investigated how visual attention allocation patterns could inform both topics, by understanding if scan-based eye tracking metrics could predict workload transition performance trends in a context-relevant domain. The eye movements of sixty Naval flight students were tracked as workload transitioned at a slow, medium, and fast pace in an unmanned aerial vehicle testbed. Four scan-based metrics were significant predictors across the different growth curve models of response time and accuracy. Stationary gaze entropy (a measure of how dispersed visual attention transitions are across tasks) was predictive across all three transition rates. The other three predictive scan-based metrics captured different aspects of visual attention, including its spread, directness, and duration. The findings specify several missing details in both theory and design guidance, which is unprecedented, and serves as a basis of future workload transition research.
... Although EEG based methods have been used in domains such as air traffic control (e.g. Aricò et al., 2019;Bernhardt et al., 2019;Radüntz et al., 2021), aviation (e.g. Blanco et al., 2018;Hebbar et al., 2021;Lee et al., 2020), construction (Chen et al., , 2017, manufacturing (Argyle et al., 2021), power plant operation (Reinerman-Jones et al., 2016), remote operation (Durantin et al., 2014;Rojas et al., 2020), ship handling and software development (Fritz et al., 2014), field studies have mostly been limited to the field of aviation (Dehais et al., 2019;Noel et al., 2005;Wilson, 2002). ...
Although the objective assessment of mental workload has been a focus of human factors research, few studies have investigated stakeholders' attitudes towards its implementation in real workplaces. The present study addresses this research gap by surveying N = 702 managers in three European countries (Germany, United Kingdom, Spain) about their expectations and concerns regarding sensor-based monitoring of employee mental workload. The data confirm the relevance of expectations regarding improvements of workplace design and employee well-being, as well as concerns about restrictions of employees' privacy and sovereignty, for the implementation of workload monitoring. Furthermore, Bayesian regression models show that the examined expectations have a substantial positive association with managers’ willingness to support workload monitoring in their company. Privacy concerns are identified as a significant barrier to the acceptance of workload monitoring, both in terms of their prevalence among managers and their strong negative relationship with monitoring support.
... As with other comprehension tasks, the goal of understanding static or dynamic images and enable learning is equivalent to an internal cognitive mental model, i.e., an image visualization medium that accurately represents an entity (an object or event depicted by a dynamic or static image). In cognitive processes, the process of selecting, organizing, and integrating representative information from different forms and constructing continuous mental cognitive models often imposes a high cognitive load on the learners [37]. The related cognitive load theory also provides additional concepts for understanding and analyzing cognitive learning using images. ...
This paper discusses the influence of dynamic images and traditional static images on user perception in web interface visualizations of products. First, thirty graduate students in industrial design participated in an eye-tracking experiment, performing visual imagery (VI) tasks of the product images with different presentation formats and durations. The results of eye movement experiment show that the visual cognitive effect was better for the dynamic images than the static image, and the efficiency of visual search was improved. However, the emotional experience of viewing dynamic images was substantially affected by the presentation time. Secondly, there were significant differences in the cognitive level and emotional experience of the users between the dynamic images with different presentation times. The optimal perception experience was observed at a presentation time of 9000 ms, indicating that the subjective responses of the users’ questionnaire survey did not represent the actual cognitive needs of the users. This study provides a scientific basis for product designers to achieve an improved browsing experience of their products.
... Similarly, it was often associated to other states or notions like cognitive load, mental effort, and cognitive performances. We can separate these notions into two categories: Cognitive States (CS), which influence or are consequences of how information is processed (e.g., reasoning, deliberation, planning) [28] such as MW or cognitive performances, and Affective States (AS), which refer to states like emotions, mood, feelings, and personality traits [258]. While there are some distinctions in their definitions and models, CS and AS have various similarities as multicomponent constructs [160,215,258] and in the methods to measure them [204]. ...
Malgré l’émergence rapide des systèmes automatisés et le développement de l’informatique affective, très peu d’études considèrent la charge mentale de travail dans la conception de scénarios de formation en RV. Cette thèse a pour objectif de contribuer au développement des systèmes adaptatifs en RV, basés sur la charge mentale de travail des utilisateurs. Nous proposons 3 axes de recherches : induction, reconnaissance, et exploitation de la charge mentale de travail en RV, ainsi qu'une définition de la « Réalité Virtuelle Affective et Cognitive ». Dans un premier temps, nous étudierons l’impact du port de casque de RV sur l’effort mental des utilisateurs. De plus, l’influence potentielle de la marche et de l’effet d’accommodation en RV seront analysés. Puis, nous proposerons une approche méthodologique pour introduire l’évaluation de la charge mentale de travail dans la conception de scénarios de formation en RV. Cette méthodologie permettra notamment de moduler le niveau de charge mentale de travail des utilisateurs au cours du temps. Des études utilisateurs seront menées dans un simulateur de vol en RV afin d'évaluer cette approche. Finalement, nous proposerons une solution tout-en-un afin d'estimer la charge mentale de travail des utilisateurs en temps-réel en utilisant des capteurs intégrés aux casques de RV. Cette configuration sera comparée aux systèmes plus répandus dans le commerce vis-à-vis des performances de prédiction. Les influence du types de mesures, des capteurs, et des méthodes de normalisation des signaux seront également analysées.
... Due to its properties, the method is expected to support solving such important problems as participants' comparison (Stanton, 2005), reference point absence, different workload definition, tasks similarity (Wickens, 2002), quantitative MWL definition (Wang et al., 2020), measurement continuity (Miller, 2001), metric sensitivity (Bernhardt et al., 2019), the "Multiple resource the-ory" further research (Bommer, 2016), dynamic MWL assessment (Iqbal et al., 2020;Kostenko et al., 2016), and workload redline determination (De Waard, 1996). It also contributes to continuous MWL profiles' construction as an alternative to analytical methods (Rusnock & Borghetti, 2018) and should be tested for such an urgent topic as MWL prediction (Bommer, 2016;Kirwan et al., 1997;Vidulich et al., 1991;Wickens, 2002). ...
Mental workload is a well-known concept with a long development history. It can be used to examine students’ attitudes at the end of the educational process and compare them in groups or separately. However, building a continuous workload profile across the range of task complexity increase is still an urgent issue. All four groups of methods used to define mental workload have such flaws for the workload profile construction process as significant time requirements, single value processing and post-processing of the received results. Only one of them can be used without modifications to construct the operator’s attitude chart (profile) regarding the workload range and it doesn’t operate with more reliable absolute values. To resolve this problem, a special workload assessment grid was developed, considering the advantages of a subjective group of methods and seven core characteristics. The reasoning for grid axes choice, threshold values, and question formulation were provided. Statistics were calculated for the full sample, different grades, and educational institutions. Comparison of the received responses with referential values, cross-comparison between institutions and different grades were performed. The results contribute to such important aspects of workload, as redlines, workload profiling, and operator’s comparison.
... An interesting EEG study conducted under real flight conditions [9] exposed that a reduction of alpha and theta frequencies bands can reveal the neural signature of intentional deafness (for example, inadvertent lack of auditory stimulation). A recent study [4] exploring air traffic controllers has calculated an EEG-based workload index, which can distinguish the task workload requirements of analyzing forebrain functions, however, they did not explore alpha and theta involvement with workload changes. ...
Increasing workload is a central notion in human factors research that can decrease the performance and yield accidents. Thus,
it is crucial to understand the impact of different internal operator’s factors including eye movements, memory and audio-visual
integration. Here, we explored the relationship between cognitive workload (low vs. high) and eye movements (saccades, fixations
and smooth pursuit). The task difficulty was induced by auditory noise, arithmetical count and working memory load. We estimated
cognitive workload using EOG and EEG-based mental state monitoring. One novelty consists in recording the EOG around the
ears (alternative EOG) and around the eyes (conventional EOG). The number of blinks and saccades amplitude increased along
with the difficulty increase (p ≤ 0.05). We found significant correlations between EOG and EEG (theta/alpha ratio) and between
conventional and alternative EOG signal. The increase in cognitive load may disturb the coding and maintenance of related visual
information. Alternative EOG metrics could be a valuable tool for detecting workload.
... This study included high engagement (Berka et al., 2007). These commercially available metrics have been shown to be valuable in both marketing and human performance areas (Bernhardt et al., 2019). ...
Neuroforecasting predicts population-wide choices based on neural data of individuals and can be used for example in neuromarketing to estimate campaign successes. To deliver true value, the brain activity metrics should deliver predictive value above and beyond traditional stated preferences. Evidence from movie trailer research has proposed neural synchrony, which compares the similarity of brain responses across participants and has shown to be a promising tool in neuroforecasting for movie popularity. The music industry might also benefit from these increasingly accurate success predictors, but only one study has been forecasting music popularity, using fMRI measures. Current research validates the strength of neural synchrony as a predictive measure for popularity of music, making use of electroencephalogram (EEG) to capture moment-to-moment neural similarity between respondents while they listen to music. Neural synchrony is demonstrated to be a significant predictor for public appreciation on Spotify three weeks and ten months after the release of the albums, especially when combined with the release of a single. On an individual level, other brain measures were shown to relate to individual subjective likeability ratings, including Frontal Alpha Asymmetry and engagement when combined with the factors artist and single release. Our results show the predictive value of brain activity measures outperforms stated preferences. Especially, neural synchrony carries high predictive value for the popularity on Spotify, providing the music industry with an essential asset for efficient decision making and investments, in addition to other practical implications that include neuromarketing and advertising industries.
... Its purpose is not only to study the structure and function of the brain, which is the domain of neuroscience but also to do so in the context of individual perceptions and behaviors in the workplaces, homes, transportation, and other everyday environments. Neuroergonomics involves interpreting real-life situations and activities performed in natural real-world settings, studying the neural basis of cognitive and motor functions by measuring electromagnetic or hemodynamics activity in the brain (Bernhardt et al., 2019;Nuamah et al., 2020;Parasuraman and Rizzo, 2008). ...
This study aims to evaluate the effect of workstation type on the neural and vascular networks of the prefrontal cortex (PFC) underlying the cognitive activity involved during mental stress. Workstation design has been reported to affect the physical and mental health of employees. However, while the functional effects of ergonomic workstations have been documented, there is little research on the influence of workstation design on the executive function of the brain. In this study, 23 healthy volunteers in ergonomic and non-ergonomic workstations completed the Montreal imaging stress task, while their brain activity was recorded using the synchronized measurement of electroencephalography and functional near-infrared spectroscopy. The results revealed desynchronization in alpha rhythms and oxygenated hemoglobin, as well as decreased functional connectivity in the PFC networks at the non-ergonomic workstations. Additionally, a significant increase in salivary alpha-amylase activity was observed in all participants at the non-ergonomic workstations, confirming the presence of induced stress. These findings suggest that workstation design can significantly impact cognitive functioning and human capabilities at work. Therefore, the use of functional neuroimaging in workplace design can provide critical information on the causes of workplace-related stress.
... The cognitive workload involves studying the dynamic relationships between the resources necessary to accomplish a task and the ability of the brain to adequately supply those resources [1], [2]. It is defined as the total mental activity imposed on a subject's cognitive system during a particular period of work. ...
Mental workload has been widely estimated based on electroencephalography (EEG) in the frequency domain. However, simple frequency features are not entirely accurate indicators of the cognitive load because surface EEG signals are weak, nonstationary and randomness. We hypothesize that graph methods, which analyse the relationship between each point and other points of the EEG signals, may provide a more precise identification of the mental load. To investigate this hypothesis, we aim to identify the optimum graph features from 14 channel EEG recordings (sampling rate=128 Hz) in order to detect the high cognitive load related to multitasking. Three graph features: mean degree d, clustering coefficient c, and degree distribution p(k), are extracted from 48 subjects EEG records. Each experimental subject conducts two tasks: without tasks and with a simultaneous capacity task, respectively. After the experiment is completed, the feeling of the subject with the cognitive load tags in three types: low load, medium load, and heavy load. The optimal features of these three levels of the subject sensation and two types of cognitive load in different tasks are selected on the basis of statistical analysis. Then all graph features are forwarded into a support vector machine (SVM) and a decision tree to conduct objective scoring classification and a three subjective rating classification, respectively. Based on the present results,channels O2, T8, FC6, F8, and AF4 are considered optimal for a more efficiently estimation of the cognitive load. c associated with F8 and T8 during low cognitive load is significantly lower than those associated with high cognitive load (p<0.001). Using three graph features, the accuracy of identifying two types of mental load is 89.6%. Current findings suggest that the mental workload associated with multi-tasks can be accurately assessed using the graph approaches to EEG data.
... The number of blinks also reduced considerably with the increasing workload in both tasks. Pupil size is a reliable measure of workload (Beatty, 1982, Mandrick et al., 2016 as it dilates with increasing workload (Batmaz and Ozturk, 2008, Kosch et al., 2018, Truschzinski et al., 2018, Bernhardt et al., 2019, Kearney et al., 2019, Marinescu et al., 2018, Wanyan et al., 2014. Recarte et al., 2008 show that blink inhibition happens in higher workload conditions and so, the blink rate is inversely correlated with the attentional levels and workload experienced by the operator (Veltman and Gaillard, 1996, Brookings et al., 1996, Wilson, 2002, Borghini et al., 2014, Widyanti et al., 2017, Wanyan et al., 2018. ...
Objective
We have designed tracking and collision prediction tasks to elucidate the differences in the physiological response to the workload variations in basic ATC tasks to untangle the impact of workload variations experienced by operators working in a complex ATC environment.
Background
Even though several factors influence the complexity of ATC tasks, keeping track of the aircraft and preventing collision are the most crucial.
Methods
Physiological measures, such as electroencephalogram (EEG), eye activity, and heart rate variability (HRV) data, were recorded from 24 participants performing tracking and collision prediction tasks with three levels of difficulty.
Results
The neurometrics of workload variations in the tracking and collision prediction tasks were markedly distinct, indicating that neurometrics can provide insights on the type of mental workload. The pupil size, number of blinks and HRV metric, root mean square of successive difference (RMSSD), varied significantly with the mental workload in both these tasks in a similar manner.
Conclusion
Our findings indicate that variations in task load are sensitively reflected in physiological signals, such as EEG, eye activity and HRV, in these basic ATC-related tasks.
Application
These findings have applicability to the design of future mental workload adaptive systems that integrate neurometrics in deciding not just ‘when’ but also ‘what’ to adapt. Our study provides compelling evidence in the viability of developing intelligent closed-loop mental workload adaptive systems that ensure efficiency and safety in ATC and beyond.
Précis
This article identifies the physiological correlates of mental workload variation in basic ATC tasks. The findings assert that neurometrics can provide more information on the task that contributes to the workload, which can aid in the design of intelligent mental workload adaptive system.
... Recent studies exploring Air Traffic Controllers (Bernhardt et al., 2019) have computed an EEG-based workload index that could differentiate between task workload requirements exploring front-parietal brain function. Yet, EEG exploration has achieved traction in aviation and space operations, current studies face challenges related to the intrusive and bulky nature of the equipment (Caldwell et al., 2002), the discomfort of long preparation time, and dependence on gel electrodes (e.g., wet electrodes). ...
Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.
... Various studies have been conducted recently to address the future ATCO simulation training issues regarding the implementation of new learning tools and technologies (Upgrove and Jafer (8,9) , Chayya et al. (10) , Coyne (11) ) and their impact on trainees' learning skills such as cross-task cue utilisation and situational awareness (Falkland and Wiggins (12) ), as well as workload and engagement metrics (Bernhardt et al. (13) ). These studies provide important insights into the development of ATCO simulation training content within the framework of regulations. ...
Air traffic controller training is highly regulated but lacks prescribed common assessment criteria and methods to evaluate trainees at the level of basic training and consideration of how trainees in fluence flight efficiency.We investigated whether there is a correlation between two parameters, viz. the trainees’ assessment score and fuel consumption, obtained and calculated after real-time human-in-the-loop radar simulations within the ATCOSIMA project. Although basic training assessment standards emphasise safety indicators, it was expected that trainees with higher assessment scores would achieve better flight efficiency, i.e. less fuel consumption. However, the results showed that trainees’ assessment scores and fuel consumption did not correlate in the expected way, leading to several conclusions.
... Another study identified that neurophysiological measures are more sensitive, in comparison to eye-tracking metrics, when discriminating the impact that task difficulty and complexity have in a high-fidelity driving scenario (Flumeri et al. 2018). In contrast, Bernhardt et al. (2019), reached the conclusion that EEG workload and pupil dilation are not correlated to each other, a finding also shared by Matthews et al. (2015), and hypothesized that it may be that pupil dilation may not be a direct measure of cognitive workload. Regardless, fNIRS and eye-tracking are portable, safe, affordable and unobtrusive systems, which can present a potential multi-sensor approach to a multidimensional evaluation of an ATC specialist's cognitive workload. ...
Air Traffic Control (ATC) specialists work in an environment where the proficient interaction between humans and computer systems is crucial to provide a safe and efficient flow of traffic. The complexity of this system may increase due to planned changes in operator roles and a projected rise in traffic volume. This increase, over an already highly complex system, will exacerbate the mental workload placed on the operator. The emergence of wearable sensors that measure physiological signals enables us to monitor the mental workload in real time without interfering in operational activity. However, the use of a single sensor approach may not provide a comprehensive assessment of cognitive workload while executing a complex task. Therefore, this study implemented a multimodal approach by using two sensors, namely functional near infrared spectroscopy (fNIRS) and eye-tracking, to evaluate the cognitive workload changes experienced by an ATC specialist. Three retired tower controllers with over 20 years of experience, underwent three sessions of experimentation where each individual fulfilled one of the following roles - observer, a Local controller or a Ground controller. During each iteration, the fNIRS and eye tracking sensors were attached to the Local controller while they commanded aircraft through verbal clearances. The task difficulty and complexity were quantified by the number of aircraft and clearances given, respectively. The number of aircraft displayed on the screen increased across time and was positively correlated with oxygenation measures assessed by the fNIRS signals of both the right and left prefrontal cortex. On the other hand, the number of fixations was positively correlated with the number of clearances. These results suggest fNIRS and eye-tracking measures are sensitive to changes in cognitive workload, and indicates that they may be amenable to complement each other for the assessment of the multidimensionality of cognitive workload induced by task difficulty and complexity.
Pseudo-pilots work in real-time simulations in air traffic control training and carry out an important task to ensure appropriate training for the trainee controllers. They are in constant communication with controllers during the training and are responsible for fulfilling the instructions given by the controllers. Unlike real operations, pseudo-pilots are required to monitor many aircraft simultaneously on the simulator’s radar screen and perform various interventions. This situation can be challenging, and the impact of such situations on their workload is usually neglected. In the literature, various studies have revealed the factors affecting controller workload; however, the factors affecting the workload of pseudo-pilots have not been emphasized. In this study, statistical analyses were carried out using the data obtained from real-time simulations and NASA-TLX surveys. As a result, the number of aircraft, controller performance, and the interruption duration were determined as the key factors that affect the overall workload of pseudo-pilots.
The purpose of this study was to examine neuroimaging technologies for quantifying cognitive load in intelligent interactive multimedia systems for experimental applications by conducting a systematic review of all relevant papers published up to April 2020. The study's most striking finding is that electroencephalography, functional magnetic resonance imaging, functional near-infrared spectroscopy, and transcranial doppler ultrasonography are the most frequently used neuroimaging equipment in cognitive load research in multimedia learning. Forty papers were selected depending on the equipment that should be understood in the field of neuroimaging in examining cognitive load in multimedia learning, the benefits and drawbacks of neuroimaging devices, and the experimental protocol for cognitive load in multimedia research. The study's findings were analyzed, and numerous discrepancies in the research on cognitive load and multimedia learning were discovered.
Predicting separation errors in the daily tasks of air traffic controllers (ATCOs) is essential for the timely implementation of mitigation strategies before performance declines and the prevention of loss of separation and aircraft collisions. However, three challenges impede accurate separation errors forecasting: 1) compounding relationships between many human factors and control processes require sufficient operation process data to capture how separation errors occur and propagate within controller-in-the-loop processes; 2) previous human factor measurement approaches are disruptive to controllers’ daily operations because they use invasive sensors, such as electroencephalography (EEG) and electrocardiography (ECG), 3) errors accumulated in using the tasks and human behaviors for estimating system dynamics challenge accurate separation error predictions with sufficient leading time for proactive control actions. This study proposed a separation error prediction framework with a long leading time (>50 s) to address the above challenges, including 1) a multi-factorial model that characterizes the inter-relationships between task complexity, behavioral activity, cognitive load, and operational performance; 2) a multimodal data analytics approach to non-intrusively extract the task features (i.e., traffic density) from high-fidelity simulation systems and visual behavioral features (i.e., head pose, eyelid movements, and facial expressions) from ATCOs’ facial videos; 3) an encoder-decoder Long Short-Term Memory (LSTM) network to predict long-time-ahead separation errors by integrating multimodal features for reducing accumulated errors. A user study with six experienced ATCOs tested the proposed framework using the Phoenix Terminal Radar Approach Control (TRACON) simulator. The authors evaluated the model performance through two types of metrics: 1) point-level metrics, including precision, recall, and F1-score, and 2) sequence-level metrics, including alignment accuracy and sequence similarity. The results showed that 1) the model using the task and visual behavioral features significantly improved the prediction performance compared to the model using one single feature (eyelid movements), with an improvement of up to 26.95% in alignment accuracy for 10s-ahead prediction; 2) the model that combined task and visual behavioral features had a higher or comparable performance to models with different hybrid features, achieving an alignment accuracy of 82.38% for 50s-ahead error prediction; and (3) the proposed method outperformed three baseline models – Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), and classic LSTM – by 8.21%, 3.47%, and 3.14% in alignment accuracy, respectively, for predicting 50s-ahead separation errors. These results suggest that the proposed model can effectively predict separation errors in air traffic control.
The rate of human errors would increase as air traffic control officers (ATCOs) lose situation awareness (SA), which could also be affected by their perceived workloads. Recognising ATCOs SA inadequacy is crucial for ensuring traffic safety. This paper aims to propose a two-phase analytical methodology for revealing SA-related neuro-physiological patterns and hierarchically recognising ATCOs SA loss with workload concerns using EEG and eye-tracking data. A simulated air traffic control (ATC) radar-monitoring experiment involving different task loads with SA-probe tests was first conducted to collect behavioural and physiological (EEG and eye-tracking) data simultaneously, and the NASA Task Load Index (NASA-TLX) scale was used to measure participants’ perceived workloads. In our two-phase methodology, behavioural data representing the task performance was analysed in Phase I using the Gaussian Mixture Model to determine sample’s SA, and the perceived workloads on samples were labelled using NASA-TLX scores. Subsequently, in order to achieve our purposes, the physiological data were annotated based on results from Phase I, the physiological feature base was extracted using a fast Fourier transformation and Hilbert transform in Phase II, and the linear discriminant analysis was then used to extract the core features as inputs to train multiple classifiers. Results showed that the neuro-physiological behaviours of SA loss during normal workloads differed from those in high workload situations. A leave-one-subject-out cross-validation was also performed, and the results demonstrated that the optimal performance was 76.1% for classifying high/low SA (1-Level classification) and 82.7% for recognising low SA associated with high workload (2-Level classification).
This study examines the utilities of a dynamic Bayesian network (DBN) to predict multitasking performance. Multitasking is the practice of conducting more than one task simultaneously. Compared with BN (Bayesian network), the DBN has the advantage of encoding both spatial and temporal relationships of the multiple variables under uncertain information. We established the DBN model based on contextual and observable variables from 19 participants to predict multitasking performance over time. The proposed DBN outperformed the BN model with smaller prediction errors and showed great potential to be used as a tool in error management and operator screening in diverse work environments.
Air traffic controllers (ATCOs) are one kinds of key personnel for aviation safety. They often accept training for learning new knowledge and skills of advanced technologies related on aviation safety. The aim of this study was to learn the age effect on the perception, mood and fatigue of ATCOs for learning. An investigation with 3 × 2 levels of two factors including load of training and age on simulators was carried out at one training center in China. 234 effective questionnaires (78 person times) were collected and analyzed with statistical methods. Results of Pearson correlation test showed that ATCOs’ perception had correlativity with their mood and their fatigue. Results of repeated measure variance analysis showed that the accumulation of load of training led to ATCOs perception decreased, mood worse, and fatigue increased very significantly. The results also showed that the age effect was significant to ATCOs’ perception, mood and fatigue, and the elder ATCOs were easier to feel perception decreased, mood worse, and fatigue than the younger ones. These results indicated that it was more difficult for elder ATCOs to learn new knowledge and skills. And the results promote to give more help to the elder ATCOs for their training on new knowledge and skills.
In a complex human-computer interaction system, estimating mental workload based on electroencephalogram (EEG) plays a vital role in the system adaption in accordance with users’ mental state. Compared to within-subject classification, cross-subject classification is more challenging due to larger variation across subjects. In this paper, we targeted the cross-subject mental workload classification and attempted to improve the performance. A capsule network capturing structural relationships between features of power spectral density and brain connectivity was proposed. The comparison results showed that it achieved a cross-subject classification accuracy of 45.11%, which was superior to the compared methods (e.g., convolutional neural network and support vector machine). The results also demonstrated feature fusion positively contributed to the cross-subject workload classification. Our study could benefit the future development of a real-time workload detection system unspecific to subjects.KeywordsMental workload classificationCapsule networkFeature fusionCross-subjectEEGBrain connectivityPower spectral density
Teams formulated by aviation professionals are essential in maintaining a safe and efficient aerodrome environment. Nonetheless, the shared situational awareness between the flight crews under adverse weather conditions might be impaired. This research aims to evaluate the impact of a proposed enhancement in communication protocol on cognitive workload and develop a human-centred classification model to identify hazardous meteorological conditions. Thirty groups of subjects completed four post-landing taxiing tasks under two visibility conditions (CAVOK/CAT IIIA) while two different communication protocols (presence/absence of turning direction information) were adopted by the air traffic control officer (ATCOs). Electroencephalography (EEG) and the NASA Task Load Index were respectively used to reflect the pilot’s mental state and to evaluate the pilot’s mental workload subjectively. Results indicated that impaired visibility increases the subjective workload significantly, while the inclusion of turning direction information in the ATCO’s instruction would not significantly intensify their cognitive workload. Mutual information was used to quantitatively assess the shared situational awareness between the pilot flying and the pilot monitoring. Finally, this research proposes a human-centred approach to identify potentially hazardous weather conditions from EEG power spectral densities with Bayesian neural networks (BNN). The classification model has outperformed other baseline algorithms with an accuracy of 66.5%, an F1 score of 61.4%, and an area under the ROC of 0.749. Using the concept of explainable AI with Shapley Additive Explanations (SHAP) values, the exploration of latent mental patterns formulates novel knowledge to gain insights into the vital physiological indicators of the pilots in response to different scenarios from the BNN model. In the long term, the model facilitates the decision regarding the necessity of providing automation and decision-making aids to pilots.
Background and Objective
Operator's capability for accurately comprehending verbal commands is critically important to maintain the performance of human-machine interaction. It can be evaluated by human mental workload measured with electroencephalography (EEG). However, the time duration of different workload conditions within a task session is unequal due to varied psychophysiological processes across individuals. It leads to data imbalance of the EEG for training workload classifiers.
Methods
In this study, we propose an EEG feature oversampling technique, Gaussian-SMOTE based feature ensemble (GSMOTE-FE), for workload recognition with imbalanced classes. First, artificial EEG instances are drawn from a Gaussian distribution in the margin between the minority and majority workload classes. Tomek links are detected as clues to remove redundant feature vectors. Then, we embed a feature selection module based on the GINI importance while an ensemble classifier committee with bootstrap aggregating is used to further enhance classification performance.
Results
We validate the GSMOTE-FE framework based on an experiment that simulates operators to understand the correct meaning of the instructions in the Chinese language. Participants’ EEG signals and reaction time data were both recorded to validate the proposed workload classifier. Workload classification accuracy and Macro-F1 values are 0.6553 and 0.5862, respectively. Corresponding G-mean and AUC achieve at 0.5757 and 0.5958, respectively.
Conclusions
The performance of the GSMOTE-FE is demonstrated to be comparable with the advanced oversampling techniques. The workload classifier has the capability to indicate low and high levels of the task demand of the Chinese language understanding task.
This study provides a systematic synthesis of empirical research on mental workload (MWL) in air traffic control (ATC). MWL is a key concept in research on innovative technologies, because the assessment of MWL is crucial to the evaluation of such technologies. Our specific focus was on physiological measures of MWL. The used search strategy identified 39 peer-reviewed publications that analysed ATC tasks, examined different levels of difficulty of the ATC task, and considered at least one physiological measure of MWL. Positive relations between measures of MWL and task difficulty were observed most frequently, indicating that the measures indeed allowed the assessment of MWL. The most commonly used physiological measures were brain measures (EEG and fNIR) and heart rate measures. The review revealed a need for more precise descriptions of crucial experimental parameters in order to permit a transition of the field towards more interactive and dynamic types of analysis.
Practitioner summary: Research on innovative technology in air traffic control (ATC) depends on assessments of mental workload (MWL). We reviewed empirical research on MWL in ATC. Brain and heart measures often allow assessments of MWL. Better descriptions of experiments are needed to allow comparisons among studies and more dynamic and interactive analyses.
At present, electroencephalogram (EEG) has been widely used in the classification of mental workload. But most of the EEG acquisition devices used in the research a use a large number of electrodes. However, this brings high hardware costs, limited portability and discomfort to the wearer. In addition, most of the channels have information redundancy and noise interference, which have a negative impact on the subsequent mental workload classification. Therefore, it is necessary to use fewer channels to accurately identify the mental load of the operator. Focusing on the above problems, a method of channel selection based on Davies–Bouldin Index (DBI) for visual manipulation tasks is proposed in this paper, it selects effective channels by analyzing the differences between the features of low and high workload data.
This book reports the most recent, advanced, successful, and real applications of ergonomics in order to improve the human well-being and performance in a short term, as well as the organizational performance in a long term. The book is organized as follows:
Physical Ergonomics. This section reports case studies where physical risk factors are presented in the workplace, such as physical risk factors including uncomfortable body postures, repetitive movements, force application, manual material handling, and physical environmental conditions. In addition, case studies must report applications from physical ergonomics methods, for instance, RULA, REBA, OWAS, NIOSH, JSI, Suzane Rodgers, ERIN, among others.
Cognitive Ergonomics. This section reports the implementation of ergonomic tools, techniques, and methods in real case studies. These applications are aimed to know, decrease, and control cognitive and psychological risk factors, such as mental workload, information processing, situation awareness, human error identification, and interface analysis. These applications may include the following methods NASA-TLX, SWAT, CWA, SHERPA, HET, TAFEI, SAGAT, SART, SACRI, QUIS, SUMI, to mention a few of them.
Macro-ergonomics. This section is focused on the analysis, design, and evaluation of work systems. It reports case studies where risk factors are beyond a specific workstation. These risk factors may include supervision styles, teamwork management, task variety, social relationships, organizational culture, organizational communication, technology, work schedules, and motivation, among others. In addition, case studies report the application of macro-ergonomic methods, such as MOQS, focus group, participatory ergonomics, HITOP, MAS, and MEAD, among others.
Current organizational and manufacturing processes imply high mental workload demands that can be evaluated through the construct of MWL (mental workload). This term is often used in new manufacturing and organizational environments, which have replaced physical tasks with cognitive activities involving a high MWL. By overusing the attentional resources given to the tasks, such work environments are placing high cognitive loads on operators, thus affecting their performance and causing them to experience mental fatigue. A formal evaluation of MWL offers the opportunity to prevent mental disorders and maintain mental health. On the other hand, the lack of evaluation and proper management of MWL in the industry can result in errors that create economic costs, accidents, injuries, or even deadly events. Finally, MWL assessment and management can be a human-oriented strategy designed to improve and sustain the future of an organization. Industries must find competitive advantages in sustainable processes from the economic, environmental, and social view. In this sense, this chapter aims to present a literature review to provide a comprehensive literature analysis of MWL evaluation and management for sustainable processes in the manufacturing industry, from a social perspective.
This research empirically evaluates the introduction of speech to existing keyboard and mouse input modalities in an application used to control aircraft in a simulated, complex and dynamic environment. Task performance and task performance degradation are assessed for three levels of workload. Previous studies have evaluated task performance using these modalities however, only a couple have evaluated task performance under varying workload. Even though speech is a common addition to modern control interfaces, the effect of varying workload on this combination of control modalities has not yet been reported. Thirty-six participants commanded simulated aircraft through generated obstacle courses to reach a Combat Air Patrol (CAP) point while also responding to a secondary task. There were nine conditions that varied the control modality (Keyboard and Mouse (KM), Voice (V), and Keyboard, Mouse and Voice (KMV)), and workload by varying the number of aircraft being controlled (low, medium and high). Results showed that KM outperformed KMV and V for the low and medium workload levels. However, task performance with KMV was found to degrade the least as workload increased. KMV and KM were found to enable significantly more correct responses to the secondary task which was delivered aurally. Participants reported a preference for the combined modalities (KMV), self-assessing that KMV most reduced their workload. This research suggests that the addition of a speech interface to existing keyboard and mouse modalities, for control of aircraft in a simulation, may help manage cognitive load and may assist in controlling more aircraft under higher workloads.
Safety-critical systems like air traffic control (ATC) are usually less automated than might be expected by the public, so human intelligence will remain at the core in the decision-making (DM) process. Meanwhile, human factors (HFs) need to be fully considered in the DM process, which can design the ATC system to be more intelligent and more adaptive to the behaviour of the user. However, the existing DM research lacks the systematic methods that fully consider human performance in a smart manner. This study proposed a human-centred adaptive DM methodology that combines subjective and objective measurements made by functional near-infrared spectroscopy (fNIRS) via intelligent automation (IA). Moreover, this paper also described a case study of radar display map operation, including descriptive and optimised maps, to illustrate the proposed approach and verify its feasibility and effectiveness. The results were determined by jointly considering the user-generated and system-generated data and suggested that the proposed approach could capture subjective and objective data, take into consideration the HFs information to provide real-time online feedback and adjust the decision support system to HFs. It is hoped that this study can promote the methodology of human-centred subjective and objective data-driven applications in the future ATC environment adaptive decision research.
Mental or cognitive workload is an essential factor in Air Traffic Control (ATC). Besides the task demand, there are various types of individual factors that affect the workload felt by the Air Traffic Controller (ATCo). Therefore, this study aims to examine the relationship between individual factors and perceived mental workload using NASA-TLX. A total of 256 questionnaires were successfully collected from Air Traffic Controllers in the busiest airport in Indonesia. All observed groups indicated high perceived workload scores, with F-Test of One-way ANOVA used to confirm the relationship between individual factors and mental workload. The result showed that the workload is significantly related to the gender specification and working experience of ATCos. Fewer working years and female gender tended to have lower perceived workload than other groups. On the other hand, the difference in unit tasks (ACC and APP) and age failed to differ the perceived workload significantly. The result confirms that the mental workload of ATCo is high, and affected by some individual factors.
In this paper, an intelligent algorithm for modeling and simulating the Air Traffic Control (ATC) is presented. Reducing the workload of ATC operators by using computers instead of humans, the algorithm decreases the human faults. In this regard, two methods, including Naïve Bayes classifier (NB) and neural network (NN), are employed to make the machines intelligent. Having used a huge number of data from radio signal strengths at determined positions, least-square and backpropagation algorithms are utilized to train and evaluation test of NB classifier and neural network, respectively. The experiments compare between these two methods; however, both of them show satisfactory results in the determination of the agents' positions.
In this paper, an intelligent algorithm for modeling and simulating the Air Traffic Control (ATC) is presented. Reducing the workload of ATC operators by using computers instead of humans, the algorithm decreases the human faults. In this regard, two methods, including Naïve Bayes classifier (NB) and neural network (NN), are employed to make the machines intelligent. Having used a huge number of data from radio signal strengths at determined positions, least-square and backpropagation algorithms are utilized to train and evaluation test of NB classifier and neural network, respectively. The experiments compare between these two methods; however, both of them show satisfactory results in the determination of the agents' positions.
In measurement and evaluation for a cognitive performance carried out on various tasks can use objective and subjective measurement tools. This study aims to review research on measuring instruments and provide its potential to be used in research related to cognitive ergonomics. The method that is used in this study is a review of article literature on studies that are subjective and objective measurement tools. the results of this study indicate that in each study usually does not only use one measuring instrument, to validate measurements, but another measurement tool is also used. And in research usually uses subjective and objective measurement tools for the same task.
The positivity effect among older adults is a tendency to process more positive and/or less negative emotional stimuli compared to younger adults, with unknown upper age boundaries. Cognitive and emotional working memory were assessed in young-old adults (60–75) and very old adults (VOAs; 80+) to determine whether emotional working memory declines similar to the age-related decline of cognitive working memory. The moderating role of valence on the link between age and emotional working memory was examined to identify change in positivity effect with advanced age. Electroencephalography (EEG) markers of cognitive workload and engagement were obtained to test the theory of cognitive resource allocation in older adults’ emotional stimuli processing. EEG recordings were collected during cognitive memory task and emotional working memory tasks that required rating emotional intensity of images pairs. Results indicate a positivity effect among VOAs that does not require additional cognitive effort and is not likely to diminish with age.
Pupillometry is a promising method for assessing mental workload and could be helpful in the optimization of systems that involve human-computer interaction. The present study focuses on replicating the studies by Ahern (1978) and Klingner (2010), which found that for three levels of difficulty of mental multiplications, the more difficult multiplications yielded larger dilations of the pupil. Using a remote eye tracker, our research expands upon these two previous studies by statistically testing for each 1.5 s interval of the calculation period (1) the mean absolute pupil diameter (MPD), (2) the mean pupil diameter change (MPDC) with respect to the pupil diameter during the pre-stimulus accommodation period, and (3) the mean pupil diameter change rate (MPDCR). An additional novelty of our research is that we compared the pupil diameter measure with a self-report measure of workload, the NASA Task Load Index (NASA-TLX), and with the mean blink rate (MBR). The results showed that the findings of Ahern and Klingner were replicated, and that the MPD and MPDC discriminated just as well between the lowest and highest difficulty levels as did the NASA-TLX. The MBR, on the other hand, did not interpretably differentiate between the difficulty levels. Moderate to strong correlations were found between the MPDC and the proportion of incorrect responses, indicating that the MPDC was higher for participants with a poorer performance. For practical applications, validity could be improved by combining pupillometry with other physiological techniques.
This research investigated controller' situation awareness by comparing COOPANS's acoustic alerts with newly designed semantic alerts. The results demonstrate that ATCOs' visual scan patterns had significant differences between acoustic and semantic designs. ATCOs established different eye movement patterns on fixations number, fixation duration and saccade velocity. Effective decision support systems require human-centered design with effective stimuli to direct ATCO's attention to critical events. It is necessary to provide ATCOs with specific alerting information to reflect the nature of the critical situation in order to minimise the side effects of startle and inattentional deafness. Consequently, the design of a semantic alert can significantly reduce ATCOs' response time, therefore providing valuable extra time in a time-limited situation to formulate and execute resolution strategies in critical air safety events. The findings of this research indicate that the context-specified design of semantic alerts could improve ATCO's situational awareness and significantly reduce response time in the event of Short Term Conflict Alert (STCA) activation which alerts to two aircraft having less than the required lateral or vertical separation. Practitioner Summary: Eye movements are closely linked with visual attention and can be analysed to explore shifting attention whilst performing monitoring tasks. This research has found that context-specific designed semantic alerts facilitated improved ATCO cognitive processing by integrating visual and auditory resources. Semantic designs have been demonstrated to be superior to acoustic design by directing the operator's attention more quickly to critical situations.
Abbreviations:
APW: area proximity warning; ASRS: aviation safety reporting system; ATC: air traffic control; ATCO: air traffic controller; ATM: air traffic management; COOPANS: cooperation between air navigation service providers; HCI: human-computer interaction; IAA: irish aviation authority; MSAW: minimum safe altitude warning; MTCD: medium-term conflict detection; SA: situation awareness; STCA: short term conflict alert; TP: trajectory prediction.
Electroencephalograph (EEG) has been increasingly studied to identify distinct mental factors when persons perform cognitively demanding tasks. However, most of these studies examined EEG correlates at channel domain, which suffers the limitation that EEG signals are the mixture of multiple underlying neuronal sources due to the volume conduction effect. Moreover, few studies have been conducted in real-world tasks. To precisely probe EEG correlates with specific neural substrates to mental factors in real-world tasks, the present study examined EEG correlates to three mental factors, i.e., mental fatigue [also known as time-on-task (TOT) effect], workload and effort, in EEG component signals, which were obtained using an independent component analysis (ICA) on high-density EEG data. EEG data were recorded when subjects performed a realistically simulated air traffic control (ATC) task for 2 h. Five EEG independent component (IC) signals that were associated with specific neural substrates (i.e., the frontal, central medial, motor, parietal, occipital areas) were identified. Their spectral powers at their corresponding dominant bands, i.e., the theta power of the frontal IC and the alpha power of the other four ICs, were detected to be correlated to mental workload and effort levels, measured by behavioral metrics. Meanwhile, a linear regression analysis indicated that spectral powers at five ICs significantly increased with TOT. These findings indicated that different levels of mental factors can be sensitively reflected in EEG signals associated with various brain functions, including visual perception, cognitive processing, and motor outputs, in real-world tasks. These results can potentially aid in the development of efficient operational interfaces to ensure productivity and safety in ATC and beyond.
The assessment of mental workload could be helpful to road safety especially if developments of vehicle automation will increasingly place drivers into roles of supervisory control. With the rapidly decreasing size and increasing resolution of cameras as well as exponential computational power gains, remote eye measurements are growing in popularity as non-obtrusive and non-distracting tools for assessing driver workload. This review summarizes literature on the relation between eye measurement parameters and drivers’ mental workload. Various eye activity measures including blinks, fixations, and saccades have previously researched and confirmed as useful estimates of a driver's mental workload. Additionally, recent studies in pupillometry have shown promise for real-time prediction and assessment of driver mental workload after effects of illumination are accounted for. Specifically, workload increases were found to be indicated by increases in blink latency, PERCLOS, fixation duration, pupil dilation, and ICA; by decreases in blink duration and gaze variability; and with mixed results regarding blink rate. Given such a range of measures available, we recommend using multiple assessment methods to increase validity and robustness in driver assessment.
The present study examined whether personality characteristics and general intelligence predict multitasking performance. The Multi-Attribute Task Battery-II was used to assess multitasking performance. Personality factors included the Big Five, openness, conscientiousness, extraversion, agreeableness, and neuroticism. The results indicated scores on general intelligence predict performance on the tracking task of the Multi-Attribute Task Battery-II, where higher scores of general intelligence predicted improved tracking performance. Additionally, conscientiousness and neuroticism were found to predict worsened performance on the resource management task of the Multi-Attribute Task Battery-II. Furthermore, agreeableness was found to predict perceived workload on the mental demand subscale of the Workload Rating Scale.
The present study examined the sensitivity of several candidate metrics of real-time workload within the spatial component of an unmanned aerial vehicle (UAV) task. Advanced Brain Monitoring's (ABM) wireless B-Alert system was used to collect participant's EEG workload and engagement data. Eye tracking data was also collected. The UAV simulation required participants to report heading information of moving vehicles, as seen from the UAV. There were four blocks of difficulty, over which a significant performance decrement was shown. Additionally, participants rated their workload significantly higher and pupil diameter significantly increased across blocks of increasing difficulty, as well as within each block during periods of highest mental demand. ABM's workload and engagement metrics however did not show a significant change over or within blocks. The results showed that pupil diameter shows promise as a correlate of mental workload.
A study was run to test the sensitivity of multiple workload indices to the differing cognitive demands of four military monitoring task scenarios and to investigate relationships between indices.
Various psychophysiological indices of mental workload exhibit sensitivity to task factors. However, the psychometric properties of multiple indices, including the extent to which they intercorrelate, have not been adequately investigated.
One hundred fifty participants performed in four task scenarios based on a simulation of unmanned ground vehicle operation. Scenarios required threat detection and/or change detection. Both single- and dual-task scenarios were used. Workload metrics for each scenario were derived from the electroencephalogram (EEG), electrocardiogram, transcranial Doppler sonography, functional near infrared, and eye tracking. Subjective workload was also assessed.
Several metrics showed sensitivity to the differing demands of the four scenarios. Eye fixation duration and the Task Load Index metric derived from EEG were diagnostic of single-versus dual-task performance. Several other metrics differentiated the two single tasks but were less effective in differentiating single- from dual-task performance. Psychometric analyses confirmed the reliability of individual metrics but failed to identify any general workload factor. An analysis of difference scores between low- and high-workload conditions suggested an effort factor defined by heart rate variability and frontal cortex oxygenation.
General workload is not well defined psychometrically, although various individual metrics may satisfy conventional criteria for workload assessment.
Practitioners should exercise caution in using multiple metrics that may not correspond well, especially at the level of the individual operator.
According to Cognitive Load Theory (CLT), one of the crucial factors for successful learning is the type and amount of working-memory load (WML) learners experience while studying instructional materials. Optimal learning conditions are characterized by providing challenges for learners without inducing cognitive over- or underload. Thus, presenting instruction in a way that WML is constantly held within an optimal range with regard to learners' working-memory capacity might be a good method to provide these optimal conditions. The current paper elaborates how digital learning environments, which achieve this goal can be developed by combining approaches from Cognitive Psychology, Neuroscience, and Computer Science. One of the biggest obstacles that needs to be overcome is the lack of an unobtrusive method of continuously assessing learners' WML in real-time. We propose to solve this problem by applying passive Brain-Computer Interface (BCI) approaches to realistic learning scenarios in digital environments. In this paper we discuss the methodological and theoretical prospects and pitfalls of this approach based on results from the literature and from our own research. We present a strategy on how several inherent challenges of applying BCIs to WML and learning can be met by refining the psychological constructs behind WML, by exploring their neural signatures, by using these insights for sophisticated task designs, and by optimizing algorithms for analyzing electroencephalography (EEG) data. Based on this strategy we applied machine-learning algorithms for cross-task classifications of different levels of WML to tasks that involve studying realistic instructional materials. We obtained very promising results that yield several recommendations for future work.
Objective: To identify psychophysiological indices of deadly force decision making in experts versus novices during simulator training.Background: Modern training techniques focus on improving decision-making skills with participative assessment between trainees and subject matter experts primarily through subjective observation. Objective metrics need to be developed. The current study explored the potential for psychophysiological metrics of decision making in deadly force judgment contexts. Method: Twenty-four participants (novice, expert) were recruited. All wore a wireless EEG device to collect psychophysiological data during high-fidelity simulated deadly force judgment and decision-making simulations using a modified Glock firearm. Participants were exposed to 27 video scenarios, one-third of which would have justified use of deadly force. Pass/fail was determined by whether the participant used deadly force appropriately. Results: Experts had a significantly higher pass rate compared to novices (p < 0.05). Multiple metrics were shown to distinguish novices from experts. Hierarchical regression analysis indicate that psycho-physiological variables are able to explain 72% of the variability in expert performance, but only 37% in novices. Discriminant function analysis using psychophysiological metrics was able to discern between experts and novices with 72.6% accuracy. Conclusion: Results suggest that expert performance is more tightly coupled with psychophysiology, compared with a weaker relationship in novices. Discriminant function measures may have the potential to objectively identify when expertise is obtained. Application: Psychophysiological metrics may create a performance model with the potential to optimize simulator-based DFJDM training. These performance models could be used for trainee feedback, and/or by the instructor to assess performance objectively.
Electroencephalography (EEG) holds promise as a neuroimaging technology that can be used to understand how the human brain functions in real-world, operational settings while individuals move freely in perceptually-rich environments. In recent years, several EEG systems have been developed that aim to increase the usability of the neuroimaging technology in real-world settings. Here, the usability of three wireless EEG systems from different companies are compared to a conventional wired EEG system, BioSemi's ActiveTwo, which serves as an established laboratory-grade 'gold standard' baseline. The wireless systems compared include Advanced Brain Monitoring's B-Alert X10, Emotiv Systems' EPOC and the 2009 version of QUASAR's Dry Sensor Interface 10-20. The design of each wireless system is discussed in relation to its impact on the system's usability as a potential real-world neuroimaging system. Evaluations are based on having participants complete a series of cognitive tasks while wearing each of the EEG acquisition systems. This report focuses on the system design, usability factors and participant comfort issues that arise during the experimental sessions. In particular, the EEG systems are assessed on five design elements: adaptability of the system for differing head sizes, subject comfort and preference, variance in scalp locations for the recording electrodes, stability of the electrical connection between the scalp and electrode, and timing integration between the EEG system, the stimulus presentation computer and other external events.
Longer lasting performance in cognitively demanding tasks leads to an exhaustion of cognitive resources and to a state commonly described as mental fatigue. More specifically, the allocation and focusing of attention become less efficient with time on task. Additionally, the selection of even simple responses becomes more error prone. With respect to the recorded EEG, mental fatigue has been reported to be associated with an increase in frontal theta and frontal and occipital alpha activity. The present study focused on the time course of changes in behavior and in the EEG to characterize fatigue-related processes. Participants performed a spatial stimulus-response-compatibility task in eight blocks for an overall duration of four hours. Error rates increased continuously with time on task. Total alpha power was larger at the end compared to the beginning of the experiment. However, alpha power increased rapidly and reached its maximal amplitude already after one hour, whereas frontal theta showed a continuous increase with time on task, possibly related to increased effort to keep the performance level high. Time frequency analyses revealed power changes in the theta band induced by task relevant information that might be assigned to a drain of executive control capacities. Thus, frontal theta turned out to be a reliable marker of distinct changes in cognitive processing with increasing fatigue.
The aim of this study was to identify the cognitive factors that predictability and adaptability during multitasking with a flight simulator.
Multitasking has become increasingly prevalent as most professions require individuals to perform multiple tasks simultaneously. Considerable research has been undertaken to identify the characteristics of people (i.e., individual differences) that predict multitasking ability. Although working memory is a reliable predictor of general multitasking ability (i.e., performance in normal conditions), there is the question of whether different cognitive faculties are needed to rapidly respond to changing task demands (adaptability).
Participants first completed a battery of cognitive individual differences tests followed by multitasking sessions with a flight simulator. After a baseline condition, difficulty of the flight simulator was incrementally increased via four experimental manipulations, and performance metrics were collected to assess multitasking ability and adaptability.
Scholastic aptitude and working memory predicted general multitasking ability (i.e., performance at baseline difficulty), but spatial manipulation (in conjunction with working memory) was a major predictor of adaptability (performance in difficult conditions after accounting for baseline performance).
Multitasking ability and adaptability may be overlapping but separate constructs that draw on overlapping (but not identical) sets of cognitive abilities.
The results of this study are applicable to practitioners and researchers in human factors to assess multitasking performance in real-world contexts and with realistic task constraints. We also present a framework for conceptualizing multitasking adaptability on the basis of five adaptability profiles derived from performance on tasks with consistent versus increased difficulty.
Perhaps the most basic issue in the study of cognitive workload is the problem of how to actually measure it. The electroencephalogram (EEG) continues to be the clinical method of choice for monitoring brain function in assessing sleep disorders, level of anaesthesia and epilepsy. This preference reflects the EEG’s high sensitivity to variations in alertness and attention, the unimposing conditions under which it can be recorded, and the low cost of the technology it requires. These characteristics also suggest that EEG-based monitoring methods might provide a useful tool in ergonomics. This paper reviews a long-term programme of research aimed at developing cognitive workload monitoring methods based on EEG measures. This research programme began with basic studies of the way neuroelectric signals change in response to highly controlled variations in task demands. The results yielded from such studies provided a basis on which to develop appropriate signal processing methodologies to automatically differentiate mental effort-related changes in brain activity from artifactual contaminants and for gauging relative magnitudes of mental effort in different task conditions. These methods were then evaluated in the context of more naturalistic computerbased work. The results obtained from these studies provide initial evidence for the scientific and technical feasibility of using EEG-based methods for monitoring cognitive load during human–computer interaction.
Fatigue is a well known stressor in aviation operations and its interaction with mental workload needs to be understood. Performance, psychophysiological, and subjective measures were collected during performance of three tasks of increasing complexity. A psychomotor vigilance task, multi-attribute task battery and an uninhabited air vehicle task were performed five times during one night's sleep loss. EEG, ECG and pupil area were recorded during task performance. Performance decrements were found at the next to last and/or last testing session. The EEG showed concomitant changes. The degree of impairment was at least partially dependent on the task being performed and the performance variable assessed.
Mental imagery is known to play a significant role in the skilled performance of complex cognitive tasks, yet is mostly overlooked in the field of air traffic control—a task that is reliant on what controllers term “the picture.” This article explores 3 strands of imagery research: the similarities between imagery and perception, individual differences in imagery, and skill learning and imagery. The research reported is discussed in terms of fundamental implications for air traffic control, implications for the measurement of imagery, implications for training, and implications for technology design.
This chapter argues that fatigue is a problem of the management of control rather than of energy. Thorndike (1900) interpreted fatigue as a problem of doing the right thing, rather than of doing too much. Bartley and Chute (1947) considered fatigue a result of conflict between competing behavioral tendencies—between doing and not doing, between doing one thing and doing another. The idea that the resolution of conflict is an effective basis for the control of action is a familiar one (Berlyne, 1960; Botvinick, Braver, Barch, Carter, & Cohen, 2001; Norman & Shallice, 1986), with cognitive control acting to maintain selected tasks and prevent disruption by competing activities. Fatigue is interpreted here as an adaptive state, serving to maintain effective overall (system-wide) management of goals. In this conceptualization, the subjective experience of fatigue arises through conflict between current and competing goals, or action tendencies. In effect, it is assumed to have a signal value for motivational control, providing a mechanism for resolving conflicts between current goals and other desired courses of action. This approach is developed in the rest of the chapter by considering the boundary conditions for the experience and impact of fatigue, especially in relation to work. The focus is necessarily broader than fatigue itself, as fatigue is considered to be one aspect of the general control system that manages goal activity in the service of motivational requirements. A fundamental assumption of traditional theory is that fatigue is caused by work, but the nature of the work may be important: Does it matter how much effort is applied by the performer or how much control he or she has over what is done? A second assumption is that fatigue causes decrements in the performance of tasks (as a result of prolonged work without rest), but this is not always found to be the case. Researchers consider the nature of performance decrement, and the reasons why task goals may fail, both with the need to maintain them over time and, in the broader context, under the impact of stress and high workload. This leads to considering the general model of motivational control in task performance, in which goal management is conceived of as an essentially compensatory process (Hockey, 1997, 2005). At least for highly motivated tasks, primary task goals appear to be stabilized by increased regulatory control, with associated costs. Over the final part of the chapter, the compensatory control model is adapted and extended to focus on the specific problem of cognitive fatigue and its implications for patterns of performance decrement. In this model, fatigue is interpreted as an adaptive state that signals a growing conflict in control activity between what is being done and what else might be done, between old goals and new goals, and between duties and desires. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
This paper reports three studies on the application of ambulatory monitoring in air traffic control (ATC). The aim of the first study was to explore a set of psychophysiological measures with respect to ATC workload sensitivity and feasibility at the workplace. Nearly all physiological measures showed the expected changes during work. Significant positive correlations were found between cardiovascular responses and the number of aircraft under control, especially heavy, fast, climbing, and descending aircraft. The following en-route (Study 2) and tower (Study 3) simulations identified the relative impact of air traffic features. Heart rate, systolic blood pressure, self-reported concentration, and upset were significantly higher in the simulations with 12 aircraft continuously under control compared to only 6. A high versus low number of potential conflicts between aircraft in the en-route setting (Study 2) also caused significant increases of heart rate, systolic blood pressure, self-reported concentration, and upset. On the basis of these results, a new workload model for air traffic controllers was suggested and implemented. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
This paper describes the origins and hisotry of multiple resource theory in accounting for difference in dual task interference. One particular application of the theory, the 4-dimensional multiple resources model, is described in detail, positing that there will be greater interference between two tasks to the extent that they share stages (perceptua/cognitive vs response) sensory modalities (auditory vs visual), codes (visual vs spatial) and channels of visual information (focal vs ambient). A computational rendering of this model is then presented. Examples are given of how the model predicts interference differences in operational environments. Finally, three challenges to the model are outlined regarding task demand coding, task allocation and visual resource competition.
Abstract
Cognitive load theory has been designed to provide guidelines intended to assist in the presentation of information in a manner that encourages learner activities that optimize intellectual performance. The theory assumes a limited capacity working memory that includes partially independent subcomponents to deal with auditory/verbal material and visual/2- or 3-dimensional information as well as an effectively unlimited long-term memory, holding schemas that vary in their degree of automation. These structures and functions of human cognitive architecture have been used to design a variety of novel instructional procedures based on the assumption that working memory load should be reduced and schema construction encouraged. This paper reviews the theory and the instructional designs generated by it.
Modern work requires cognitively demanding multitasking and the need for sustained vigilance, which may result in work-related stress and may increase the possibility of human error. Objective methods for estimating cognitive overload and mental fatigue of the brain on-line, during work performance, are needed. We present a two-channel electroencephalography (EEG)-based index, theta Fz/alpha Pz ratio, potentially implementable into a compact wearable device. The index reacts to both acute external and cumulative internal load. The index increased with the number of tasks to be performed concurrently (p = 0.004) and with increased time awake, both after normal sleep (p = 0.002) and sleep restriction (p = 0.004). Moreover, the increase of the index was more pronounced in the afternoon after sleep restriction (p = 0.006). As a measure of brain state and its dynamics, the index can be considered equivalent to the heartbeat, an indicator of the cardiovascular state, thus inspiring the name "brainbeat".
This study examined the relation between sleep quality and cognitive performance in older adults, controlling for common medical
comorbidities. Participants were community volunteers who, while not selected on the basis of their sleep, did report substantial
variability in sleep quality. Good and poor sleepers differed on tests of working memory, attentional set shifting, and abstract
problem solving but not on processing speed, inhibitory function, or episodic memory. Poor sleep was also associated with
increased depressive symptomatology but only for functional symptoms (e.g., decreased concentration) and not for mood (e.g.,
sadness). The relationships between sleep quality and cognition were not explained by confound factors such as cerebrovascular
disease, depression, or medication usage. Sleep problems may contribute to performance variability between elderly individuals
but only in certain cognitive domains.
A physiological measure of processing load or "mental effort" required to perform a cognitive task should accurately reflect within-task, between-task, and between-individual variations in processing demands. The present article reviews all available experimental data on task-evoked pupillary response. It is concluded that the task-evoked pupillary response fulfills the criteria. Alternative explanations are considered and rejected. Implications for neurophysiological and cognitive theories of processing resources are discussed. (47 ref)
Air traffic controllers are required to perform complex tasks which require attention and high precision. This study investigates how the difficulty of such tasks influences emotional states, cognitive workload and task performance. We use quantitative and qualitative measurements, including the recording of pupil dilation and changes in affect using questionnaires. Participants were required to perform a number of air traffic control tasks using the immersive human accessible Virtual Reality space in the "eXperience Induction Machine".
Based on the data collected, we developed and validated a model which integrates personality, workload and affective theories. Our results indicate that the difficulty of an air traffic control task has a direct influence on cognitive workload as well as on the self-reported mood; whereas both mood and workload seem to change independently. In addition, we show that personality, in particular neuroticism, affects both mood and performance of the participants.
This article provides the reader a focused and organised review of the research progresses on neurophysiological indicators, also called “neurometrics”, to show how neurometrics could effectively address some of the most important Human Factors (HFs) needs in the Air Traffic Management (ATM) field. The state of the art on the most involved HFs and related cognitive processes (e.g. mental workload, cognitive training) is presented together with examples of possible applications in the current and future ATM scenarios, in order to better understand and highlight the available opportunities of such neuroscientific applications. Furthermore, the paper will discuss the potential enhancement that further research and development activities could bring to the efficiency and safety of the ATM service.
reviews major categories of empirical workload measurement techniques and provides guidelines for the choice of appropriate assessment procedures for particular applications
sensitivity / diagnosticity
rating scales / psychometric techniques (PsycINFO Database Record (c) 2012 APA, all rights reserved)
In our anxiogenic and stressful world, the maintenance of an optimal cognitive performance is a constant challenge. It is particularly true in complex working environments (e.g. flight deck, air traffic control tower), where individuals have sometimes to cope with a high mental workload and stressful situations. Several models (i.e. processing efficiency theory, cognitive-energetical framework) have attempted to provide a conceptual basis on how human performance is modulated by high workload and stress/anxiety. These models predict that stress can reduce human cognitive efficiency, even in the absence of a visible impact on the task performance. Performance may be protected under stress thanks to compensatory effort, but only at the expense of a cognitive cost. Yet, the psychophysiological cost of this regulation remains unclear. We designed two experiments involving pupil diameter, cardiovascular and prefrontal oxygenation measurements. Participants performed the Toulouse N-back Task that intensively engaged both working memory and mental calculation processes under the threat (or not) of unpredictable aversive sounds. The results revealed that higher task difficulty (higher n level) degraded the performance and induced an increased tonic pupil diameter, heart rate and activity in the lateral prefrontal cortex, and a decreased phasic pupil response and heart rate variability. Importantly, the condition of stress did not impact the performance, but at the expense of a psychophysiological cost as demonstrated by lower phasic pupil response, and greater heart rate and prefrontal activity. Prefrontal cortex seems to be a central region for mitigating the influence of stress because it subserves crucial functions (e.g. inhibition, working memory) that can promote the engagement of coping strategies. Overall, findings confirmed the psychophysiological cost of both mental effort and stress. Stress likely triggered increased motivation and the recruitment of additional cognitive resources that minimize its aversive effects on task performance (effectiveness), but these compensatory efforts consumed resources that caused a loss of cognitive efficiency (ratio between performance effectiveness and mental effort).
The aim of this study was to evaluate the sensitivity of a range of EEG indices to time-on-task effects and to a workload manipulation (cueing), during performance of a resource-limited vigilance task. Effects of task period and cueing on performance and subjective state response were consistent with previous vigilance studies and with resource theory. Two EEG indices-the Task Load Index (TLI) and global lower frequency (LF) alpha power-showed effects of task period and cueing similar to those seen with correct detections. Across four successive task periods, the TLI declined and LF alpha power increased. Cueing increased TLI and decreased LF alpha. Other indices-the Engagement Index (EI), frontal theta and upper frequency (UF) alpha failed to show these effects. However, EI and frontal theta were sensitive to interactive effects of task period and cueing, which may correspond to a stronger anxiety response to the uncued task.
This effort investigated the ability of a neurophysiological measure to detect changes in workload during a task which is sensitive to cognitive function. A growing collection of research suggests that physiological measures such as EEG can be used to inform the adaptation of systems. However, it has been proposed that such measures often provide a gross interpretation of cognitive workload during complex tasks and are not sensitive to differences in specific cognitive function. To understand the utility of neurophysiological measures for human-machine interaction, we must know if these measures are sensitive to tasks which are sensitive to changes in cognitive function. To begin to answer this question, we investigated the sensitivity of Advanced Brain Monitoring’s EEG-based measures to changes in workload experienced during a Stroop task. Results indicated that ABM’s workload measure can detect changes associated with the attentional demands and cognitive processes linked to the ability to inhibit word naming during tasks involving semantic interference. This indicates that changes in workload associated with the ability to inhibit competing cognitive processes can be identified using neurophysiological workload measures.
Performance on measures of working memory (WM) capacity predicts performance on a wide range of real-world cognitive tasks. I review the idea that WM capacity (a) is separable from short-term memory, (b) is an important component of general fluid intelligence, and (c) represents a domain-free limitation in ability to control attention. Studies show that individual differences in WM capacity are reflected in performance on antisaccade, Stroop, and dichotic-listening tasks. WM capacity, or executive attention, is most important under conditions in which interference leads to retrieval of response tendencies that conflict with the current task.
To better understand the mechanisms by which working memory training can augment human performance we continuously monitored trainees with near infrared spectroscopy (NIRS) while they performed a dual verbal-spatial working memory task. Linear mixed effects models were used to model the changes in cerebral hemodynamic response as a result of time spent training working memory. Nonlinear increases in left dorsolateral prefrontal cortex (DLPFC) and right ventrolateral prefrontal cortex (VLPFC) were observed with increased exposure to working memory training. Adaptive and yoked training groups also showed differential effects in rostral prefrontal cortex with increased exposure to working memory training. There was also a significant negative relationship between verbal working memory performance and bilateral VLPFC activation. These results are interpreted in terms of decreased proactive interference, increased neural efficiency, reduced mental workload for stimulus processing, and increased working memory capacity with training.
BORGHINI et al. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness NEUROSCI BIOBEHAV REV 21(1) XXX-XXX, 2012. This paper reviews published papers related to neurophysiological measurements (electroencephalography: EEG, electrooculography EOG; heart rate: HR) in pilots/drivers during their driving tasks. The aim is to summarise the main neurophysiological findings related to the measurements of pilot/driver's brain activity during drive performance and how particular aspects of this brain activity could be connected with the important concepts of "mental workload", "mental fatigue" or "situational awareness". Review of the literature suggest that exists a coherent sequence of changes for EEG, EOG and HR variables during the transition from normal drive, high mental workload and eventually mental fatigue and drowsiness. In particular, increased EEG power in theta band and a decrease in alpha band occurred in high mental workload. Successively, increased EEG power in theta as well as delta and alpha bands characterize the transition between mental workload and mental fatigue. Drowsiness is also characterized by increased blink rate and decreased HR values. The detection of such mental states is actually performed "off-line" with accuracy around 90% but not on-line. A discussion on the possible future applications of findings provided by these neurophysiological measurements in order to improve the safety of the vehicles will be also presented.