No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
An Evaluation of EEG-based Metrics for Engagement Assessment of Distance Learners
Abstract
Maintaining students' cognitive engagement in educational settings is crucial to their performance, though quantifying this mental state in real-time for distance learners has not been studied extensively in natural distance learning environments. We record electroencephalographic (EEG) data of students watching online lecture videos and use it to predict engagement rated by human annotators. An evaluation of prior EEG-based engagement metrics that utilize power spectral density (PSD) features is presented. We examine the predictive power of various supervised machine learning approaches with both subject-independent and individualized models when using simple PSD feature functions. Our results show that engagement metrics with few power band variables, including those proposed in prior research, do not produce predictions consistent with human observations. We quantify the performance disparity between cross-subject and per-subject models and demonstrate that individual differences in EEG patterns necessitate a more complex metric for educational engagement assessment in natural distance learning environments.
... Learning style preferences: v: visual, a: auditory, r: reading, k: kinesthetic. Continuous update of the working memory is accompanied by higher frontal theta activity, while mental effort causes a decrease in posterior alpha activity, resulting in higher cognitive load values (Booth et al., 2018). There is an optimal level of cognitive load, which ensures sufficient mental resources without the overload of the working memory. ...
... We also measured the participant's engagement level since learning outcomes highly depend on the continuous maintenance of a high level of engagement (Booth et al., 2018). In our study, we did not find any change in engagement level, which result could be easily explained by the fact that probably only those students who volunteered for the EEG measurement were already motivated and tried to perform well. ...
Affordable, portable, and easy-to-use devices, as well as computer programs that allow the quantitative analysis of EEG signals, provide the opportunity for wide-ranging educational applications of EEG. However, the use of EEG data is complicated by the presence of significant individual differences in resting activity; therefore, this technology is not suitable for analyzing educational processes on its own. However, when used together with other methods, it could provide valuable data on learning-related brain activity. In our study, we used the Emotiv EPOC EEG headset and monitored brain activity during closed- and open-eye relaxation, reading, and arithmetic calculation. We used the Fourier Transform (FFT) to analyze the spectral content of the EEG signal, determined the power of the theta, alpha, and beta EEG bands, and then calculated the attention, engagement, and cognitive load values for each task. Participants also solved a test related to the tasks, evaluated their performance, and filled out a questionnaire about their impressions of the measurement. We made our conclusions by combining the data from the EEG, the test, and the questionnaire results. We found an increase in the alpha power related to tiredness, increased attention during reading, and high cognitive load during calculation, especially when the result was correct. Participants diagnosed with dyslexia showed lower alpha peak frequency during open-eye relaxation and increased theta activity during reading. Although they needed more time to complete the reading task, their test results were similar to the results of the control group. Our results suggest several possible usage of EEG in education. It enables the continuous monitoring of alertness, can detect fatigue, and helps keep task difficulty at an optimal level. However, the beta theta ratio used in this study to determine attention level does not seem suitable for monitoring sustained attention. EEG can also give us useful insight into the performance of students with special needs, helping teachers provide adequate tasks and guidance.
... 10 School Student [165]; [166]; [167]; [168]; [169]; [170]; [171]; [172]; [173]; [174]; [175]; [176]; [177]; [178]; [179]; [180]; [181]; [182]; [183]; [184]; [185]; [186]; [187]; [120]; [35]; [188]; [189]; [190]; [191]; [192] 30 University / College Student [193]; [194]; [195]; [196]; [197]; [198]; [199]; [200]; [201]; [202]; [203]; [204]; [205]; [206]; [207]; [48]; [208]; [209]; [115]; [210]; [211]; [212]; [213]; [214]; [215]; [167]; [216]; [217]; [218]; [219]; [220]; [119]; [221]; [222]; [108]; [223]; [224]; [225]; [226]; [227]; [228]; [176]; [229]; [230]; [231]; [232]; [233]; [234]; [235]; [236]; [237]; [238]; [239]; [240]; [241]; [242]; [243]; [244]; [245]; [246]; [247]; [248]; [249]; [250]; [251]; [139]; [135]; [112]; [138]; [252]; [253]; [254]; [255]; [256]; [257] [258]; [259]; [136]; [260]; [137]; [261]; [262]; [263]; [264]; [265]; [266]; [267]); [268]; [114]; [269]; [270]; [117]; [271]; [116]; [272]; [273]; [141]; [274]; [143]; [275]; [276]; [144]; [277]; [278]; [279]; [280]; [281]; [282]; [283]; [284]; [285]; [145]; [286]; [287]; [288]; [289]; [290]; [291]; [292]; [293]; [294]; [295]; [296]; [297]; [298]; [299]; [300]; [301]; [302]; [192] ; [303]; [304]. ...
... School Student [310]; [311]; [190]. [118]; [124]; [165]; [166]; [167]; [168]; [169]; [170]; [171]; [193]; G [194]; [195]; [196]; [197]; [198]; [199]; [200]; [201]; [202]; [203]; [204]; [205]; [206]; [207]; [48]; [208]; [209]; [115]; [210]; [211]; [212]; [213]; [214]; [215]; [305]; [105]; [128]; [129]; [130]; [306]; [307]; [242]; [216]; [217]; [218]; [219]; [220]; [119]; [221]; [222]; [172]; [173]; [174]; [160]; [108]; [223]; [175]; [224]; [225]; [226]; [227]; [228]; [176]; [110]; [229]; [230]; [231]; [232]; [177]; [125]; [233]; [234]; [235]; [236]; [237]; [238]; [239]; [240]; [241]; [107]; [243]; [244]; [245]; [246]; [247]; [248]; [249]; [250]; [178]; [179]; [180]; [161]; [162] ; [132]; [308]; [113]; [111]; [310]; [311]; [309]; [290]; [292]; [294]; [295]; [296]; [297]; [298]; [188]; [299]; [163]; [300]; [301]; [302]; [190]; [192] ; [303]; [164]. ...
Brain–computer interface (BCI) technology has the potential to positively contribute to the educational learning environment, which faces many challenges and shortcomings. Cognitive and affective BCIs can offer a deep understanding of brain mechanisms, which may improve learning strategies and increase brain-based skills. They can offer a better empirical foundation for teaching–learning methodologies, including adjusting learning content based on brain workload, measuring student interest of a topic, or even helping students focus on specific tasks. The latest findings from emerging BCI technology, neuroscience, cognitive sciences, and psychology could be used in learning and teaching strategies to improve student abilities in education. This study investigates and analyzes the research on BCI patterns and its implementation for enhancing cognitive capabilities of students. The results showed that there is insufficient literature on BCI that addresses students with disabilities in the learning process. Further, our analysis revealed a bias toward the significance of cognitive process factors compared with other influential factors, such as the learning environment and emotions that influence learning. Finally, we concluded that BCI technology could improve students’ learning and cognitive skills—when consistently associated with the different pedagogical teaching–learning strategies—for better academic achievement.
... Uses Continuous annotations have been used to represent subjective human constructs such as dimensional affect (e.g., valence and arousal Abadi et al. 2015;Koelstra et al. 2012;Kossaifi et al. 2019;McKeown et al. 2011;Metallinou and Narayanan 2013;Ringeval et al. 2013;Soleymani et al. 2011), challenge/immersion (e.g., Beaudoin-Gagnon et al. 2019), and student engagement (e.g., Booth et al. 2017Booth et al. , 2018. These annotations help uncover latent information about the dynamics of human mental states and aid in modeling and understanding human perception of complex constructs. ...
Accurately representing changes in mental states over time is crucial for understanding their complex dynamics. However, there is little methodological research on the validity and reliability of human-produced continuous-time annotation of these states. We present a psychometric perspective on valid and reliable construct assessment, examine the robustness of interval-scale (e.g., values between zero and one) continuous-time annotation, and identify three major threats to validity and reliability in current approaches. We then propose a novel ground truth generation pipeline that combines emerging techniques for improving validity and robustness. We demonstrate its effectiveness in a case study involving crowd-sourced annotation of perceived violence in movies, where our pipeline achieves a .95 Spearman correlation in summarized ratings compared to a .15 baseline. These results suggest that highly accurate ground truth signals can be produced from continuous annotations using additional comparative annotation (e.g., a versus b) to correct structured errors, highlighting the need for a paradigm shift in robust construct measurement over time.
... High accuracies were obtained based on all the IMU models, PPG models, and combined models, which is significantly higher than previous studies based on research-grade devices and complex machine learning models [36], [37]. The PPGbased HRV features, which could reflect physiological fluctuation rather than behavioral performance, are also able to address the engagement state in classification models. ...
... The most widely used feature in engagement level classification is spectral power. Booth et al. [29] predicted engagement utilizing power spectral density (PSD) features. The PSD features in δ (1-4 Hz), θ (4-8 Hz), α (8)(9)(10)(11)(12)(13), and β (13-30 Hz) were extracted as features to feed into classifiers to predict engagement level. ...
Engagement ability plays a fundamental role in allocating attentional resources and helps us perform daily tasks efficiently. Therefore, it is of great importance to recognize engagement level. Electroencephalography is frequently employed to recognize engagement for its objective and harmless nature. To fully exploit the information contained in EEG signals, an engagement recognition method integrating multi-domain information is proposed. The proposed method extracts frequency information by a filter bank. In order to utilize spatial information, the correlation-based common spatial patterns method is introduced and extended into three versions by replacing different correlation coefficients. In addition, the Hilbert transform helps to obtain both amplitude and phase information. Finally, features in three domains are combined and fed into a support vector machine to realize engagement recognition. The proposed method is experimentally validated on an open dataset composed of 29 subjects. In the comparison with six existing methods, it achieves the best accuracy of 87.74±5.98% in binary engagement recognition with an improvement of 4.03%, which proves its efficiency in the engagement recognition field.
... In a recent example, an adaptive tutoring system based on EEG measures of cognitive engagement and load had a positive impact on learning outcomes (Chaouachi et al., 2019). However, other studies reported that attention classifiers are not easily generalizable across students (Dhindsa et al., 2019) and are not predictive of how student engagement is rated by annotators (Booth et al., 2018). These findings call for additional research on how EEG measures relate to student engagement. ...
Video lectures are commonly used in online and flipped courses, but students often find it challenging to stay engaged and retain lecture content. The current study examined to what extent the power of electroencephalography (EEG) brain activity in the theta (4–7 Hz), alpha (8–12 Hz) and beta (13–20 Hz) bands can dynamically capture fluctuations of student engagement and retention throughout pre‐recorded lectures. EEG activity was recorded from 33 college students throughout four video‐based chemistry lectures. In‐video probes were used to assess both student engagement and content retention at random moments during the video. Our findings reveal that there are significant fluctuations in self‐reported engagement throughout pre‐recorded lectures. Further, among the three frequency bands that were tested, only alpha power closely tracked fluctuations in self‐reported engagement at the individual student level. In‐lecture fluctuations in engagement were associated with content retention, but content retention itself was not well captured by EEG activity in any of the frequency bands that were examined. These findings suggest that the design of video lectures should consider fluctuations in student engagement and potentially incorporate self‐reported and physiological indicators of engagement. Future research should further investigate how EEG and other physiological engagement indicators can be used in real time to personalize online instruction.
Practitioner Notes
What is already known about this topic Students often find it challenging to stay engaged during online lectures and retain lecture content.
Measuring engagement and retention throughout an online lecture (rather than only at its end) is important but challenging because it requires the insertion of in‐lecture questions that interrupt the learning process.
Electroencephalography (EEG) could potentially provide a continuous and implicit measure of engagement and retention throughout online lectures.
What this paper adds Self‐reported engagement tends to gradually decrease throughout the duration of video lectures with substantial variation both within and between students.
Fluctuations in student engagement are predictive of content retention throughout video lectures.
EEG power in the alpha band (8–12 Hz) dynamically tracks fluctuations in student engagement.
EEG power in the alpha band significantly predicts overall lecture engagement as well as learning confidence. However, EEG power might not be sensitive to variations in post‐lecture test performance.
Implications for practice and/or policy The design of online lectures should take into consideration the dynamic and idiosyncratic nature of student engagement.
In‐video self‐report probes and EEG power measures can be useful sources of information on students' level of engagement during online lectures.
It should be further investigated whether EEG and other physiological indicators of engagement can be used in real time to personalize online instruction.
... (2) Cross-task methods are innovatively applied to crosssubject research. In 2016, Touryan et al. studied the PSD features of cross-task EEG signals (Touryan et al., 2016), and 2 years later, Booth et al. carried out in-depth research on the PSD features of cross-subject EEG signals (Booth et al., 2018). ...
... The BCI technology can help monitor students' learning approaches, which can be influenced by how they are supervised, act, and study the given material. Teachers or instructors using online platforms to keep track of and react to students who are struggling to concentrate in virtual learning environments could lead to a reduction in drop-out rates and increase students learning outcomes [1]. This paper proposes a new way to obtain actual data by exploring a multifunctional neuro-feedback with a BCI approach by monitoring and analyzing students' brain waves/signals. ...
... On the other hand, the rapid development in sensory technology has enabled researchers and practitioners to push the boundaries of learning engagement detection and its analysis by investigating various machine-readable signals or behaviors, such as electroencephalogram (EEG) signal, physiological signal, electrodermal activity, facial expression, gaze, keystroke and mouse movement [17]. In order to obtain high quality data generated from the learning activities, intrusive or wearable devices such as electrode headset [18], wristband [19], or costly sophisticated eye tracker [20] have to be adopted. The adoption of these devices may cause inconvenience or discomfort to learners during data acquisition process. ...
“Lack of supervision” is a particularly challenging problem in E-learning or distance learning environments. A wide range of research efforts and technologies have been explored to alleviate its impact by monitoring students’ engagement, such as emotion or learning behaviors. However, the current research still lacks multi-dimensional computational measures for analyzing learner’s engagement from the interactions that occur in digital learning environment. In this paper, we propose an integrated framework to identify learning engagement from three facets: affect, behavior and cognitive state, which are conveyed by learner’s facial expressions, eye movement behaviors and the overall performance during short video learning session. To recognize the three states of learners, three channel data is recorded: 1) video/image sequence captured by camera; 2) eye movement information from a non-intrusive and cost-effective eye tracker; and 3) click stream data from mouse. Based on these modalities, a multi-channel data fusion strategy is designed that concatenates time series features of three channels in the same time segment to predict course learning performance. We also presented a new method to make the self-reported annotations more reliable without using external observers’ verification. To validate the approach and methods, 46 participants were invited to attend a representative on-line course that consists of short videos in our designed learning environment. The results demonstrated the effectiveness of the proposed framework and methods in monitoring learning engagement. More importantly, a prototype system was developed to detect learner’s emotional and eye behavioral engagement in real-time as well as predict the learning performance of learners after they had completed each short video course.
Student characteristics affect their willingness and ability to acquire new knowledge. Assessing and identifying the effects of student characteristics is important for online educational systems. Machine learning (ML) is becoming significant in utilizing learning data for student modeling, decision support systems, adaptive systems, and evaluation systems. The growing need for dynamic assessment of student characteristics in online educational systems has led to application of machine learning methods in modeling the characteristics. Being able to automatically model student characteristics during learning processes is essential for dynamic and continuous adaptation of teaching and learning to each student's needs. This paper provides a review of 8 years (from 2015 to 2022) of literature on the application of machine learning methods for automatic modeling of various student characteristics. The review found six student characteristics that can be modeled automatically and highlighted the data types, collection methods, and machine learning techniques used to model them. Researchers, educators, and online educational systems designers will benefit from this study as it could be used as a guide for decision-making when creating student models for adaptive educational systems. Such systems can detect students' needs during the learning process and adapt the learning interventions based on the detected needs. Moreover, the study revealed the progress made in the application of machine learning for automatic modeling of student characteristics and suggested new future research directions for the field. Therefore, machine learning researchers could benefit from this study as they can further advance this area by investigating new, unexplored techniques and find new ways to improve the accuracy of the created student models.
Language learning has long been a topic of interest, and instructional videos which allow students to learn anywhere and anytime have become an important language learning tool. However, the emotional characteristics of both instructors and students, which have the potential to influence students’ second language learning from instructional videos, have yet to be fully explored. The current study investigated the interaction effects of an instructor’s emotions (positive vs. negative vs. neutral) and students’ emotional intelligence (low vs. high) on students’ second language vocabulary learning from instructional videos with consideration of attention paid to the learning material (i.e., average fixation time, referring to the duration of each fixation on the learning material), learning experience (i.e., motivation, engagement, interaction), and learning performance (both immediate and delayed). Results showed that (1) only the interaction effect on attention was verified, and that (2) students with high emotional intelligence showed a larger average fixation time in the positive condition than in the negative condition, while (3) students with low emotional intelligence showed a smaller average fixation time in the neutral condition than in the negative condition. Furthermore, the results verified the benefits of the instructor’s positive emotion on students’ motivation, interaction, and immediate performance. Our findings shine a light onto the influence of an instructor’s emotions and students’ emotional intelligence on second language learning, and provide practical implications for the design of instructional videos and second language learning.
The objective of this experiment was to determine the best possible input EEG feature for classification of the workload while designing load balancing logic for an automated operator. The input features compared in this study consisted of spectral features of Electroencephalography, objective scoring and subjective scoring. Method utilizes to identify best EEG feature as an input in Neural Network Classifiers for workload classification, to identify channels which could provide classification with the highest accuracy and for identification of EEG feature which could give discrimination among workload level without adding any classifiers. The result had shown Engagement Index is the best feature for neural network classification.
Massive open online courses (MOOCs) have received wide publicity and many institutions have invested considerable effort in developing, promoting and delivering such courses. However, there are still many unresolved questions relating to MOOCs and their effectiveness. One of the major recurring issues raised in both academic literature and the popular press is the consistently high dropout rate of MOOC learners. Although many thousands of participants enrol on these courses, the completion rate for most courses is below 13%. This paper investigates MOOC attrition from several different perspectives. Firstly, we review existing literature relating to MOOC dropout rates, bringing together existing findings on completion rates and analyses of several specific courses, which identify factors that correlate to likelihood of dropout. We provide a meta-analysis of the basic figures on overall dropout rates previously collected to identify relationships between course factors and dropout rates. In addition, the literature is reviewed from a qualitative perspective drawing together findings on the reasons for dropout and methods suggested for resolving or reducing the dropout rate. Secondly, using themes emerging from the initial investigation, we provide a preliminary analysis of data gathered from a Computing MOOC run by the University of Warwick, UK and presented using a Moodle platform. Different aspects of students' demographic data are examined to see if relationships to persistence exist. An important feature of this course is that it has been run in two different parallel modes (" traditional " MOOC mode with peer support, and " supported " mode with real time, tutored programming labs). This allows direct comparison between the dropout figures for the two different modes. Qualitative information from student evaluations is also considered. Finally, we discuss our findings relating MOOC dropout rates, considering what factors are within the control of a MOOC provider and suggesting the most promising avenues for improvement. Our results indicate that many participants who may be classed as dropouts (for example, because they do not complete the necessary components to gain a certificate) are still participating in the course in their own preferred way (either at a slower pace or with selective engagement). This suggests that the structure of " a course " may not be helpful to all participants and supporting different patterns of engagement and presentation of material may be beneficial.
Magnetoencephalography and electroencephalography (M/EEG) measure the weak electromagnetic signals generated by neuronal activity in the brain. Using these signals to characterize and locate neural activation in the brain is a challenge that requires expertise in physics, signal processing, statistics, and numerical methods. As part of the MNE software suite, MNE-Python is an open-source software package that addresses this challenge by providing state-of-the-art algorithms implemented in Python that cover multiple methods of data preprocessing, source localization, statistical analysis, and estimation of functional connectivity between distributed brain regions. All algorithms and utility functions are implemented in a consistent manner with well-documented interfaces, enabling users to create M/EEG data analysis pipelines by writing Python scripts. Moreover, MNE-Python is tightly integrated with the core Python libraries for scientific comptutation (NumPy, SciPy) and visualization (matplotlib and Mayavi), as well as the greater neuroimaging ecosystem in Python via the Nibabel package. The code is provided under the new BSD license allowing code reuse, even in commercial products. Although MNE-Python has only been under heavy development for a couple of years, it has rapidly evolved with expanded analysis capabilities and pedagogical tutorials because multiple labs have collaborated during code development to help share best practices. MNE-Python also gives easy access to preprocessed datasets, helping users to get started quickly and facilitating reproducibility of methods by other researchers. Full documentation, including dozens of examples, is available at http://martinos.org/mne.
Changes in cortical activity during working memory tasks were examined with electroencephalograms (EEGs) sampled from 115 channels and spatially sharpened with magnetic resonance imaging (MRI)-based finite element deblurring. Eight subjects performed tasks requiring comparison of each stimulus to a preceding one on verbal or spatial attributes. A frontal midline theta rhythm increased in magnitude with increased memory load. Dipole models localized this signal to the region of the anterior cingulate cortex. A slow (low-frequency), parietocentral, alpha signal decreased with increased working memory load. These signals were insensitive to the type of stimulus attribute being processed. A faster (higher-frequency), occipitoparietal, alpha signal was relatively attenuated in the spatial version of the task, especially over the posterior right hemisphere. Theta and alpha signals increased, and overt performance improved, after practice on the tasks. Increases in theta with both increased task difficulty and with practice suggests that focusing attention required more effort after an extended test session. Decreased alpha in the difficult tasks indicates that this signal is inversely related to the amount of cortical resources allocated to task performance. Practice-related increases in alpha suggest that fewer cortical resources are required after skill development. These results serve: (i) to dissociate the effects of task difficulty and practice; (ii) to differentiate the involvement of posterior cortex in spatial versus verbal tasks; (iii) to localize frontal midline theta to the anteromedial cortex; and (iv) to demonstrate the feasibility of using anatomical MRIs to remove the blurring effect of the skull and scalp from the ongoing EEG. The results are discussed with respect to those obtained in a prior study of transient evoked potentials during working memory.
Two experiments are presented examining adaptive and adaptable methods for invoking automation.
Empirical investigations of adaptive automation have focused on methods used to invoke automation or on automation-related performance implications. However, no research has addressed whether performance benefits associated with brain-based systems exceed those in which users have control over task allocations.
Participants performed monitoring and resource management tasks as well as a tracking task that shifted between automatic and manual modes. In the first experiment, participants worked with an adaptive system that used their electroencephalographic signals to switch the tracking task between automatic and manual modes. Participants were also divided between high- and low-reliability conditions for the system-monitoring task as well as high- and low-complacency potential. For the second experiment, participants operated an adaptable system that gave them manual control over task allocations.
Results indicated increased situation awareness (SA) of gauge instrument settings for individuals high in complacency potential using the adaptive system. In addition, participants who had control over automation performed more poorly on the resource management task and reported higher levels of workload. A comparison between systems also revealed enhanced SA of gauge instrument settings and decreased workload in the adaptive condition.
The present results suggest that brain-based adaptive automation systems may enhance perceptual level SA while reducing mental workload relative to systems requiring user-initiated control.
Potential applications include automated systems for which operator monitoring performance and high-workload conditions are of concern.
The ability to continuously and unobtrusively monitor levels of task engagement and mental workload in an operational environment could be useful in identifying more accurate and efficient methods for humans to interact with technology. This information could also be used to optimize the design of safer, more efficient work environments that increase motivation and productivity.
The present study explored the feasibility of monitoring electroencephalo-graphic (EEG) indices of engagement and workload acquired unobtrusively and quantified during performance of cognitive tests. EEG was acquired from 80 healthy participants with a wireless sensor headset (F3-F4,C3-C4,Cz-POz,F3-Cz,Fz-C3,Fz-POz) during tasks including: multi-level forward/backward-digit-span, grid-recall, trails, mental-addition, 20-min 3-Choice Vigilance, and image-learning and memory tests. EEG metrics for engagement and workload were calculated for each 1 -s of EEG.
Across participants, engagement but not workload decreased over the 20-min vigilance test. Engagement and workload were significantly increased during the encoding period of verbal and image-learning and memory tests when compared with the recognition/ recall period. Workload but not engagement increased linearly as level of difficulty increased in forward and backward-digit-span, grid-recall, and mental-addition tests. EEG measures correlated with both subjective and objective performance metrics.
These data in combination with previous studies suggest that EEG engagement reflects information-gathering, visual processing, and allocation of attention. EEG workload increases with increasing working memory load and during problem solving, integration of information, analytical reasoning, and may be more reflective of executive functions. Inspection of EEG on a second-by-second timescale revealed associations between workload and engagement levels when aligned with specific task events providing preliminary evidence that second-by-second classifications reflect parameters of task performance.
Advances in affective computing technologies have made it possible for researchers to investigate brain function while users interact in virtual environments. Progress in sensors and algorithms for off-the-shelf EEG systems has made it possible for gaming researchers to perform real-time estimation of human cognitive and affective states using EEG. In this study our aim was to coordinate " Task Engagement " data with " Arousal-Valence " data. The resulting coordinate was designed for application to expressive transformations to video game play in real time by tuning different performance parameters in an Engagement-Arousal rule system. Results revealed that the engagement index (Beta / (Alpha + Theta)) was capable of differentiating high intensity game events (Player Death) from general game play. Given that higher levels of engagement during death events may reflect increase in autonomic response, we also measured arousal by using (BetaF3 + BetaF4) / (AlphaF3 + AlphaF4) and valence using (AlphaF4 / BetaF4)-(AlphaF3 / BetaF3). Results revealed that arousal increases and valence decreases during high intensity game events (Player Death) when compared to lower intensity game events (General Game Play). Given our desire to establish " Task Engagement " data with " Arousal-Valence " coordinates for a flow model, we divided the data into quartiles, which allowed us to establish upper and lower thresholds to indicate when the player has left a state of flow. Our aim was to use an off-the-shelf EEG system to establish " Task Engagement " and " Arousal-Valence " coordinates during video game play that can be used for a flow model. It is believed that this model will allow for future use of the Emotiv for assessing the cognitive and emotional processing of the player.
A bio-cybernetic, closed-loop system was validated for use in an adaptive automation environment. Subjects were asked to perform either a single task or multiple tasks from the Multi-Attribute Task Battery. EEG was continuously sampled while they performed the task(s) and an EEG index was derived (20 Beta/Alpha + Theta). The system switched between manual and automatic modes according to the level of operator engagement based upon the EEG index. The NASA-TLX was administered after each trial. The results of the study demonstrated that it was possible to moderate an operator's level of engagement through a closed-loop system driven by the operator's EEG. In addition, the system was sensitive to increases in task load. These findings show promise for designing adaptive automation technology around psychophysiological input.
Student engagement is a key concept in contemporary education, where it is valued as a goal in its own right. In this paper we explore approaches for automatic recognition of engagement from students’ facial expressions. We studied whether human observers can reliably judge engagement from the face; analyzed the signals observers use to make these judgments; and automated the process using machine learning. We found that human observers reliably agree when discriminating low versus high degrees of engagement (Cohen’s ). When fine discrimination is required (four distinct levels) the reliability decreases, but is still quite high (). Furthermore, we found that engagement labels of 10-second video clips can be reliably predicted from the average labels of their constituent frames (Pearson r=0.85), suggesting that static expressions contain the bulk of the information used by observers. We used machine learning to develop automatic engagement detectors and found that for binary classification (e.g., high engagement versus low engagement), automated engagement detectors perform with comparable accuracy to humans. Finally, we show that both human and automatic engagement judgments correlate with task performance. In our experiment, student post-test performance was predicted with comparable accuracy from engagement labels (r=0.47) as from pre-test scores (r=0.44).
This text is the second edition of this book. It expands the widely acclaimed 1981 book, filling more gaps between EEG and the physical sciences. EEG opens a "window on the mind" by finding new connections between psychology and physiology. Topics include synaptic sources, electrode placement, choice of reference, volume conduction, power and coherence, projection of scalp potentials to dura surface, dynamic signatures of conscious experience, and neural networks immersed in global fields of synaptic action.
A biocybernetic system has been developed as a method to evaluate automated flight deck concepts for compatibility with human capabilities. A biocybernetic loop is formed by adjusting the mode of operation of a task set (e.g., manual/automated mix) based on electroencephalographic (EEG) signals reflecting an operator's engagement in the task set. A critical issue for the loop operation is the selection of features of the EEG to provide an index of engagement upon which to base decisions to adjust task mode. Subjects were run in the closed-loop feedback configuration under four candidate and three experimental control definitions of an engagement index. The temporal patterning of system mode switching was observed for both positive and negative feedback of the index. The indices were judged on the basis of their relative strength in exhibiting expected feedback control system phenomena (stable operation under negative feedback and unstable operation under positive feedback). Of the candidate indices evaluated in this study, an index constructed according to the formula, beta power/(alpha power + theta power), reflected task engagement best.
The necessity of psychophysiological research in ergonomics has gradually been recognized in Japan. In this paper, frontal midline theta rhythm (Fm-theta) and eyeblinking are recommended as tools in this field, especially for assessing workers' attention concentration, mental workload, fatigue, and interest during VDT work at the workplace and playing video games at home. In experiment 1, Fm-theta and eyeblink rates were measured in 10 Japanese abacus experts (Group E) and 10 normal students (Group C) during a visual search task with VDT. Memory load affected all measures. The amount of Fm-theta appeared more in Group E than Group C, but blink rate was lower in Group E than in Group C. As abacus experts have such highly developed skills in concentration, the result indicates that the amount of Fm-theta would be a good index of attention concentration in VDT workers. The second experiment was done with 10 school-aged children as subjects during three visual tasks: video game, mental test and animation. Amounts of Fm-theta and the degree of blink inhibition were maximum while playing the video game, which all subjects reported they most preferred, and minimum while watching animation, which eight subjects reported to be most boring. An interesting task would seem to provoke Fm-theta and inhibit eyeblink activity. From these two experiments, Fm-theta and eyeblink rate would appear to be good indices of attention concentration and task pleasantness of a mental task using VDT.
The present study was designed to determine whether a biocybernetic, adaptive system could enhance vigilance performance. Participants were asked to monitor the repetitive presentation of white bars on a computer screen for occasional increases in length. An index of task engagement was derived from participants' electroencephalographic (EEG) activity and was used to change the presentation rate of events among 3 values (6, 20, and 60 events/min). Under a negative feedback contingency, event rates increased if the engagement index decreased and, conversely, decreased if the index increased. Under positive feedback, the opposite contingency existed. Each experimental participant had a yoked control partner who received the same pattern of changes in event rates irrespective of his or her EEG activity. The results showed that better vigilance performance was obtained under negative feedback and that the performance of the yoked participants was similar to that of their experimental partners. These findings suggest that it may be possible to improve monitoring performance on critical activities such as air traffic control and radar and sonar operation through a pattern of event rate changes that do not rely on an operator's overt behavior.
The present study examined the effects of an electroencephalographic- (EEG-) based system for adaptive automation on tracking performance and workload. In addition, event-related potentials (ERPs) to a secondary task were derived to determine whether they would provide an additional degree of workload specificity. Participants were run in an adaptive automation condition, in which the system switched between manual and automatic task modes based on the value of each individual's own EEG engagement index; a yoked control condition; or another control group, in which task mode switches followed a random pattern. Adaptive automation improved performance and resulted in lower levels of workload. Further, the P300 component of the ERP paralleled the sensitivity to task demands of the performance and subjective measures across conditions. These results indicate that it is possible to improve performance with a psychophysiological adaptive automation system and that ERPs may provide an alternative means for distinguishing among levels of cognitive task demand in such systems. Actual or potential applications of this research include improved methods for assessing operator workload and performance.
The performance of an adaptive automation system was evaluated using a cognitive vigilance task. Participants responded to the presence of a green "K" in an array of two, five, or nine distractor stimuli during a 40-min vigil. The array with the target stimulus was presented once each minute. Participants EEG was recorded and an engagement index (EI = 20 x beta/(alpha + theta)) was derived. In the negative feedback condition, increases in the EI caused the number of stimuli in the array to decrease while decreases in the EI caused the number of stimuli to increase. For the positive feedback condition, increases in the index caused an increase in the array size (AS) while decreases caused a decrease in the array size. Each experimental participant had a yoked control partner who received the same pattern of changes in array irrespective of their engagement index. A vigilance decrement was seen only for the positive feedback, experimental group.
We describe a set of computational tools able to estimate cortical activity and connectivity from high-resolution EEG and fMRI recordings in humans. These methods comprise the estimation of cortical activity using realistic geometry head volume conductor models and distributed cortical source models, followed by the evaluation of cortical connectivity between regions of interest coincident with the Brodmann areas via the use of Partial Directed Coherence. Connectivity patterns estimated on the cortical surface in different frequency bands are then imaged and interpreted with measures based on graph theory. These computational tools were applied on a set of EEG and fMRI data from a Stroop task to demonstrate the potential of the proposed approach. The present findings suggest that the methodology is able to identify differences in functional connectivity patterns elicited by different experimental tasks or conditions.
EEG based cognitive workload classification during NASA MATB-II multitasking
- Sushil Chandra
- Greeshma Verma
- Alok Sharma
- Devendra Mittal
- Jha
Sushil Chandra, Kundan Lal Verma, Greeshma Sharma, Alok Mittal,
and Devendra Jha, "EEG based cognitive workload classification during NASA MATB-II multitasking," International Journal of Cognitive
Research in Science, Engineering and Education (IJCRSEE), vol. 3,
no. 1, pp. 35-41, 2015.