No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Unraveling the Development of an Algorithm for Recognizing Primary Emotions Through Electroencephalography
Abstract
The large range of potential applications, not only for patients but also for healthy people, that could be achieved by affective brain-computer interface (aBCI) makes more latent the necessity of finding a commonly accepted protocol for real-time EEG-based emotion recognition. Based on wavelet package for spectral feature extraction, attending to the nature of the EEG signal, we have specified some of the main parameters needed for the implementation of robust positive and negative emotion classification. Twelve seconds has resulted as the most appropriate sliding window size; from that, a set of 20 target frequency-location variables have been proposed as the most relevant features that carry the emotional information. Lastly, QDA and KNN classifiers and population rating criterion for stimuli labeling have been suggested as the most suitable approaches for EEG-based emotion recognition. The proposed model reached a mean accuracy of 98% (s.d. 1.4) and 98.96% (s.d. 1.28) in a subject-dependent (SD) approach for QDA and KNN classifier, respectively. This new model represents a step forward towards real-time classification. Moreover, new insights regarding subject-independent (SI) approximation have been discussed, although the results were not conclusive.
This study presents a Subject-Aware Transformer-based neural network designed for the Electroencephalogram (EEG) Emotion Recognition task (SATEER), which entails the analysis of EEG signals to classify and interpret human emotional states. SATEER processes the EEG waveforms by transforming them into Mel spectrograms, which can be seen as particular cases of images with the number of channels equal to the number of electrodes used during the recording process; this type of data can thus be processed using a Computer Vision pipeline. Distinct from preceding approaches, this model addresses the variability in individual responses to identical stimuli by incorporating a User Embedder module. This module enables the association of individual profiles with their EEGs, thereby enhancing classification accuracy. The efficacy of the model was rigorously evaluated using four publicly available datasets, demonstrating superior performance over existing methods in all conducted benchmarks. For instance, on the AMIGOS dataset (A dataset for Multimodal research of affect, personality traits, and mood on Individuals and GrOupS), SATEER’s accuracy exceeds 99.8% accuracy across all labels and showcases an improvement of 0.47% over the state of the art. Furthermore, an exhaustive ablation study underscores the pivotal role of the User Embedder module and each other component of the presented model in achieving these advancements.
This study proposes an innovative expert system that uses exclusively EEG signals to diagnose schizophrenia in its early stages. For diagnosing psychiatric/neurological disorders, electroencephalogram (EEG) testing is considered a financially viable, safe, and reliable alternative. Using the reconstructed phase space (RPS) and the continuous wavelet transform, the researchers maximized the differences between the EEG nonstationary signals of normal and schizophrenia individuals, which cannot be observed in the time, frequency, or time-frequency domains. This reveals significant information, highlighting more distinguishable features. Then, a deep learning network was trained to enhance the accuracy of the resulting image classification. The algorithm’s efficacy was confirmed through three distinct methods: employing 70% of the dataset for training, 15% for validation, and the remaining 15% for testing. This was followed by a 5-fold cross-validation technique and a leave-one-out classification approach. Each method was iterated 100 times to ascertain the algorithm’s robustness. The performance metrics derived from these tests — accuracy, precision, sensitivity, F1 score, Matthews correlation coefficient, and Kappa — indicated remarkable outcomes. The algorithm demonstrated steady performance across all evaluation strategies, underscoring its relevance and reliability. The outcomes validate the system’s accuracy, precision, sensitivity, and robustness by showcasing its capability to autonomously differentiate individuals diagnosed with schizophrenia from those in a state of normal health.
Currently, brain-computer interface technology still poses hidden dangers in the complete control of ideas. Additionally, there are issues with the low sampling frequency and accuracy of EEG signal acquisition equipment. To address these concerns, this study proposes a combined model for EEG signal recognition and classification analysis by combining frequency division multi-feature complex brain networks with parallel convolutional neural networks. The effectiveness of this model has been verified. In model visualization analysis, the visualization results of t-distribution random neighborhood embedding in the third separable convolutional layer indicate that the two types of imagination have already experienced separation. There is a clear boundary between the two at position 0 on both the horizontal and vertical axes. This is a significant improvement compared to the comparative model. In the model performance verification, the full band classification accuracy in the synchronous network was maintained between 60% to 84%, and the μ-rhythm was maintained between 59% to 81%. The average classification accuracy of the combined model was 77.40% with higher performance, which was higher than 68.53% and 70.87% of the single scale convolutional neural network. In comparison with deep learning algorithms, the average classification accuracy of the combined model was 85.74%, much higher than the 66.20% and 76.69% of the comparative models. The composite model constructed has good performance in recognizing and classifying electroencephalogram signals. It can be effectively applied in practical brain-computer interface technology or electroencephalogram signal analysis.
Six hundred eighty seven individuals ages 25–103 years were studied cross-sectionally to examine the relationship between measures of sensory functioning (visual and auditory acuity) and intelligence (14 cognitive tasks representing a 5-factor space of psychometric intelligence). As predicted, the average proportion of individual differences in intellectual functioning connected to sensory functioning increased from 11% in adulthood (25–69 years) to 31% in old age (70–103 years). However, the link between fluid intellectual abilities and sensory functioning, albeit of different size, displayed a similarly high connection to age in both age groups. Several explanations are discussed, including a “common cause” hypothesis. In this vein, we argue that the increase in the age-associated link between sensory and intellectual functioning may reflect brain aging and that the search for explanations of cognitive aging phenomena would benefit from attending to factors that are shared between the 2 domains.
Few studies in the literature have researched the use of surface electromyography (sEMG) for motor assessment post-stroke due to the complexity of this type of signal. However, recent advances in signal processing and machine learning have provided fresh opportunities for analyzing complex, non-linear, non-stationary signals, such as sEMG. This paper presents a method for identification of the upper limb movements from sEMG signals using a combination of digital signal processing, that is discrete wavelet transform, and the enhanced probabilistic neural network (EPNN). To explore the potential of sEMG signals for monitoring motor rehabilitation progress, this study used sEMG signals from a subset of movements of the Arm Motor Ability Test (AMAT) as inputs into a movement classification algorithm. The importance of a particular frequency domain feature, that is the ratio of the mean absolute values between sub-bands, was discovered in this work. An average classification accuracy of 75.5% was achieved using the proposed approach with a maximum accuracy of 100%. The performance of the proposed method was compared with results obtained using three other classification algorithms: support vector machine (SVM), k-Nearest Neighbors (k-NN), and probabilistic neural network (PNN) in terms of sEMG movement classification. The study demonstrated the capability of using upper limb sEMG signals to identify and distinguish between functional movements used in standard upper limb motor assessments for stroke patients. The classification algorithm used in the proposed method, EPNN, outperformed SVM, k-NN, and PNN.
In order to develop more precise and functional affective applications, it is necessary to achieve a balance between the psychology and the engineering applied to emotions. Signals from the central and peripheral nervous systems have been used for emotion recognition purposes, however, their operation and the relationship between them remains unknown. In this context, in the present work, we have tried to approach the study of the psychobiology of both systems in order to generate a computational model for the recognition of emotions in the dimension of valence. To this end, the electroencephalography (EEG) signal, electrocardiography (ECG) signal and skin temperature of 24 subjects have been studied. Each methodology has been evaluated individually, finding characteristic patterns of positive and negative emotions in each of them. After feature selection of each methodology, the results of the classification showed that, although the classification of emotions is possible at both central and peripheral levels, the multimodal approach did not improve the results obtained through the EEG alone. In addition, differences have been observed between cerebral and peripheral responses in the processing of emotions by separating the sample by sex; though, the differences between men and women were only notable at the peripheral nervous system level.
Objective. Most current electroencephalography (EEG)-based brain–computer interfaces (BCIs) are based on machine learning algorithms. There is a large diversity of classifier types that are used in this field, as described in our 2007 review paper. Now, approximately ten years after this review publication, many new algorithms have been developed and tested to classify EEG signals in BCIs. The time is therefore ripe for an updated review of EEG classification algorithms for BCIs. Approach. We surveyed the BCI and machine learning literature from 2007 to 2017 to identify the new classification approaches that have been investigated to design BCIs. We synthesize these studies in order to present such algorithms, to report how they were used for BCIs, what were the outcomes, and to identify their pros and cons. Main results. We found that the recently designed classification algorithms for EEG-based BCIs can be divided into four main categories: adaptive classifiers, matrix and tensor classifiers, transfer learning and deep learning, plus a few other miscellaneous classifiers. Among these, adaptive classifiers were demonstrated to be generally superior to static ones, even with unsupervised adaptation. Transfer learning can also prove useful although the benefits of transfer learning remain unpredictable. Riemannian geometry-based methods have reached state-of-the-art performances on multiple BCI problems and deserve to be explored more thoroughly, along with tensor-based methods. Shrinkage linear discriminant analysis and random forests also appear particularly useful for small training samples settings. On the other hand, deep learning methods have not yet shown convincing improvement over state-of-the-art BCI methods. Significance. This paper provides a comprehensive overview of the modern classification algorithms used in EEG-based BCIs, presents the principles of these methods and guidelines on when and how to use them. It also identifies a number of challenges to further advance EEG classification in BCI.
The present article highlights important concepts of personality including stability issues from the perspective of situational demands and stability over the life-course. Following this more introductory section, we argue why individual differences in primary emotional systems may represent the phylogenetically oldest parts of human personality. Our argumentation leads to the need to increasingly consider individual differences in the raw affects/emotions of people to understand human personality in a bottom–up fashion, which can be coordinated with top–down perspectives. In support of this idea, we also review existing evidence linking individual differences in primal emotions as assessed with the Affective Neuroscience Personality Scales and the widely accepted Big Five Model of Personality. In this context, we provide additional evidence on the link between primal emotions and personality in German and Chinese sample populations. In short, this article addresses evolutionary perspectives in the evaluation of human personality, highlighting some of the ancestral emotional urges that probably still control variations in the construction of human personality structures. Moreover, we address how individual differences in primary emotional systems can illuminate linkages to major human psychopathologies and the potential advantages and disadvantages of carrying a certain personality trait within certain cultural/environmental niches.
Although emotion detection using electroencephalogram (EEG) data has become a highly active area of research over the last decades, little attention has been paid to stimulus familiarity, a crucial subjectivity issue. Using both our experimental data and a sophisticated database (DEAP dataset), we investigated the effects of familiarity on brain activity based on EEG signals. Focusing on familiarity studies, we allowed subjects to select the same number of familiar and unfamiliar songs; both resulting datasets demonstrated the importance of reporting self-emotion based on the assumption that the emotional state when experiencing music is subjective. We found evidence that music familiarity influences both the power spectra of brainwaves and the brain functional connectivity to a certain level. We conducted an additional experiment using music familiarity in an attempt to recognize emotional states; our empirical results suggested that the use of only songs with low familiarity levels can enhance the performance of EEG-based emotion classification systems that adopt fractal dimension or power spectral density features and support vector machine, multilayer perceptron or C4.5 classifier. This suggests that unfamiliar songs are most appropriate for the construction of an emotion recognition system.
Technically, a feature represents a distinguishing property, a recognizable measurement, and a functional component obtained from a section of a pattern. Extracted features are meant to minimize the loss of important information embedded in the signal. In addition, they also simplify the amount of resources needed to describe a huge set of data accurately. This is necessary to minimize the complexity of implementation, to reduce the cost of information processing, and to cancel the potential need to compress the information. More recently, a variety of methods have been widely used to extract the features from EEG signals, among these methods are time frequency distributions (TFD), fast fourier transform (FFT), eigenvector methods (EM), wavelet transform (WT), and auto regressive method (ARM), and so on. In general, the analysis of EEG signal has been the subject of several studies, because of its ability to yield an objective mode of recording brain stimulation which is widely used in brain-computer interface researches with application in medical diagnosis and rehabilitation engineering. The purposes of this paper, therefore, shall be discussing some conventional methods of EEG feature extraction methods, comparing their performances for specific task, and finally, recommending the most suitable method for feature extraction based on performance.
Affective states, moods and emotions, are an integral part of human nature: they shape our thoughts, govern the behavior of the individual, and influence our interpersonal relationships. The last decades have seen a growing interest in the automatic detection of such states from voice, facial expression, and physiological signals, primarily with the goal of enhancing human-computer interaction with an affective component. With the advent of brain-computer interface research, the idea of affective brain-computer interfaces (aBCI), enabling affect detection from brain signals, arose. In this article, we set out to survey the field of neurophysiology-based affect detection. We outline possible applications of aBCI in a general taxonomy of brain-computer interface approaches and introduce the core concepts of affect and their neurophysiological fundamentals. We show that there is a growing body of literature that evidences the capabilities, but also the limitations and challenges of affect detection from neurophysiological activity.
Recent theoretical and empirical work has focused on the variability of network dynamics in maturation. Such variability seems to reflect the spontaneous formation and dissolution of different functional networks. We sought to extend these observations into healthy aging. Two different data sets, one EEG (total n = 48, ages 18-72) and one magnetoencephalography (n = 31, ages 20-75) were analyzed for such spatiotemporal dependency using multiscale entropy (MSE) from regional brain sources. In both data sets, the changes in MSE were timescale dependent, with higher entropy at fine scales and lower at more coarse scales with greater age. The signals were parsed further into local entropy, related to information processed within a regional source, and distributed entropy (information shared between two sources, i.e., functional connectivity). Local entropy increased for most regions, whereas the dominant change in distributed entropy was age-related reductions across hemispheres. These data further the understanding of changes in brain signal variability across the lifespan, suggesting an inverted U-shaped curve, but with an important qualifier. Unlike earlier in maturation, where the changes are more widespread, changes in adulthood show strong spatiotemporal dependence.
Factor-analytic evidence has led most psychologists to describe affect as a set of dimensions, such as displeasure, distress, depression, excitement, and so on, with each dimension varying independently of the others. However, there is other evidence that rather than being independent, these affective dimensions are interrelated in a highly systematic fashion. The evidence suggests that these interrelationships can be represented by a spatial model in which affective concepts fall in a circle in the following order: pleasure (0), excitement (45), arousal (90), distress (135), displeasure (180), depression (225), sleepiness (270), and relaxation (315). This model was offered both as a way psychologists can represent the structure of affective experience, as assessed through self-report, and as a representation of the cognitive structure that laymen utilize in conceptualizing affect. Supportive evidence was obtained by scaling 28 emotion-denoting adjectives in 4 different ways: R. T. Ross's (1938) technique for a circular ordering of variables, a multidimensional scaling procedure based on perceived similarity among the terms, a unidimensional scaling on hypothesized pleasure–displeasure and degree-of-arousal dimensions, and a principal-components analysis of 343 Ss' self-reports of their current affective states. (70 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.
In this paper, we summarize the human emotion rec-ognition using different set of electroencephalogram (EEG) channels using discrete wavelet transform. An audio-visual induction based protocol has been de-signed with more dynamic emotional content for in-ducing discrete emotions (disgust, happy, surprise, fear and neutral). EEG signals are collected using 64 electrodes from 20 subjects and are placed over the entire scalp using International 10-10 system. The raw EEG signals are preprocessed using Surface Laplacian (SL) filtering method and decomposed into three different frequency bands (alpha, beta and gamma) using Discrete Wavelet Transform (DWT). We have used "db4" wavelet function for deriving a set of conventional and modified energy based fea-tures from the EEG signals for classifying emotions. Two simple pattern classification methods, K Nearest Neighbor (KNN) and Linear Discriminant Analysis (LDA) methods are used and their performances are compared for emotional states classification. The ex-perimental results indicate that, one of the proposed features (ALREE) gives the maximum average classi-fication rate of 83.26% using KNN and 75.21% using LDA compared to those of conventional features. Fi-nally, we present the average classification rate and subsets of emotions classification rate of these two different classifiers for justifying the performance of our emotion recognition system.
Schizophrenia is sometimes considered one of the most devastating of mental illnesses because its onset is early in a patient's life and its symptoms can be destructive to the patient, the family, and friends. Schizophrenia affects 1 in 100 people at some point during their lives, and while there is no cure, it is treatable with antipsychotic medications. According to the Clinical Antipsychotic Trials for Interventions Effectiveness (CATIE), about 74% of the patients who have discontinued the first medication prescribed within a year will have a relapse afterward. This shows an enormous need for developing better treatment methods and better ways to manage the disease, since current therapies do not have sufficient impact on negative symptoms, cognitive dysfunction, and compliance to treatment. In this clinical case series, we investigate the efficacy of quantitative electroencephalography (qEEG)-guided neurofeedback (NF) treatment in this population, and whether this method has an effect on concurrent medical treatment and on the patients. Fifty-one participants (25 males and 26 females) ranging from 17 to 54 years of age (mean: 28.82 years and SD: 7.94 years) were included. Signed consent was received from all patients. Most of the participants were previously diagnosed with chronic schizophrenia, and their symptoms did not improve with medication. All 51 patients were evaluated using qEEG, which was recorded at baseline and following treatment. Before recording the qEEG, participants were washed out for up to 7 half-lives of the medication. After Food and Drug Administration (FDA)-approved Nx-Link Neurometric analysis, qEEGs suggested a diagnosis of chronic schizophrenia for all participants. This was consistent with the clinical judgment of the authors. The participants' symptoms were assessed by means of the Positive and Negative Syndrome Scale (PANSS). Besides the PANSS, 33 out of 51 participants were also evaluated by the Minnesota Multiphasic Personality Inventory (MMPI) and the Test of Variables of Attention (TOVA), both at baseline and following treatment. Each participant was prescribed an NF treatment protocol based on the results of their qEEG neurometric analysis. Each session was 60 minutes in duration, with 1 to 2 sessions per day. When 2 sessions were administered during a single day, a 30-minute rest was given between the sessions. Changes in the PANSS, MMPI, and TOVA were analyzed to evaluate the effectiveness of NF treatment. The mean number of sessions completed by the participants was 58.5 sessions within 24 to 91 days. Three dropped out of treatment between 30 and 40 sessions of NF, and one did not show any response. Of the remaining 48 participants 47 showed clinical improvement after NF treatment, based on changes in their PANSS scores. The participants who were able to take the MMPI and the TOVA showed significant improvements in these measures as well. Forty were followed up for more than 22 months, 2 for 1 year, 1 for 9 months, and 3 for between 1 and 3 months after completion of NF. Overall NF was shown to be effective. This study provides the first evidence for positive effects of NF in schizophrenia.
A brain-computer interface (BCI) is a hardware and software communications system that permits cerebral activity alone to control computers or external devices. The immediate goal of BCI research is to provide communications capabilities to severely disabled people who are totally paralyzed or ‘locked in’ by neurological neuromuscular disorders, such as amyotrophic lateral sclerosis, brain stem stroke, or spinal cord injury. Here, we review the state-of-the-art of BCIs, looking at the different steps that form a standard BCI: signal acquisition, preprocessing or signal enhancement, feature extraction, classification and the control interface. We discuss their advantages, drawbacks, and latest advances, and we survey the numerous technologies reported in the scientific literature to design each step of a BCI. First, the review examines the neuroimaging modalities used in the signal acquisition step, each of which monitors a different functional brain activity such as electrical, magnetic or metabolic activity. Second, the review discusses different electrophysiological control signals that determine user intentions, which can be detected in brain activity. Third, the review includes some techniques used in the signal enhancement step to deal with the artifacts in the control signals and improve the performance. Fourth, the review studies some mathematic algorithms used in the feature extraction and classification steps which translate the information in the control signals into commands that operate a computer or other device. Finally, the review provides an overview of various BCI applications that control a range of devices.
The analysis of biological fluctuations provides an excellent route to probe the underlying mechanisms in maintaining internal homeostasis of the body, especially under the challenges of the ever-changing environment or disease processes. However, the features of nonlinearity and nonstationarity in physiological time series limit the reliability of the conventional analysis. Hilbert–Huang transform (HHT), based on nonlinear theory, is an innovative approach to extract the dynamic information at different time scales, in particular, from nonstationary signals. In this paper, HHT is introduced to analyze the alpha waves of human's electroencephalography (EEG), which seemly oscillate regularly between 8 and 12 Hz in healthy subject but getting irregular or disappeared in different demented status. Furthermore, conventional time–frequency analyses are adopted to collate the results from those methods and HHT. Finally, the potential usages of HHT are demonstrated in characterizing the biological signals qualitatively and quantitatively, including stationarity analysis, instantaneous frequency and amplitude modulation or correlation analysis. Such applications on EEG have successively disclosed the differences of alpha rhythms between normal and demented brains and the nonlinear characteristics of the underlying mechanisms. Hopefully, in addition to empower the studies of EEG varied in diseased, aging, and physiological processes, these methods might find other applications in EEG analysis.
Functional significance of delta oscillations is not fully understood. One way to approach this question would be from an evolutionary perspective. Delta oscillations dominate the EEG of waking reptiles. In humans, they are prominent only in early developmental stages and during slow-wave sleep. Increase of delta power has been documented in a wide array of developmental disorders and pathological conditions. Considerable evidence on the association between delta waves and autonomic and metabolic processes hints that they may be involved in integration of cerebral activity with homeostatic processes. Much evidence suggests the involvement of delta oscillations in motivation. They increase during hunger, sexual arousal, and in substance users. They also increase during panic attacks and sustained pain. In cognitive domain, they are implicated in attention, salience detection, and subliminal perception. This evidence shows that delta oscillations are associated with evolutionary old basic processes, which in waking adults are overshadowed by more advanced processes associated with higher frequency oscillations. The former processes rise in activity, however, when the latter are dysfunctional.
Although multiple neuroimaging studies suggest that affect labeling (i.e., putting feelings into words) can dampen affect-related responses in the amygdala, the consequences of affect labeling have not been examined in other channels of emotional responding. We conducted four studies examining the effect of affect labeling on self-reported emotional experience. In study one, self-reported distress was lower during affect labeling, compared to passive watching, of negative emotional pictures. Studies two and three added reappraisal and distraction conditions, respectively. Affect labeling showed similar effects on self-reported distress as both of these intentional emotion regulation strategies. In each of the first three studies, however, participant predictions about the effects of affect labeling suggest that unlike reappraisal and distraction, people do not believe affect labeling to be an effective emotion regulation strategy. Even after having the experience of affect labels leading to lower distress, participants still predicted that affect labeling would increase distress in the future. Thus, affect labeling is best described as an incidental emotion regulation process. Finally, study four employed positive emotional pictures and here, affect labeling was associated with diminished self-reported pleasure, relative to passive watching. This suggests that affect labeling tends to dampen affective responses in general, rather than specifically alleviating negative affect.
New work suggests that blood oxygen level-dependent (BOLD) signal variability can be a much more powerful index of human age than mean activation, and that older brains are actually less variable than younger brains. However, little is known of how BOLD variability and task performance may relate. In the current study, we examined BOLD variability in relation to age, and reaction time speed and consistency in healthy younger (20-30 years) and older (56-85 years) adults on three cognitive tasks (perceptual matching, attentional cueing, and delayed match-to-sample). Results indicated that younger, faster, and more consistent performers exhibited significantly higher brain variability across tasks, and showed greater variability-based regional differentiation compared with older, poorer-performing adults. Also, when we compared brain variability- and typical mean-based effects, the respective spatial patterns were essentially orthogonal across brain measures, and any regions that did overlap were largely opposite in directionality of effect. These findings help establish the functional basis of BOLD variability, and further support the statistical and spatial differentiation between BOLD variability and BOLD mean. We thus argue that the precise nature of relations between aging, cognition, and brain function is underappreciated by using mean-based brain measures exclusively.
Working memory (WM) tasks require not only distinct functions such as a storage buffer and central executive functions, but also coordination among these functions. Neuroimaging studies have revealed the contributions of different brain regions to different functional roles in WM tasks; however, little is known about the neural mechanism governing their coordination. Electroencephalographic (EEG) rhythms, especially theta and alpha, are known to appear over distributed brain regions during WM tasks, but the rhythms associated with task-relevant regional coupling have not been obtained thus far. In this study, we conducted time-frequency analyses for EEG data in WM tasks that include manipulation periods and memory storage buffer periods. We used both auditory WM tasks and visual WM tasks. The results successfully demonstrated function-specific EEG activities. The frontal theta amplitudes increased during the manipulation periods of both tasks. The alpha amplitudes increased during not only the manipulation but also the maintenance periods in the temporal area for the auditory WM and the parietal area for the visual WM. The phase synchronization analyses indicated that, under the relevant task conditions, the temporal and parietal regions show enhanced phase synchronization in the theta bands with the frontal region, whereas phase synchronization between theta and alpha is significantly enhanced only within the individual areas. Our results suggest that WM task-relevant brain regions are coordinated by distant theta synchronization for central executive functions, by local alpha synchronization for the memory storage buffer, and by theta-alpha coupling for inter-functional integration.
There is a need for effective interventions to address the core symptoms and problems associated with autistic spectrum disorder (ASD). Behavior therapy improves communication and behavioral functioning. Additional treatment options include psychopharmacological and biomedical interventions. Although these approaches help children with autistic problems, they may be associated with side effects, risks or require ongoing or long-term treatment. Neurofeedback is a noninvasive approach shown to enhance neuroregulation and metabolic function in ASD. We present a review of the literature on the application of Neurofeedback to the multiple problems associated with ASD. Directions for future research are discussed.
In this paper we review classification algorithms used to design brain–computer interface (BCI) systems based on electroencephalography (EEG). We briefly present the commonly employed algorithms and describe their critical properties. Based on the literature, we compare them in terms of performance and provide guidelines to choose the suitable classification algorithm(s) for a specific BCI.
Neuronal gamma-band synchronization is found in many cortical areas, is induced by different stimuli or tasks, and is related to several cognitive capacities. Thus, it appears as if many different gamma-band synchronization phenomena subserve many different functions. I argue that gamma-band synchronization is a fundamental process that subserves an elemental operation of cortical computation. Cortical computation unfolds in the interplay between neuronal dynamics and structural neuronal connectivity. A core motif of neuronal connectivity is convergence, which brings about both selectivity and invariance of neuronal responses. However, those core functions can be achieved simultaneously only if converging neuronal inputs are functionally segmented and if only one segment is selected at a time. This segmentation and selection can be elegantly achieved if structural connectivity interacts with neuronal synchronization. I propose that this process is at least one of the fundamental functions of gamma-band synchronization, which then subserves numerous higher cognitive functions.
There are declines with age in speed of processing, working memory, inhibitory function, and long-term memory, as well as decreases in brain structure size and white matter integrity. In the face of these decreases, functional imaging studies have demonstrated, somewhat surprisingly, reliable increases in prefrontal activation. To account for these joint phenomena, we propose the scaffolding theory of aging and cognition (STAC). STAC provides an integrative view of the aging mind, suggesting that pervasive increased frontal activation with age is a marker of an adaptive brain that engages in compensatory scaffolding in response to the challenges posed by declining neural structures and function. Scaffolding is a normal process present across the lifespan that involves use and development of complementary, alternative neural circuits to achieve a particular cognitive goal. Scaffolding is protective of cognitive function in the aging brain, and available evidence suggests that the ability to use this mechanism is strengthened by cognitive engagement, exercise, and low levels of default network engagement.
In this experiment, we combined the measurement of observable facial behavior with simultaneous measures of brain electrical activity to assess patterns of hemispheric activation in different regions during the experience of happiness and disgust. Disgust was found to be associated with right-sided activation in the frontal and anterior temporal regions compared with the happy condition. Happiness was accompanied by left-sided activation in the anterior temporal region compared with disgust. No differences in asymmetry were found between emotions in the central and parietal regions. When data aggregated across positive films were compared to aggregate negative film data, no reliable differences in brain activity were found. These findings illustrate the utility of using facial behavior to verify the presence of emotion, are consistent with the notion of emotion-specific physiological patterning, and underscore the importance of anterior cerebral asymmetries for emotions associated with approach and withdrawal.
Emotion recognition (ER) from Electroencephalogram (EEG) signals is a challenging task due to the non-linearity and non-stationarity nature of EEG signals. Existing feature extraction methods cannot extract the deep concealed characteristics of EEG signals from different layers for efficient classification scheme and also hard to select appropriate and effective feature extraction methods for different types of EEG data. Hence this study intends to develop an efficient deep feature extraction based method to automatically classify emotion status of people. In order to discover reliable deep features, five deep convolutional neural networks (CNN) models are considered: AlexNet, VGG16, ResNet50, SqueezeNet and MobilNetv2. Pre-processing, Wavelet Transform (WT), and Continuous Wavelet Transform (CWT) are employed to convert the EEG signals into EEG rhythm images then five well-known pretrained CNN models are employed for feature extraction. Finally, the proposed method puts the obtained features as input to the support vector machine (SVM) method for classifying them into binary emotion classes: valence and arousal classes. The DEAP dataset was used in experimental works. The experimental results demonstrate that the AlexNet features with Alpha rhythm produces better accuracy scores (91.07% in channel Oz) than the other deep features for the valence discrimination, and the MobilNetv2 features yields the highest accuracy score (98.93% in Delta rhythm ( with channel C3) for arousal discrimination.
Emotion recognition based on electroencephalography (EEG) signal features is now one of the booming big data research areas. As the number of commercial EEG devices in the current market increases, there is a need to understand current trends and provide researchers and young practitioners with insights into future investigations of emotion recognition systems. This paper aims to evaluate popular consumer-grade EEG devices’ status and review relevant studies that examined the reliability of these low-cost devices for emotion recognition over the last five years. Additionally, a comparison with research-grade devices is conducted. This paper also highlights EEG-based emotion recognition research’s key areas, including different feature extraction capabilities, characteristics, and machine learning algorithms. Finally, the main challenges for building an EEG-based emotion recognition system, focusing on the data collection process with commercial EEG devices and machine learning algorithms’ performance, are presented.
The development of suitable EEG-based emotion recognition systems has become a main target in the last decades for Brain Computer Interface applications (BCI). However, there are scarce algorithms and procedures for real-time classification of emotions. The present study aims to investigate the feasibility of real-time emotion recognition implementation by the selection of parameters such as the appropriate time window segmentation and target bandwidths and cortical regions. We recorded the EEG-neural activity of 24 participants while they were looking and listening to an audiovisual database composed of positive and negative emotional video clips. We tested 12 different temporal window sizes, 6 ranges of frequency bands and 60 electrodes located along the entire scalp. Our results showed a correct classification of 86.96% for positive stimuli. The correct classification for negative stimuli was a little bit less (80.88%). The best time window size, from the tested 1
s to 12
s segments, was 12
s. Although more studies are still needed, these preliminary results provide a reliable way to develop accurate EEG-based emotion classification.
Electroencephalography is widely used to study the dynamics of neural information processing in the brain and to diagnose brain disorder and cognitive processes. In this paper, we proposed EEG based emotion recognition system using Discrete Wavelet Transformation. A set of highly significant features based on wavelets coefficients has been extracted which also includes modified wavelet energy features. In order to minimize redundancy and maximize relevancy among features, mRMR algorithm is significantly applied for feature selection. Multi class Support Vector Machine is used to perform classification of four classes of human emotions. EEG recordings of “DEAP” database are used in this experiment. The proposed approach shows significant performance compared to existing algorithms.
In electroencephalography (EEG)-based emotion recognition systems, the distribution between the training samples and the testing samples may be mismatched if they are sampled from different experimental sessions or subjects because of user fatigue, different electrode placements, varying impedances, etc. Therefore, it is difficult to directly classify the EEG patterns with a conventional classifier. The domain adaptation method, which is aimed at obtaining a common representation across training and test domains, is an effective method for reducing the distribution discrepancy. However, the existing domain adaptation strategies either employ a linear transformation or learn the nonlinearity mapping without a consistency constraint; they are not sufficiently powerful to obtain a similar distribution from highly non-stationary EEG signals. To address this problem, in this paper, a novel component, called the subspace alignment auto-encoder (SAAE), is proposed. Taking advantage of both nonlinear transformation and a consistency constraint, we combine an auto-encoder network and a subspace alignment solution in a unified framework. As a result, the source domain can be aligned with the target domain together with its class label, and any supervised method can be applied to the new source domain to train a classifier for classification in the target domain, as the aligned source domain follows a distribution similar to that of the target domain. We compared our SAAE method with six typical approaches using a public EEG dataset containing three affective states: positive, neutral, and negative. Subject-to-subject and session-to-session evaluations were performed. The subject-to-subject experimental results demonstrate that our component achieves a mean accuracy of 77.88% in comparison with a state-of-the-art method, TCA, which achieves 73.82% on average. In addition, the average classification accuracy of SAAE in the session-to-session evaluation for all the 15 subjects in a dataset is 81.81%, an improvement of up to 1.62% on average as compared to the best baseline TCA. The experimental results show the effectiveness of the proposed method relative to state-of-the-art methods. It can be concluded that SAAE is a useful and effective tool for decreasing domain discrepancy and reducing performance degradation across subjects and sessions in the EEG-based emotion recognition field.
In this paper, we review different classification methods for emotion recognition from EEG and perform a detailed comparison of these methods on a relatively larger dataset of 45 experiments. We propose to combine the classifiers using stacking to improve the emotion recognition accuracies. Experimental results show that the combination of classifiers using stacking can achieve higher average accuracies than that without stacking methods. The weights derived from the classifiers are investigated to extract the relevant features and present their biological interpretation as critical brain areas and critical frequency bands.
It is often a problem in various fields that one runs into a series of tasks that appear - to a human - to be highly related to each other, yet applying the optimal machine learning solution of one problem to another results in poor performance. Specifically in the field of brain-computer interfaces (BCIs), it has long been known that a subject with good classification of some brain signal today could come into the experimental setup tomorrow and perform terribly using the exact same classifier. One initial approach to get over this problem was to fix the classification rule beforehand and train the patient to force brain activity to conform to this rule.
Brain-computer interface (BCI) is combination of hardware and software systems that allows the severely or partially disabled persons to communicate with their surroundings. The study of brain activities precisely is an important step in BCI system. Many invasive and non-invasive neuro-imaging techniques are being conducted. In this paper, a comparative analysis of these different approaches has been reviewed such as electro-encephalography (EEG), electro-corticograph (ECoG), magneto-encephalograph (MEG), intra-cortical neuron recording (INR), and magnetic resonance imaging (MRI).
An 81-electrode system is described which is designed for topographic studies of spontaneous and evoked EEG activities. This method combines the standard leads of the International 10-20 System with supplementary electrodes applied midway between leads of the 10-20 system or electrodes in turn situated between 10-20 leads. Auxiliary electrode designations refer to the underlying brain areas and to adjacent leads of the 10-20 method. The utilization of this '10% system' is suggested to promote standardization.
Brain signals measured over a series of experiments have inherent variability because of different physical and mental conditions among multiple subjects and sessions. Such variability complicates the analysis of data from multiple subjects and sessions in a consistent way, and degrades the performance of subject-transfer decoding in a brain–machine interface (BMI). To accommodate the variability in brain signals, we propose 1) a method for extracting spatial bases (or a dictionary) shared by multiple subjects, by employing a signal-processing technique of dictionary learning modified to compensate for variations between subjects and sessions, and 2) an approach to subject-transfer decoding that uses the resting-state activity of a previously unseen target subject as calibration data for compensating for variations, eliminating the need for a standard calibration based on task sessions. Applying our methodology to a dataset of electroencephalography (EEG) recordings during a selective visual–spatial attention task from multiple subjects and sessions, where the variability compensation was essential for reducing the redundancy of the dictionary, we found that the extracted common brain activities were reasonable in the light of neuroscience knowledge. The applicability to subject-transfer decoding was confirmed by improved performance over existing decoding methods. These results suggest that analyzing multisubject brain activities on common bases by the proposed method enables information sharing across subjects with low-burden resting calibration, and is effective for practical use of BMI in variable environments.
The detection of nonconvulsive seizures (NCSz) is a challenge because of the lack of physical symptoms, which may delay the diagnosis of the disease. Many researchers have reported automatic detection of seizures. However, few investigators have concentrated on detection of NCSz. This article proposes a method for reliable detection of NCSz. The electroencephalography (EEG) signal is usually contaminated by various nonstationary noises. Signal denoising is an important preprocessing step in the analysis of such signals. In this study, a new wavelet-based denoising approach using cubical thresholding has been proposed to reduce noise from the EEG signal prior to analysis. Three statistical features were extracted from wavelet frequency bands, encompassing the frequency range of 0 to 8, 8 to 16, 16 to 32, and 0 to 32 Hz. Extracted features were used to train linear classifier to discriminate between normal and seizure EEGs. The performance of the method was tested on a database of nine patients with 24 seizures in 80 hours of EEG recording. All the seizures were successfully detected, and false positive rate was found to be 0.7 per hour.
Abstract Here, we present a state-of-the-art review of the research performed on the brain-computer interface (BCI) technologies with a focus on signal processing approaches. BCI can be divided into three main components: signal acquisition, signal processing, and effector device. The signal acquisition component is generally divided into two categories: noninvasive and invasive. For noninvasive, this review focuses on electroencephalogram. For the invasive, the review includes electrocorticography, local field potentials, multiple-unit activity, and single-unit action potentials. Signal processing techniques reviewed are divided into time-frequency methods such as Fourier transform, autoregressive models, wavelets, and Kalman filter and spatiotemporal techniques such as Laplacian filter and common spatial patterns. Additionally, various signal feature classification algorithms are discussed such as linear discriminant analysis, support vector machines, artificial neural networks, and Bayesian classifiers. The article ends with a discussion of challenges facing BCI and concluding remarks on the future of the technology.
In the study of brain computer interfaces, a novel method was proposed in this paper for the feature extraction of electroencephalogram (EEG). It was based on wavelet packet decomposition (WPD). The energy of special sub-bands and corresponding coefficients of wavelet packet decomposition were selected as features which have maximal separability according to the Fisher distance criterion. The eigenvector was obtained for classification by combining the effective features from different channels; its performance was evaluated by separability and pattern recognition accuracy using the datasets of BCI 2003 Competition. The classification results have proved the effectiveness of the proposed method. This technology provides another useful way to EEG feature extraction in BCIs.
Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.
The test is based on the maximum difference between an empirical and a hypothetical cumulative distribution. Percentage points are tabled, and a lower bound to the power function is charted. Confidence limits for a cumulative distribution are described. Examples are given. Indications that the test is superior to the chi-square test are cited.
Film clips are an important tool for evoking emotional responses in the laboratory. When compared with other emotionally potent visual stimuli (e.g., pictures), film clips seem to be more effective in eliciting emotions for longer periods of time at both the subjective and physiological levels. The main objective of the present study was to develop a new database of affective film clips without auditory content, based on a dimensional approach to emotional stimuli (valence, arousal and dominance). The study had three different phases: (1) the pre-selection and editing of 52 film clips (2) the self-report rating of these film clips by a sample of 113 participants and (3) psychophysiological assessment [skin conductance level (SCL) and the heart rate (HR)] on 32 volunteers. Film clips from different categories were selected to elicit emotional states from different quadrants of affective space. The results also showed that sustained exposure to the affective film clips resulted in a pattern of a SCL increase and HR deceleration in high arousal conditions (i.e., horror and erotic conditions). The resulting emotional movie database can reliably be used in research requiring the presentation of non-auditory film clips with different ratings of valence, arousal and dominance.
The articles published by the Annals of Eugenics (1925–1954) have been made available online as an historical archive intended for scholarly use. The work of eugenicists was often pervaded by prejudice against racial, ethnic and disabled groups. The online publication of this material for scholarly research purposes is not an endorsement of those views nor a promotion of eugenics in any way.
Neurofeedback represents an exciting complementary option in the treatment of depression that builds upon a huge body of research on electroencephalographic correlates of depression.
The objectives of this article are threefold: review the literature on neurofeedback protocols for depression; introduce a new protocol, which aims to synthesize the best qualities of the currently available protocols; and present the results of a small clinical experiment with the new protocol.
Structured survey of the literature; software development; clinical trial with one subject, submitted to ten sessions of neurofeedback (one hour each).
Currently there are twenty-one articles in neurofeedback for depression, among which only six present original experimental results. All of them report positive results with the technique. The most used protocols focus on Alpha inter-hemispheric asymmetry, and Theta/Beta ratio within the left prefrontal cortex. Our new protocol integrates both dimensions in a single circuit, adding to it a third programming line, which divides Beta frequencies and reinforces the decrease of Beta-3, in order to reduce anxiety. The favorable outcome of our clinical experiment, suggests that new research with this protocol is worthwhile.
This study explores the electroencephalographic (EEG) correlates of emotional experience during music listening. Independent component analysis and analysis of variance were used to separate statistically independent spectral changes of the EEG in response to music-induced emotional processes. An independent brain process with equivalent dipole located in the fronto-central region exhibited distinct δ-band and θ-band power changes associated with self-reported emotional states. Specifically, the emotional valence was associated with δ-power decreases and θ-power increases in the frontal-central area, whereas the emotional arousal was accompanied by increases in both δ and θ powers. The resultant emotion-related component activations that were less interfered by the activities from other brain processes complement previous EEG studies of emotion perception to music.
A phenomenon often found in session-to-session transfers of brain-computer interfaces (BCIs) is nonstationarity. It can be caused by fatigue and changing attention level of the user, differing electrode placements, varying impedances, among other reasons. Covariate shift adaptation is an effective method that can adapt to the testing sessions without the need for labeling the testing session data. The method was applied on a BCI Competition III dataset. Results showed that covariate shift adaptation compares favorably with methods used in the BCI competition in coping with nonstationarities. Specifically, bagging combined with covariate shift helped to increase stability, when applied to the competition dataset. An online experiment also proved the effectiveness of bagged-covariate shift method. Thus, it can be summarized that covariate shift adaptation is helpful to realize adaptive BCI systems.
When making a decision, one must first accumulate evidence, often over time, and then select the appropriate action. Here, we present a neural model of decision making that can perform both evidence accumulation and action selection optimally. More specifically, we show that, given a Poisson-like distribution of spike counts, biological neural networks can accumulate evidence without loss of information through linear integration of neural activity and can select the most likely action through attractor dynamics. This holds for arbitrary correlations, any tuning curves, continuous and discrete variables, and sensory evidence whose reliability varies over time. Our model predicts that the neurons in the lateral intraparietal cortex involved in evidence accumulation encode, on every trial, a probability distribution which predicts the animal's performance. We present experimental evidence consistent with this prediction and discuss other predictions applicable to more general settings.
This research brings together two separate areas: that of EEG processes associated with positive and negatively valenced emotional material; and that of traditional psychophysiological research related to the "intake" and "rejection" of environmental stimuli. Forty males on each of two days were presented with tasks reflecting both attentional demands and affectual processing. Heart rate and bilateral EEG measures from frontal, parietal and temporal sites were recorded. Using a FFT (fast Fourier transform) electrocortical activity in the 2-7 Hz, 8-15 Hz, and 16-24 Hz was determined and analyzed. The results suggest emotional valence (i.e. positive and negative) and attentional demands (i.e. intake vs rejection) are differentially represented in terms of EEG functioning. An interaction of attentional demand with hemisphere was found for EEG alpha activity in the temporal and parietal areas. For emotional valence there was a significant main effect for EEG beta activity in both the temporal and parietal areas. Differential hemispheric activity was found using a factor analytic technique (PARAFAC) with positively valenced tasks being associated with right temporal beta. Heart rate changes for the attentional dimension were consistent with previous research.
The present study was designed to test differential hemispheric activation induced by emotional stimuli in the gamma band range (30-90 Hz). Subjects viewed slides with differing emotional content (from the International Affective Picture System). A significant valence by hemisphere interaction emerged in the gamma band from 30-50 Hz. Other bands, including alpha and beta, did not show such an interaction. Previous hypotheses suggested that the left hemisphere is more involved in positive affective processing as compared to the right hemisphere, while the latter dominates during negative emotions. Contrary to this expectation, the 30-50 Hz band showed relatively more power for negative valence over the left temporal region as compared to the right and a laterality shift towards the right hemisphere for positive valence. In addition, emotional processing enhanced gamma band power at right frontal electrodes regardless of the particular valence as compared to processing neutral pictures. The extended distribution of specific activity in the gamma band may be the signature of cell assemblies with members in limbic, temporal and frontal neocortical structures that differ in spatial distribution depending on the particular type of emotional processing.
The use of neurofeedback as an operant conditioning paradigm has disclosed that participants are able to gain some control over particular aspects of their electroencephalogram (EEG). Based on the association between theta activity (4-7 Hz) and working memory performance, and sensorimotor rhythm (SMR) activity (12-15 Hz) and attentional processing, we investigated the possibility that training healthy individuals to enhance either of these frequencies would specifically influence a particular aspect of cognitive performance, relative to a non-neurofeedback control-group. The results revealed that after eight sessions of neurofeedback the SMR-group were able to selectively enhance their SMR activity, as indexed by increased SMR/theta and SMR/beta ratios. In contrast, those trained to selectively enhance theta activity failed to exhibit any changes in their EEG. Furthermore, the SMR-group exhibited a significant and clear improvement in cued recall performance, using a semantic working memory task, and to a lesser extent showed improved accuracy of focused attentional processing using a 2-sequence continuous performance task. This suggests that normal healthy individuals can learn to increase a specific component of their EEG activity, and that such enhanced activity may facilitate semantic processing in a working memory task and to a lesser extent focused attention. We discuss possible mechanisms that could mediate such effects and indicate a number of directions for future research.