No file available
To read the file of this research,
you can request a copy directly from the authors.
2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
Transfer learning and meta-learning offer some of the most promising avenues to unlock the scalability of healthcare and consumer technologies driven by biosignal data. This is because current methods cannot generalise well across human subjects' data and handle learning from different heterogeneously collected data sets, thus limiting the scale of training data. On the other side, developments in transfer learning would benefit significantly from a real-world benchmark with immediate practical application. Therefore, we pick electroencephalography (EEG) as an exemplar for what makes biosignal machine learning hard. We design two transfer learning challenges around diagnostics and Brain-Computer-Interfacing (BCI), that have to be solved in the face of low signal-to-noise ratios, major variability among subjects, differences in the data recording sessions and techniques, and even between the specific BCI tasks recorded in the dataset. Task 1 is centred on the field of medical diagnostics, addressing automatic sleep stage annotation across subjects. Task 2 is centred on Brain-Computer Interfacing (BCI), addressing motor imagery decoding across both subjects and data sets. The BEETL competition with its over 30 competing teams and its 3 winning entries brought attention to the potential of deep transfer learning and combinations of set theory and conventional machine learning techniques to overcome the challenges. The results set a new state-of-the-art for the real-world BEETL benchmark.
... Notably, the results of the CNN pipelines show a more pronounced bimodal distribution compared to the Riemannian pipelines. The observation that deep convolutional networks may be particularly useful in transfer learning settings is in line with the results of a recent BCI decoding competition that was also won by a CNN approach [37]. However, we remark that the methods benchmarked here were not explicitly designed for cross-subject decoding. ...
Objective. To date, a comprehensive comparison of Riemannian decoding methods with deep convolutional neural networks for EEG-based Brain-Computer Interfaces remains absent from published work. We address this research gap by using MOABB, The Mother Of All BCI Benchmarks, to compare novel convolutional neural networks to state-of-the-art Riemannian approaches across a broad range of EEG datasets, including motor imagery, P300, and steady-state visual evoked potentials paradigms.
Approach. We systematically evaluated the performance of convolutional neural networks, specifically EEGNet, shallow ConvNet, and deep ConvNet, against well-established Riemannian decoding methods using MOABB processing pipelines. This evaluation included within- session, cross-session, and cross-subject methods, to provide a practical analysis of model effectiveness and to find an overall solution that performs well across different experimental settings.
Main results. We find no significant differences in decoding performance between convolutional neural networks and Riemannian methods for within-session, cross-session, and cross-subject analyses.
Significance. The results show that, when using traditional Brain-Computer Interface paradigms, the choice between CNNs and Riemannian methods may not heavily impact decoding performances in many experimental settings. These findings provide researchers with flexibility in choosing decoding approaches based on factors such as ease of implementation, computational efficiency or individual preferences.
In recent years, deep-learning models gained attention for electroencephalography (EEG) classification tasks due to their excellent performance and ability to extract complex features from raw data. In particular, convolutional neural networks (CNN) showed adequate results in brain-computer interfaces (BCI) based on different control signals, including event-related potentials (ERP). In this study, we propose a novel CNN, called EEG-Inception, that improves the accuracy and calibration time of assistive ERP-based BCIs. To the best of our knowledge, EEG-Inception is the first model to integrate Inception modules for ERP detection, which combined efficiently with other structures in a light architecture, improved the performance of our approach. The model was validated in a population of 73 subjects, of which 31 present motor disabilities. Results show that EEG-Inception outperforms 5 previous approaches, yielding significant improvements for command decoding accuracy up to 16.0%, 10.7%, 7.2%, 5.7% and 5.1% in comparison to rLDA, xDAWN + Riemannian geometry, CNN-BLSTM, DeepConvNet and EEGNet, respectively. Moreover, EEG-Inception requires very few calibration trials to achieve state-of-the-art performances taking advantage of a novel training strategy that combines cross-subject transfer learning and fine-tuning to increase the feasibility of this approach for practical use in assistive applications.
Explainable AI is not a new field. Since at least the early exploitation of C.S. Pierce’s abductive reasoning in expert systems of the 1980s, there were reasoning architectures to support an explanation function for complex AI systems, including applications in medical diagnosis, complex multi-component design, and reasoning about the real world. So explainability is at least as old as early AI, and a natural consequence of the design of AI systems. While early expert systems consisted of handcrafted knowledge bases that enabled reasoning over narrowly well-defined domains (e.g., INTERNIST, MYCIN), such systems had no learning capabilities and had only primitive uncertainty handling. But the evolution of formal reasoning architectures to incorporate principled probabilistic reasoning helped address the capture and use of uncertain knowledge.
Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end-to-end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end-to-end EEG analysis, but a better understanding of how to design and train ConvNets for end-to-end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG-based brain mapping. Hum Brain Mapp, 2017.
Sleep stage classification constitutes an important preliminary exam in the diagnosis of sleep disorders and is traditionally performed by a sleep expert who assigns to each 30s of signal a sleep stage, based on the visual inspection of signals such as electroencephalograms (EEG), electrooculograms (EOG), electrocardiograms (ECG) and electromyograms (EMG). In this paper, we introduce the first end-to-end deep learning approach that performs automatic temporal sleep stage classification from multivariate and multimodal Polysomnography (PSG) signals. We build a general deep architecture which can extract information from EEG, EOG and EMG channels and pools the learnt representations into a final softmax classifier. The architecture is light enough to be distributed in time in order to learn from the temporal context of each sample, namely previous and following data segments. Our model, which is unique in its ability to learn a feature representation from multiple modalities, is compared to alternative automatic approaches based on convolutional networks or decisions trees. Results obtained on 61 publicly available PSG records with up to 20 EEG channels demonstrate that our network architecture yields state-of-the-art performance. Our study reveals a number of insights on the spatio-temporal distribution of the signal of interest: a good trade-off for optimal classification performance measured with balanced accuracy is to use 6 EEG with some EOG and EMG channels. Also exploiting one minute of data before and after each data segment to be classified offers the strongest improvement when a limited number of channels is available. Our approach aims to improve a key step in the study of sleep disorders. As sleep experts, our system exploits the multivariate and multimodal character of PSG signals to deliver state-of-the-art classification performance at a very low complexity cost.
Background:
Most investigators of brain computer interface (BCI) research believe that BCI can be achieved through induced neuronal activity from the cortex, but not by evoked neuronal activity. Motor imagery (MI) based BCI is one of the standard concepts of BCI, in that the user can generate induced activity by imagining motor movements. However, variations in performance over sessions and subjects are too severe to overcome easily; therefore, a basic understanding and investigation of BCI performance variation is necessary to find critical evidence of performance variation. Here we present not only EEG datasets for MI BCI from 52 subjects, but also the results of a psychological and physiological questionnaire, EMG datasets, the locations of 3D EEG electrodes, and EEGs for non-task related states.
Findings:
We validated our EEG datasets by using the percentage of bad trials, event-related desynchronization/synchronization (ERD/ERS) analysis, and classification analysis. After conventional rejection of bad trials, we showed contralateral ERD and ipsilateral ERS in the somatosensory area, which are well-known patterns of MI. Finally, we showed that 73.08% of datasets (38 subjects) included reasonably discriminative information.
Conclusions:
Our EEG datasets included good information necessary to determine statistical significance; they consisted of well-discriminated datasets (38 subjects) and less-discriminative datasets. These may provide researchers with opportunities to investigate human factors related to MI BCI performance variation, and may also achieve subject-to-subject transfer by using metadata, including a questionnaire, EEG coordinates, and EEGs for non-task related states.
Objective:
Brain computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional Neural Networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible.
Approach:
In this work we introduce EEGNet, a compact convolutional neural network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR).
Results:
We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, the reference algorithms when only limited training data is available across all tested paradigms. In addition, we demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features.
Significance:
Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks.Our models can be found at: https://github.com/vlawhern/arl-eegmodels.
Anaesthetic drugs act at sites within the brain that undergo profound changes during typical ageing. We postulated that anaesthesia-induced brain dynamics observed in the EEG change with age.
We analysed the EEG in 155 patients aged 18-90 yr who received propofol (n=60) or sevoflurane (n=95) as the primary anaesthetic. The EEG spectrum and coherence were estimated throughout a 2 min period of stable anaesthetic maintenance. Age-related effects were characterized by analysing power and coherence as a function of age using linear regression and by comparing the power spectrum and coherence in young (18- to 38-yr-old) and elderly (70- to 90-yr-old) patients.
Power across all frequency bands decreased significantly with age for both propofol and sevoflurane; elderly patients showed EEG oscillations ∼2- to 3-fold smaller in amplitude than younger adults. The qualitative form of the EEG appeared similar regardless of age, showing prominent alpha (8-12 Hz) and slow (0.1-1 Hz) oscillations. However, alpha band dynamics showed specific age-related changes. In elderly compared with young patients, alpha power decreased more than slow power, and alpha coherence and peak frequency were significantly lower. Older patients were more likely to experience burst suppression.
These profound age-related changes in the EEG are consistent with known neurobiological and neuroanatomical changes that occur during typical ageing. Commercial EEG-based depth-of-anaesthesia indices do not account for age and are therefore likely to be inaccurate in elderly patients. In contrast, monitoring the unprocessed EEG and its spectrogram can account for age and individual patient characteristics.
© The Author 2015. Published by Oxford University Press on behalf of the British Journal of Anaesthesia. All rights reserved. For Permissions, please email: journals.permissions@oup.com.
Motor imagery (MI), sharing similar neural representations to motor execution, is regarded as a window to investigate the cognitive motor processes. However, in comparison to simple limb motor imagery, significantly less work has been reported on brain oscillatory patterns induced by compound limb motor imagery which involves several parts of limbs. This study aims to investigate differences of the electroencephalogram (EEG) patterns as well as cognitive process between simple limb motor imagery and compound limb motor imagery. Ten subjects participated in the experiment involving three tasks of simple limb motor imagery (left hand, right hand, feet) and three tasks of compound limb motor imagery (both hands, left hand combined with right foot, right hand combined with left foot). Simultaneous imagination of different limbs contributes to the activation of larger cortical areas as well as two estimated sources located at corresponding motor areas within beta rhythm. Compared with simple limb motor imagery, compound limb motor imagery presents a network with more effective interactions overlying larger brain regions, additionally shows significantly larger causal flow over sensorimotor areas and larger causal density over both sensorimotor areas and neighboring regions. On the other hand, compound limb motor imagery also shows significantly larger 10-11 Hz alpha desynchronization at occipital areas and central theta synchronization. Furthermore, the phase-locking value (PLV) between central and occipital areas of left/right hand combined with contralateral foot imagery is significantly larger than that of simple limb motor imagery. All these findings imply that there exist apparent intrinsic distinctions of neural mechanism between simple and compound limb motor imagery, which presents a more complex effective connectivity network and may involve a more complex cognitive process during information processing.
Many deep neural networks trained on natural images exhibit a curious
phenomenon in common: on the first layer they learn features similar to Gabor
filters and color blobs. Such first-layer features appear not to be specific to
a particular dataset or task, but general in that they are applicable to many
datasets and tasks. Features must eventually transition from general to
specific by the last layer of the network, but this transition has not been
studied extensively. In this paper we experimentally quantify the generality
versus specificity of neurons in each layer of a deep convolutional neural
network and report a few surprising results. Transferability is negatively
affected by two distinct issues: (1) the specialization of higher layer neurons
to their original task at the expense of performance on the target task, which
was expected, and (2) optimization difficulties related to splitting networks
between co-adapted neurons, which was not expected. In an example network
trained on ImageNet, we demonstrate that either of these two issues may
dominate, depending on whether features are transferred from the bottom,
middle, or top of the network. We also document that the transferability of
features decreases as the distance between the base task and target task
increases, but that transferring features even from distant tasks can be better
than using random features. A final surprising result is that initializing a
network with transferred features from almost any number of layers can produce
a boost to generalization that lingers even after fine-tuning to the target
dataset.
The BCI competition IV stands in the tradition of prior BCI competitions that aim to provide high quality neuroscientific data for open access to the scientific community. As experienced already in prior competitions not only scientists from the narrow field of BCI compete, but scholars with a broad variety of backgrounds and nationalities. They include high specialists as well as students. The goals of all BCI competitions have always been to challenge with respect to novel paradigms and complex data. We report on the following challenges: (1) asynchronous data, (2) synthetic, (3) multi-class continuous data, (4) session-to-session transfer, (5) directionally modulated MEG, (6) finger movements recorded by ECoG. As after past competitions, our hope is that winning entries may enhance the analysis methods of future BCIs.
The effect of age on sleep and the sleep EEG was investigated in middle-aged men (mean age: 62.0 years) and in young men (mean age: 22.4 years). Even though the older men reported a higher number of nocturnal awakenings, subjective sleep quality did not differ. Total sleep time, sleep efficiency, and slow wave sleep were lower in the middle-aged, while stage 1 and wakefulness after sleep onset were higher. The difference in wakefulness within nonREM-REM sleep cycles was most pronounced in the third and fourth cycle. In the older men, EEG power density in nonREM sleep was reduced in frequencies below 14.0 Hz, whereas in REM sleep age-related reductions were limited to the delta-theta (0.25-7.0 Hz) and low alpha (8.25-10.0 Hz) band. Slow-wave activity (SWA, power density in the 0.75-4.5 Hz range) decreased in the course of sleep in both age groups. The between-group difference in SWA diminished in the course of sleep, whereas the difference in activity in the frequency range of sleep spindles (12.25-14.0 Hz) increased. It is concluded that frequency and state specific changes occur as a function of age, and that the sleep dependent decline in SWA and increase in sleep spindle activity are attenuated with age.
Many laboratories have begun to develop brain-computer interface (BCI) systems that provide communication and control capabilities to people with severe motor disabilities. Further progress and realization of practical applications depends on systematic evaluations and comparisons of different brain signals, recording methods, processing algorithms, output formats, and operating protocols. However, the typical BCI system is designed specifically for one particular BCI method and is, therefore, not suited to the systematic studies that are essential for continued progress. In response to this problem, we have developed a documented general-purpose BCI research and development platform called BCI2000. BCI2000 can incorporate alone or in combination any brain signals, signal processing methods, output devices, and operating protocols. This report is intended to describe to investigators, biomedical engineers, and computer scientists the concepts that the BCI2000 system is based upon and gives examples of successful BCI implementations using this system. To date, we have used BCI2000 to create BCI systems for a variety of brain signals, processing methods, and applications. The data show that these systems function well in online operation and that BCI2000 satisfies the stringent real-time requirements of BCI systems. By substantially reducing labor and cost, BCI2000 facilitates the implementation of different BCI systems and other psychophysiological experiments. It is available with full documentation and free of charge for research or educational purposes and is currently being used in a variety of studies by many research groups.
This paper presents a new classification framework for brain-computer interface (BCI) based on motor imagery. This framework involves the concept of Riemannian geometry in the manifold of covariance matrices. The main idea is to use spatial covariance matrices as EEG signal descriptors and to rely on Riemannian geometry to directly classify these matrices using the topology of the manifold of symmetric and positive definite (SPD) matrices. This framework allows to extract the spatial information contained in EEG signals without using spatial filtering. Two methods are proposed and compared with a reference method [multiclass Common Spatial Pattern (CSP) and Linear Discriminant Analysis (LDA)] on the multiclass dataset IIa from the BCI Competition IV. The first method, named minimum distance to Riemannian mean (MDRM), is an implementation of the minimum distance to mean (MDM) classification algorithm using Riemannian distance and Riemannian mean. This simple method shows comparable results with the reference method. The second method, named tangent space LDA (TSLDA), maps the covariance matrices onto the Riemannian tangent space where matrices can be vectorized and treated as Euclidean objects. Then, a variable selection procedure is applied in order to decrease dimensionality and a classification by LDA is performed. This latter method outperforms the reference method increasing the mean classification accuracy from 65.1% to 70.2%.
EEG arousals were quantified in 40 nocturnal polysomnographic recordings belonging to four age groups (teenagers: 10 to 19 years; young adults: 20 to 39 years; middle-aged: 40 to 59 years; elderly: > or = 60 years). Ten subjects (five males and five females) participated in each group. The subjects were healthy and sound sleepers. All sleep recordings were preceded by an adaptation night which aimed at excluding the presence of sleep-related disorders. The recordings were carried out in a partially soundproof recording chamber and in a standard laboratory setting. Arousal indices (AI), defined as the number of arousals per hour of sleep, were calculated for total sleep time (AI/TST) and for all the sleep stages. AI/TST increased linearly with age (r = 0.852; p < 0.00001): teenagers (13.8), young adults (14.7), middle-aged (17.8), elderly (27.1). An age-related positive linear correlation was found also for the arousal indices referred to NREM sleep (r = 0.811; p < 0.00001) and to stages 1 and 2 (r = 0.712; p < 0.00001), while in stages 3 and 4 and in REM sleep, arousal indices showed stable values across the ages. Overall, arousals lasted 14.9 +/- 2.3 seconds, with arousal duration stable across the ages (range of means: 13.3-16.6 seconds) and no relevant differences between NREM sleep (14.6 +/- 2.5 seconds) and REM sleep (16.2 +/- 5 seconds). The paper discusses the impact of age on arousals, the similarities between arousals and the phases d'activation transitoire, and the consideration that arousals are physiological components of sleep.
Recently, transfer learning and deep learning have been introduced to solve intra- and inter-subject variability problems in Brain-Computer Interfaces. However, the generalization ability of these BCIs is still to be further verified in a cross-dataset scenario. This study compared the transfer performance of manifold embedded knowledge transfer and pre-trained EEGNet with three preprocessing strategies. This study also introduced AdaBN for target domain adaptation. The results showed that EEGNet with Riemannian alignment and AdaBN could achieve the best transfer accuracy about 65.6% on the target dataset. This study may provide new insights into the design of transfer neural networks for BCIs by separating source and target batch normalization layers in the domain adaptation process.
Situation awareness (SA) has received much attention in recent years because of its importance for operators of dynamic systems. Electroencephalography (EEG) can be used to measure mental states of operators related to SA. However, cross-subject EEG-based SA recognition is a critical challenge, as data distributions of different subjects vary significantly. Subject variability is considered as a domain shift problem. Several attempts have been made to find domain-invariant features among subjects, where subject-specific information is neglected. In this work, we propose a simple but efficient subject matching framework by finding a connection between a target (test) subject and source (training) subjects. Specifically, the framework includes two stages: (1) we train the model with multi-source domain alignment layers to collect source domain statistics. (2) During testing, a distance is computed to perform subject matching in the latent representation space. We use a reciprocal exponential function as a similarity measure to dynamically select similar source subjects. Experiment results show that our framework achieves a state-of-the-art accuracy 74.14% for the Taiwan driving dataset.
Electroencephalogram (EEG) signal analysis, which is widely used for human-computer interaction and neurological disease diagnosis, requires a large amount of labeled data for training. However, the collection of substantial EEG data could be difficult owing to its randomness and non-stationary. Moreover, there is notable individual difference in EEG data, which affects the reusability and generalization of models. For mitigating the adverse effects from the above factors, transfer learning is applied in this field to transfer the knowledge learnt in one domain into a different but related domain. Transfer learning adjusts models with small-scale data of the task, and also maintains the learning ability with individual difference. This paper describes four main methods of transfer learning and explores their practical applications in EEG signal analysis in recent years. Finally, we discuss challenges and opportunities of transfer learning and suggest areas for further study.
A brain-computer interface (BCI) enables a user to communicate with a computer directly using brain signals. The most common non-invasive BCI modality, electroencephalogram (EEG), is sensitive to noise/artifact and suffers between-subject/within-subject non-stationarity. Therefore, it is difficult to build a generic pattern recognition model in an EEG-based BCI system that is optimal for different subjects, during different sessions, for different devices and tasks. Usually, a calibration session is needed to collect some training data for a new subject, which is time-consuming and user unfriendly. Transfer learning (TL), which utilizes data or knowledge from similar or relevant subjects/sessions/devices/tasks to facilitate learning for a new subject/session/device/task, is frequently used to reduce the amount of calibration effort. This paper reviews journal publications on TL approaches in EEG-based BCIs in the last few years, i.e., since 2016. Six paradigms and applications – motor imagery, event-related potentials, steady-state visual evoked potentials, affective BCIs, regression problems, and adversarial attacks – are considered. For each paradigm/application, we group the TL approaches into cross-subject/session, cross-device, and cross-task settings and review them separately. Observations and conclusions are made at the end of the paper, which may point to future research directions.
A brain-computer interface (BCI) system usually needs a long calibration session for each new subject/task to adjust its parameters, which impedes its transition from the laboratory to real-world applications. Domain adaptation, which leverages labeled data from auxiliary subjects/tasks (source domains), has demonstrated its effectiveness in reducing such calibration effort. Currently, most domain adaptation approaches require the source domains to have the same feature space and label space as the target domain, which limits their applications, as the auxiliary data may have different feature spaces and/or different label spaces. This paper considers different set domain adaptation for BCIs, i.e., the source and target domains have different label spaces.We introduce a practical setting of different label sets for BCIs, and propose a novel label alignment (LA) approach to align the source label space with the target label space. It has three desirable properties: 1) LA only needs as few as one labeled sample from each class of the target subject; 2) LA can be used as a preprocessing step before different feature extraction and classification algorithms; and, 3) LA can be integrated with other domain adaptation approaches to achieve even better performance. Experiments on two motor imagery datasets demonstrated the effectiveness of LA.
The development of “intelligent” systems that can take decisions and perform autonomously might lead to faster and more consistent decisions. A limiting factor for a broader adoption of AI technology is the inherent risks that come with giving up human control and oversight to “intelligent” machines. Forsensitive tasks involving critical infrastructures and affecting human well-being or health, it is crucial to limit the possibility of improper, non-robust and unsafe decisions and actions. Before deploying an AI system, we see a strong need to validate its behavior, and thus establish guarantees that it will continue to perform as expected when deployed in a real-world environment. In pursuit of that objective, ways for humans to verify the agreement between the AI decision structure and their own ground-truth knowledge have been explored. Explainable AI (XAI) has developed as a subfield of AI, focused on exposing complex AI models to humans in a systematic and interpretable manner.
The 22 chapters included in this book provide a timely snapshot of algorithms, theory, and applications of interpretable and explainable AI and AI techniques that have been proposed recently reflecting the current discourse in this field and providing directions of future development. The book is organized in six parts: towards AI transparency; methods for interpreting AI systems; explaining the decisions of AI systems; evaluating interpretability and explanations; applications of explainable AI; and software for explainable AI.
Objective:
This paper targets a major challenge in developing practical electroencephalogram (EEG)-based brain-computer interfaces (BCIs): how to cope with individual differences so that better learning performance can be obtained for a new subject, with minimum or even no subject-specific data?
Methods:
We propose a novel approach to align EEG trials from different subjects in the Euclidean space to make them more similar, and hence improve the learning performance for a new subject. Our approach has three desirable properties: first, it aligns the EEG trials directly in the Euclidean space, and any signal processing, feature extraction, and machine learning algorithms can then be applied to the aligned trials; second, its computational cost is very low; and third, it is unsupervised and does not need any label information from the new subject.
Results:
Both offline and simulated online experiments on motor imagery classification and event-related potential classification verified that our proposed approach outperformed a state-of-the-art Riemannian space data alignment approach, and several approaches without data alignment.
Conclusion:
The proposed Euclidean space EEG data alignment approach can greatly facilitate transfer learning in BCIs.
Significance:
Our proposed approach is effective, efficient, and easy to implement. It could be an essential pre-processing step for EEG-based BCIs.
Current sleep medicine relies on the supervised analysis of polysomnographic measurements, comprising amongst others electroencephalogram (EEG), electromyogram (EMG), and electrooculogram (EOG) signals. Convolutional neural networks (CNN) provide an interesting framework to automated classification of sleep based on these raw waveforms. In this study, we compare existing CNN approaches to four databases of pathological and physiological subjects. The best performing model resulted in Cohen's Kappa of on healthy subjects and on patients suffering from a variety of sleep disorders. Further, we show the advantages of additional sensor data (i.e., EOG and EMG). Deep learning approaches require a lot of data which is scarce for less prevalent diseases. For this, we propose a transfer learning procedure by pretraining a model on large public data and fine-tune this on each subject from a smaller dataset. This procedure is demonstrated using a private REM Behaviour Disorder database, improving sleep classification by 24.4%.
Objective:
This paper tackles the problem of transfer learning in the context of EEG-based Brain Computer Interface (BCI) classification. In particular the problems of cross-session and cross-subject classification are considered. These problems concern the ability to use data from previous sessions or from a database of past users to calibrate and initialize the classifier, allowing a calibration-less BCI mode of operation.
Methods:
Data are represented using spatial covariance matrices of the EEG signals, exploiting the recent successful techniques based on the Riemannian geometry of the manifold of Symmetric Positive Definite (SPD) matrices. Cross-session and cross-subject classification can be difficult, due to the many changes intervening between sessions and between subjects, including physiological, environmental, as well as instrumental changes. Here we propose to affine transform the covariance matrices of every session/subject in order to center them with respect to a reference covariance matrix, making data from different sessions/subjects comparable. Then, classification is performed both using a standard Minimum Distance to Mean (MDM) classifier, and through a probabilistic classifier recently developed in the literature, based on a density function (mixture of Riemannian Gaussian distributions) defined on the SPD manifold.
Results:
The improvements in terms of classification performances achieved by introducing the affine transformation are documented with the analysis of two BCI data sets.
Conclusion and significance:
Hence, we make, through the affine transformation proposed, data from different sessions and subject comparable, providing a significant improvement in the BCI transfer learning problem.
Multi-task learning (MTL) has led to successes in many applications of machine learning, from natural language processing and speech recognition to computer vision and drug discovery. This article aims to give a general overview of MTL, particularly in deep neural networks. It introduces the two most common methods for MTL in Deep Learning, gives an overview of the literature, and discusses recent advances. In particular, it seeks to help ML practitioners apply MTL by shedding light on how MTL works and providing guidelines for choosing appropriate auxiliary tasks.
The performance of brain-computer interfaces (BCIs) improves with the amount
of available training data, the statistical distribution of this data, however,
varies across subjects as well as across sessions within individual subjects,
limiting the transferability of training data or trained models between them.
In this article, we review current transfer learning techniques in BCIs that
exploit shared structure between training data of multiple subjects and/or
sessions to increase performance. We then present a framework for transfer
learning in the context of BCIs that can be applied to any arbitrary feature
space, as well as a novel regression estimation method that is specifically
designed for the structure of a system based on the electroencephalogram (EEG).
We demonstrate the utility of our framework and method on subject-to-subject
transfer in a motor-imagery paradigm as well as on session-to-session transfer
in one patient diagnosed with amyotrophic lateral sclerosis (ALS), showing that
it is able to outperform other comparable methods on an identical dataset.
In the deep neural network (DNN), the hidden layers can be considered as increasingly complex feature transformations and the final softmax layer as a log-linear classifier making use of the most abstract features computed in the hidden layers. While the loglinear classifier should be different for different languages, the feature transformations can be shared across languages. In this paper we propose a shared-hidden-layer multilingual DNN (SHL-MDNN), in which the hidden layers are made common across many languages while the softmax layers are made language dependent. We demonstrate that the SHL-MDNN can reduce errors by 3-5%, relatively, for all the languages decodable with the SHL-MDNN, over the monolingual DNNs trained using only the language specific data. Further, we show that the learned hidden layers sharing across languages can be transferred to improve recognition accuracy of new languages, with relative error reductions ranging from 6% to 28% against DNNs trained without exploiting the transferred hidden layers. It is particularly interesting that the error reduction can be achieved for the target language that is in different families of the languages used to learn the hidden layers.
A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.
To assess age-related topographic changes in the sleep electroencephalogram (EEG).
The sleep EEG records of young (mean age, 22.3 years) and middle-aged (mean age, 62.0 years) healthy men were compared. The EEG was obtained from 3 bipolar derivations (frontal-central (FC), central-parietal (CP), and parietal-occipital (PO)) along the antero-posterior axis.
The total sleep time, sleep efficiency, stage 2 and slow wave sleep (SWS) were lower in the middle-aged group, while sleep latency, stage 1 and wakefulness after sleep onset were higher. Spectral analysis documented the age-related reduction of EEG power in non-REM sleep (0.25-14 Hz), and REM sleep (0.75-10 Hz). However, the reduction was not uniform over the 3 derivations, but was most pronounced in the anterior derivation (FC) in the theta (both sleep states) and high-alpha/low-sigma bands (non-REM sleep).
These changes can be interpreted as age-related shifts of power from the anterior (FC) towards the middle derivation (CP). Aging not only reduces power in the sleep EEG, but causes frequency-specific changes in the brain topography. The results are consistent with the notion of sleep as a local process.
The balanced accuracy and its posterior distribution
- Kay Henning Brodersen
- Cheng Soon Ong
- Klaas Enno Stephan
- Joachim M Buhmann
Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. The balanced accuracy and its posterior distribution. In 2010 20th international
conference on pattern recognition, pages 3121-3124. IEEE, 2010.
Riemannian batch normalization for spd neural networks
- Daniel Brooks
- Olivier Schwander
- Frederic Barbaresco
- Jean-Yves Schneider
- Matthieu Cord
Daniel Brooks, Olivier Schwander, Frederic Barbaresco, Jean-Yves Schneider, and Matthieu
Cord. Riemannian batch normalization for spd neural networks. In H. Wallach,
H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.,
2019.
A riemannian network for spd matrix learning
- Zhiwu Huang
- Luc Van Gool
Zhiwu Huang and Luc Van Gool. A riemannian network for spd matrix learning. In
Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI'17,
page 2036-2042. AAAI Press, 2017.
Adam: A method for stochastic optimization
- P Diederik
- Jimmy Kingma
- Ba
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv
preprint arXiv:1412.6980, 2014.
Meta-Learning EEG Motor Imagery Decoder for Brain Computer Interface
- Denghao Li
- Pablo Ortega
- Xiaoxi Wei
Denghao Li, Pablo Ortega, Xiaoxi Wei, and A.A. Faisal. Meta-Learning EEG Motor Imagery
Decoder for Brain Computer Interface. IEEE Neural Engineering (NER), 10, 2021a.
- Mingsheng Long
- Yue Cao
- Jianmin Wang
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. Learning transferable
features with deep adaptation networks. arXiv preprint arXiv:1502.02791, 2015.
Visualizing data using t-sne
- Laurens Van Der Maaten
- Geoffrey Hinton
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of
Machine Learning Research, 9(86):2579-2605, 2008.
Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola
- Manzil Zaheer
- Satwik Kottur
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. Deep sets. arXiv preprint arXiv:1703.06114, 2017.