Figure 4 - available via license: CC BY
Content may be subject to copyright.
Model for binary classification using (a) Fully Connected Neural Network (FCNN) architecture and (b) a Convolutional Neural Network (CNN) architecture.
Source publication
Multiplexed deep neural networks (DNN) have engendered high-performance predictive models gaining popularity for decoding brain waves, extensively collected in the form of electroencephalogram (EEG) signals. In this paper, to the best of our knowledge, we introduce a first-ever DNN-based generalized approach to estimate reaction time (RT) using the...
Context in source publication
Citations
... With the ever-increasing access to more data, there is a growing interest in utilising deep learning technologies for data analysis [4]. The most commonly implemented networks are Convolutional Neural Networks (CNN, 1D and 2D) [5], Recurrent Neural Networks (RNN) [6], Auto-Encoders (AE) [7], Deep Belief Networks (DBN) and Multi-Layer Perceptron (MLP) [8]. CNNs are proving to be powerful tools in deep learning due to their computational efficiency and ability to extract features with high inter-class variance [9,10,11]. ...
Epilepsy is a highly prevalent brain condition with many serious complications arising from it. The majority of patients which present to a clinic and undergo electroencephalogram (EEG) monitoring would be unlikely to experience seizures during the examination period, thus the presence of interictal epileptiform discharges (IEDs) become effective markers for the diagnosis of epilepsy. Furthermore, IED shapes and patterns are highly variable across individuals, yet trained experts are still able to identify them through EEG recordings - meaning that commonalities exist across IEDs that an algorithm can be trained on to detect and generalise to the larger population. This research proposes an IED detection system for the binary classification of epilepsy using scalp EEG recordings. The proposed system features an ensemble based deep learning method to boost the performance of a residual convolutional neural network, and a bidirectional long short-term memory network. This is implemented using raw EEG data, sourced from Temple University Hospital's EEG Epilepsy Corpus, and is found to outperform the current state of the art model for IED detection across the same dataset. The achieved accuracy and Area Under Curve (AUC) of 94.92% and 97.45% demonstrates the effectiveness of an ensemble method, and that IED detection can be achieved with high performance using raw scalp EEG data, thus showing promise for the proposed approach in clinical settings.
... Periodogram is the simplest form of estimating power spectral density values and it is applied directly to the raw EEG signals. It is the square of the absolute value Fourier transform of the time domain signal [22]. The formula of the multitaper method is given in Eq. 1 [23]: ...
Schizophrenia is a complex psychiatric disorder characterized by delusions, hallucinations, disorganized speech, mood disturbances, and abnormal behavior. Early diagnosis of schizophrenia depends on the manifestation of the disorder, its symptoms are complex, heterogeneous and cannot be clearly separated from other neurological categories. Therefore, its early diagnosis is quite difficult. An objective, effective and simple diagnostic model and procedure are essential for diagnosing schizophrenia. Electroencephalography (EEG)-based models are a strong candidate to overcome these limits. In this study, we proposed an EEG-based solution for the diagnosis of schizophrenia using 1D-convolutional neural network deep learning approach and multitaper method. Firstly, the raw EEG signals were segmented and denoised using multiscale principal component analysis. Then, three different feature sets were extracted using leading feature extraction methods such as periodogram, welch, and multitaper. The performance of each feature extraction method was compared. Finally, classification performance of support vector machine, decision trees, k-nearest neighbors, and 1D-convolutional neural network algorithms were tested according to model evaluation criteria. The highest performance was obtained with the multitaper and 1D-convolutional neural network approach, and the highest accuracy was 98.76%. The results of the model were found to be 0.991 sensitivity, 0.984 precision, 0.983 specificity, 0.975 Matthews correlation coefficient, 0.987 f1-score, and 0.975 kappa statistic. This study presents the multitaper and 1D-convolutional neural network approach framework for the first time in the diagnosis of schizophrenia. Moreover, this study achieved satisfactorily high classification performance for the diagnosis of schizophrenia compared to methods in the relevant literature.
... Our aim was to evaluate the capability of a deep learning model to provide a superior method for classification of a motor task versus rest using MRCP features of EEG. Deep learning has recently demonstrated promising results in the classification of brain activity for applications such as hand movement classification [10], virtual reality based BCI-NFT [11] and exoskeleton-based BCI-NFT [12]. ...
Brain computer interface (BCI) systems were initially developed to replace lost function; however, they are being increasingly utilized in rehabilitation to restore motor functioning after brain injury. In such BCI-mediated neurofeedback training (BCI-NFT), the brain-state associated with movement attempt or intention is used to activate an external device which assists the movement while providing sensory feedback to enhance neuroplasticity. A critical element in the success of BCI-NFT is accurate timing of the feedback within the active period of the brain state. The overarching goal of this work was to develop a reliable deep learning model that can predict motion before its onset, and thereby deliver the sensory stimuli in a timely manner for BCI-NFT applications. To this end, the main objective of the current study was to design and evaluate a Multi-layer Perceptron Neural Network (MLP-NN). Movement-related cortical potentials (MRCP) during planning and execution of ankle dorsiflexion was used to train the model to classify dorsiflexion planning vs. rest. The accuracy and reliability of the model was evaluated offline using data from eight healthy individuals (age: 26.3 ± 7.6 years). First, we evaluated three different epoching strategies for defining our 2 classes, to identify the one which best discriminated rest from dorsiflexion. The best model accuracy for predicting ankle dorsiflexion from EEG before movement execution was 84.7%. Second, the effect of various spatial filters on the model accuracy was evaluated, demonstrating that the spatial filtering had minimal effect on model accuracy and reliability.
... Based on a sensitivity analysis of the GLM and NN models, we found that right frontal low gamma was the most important EEG feature for predicting RT. Additionally, groups of EEG features in the high beta (20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35) and low gamma Hz) bands modulated with RT. Overall, by using a machine learning approach, we identified EEG features that may help us to better understand which areas and connections in the brain underlie participants' RT variability related to both distraction control and working memory maintenance. ...
... After, each component was plotted onto a 2D map of the scalp, and components associated with eye blinks were selected manually and removed by projecting the sum of selected non-artifactual components back onto the scalp. Then, for each trial and electrode, we filtered the cleaned EEG signal, using another 1650th order FIR filter, into five different frequency bands: theta (4-8 Hz), alpha (8-12 Hz), low beta (12)(13)(14)(15)(16)(17)(18)(19)(20), high beta (20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30), and low gamma . Finally, the time period of interest, 1400 ms prior to the distractor (Delay 1), was extracted for each filtered signal. ...
... After, each component was plotted onto a 2D map of the scalp, and components associated with eye blinks were selected manually and removed by projecting the sum of selected non-artifactual components back onto the scalp. Then, for each trial and electrode, we filtered the cleaned EEG signal, using another 1650th order FIR filter, into five different frequency bands: theta (4-8 Hz), alpha (8-12 Hz), low beta (12)(13)(14)(15)(16)(17)(18)(19)(20), high beta (20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30), and low gamma . Finally, the time period of interest, 1400 ms prior to the distractor (Delay 1), was extracted for each filtered signal. ...
Value-driven attention capture (VDAC) occurs when previously rewarded stimuli capture attention and impair goal-directed behavior. In a working memory (WM) task with VDAC-related distractors, we observe behavioral variability both within and across individuals. Individuals differ in their ability to maintain relevant information and ignore distractions. These cognitive components shift over time with changes in motivation and attention, making it difficult to identify underlying neural mechanisms of individual differences. In this study, we develop the first participant-specific feedforward neural network models of reaction time from neural data during a VDAC WM task. We used short epochs of electroencephalography (EEG) data from 16 participants to develop the feedforward neural network (NN) models of RT aimed at understanding both WM and VDAC. Using general linear models (GLM), we identified 20 EEG features to predict RT across participants (r=0.53±0.08). The linear model was compared to the NN model, which improved the predicted trial-by-trial RT for all participants (r=0.87±0.04). We found that right frontal gamma-band activity and fronto-posterior functional connectivity in the alpha, beta, and gamma bands explain individual differences. Our study shows that NN models can link neural activity to highly variable behavior and can identify potential new targets for neuromodulation interventions.
... Raw multichannel EEG data were first filtered using an FIR bandpass filter using the 4 Hz and 40 Hz limits. Theta waves (4-8 Hz) and alpha waves (8-12 Hz) have been shown to be the most active in visual tasks [62]. However, beta (15-31 Hz) and gamma bands provide information about cognitive processes related to visual perception [19]. ...
Visual perception is an important part of human life. In the context of facial recognition, it allows us to distinguish between emotions and important facial features that distinguish one person from another. However, subjects suffering from memory loss face significant facial processing problems. If the perception of facial features is affected by memory impairment, then it is possible to classify visual stimuli using brain activity data from the visual processing regions of the brain. This study differentiates the aspects of familiarity and emotion by the inversion effect of the face and uses convolutional neural network (CNN) models (EEGNet, EEGNet SSVEP (steady-state visual evoked potentials), and DeepConvNet) to learn discriminative features from raw electroencephalography (EEG) signals. Due to the limited number of available EEG data samples, Generative Adversarial Networks (GAN) and Variational Autoencoders (VAE) are introduced to generate synthetic EEG signals. The generated data are used to pretrain the models, and the learned weights are initialized to train them on the real EEG data. We investigate minor facial characteristics in brain signals and the ability of deep CNN models to learn them. The effect of face inversion was studied, and it was observed that the N170 component has a considerable and sustained delay. As a result, emotional and familiarity stimuli were divided into two categories based on the posture of the face. The categories of upright and inverted stimuli have the smallest incidences of confusion. The model’s ability to learn the face-inversion effect is demonstrated once more.
... These studies portrayed subject-specific prediction methods. In our previous works [7], [8], [9] we created generalized models using machine learning and deep learning-based techniques to solve this problem and achieved a maximum CC 0f 0.80 and a minimum RMSE of 108.6 ms. This study tried to go a step further, using a more meaningful periodogram-based EEG data structure and developing a novel 3D neural network architecture to predict RT. ...
... In light of prior studies [9], [12], [13], we designed a simple 3D convolutional neural network (CNN) to predict RTs from the cuboids computed earlier. The model consists of 2 3D convolutional (Conv3D) layers and 3 fully connected (FC) layers. ...
... Using the CNN model, we achieved an RMSE of 91.5 ms and a CC of 0.83. The actual vs. predicted RTs for the testing set using our model are shown in figure 5. Also, a comparison of obtained RMSE and CC values for RT estimation among previous models [7], [8], [9] and the 3D CNN architecture is shown in Table I. Clearly, the 3D CNN model exhibited the best performance with improved results. ...
The study of human reaction time (RT) is invaluable not only to understand the sensory-motor functions but also to translate brain signals into machine comprehensible commands that can facilitate augmentative and alternative communication using brain-computer interfaces (BCI). Recent developments in sensor technologies, hardware computational capabilities, and neural network models have significantly helped advance biomedical signal processing research. This study is an attempt to utilize state-of-the-art resources to explore the relationship between human behavioral responses during perceptual decision-making and corresponding brain signals in the form of electroencephalograms (EEG). In this paper, a generalized 3D convolutional neural network (CNN) architecture is introduced to estimate RT for a simple visual task using single-trial multi-channel EEG. Earlier comparable studies have also employed a number of machine learning and deep learning-based models, but none of them considered inter-channel relationships while estimating RT. On the contrary, the use of 3D convolutional layers enabled us to consider the spatial relationship among adjacent channels while simultaneously utilizing spectral information from individual channels. Our model can predict RT with a root mean square error of 91.5 ms and a correlation coefficient of 0.83. These results surpass all the previous results attained from different studies.Clinical relevance Novel approaches to decode brain signals can facilitate research on brain-computer interfaces (BCIs), psychology, and neuroscience, enabling people to utilize assistive devices by root-causing psychological or neuromuscular disorders.
... As a result, they observed that images with higher complexity caused fixation eye movements with lower fractality. Chowdhury et al. (Chowdhury et al. 2020), presented a DNN-based approach to estimate reaction time (RT) using a single-attempt EEG representation in a visual stimulus-response experiment with 48 participants, achieving 94% accuracy for binary classification. Zheng et al. (Zheng et al. 2020), proposed a method for estimation by classifying the EEG signals from the Long Short Term Memory (LSTM) based EEG encoder and ResNet deep learning algorithm by matching the features, they extracted from the visuals shown to the person to be taken, and as a result, they made the classification with 90.16% accuracy. ...
... In this case, it could be possible to leverage the InstaGATs's spatial-dependant embeddings to identify the most involved brain areas. Also, in case of reaction tasks [43] (when the individual is expected to promptly react to an external stimulus), time-dependant features could be filtered out more accurately by an architecture such as LSTM+Att. We can also highlight 9 that, despite their simple architectures, the attention mechanism allowed the models to achieve high accuracy levels across different real-world scenarios with minimal pre-processing. ...
Objective: To evaluate the impact on Electroencephalography (EEG) classification of different kinds of attention mechanisms in Deep Learning (DL) models. Methods: We compared three attention-enhanced DL models, the brand-new InstaGATs, an LSTM with attention and a CNN with attention. We used these models to classify normal and abnormal (i.e., artifactual or pathological) EEG patterns. Results: We achieved the state of the art in all classification problems, regardless the large variability of the datasets and the simple architecture of the attention-enhanced models. We could also prove that, depending on how the attention mechanism is applied and where the attention layer is located in the model, we can alternatively leverage the information contained in the time, frequency or space domain of the dataset. Conclusions: with this work, we shed light over the role of different attention mechanisms in the classification of normal and abnormal EEG patterns. Moreover, we discussed how they can exploit the intrinsic relationships in the temporal, frequency and spatial domains of our brain activity. Significance: Attention represents a promising strategy to evaluate the quality of the EEG information, and its relevance, in different real-world scenarios. Moreover, it can make it easier to parallelize the computation and, thus, to speed up the analysis of big electrophysiological (e.g., EEG) datasets.
