Article

Data Analytics in Steady-State Visual Evoked Potential-Based Brain–Computer Interface: A Review

Authors:
If you want to read the PDF, try requesting it from the authors.

Abstract

Electroencephalograph (EEG) has been widely applied for brain-computer interface (BCI) which enables paralyzed people to directly communicate with and control of external devices, due to its portability, high temporal resolution, ease of use and low cost. Of various EEG paradigms, steady-state visual evoked potential (SSVEP)-based BCI system which uses multiple visual stimuli (such as LEDs or boxes on a computer screen) flickering at different frequencies has been widely explored in the past decades due to its fast communication rate and high signal-to-noise ratio. In this paper, we review the current research in SSVEP-based BCI, focusing on the data analytics that enables continuous, accurate detection of SSVEPs and thus high information transfer rate. The main technical challenges, including signal pre-processing, spectrum analysis, signal decomposition, spatial filtering in particular canonical correlation analysis and its variations, and classification techniques are described in this paper. Research challenges and opportunities in spontaneous brain activities, mental fatigue, transfer learning as well as hybrid BCI are also discussed.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Commonly used control signal in EEG-based BCIs is SSVEP [3,4]. SSVEP is a resonance phenomenon that occurs mainly in the visual cortex when an individual's visual attention focuses on a light source that flickers with a frequency above 6 Hz [5,6,7]. Also, SSVEP consists of a periodic component of the same frequency as the flickering light source, likewise of many harmonic frequencies [5]. ...
... The interest in SSVEP based BCI studies is mainly owing to the robustness of the SSVEP phenomenon. Besides, it has advantages such as high information transfer rate (ITR), simple system structure, short user training, and short time requirement [5][6][7][8][9]. ...
... In the classification phase, a single classifier was used in many EEG-based BCI systems [7,8,21,22]. On the other hand, combinations of classifiers are very useful in synchronous experiments [13-17, 26, 27]. ...
Article
Full-text available
Brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEP) have been acceleratingly used in different application areas from entertainment to rehabilitation, like clinical neuroscience, cognitive, and use of engineering researches. Of various electroencephalography paradigms, SSVEP-based BCI systems enable apoplectic people to communicate with the outside world easily, due to their simple system structure, short or no training time, high temporal resolution, high information transfer rate, and affordable by comparing to other methods. SSVEP-based BCIs use multiple visual stimuli flickering at different frequencies to generate distinct commands. In this paper, we compared the classifier performances of combinations of binary commands flickering at seven different frequencies to determine which frequency pair gives the highest performance using temporal and spectral methods. For SSVEP frequency recognition, in total 25 temporal change characteristics of the signals and 15 frequency-based feature vectors extracted from the SSVEP signal. These feature vectors were applied to the input of seven well-known machine learning algorithms (Decision Tree, Discriminant Analysis, Logistic Regression, Naive Bayes, Support Vector Machines, Nearest Neighbour, and Ensemble Learning). In conclusion, we achieved 100% accuracy in 7.5 - 10 frequency pairs among these 2,520 distinct runs and we found that the most successful classifier is the Ensemble Learning classifier. The combination of these methods leads to an appropriate detailed and comparative analysis that represents the robustness and effectiveness of classical approaches.
... Commonly used control signal in EEG-based BCIs is SSVEP [3,4]. SSVEP is a resonance phenomenon that occurs mainly in the visual cortex when an individual's visual attention focuses on a light source that flickers with a frequency above 6 Hz [5,6,7]. Also, SSVEP consists of a periodic component of the same frequency as the flickering light source, likewise of many harmonic frequencies [5]. ...
... The interest in SSVEP based BCI studies is mainly owing to the robustness of the SSVEP phenomenon. Besides, it has advantages such as high information transfer rate (ITR), simple system structure, short user training, and short time requirement [5][6][7][8][9]. ...
... In the classification phase, a single classifier was used in many EEG-based BCI systems [7,8,21,22]. On the other hand, combinations of classifiers are very useful in synchronous experiments [13-17, 26, 27]. ...
Conference Paper
Abstract-Brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEP) have been acceleratingly used in different application areas from entertainment to rehabilitation, like clinical neuroscience, cognitive, and use of engineering researches. Of various electroencephalography paradigms, SSVEP-based BCI systems enable apoplectic people to communicate with outside world easily, due to their simple system structure, short or no training time, high temporal resolution, high information transfer rate, and affordable by comparing to other methods. SSVEP-based BCIs use multiple visual stimuli flickering at different frequencies to generate distinct commands. In this paper, we compared the classifier performances of combinations of binary commands flickering at seven different frequencies to determine which frequency pair gives the highest performance using temporal and spectral methods. For SSVEP frequency recognition, in total 25 temporal change characteristics of the signals and 15 frequency-based feature vectors extracted from the SSVEP signal. These feature vectors were applied to the input of seven well-known machine learning algorithms (Decision Tree, Discriminant Analysis, Logistic Regression, Naive Bayes, Support Vector Machines, Nearest Neighbour, and Ensemble Learning). In conclusion, we achieved 100% accuracy in 7.5-10 frequency pairs among these 2,520 distinct runs and we found that the most successful classifier is the Ensemble Learning classifier. The combination of these methods leads to an appropriate detailed and comparative analysis that represents the robustness and effectiveness of classical approaches.
... The response EEG has peak frequencies that are the fundamental and harmonic frequencies of the flicker stimulation. An SSVEP-based BCI is realized by assigning commands or alphabets to flickers having different frequencies [16,17]. BCIs can determine the intended commands of users by detecting the peak frequency from observed EEGs. ...
... BCIs can determine the intended commands of users by detecting the peak frequency from observed EEGs. SSVEP-based BCIs achieve the highest information transfer rate (ITR) among state-of-the-art BCI implementations [16][17][18]. There are many extensions and hybrid BCIs for SSVEP-based BCIs. ...
Article
Full-text available
The steady-state visual evoked potential (SSVEP), which is a kind of event-related potential in electroencephalograms (EEGs), has been applied to brain–computer interfaces (BCIs). SSVEP-based BCIs currently perform the best in terms of information transfer rate (ITR) among various BCI implementation methods. Canonical component analysis (CCA) or spectrum estimation, such as the Fourier transform, and their extensions have been used to extract features of SSVEPs. However, these signal extraction methods have a limitation in the available stimulation frequency; thus, the number of commands is limited. In this paper, we propose a complex valued convolutional neural network (CVCNN) to overcome the limitation of SSVEP-based BCIs. The experimental results demonstrate that the proposed method overcomes the limitation of the stimulation frequency, and it outperforms conventional SSVEP feature extraction methods.
... Several commonly used EEG paradigms include steady-state visual evoked potential (SSVEP), P300 (Farwell et al., 2014;Yin et al., 2016), and motor imagery (MI) (Song and Kim, 2019;Xu et al., 2021). Compared to the EEG paradigms of P300 and MI, the SSVEP-based BCI system is preferable in robotic arms control owing to the little training and relatively high recognition accuracy (Ge et al., 2019;Chen et al., 2020;Zhang et al., 2020). Besides, due to the limited output commands, it is not able to perform a real-time motion control task in a 3D space for the P300-based and MI-based robotic arm systems with multiple degrees of freedom (DOF) (Xu et al., 2019). ...
... The CCA and its extended methods are the mainstream approaches for feature extraction in SSVEP-based BCI since their simple implementation and enhancing the SNR of SSVEP signals when using multiple channels (Wang et al., 2014). However, the reference signal of the CCA method is an overly idealized model, which is powerless to weaken the effect of spontaneous EEG and other background noise in the multi-channel signal due to the lack of real information Zhang et al., 2020). At the same time, the template-based CCA method can optimize the reference signals by extracting more time-domain feature information from the EEG data. ...
Article
Full-text available
Recently, the robotic arm control system based on a brain-computer interface (BCI) has been employed to help the disabilities to improve their interaction abilities without body movement. However, it's the main challenge to implement the desired task by a robotic arm in a three-dimensional (3D) space because of the instability of electroencephalogram (EEG) signals and the interference by the spontaneous EEG activities. Moreover, the free motion control of a manipulator in 3D space is a complicated operation that requires more output commands and higher accuracy for brain activity recognition. Based on the above, a steady-state visual evoked potential (SSVEP)-based synchronous BCI system with six stimulus targets was designed to realize the motion control function of the seven degrees of freedom (7-DOF) robotic arm. Meanwhile, a novel template-based method, which builds the optimized common templates (OCTs) from various subjects and learns spatial filters from the common templates and the multichannel EEG signal, was applied to enhance the SSVEP recognition accuracy, called OCT-based canonical correlation analysis (OCT-CCA). The comparison results of offline experimental based on a public benchmark dataset indicated that the proposed OCT-CCA method achieved significant improvement of detection accuracy in contrast to CCA and individual template-based CCA (IT-CCA), especially using a short data length. In the end, online experiments with five healthy subjects were implemented for achieving the manipulator real-time control system. The results showed that all five subjects can accomplish the tasks of controlling the manipulator to reach the designated position in the 3D space independently.
... The decoding algorithm is an important aspect of SSVEPbased BCIs. In single-frequency stimulated SSVEP decoding, the canonical correlation analysis (CCA) is one of the most popular methods [11]. CCA evaluates correlations between the recorded data and pre-determined reference signal templates that each represent the candidate frequencies. ...
Preprint
Full-text available
Stimulation methods that utilise more than one stimulation frequency have been developed in steady-state visual evoked potential (SSVEP) brain-computer interfaces (BCIs) for the purpose of increasing the number of targets that can be presented simultaneously. However, there is no unified decoding algorithm that can be applied to a large class of multi-frequency stimulated SSVEP settings. This paper extends the widely used canonical correlation analysis (CCA) decoder to explicitly accommodate multi-frequency SSVEP by exploiting the interactions between the multiple stimulation frequencies. A concept "order" was defined as the sum of absolute values of the coefficients in the linear interaction. The probability distribution of the order in the resulting SSVEP response was then used to improve decoding accuracy. Results show that, compared to the standard CCA formulation, the proposed multi-frequency CCA (MFCCA) has a 20% improvement in decoding accuracy on average at order 2. Although the proposed methods were only tested with two input frequencies, the technique is capable of handling more than two simultaneous input frequencies.
... filters applied transformations in the channel domain, which enhanced the SSVEP identification effectively via removing background artifacts [10]- [12]. As one of the most popular spatial filtering methods, canonical correlation analysis (CCA) seeks a pair of weights to maximise the correlation between SSVEP signals and sine-cosine reference signals [13]. ...
... BCI spellers developed using the SSVEP have received considerable research attention, and are being developed rapidly. A comprehensive review of BCI spellers has been provided in Reference [21], and reviews of specific classification algorithms, data analytics, and language models can be found in Reference [22][23][24], respectively. The paradigm of visual stimuli, including the procedure of target selection, layout of targets, manner of encoding of the stimuli (i.e., frequency and phase), and their combination with other triggering methods has an important influence on the performance of the BCI speller. ...
Article
Full-text available
The steady-state visual evoked potential (SSVEP), measured by the electroencephalograph (EEG), has high rates of information transfer and signal-to-noise ratio, and has been used to construct brain–computer interface (BCI) spellers. In BCI spellers, the targets of alphanumeric characters are assigned different visual stimuli and the fixation of each target generates a unique SSVEP. Matching the SSVEP to the stimulus allows users to select target letters and numbers. Many BCI spellers that harness the SSVEP have been proposed over the past two decades. Various paradigms of visual stimuli, including the procedure of target selection, layout of targets, stimulus encoding, and the combination with other triggering methods are used and considered to influence on the BCI speller performance significantly. This paper reviews these stimulus paradigms and analyzes factors influencing their performance. The fundamentals of BCI spellers are first briefly described. SSVEP-based BCI spellers, where only the SSVEP is used, are classified by stimulus paradigms and described in chronological order. Furthermore, hybrid spellers that involve the use of the SSVEP are presented in parallel. Factors influencing the performance and visual fatigue of BCI spellers are provided. Finally, prevailing challenges and prospective research directions are discussed to promote the development of BCI spellers.
... The flickering frequency is radiated throughout the brain. This stimulation produces electrical signals in the brain at the base frequency of the flashing light, as well as at its harmonics [10]. Practically, there is a marked reduction in the power of the SSVEP signals from the second harmonics onwards. ...
Chapter
Full-text available
Steady-state visual evoked potentials (SSVEPs) have been designated to be appropriate and are in use in many areas such as clinical neuroscience, cognitive science, and engineering. SSVEPs have become popular recently, due to their advantages including high bit rate, simple system structure and short training time. To design SSVEP-based BCI system, signal processing methods appropriate to the signal structure should be applied. One of the most appropriate signal processing methods of these non-stationary signals is the Wavelet Transform. In this study, we investigated both the effect of choosing a mother wavelet function and the most successful combination of classifier algorithm, wavelet features, and frequency pairs assigned to BCI commands. SSVEP signals that were recorded at seven different stimulus frequencies (6 – 6.5 – 7 – 7.5 – 8.2 – 9.3 – 10 Hz) were used in this study. A total of 115 features were extracted from time, frequency, and time-frequency domains. These features were classified by a total of seven different classification processes. Classification evaluation was presented with the 5-fold cross-validation method and accuracy values. According to the results, (I) the most successful wavelet function was Haar wavelet, (II) the most successful classifier was Ensemble Learning, (III) using the feature vector consisting of energy, entropy, and variance features yielded higher accuracy than using one of these features alone, and (IV) the highest performances were obtained in the frequency pairs with "6-10", "6.5-10", "7-10", and "7.5-10" Hz.
... Hence, the design of a passive BCI scheme is a comparatively difficult task. Some common EEG brain signals are P300 wave, evoked potential (EP), visually evoked potential (VEP), steady-state visually evoked potential (SSVEP), sensory evoked potential (SEP), somatosensory evoked potential (SSEP), auditory evoked potential (AEP), motor evoked potential (MEP), event-related potential (ERP), and error-related potential (ErrP), etc. [32][33][34]. ...
Conference Paper
Full-text available
A passive brain-computer interface (BCI) based upon electroencephalography (EEG) brain signals was developed to classify alert and drowsy states during the driving task. This BCI modality acquired electrical neuronal activity of five healthy male subjects from prefrontal and occipital cortices of the human brain for earlier drowsiness detection. Brain activity is recorded using a 16-channel EEG headset from these brain locations. Sleep-deprived subjects drove the vehicle in a simulated driving environment while neuronal activity was continuously monitored in prefrontal and occipital regions. Spectral band power and power spectral density estimate for α and β frequency bands were used as features along with k-nearest neighbor (kNN) and support vector machine (SVM) classifiers. Average classification accuracies are 95.8% for kNN and 93.8% for SVM with a 10-fold cross-validation model. Spectral analysis shows that α-rhythms are more prominent in the occipital region as compared to the prefrontal region during drowsy driving and hence vision-based brain data is more effective for earlier detection as compared to the focus-based brain data. The proposed EEG-based passive BCI scheme is promising for earlier differentiation between drowsy and alert states from the occipital region of the human brain.
... SSVEP complies with all of these aspects since its records are minimally invasive, collects multiple trials in a short period, and provides a high information transfer rate [25]. A well-designed SSVEP-based BCI system would decrease ocular fatigue by reducing the user's exposure to the flickering exogenous inputs. ...
Article
Full-text available
This paper presents the implementation of nonlinear canonical correlation analysis (NLCCA) approach to detect steady-state visual evoked potentials (SSVEP) quickly. The need for the fast recognition of proper stimulus to help end an SSVEP task in a BCI system is justified due to the flickering external stimulus exposure that causes users to start to feel fatigued. Measuring the accuracy and exposure time can be carried out through the information transfer rate—ITR, which is defined as a relationship between the precision, the number of stimuli, and the required time to obtain a result. NLCCA performance was evaluated by comparing it with two other approaches—the well-known canonical correlation analysis (CCA) and the least absolute reduction and selection operator (LASSO), both commonly used to solve the SSVEP paradigm. First, the best average ITR value was found from a dataset comprising ten healthy users with an average age of 28, where an exposure time of one second was obtained. In addition, the time sliding window responses were observed immediately after and around 200 ms after the flickering exposure to obtain the phase effects through the coefficient of variation (CV), where NLCCA obtained the lowest value. Finally, in order to obtain statistical significance to demonstrate that all approaches differ, the accuracy and ITR from the time sliding window responses was compared using a statistical analysis of variance per approach to identify differences between them using Tukey’s test.
... Compared with its invasive counterpart, noninvasive BCI boasts its safety and ease of use, which has the potential for wide applicability in able-bodied users. Among the noninvasive paradigms, steady-state visual evoked potential based BCI (SSVEP-BCI) has received increasing attention and constitutes one of the dominant paradigms to date [3]. Physiologically, SSVEP is elicited by periodic visual stimuli, e.g., flickers in a visual speller, and its frequency tagging attribute provides it with a relatively high signal-to-noise ratio (SNR). ...
Article
Objective: The steady-state visual evoked potential based brain-computer interface (SSVEP-BCI) implemented in dry electrodes is a promising paradigm for alternative and augmentative communication in real-world applications. To improve its performance and reduce the calibration effort for dry-electrode systems, we utilize cross-device transfer learning by exploiting auxiliary individual wet-electrode electroencephalogram (EEG). Methods: We proposed a novel transfer learning framework named ALign and Pool for EEG Headset domain Adaptation (ALPHA), which aligns the spatial pattern and the covariance for domain adaptation. To evaluate its efficacy, 75 subjects performed an experiment of 2 sessions involving a 12-target SSVEP-BCI task. Results: ALPHA significantly outperformed a baseline approach (canonical correlation analysis, CCA) and two competing transfer learning approaches (transfer template CCA, ttCCA and least square transformation, LST) in two transferring directions. When transferring from wet to dry EEG headsets, ALPHA significantly outperformed the fully calibrated approach of task-related component analysis (TRCA). Conclusion: ALPHA advances the frontier of recalibration-free cross-device transfer learning for SSVEP-BCIs and boosts the performance of dry electrode based systems. Significance: ALPHA has methodological and practical implications and pushes the boundary of dry electrode based SSVEP-BCI toward real-world applications.
... The CCA-based SSVEP visual acuity achieved a difference of 0.039 logMAR and a limit of agreement of 0.202 logMAR from FrACT visual acuity, and that for MSI-based SSVEP visual acuity were −0.080 logMAR and 0.208 logMAR, which was all lower than them of SSVEP visual acuity for the native combination with a difference and a limit of agreement of −0.095 logMAR and 0.253 logMAR. Since the spatial filtering methods can enhance the SNR of SSVEPs and suppress the non-SSVEP noise (Nakanishi et al., 2018b), this result illustrated that the unrelated noise, e.g., EMG and EOG (Friman et al., 2007;Zhang et al., 2021), was one of the reasons for the difference between SSVEP and behavioral visual acuity (Hamilton et al., 2021b), and the other methods of enhancing the SNR, such as signal preprocessing (Kołodziej et al., 2016), e.g., time-domain filtering and blind source separation (BSS) (Ji et al., 2019), and SSVEP recognition algorithms , e.g., wavelet transform (WT) (Rejer, 2017) and empirical mode decomposition (EMD) (Huang et al., 2013;Tello et al., 2014), may also have the property to improve the agreement between SSVEP and behavioral visual acuity. ...
Article
Full-text available
The purpose of this study was to enhance the performance of steady-state visual evoked potential (SSVEP)-based visual acuity assessment with spatial filtering methods. Using the vertical sinusoidal gratings at six spatial frequency steps as the visual stimuli for 11 subjects, SSVEPs were recorded from six occipital electrodes (O1, Oz, O2, PO3, POz, and PO4). Ten commonly used training-free spatial filtering methods, i.e., native combination (single-electrode), bipolar combination, Laplacian combination, average combination, common average reference (CAR), minimum energy combination (MEC), maximum contrast combination (MCC), canonical correlation analysis (CCA), multivariate synchronization index (MSI), and partial least squares (PLS), were compared for multielectrode signals combination in SSVEP visual acuity assessment by statistical analyses, e.g., Bland–Altman analysis and repeated-measures ANOVA. The SSVEP signal characteristics corresponding to each spatial filtering method were compared, determining the chosen spatial filtering methods of CCA and MSI with a higher performance than the native combination for further signal processing. After the visual acuity threshold estimation criterion, the agreement between the subjective Freiburg Visual Acuity and Contrast Test (FrACT) and SSVEP visual acuity for the native combination (0.253 logMAR), CCA (0.202 logMAR), and MSI (0.208 logMAR) was all good, and the difference between FrACT and SSVEP visual acuity was also all acceptable for the native combination (−0.095 logMAR), CCA (0.039 logMAR), and MSI (−0.080 logMAR), where CCA-based SSVEP visual acuity had the best performance and the native combination had the worst. The study proved that the performance of SSVEP-based visual acuity can be enhanced by spatial filtering methods of CCA and MSI and also recommended CCA as the spatial filtering method for multielectrode signals combination in SSVEP visual acuity assessment.
... Electroencephalographic (EEG)-based brain-computer interface (BCI) system has been widely explored in the past years due to its many advantages, such as portability, low cost and high temporal resolution [1]. Among various typical paradigms in the EEG, steady-state visual evoked potential (SSVEP) is the most employed one for analysing brain activities because it has a high signal-to-noise ratio (SNR) and fast communication rate [2]. Recent researches in many application scenarios such as character spelling and cleaning robot [3], [4] have also indicated the importance of SSVEP-based BCI technologies. ...
Conference Paper
Data-driven spatial filtering approaches have been widely used for steady-state visual evoked potentials (SSVEPs) detection toward the brain-computer interface (BCI). The existing methods tend to learn the spatial filter parameters for a certain stimulation frequency only using the training trials from the same stimulus, which may ignore the information from the other stimuli. In this paper, we propose a novel multi-objective optimisation-based spatial filtering method for enhancing SSVEP recognition. Spatial filters are defined via maximising the correlation among the training data from the same stimulus whilst minimising the correlation from different stimuli. We collected SSVEP signals using 16 electrodes from six healthy subjects at 4 different stimulation frequencies: 14Hz, 15Hz, 16Hz, and 17Hz. The experimental study was implemented, and our method can achieve an average recognition accuracy of 94.17%, which illustrates its effectiveness.
... CCA has been widely applied for frequency detection in multichannel visual-based BCIs (Lin et al., 2007;Zhang et al., 2020) due to its high efficiency, high robustness, high signalto-noise ratio, and simple implementation (Bin et al., 2009;Kalunga et al., 2013;Nakanishi et al., 2015). Therefore, CCA was implemented to detect frequency components in our research. ...
Article
Full-text available
The refresh rate is one of the important parameters of visual presentation devices, and assessing the effect of the refresh rate of a device on motion perception has always been an important direction in the field of visual research. This study examined the effect of the refresh rate of a device on the motion perception response at different stimulation frequencies and provided an objective visual electrophysiological assessment method for the correct selection of display parameters in a visual perception experiment. In this study, a flicker-free steady-state motion visual stimulation with continuous scanning frequency and different forms (sinusoidal or triangular) was presented on a low-latency LCD monitor at different refresh rates. Seventeen participants were asked to observe the visual stimulation without head movement or eye movement, and the effect of the refresh rate was assessed by analyzing the changes in the intensity of their visual evoked potentials. The results demonstrated that an increased refresh rate significantly improved the intensity of motion visual evoked potentials at stimulation frequency ranges of 7–28 Hz, and there was a significant interaction between the refresh rate and motion frequency. Furthermore, the increased refresh rate also had the potential to enhance the ability to perceive similar motion. Therefore, we recommended using a refresh rate of at least 120 Hz in motion visual perception experiments to ensure a better stimulation effect. If the motion frequency or velocity is high, a refresh rate of≥240 Hz is also recommended.
... BCI systems which use P300 components often use visual tasks. However, P300-based BCI have been implemented using auditory or tactile stimuli as well [2,3]. Another prevalent method used to extract specific neural activity patterns from EEGs is motor imagery (MI), in which particular patterns are elicited in the motor cortex during the imagination of movements [4]. ...
Chapter
Full-text available
A motor imagery-based brain-computer interface (MI-BCI) creates a path through which the brain interacts with the external environment by recording and processing electroencephalograph (EEG) signals made by imagining the movement of a particular limb. In this study, with the aim of improving classification accuracy, we modified the feature extraction stage. After preprocessing and decomposing EEG signals into their frequency bands, we applied a common spatial pattern (CSP) algorithm to each subband. Then, using spatial filters and blind source separation (BSS) techniques, we identified features of brain sources rather than just channels. Finally, by selecting the appropriate features as the input of the classification block, we discerned the imagined classes. To evaluate our proposed method, we used the dataset IVa of BCI competition III, which includes two classes: right hand and foot. The results of various experiments explain that the accuracy of the system is on average 98.8% and the sensitivity is 100% for all subjects. This indicates that the proposed method can be used to improve the MI-based BCI.
... Canonical correlation analysis is the most popular and widely used method for generating the spatial filters, which finds a linear transformation that maximizes the correlation between the recorded signal and a template signal, e.g., sine-cosine signals or averaged EEG template signals [28,29]. Typically, only the first canonical correlation and corresponding weights (w) are used for the classification and construction of filters [11]. ...
Article
This paper investigates the effects of the repetitive block-wise training process on the classification accuracy for a code-modulated visual evoked potentials (cVEP)-based brain-computer interface (BCI). The cVEP-based BCIs are popular thanks to their autocorrelation feature. The cVEP-based stimuli are generated by a specific code pattern, usually the m-sequence, which is phase-shifted between the individual targets. Typically, the cVEP classification requires a subject-specific template (individually created from the user's own pre-recorded EEG responses to the same stimulus target), which is compared to the incoming electroencephalography (EEG) data, using the correlation algorithms. The amount of the collected user training data determines the accuracy of the system. In this offline study, previously recorded EEG data collected during an online experiment with 10 participants from multiple sessions were used. A template matching target identification, with similar models as the task-related component analysis (TRCA), was used for target classification. The spatial filter was generated by the canonical correlation analysis (CCA). When comparing the training models from one session with the same session's data (intra-session) and the model from one session with the data from the other session (inter-session), the accuracies were (94.84%, 94.53%) and (76.67%, 77.34%) for intra-sessions and inter-sessions, respectively. In order to investigate the most reliable configuration for accurate classification, the training data blocks from different sessions (days) were compared interchangeably. In the best training set composition, the participants achieved an average accuracy of 82.66% for models based only on two training blocks from two different sessions. Similarly, at least five blocks were necessary for the average accuracy to exceed 90%. The presented method can further improve cVEP-based BCI performance by reusing previously recorded training data.
... B RAIN-COMPUTER Interfaces (BCIs) are an emerging technology able to create a direct communication path between the human brain and external devices, without the use of peripheral nerves and muscles [1]- [4]. Among the major BCI paradigms, Steady-State Visually Evoked Potential (SSVEP) has rapidly gained interest for developing applications in several fields, such as rehabilitation [5], [6], gaming [7], entertainment [8], industrial inspection [9], [10], and health monitoring [11], since it is characterized by easier detection and higher Information Transfer Rates (ITRs) with respect to other available BCIs [12], [13]. ...
... One of the most suitable application scenarios for EEG compression is the steady-state visual evoked potential (SSVEP) experiment [5][6][7][8][9][10]. The BCI speller based on SSVEP has many advantages in that it does not require user training, and its detection algorithm can be trained without relying on a large amount of training data. ...
Article
Full-text available
As an informative electroencephalogram (EEG) signal, steady-state visual evoked potential (SSVEP) stands out from many paradigms for application in wireless wearable devices. However, its data are usually enormous, occupy too many bandwidth sources and require immense power when transmitted in the raw data form, so it is necessary to compress the signal. This paper proposes a personalized EEG compression and reconstruction algorithm for the SSVEP application. In the algorithm, to realize personalization, a primary artificial neural network (ANN) model is first pre-trained with the open benchmark database towards BCI application (BETA). Then, an adaptive ANN model is generated with incremental learning for each subject to compress their individual data. Additionally, a personalized, non-uniform quantization method is proposed to reduce the errors caused by compression. The recognition accuracy only decreases by 3.79% when the compression rate is 12.7 times, and is tested on BETA. The proposed algorithm can reduce signal loss by from 50.43% to 81.08% in the accuracy test compared to the case without ANN and uniform quantization.
... The PSD method uses the power distribution of electrical brain signals in the frequency domain to apply the results to commands or decision-making. It looks at changes in density in different regions of the brain, focusing on how the stimulus area changes [39,40]. This study focused on the stimulation of regions in the occipital cortex, activated by visual perception, to determine the relationship between the target stimuli and brain signal modulation. ...
Article
Full-text available
Steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems suffer from low SSVEP response intensity and visual fatigue, resulting in lower accuracy when operating the system for continuous commands, such as an electric wheelchair control. This study proposes two SSVEP improvements to create a practical BCI for communication and control in disabled people. The first is flicker pattern modification for increasing SSVEP response through mixing (1) fundamental and first harmonic frequencies, and (2) two fundamental frequencies for an additional number of commands. The second method utilizes a quick response (QR) code for visual stimulus patterns to increase the SSVEP response and reduce visual fatigue. Eight different stimulus patterns from three flickering frequencies (7, 13, and 17 Hz) were presented to twelve participants for the test and score levels of visual fatigue. Two popular SSVEP methods, i.e., power spectral density (PSD) with Welch periodogram and canonical correlation analysis (CCA) with overlapping sliding window, are used to detect SSVEP intensity and response, compared to the checkerboard pattern. The results suggest that the QR code patterns can yield higher accuracy than checkerboard patterns for both PSD and CCA methods. Moreover, a QR code pattern with low frequency can reduce visual fatigue; however, visual fatigue can be easily affected by high flickering frequency. The findings can be used in the future to implement a real-time, SSVEP-based BCI for verifying user and system performance in actual environments.
Conference Paper
Canonical correlation analysis (CCA) is one of the most used algorithms in the steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI) systems due to its simplicity, efficiency, and robustness. Researchers have proposed modifications to CCA to improve its speed, allowing high-speed spelling and thus a more natural communication. In this work, we combine two approaches, the filter-bank (FB) approach to extract more information from the harmonics, and a range of different supervised methods which optimize the reference signals to improve the SSVEP detection. The proposed models are tested on the publicly available benchmark dataset for SSVEP-based BCIs and the results show improved performance compared to the state-of-the-art methods and, in particular, the proposed FBMwayCCA approach achieves the best results with an information transfer rate (ITR) of 134.8±8.4 bits/minute. This study indeed suggests the feasibility of combining the fundamental and harmonic SSVEP components with supervised methods in target identification to develop high-speed BCI spellers.
Chapter
Full-text available
Motor imagery (MI)-based electroencephalogram (EEG) signals classification has been utilized for developing assistive human–robotic interaction in the past few decades. EEG signals have excessive importance in the field of brain–computer interface (BCI) which has different applications in the field of bio-medical. BCI acquires brain signals, extracts features, reduced noise and produces control signal. The objective of this research work is compared to the effectiveness of time-domain (TD) features using two different classifiers, namely linear discriminant analysis (LDA) and support vector machine (SVM), to discriminate between right- and left-hand movements of EEG signals. In this context, the ocular artifact, i.e., electrooculogram (EOG) signal, was rejected by using the independent component analysis (ICA) approach, whereas dimension reduction was done by principal component analysis (PCA). The EEG dataset was acquired from ten healthy human subjects in two sessions followed by band-pass Butterworth filtering for de-noising. In this work, the 12-TD features were compared to each other in terms of classification accuracy with SVM and LDA classifiers. Finally, the top ten features were utilized to make the final feature vector which exhibited the best performance with an SVM classifier with an accuracy of 98.75% as compared to LDA (95.2%). The finding of this study would be utilized for designing the EEG-based wheelchair as well as a prosthetic limb.
Chapter
Brain–computer interfaces, also known as BCIs, are systems that analyze brain signals and translate them into meaningful information in real-time. These interfaces provide an alternative communication channel to manipulate a computer or a robotic device by modulating the neural activity. This chapter describes a general-purpose P300-based BCI developed for selecting different options presented on a computer screen. This system can be used to send instructions to a robot or any other device that accepts high-level commands representing actions or complex sequences of operations. This chapter includes a detailed description of the processing and classification stages of this BCI, and presents the results obtained from an online evaluation of this interface with 15 healthy participants. The classification accuracy observed in the system evaluation (93.89%) illustrates how computational intelligence and signal processing techniques allow the development of real-time tools for translating brain activity.
Chapter
Full-text available
Neuroimaging techniques—for example, electroencephalographic (EEG), optical (fNIRS, near infrared spectroscopy), and magnetic resonance imaging (MRI)—make it possible to study brain function and structure, and have made an essential contribution to the diagnosis and treatment of neurological and cognitive problems. The EEG detects electrical activity in the brain using electrodes attached to the scalp. From the electrical signals of the EEG, it is possible to construct maps of the oscillatory brain activity, according to its frequency, for the assessment of brain function during information processing or at rest. fNIRS is an optical neuroimaging technique that measures brain's metabolic response. From the fNIRS signal, it is possible to build brain function maps by measuring the cerebral cortex's optical changes. Magnetic resonance imaging is another neuroimaging technique that uses a magnetic field and radio waves to generate images of the brain's function and structure without using radioactive tracers. In this chapter, these three techniques are discussed. In the first section of this chapter, the electrical signals produced by the brain are reviewed. These signals are involved in many cognitive and behavioral processes, but will only be reviewed when subjects are at rest. The second section of this chapter introduces the fNIRS technique, including the physical and physiological principles underlying optical imaging and the technical considerations for conducting an fNIRS experiment. The last section corresponds to the generation of conventional and functional magnetic resonance imaging (MRI), starting from the basic physical principles and ending with its immediate clinical application.
Article
Due to the high robustness to artifacts, steady-state visual evoked potential (SSVEP) has been widely applied to construct high-speed brain-computer interfaces (BCIs). Thus far, many spatial filtering methods have been proposed to enhance the target identification performance for SSVEP-based BCIs, and task-related component analysis (TRCA) is among the most effective ones. In this paper, we further extend TRCA and propose a new method called Latency Aligning TRCA (LA-TRCA), which aligns visual latencies on channels to obtain accurate phase information from task-related signals. Based on the SSVEP wave propagation theory, SSVEP spreads from posterior occipital areas over the cortex with a fixed phase velocity. Via estimation of the phase velocity using phase shifts of channels, the visual latencies on different channels can be determined for inter-channel alignment. TRCA is then applied to aligned data epochs for target recognition. For the validation purpose, the classification performance comparison between the proposed LA-TRCA and TRCA-based expansions were performed on two different SSVEP datasets. The experimental results illustrated that the proposed LA-TRCA method outperformed the other TRCA-based expansions, which thus demonstrated the effectiveness of the proposed approach for enhancing the SSVEP detection performance.
Article
Objective: A novel flexible hydrogel electrode with a strong moisturizing ability was prepared for long-term electroencephalography (EEG) monitoring. Approach: The hydrogel was synthesized by polymerizing the N-acryloyl glycinamide (NAGA) monomer. And a proper amount of glycerin was added to the hydrogel to increase the moisture retention ability of the electrodes. The hydrogel shows high mechanical properties, and the liquid in the hydrogel produces a hydrating effect on the skin stratum corneum, which could decrease the contact impedance between skin and electrode. In addition, the installation of hydrogel electrode is very convenient, and the skin of the subject does not need to be abraded. Main results: SEM images show that there are a large number of micropores in the hydrogel, which provide storage space for water molecules. The average potential drift of the hydrogel electrode is relatively low (1.974 ± 0.560 µV min-1). The average contact impedance of hydrogel electrode in forehead region and hair region are 6.43 ± 0.84 kΩ·cm2 and 13.15 ± 3.72 kΩ·cm2, respectively. The result of open/closed paradigm, steady-state visual evoked potentials (SSVEP), and P300 visual evoked potential show that hydrogel electrode has excellent performance. Compared with the hydrogel without glycerol, the moisture retention ability of hydrogel with glycerol was greatly improved. Significance: Compared with standard Ag/AgCl wet electrode, hydrogel electrode is more convenient to install and has strong moisture retention ability, which makes it have great potential in daily life for long-term EEG recording.
Chapter
Brain-computer interface (BCI) has gained popularity for few decades in identifying brain disorders such as stress, apnea, seizure, and dizziness. Early detection of such disorders with proper medication may increase the patients’ quality of life. However, manual analysis of the recorded electroencephalogram (EEG) signals collected from the scalp is a complex task. Therefore, an automated tool for the EEG signals’ analysis and classification is helpful where a significant research contribution is being made in the literature. This chapter proposes a parallel combination of convolutional neural network (CNN) and long-short-term memory (LSTM) structure for EEG signal analysis and apnea classification. The performance of the proposed model is compared with other works and assessed in terms of performance parameters.
Chapter
Evaluation and interpretation of massive amounts of brain data are a big challenge for the design of functional brain-computer interface (BCI) devices. In this chapter, three power spectral methods: Welch, Burg, and multiple signal classification (MUSIC) are investigated to improve the pattern mining of two-class motor imagery electroencephalography (EEG) signals. Specifically, freely available dataset IVa from BCI competition III was used for evaluation. This dataset comprises a total of 118 electrodes, while 18 electrodes around the motor cortex region were included in our experiments. The proposed study is described in threefold. First, the multiscale principal component analysis (MSPCA) method was used to obtain clean EEG signals. Second, power spectral density (PSD) values were determined using the Welch, Burg, and MUSIC methods, and these PSD vectors were used as feature vectors. At last, all feature vectors were provided to logistic regression (LR), multilayer neural perceptron, and support vector machine classifiers by varying different parameters for classification tests. The results showed that the Welch PSD process gives a cumulative sensitivity, specificity, and accuracy of 99.7%, 100%, and 99.8%, respectively, which is better than Burg and MUSIC. In comparison with other methods on dataset IVa, the proposed system obtains an increase of up to 26.3% in classification. The findings suggest that the combination of MSPCA, Welch method, and LR is very efficient that can be used to build a subject-specific BCI system for different applications such as games and movies, artificial intelligence, robotics, and rehabilitation.
Article
Objective: Electroencephalogram (EEG) is an objective reflection of the brain activities, which provides potential possibilities for brain state estimation based on EEG characteristics. However, how to mine the effective EEG characteristics is still a distressing problem in brain state monitoring. Approach: The phase-scrambled method was used to generate images with different noise levels. Images were encoded into a rapid serial visual presentation (RSVP) paradigm. N-back working memory method was employed to induce and assess fatigue state. The irregular-resampling auto-spectral analysis (IRASA) method was adopted to extract and parameterize (exponent and offset) the characteristics of EEG fractal components, which were analyzed in the four dimensions: fatigue, sustained attention, visual noise and experimental tasks. Results: The degree of fatigue and visual noise level had positive effects on exponent and offset in the prefrontal lobe, and the ability of sustained attention negatively affected exponent and offset. Compared with visual stimuli task, rest task induced even larger values of exponent and offset and statistically significant in the most cerebral cortex. In addition, the SSVEP amplitudes were negatively and positively affected by the degree of fatigue and noise levels, respectively. Significance: The conclusions of this study provide insights into the relationship between brain states and EEG characteristics. In addition, this study has the potential to provide objective methods for brain states monitoring and EEG modeling.
Article
Introduction Estimation of reaching movement commands for controlling human–machine interfaces (HMIs) based on electromyogram (EMG) signals, such as prostheses or robotic assistive devices, for the disabled is challenging due to the absence of healthy and synergistic muscle activations. Methods In this study, to improve the kinematic estimation of an EMG-based decoder during goal-directed reaching movements, steady-state visual evoked potentials (SSVEP) were considered to identify the motion target. The EMG signal of the shoulder and arm muscles was mapped to the elbow angle with a model selected by SSVEP signals during the reaching movement to a flashing target on display. Results The accuracy of decoding based on the combination of EMG and EEG was significantly different (RMSE = 3.7° and R2 = 92.13%) from the counterpart EMG-based decoder (RMSE = 4.26° and R2 = 91.02%). Indeed, the proposed dual-modality structure outperformed the single-modality structure when the EMG signals altered in permanent conditions, such as spinal cord injury or stroke. Especially in simulated muscle weakness situations where the signal-to-noise ratio declines, the dual-modality framework was more robust than the single-modality, and the determinant coefficient of the decoder was stable above 80%. Conclusion Because of the additional EEG information to recognize the target of reaching, the performance of the EMG-based decoding improved and could be more robust during EMG signal alterations. The outcomes of this study can be applied to control HMIs or evaluate real-time rehabilitation trials, particularly for severe motor injury patients.
Article
In this study, inconspicuous visual stimuli with hidden targets are considered as a new paradigm for reactive brain-computer interface (BCI) applications suitable for actuating a motorized system with a dynamic command from user. To drive the motor’s control signal level as close as possible to the targeted dynamic command via wireless transmission, an embeddable intelligent control scheme is introduced to improve the overall electroencephalography (EEG)-based decoding strategy of the BCI system. The proposed technique which can induce motivated attention only requires a single EEG channel, and the intelligent control scheme is constructed with a decoder consisting of a multilayer perceptron (MLP) neural network and a recursive digital Boxcar filter to suppress the influence of noise and ocular artifacts that are typically caused by user’s involuntary movements. Results from thirty subjects showed that the predictive ability of the BCI system was significantly improved via the proposed decoder compared to the performance of MLP models alone and those with low pass and Kalman filters which were two existing methods commonly used to alleviate the aforementioned perturbations in real-time applications. The BCI’s predictive ability could also be further enhanced by selecting a suitable stimulus to construct a generic MLP model for each gender due to notable performance disparities between male and female groups. The findings of this study will have the potential to increase the degree of freedom in reactive BCI applications particularly when embedded control systems with multiple actuator speeds or accelerations are desired such as those used to control mobile robotics.
Article
Full-text available
In this study, we proposed a new type of hybrid visual stimuli for steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs), which incorporate various periodic motions into conventional flickering stimuli (FS) or pattern reversal stimuli (PRS). Furthermore, we investigated optimal periodic motions for each FS and PRS to enhance the performance of SSVEP-based BCIs. Periodic motions were implemented by changing the size of the stimulus according to four different temporal functions denoted by none, square, triangular, and sine, yielding a total of eight hybrid visual stimuli. Additionally, we developed the extended version of filter bank canonical correlation analysis (FBCCA), which is a state-of-the-art training-free classification algorithm for SSVEP-based BCIs, to enhance the classification accuracy for PRS-based hybrid visual stimuli. Twenty healthy individuals participated in the SSVEP-based BCI experiment to discriminate four visual stimuli with different frequencies. An average classification accuracy and information transfer rate (ITR) were evaluated to compare the performances of SSVEP-based BCIs for different hybrid visual stimuli. Additionally, the user's visual fatigue for each of the hybrid visual stimuli was also evaluated. As the result, for FS, the highest performances were reported when the periodic motion of the sine waveform was incorporated for all window sizes except for 3 s. For PRS, the periodic motion of the square waveform showed the highest classification accuracies for all tested window sizes. A significant statistical difference in the performance between the two best stimuli was not observed. The averaged fatigue scores were reported to be 5.3 ± 2.05 and 4.05 ± 1.28 for FS with sine-wave periodic motion and PRS with square-wave periodic motion, respectively. Consequently, our results demonstrated that FS with sine-wave periodic motion and PRS with square-wave periodic motion could effectively improve the BCI performances compared to conventional FS and PRS. In addition, thanks to its low visual fatigue, PRS with square-wave periodic motion can be regarded as the most appropriate visual stimulus for the long-term use of SSVEP-based BCIs, particularly for window sizes equal to or larger than 2 s.
Article
Full-text available
Among the Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs), the phase-tagged SSVEP (p-SSVEP) has been proved a reliable paradigm to extend the number of available targets, especially for high-frequency SSVEP-based BCIs. However, the recognition efficiency of the high-frequency p-SSVEP still remains relatively low. A longer data segment may achieve a higher classification accuracy, but the time consumption of computation leads to the decrease of information transfer rate. This paper presents a recursive Bayesian-based approach to improve the high-frequency p-SSVEP classification efficiency. In each signal processing period, the classification result is generated by the current scores, the condition probability and a recursive prior probability (dynamic prior probability). The experiment displays the SSVEP stimuli with 20 Hz and 30 Hz respectively, and each frequency contains six phases. This paper compared three classification approaches and the recursive Bayesian-based approach could obtain the highest classification accuracy and practical bit rate under the same data length. The mean accuracy and practical bit rate were 89.7% and 37.8 bits/min with 20Hz, and 89.0% and 36.5 bits/min with 30Hz, respectively Furthermore, the recursive Bayesian-based approach could reduce the individual differences among different subjects. Therefore, the recursive Bayesian-based approach can lead to high classification efficiency in high-frequency p-SSVEP.
Article
Full-text available
The assistive, adaptive, and rehabilitative applications of EEG-based robot control and navigation are undergoing a major transformation in dimension as well as scope. Under the background of artificial intelligence, medical and nonmedical robots have rapidly developed and have gradually been applied to enhance the quality of people’s lives. We focus on connecting the brain with a mobile home robot by translating brain signals to computer commands to build a brain-computer interface that may offer the promise of greatly enhancing the quality of life of disabled and able-bodied people by considerably improving their autonomy, mobility, and abilities. Several types of robots have been controlled using BCI systems to complete real-time simple and/or complicated tasks with high performances. In this paper, a new EEG-based intelligent teleoperation system was designed for a mobile wall-crawling cleaning robot. This robot uses crawler type instead of the traditional wheel type to be used for window or floor cleaning. For EEG-based system controlling the robot position to climb the wall and complete the tasks of cleaning, we extracted steady state visually evoked potential (SSVEP) from the collected electroencephalography (EEG) signal. The visual stimulation interface in the proposed SSVEP-based BCI was composed of four flicker pieces with different frequencies (e.g., 6 Hz, 7.5 Hz, 8.57 Hz, and 10 Hz). Seven subjects were able to smoothly control the movement directions of the cleaning robot by looking at the corresponding flicker using their brain activity. To solve the multiclass problem, thereby achieving the purpose of cleaning the wall within a short period, the canonical correlation analysis (CCA) classification algorithm had been used. Offline and online experiments were held to analyze/classify EEG signals and use them as real-time commands. The proposed system was efficient in the classification and control phases with an obtained accuracy of 89.92% and had an efficient response speed and timing with a bit rate of 22.23 bits/min. These results suggested that the proposed EEG-based clean robot system is promising for smart home control in terms of completing the tasks of cleaning the walls with efficiency, safety, and robustness. 1. Introduction The idea of interfacing brains with machines/robots has long captured the human imagination. Brain-computer interface (BCI) technology intend to build an interface between the brain and any electrical/electronic device (e.g., a wheelchair, smart home appliances, and robotic devices) using electroencephalogram (EEG) which is a noninvasive technique for measuring electrical potentials from electrodes placed on the scalp produced by brain activity. Nowadays, the EEG technique has been used to establish portable synchronous and asynchronous controls for BCI applications. Noninvasive EEG-based BCIs are the most promising interface for space of applications for people with severe motor disabilities because of their noninvasiveness, low cost, practicality, portability, and being easy to use. For some disabled patients with physical disability or paralysis while the brain function is still normal, although they have a normal large brain consciousness and thought, they cannot communicate with the external environment through the severely damaged muscle and nervous system and complete the daily work independently. This has caused serious physical and mental trauma, and their lives are very painful, which will affect their recovery process to some extent. How to restore or enhance their control and communication capabilities to the outside world has been the goal that has been pursued for many years in the field of medical rehabilitation. Therefore, BCIs can be used for helping patients with severe brain disorders or muscle damages to regain their ability to communicate directly with the outside environment through the brain electrophysiology response [1–3]. BCI can also be beneficial for the elderly as advanced assistive and rehabilitative technologies and useful for young able-bodied for controlling video games for entertainment [4, 5] or controlling a robotic arm for several purposes [6–9]. However, most of the traditional brain-computer interface equipment is expensive, bulky, and tedious, which makes it difficult to popularize and apply brain-computer interface technology in real life. The portable brain-computer interface has become one of the hotspots in the field of the brain-computer interface because of its advantages of easy to carry, easy to use, safe, and reliable. BCI technology is mainly divided into two types of brain activity measurement, invasive BCI, and noninvasive BCI, depending on the way of putting the electrodes to record the electrical brain activity [10–14]. Among them, the invasive BCI might lead to an immune reaction, which causes serious harm to the user, and it is hardly accepted by disabled people because of the invasiveness of the technique which requires a dedicated surgery, and its cost with equipment is very expensive and not covered by many governments yet. Although the noninvasive brain-computer interface is less accurate than the invasive BCI, it is still relatively cheaper compared with all other techniques and everyone can easily accept it. There are several paradigms to control machine or computer using our brain signal characteristics and the most popular ones are motor imagery [15, 16], P300 wave [17, 18], steady state visual evoked potentials (SSVEP) [19–21] for building practical brain-computer interface systems. So far, the SSVEP method was applied widely because of the high signal-to-noise ratio and robustness [22]. SSVEP induction means that when the human brain receives the stimulation of a fixed frequency scintillation block, an uninterrupted response related to the stimulation frequency will be generated in the visual cortex of the human brain. This SSVEP brain response is a very useful natural involuntary phenomenon which has been tested by researchers many times. The earliest SSVEP-BCI system, designed by Regan , in 1979, allowed subjects to select a flashing button on the computer screen by simply looking at the computer screen [23], basically achieving the desired design goals. Then, Mullerputz and Guneysu and Akin applied the SSVEP-BCI system to the physical control of neural limb and humanoid robot, respectively, and achieved good control results [24]. In this paper, we chose SSVEP because it does not need any training phase for subjects and has very high accuracy compared with P300 or motor imagery using single trial electroencephalography (EEG) signal. The commonly used signal processing and classification methods of SSVEP include fast Fourier transform (FFT), wavelet transform, canonical correlation analysis (CCA), linear discriminant analysis (LDA), and support vector machine (SVM). In this paper, CCA was used for developing our signal processing algorithm. Compared with other SSVEP signal classification algorithms [10–14, 25], CCA classification algorithm is fast, efficient, simple, and easy to use. In some previous researches, the SSVEP paradigm was successfully used in writing tasks [26]. In the paper [27], we can see that the authors proposed a hybrid brain-computer interface system that combines P300 and SSVEP modalities. This combined system has improved the accuracy of EEG-based wheelchair control. In addition, SSVEP has been also used in the mental spelling system [28, 29]. In the paper [30], the authors used three flash speeds to control the small robot car. Lee et al. only use OZ as the reference electrode to collect and process EEG signals. In the paper [31], Lu and Bi have proposed a longitudinal control system for brain-controlled vehicles based on EEG signals. However, it is still unknown whether it can be used in the industry. In this paper, a new type of intelligent crawler robot is designed for cleaning the walls, which is considered as one of the smart home appliances. Compared with the wheeled robot [32], the crawler robot has the advantages of long life and high carrying capacity. The intelligent crawling robot for the walls used in this experiment adds an adsorption device using negative vacuum pressure, which effectively solves the problem of sliding of the cleaning robot on a wall with a certain inclination angle. The BCI based on SSVEP can usually provide a high information transmission rate, the verification process of the system is relatively simple, and no training of the subjects is required. However, because the SSVEP of some subjects is very weak and vulnerable to the interference of other noise signals, how to accurately identify SSVEP from a short time window is still a challenging problem in BCI research based on SSVEP. This is also the subject that we will continue to study in the future. In this study, the SSVEP paradigm was designed to control the crawler robot for cleaning the dust on the walls. We used the high accuracy SSVEP paradigm to cooperate with our cleaning robot to complete the designed experiment. To our best knowledge, this is the first report, which used brain machine interface for crawling cleaning robot control to help persons with disabilities to improve their quality of life. This paper is arranged as follows: in the Materials and Methods section, the experimental paradigm and analysis method of brain signal and the motion model of the intelligent crawling robot were introduced. At the same time, the offline experiment and online experiment are completed, and the data analysis is carried out. In the Results section, the offline and online experiments were summarized and discussed separately, and the accuracy and ITR of the experiment were obtained. Our experiments validate our views and achieve the desired results. In the Discussion part, we mainly talk about the limitations of the system and put forward the future changes. Finally, conclusion and prospects of future work are given in Section 5. 2. Materials and Methods 2.1. Participants and Experimental Paradigm Seven healthy volunteers (4 males and 3 females, 23–27 years of age) were invited to join the experiment for performing some robot control tasks using their brain activity. None of the subjects have prior experience on brain-computer interfaces. Clear written informed consent was obtained from all the participants, who were informed in detail about the purpose and possible consequences of the experiment. The experimental protocol was carried out in accordance with the latest version of the Declaration of Helsinki. The experiments were carried out in a quiet and comfortable environment to reduce the noise effect on our EEG recording. Subjects sat on a chair which is 60 cm away from the screen which contains the stimulation interface. In order to ensure the accuracy of the experiment, participants were required to avoid gnashing during the experiment. Because the SSVEP paradigm was easy to cause fatigue, the subjects can take a rest after one session. The flow of the experiment is shown in Figure 1. The experimental process is mainly divided into three parts. Firstly, the EEG acquisition device should be worn correctly for the subject and the subject’s position should be adjusted. Secondly, the collected data are processed and classified by a signal processing computer. Finally, the processed instructions are sent to the lower computer, that is, the intelligent crawling robot.
Article
Full-text available
This study illustrates and evaluates a novel subject-specific target detection framework, sum of squared correlations (SSCOR), for improving the performance in steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs). The SSCOR spatial filter learns a common SSVEP representation space through the optimization of the individual SSVEP templates. The projection onto this SSVEP response subspace improves the signal to noise ratio (SNR) of the SSVEP components embedded in the recorded electroencephalographic (EEG) data. To demonstrate the effectiveness of the proposed framework, the target detection performance of the SSCOR method is compared with the state of the art task-related component analysis (TRCA). The evaluation is conducted on a 40 target SSVEP benchmark data collected from 35 subjects. The results of the extensive comparisons of the performance metrics show that the proposed SSCOR method outperforms the TRCA method. The ensemble version of the SSCOR framework provides an offline simulated information transfer rate (ITR) of 387±9 bits/min which is much higher than that of the ensemble TRCA approach (max. ITR 216±27 bits/min). The significant improvement in the detection accuracy and simulated ITR demonstrates the efficacy of the proposed framework for target detection in SSVEP based BCI applications.
Article
Full-text available
Rhythmic visual stimuli (flicker) elicit rhythmic brain responses at the frequency of the stimulus, and attention generally enhances these oscillatory brain responses (steady state visual evoked potentials, SSVEPs). Although SSVEP responses have been tested for flicker frequencies up to 100 Hz [Herrmann, 2001], effects of attention on SSVEP amplitude have only been reported for lower frequencies (up to ~30 Hz), with no systematic comparison across a wide, finely sampled frequency range. Does attention modulate SSVEP amplitude at higher flicker frequencies (gamma band, 30-80 Hz), and is attentional modulation constant across frequencies? By isolating SSVEP responses from the broadband EEG signal using a multivariate spatiotemporal source separation method, we demonstrate that flicker in the alpha and gamma bands elicit strongest and maximally phase stable brain responses (resonance), on which the effect of attention is opposite: positive for gamma and negative for alpha. Finding subject-specific gamma resonance frequency and a positive attentional modulation of gamma-band SSVEPs points to the untapped potential of flicker as a non-invasive tool for studying the causal effects of interactions between visual gamma-band rhythmic stimuli and endogenous gamma oscillations on perception and attention.
Article
Full-text available
A brain-computer interface (BCI) provides a novel non-muscular communication pathway for individuals with severe neuromuscular diseases. BCI systems based on steady-state visual evoked potentials (SSVEPs) have high classification accuracy, information transfer rate, and signal-to-noise ratio, giving them high research and application value. However, SSVEP-based BCI has several limitations in real-world applications. The main challenge is how to reduce or eliminate the need for a dedicated training process while maintaining high classification accuracy. Filter bank canonical correlation analysis (FBCCA) is a powerful and widely used feature extraction method for SSVEP-based BCI systems. However, the reference signals of FBCCA are fixed-frequency sine-cosine waves, which makes it difficult to accurately describe the complex, mutative, and individually different physiological SSVEPs. Therefore, there is huge room for improvement in classification performance based on the FBCCA method. In contrast, although spatiotemporal beamforming (BF) detects SSVEPs with high accuracy, it needs an additional training process, which limits its application. In this study, we propose a bimodal decoding algorithm (FBCCA+BF), which combines the advantages of the training-free classification of FBCCA and the data-driven and adaptive features of BF. Six-channel SSVEP data corresponding to eight targets measured from 15 subjects were used to test the effectiveness of three different CCA-based methods, BF, and our proposed FBCCA+BF methods. It was found that the classification accuracies for BF and FBCCA+BF are 95.6% and 92.2%, respectively, which are significantly higher than the other CCA-based methods. Notably, both BF and FBCCA+BF obtain state-of-the-art performance, but FBCCA+BF does this without the need for a dedicated training process. Therefore, we conclude that our proposed FBCCA+BF method provides a training-free and high-accuracy approach for SSVEP-based BCIs.
Article
Full-text available
The process of recording Electroencephalography (EEG) signals is onerous and requires massive storage to store signals at an applicable frequency rate. In this work, we propose the Event-Related Potential Encoder Network (ERPENet); a multi-task autoencoder-based model, that can be applied to any ERP-related tasks. The strength of ERPENet lies in its capability to handle various kinds of ERP datasets and its robustness across multiple recording setups, enabling joint training across datasets. ERPENet incorporates Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM), in an autoencoder setup, which tries to simultaneously compress the input EEG signal and extract related P300 features into a latent vector. Here, we can infer the process for generating the latent vector as universal joint feature extraction. The network also includes a classification part for attended and unattended events classification as an auxiliary task. We experimented on six different P300 datasets. The results show that the latent vector exhibits better compression capability than the previous state-of-the-art semi-supervised autoencoder model. For attended and unattended events classification, pre-trained weights are adopted as initial weights and tested on unseen P300 datasets to evaluate the adaptability of the model, which shortens the training process as compared to using random Xavier weight initialization. At the compression rate of 6.84, the classification accuracy outperforms conventional P300 classification models: XdawnLDA, DeepConvNet, and EEGNet achieving 79.37% -88.52% classification accuracy depending on the dataset.
Article
Full-text available
Cognitive deterioration caused by illness or aging often occurs before symptoms arise, and its timely diagnosis is crucial to reducing its medical, personal, and societal impacts. Brain–computer interfaces (BCIs) stimulate and analyze key cerebral rhythms, enabling reliable cognitive assessment that can accelerate diagnosis. The BCI system presented analyzes steady-state visually evoked potentials (SSVEPs) elicited in subjects of varying age to detect cognitive aging, predict its magnitude, and identify its relationship with SSVEP features (band power and frequency detection accuracy), which were hypothesized to indicate cognitive decline due to aging. The BCI system was tested with subjects of varying age to assess its ability to detect aging-induced cognitive deterioration. Rectangular stimuli flickering at theta, alpha, and beta frequencies were presented to subjects, and frontal and occipital Electroencephalographic (EEG) responses were recorded. These were processed to calculate detection accuracy for each subject and calculate SSVEP band power. A neural network was trained using the features to predict cognitive age. The results showed potential cognitive deterioration through age-related variations in SSVEP features. Frequency detection accuracy declined after age group 20–40, and band power declined throughout all age groups. SSVEPs generated at theta and alpha frequencies, especially 7.5 Hz, were the best indicators of cognitive deterioration. Here, frequency detection accuracy consistently declined after age group 20–40 from an average of 96.64% to 69.23%. The presented system can be used as an effective diagnosis tool for age-related cognitive decline.
Article
Full-text available
Neurorobotics is one of the most ambitious fields in robotics, driving integration of interdisciplinary data and knowledge. One of the most productive areas of interdisciplinary research in this area has been the implementation of biologically-inspired mechanisms in the development of autonomous systems. Specifically, enabling such systems to display adaptive behavior such as learning from good and bad outcomes, has been achieved by quantifying and understanding the neural mechanisms of the brain networks mediating adaptive behaviors in humans and animals. For example, associative learning from aversive or dangerous outcomes is crucial for an autonomous system, to avoid dangerous situations in the future. A body of neuroscience research has suggested that the neurocomputations in the human brain during associative learning involve re-shaping of sensory responses. The nature of these adaptive changes in sensory processing during learning however are not yet well enough understood to be readily implemented into on-board algorithms for robotics application. Toward this overall goal, we record the simultaneous electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), characterizing one candidate mechanism, i.e., large-scale brain oscillations. The present report examines the use of Functional Source Separation (FSS) as an optimization step in EEG-fMRI fusion that harnesses timing information to constrain the solutions that satisfy physiological assumptions. We applied this approach to the voxel-wise correlation of steady-state visual evoked potential (ssVEP) amplitude and blood oxygen level-dependent imaging (BOLD), across both time series. The results showed the benefit of FSS for the extraction of robust ssVEP signals during simultaneous EEG-fMRI recordings. Applied to data from a 3-phase aversive conditioning paradigm, the correlation maps across the three phases (habituation, acquisition, extinction) show converging results, notably major overlapping areas in both primary and extended visual cortical regions, including calcarine sulcus, lingual cortex, and cuneus. In addition, during the acquisition phase when aversive learning occurs, we observed additional correlations between ssVEP and BOLD in the anterior cingulate cortex (ACC) as well as the precuneus and superior temporal gyrus.
Article
Full-text available
Objective: This study introduces and evaluates a novel target identification method, latent common source extraction (LCSE), that uses subject-specific training data for the enhancement of detection of steady-state visual evoked potential (SSVEP). Approach: LCSE seeks to construct a common latent representation of the SSVEP signal subspace that is stable across multiple trials of electroencephalographic (EEG) data. The spatial filter thus obtained improves the signal-to-noise ratio (SNR) of the SSVEP components by removing nuisance signals that are irrelevant to the generalized signal representation learnt from the given data. In this study a comparison of SSVEP identification performance between the proposed method, extended canonical correlation analysis (ExtCCA) and multiset canonical correlation analysis (MsetCCA) was conducted using SSVEP benchmark data of 40 targets recorded from 35 subjects to validate the effectiveness of the LCSE framework. Main results: The results indicate that the LCSE framework significantly outperforms the other two methods in terms of both classification accuracy and information transfer rates (ITRs). Significance: The significant improvement in the target identification performance demonstrates that the proposed LCSE method can be seen as a promising potential candidate for efficient SSVEP detection in brain-computer interface (BCI) systems.
Article
Full-text available
This paper proposes a novel phase estimator based on fully-traversed Discrete Fourier Transform (DFT) which takes all possible truncated DFT spectra into account such that it possesses two merits of `direct phase extraction’ (namely accurate instantaneous phase information can be extracted without any correction) and suppressing spectral leakage. This paper also proves that the proposed phase estimator complies with the 2-parameter joint estimation model rather than the conventional 3-parameter joint model. Numerical results verify the above two merits and demonstrate that the proposed estimator can extract phase information from noisy multi-tone signals. Finally, real data analysis shows that fully-traversed DFT can achieve a better classification on the phase of steady-state visual evoked potential (SSVEP) brain-computer interface (BCI) than the conventional DFT estimator does. Besides, the proposed phase estimator imposes no restrictions on the relationship between the sampling rates and the stimulus frequencies, thus it is capable of wider applications in phase-coded SSVEP BCIs, when compared with the existing estimators.
Article
Full-text available
Generally, the more channels are used to acquire EEG signals, the better the performance of the brain–computer interface (BCI). However, from the user’s point of view, a BCI system comprising a large number of channels is not desirable because of the lower comfort and extended application time. Therefore, the current trend in BCI design is to use the smallest number of channels possible. The problem is, however, that usually when we decrease the number of channels, the interface accuracy also drops significantly. In the paper, we examined whether it is possible to maintain the high accuracy of a BCI based on steady-state visual evoked potentials (SSVEP-BCI) in a low-channel setup using a preprocessing procedure successfully used in a multichannel setting: independent component analysis (ICA). The influence of ICA on the BCI performance was measured in an off-line (24 subjects) mode and online (eight subjects) mode. In the off-line mode, we compared the number of correctly recognized different stimulation frequencies, and in the online mode, we compared the classification accuracy. In both experiments, we noted the predominance of signals that underwent ICA preprocessing. In the off-line mode, we detected 50% more stimulation frequencies after ICA preprocessing than before (in the case of four EEG channels), and in the online mode, we noted a classification accuracy increase of 8%. The most important results, however, were the results obtained for a very low luminance (350 lx), where we noted 71% gain in the off-line mode and 11% gain in the online mode.
Conference Paper
Full-text available
Among various brain activity patterns, Steady State Visual Evoked Potential (SSVEP) based Brain Computer Inter-face (BCI) requires the least training time while carrying the fastest information transfer rate, making it highly suitable for deploying efficient self-paced BCI systems. In this study, we propose a Spectrum and Phase Adaptive CCA (SPACCA) for subject-and device-specific SSVEP-based BCI. Cross subject heterogeneity of spectrum distribution is taken into consideration to improve the prediction accuracy. We design a library of phase shifting reference signals to accommodate subjective and device-related response time lag. With the flexible reference signal generating approach, the system can be optimized for any specific flickering source, include LED, computer screen and mobile devices. We evaluated the performance of SPACCA using three sets of data that use LED, computer screen and mobile device (tablet) as stimuli sources respectively. The first two data sets are publicly available whereas the third data set is collected in our BCI lab. Across different data sets, SPACCA consistently performs better than the baseline, i.e. standard CCA approach. Statistical test to compare the overall results across three data sets yield a p-value of 1.66e-6, implying the improvement is significant.
Chapter
Full-text available
One of the major limitations of brain-computer interface (BCI) is its long calibration time. Typically, a big amount of training data needs to be collected at the beginning of each session in order to tune the parameters of the system for the target user due to between sessions/subjects non-stationarity. To mitigate this limitation, transfer learning can be potentially one useful solution. Transfer learning extracts information from different domains (raw data, features, or classification domain) to compensate the lack of labelled data from the test subject. Within this chapter transfer learning definitions and techniques are fully explained.After that, some of the available transfer learning applications in BCI are explored. Then, a brief discussion about applying transfer learning in the different domains is included. The discussion shows that despite some advances, a successful transfer learning framework for BCI still needs to be developed. Finally, future research directions in this topic are suggested in order to successfully and reliably reduce the calibration time for new subjects and increase the accuracy of the system.
Article
Full-text available
The accurate discrimination between bipolar disorder (BD) and schizophrenic patients is crucial because of the considerable overlap between their clinical signs and symptoms (e.g., hallucination and delusion). Recently, electroencephalograms (EEGs) measured in the resting state have been vastly analyzed as a means for classifying BD and schizophrenic patients, but EEGs evoked by external audio/visual stimuli have been rarely investigated, despite their high signal-to-noise ratio (SNR). In this study, in order to investigate whether EEGs evoked by external stimuli can be used for classifying BD and schizophrenic patients, we used a visual stimulus modulated at a specific frequency to induce steady-state visual evoked potential (SSVEP). In the experiment, a photic stimulation modulated at 16 Hz was presented to two groups of schizophrenic and BD patients for 95 s, during which EEG data were recorded. Statistical measures of SSVEPs (mean, skewness, and kurtosis) described in SNR units were extracted as features to characterize and classify variations of brain activity patterns in the two groups. Two brain areas, O1 and Fz, showed a statistically significant difference between the two groups for SNR mean and kurtosis, respectively. Among five applied classifiers, k-nearest neighbor provided the highest classification accuracy of 91.30% with the best feature set selected by Fisher score. An acceptable accuracy for binary classification (> 70%) was retained until analysis time was reduced up to 10 s using a support vector machine classifier, and 20 s for other classifiers. Our results demonstrate the potential applicability of the proposed SSVEP-based classification approach with relatively short single-trial EEG signals. OAPA
Article
Full-text available
A Brain–Computer Interface (BCI) provides a novel non-muscular communication method via brain signals. A BCI-speller can be considered as one of the first published BCI applications and has opened the gate for many advances in the field. Although many BCI-spellers have been developed during the last few decades, to our knowledge, no reviews have described the different spellers proposed and studied in this vital field. The presented speller systems are categorized according to major BCI paradigms: P300, steady-state visual evoked potential (SSVEP), and motor imagery (MI). Different BCI paradigms require specific electroencephalogram (EEG) signal features and lead to the development of appropriate Graphical User Interfaces (GUIs). The purpose of this review is to consolidate the most successful BCI-spellers published since 2010, while mentioning some other older systems which were built explicitly for spelling purposes. We aim to assist researchers and concerned individuals in the field by illustrating the highlights of different spellers and presenting them in one review. It is almost impossible to carry out an objective comparison between different spellers, as each has its variables, parameters, and conditions. However, the gathered information and the provided taxonomy about different BCI-spellers can be helpful, as it could identify suitable systems for first-hand users, as well as opportunities of development and learning from previous studies for BCI researchers.
Article
Full-text available
People with severe neurological impairments face many challenges in sensorimotor functions and communication with the environment; therefore they have increased demand for advanced, adaptive and personalized rehabilitation. During the last several decades, numerous studies have developed brain–computer interfaces (BCIs) with the goals ranging from providing means of communication to functional rehabilitation. Here we review the research on non-invasive, electroencephalography (EEG)-based BCI systems for communication and rehabilitation. We focus on the approaches intended to help severely paralyzed and locked-in patients regain communication using three different BCI modalities: slow cortical potentials, sensorimotor rhythms and P300 potentials, as operational mechanisms. We also review BCI systems for restoration of motor function in patients with spinal cord injury and chronic stroke. We discuss the advantages and limitations of these approaches and the challenges that need to be addressed in the future.
Article
Full-text available
Brain-computer interface (BCI) paradigms are usually tested when environmental and biological artifacts are intentionally avoided. In this study, we deliberately introduced different perturbations in order to test the robustness of a steady state visual evoked potential (SSVEP) based BCI. Specifically we investigated to what extent a drop in performance is related to the degraded quality of EEG signals or rather due to increased cognitive load. In the online tasks, subjects focused on one of the four circles and gave feedback on the correctness of the classification under four conditions randomized across subjects: Control (no perturbation), Speaking (counting loudly and repeatedly from one to ten), Thinking (mentally counting repeatedly from one to ten), and Listening (listening to verbal counting from one to ten). Decision tree, Naïve Bayes and K-Nearest Neighbor classifiers were used to evaluate the classification performance using features generated by canonical correlation analysis. During the online condition, Speaking and Thinking decreased moderately the mean classification accuracy compared to Control condition whereas there was no significant difference between Listening and Control conditions across subjects. The performances were sensitive to the classification method and to the perturbation conditions. We have not observed significant artifacts in EEG during perturbations in the frequency range of interest except in theta band. Therefore we concluded that the drop in the performance is likely to have a cognitive origin. During the Listening condition relative alpha power in a broad area including central and temporal regions primarily over the left hemisphere correlated negatively with the performance thus most likely indicating active suppression of the distracting presentation of the playback. This is the first study that systematically evaluates the effects of natural artifacts (i.e. mental, verbal and audio perturbations) on SSVEP-based BCIs. The results can be used to improve individual classification performance taking into account effects of perturbations.
Article
Full-text available
Objective This study aims to establish a steady-state visual evoked potential- (SSVEP-) based passive training protocol on an ankle rehabilitation robot and validate its feasibility. Method This paper combines SSVEP signals and the virtual reality circumstance through constructing information transmission loops between brains and ankle robots. The robot can judge motion intentions of subjects and trigger the training when subjects pay their attention on one of the four flickering circles. The virtual reality training circumstance provides real-time visual feedback of ankle rotation. Result All five subjects succeeded in conducting ankle training based on the SSVEP-triggered training strategy following their motion intentions. The lowest success rate is 80%, and the highest one is 100%. The lowest information transfer rate (ITR) is 11.5 bits/min when the biggest one of the robots for this proposed training is set as 24 bits/min. Conclusion The proposed training strategy is feasible and promising to be combined with a robot for ankle rehabilitation. Future work will focus on adopting more advanced data process techniques to improve the reliability of intention detection and investigating how patients respond to such a training strategy.
Article
Full-text available
Multiset canonical correlation analysis (MsetCCA) has been successfully applied to optimize the reference signals by extracting common features from multiple sets of electroencephalogram (EEG) for steady-state visual evoked potential (SSVEP) recognition in brain-computer interface application. To avoid extracting the possible noise components as common features, this study proposes a sophisticated extension of MsetCCA, called multilayer correlation maximization (MCM) model for further improving SSVEP recognition accuracy. MCM combines advantages of both CCA and MsetCCA by carrying out three layers of correlation maximization processes. The first layer is to extract the stimulus frequency-related information in using CCA between EEG samples and sine-cosine reference signals. The second layer is to learn reference signals by extracting the common features with MsetCCA. The third layer is to re-optimize the reference signals set in using CCA with sine-cosine reference signals again. Experimental study is implemented to validate effectiveness of the proposed MCM model in comparison with the standard CCA and MsetCCA algorithms. Superior performance of MCM demonstrates its promising potential for the development of an improved SSVEP-based brain-computer interface.
Article
Full-text available
In this article, non-invasive hybrid brain–computer interface (hBCI) technologies for improving classification accuracy and increasing the number of commands are reviewed. Hybridization combining more than two modalities is a new trend in brain imaging and prosthesis control. Electroencephalography (EEG), due to its easy use and fast temporal resolution, is most widely utilized in combination with other brain/non-brain signal acquisition modalities, for instance, functional near infrared spectroscopy (fNIRS), electromyography (EMG), electrooculography (EOG), and eye tracker. Three main purposes of hybridization are to increase the number of control commands, improve classification accuracy and reduce the signal detection time. Currently, such combinations of EEG + fNIRS and EEG + EOG are most commonly employed. Four principal components (i.e., hardware, paradigm, classifiers, and features) relevant to accuracy improvement are discussed. In the case of brain signals, motor imagination/movement tasks are combined with cognitive tasks to increase active brain–computer interface (BCI) accuracy. Active and reactive tasks sometimes are combined: motor imagination with steady-state evoked visual potentials (SSVEP) and motor imagination with P300. In the case of reactive tasks, SSVEP is most widely combined with P300 to increase the number of commands. Passive BCIs, however, are rare. After discussing the hardware and strategies involved in the development of hBCI, the second part examines the approaches used to increase the number of control commands and to enhance classification accuracy. The future prospects and the extension of hBCI in real-time applications for daily life scenarios are provided.
Article
Maximum signal fraction analysis (MSFA) is an efficient method for frequency identification in steady-state visual evoked potential (SSVEP) based brain-computer interface (BCI). However, standard MSFA only utilizes the spatial filter corresponding to the maximum eigenvalue for multichannel EEG signals de-noising and frequency identification, and discards other weight vectors containing discriminant information. In this work, we proposed a new SSVEP frequency recognition method with spatial dimension fusion strategy, i.e., FoMSFA, which employs the information of all spatial filters from standard MSFA, and uses a nonlinear weighting function to fuse multiple sets of correlation coefficients to identify the frequency of SSVEP signal. Numerical results of the benchmark dataset with 35 subjects and our laboratory dataset with 10 subjects show that FoMSFA outperforms the standard MSFA and CCA-based methods. The proposed method has the potential to design high-performance SSVEP-based BCI systems for communication and control.
Article
Common Spatial Pattern (CSP) is the most popular method in motor imagery (MI) based Brain–Computer Interfaces (BCI) for extracting features from electroencephalogram (EEG) signals. Due to the non-stationary nature of EEG signals, the CSP computed on the training data may not be optimal for the evaluation data. One of the major causes of such non-stationarity is the change in user's cognitive state due to fatigue, frustration, low arousal level, etc. This paper proposes an adaptive scheme for the CSP based on the mental fatigue of the user. The proposed method uses Linear Discriminant Analysis (LDA) active learning to adapt the CSP. Breaking ties criterion is used for selecting samples from the evaluation data. The separability of MI EEG features extracted with the proposed adaptive CSP has been compared with that of conventional CSP in terms of three separability metrics: Davies Bouldin Index (DBI), Fisher's Score (FS) and Dunn's Index (DI). Experimental results show significantly higher separability of features extracted with adaptive CSP as compared to that with conventional CSP.
Article
Steady State Motion Visual Evoked Potential (SSMVEP)-based Brain Computer Interface (BCI) is widely studied and has been used to varies of occasions on account of its good performance, mild stimulation, and free of additional training. We design a trolley control system based on SSMVEP signals and observe a phenomenon named “BCI Illiterate”, in which case some subjects present unsatisfactory performance with low classification accuracies. In order to cope with this challenging problem in real-world contexts, we introduce a deep learning (DL) method. The method allows improving the accuracies for both EEG literate and EEG illiterate. In particular, we firstly conduct SSMVEP experiments to obtain EEG signals from 10 subjects, including 5 EEG literates and 5 EEG illiterates. Then we construct a convolutional neural network with long short-term memory (CNN-LSTM) framework, which allows extracting the spectral, spatial, and temporal features of EEG signals, to realize the high classification accuracies of SSMVEP signals. The results show that DL method can reach 96.83% and 91.86% for EEG literate and EEG illiterate respectively, which are 12.68% and 31.08% higher than the results of traditional methods. These results indicate that DL method is not only suitable for EEG literate, but more importantly, can greatly improve the performance for EEG illiterate, which finally can enhance the robustness and universality of the SSMVEP-based BCI.
Article
Background BCI systems based on steady-state visual evoked potentials (SSVEP) have formed an immense contribution to practical applications, due to their high recognition accuracy and ease of use. The MLR method has a better frequency recognition accuracy for short-term windows, and the MsetCCA method works more accurately in long-term windows. New Method The proposed fuzzy ensemble system can analyze the relevant SSVEP signals of each subject from 0.5 to 4-second windows with 0.5-second incremental steps. It is capable of taking decisions to improve the accuracy of SSVEP stimulation frequency recognition using the MLR and MsetCCA methods. Results Our fuzzy system provides high-accuracy results for the stimulation frequency recognition in signals with the length of 1second and more. Specifically, the average accuracy of 2-second windows has improved to 100 percent. Comparison with existing methods The recognition accuracy of the presented system is always better than both MLR and MsetCCA methods. Conclusion One of the capabilities of fuzzy systems is that they can use human information and knowledge to build engineering systems. The fuzzy ensemble system can utilize various methods or classifiers simultaneously. The new system has proposed to combine multiple methods using the fuzzy ensemble, which encompasses the benefits of all the subsystems.
Article
A novel exactly periodic spatial filtering (EPSD) approach, that provides a robust detection performance, is introduced and evaluated in this study. The proposed method exploits the temporal properties of the steady-state visual evoked potential (SSVEP) response to construct an orthogonal and exactly periodic mapping that enhances the signal to noise ratio (SNR) of the SSVEP embedded in the electroencephalogram (EEG) data. The subspace of interest is constructed via the elimination of the signals spaces that does not constitute the exact period of the target frequency. The EPSD is evaluated on a 35 subject benchmark dataset collected using a 40 target SSVEP BCI system. The results reveal that the proposed EPSD spatial filter significantly enhances the performance of target detection. Further statistical tests also confirm that the EPSD is a potential alternative to the existing SSVEP spatial filters for realizing an efficient BCI system.
Article
Background and objective: Steady-state visual evoked potential (SSVEP) and rapid serial visual presentation (RSVP) are useful methods in the brain-computer interface (BCI) systems. Hybrid BCI systems that combine these two approaches can enhance the proficiency of the P300 spellers. Methods: In this study, a new hybrid RSVP/SSVEP BCI is proposed to increase the classification accuracy and information transfer rate (ITR) as compared with the other RSVP speller paradigms. In this paradigm, RSVP (eliciting a P300 response) and SSVEP stimulations are presented in such a way that the target group of characters is identified by RSVP stimuli, and the target character is recognized by SSVEP stimuli. Results: The proposed paradigm achieved accuracy of 93.06%, and ITR of 23.41 bit/min averaged across six subjects. Conclusions: The new hybrid system demonstrates that by using SSVEP stimulation in Triple RSVP speller paradigm, we could enhance the performance of the system as compared with the traditional Triple RSVP paradigm. Our work is the first hybrid paradigm in RSVP spellers that could obtain the higher classification accuracy and information transfer rate in comparison with the previous RSVP spellers.
Preprint
(datasets: https://github.com/IoBT-VISTEC/EEG-Emotion-Recognition) Since the launch of the first consumer grade EEG measuring sensors 'NeuroSky Mindset' in 2007, the market has witnessed an introduction of at least one new product every year by competing manufacturers, which include NeuroSky, Emotiv, interaXon and OpenBCI. There are numerous variations in the make and versions, but these products clearly share the key selling points of affordability, portability, and ease of use. These features are patently well placed provided one of the main objectives for their development is to attract a new target group of commercial users. Nevertheless, with several decades of traditional EEG usage in clinical and experimental settings, the shift toward commercial and engineering sides has not been achieved without skepticism. With this in mind, researchers in related fields have been tirelessly working to ensure that these putatively novel features were not introduced at the expense of efficiency and accuracy by conducting validation studies to compare the performance of data derived from consumer grade EEG devices with ones from standard research grade counterparts. In this review, we seek to provide the detail of the products supplied by the major players, summarize studies that evaluate consumer product's performance against research grade devices, the key areas of applications that these consumer grade devices have been employed in over the past five years or so, and finally give our perspectives on the limitations and what these innovative tools could offer going forward in terms of research and commercial applications.
Article
To enhance the performance of the brain-actuated robot system, a novel shared controller based on Bayesian approach is proposed for intelligently combining robot automatic control and brain-actuated control, which takes into account the uncertainty of robot perception, action and human control. Based on maximum a posteriori probability (MAP), this method establishes the probabilistic models of human and robot control commands to realize the optimal control of a brain-actuated shared control system. Application on an intelligent Bayesian shared control system based on steady-state visual evoked potential (SSVEP)-based brain machine interface (BMI) is presented for all-time continuous wheelchair navigation task. Moreover, to obtain more accurate brain control commands for shared controller and adapt the proposed system to the uncertainty of electroencephalogram (EEG), a hierarchical brain control mechanism with feedback rule is designed. Experiments have been conducted to verify the proposed system in several scenarios. Eleven subjects participated in our experiments and the results illustrate the effectiveness of the proposed method.
Article
Canonical correlation analysis (CCA) has been widely used for frequency recognition in steady-state visual evoked potential (SSVEP) based brain–computer interfaces (BCIs). However, linear CCA-based methods may be insufficient given the complexity of EEG signals. A nonlinear feature extraction method based on deep multiset CCA (DMCCA) is proposed for SSVEP recognition to fully utilize the real EEG and constructed sine–cosine signals. In DMCCA, neural networks are trained to learn the nonlinear representations of multiple sets of EEG signals at the same frequency by maximizing the overall correlation within the representations and reference signals. Therefore, reference signals are augmented with the extracted features for frequency recognition. Finally, the proposed method is evaluated using SSVEP signals collected from 10 subjects. DMCCA-based method outperforms others in terms of classification accuracy compared with CCA- and multiset CCA-based methods. The proposed DMCCA-based method has substantial potential for improving the recognition performance of SSVEP signals.
Conference Paper
A four-class brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEPs) was developed by presenting phase-coded 60Hz stimulations on a 240Hz LCD monitor. The task-related component analysis (TRCA) algorithm was used to detect SSVEPs with individual training data. In the BCI experiment with 10 subjects, the system achieved high classification accuracy of 94.50±6.70% and 92.71±7.56% in offline and online BCI experiments, resulting in information transfer rates (ITR) of 19.95±4.36 and 18.81±4.74 bpm, respectively. The behavioral tests on visual comfortableness and perception of flickering reveal that the proposed BCI system is very comfortable to use without any perception of flicker.
Article
The steady-state visually evoked potential (SSVEP) based brain-computer interfaces (BCIs) generally deploy flickering stimuli with different frequencies in order to generate different commands. This paper presents a setup that can be used to generate multiple commands from a single flickering stimulus using magnitude modulation of SSVEP through eye-accommodation. In this setup, a flickering stimulus was shown on the computer screen and a passive fixation target was placed between the screen and the subject. The eye-accommodation mechanism to focus on the target between the screen and the subject, caused the flickering stimulus to become blurred which reduced the magnitude of the evoked SSVEP response. The reduced magnitude SSVEP response can be used to generate another command over the command generated when the subject focuses directly on the stimulus. The fixation target was placed at 3 different positions that can provide up to 4 commands from the single flicker stimulus. Fifteen healthy human subjects participated in the experiments. The mean offline accuracies obtained for 2-class, 3-class, and 4-class extraction were 100%, 94.2 ± 6.1%, and 80.9 ± 9.7% respectively for a 4-seconds time window.
Article
Autism has traditionally been regarded as a disorder of the social brain. Recent reports of differences in visual perception have challenged this notion, but little evidence for altered visual processing in the autistic brain exists. We have previously observed slower behaviorally reported rates of a basic visual phenomenon, binocular rivalry, in autism [1, 2]. During rivalry, two images-one presented to each eye-vie for awareness, alternating back and forth in perception. This competition is modeled to rely, in part, on the balance of excitation and inhibition in visual cortex [3-8], which may be altered in autism [2, 9-14]. Yet direct neural evidence for this potential marker of excitation/inhibition (E/I) balance in autism is lacking. Here, we report a striking alteration in the neural dynamics of binocular rivalry in individuals with autism. Participants viewed true and simulated frequency-tagged binocular rivalry displays while steady-state visually evoked potentials (SSVEPs) were measured over occipital cortex using electroencephalography (EEG). First, we replicate our prior behavioral findings of slower rivalry and reduced perceptual suppression in individuals with autism compared with controls. Second, we provide direct neural evidence for slower rivalry in autism compared with controls, which strongly predicted individuals' behavioral switch rates. Finally, using neural data alone, we were able to predict autism symptom severity (ADOS) and correctly classify individuals' diagnostic status (autistic versus control; 87% accuracy). These findings clearly implicate atypical visual processing in the neurobiology of autism. Down the road, this paradigm may serve as a non-verbal marker of autism for developmental and cross-species research.
Preprint
Brain-Computer interfaces (BCIs) play a significant role in easing neuromuscular patients on controlling computers and prosthetics. Due to their high signal-to-noise ratio, steady state visually evoked potentials (SSVEPs) has been widely used to build BCIs. However, currently developed algorithms do not predict the modulation of SSVEP amplitude, which is known to change as a function of stimulus luminance contrast. In this study, we aim to develop an integrated approach to simultaneously estimate the frequency and contrast-related amplitude modulations of the SSVEP signal. To achieve that, we developed a behavioral task in which human participants focused on a visual flicking target which the luminance contrast can change through time in several ways. SSVEP signals from 16 subjects were then recorded from electrodes placed at the central occipital site using a lowcost, consumer-grade EEG. Our results demonstrate that the filter bank canonical correlation analysis (FBCCA) performed well in SSVEP frequency recognition, while the support vector regression (SVR) outperformed the other supervised machine learning algorithms in predicting the contrast-dependent amplitude modulations of the SSVEPs. These findings indicate the applicability and strong performance of our integrated method at simultaneously predicting both frequency and amplitude of visually evoked signals, and have proven to be useful for advancing SSVEP-based applications.
Article
Inherent fuzzy entropy is an objective measurement of electroencephalography (EEG) complexity reflecting the robustness of brain systems. In this study, we present a novel application of multiscale relative inherent fuzzy entropy using repetitive steady-state visual evoked potentials (SSVEPs) to investigate EEG complexity change between two migraine phases, i.e., interictal (baseline) and preictal (before migraine attacks) phases. We used a wearable headband EEG device with O1, Oz, O2, and Fpz electrodes to collect EEG signals from 80 participants [40 migraine patients and 40 healthy controls (HCs)] under the following two conditions: During resting state and SSVEPs with five 15-Hz photic stimuli. We found a significant enhancement in occipital EEG entropy with increasing stimulus times in both HCs and patients in the interictal phase, but a reverse trend in patients in the preictal phase. In the 1st SSVEP, occipital EEG entropy of the HCs was significantly lower than that of patents in the preictal phase (FDR-adjusted p < 0.05). Regarding the transitional variance of EEG entropy between the 1st and 5th SSVEPs, patients in the preictal phase exhibited significantly lower values than patients in the interictal phase (FDR-adjusted p < 0.05). Furthermore, in the classification model, the AdaBoost ensemble learning showed an accuracy of 81 $ \pm $ 6% and area under the curve of 0.87 for classifying interictal and preictal phases. In contrast, there were no differences in EEG entropy among groups or sessions by using other competing entropy models, including approximate entropy, sample entropy, and fuzzy entropy on the same dataset. In conclusion, inherent fuzzy entropy offers novel applications in visual stimulus environments and may have the potential to provide a preictal alert to migraine patients.
Article
Brain–computer interfaces (BCI) harnessing steady state visual evoked potentials (SSVEPs) manipulate the frequency and phase of visual stimuli to generate predictable oscillations in neural activity. For BCI spellers, oscillations are matched with alphanumeric characters allowing users to select target numbers and letters. Advances in BCI spellers can, in part, be accredited to subject-specific optimization, including; 1) custom electrode arrangements; 2) filter sub-band assessments; and 3) stimulus parameter tuning. Here, we apply deep convolutional neural networks (DCNNs) demonstrating cross-subject functionality for the classification of frequency and phase encoded SSVEP. Electroencephalogram (EEG) data are collected and classified using the same parameters across subjects. Subjects fixate forty randomly cued flickering characters ( $\textsf {5} \times \textsf {8}$ keyboard array) during concurrent wet-EEG acquisition. These data are provided by an open source SSVEP dataset. Our proposed DCNN, PodNet, achieves 86% and 77% offline accuracy of classification across-subjects for two data capture periods, respectively, 6-seconds (information transfer rate = 40 bpm) and 2-seconds (information transfer rate = 101 bpm). Subjects demonstrating sub-optimal (<70%) performance are classified to similar levels after a short subject-specific training period. PodNet outperforms filter-bank canonical correlation analysis for a low volume (3-channel) clinically feasible occipital electrode configuration. The networks defined in this study achieve functional performance for the largest number of SSVEP classes decoded via DCNN to date. Our results demonstrate PodNet achieves cross-subject, calibrationless classification and adaptability to sub-optimal subject data, and low-volume EEG electrode arrangements.
Chapter
One of the procedures often used in an SSVEP-BCI (Steady State Evoked Potential Brain Computer Interface) processing pipeline is multichannel spatial filtering. This procedure not only improves SSVEP-BCI classification accuracy but also provides higher flexibility in choosing the localization of EEG electrodes on the user scalp. The problem is, however, how to choose the spatial filter that provides the highest classification accuracy for the given BCI settings. Although there are some papers comparing filtering procedures, the comparison is usually done in terms of one, strictly defined BCI setup [1, 2]. Such comparisons do not inform, however, whether some filtering procedures are superior to the others regardless of the experimental conditions. The research reported in this paper partially fills this gap. During the research four spatial filtering procedures (MEC, MCC, CCA, and FBCCA) were compared under 15 slightly different SSVEP-BCI setups. The main finding was that none of the procedures showed clear predominance in all 15 setups. By applying not-the-best procedure the classification accuracy dropped significantly, even of more than 30%.
Article
Background: Brain computer interface (BCI) technology is a communication and control approach. Up to now many studies have attempted to develop an EEG-based BCI system to improve the quality of life of people with severe disabilities, such as amyotrophic lateral sclerosis (ALS), paralysis, brain stroke and so on. The proposed BCIBSHS could help to provide a new way for supporting life of paralyzed people and elderly people. Objective: The goal of this paper is to explore how to set up a cost-effective and safe-to-use online BCIBSHS to recognize multi-commands and control smart devices based on SSVEP. Methods: The portable EEG acquisition device (Emotiv EPOC) was used to collect EEG signals. The raw signals were denoised by discrete wavelet transform (DWT) method, and then the canonical correlation analysis (CCA) method was used for feature extraction and classification. Another part is the control of smart home devices. The classification results of SSVEP can be translated into commands to control several devices for the smart home. Results: Here, the Power over Ethernet (PoE) technology was utilized to provide electrical energy and communication for those devices. During online experiments, four different control commands have been achieved to control four smart home devices (lamp, web camera, guardianship telephone and intelligent blinds). Experimental results showed that the online BCIBSHS obtained 86.88 ± 5.30% average classification accuracy rate. Conclusion: The BCI and PoE technology, combined with smart home system, overcoming the shortcomings of traditional systems and achieving home applications management rely on EEG signal. In this paper, we proposed an online steady-state visual evoked potential (SSVEP) based BCI system on controlling several smart home devices.
Article
Objective: Despite the vast research aimed at improving the performance of steady-state visually evoked potential (SSVEP)-based brain-computer interfaces (BCIs), several limitations exist that restrict the use of such applications for long-term users in the real-world. One of the main challenges has been to reduce user training while maintaining good BCI performance. Motivated by the same aim, this survey identifies and compares the different training requirements of feature extraction methods for SSVEP-based BCIs. Approach: This paper reviews the various state-of-the-art SSVEP feature extraction methods that have been developed and are most widely used in the literature. Main results: The main contributions compared to existing reviews are the following: (i) a detailed summary, including a brief mathematical description of each feature extraction algorithm, providing a guide to the basic concepts of the state-of-the-art techniques for SSVEP-based BCIs found in literature; (ii) a categorisation of the training requirements of SSVEP-based methods into three categories, defined as training-less methods and subject-specific and subject-independent training methods; (iii) a comparative review of the training requirements of SSVEP feature extraction methods, providing a reference for future work on SSVEP-based BCIs. Significance: This review highlights the strengths and weaknesses of the three categories of SSVEP training methods. Training-less systems are more practical but their performance is limited due to inter-subject variability resulting from the complex EEG activity. Feature extraction methods that incorporate some training data address this issue and in fact have outperformed training-less methods: subject-specific BCIs are tuned to the individual yielding the best performance at the cost of long, tiring training sessions making these methods unsuitable for everyday use; subject-independent BCIs that make use of training data from various subjects offer a good trade-off between training effort and performance, making these BCIs better suited for practical use.
Article
Goal: A new method for steady-state visual evoked potentials (SSVEPs) frequency recognition is proposed to enhance the performance of SSVEP-based brain-computer interface (BCI). Methods: Correlated component analysis (CORCA) is introduced, which originally was designed to find linear combinations of electrodes that are consistent across subjects and maximally correlated between them. We propose a CORCA algorithm to learn spatial filters with multiple blocks of individual training data for SSVEP-based BCI scenario. The spatial filters are used to remove background noises by combining the multichannel EEG signals. We conduct a comparison between the proposed CORCA-based and the task-related component analysis (TRCA) based methods using a 40-class SSVEP benchmark dataset recorded from 35 subjects. Results: Our experimental study validates the efficiency of the CORCA-based method, and the extensive comparison results indicate that the CORCAbased method significantly outperforms the TRCA-based method. Conclusion and Significance: Superior performance demonstrates that the proposed method holds the promising potential to achieve satisfactory performance for SSVEP-based BCI with a large number of targets.
Article
The Steady State Visual Evoked Potential (SSVEP)-based Brain Computer Interface (BCI) system has seen extensively applications in many fields, such as physical recovery of handicap persons, obstacle avoidance of intelligent vehicles, entertainment and smart homes. However, subjects easily get fatigued because of the involving long-time operations. The presence of fatigue symptoms typically affect the efficiency of the BCI system, so investigating the effects of fatigue on the SSVEP classification accuracy from the perspective of brain network becomes a challenging issue of significant importance. In this paper, we develop an adaptive optimal-Kernel time-frequency representation (AOK-TFR)-based complex network method for characterizing fatigued behavior using the SSVEP-based BCI system. We apply the traditional Canonical Correlation Analysis (CCA) and Fisher Linear Discriminant Analysis (FLDA) to classify SSVEP signals. We find that the classification accuracy at the fatigue states is significantly lower than that at the normal states. To reveal the reasons, we infer and analyze the AOK-TFR-based functional brain network with SSVEP signals. In particular, we calculate the AOK-TFR of the acquired 30-channel SSVEP signals under both normal and fatigue conditions and then construct a brain network in terms of the two-norm distance between different channels. Our results suggest that the small-world-ness of the network at normal states is prominent, and the main brain regions associated with SSVEP are in the prefrontal cortex and occipital lobe. Our analysis sheds new insights into the understanding and management of the fatigued behavior using the SSVEP-based BCI system.
Article
Brain-computer interfaces (BCIs) not only can allow individuals to voluntarily control external devices, helping to restore lost motor functions of the disabled, but can also be used by healthy users for entertainment and gaming applications. In this study, we proposed a hybrid BCI paradigm to explore a feasible and natural way to play games by using electroencephalogram (EEG) signals in a practical environment. In this paradigm, we combined motor imagery (MI) and steady-state visually evoked potentials (SSVEPs) to generate multiple commands. A classic game, Tetris, was chosen as the control object. The novelty of this study includes the effective usage of a “dwell time” approach and fusion rules to design BCI games. To demonstrate the feasibility of the proposed hybrid paradigm, ten subjects were chosen to participate in online control experiments. The experimental results showed that all subjects successfully completed the predefined tasks with high accuracy. This proposed hybrid BCI paradigm could potentially provide those who suffer disability or paralysis with additional entertainment options, such as brain-actuated games, that could improve their happiness and quality of life. Abbreviations: BCI: brain-computer interface; EEG: electroencephalogram; MI: motor imagery; SSVEP: steady-state visually evoked potential; ERP: event-related potential; SMR: sensorimotor rhythm; VEP: visual evoked potential; TCP/IP: transmission control protocol/internet protocol; GUI: graphical user interface; ERD/ERS: event-related desynchronization/synchronization; CIC: control intention classifier; LRC: left/right classifier; CSP: common spatial pattern; LDA: linear discriminant analysis; ROC: receiver operating characteristic; TPR: true positive rate; FPR: false positive rate; CCA: canonical correlation analysis.