No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The current state of electrocorticography-based brain–computer interfaces
Abstract
Brain–computer interfaces (BCIs) provide a way for the brain to interface directly with a computer. Many different brain signals can be used to control a device, varying in ease of recording, reliability, stability, temporal and spatial resolution, and noise. Electrocorticography (ECoG) electrodes provide a highly reliable signal from the human brain surface, and these signals have been used to decode movements, vision, and speech. ECoG-based BCIs are being developed to provide increased options for treatment and assistive devices for patients who have functional limitations. Decoding ECoG signals in real time provides direct feedback to the patient and can be used to control a cursor on a computer or an exoskeleton. In this review, the authors describe the current state of ECoG-based BCIs that are approaching clinical viability for restoring lost communication and motor function in patients with amyotrophic lateral sclerosis or tetraplegia. These studies provide a proof of principle and the possibility that ECoG-based BCI technology may also be useful in the future for assisting in the cortical rehabilitation of patients who have suffered a stroke.
... During this period, many patients voluntarily enrolled in studies that correlate the real-time recorded signals with specific actions or tasks. Commercial ECoG arrays for clinical use as well as BMI-related items are available from various companies such as NeuroPace, Medtronic, and PMT Corp [31]. Standard clinical ECoG arrays consist of 2.3 mm-diameter electrodes that are spaced 1 cm apart [31]. ...
... Commercial ECoG arrays for clinical use as well as BMI-related items are available from various companies such as NeuroPace, Medtronic, and PMT Corp [31]. Standard clinical ECoG arrays consist of 2.3 mm-diameter electrodes that are spaced 1 cm apart [31]. Researchers have been working to enhance the spatiotemporal resolution of ECoG. ...
... The researchers used a wireless 64-channel ECoG recording implant developed for long-term clinical applications called WIMAGINE®. Although the system did not work as intended all the time, the patient was able to walk around and control a multi-jointed upper limb with 8 degrees of movement over 2 years [31]. Regardless of these encouraging results, the overall performance of noninvasive/ partially invasive BMI is restricted by a small number of commands. ...
... Second, electrocorticography (ECoG) involves surgically implanting electrodes under the dura (the external lining of the brain), directly on the surface of the brain. ECoG gathers electrical data from populations of about a million cells at a time (Miller et al., 2020). Third, intracortical devices involve surgically implanting electrodes within the cortex of the brain, measuring the activity of single to small groups of cells (Milekovic et al., 2018). ...
... With ECoG BCI systems, an analogous ERD can be seen, as well as a more spatially specific increase in power at higher frequencies, representing an increase in activity in a population of cells numbering in the millions (Miller et al., 2020). These BCI systems are invasive, requiring implantation of electrodes on the surface of the brain; such electrodes allow better signal fidelity and greater spatial resolution compared to EEG, but detect signals from limited regions of the brain. ...
... These BCI systems are invasive, requiring implantation of electrodes on the surface of the brain; such electrodes allow better signal fidelity and greater spatial resolution compared to EEG, but detect signals from limited regions of the brain. Currently, implantable systems allow anywhere from 1 to 64 channels of data and have been used to translate mental imagery into 1-, 2-, and 3-dimensional control signals (Miller et al., 2020). The nature of these brain-derived control signals is generally stable over years (Fraczek et al., 2021b), and long-term feasibility of BCI implants with small numbers of electrodes has been reported in multiple series. ...
Intelligent neurotechnology is an emerging field that combines neurotechnologies like brain-computer interface (BCI) with artificial intelligence. This paper introduces a capability framework to assess the responsible use of intelligent BCI systems and provide practical ethical guidance. It proposes two tests, the threshold and flourishing tests, that BCI applications must meet, and illustrates them in a series of cases. After a brief introduction (Section 1), Section 2 sets forth the capability view and the two tests. It illustrates the threshold test using examples from clinical medicine of BCI applications that enable patients with profound disabilities to function at a threshold level through computer mediation. Section 3 illustrates the flourishing test by exploring possible future applications of BCI involving neuroenhancements for healthy people, using examples adapted from research currently underway in the US military. Section 3 applies a capability lens to a complex case involving dual effects, both therapeutic and non-therapeutic, showing how the threshold and flourishing tests resolve the case. Section 4 replies to three objections: neurorights are the best tool for assessing BCI; the two tests are moving targets; and the analysis utilizes a capability view to do work it is not designed for. The paper concludes that a capability view offers unique advantages and gives practical guidance for evaluating the responsible use of present and future BCI applications. Extrapolating from our analysis may help guide other emerging technologies, such as germline gene editing, expected to impact central human capabilities.
... To begin this process, we performed a scoping review to identify and catalog the ways in which caregivers are mentioned in published studies of implantable BCIs. We defined the scope of our review to include BCI research for paralysis, aphasia, and blindness, as these conditions are currently the major experimental applications for BCIs (Miller et al., 2020). Understanding how caregivers are discussed in BCI research publications is not only important to honor the important work that they do and respect the integral role they play in the lives of the participants, but is also a step towards developing a comprehensive methodology that responsibly integrates caregivers in future trials. ...
... Guidelines given by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) were followed throughout, and the review protocol was published with the Open Science Framework. To identify the keywords for our search, we looked at the keywords used in previous reviews of BCI research (Livanis et al., 2024;Miller et al., 2020), and iteratively refined them to best capture all relevant implantable BCI research studies for motor dysfunction, communication impairment (caused by stroke, ALS, or spinal cord injury), and blindness. This process culminated in the query terms found in Table 1. ...
Introduction
While the ethical significance of caregivers in neurological research has increasingly been recognized, the role of caregivers in brain-computer interface (BCI) research has received relatively less attention.
Objectives
This report investigates the extent to which caregivers are mentioned in publications describing implantable BCI (iBCI) research for individuals with motor dysfunction, communication impairment, and blindness.
Methods
The scoping review was conducted in June 2024 using the PubMed and Web of Science bibliographic databases. The articles were systematically searched using query terms for caregivers, family members, and guardians, and the results were quantitatively and qualitatively analyzed.
Results
Our search yielded 315 unique studies, 78 of which were included in this scoping review. Thirty-four (43.6%) of the 78 articles mentioned the study participant’s caregivers. We sorted these into 5 categories: Twenty-two (64.7%) of the 34 articles thanked caregivers in the acknowledgement section, 6 (17.6%) articles described the caregiver’s role with regard to the consent process, 12 (35.3%) described the caregiver’s role in the technical maintenance and upkeep of the BCI system or in other procedural aspects of the study, 9 (26.5%) discussed how the BCI enhanced participant communication and goal-directed behavior with the help of a caregiver, and 3 (8.8%) articles included general comments that did not fit into the other categories but still related to the importance of caregivers in the lives of the research participants.
Discussion
Caregivers were mentioned in less than half of BCI studies in this review. The studies that offered more robust discussions of caregivers provide valuable insight into the integral role that caregivers play in supporting the study participants and the research process. Attention to the role of caregivers in successful BCI research studies can help guide the responsible development of future BCI study protocols.
... sharp-wave ripples, [25,35]). High gamma power is a reliable indicator of population firing rate [36,37,38,39,40,41,42,1,43] and dendritic activity [29,44,45] in the absence of rhythms. The other biomarker of the underlying network activity, spectral slope, refers to the power-law exponent required to fit a 1/f β -like distribution to the measured power spectrum, i.e. the slope of the spectrum on a log-log plot. ...
... No reuse allowed without permission. interfaces [36,37,38,39,88], where high gamma power Hz) is a common proxy for population firing rates. The success of these models in both theoretical and empirical research speaks to their substantial explanatory power, despite the simplicity (and linearity) of their structure. ...
Neural electrophysiological recordings arise from interacting rhythmic (oscillatory) and broadband (aperiodic) biological subprocesses. Both rhythmic and broadband processes contribute to the neural power spectrum, which decomposes the variance of a neural recording across frequencies. Although an extensive body of literature has successfully studied rhythms in various diseases and brain states, researchers only recently have systematically studied the characteristics of broadband effects in the power spectrum. Broadband effects can generally be categorized as 1) shifts in power across all frequencies, which correlate with changes in local firing rates and 2) changes in the overall shape of the power spectrum, such as the spectral slope or power law exponent. Shape changes are evident in various conditions and brain states, influenced by factors such as excitation to inhibition balance, age, and various diseases. It is increasingly recognized that broadband and rhythmic effects can interact on a sub-second timescale. For example, broadband power is time-locked to the phase of <1 Hz rhythms in propofol induced unconsciousness. Modeling tools that explicitly deal with both rhythmic and broadband contributors to the power spectrum and that capture their interactions are essential to help improve the interpretability of power spectral effects. Here, we introduce a tractable stochastic forward modeling framework designed to capture both narrowband and broadband spectral effects when prior knowledge or theory about the primary biophysical processes involved is available. Population-level neural recordings are modeled as the sum of filtered point processes (FPPs), each representing the contribution of a different biophysical process such as action potentials or postsynaptic potentials of different types. Our approach builds on prior neuroscience FPP work by allowing multiple interacting processes and time-varying firing rates and by deriving theoretical power spectra and cross-spectra. We demonstrate several properties of the models, including that they divide the power spectrum into frequency ranges dominated by rhythmic and broadband effects, and that they can capture spectral effects across multiple timescales, including sub-second cross-frequency coupling. The framework can be used to interpret empirically observed power spectra and cross-frequency coupling effects in biophysical terms, which bridges the gap between theoretical models and experimental results.
... By attaching skin electrodes outside the eye near the lateral and medial canthus, the potential can be measured by having the patient move the eyes horizontally a set distance [38] Eyes EMG Needle electrodes or surface electrodes coupled to a chart recorder or a computer [39] EMG is a technique for recording biomedical electrical signals obtained from the neuromuscular activities [39] Muscles EEG Neurosky Mind-Wave, InteraXon Muse, Emotiv Epoc, Emotiv Insight, and OpenBCI [40] EEG is an electrophysiological technique for the recording of electrical activity arising from the human brain. EEG devices record voltage differences between different points by comparing different electrical signals comming from a pair of electrodes [41] Brain ECoG Electrodes coupled to a subcutaneous amplifier and transmitter unit (with leads connected to the electrode array) and a a processing computer for realtime analysis of the signal [42] ECoG is a technique for recording brain signals placing electrodes subdurally on the arachnoidal surface of the brain [42] Brain Bioacoustic PCG Phonocardiograph, smart stethoscopes PCG consists on recording all sounds made by heart during a cardiac cycle [43] Heart Speech Voice Recorders, microphones, sound recorders, etc [44]. ...
... By attaching skin electrodes outside the eye near the lateral and medial canthus, the potential can be measured by having the patient move the eyes horizontally a set distance [38] Eyes EMG Needle electrodes or surface electrodes coupled to a chart recorder or a computer [39] EMG is a technique for recording biomedical electrical signals obtained from the neuromuscular activities [39] Muscles EEG Neurosky Mind-Wave, InteraXon Muse, Emotiv Epoc, Emotiv Insight, and OpenBCI [40] EEG is an electrophysiological technique for the recording of electrical activity arising from the human brain. EEG devices record voltage differences between different points by comparing different electrical signals comming from a pair of electrodes [41] Brain ECoG Electrodes coupled to a subcutaneous amplifier and transmitter unit (with leads connected to the electrode array) and a a processing computer for realtime analysis of the signal [42] ECoG is a technique for recording brain signals placing electrodes subdurally on the arachnoidal surface of the brain [42] Brain Bioacoustic PCG Phonocardiograph, smart stethoscopes PCG consists on recording all sounds made by heart during a cardiac cycle [43] Heart Speech Voice Recorders, microphones, sound recorders, etc [44]. ...
... Speech loss due to neurological deficits is a severe disability that limits social and work life. Advances in machine learning and Brain-computer interface (BCI) systems have pushed the envelope to develop neural speech prostheses to enable people with speech loss to communicate [1][2][3][4][5]. An effective modality for acquiring data to develop such decoders involves Electrocorticographic (ECoG) recordings obtained in epilepsy surgery patients [4][5][6][7][8][9][10]. ...
... x(k, m) 2 where k 1 (j) and k 2 (j) denote the one-third octave band edges rounded to the nearest DFT-bin. The TF representation of the processed speechŷ is obtained similarly and denoted by Y j (m). ...
Decoding human speech from neural signals is essential for brain-computer interface (BCI) technologies restoring speech function in populations with neurological deficits. However, it remains a highly challenging task, compounded by the scarce availability of neural signals with corresponding speech, data complexity, and high dimensionality, and the limited publicly available source code. Here, we present a novel deep learning-based neural speech decoding framework that includes an ECoG Decoder that translates electrocorticographic (ECoG) signals from the cortex into interpretable speech parameters and a novel differentiable Speech Synthesizer that maps speech parameters to spectrograms. We develop a companion audio-to-audio auto-encoder consisting of a Speech Encoder and the same Speech Synthesizer to generate reference speech parameters to facilitate the ECoG Decoder training. This framework generates natural-sounding speech and is highly reproducible across a cohort of 48 participants. Among three neural network architectures for the ECoG Decoder, the 3D ResNet model has the best decoding performance (PCC=0.804) in predicting the original speech spectrogram, closely followed by the SWIN model (PCC=0.796). Our experimental results show that our models can decode speech with high correlation even when limited to only causal operations, which is necessary for adoption by real-time neural prostheses. We successfully decode speech in participants with either left or right hemisphere coverage, which could lead to speech prostheses in patients with speech deficits resulting from left hemisphere damage. Further, we use an occlusion analysis to identify cortical regions contributing to speech decoding across our models. Finally, we provide open-source code for our two-stage training pipeline along with associated preprocessing and visualization tools to enable reproducible research and drive research across the speech science and prostheses communities.
... Another variety of invasive BCI is based on electrocorticographic (ECoG) recordings, which involves surgical implantation of a grid of electrodes that lie on the surface of the cortex, beneath the skull, but which do not penetrate the neural tissue [5]. ECoG grids provide wider coverage than implanted electrodes, but do not allow the identification of individual neurons. ...
Brain–computer interfaces (BCIs) enable direct communication between the brain and external computers, allowing processing of brain activity and the ability to control external devices. While often used for medical purposes, BCIs may also hold great promise for nonmedical purposes to unlock human neurocognitive potential. In this Essay, we discuss the prospects and challenges of using BCIs for cognitive enhancement, focusing specifically on invasive enhancement BCIs (eBCIs). We discuss the ethical, legal, and scientific implications of eBCIs, including issues related to privacy, autonomy, inequality, and the broader societal impact of cognitive enhancement technologies. We conclude that the development of eBCIs raises challenges far beyond practical pros and cons, prompting fundamental questions regarding the nature of conscious selfhood and about who—and what—we are, and ought, to be.
... BCI-based brain signals, including magnetoencephalography (Mridha et al., 2021;Peksa and Mamchur, 2023), electrocorticography (Miller et al., 2020), functional magnetic resonance imaging (Guo, 2020), and electroencephalogram (EEG)-based systems (Rashid et al., 2020;Ydaav and Maini, 2023). EEG-based BCIs have gained significant attention owing to their non-invasiveness, affordability, high temporal resolution, and widespread availability (Rashid et al., 2020;Ydaav and Maini, 2023). ...
Introduction
Event-related potentials (ERPs), such as P300, are widely utilized for non-invasive monitoring of brain activity in brain-computer interfaces (BCIs) via electroencephalogram (EEG). However, the non-stationary nature of EEG signals and different data distributions among subjects create significant challenges for implementing real-time P300-based BCIs. This requires time-consuming calibration and a large number of training samples.
Methods
To address these challenges, this study proposes a transfer learning-based approach that uses a convolutional neural network for high-level feature extraction, followed by Euclidean space data alignment to ensure similar distributions of extracted features. Furthermore, a source selection technique based on the Euclidean distance metric was applied to measure the distance between each source feature sample and a reference point from the target domain. The samples with the lowest distance were then chosen to increase the similarity between source and target datasets. Finally, the transferred features are applied to a discriminative restricted Boltzmann machine classifier for P300 detection.
Results
The proposed method was evaluated on the state-of-the-art BCI Competition III dataset II and rapid serial visual presentation dataset. The results demonstrate that the proposed technique achieves an average accuracy of 97% for both online and offline after 15 repetitions, which is comparable to the state-of-the-art methods. Notably, the proposed approach requires <½ of the training samples needed by previous studies.
Discussion
Therefore, this technique offers an efficient solution for developing ERP-based BCIs with robust performance against reduced a number of training data.
... The limited use of ECoG in mental health conditions may be due to the invasive nature of the procedure, as well as the lack of clear guidelines for its use in this field [149][150][151]. While the number of studies on ECoG applications is still confined, there is a flourish-ing evolution of decoding algorithms happening in recent years [125,[152][153][154]. Various strategies are being considered, such as the use of switching models and the adaptation of algorithms. ...
This review provides a comprehensive examination of recent developments in both neurofeedback and brain-computer interface (BCI) within the medical field and rehabilitation. By analyzing and comparing results obtained with various tools and techniques, we aim to offer a systematic understanding of BCI applications concerning different modalities of neurofeedback and input data utilized. Our primary objective is to address the existing gap in the area of meta-reviews, which provides a more comprehensive outlook on the field, allowing for the assessment of the current landscape and developments within the scope of BCI. Our main methodologies include meta-analysis, search queries employing relevant keywords, and a network-based approach. We are dedicated to delivering an unbiased evaluation of BCI studies, elucidating the primary vectors of research development in this field. Our review encompasses a diverse range of applications, incorporating the use of brain-computer interfaces for rehabilitation and the treatment of various diagnoses, including those related to affective spectrum disorders. By encompassing a wide variety of use cases, we aim to offer a more comprehensive perspective on the utilization of neurofeedback treatments across different contexts. The structured and organized presentation of information, complemented by accompanying visualizations and diagrams, renders this review a valuable resource for scientists and researchers engaged in the domains of biofeedback and brain-computer interfaces.
... Input signals usually belong to one of the following categories: the P300 wave of event-related potentials (ERP); SSVEP; slow cortical potentials and motor imagery (MI). The brain activity (i.e., the electroencephalogram-EEG) is usually measured noninvasively by means of electrodes (e.g., gel, water, or dry electrodes) mounted on the human scalp [15], [16]. In the proposed solution, the BCI is implemented by the NextMind device that falls under the SSVEP category. ...
This article presents a brain–computer interface (BCI) coupled with an augmented reality (AR) system to support human–robot interaction in controlling a robotic arm for pick-and-place tasks. BCIs can process steady-state visual evoked potentials (SSVEPs), which are signals generated through visual stimuli. The visual stimuli may be conveyed to the user with AR systems, expanding the range of possible applications. The proposed approach leverages the capabilities of the NextMind BCI to enable users to select objects in the range of the robotic arm. By displaying a visual anchor associated with each object in the scene with projected AR, the NextMind device can detect when users focus their eyesight on one of them, thus triggering the pick-up action of the robotic arm. The proposed system has been designed considering the needs and limitations of mobility-impaired people to support them when controlling a robotic arm for pick-and-place tasks. Two different approaches for positioning the visual anchors are proposed and analyzed. Experimental tests involving users show that both approaches are highly appreciated. The system performances are extremely robust, thus allowing the users to select objects in an easy, fast, and reliable way.
... One of the primary shortcomings of the invasive method is that it can only be used for a very limited time before it needs to be withdrawn because it can damage nearby tissue [14]. The most widely used BCI systems are non-invasive because they do not need to be implanted, and their use is neither challenging nor dangerous [15]. The purpose of this work is to explore and develop BCIs as transformative tools for translating neural signals into commands for external devices. ...
Brain-computer interfaces (BCIs) have emerged as transformative tools for translating users’ neural signals into commands for external devices. The urgent need for innovative treatments to enhance upper limb motor function in stroke survivors is underscored by the limitations of traditional rehabilitation methods. The development of communication and control technology for individuals with severe neuromuscular diseases, particularly stroke patients, is centered on utilizing electroencephalographic (EEG) signals to accurately decode users’ intentions and operate external devices. Two healthy subjects and a stroke patient were enrolled to acquire EEG signals using the EMOTIV EPOC+ sensor. The experimental procedure involved recording five actions for both motor imagery and facial expression signals to control the 3D-printed upper limb exoskeleton. EEGLAB and BCILAB software were used for preprocessing and classification. The results showed successful EEG-based control of the exoskeleton, representing a significant advancement in assistive technology for individuals with motor impairments. The support vector machine (SVM) classifier achieved higher accuracy in both offline and online modes for both motor imaginary and facial expression tasks. The conclusion highlights the appropriateness of using EEGLAB for offline EEG data analysis and BCILAB for both offline and online analysis and classification. The integration of servo motors in the exoskeleton, allowing movements in five Degrees of Freedom (DOF), positions it as an effective rehabilitation solution for individuals with upper limb impairments.
... Invasive and non-invasive recording techniques are utilized for capturing neural activity. While invasive approaches offer greater accuracy in brain activity recognition, they necessitate surgical implantation of sensors beneath the scalp [12], [13]. Conversely, non-invasive BCIs capture neural signals by pacing sensors on the scalp. ...
Objective: Electroencephalogram (EEG) signals-based motor kinematics prediction (MKP) has been an active area of research to develop brain-computer interface (BCI) systems such as exosuits, prostheses, and rehabilitation devices. However, EEG source imaging (ESI) based kinematics prediction is sparsely explored in the literature. Approach: In this study, pre-movement EEG features are utilized to predict three-dimensional (3D) hand kinematics for the grasp-and-lift motor task. A public dataset, WAY-EEG-GAL, is utilized for MKP analysis. In particular, sensor-domain (EEG data) and source-domain (ESI data) based features from the frontoparietal region are explored for MKP. Deep learning-based models are explored to achieve efficient kinematics decoding. Various time-lagged and window sizes are analyzed for hand kinematics prediction. Subsequently, intra-subject and inter-subject MKP analysis is performed to investigate the subject-specific and subject-independent motor-learning capabilities of the neural decoders. The Pearson correlation coefficient (PCC) is used as the performance metric for kinematics trajectory decoding. Main results: The rEEGNet neural decoder achieved the best performance with sensor-domain and source-domain features with the time lag and window size of 100 ms and 450 ms, respectively. The highest mean PCC values of 0.790, 0.795, and 0.637 are achieved using sensor-domain features, while 0.769, 0.777, and 0.647 are achieved using source-domain features in x, y, and z-directions, respectively. Significance: This study explores the feasibility of trajectory prediction using EEG sensor-domain and source-domain EEG features for the grasp-and-lift task. Furthermore, inter-subject trajectory estimation is performed using the proposed deep learning decoder with EEG source domain features.
... 80 Particularly, no events necessitated device explanation or resulted in death or permanently increased disability, suggesting a promising risk-benefit ratio for BCI use in individuals with SCI. 80 Moreover, BCIs extend their utility to diagnosing and predicting abnormal CNS structures like tumors, offering a cost-effective alternative to computed tomography scans and magnetic resonance imaging through EEG. 15 Looking ahead, ECoG devices show promise in aiding rehabilitation post-tumor resection by enhancing brain-spinal cord plasticity and 81 Implementing BCIs requires precise preoperative planning to ensure optimal implant placement on the brain and spinal cord. This involves using computerized tomography to identify cerebral cortex regions most responsive to BCI stimulation, ensuring effective detection of limb movement intentions. ...
Brain-Computer Interfaces (BCIs), a remarkable technological advancement in neurology and neurosurgery, mark a significant leap since the inception of electroencephalography (EEG) in 1924. These interfaces effectively convert central nervous system signals into commands for external devices, offering revolutionary benefits to patients with severe communication and motor impairments due to a myriad of neurological conditions like stroke, spinal cord injuries, and neurodegenerative disorders. BCIs enable these individuals to communicate and interact with their environment, using their brain signals to operate interfaces for communication and environmental control. This technology is especially crucial for those completely locked in, providing a communication lifeline where other methods fall short. The advantages of BCIs are profound, offering autonomy and an improved quality of life for patients with severe disabilities. They allow for direct interaction with various devices and prostheses, bypassing damaged or non-functional neural pathways. However, challenges persist, including the complexity of accurately interpreting brain signals, the need for individual calibration, and ensuring reliable, long-term use. Additionally, ethical considerations arise regarding autonomy, consent, and the potential for dependence on technology. Despite these challenges, BCIs represent a transformative development in neurotechnology, promising enhanced patient outcomes and a deeper understanding of brain-machine interfaces.
... Remarkable progress has been made in decoding motor execution and intention in recent years (49)(50)(51)(52). However, limited attention has been paid to motor recognition from this compromised eloquent cortex. ...
Glioma cells can reshape functional neuronal networks by hijacking neuronal synapses, leading to partial or complete neurological dysfunction. These mechanisms have been previously explored for language functions. However, the impact of glioma on sensorimotor functions is still unknown. Therefore, we recruited a control group of patients with unaffected motor cortex and a group of patients with glioma-infiltrated motor cortex, and recorded high-density electrocortical signals during finger movement tasks. The results showed that glioma suppresses task-related synchronization in the high-gamma band and reduces the power across all frequency bands. The resulting atypical motor information transmission model with discrete signaling pathways and delayed responses disrupts the stability of neuronal encoding patterns for finger movement kinematics across various temporal-spatial scales. These findings demonstrate that gliomas functionally invade neural circuits within the motor cortex. This result advances our understanding of motor function processing in chronic disease states, which is important to advance the surgical strategies and neurorehabilitation approaches for patients with malignant gliomas.
... The EEG signals have high temporal resolution [9,39], and the approach involves a non-invasive process of measuring the electrical activity of the cerebral cortex and does not require surgical intervention. This provides a high safety level for users of such BCIs, compared to BCIs based on electrocorticography (ECoG) [40]. The latter requires mandatory surgical intervention and, thus, poses a high level of risk, making EEG-based BCIs a safer and more promising technology for universal application. ...
Brain–computer interfaces (BCIs) based on steady-state visually evoked potentials (SSVEPs) are inexpensive and do not require user training. However, the highly personalized reaction to visual stimulation is an obstacle to the wider application of this technique, as it can be ineffective, tiring, or even harmful at certain frequencies. In our experimental study, we proposed a new approach to the selection of optimal frequencies of photostimulation. By using a custom photostimulation device, we covered a frequency range from 5 to 25 Hz with 1 Hz increments, recording the subjects’ brainwave activity (EEG) and analyzing the signal-to-noise ratio (SNR) changes at the corresponding frequencies. The proposed set of SNR-based coefficients and the discomfort index, determined by the ratio of theta and beta rhythms in the EEG signal, enables the automation of obtaining the recommended stimulation frequencies for use in SSVEP-based BCIs.
... Implantable neural interfacing electrodes serve as the basic research tools for neuroscience [1][2][3][4] as well as the clinical application tools for the diagnosis and treatment of neurological disorders [5][6][7][8][9] , thanks to the functions of electrophysiological recording and/or neural stimulation or modulation. Electrocorticography (ECoG) records the electrical activity in the brain from the sum of the local field potentials of the population of neurons by flexible electrodes implanted on the epidural or subdural surface of the brain 10 . In the last decade, minimally invasive ECoG electrodes have no longer been limited to short-term neural signal monitoring and intraoperative focal localization in surgery for epilepsy [11][12][13] . ...
Bacterial cellulose (BC), a natural biomaterial synthesized by bacteria, has a unique structure of a cellulose nanofiber-weaved three-dimensional reticulated network. BC films can be ultrasoft with sufficient mechanical strength, strong water absorption and moisture retention and have been widely used in facial masks. These films have the potential to be applied to implantable neural interfaces due to their conformality and moisture, which are two critical issues for traditional polymer or silicone electrodes. In this work, we propose a micro-electrocorticography (micro-ECoG) electrode named “Brainmask”, which comprises a BC film as the substrate and separated multichannel parylene-C microelectrodes bonded on the top surface. Brainmask can not only guarantee the precise position of microelectrode sites attached to any nonplanar epidural surface but also improve the long-lasting signal quality during acute implantation with an exposed cranial window for at least one hour, as well as the in vivo recording validated for one week. This novel ultrasoft and moist device stands as a next-generation neural interface regardless of complex surface or time of duration.
... Clinically, ECoG has been used to diagnose epileptogenic zones in presurgical monitoring since the 1940s (Nakasato et al., 1994). Nowadays, there is growing interest in using chronic ECoG electrodes in brain-machine interface (BMI, also known as brain-computer interface) applications to control neuro-prosthetic limbs or synthesize speech from neural activity in paralyzed patients (Bouchard et al., 2013;Vansteensel et al., 2016;Anumanchipalli et al., 2019;Benabid et al., 2019;Miller et al., 2020). ...
Background
Subdural electrocorticography (ECoG) signals have been proposed as a stable, good-quality source for brain-machine interfaces (BMIs), with a higher spatial and temporal resolution than electroencephalography (EEG). However, long-term implantation may lead to chronic inflammatory reactions and connective tissue encapsulation, resulting in a decline in signal recording quality. However, no study has reported the effects of the surrounding tissue on signal recording and device functionality thus far.
Methods
In this study, we implanted a wireless recording device with a customized 32-electrode-ECoG array subdurally in two nonhuman primates for 15 months. We evaluated the neural activities recorded from and wirelessly transmitted to the devices and the chronic tissue reactions around the electrodes. In addition, we measured the gain factor of the newly formed ventral fibrous tissue in vivo.
Results
Time-frequency analyses of the acute and chronic phases showed similar signal features. The average root mean square voltage and power spectral density showed relatively stable signal quality after chronic implantation. Histological examination revealed thickening of the reactive tissue around the electrode array; however, no evident inflammation in the cortex. From gain factor analysis, we found that tissue proliferation under electrodes reduced the amplitude power of signals.
Conclusion
This study suggests that subdural ECoG may provide chronic signal recordings for future clinical applications and neuroscience research. This study also highlights the need to reduce proliferation of reactive tissue ventral to the electrodes to enhance long-term stability.
... Several BCIs have been proposed over the years, leveraging information provided by brain electrical activity. Depending on the specific application, brain activity can be acquired either invasively through electrocorticogram (ECoG) or non-invasively through electroencephalogram (EEG) [2], [3]. EEG is commonly preferred as non-invasive, but scalp-measured brain electrical activity has lower signal-to-noise ratio, which poses significant challenges to develop effective BCIs [4]. ...
Brain-computer interfaces (BCIs) have revolutionized the way humans interact with machines, particularly for patients with severe motor impairments. EEG-based BCIs have limited functionality due to the restricted pool of stimuli that they can distinguish, while those elaborating event-related potentials up to now employ paradigms that require the patient's perception of the eliciting stimulus. In this work, we propose MIRACLE: a novel BCI system that combines functional data analysis and machine-learning techniques to decode patients' minds from the elicited potentials. MIRACLE relies on a hierarchical ensemble classifier recognizing 10 different semantic categories of imagined stimuli. We validated MIRACLE on an extensive dataset collected from 20 volunteers, with both imagined and perceived stimuli, to compare the system performance on the two. Furthermore, we quantify the importance of each EEG channel in the decision-making process of the classifier, which can help reduce the number of electrodes required for data acquisition, enhancing patients' comfort.
... Since the implantation of a wireless intracortical electrode in a patient with locked-in syndrome who achieved real-time vowel synthesis more than a decade ago [10], the field of speech neuroprostheses has continued its rapid transition toward clinical trials with more target patients currently benefiting from chronic brain implants [3][4][5]. However, enabling research with non-target participants remains essential to accelerate the development of such devices [11,12]. To date, the use of human intracranial recordings is mostly limited to clinical monitoring prior to brain resection for the treatment of intractable epilepsy [12,13]. ...
Objective:
Brain-computer interfaces can restore various forms of communication in paralyzed patients who have lost their ability to articulate intelligible speech. This study aimed to demonstrate the feasibility of closed-loop synthesis of artificial speech sounds from human cortical surface recordings during silent speech production.
Approach:
Ten participants with intractable epilepsy were temporarily implanted with intracranial electrode arrays over cortical surfaces. A decoding model that predicted audible outputs directly from patient-specific neural feature inputs was trained during overt word reading and immediately tested with overt, mimed and imagined word reading. Predicted outputs were later assessed objectively against corresponding voice recordings and subjectively through human perceptual judgments.
Main results:
Artificial speech sounds were successfully synthesized during overt and mimed utterances by two participants with some coverage of the precentral gyrus. About a third of these sounds were correctly identified by naïve listeners in two-alternative forced-choice tasks. A similar outcome could not be achieved during imagined utterances by any of the participants. However, neural feature contribution analyses suggested the presence of exploitable activation patterns during imagined speech in the postcentral gyrus and the superior temporal gyrus. In future work, a more comprehensive coverage of cortical surfaces, including posterior parts of the middle frontal gyrus and the inferior frontal gyrus, could improve synthesis performance during imagined speech.
Significance:
As the field of speech neuroprostheses is rapidly moving toward clinical trials, this study addressed important considerations about task instructions and brain coverage when conducting research on silent speech with non-target participants.
... In the field of neural decoding for direct communication in brain-computer interfaces (BCIs), research is progressing for detecting spoken signals from multi-channel electrocorticograms (ECoGs) at the brain cortex (Knight and Heinze, 2008;Pasley et al., 2012;Bouchard et al., 2013;Flinker et al., 2015;Herff and Schultz, 2016;Martin et al., 2018;Anumanchipalli et al., 2019;Miller et al., 2020). If we could instead detect linguistic information from scalp EEGs, then BCIs could enjoy much wider practical applications, for instance improving the quality of life (QoL) of amyotrophic lateral sclerosis (ALS) patients, but this goal is hampered by many unsolved problems (Wang et al., 2012;Min et al., 2016;Rojas and Ramos, 2016;Yoshimura et al., 2016;Yu and Shafer, 2021;Zhao et al., 2021). ...
Speech imagery recognition from electroencephalograms (EEGs) could potentially become a strong contender among non-invasive brain-computer interfaces (BCIs). In this report, first we extract language representations as the difference of line-spectra of phones by statistically analyzing many EEG signals from the Broca area. Then we extract vowels by using iterative search from hand-labeled short-syllable data. The iterative search process consists of principal component analysis (PCA) that visualizes linguistic representation of vowels through eigen-vectors φ(m), and subspace method (SM) that searches an optimum line-spectrum for redesigning φ(m). The extracted linguistic representation of Japanese vowels /i/ /e/ /a/ /o/ /u/ shows 2 distinguished spectral peaks (P1, P2) in the upper frequency range. The 5 vowels are aligned on the P1-P2 chart. A 5-vowel recognition experiment using a data set of 5 subjects and a convolutional neural network (CNN) classifier gave a mean accuracy rate of 72.6%.
... In recent years, there have been significant advances in the control of biomechatronic devices driven by biological signals derived from the user [1][2][3][4][5][6][7][8][9] . Various biological signals have been used for gesture intent recognition to drive such biomechatronic devices, including electrical signals such as surface electromyography [10][11][12] , electroencephalography 13,14 , electrocorticography 15,16 , as well as mechanical signals such as mechanomyography 17 and sonomyography [18][19][20][21][22][23][24] . These signal extraction techniques have found broad applications in rehabilitation engineering [25][26][27] . ...
There have been significant advances in biosignal extraction techniques to drive external biomechatronic devices or to use as inputs to sophisticated human machine interfaces. The control signals are typically derived from biological signals such as myoelectric measurements made either from the surface of the skin or subcutaneously. Other biosignal sensing modalities are emerging. With improvements in sensing modalities and control algorithms, it is becoming possible to robustly control the target position of a end effector. It remains largely unknown to what extent these improvements can lead to naturalistic human-like movement. In this paper, we sought to answer this question. We utilized a sensing paradigm called sonomyography based on continuous ultrasound imaging of forearm muscles. Unlike myoelectric control strategies which measure electrical activation and use the extracted signals to determine the velocity of an end-effector; sonomyography measures muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Previously, we showed that users were able to accurately and precisely perform a virtual target acquisition task using sonomyography. In this work, we investigate the time course of the control trajectories derived from sonomyography. We show that the time course of the sonomyography-derived trajectories that users take to reach virtual targets reflect the trajectories shown to be typical for kinematic characteristics observed in biological limbs. Specifically, during a target acquisition task, the velocity profiles followed a minimum jerk trajectory shown for point-to-point arm reaching movements, with similar time to target. In addition, the trajectories based on ultrasound imaging result in a systematic delay and scaling of peak movement velocity as the movement distance increased. We believe this is the first evaluation of similarities in control policies in coordinated movements in jointed limbs, and those based on position control signals extracted at the individual muscle level. These results have strong implications for the future development of control paradigms for assistive technologies.
... Most BMIs collect the electrical signals from the brain to decode neural or brain states. There are various types of neural electrical signals that can be recorded, such as electroencephalogram (EEG) 16,17 and electrocorticography (ECoG) 18,19 obtained through surface electrodes attached to the surface of the skull or brain tissue. However, EEG and ECoG, which average signals from millions of neurons, are limited by relatively low temporal and spatial resolution, restricted information for brain state decoding, and signal instability caused by motion artifacts. ...
Brain–machine interfaces (BMIs) offer the potential for the development of communication tools between the brain and external devices. The current BMI technologies for recording and modulation of electric signals from the brain have made significant contributions to areas such as neuroscience, disease diagnosis, and rehabilitation. Next-generation BMIs require long-term stable recording and modulation of electrical signals from statistically significant neuron populations with millisecond single-cell spatiotemporal resolution. However, there are challenges to achieving this stability due to the mechanical and geometrical mismatches between electronics and the brain tissue. In addition, the requirement to achieve cell-type-specific neuromodulation and transmit and process the ever-increasing volume of data on-the-fly necessitates the implementation of smart electronics. In this review, we first summarize the requirements, challenges, and current limitations of BMIs. We then highlight three major approaches to the fabrication of flexible electronics as implantable electronics, aimed at enabling long-term stable and gliosis-free BMIs. The progress of multifunctional electronics for multimodal recording and modulation of cell-type-specific components in the brain is also discussed. Furthermore, we discuss the integration of wireless and closed-loop modulation, and on-chip processing as smart electronic components for BMIs. Finally, we examine the remaining challenges in this field and the future perspectives for how flexible and smart electronics can address these problems and continue to advance the field of BMIs.
... Since ECoG has not yet been able to be used to fully restore movement (Miller et al., 2020) and ...
Objective. Previous electrophysiological research has characterized canonical oscillatory patterns associated with movement mostly from recordings of primary sensorimotor cortex. Less work has attempted to decode movement based on electrophysiological recordings from a broader array of brain areas such as those sampled by stereoelectroencephalography (sEEG), especially in humans. We aimed to identify and characterize different movement-related oscillations across a relatively broad sampling of brain areas in humans and if they extended beyond brain areas previously associated with movement. Approach. We used a linear support vector machine to decode time-frequency spectrograms time-locked to movement, and we validated our results with cluster permutation testing and common spatial pattern decoding. Main results. We were able to accurately classify sEEG spectrograms during a keypress movement task versus the inter-trial interval. Specifically, we found these previously-described patterns: beta (13–30 Hz) desynchronization, beta synchronization (rebound), pre-movement alpha (8–15 Hz) modulation, a post-movement broadband gamma (60–90 Hz) increase and an event-related potential. These oscillatory patterns were newly observed in a wide range of brain areas accessible with sEEG that are not accessible with other electrophysiology recording methods. For example, the presence of beta desynchronization in the frontal lobe was more widespread than previously described, extending outside primary and secondary motor cortices. Significance. Our classification revealed prominent time-frequency patterns which were also observed in previous studies that used non-invasive electroencephalography and electrocorticography, but here we identified these patterns in brain regions that had not yet been associated with movement. This provides new evidence for the anatomical extent of the system of putative motor networks that exhibit each of these oscillatory patterns.
... There are plenty of studies on decoding MI, as it is often used in braincomputer interfaces (BCIs) for fully or partially paralyzed patients [35]. Many of them are based on unilateral or bilateral hand movements, while many other types of movement can happen in dreams, such as running and flying. ...
Dreams are often forgotten despite their impact on our emotions and memory. In our pursuit of developing an objective dream-content recording methodology (reaDream), we focused on the motor imagery (MI)-related dream component, which is reported to be present in dreams along with other sensory, perceptual, and cognitive phenomena. It has been shown that brain activation during dreamed actions corresponds to the brain activation for the same actions in a wakeful state. This allows one to decode electrocorticographic (ECoG) brain activity during sleep using a machine learning (ML) model trained on wakeful data. ECoG data is very specific to each individual and not generalized between subjects; deep ML models are prone to overfit on small amounts of data. We propose to generalize ECoG data by combining recordings from several subjects. For that, we developed a Convolutional Neural Network (CNN)-based classifier that discriminates between hand and tongue movements in different subjects. We tested a hypothesis on whether a MI classifier can be trained on motor execution (ME) data. We demonstrate that ME types are easier to distinguish compared to MI. We showed that power features are more informative than temporal features. Finally, we demonstrated how our trained models could be used to predict MI during Rapid Eye Movement (REM) sleep.
KeywordsDream researchECoGMotor imageryCNN
... In this paragraph, we review some of the cuttingedge neurosurgical innovations in the BCI research field. In [138], a systematic review on the current state-of-art of ECoG-based BCIs for decoding movements, vision and speech and their clinical implementation in patients with ALS and with tetraplegia is presented. Alternatively, Soldozy et al. review the use of minimally invasive neurovascular approaches (stent-electrodes) in BMIs [139]. ...
With the recent developments in neuroscience and engineering, it is now possible to record brain signals and decode them. Also, a growing number of stimulation methods have emerged to modulate and influence brain activity. Current brain-computer interface (BCI) technology is mainly on therapeutic outcomes, it already demonstrated its efficiency as assistive and rehabilitative technology for patients with severe motor impairments. Recently, artificial intelligence (AI) and machine learning (ML) technologies have been used to decode brain signals. Beyond this progress, combining AI with advanced BCIs in the form of implantable neurotechnologies grants new possibilities for the diagnosis, prediction, and treatment of neurological and psychiatric disorders. In this context, we envision the development of closed loop, intelligent, low-power, and miniaturized neural interfaces that will use brain inspired AI techniques with neuromorphic hardware to process the data from the brain. This will be referred to as Brain Inspired Brain Computer Interfaces (BI-BCIs). Such neural interfaces would offer access to deeper brain regions and better understanding for brain's functions and working mechanism, which improves BCIs operative stability and system's efficiency. On one hand, brain inspired AI algorithms represented by spiking neural networks (SNNs) would be used to interpret the multimodal neural signals in the BCI system. On the other hand, due to the ability of SNNs to capture rich dynamics of biological neurons and to represent and integrate different information dimensions such as time, frequency, and phase, it would be used to model and encode complex information processing in the brain and to provide feedback to the users. This paper provides an overview of the different methods to interface with the brain, presents future applications and discusses the merger of AI and BCIs.
... At the same time, it captures the high-frequency components of neural oscillations that were attenuated in EEG. Indeed, the ECoG has undoubtedly been one of the most suitable measures in BCI studies (Miller et al 2020) with various applications not limited to offline analyses but also online (Benabid et al 2019, Degenhart et al 2018). Particularly, the wide frequency spectrum of the ECoG possibly offers a chance for optimizing neural decoding using features that are beyond the limits of previous EEG studies. ...
Objective:
Reaching hand movement is an important motor skill actively examined in brain-computer interface (BCI). Among various components of movement analyzed is the hand's trajectory, which describes the hand's continuous positions in three-dimensional space. While a large body of studies have investigated the decoding of real movements and the reconstruction of real hand movement trajectories from neural signals, fewer studies have attempted to decode the trajectory of imagined hand movement. To develop BCI systems for patients with hand motor dysfunctions, the systems essentially require to achieve movement-free control of external devices, which is only possible through successful decoding of purely imagined hand movement.
Approach:
To achieve this goal, this study used a machine learning technique (i.e., the variational Bayesian least square) to analyze the electrocorticogram (ECoG) of eighteen epilepsy patients obtained from when they performed movement execution (ME) and kinesthetic movement imagination (KMI) of the reach-and-grasp hand action.
Main results:
The variational Bayesian decoding model was able to successfully predict the imagined trajectories of hand movement significantly above chance level. The Pearson's correlation coefficient between imagined and predicted trajectories was 0.3393 and 0.4936 for the KMI (KMI trials only) and MEKMI paradigm (alternating trials of ME and KMI) respectively.
Significance:
This study demonstrated a high accuracy of prediction for trajectories of imagined hand movement, and more importantly, higher decoding accuracy of imagined trajectories in the MEKMI paradigm than in the KMI paradigm solely.
... In that regard, ECoG and sEEG recordings have been instrumental in development, testing and deployment of various examples of emerging clinical neurotechnology. See [196,197,198,199,200,201,202,203,204,205,206,207] for reviews. ...
Artificial intelligence (AI) is a fast-growing field focused on modeling and machine implementation of various cognitive functions with an increasing number of applications in computer vision, text processing, robotics, neurotechnology, bio-inspired computing and others. In this chapter, we describe how AI methods can be applied in the context of intracranial electroencephalography (iEEG) research. IEEG data is unique as it provides extremely high-quality signals recorded directly from brain tissue. Applying advanced AI models to these data carries the potential to further our understanding of many fundamental questions in neuroscience. At the same time, as an invasive technique, iEEG lends itself well to long-term, mobile brain-computer interface applications, particularly for communication in severely paralyzed individuals. We provide a detailed overview of these two research directions in the application of AI techniques to iEEG. That is, (1) the development of computational models that target fundamental questions about the neurobiological nature of cognition (AI-iEEG for neuroscience) and (2) applied research on monitoring and identification of event-driven brain states for the development of clinical brain-computer interface systems (AI-iEEG for neurotechnology). We explain key machine learning concepts, specifics of processing and modeling iEEG data and details of state-of-the-art iEEG-based neurotechnology and brain-computer interfaces.
... Semi invasive BCIs are different from the invasive ones in that the sensing implants are placed inside the skull but outside the brain. Semi invasive BCIs record the electrocorticography (ECoG) signals from beneath the skull and are used to decode movements, vision, and speech (Miller et al., 2020). Compared to the electroencephalography (EEG) signals, that are recorded from outside the brain in non-invasive BCIs, ECoG signals exhibit higher spatial resolution, better signal-to-noise ratio, wider frequency range, and less training requirements. ...
Affordable, accessible biosensors for the diagnosis, management, and treatment of chronic diseases are critical to reducing healthcare costs. Among all electronic instruments or chemical compounds that are capable of sensing biological signals, those that are implanted in vivo pose a significant advantage. Implantable sensors can directly monitor biological metabolites, electrically stimulate and detect nerve activities, restore body functions, and be used for drug delivery. This chapter provides a comprehensive overview of the ongoing efforts to develop different modalities of implantable sensors and also covers the necessary technical considerations for developing functional and stable chemical compounds that are capable of measuring the concentration of different bioanalytes of interest.
... Similar to intracortical micro arrays are the use of invasive electrodes on the surface of the brain via electrocorticography (ECoG). This method has enabled long-term and stable signal acquisition [44] with higher spatial resolution and signal-to-noise ratio [45]. Such applications have successfully controlled cursors [11,46], computers effectors, providing the end-user the flexibility to choose the end effector to accomplish specific motor tasks for daily needs. ...
Objective
The objective of this study was to develop a portable and modular brain–computer interface (BCI) software platform independent of input and output devices. We implemented this platform in a case study of a subject with cervical spinal cord injury (C5 ASIA A).
Background
BCIs can restore independence for individuals with paralysis by using brain signals to control prosthetics or trigger functional electrical stimulation. Though several studies have successfully implemented this technology in the laboratory and the home, portability, device configuration, and caregiver setup remain challenges that limit deployment to the home environment. Portability is essential for transitioning BCI from the laboratory to the home.
Methods
The BCI platform implementation consisted of an Activa PC + S generator with two subdural four-contact electrodes implanted over the dominant left hand-arm region of the sensorimotor cortex, a minicomputer fixed to the back of the subject’s wheelchair, a custom mobile phone application, and a mechanical glove as the end effector. To quantify the performance for this at-home implementation of the BCI, we quantified system setup time at home, chronic (14-month) decoding accuracy, hardware and software profiling, and Bluetooth communication latency between the App and the minicomputer. We created a dataset of motor-imagery labeled signals to train a binary motor imagery classifier on a remote computer for online, at-home use.
Results
Average bluetooth data transmission delay between the minicomputer and mobile App was 23 ± 0.014 ms. The average setup time for the subject’s caregiver was 5.6 ± 0.83 min. The average times to acquire and decode neural signals and to send those decoded signals to the end-effector were respectively 404.1 ms and 1.02 ms. The 14-month median accuracy of the trained motor imagery classifier was 87.5 ± 4.71% without retraining.
Conclusions
The study presents the feasibility of an at-home BCI system that subjects can seamlessly operate using a friendly mobile user interface, which does not require daily calibration nor the presence of a technical person for at-home setup. The study also describes the portability of the BCI system and the ability to plug-and-play multiple end effectors, providing the end-user the flexibility to choose the end effector to accomplish specific motor tasks for daily needs.
Trial registration ClinicalTrials.gov: NCT02564419. First posted on 9/30/2015
... Patients with motor deficits are often willing to undergo invasive surgery to restore function with a BCI. Since ECoG has not yet been able to fully restore movement (Miller et al., 2020) and since sEEG uses minor 2.4 mm bolt holes, which are less invasive than the craniotomy required for ECoG, improving BCI control by adding sEEG to sample different motor network nodes is an option that should be considered. Lastly, our results are proof-of-concept that intraoperative functional brain mapping could be more efficient with machine learning. ...
Previous electrophysiological research has characterized canonical oscillatory patterns associated with movement mostly from recordings of primary sensorimotor cortex. Less work has attempted to decode movement based on electrophysiological recordings from a broader array of brain areas such as those sampled by stereoelectroencephalography (sEEG). Here we decoded movement using a linear support vector machine (SVM). We were able to accurately classify sEEG spectrograms during a keypress movement in a task versus those during the inter-trial interval. Furthermore, the important time-frequency patterns for this classification recapitulated findings from previous studies that used non-invasive electroencephalography (EEG) and electrocorticography (ECoG) and identified brain regions that were not associated with movement in previous studies. Specifically, we found these previously described patterns: beta (13 - 30 Hz) desynchronization, beta synchronization (rebound), pre-movement alpha (8 - 15 Hz) modulation, a post-movement broadband gamma (60 - 90 Hz) increase and an event-related potential. These oscillatory patterns were newly observed in a wide range of brain areas accessible with sEEG that are not accessible with other electrophysiology recording methods. For example, the presence of beta desynchronization in the frontal lobe was more widespread than previously described, extending outside primary and secondary motor cortices. We provide evidence for a system of putative motor networks that exhibit unique oscillatory patterns by describing the anatomical extent of the movement-related oscillations that were observed most frequently across all sEEG contacts.
Significance Statement
Several major motor networks have been previously delineated in humans, however, much less is known about the population-level oscillations that coordinate this neural circuitry, especially in cortex. Therapies that modulate brain circuits to treat movement disorders, such as deep brain stimulation (DBS), or use brain signals to control movement, such as brain-computer interfaces (BCIs), rely on our basic scientific understanding of this movement neural circuitry. In order to bridge this gap, we used stereoelectroencephalography (sEEG) collected in human patients being monitored for epilepsy to assess oscillatory patterns during movement.
Imagined speech, also known as inner, covert, or silent speech, means how to express thoughts silently without moving the vocal apparatus. Imagined speech reconstruction (ISR) refers to the innovative process of decoding and reconstructing the imagined speech in the human brain, using kinds of neural signals and advanced signal processing techniques. The ISR has become a popular research topic in the field of brain-computer interface (BCI), offering a promising solution to improve the quality of life for individuals with disabilities. Some innovative works have been published in recent years, however, there is a lack of review to give general readers a clear picture of ISR advances. To this end, our paper comprehensively summarizes the recent published ISR-related works. The data acquisition and preprocessing methods of the involved neural signals are included, and the ISR advances from these neural signals are further summarized. The included concepts, methods, datasets, discussions, are expected to be beneficial for both the general reader and academic researchers in this field.
Objective: Micro-electrocorticographic (μECoG) arrays are able to record neural activities from the cortical surface, without the need to penetrate the brain parenchyma. Owing in part to small electrode sizes, previous studies have demonstrated that single-unit spikes could be detected from the cortical surface, and likely from Layer I neurons of the neocortex. Here we tested the ability to use μECoG array to decode, in rats, body position during open field navigation, through isolated single-unit activities.
Approach: μECoG arrays were chronically implanted onto primary motor cortex (M1) of Wistar rats, and neural recording was performed in awake, behaving rats in an open-field enclosure. The signals were band-pass filtered between 300 to 3000 Hz. Threshold-crossing spikes were identified and sorted into distinct units based on defined criteria including waveform morphology and refractory period. Body positions were derived from video recordings. We used gradient-boosting machine to predict body position based on previous 100 ms of spike data, and correlation analyses to elucidate the relationship between position and spike patterns.
Main results: Single-unit spikes could be extracted during chronic recording from μECoG, and spatial position could be decoded from these spikes with a mean absolute error of prediction of 0.135 and 0.090 in the x- and y- dimensions (of a normalized range from 0 to 1), and Pearson’s r of 0.607 and 0.571, respectively.
Significance: μECoG can detect single-unit activities that likely arise from superficial neurons in the cortex and is a promising alternative to intracortical arrays, with the added benefit of scalability to cover large cortical surface with minimal incremental risks. More studies should be performed in human related to its use as brain-machine interface.
Decoding human speech from neural signals is essential for brain–computer interface (BCI) technologies that aim to restore speech in populations with neurological deficits. However, it remains a highly challenging task, compounded by the scarce availability of neural signals with corresponding speech, data complexity and high dimensionality. Here we present a novel deep learning-based neural speech decoding framework that includes an ECoG decoder that translates electrocorticographic (ECoG) signals from the cortex into interpretable speech parameters and a novel differentiable speech synthesizer that maps speech parameters to spectrograms. We have developed a companion speech-to-speech auto-encoder consisting of a speech encoder and the same speech synthesizer to generate reference speech parameters to facilitate the ECoG decoder training. This framework generates natural-sounding speech and is highly reproducible across a cohort of 48 participants. Our experimental results show that our models can decode speech with high correlation, even when limited to only causal operations, which is necessary for adoption by real-time neural prostheses. Finally, we successfully decode speech in participants with either left or right hemisphere coverage, which could lead to speech prostheses in patients with deficits resulting from left hemisphere damage.
Brain–computer interfaces (BCIs) can revolutionize how humans interact with technology, but several scientific and technological challenges must be addressed to realize their full potential. Recent developments in quantum‐based sensing methods offer promising solutions to some of these challenges. This review provides an overview of the progress, challenges, and prospects of BCIs research and discuss the feasibility of integrating quantum sensor technology in BCI systems. The applications of quantum sensing in BCIs research are reviewed and the solution based on quantum sensor technology to overcome some of the challenges associated with BCI systems is proposed. The potential of quantum sensor technology for the future development of BCIs is emphasized. Overall, this review highlights quantum sensor technology's significant potential for future development of BCI.
Objective:
Electrocorticography (ECoG)-based brain-computer interface (BCI) systems have the potential to improve quality of life of people with locked-in syndrome (LIS) by restoring their ability to communicate independently. Before implantation of such a system, it is important to localize ECoG electrode target regions. Here, we assessed the predictive value of functional magnetic resonance imaging (fMRI) for the localization of suitable target regions on the sensorimotor cortex for ECoG-based BCI in people with locked-in syndrome.
Methods:
Three people with locked-in syndrome were implanted with a chronic, fully implantable ECoG-BCI system. We compared pre-surgical fMRI activity with post-implantation ECoG activity from areas known to be active and inactive during attempted hand movement (sensorimotor hand region and dorsolateral prefrontal cortex, respectively).
Results:
Results showed a spatial match between fMRI activity and changes in ECoG low and high frequency band power (10 - 30 and 65 - 95 Hz, respectively) during attempted movement. Also, we found that fMRI can be used to select a sub-set of electrodes that show strong task-related signal changes that are therefore likely to generate adequate BCI control.
Conclusions:
Our findings indicate that fMRI is a useful non-invasive tool for the pre-surgical workup of BCI implant candidates.
Significance:
If these results are confirmed in more BCI studies, fMRI might be used for more efficient surgical BCI procedures with focused cortical coverage and lower participant burden.
There have been significant advances in biosignal extraction techniques to drive external biomechatronic devices or to use as inputs to sophisticated human machine interfaces. The control signals are typically derived from biological signals such as myoelectric measurements made either from the surface of the skin or subcutaneously. Other biosignal sensing modalities are emerging. With improvements in sensing modalities and control algorithms, it is becoming possible to robustly control the target position of an end-effector. It remains largely unknown to what extent these improvements can lead to naturalistic human-like movement. In this paper, we sought to answer this question. We utilized a sensing paradigm called sonomyography based on continuous ultrasound imaging of forearm muscles. Unlike myoelectric control strategies which measure electrical activation and use the extracted signals to determine the velocity of an end-effector; sonomyography measures muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Previously, we showed that users were able to accurately and precisely perform a virtual target acquisition task using sonomyography. In this work, we investigate the time course of the control trajectories derived from sonomyography. We show that the time course of the sonomyography-derived trajectories that users take to reach virtual targets reflect the trajectories shown to be typical for kinematic characteristics observed in biological limbs. Specifically, during a target acquisition task, the velocity profiles followed a minimum jerk trajectory shown for point-to-point arm reaching movements, with similar time to target. In addition, the trajectories based on ultrasound imaging result in a systematic delay and scaling of peak movement velocity as the movement distance increased. We believe this is the first evaluation of similarities in control policies in coordinated movements in jointed limbs, and those based on position control signals extracted at the individual muscle level. These results have strong implications for the future development of control paradigms for assistive technologies.
Artificial intelligence (AI) is a fast-growing field focused on modeling and machine implementation of various cognitive functions with an increasing number of applications in computer vision, text processing, robotics, neurotechnology, bio-inspired computing and others. In this chapter, we describe how AI methods can be applied in the context of intracranial electroencephalography (iEEG) research. IEEG data is unique as it provides extremely high-quality signals recorded directly from brain tissue. Applying advanced AI models to this data carries the potential to further our understanding of many fundamental questions in neuroscience. At the same time, as an invasive technique, iEEG lends itself well to long-term, mobile brain-computer interface applications, particularly for communication in severely paralyzed individuals. We provide a detailed overview of these two research directions in the application of AI techniques to iEEG. That is, (1) the development of computational models that target fundamental questions about the neurobiological nature of cognition (AI-iEEG for neuroscience) and (2) applied research on monitoring and identification of event-driven brain states for the development of clinical brain-computer interface systems (AI-iEEG for neurotechnology). We explain key machine learning concepts, specifics of processing and modeling iEEG data and details of state-of-the-art iEEG-based neurotechnology and brain-computer interfaces.
Injectable bioprobes record single-neuron activity from within blood vessels.
Highlights
Various 3D printing techniques for neural tissue-engineered scaffolds or living cell-laden constructs are summarized and compared.
Strategies by integrating topographical, biochemical and electroactive cues inside 3D-printed neural constructs for functional neural regeneration were introduced.
The typical applications of 3D-printed bioengineered constructs are demonstrated.
The challenges and future outlook associated with 3D printing for functional neural constructs in various categories are discussed.
As scientists discovered that raw neurological signals could translate into bioelectric information, brain–machine interfaces (BMI) for experimental and clinical studies have experienced massive growth. Developing suitable materials for bioelectronic devices to be used for real‐time recording and data digitalizing has three important necessitates which should be covered. Biocompatibility, electrical conductivity, and having mechanical properties similar to soft brain tissue to decrease mechanical mismatch should be adopted for all materials. In this review, inorganic nanoparticles and intrinsically conducting polymers are discussed to impart electrical conductivity to systems, where soft materials such as hydrogels can offer reliable mechanical properties and a biocompatible substrate. Interpenetrating hydrogel networks offer more mechanical stability and provide a path for incorporating polymers with desired properties into one strong network. Promising fabrication methods, like electrospinning and additive manufacturing, allow scientists to customize designs for each application and reach the maximum potential for the system. In the near future, it is desired to fabricate biohybrid conducting polymer‐based interfaces loaded with cells, giving the opportunity for simultaneous stimulation and regeneration. Developing multi‐modal BMIs, Using artificial intelligence and machine learning to design advanced materials are among the future goals for this field.
This article is categorized under: Therapeutic Approaches and Drug Discovery > Nanomedicine for Neurological Disease
Brain–Computer Interface (BCI) technology is a promising research area in many domains. Brain activity can be interpreted through both invasive and noninvasive monitoring devices, allowing for novel, therapeutic solutions for individuals with disabilities and for other non-medical applications. However, a number of ethical issues have been identified from the use of BCI technology. In previous work published in 2020, we reviewed the academic discussion of the ethical implications of BCI technology in the previous 5 years by using a limited sample to identify trends and areas of concern or debate among researchers and ethicists. In this chapter, we provide an overview on the academic discussion of BCI ethics and report on the findings for the next phase of this work, which systematically categorizes the entire sample. The aim of this work is to collect and synthesize all the pertinent academic scholarship into the ethical, legal, and social implications (ELSI) of BCI technology. We hope this study will provide a foundation for future scholars, ethicists, and policy makers to understand the landscape of the relevant ELSI concepts and pave the way for assessing the need for regulatory action. We conclude that some emerging applications of BCI technology—including commercial ventures that seek to meld human intelligence with AI—present new and unique ethical concerns.KeywordsBrain–computer interface (BCI)Brain–machine interface (BMI)Ethical, legal, and social issues (ELSI)NeuroethicsScoping review
The idea of creating a direct link between the brain and a computer, once exotic and for many people inconceivable, is steadily becoming mainstream. Building on decade-long academic research and fueled by successful demonstrations of brain-controlled devices, tech investors are now increasingly engaged in developing and patenting such technology. While early conceptions focused on assistive applications, e.g., to control a prosthesis or restore communication, new paradigms are implemented to enhance brain function to or above the norm. Moreover, so-called bidirectional brain/neural-computer interfaces could restore sensory capacities or suppress pathological brain activity in neuropsychiatric disorders. Merging this technology with artificial intelligence (AI) promises to increase applicability in various medical and non-medical applications. The rapid advancements of such AI-enhanced brain–computer interfaces (BCIs) beyond the medical field raise many concerns, including dual-use, cybersecurity, and brain-hacking. This chapter will provide an overview of the most innovative trends in current brain/neural–interface technology and elaborate on the associated technical and conceptual challenges ahead. While it is very difficult to anticipate which applications will evolve, we will outline BCI technology’s most likely course of evolution and its implications for society.KeywordsBrain–computer interfaceArtificial intelligenceCybersecurityAdaptive neuromodulationAssistive technologyNeurorestorationHybridmind
Brain-computer interface (BCI) is the technology interfacing subjects directly by their brain signals to the computer or external environment. BCI provides alternative communication channels for locked-in patients, who can sense but cannot respond due to the complete paralysis of all voluntary muscles except the control of the eye movements. BCI system measures the brain activities reflecting the intent of subjects and converts them into artificial output to interact with the environment. BCI using local field potential and electrocorticogram produce higher performance, but their invasiveness limits their applications. Noninvasive BCIs using EEG are most actively pursued. Slow cortical potential (SCP), event-related potentials (ERP) like P300, steady-state visual evoked potential (SSVEP), and sensory-motor rhythms (SMR) are most actively used in BCI systems. BCI systems using different types of EEG signals have their own advantages and disadvantages. In a BCI using SSVEP, the highest information transfer rate (ITR) of 325.3 bit/min has been reported with a cue-guided task using a 40-characters speller.KeywordsAsynchronous BCIBrain-computer interface (BCI)BCI spellerEvent-related desynchronization (ERD)Hybrid BCIInformation transfer rate (ITR)Invasive BCIsLocked-in syndrome (LIS)Motor imageryP300Sensory-motor rhythmsSlow cortical potentialSteady-state visual evoked potential (SSVEP)
In recent years, electrocorticography (ECoG) has arisen as a neural signal recording tool in the development of clinically viable neural interfaces. ECoG electrodes are generally placed below the dura mater (subdural) but can also be placed on top of the dura (epidural). In deciding which of these modalities best suits long-term implants, complications and signal quality are important considerations. Conceptually, epidural placement may present a lower risk of complications as the dura is left intact, but also a lower signal quality due to the dura acting as a signal attenuator. The extent to which complications and signal quality are affected by the dura, however, has been a matter of debate. To improve our understanding of the effects of the dura on complications and signal quality, we conducted a literature review. We inventorized the effect of the dura on signal quality, decodability and longevity of acute and chronic ECoG recordings in humans and non-human primates. Also, we compared the incidence and nature of serious complications in studies that employed epidural and subdural ECoG. Overall, we found that, even though epidural recordings exhibit attenuated signal amplitude over subdural recordings, particularly for high-density grids, the decodability of epidural recorded signals does not seem to be markedly affected. Additionally, we found that the nature of serious complications was comparable between epidural and subdural recordings. These results indicate that both epidural and subdural ECoG may be suited for long-term neural signal recordings, at least for current generations of clinical and high-density ECoG grids.
Background
Subdural electrocorticography (ECoG) signals have been proposed as a stable, good-quality source for brain-machine interfaces (BMIs), with a higher spatial and temporal resolution than electroencephalogram (EEG). However, long-term implantation may lead to chronic inflammatory reactions and connective tissue encapsulation, resulting in a decline in the signal recording quality. However, no study has reported the effects of the surrounding tissue on signal recording and device functionality thus far.
Methods
In this study, we implanted a wireless recording device with a customized 32-electrode-ECoG array subdurally in two nonhuman primates for 15 months. We evaluated the neural activities recorded and wirelessly transmitted to the devices and the chronic tissue reactions around the electrodes.
Results
Time-frequency analyses of the acute and chronic phases showed similar signal features. The average root mean square voltage and power spectral density remained relatively stable after chronic implantation. Histological examination revealed thickening of the reactive tissue around the electrode array; however, no evident inflammation in the cortex. In addition, we measured the gain factor of the newly formed ventral fibrous tissue in vivo.
Conclusions
This study suggests that subdural ECoG may provide stable chronic signal recordings for future clinical applications and neuroscience research and highlights the role of reducing the thickness of ventral tissue proliferation.
A neurochip comprises a small device based on the brain–machine interfaces (BMI) that emulates the functioning synapses. Its implant in the human body allows the interaction of the brain with a computer. Although the data-processing speed is still slower than that of the human brain, they are being developed. There is no ethical conflict as long as it is used for neural rehabilitation or to supply impaired or missing neurological functions. However, other applications emerge as controversial. Deliberation on neurochips is primarily limited to a small circle of scholars such as neurotechnological engineers, artists, philosophers, and bioethicists. Why do we address neurosurgeons? They will be directly involved as they could be required to perform invasive procedures. Future neurosurgeons will have to be a different type of neurosurgeon. They will be part of interdisciplinary teams interacting with computer engineers, neurobiologist, and ethicists. Although a neurosurgeon is not expected to be an expert in all areas, they have to be familiar with them; they have to be prepared to determine indications, contraindications, and risks of the procedures, participating in the decision-making processes, and even collaborating in the design of devices in order to preserve anatomic structures. Social, economic, and legal aspects are also inherent to the neurosurgical activity; therefore, these aspects should also be considered. The neurosurgical societies, and the directors of training programs, should start to prepare young doctors to anticipate these kind of neuroethical issues. Perhaps, the neurosurgical community, even in collaboration with the WHO (World Health Organization) and PAHO (Pan American Health Organization), should anticipate worldwide ethical recommendations.
Objective:
Recent advances in neural decoding have accelerated the development of brain-computer interfaces aimed at assisting users with everyday tasks such as speaking, walking, and manipulating objects. However, current approaches for training neural decoders commonly require large quantities of labeled data, which can be laborious or infeasible to obtain in real-world settings. Alternatively, self-supervised models that share self-generated pseudo-labels between two data streams have shown exceptional performance on unlabeled audio and video data, but it remains unclear how well they extend to neural decoding.
Approach:
We learn neural decoders without labels by leveraging multiple simultaneously recorded data streams, including neural, kinematic, and physiological signals. Specifically, we apply cross-modal, self-supervised deep clustering to train decoders that can classify movements from brain recordings. After training, we then isolate the decoders for each input data stream and compare the accuracy of decoders trained using cross-modal deep clustering against supervised and unimodal, self-supervised models.
Main results:
We find that sharing pseudo-labels between two data streams during training substantially increases decoding performance compared to unimodal, self-supervised models, with accuracies approaching those of supervised decoders trained on labeled data. Next, we extend cross-modal decoder training to three or more modalities, achieving state-of-the-art neural decoding accuracy that matches or slightly exceeds the performance of supervised models. Significance: We demonstrate that cross-modal, self-supervised decoding can be applied to train neural decoders when few or no labels are available and extend the cross-modal framework to share information among three or more data streams, further improving self-supervised training.
Investigators in neuroscience have brought the machinations of the nervous system into a clearer view, especially with the initiative called BRAIN (Brain Research through Advancing Innovative Neurotechnologies) led by the president of the United States, Barack Obama. While this initiative focused on the understanding of how the brain and nervous system work, it also allowed great leaps in technology with which to investigate the system. Not only could this technology help understand and unravel the circuitry of the brain, but it also allowed many to imagine various therapeutics which could help individuals suffering from neurological disorders or trauma.
The sensorimotor cortex is a frequently targeted brain area for the development of Brain-Computer Interfaces (BCIs) for communication in people with severe paralysis and communication problems (locked-in syndrome; LIS). It is widely acknowledged that this area displays an increase in high-frequency band (HFB) power and a decrease in the power of the low frequency band (LFB) during movement of, for example, the hand. Upon termination of hand movement, activity in the LFB band typically shows a short increase (rebound). The ability to modulate the neural signal in the sensorimotor cortex by imagining or attempting to move is crucial for the implementation of sensorimotor BCI in people who are unable to execute movements. This may not always be self-evident, since the most common causes of LIS, amyotrophic lateral sclerosis (ALS) and brain stem stroke, are associated with significant damage to the brain, potentially affecting the generation of baseline neural activity in the sensorimotor cortex and the modulation thereof by imagined or attempted hand movement. In the Utrecht NeuroProsthesis (UNP) study, a participant with LIS caused by ALS and a participant with LIS due to brain stem stroke were implanted with a fully implantable BCI, including subdural electrocorticography (ECoG) electrodes over the sensorimotor area, with the purpose of achieving ECoG-BCI-based communication. We noted differences between these participants in the spectral power changes generated by attempted movement of the hand. To better understand the nature and origin of these differences, we compared the baseline spectral features and task-induced modulation of the neural signal of the LIS participants, with those of a group of able-bodied people with epilepsy who received a subchronic implant with ECoG electrodes for diagnostic purposes. Our data show that baseline LFB oscillatory components and changes generated in the LFB power of the sensorimotor cortex by (attempted) hand movement differ between participants, despite consistent HFB responses in this area. We conclude that the etiology of LIS may have significant effects on the LFB spectral components in the sensorimotor cortex, which is relevant for the development of communication-BCIs for this population.
Electrophysiological data from implanted electrodes in the human brain are rare, and therefore scientific access to such data has remained somewhat exclusive. Here we present a freely available curated library of implanted electrocorticographic data and analyses for 16 behavioural experiments, with 204 individual datasets from 34 patients recorded with the same amplifiers and at the same settings. For each dataset, electrode positions were carefully registered to brain anatomy. A large set of fully annotated analysis scripts with which to interpret these data is embedded in the library alongside them. All data, anatomical locations and analysis files (MATLAB code) are provided in a shared file structure at https://searchworks.stanford.edu/view/zk881ps0522.
This article deals with the long-term preclinical validation of WIMAGINE® (Wireless Implantable Multi-channel Acquisition system for Generic Interface with Neurons), a 64-channel wireless implantable recorder that measures the electrical activity at the cortical surface (electrocorticography, ECoG). The WIMAGINE® implant was designed for chronic wireless neuronal signal acquisition, to be used e.g., as an intracranial Brain–Computer Interface (BCI) for severely motor-impaired patients. Due to the size and shape of WIMAGINE®, sheep appeared to be the best animal model on which to carry out long-term in vivo validation. The devices were implanted in two sheep for a follow-up period of 10 months, including idle state cortical recordings and Somato-Sensory Evoked Potential (SSEP) sessions. ECoG and SSEP demonstrated relatively stable behavior during the 10-month observation period. Information recorded from the SensoriMotor Cortex (SMC) showed an SSEP phase reversal, indicating the cortical site of the sensorimotor activity was retained after 10 months of contact. Based on weekly recordings of raw ECoG signals, the effective bandwidth was in the range of 230 Hz for both animals and remarkably stable over time, meaning preservation of the high frequency bands valuable for decoding of the brain activity using BCIs. The power spectral density (in dB/Hz), on a log scale, was of the order of 2.2, –4.5 and –18 for the frequency bands (10–40), (40–100), and (100–200) Hz, respectively. The outcome of this preclinical work is the first long-term in vivo validation of the WIMAGINE® implant, highlighting its ability to record the brain electrical activity through the dura mater and to send wireless digitized data to the external base station. Apart from local adhesion of the dura to the skull, the neurosurgeon did not face any difficulty in the implantation of the WIMAGINE® device and post-mortem analysis of the brain revealed no side effect related to the implantation. We also report on the reliability of the system; including the implantable device, the antennas module and the external base station.
Objective:
We investigated the long-term functional stability and home use of a fully implanted electrocorticography (ECoG)-based brain-computer interface (BCI) for communication by an individual with late-stage Amyotrophic Lateral Sclerosis (ALS).
Methods:
Data recorded from the cortical surface of the motor and prefrontal cortex with an implanted brain-computer interface device was evaluated for 36 months after implantation of the system in an individual with late-stage ALS. In addition, electrode impedance and BCI control accuracy were assessed. Key measures included frequency of use of the system for communication, user and system performance, and electrical signal characteristics.
Results:
User performance was high consistently over the three years. Power in the high frequency band, used for the control signal, declined slowly in the motor cortex, but control over the signal remained unaffected by time. Impedance increased until month 5, and then remained constant. Frequency of home use increased steadily, indicating adoption of the system by the user.
Conclusions:
The implanted brain-computer interface proves to be robust in an individual with late-stage ALS, given stable performance and control signal for over 36 months.
Significance:
These findings are relevant for the future of implantable brain-computer interfaces along with other brain-sensing technologies, such as responsive neurostimulation.
The development of implantable neuroelectrodes is advancing rapidly as these tools are becoming increasingly ubiquitous in clinical practice, especially for the treatment of traumatic and neurodegenerative disorders. Electrodes have been exploited in a wide number of neural interface devices, such as deep brain stimulation, which is one of the most successful therapies with proven efficacy in the treatment of diseases like Parkinson or epilepsy. However, one of the main caveats related to the clinical application of electrodes is the nervous tissue response at the injury site, characterized by a cascade of inflammatory events, which culminate in chronic inflammation, and, in turn, result in the failure of the implant over extended periods of time. To overcome current limitations of the most widespread macroelectrode based systems, new design strategies and the development of innovative materials with superior biocompatibility characteristics are currently being investigated. This review describes the current state of the art of in vitro, ex vivo, and in vivo models available for the study of neural tissue response to implantable microelectrodes. We particularly highlight new models with increased complexity that closely mimic in vivo scenarios and that can serve as promising alternatives to animal studies for investigation of microelectrodes in neural tissues. Additionally, we also express our view on the impact of the progress in the field of neural tissue engineering on neural implant research.
Technology that translates neural activity into speech would be transformative for people who are unable to communicate as a result of neurological impairments. Decoding speech from neural activity is challenging because speaking requires very precise and rapid multi-dimensional control of vocal tract articulators. Here we designed a neural decoder that explicitly leverages kinematic and sound representations encoded in human cortical activity to synthesize audible speech. Recurrent neural networks first decoded directly recorded cortical activity into representations of articulatory movement, and then transformed these representations into speech acoustics. In closed vocabulary tests, listeners could readily identify and transcribe speech synthesized from cortical activity. Intermediate articulatory dynamics enhanced performance even with limited data. Decoded articulatory representations were highly conserved across speakers, enabling a component of the decoder to be transferrable across participants. Furthermore, the decoder could synthesize speech when a participant silently mimed sentences. These findings advance the clinical viability of using speech neuroprosthetic technology to restore spoken communication.
Background
Lifetime stroke risk has been calculated in a limited number of selected populations. We determined lifetime risk of stroke globally and at the regional and country level.
Methods
Using Global Burden of Disease Study estimates of stroke incidence and the competing risks of non-stroke mortality, we estimated the cumulative lifetime risk of ischemic stroke, hemorrhagic stroke, and total stroke (with 95% uncertainty intervals [UI]) for 195 countries among adults over 25 years) for the years 1990 and 2016 and according to the GBD Study Socio-Demographic Index (SDI).
Results
The global estimated lifetime risk of stroke from age 25 onward was 24.9% (95% UI: 23.5–26.2): 24.7% (23.3–26.0) in men and 25.1% (23.7–26.5) in women. The lifetime risk of ischemic stroke was 18.3% and of hemorrhagic stroke was 8.2%. The risk of stroke was 23.5% in high SDI countries, 31.1% in high-middle SDI countries, and 13.2% in low SDI countries with UIs not overlapping for these categories. The greatest estimated risk of stroke was in East Asia (38.8%) and Central and Eastern Europe (31.7 and 31.6 %%), and lowest in Eastern Sub-Saharan Africa (11.8%). From 1990 to 2016, there was a relative increase of 8.9% in global lifetime risk.
Conclusions
The global lifetime risk of stroke is approximately 25% starting at age 25 in both men and women. There is geographical variation in the lifetime risk of stroke, with particularly high risk in East Asia, Central and Eastern Europe.
Brain-Computer Interfaces Handbook: Technological and Theoretical Advances. CRC Press, Taylor & Francis Group, Oxford, UK
Brain‒machine interface (BMI) is a promising technology that looks set to contribute to the development of artificial limbs and new input devices by integrating various recent technological advances, including neural electrodes, wireless communication, signal analysis, and robot control. Neural electrodes are a key technological component of BMI, as they can record the rapid and numerous signals emitted by neurons. To receive stable, consistent, and accurate signals, electrodes are designed in accordance with various templates using diverse materials. With the development of microelectromechanical systems (MEMS) technology, electrodes have become more integrated, and their performance has gradually evolved through surface modification and advances in biotechnology. In this paper, we review the development of the extracellular/intracellular type of in vitro microelectrode array (MEA) to investigate neural interface technology and the penetrating/surface (non-penetrating) type of in vivo electrodes. We briefly examine the history and study the recently developed shapes and various uses of the electrode. Also, electrode materials and surface modification techniques are reviewed to measure high-quality neural signals that can be used in BMI.
Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease affecting motor neurons and other neuronal cells, leading to severe disability and eventually death from ventilatory failure. It has a prevalence of 5 in 100,000, with an incidence of 1.7 per 100,000, reflecting short average survival. The pathogenesis is incompletely understood, but defects of RNA processing and protein clearance may be fundamental. Repeat expansions in the chromosome 9 open reading frame 72 gene (C9orf72) are the most common known genetic cause of ALS and are seen in approximately 40% of patients with a family history and approximately 10% of those without. No environmental risk factors are proved to be causative, but many have been proposed, including military service. The diagnosis of ALS rests on a history of painless progressive weakness coupled with examination findings of upper and lower motor dysfunction. No diagnostic test is yet available, but electromyography and genetic tests can support the diagnosis. Care for patients is best provided by a multidisciplinary team, and most interventions are directed at managing symptoms. Two medications with modest benefits have Food and Drug Administration approval for the treatment of ALS: riluzole, a glutamate receptor antagonist, and, new in 2017, edaravone, a free radical scavenger. Many other encouraging treatment strategies are being explored in clinical trials for ALS; herein we review stem cell and antisense oligonucleotide gene therapies.
Brain-computer interfaces (BCIs) have the potential to restore communication for people with tetraplegia and anarthria by translating neural activity into control signals for assistive communication devices. While previous pre-clinical and clinical studies have demonstrated promising proofs-of-concept (Serruya et al., 2002; Simeral et al., 2011; Bacher et al., 2015; Nuyujukian et al., 2015; Aflalo et al., 2015; Gilja et al., 2015; Jarosiewicz et al., 2015; Wolpaw et al., 1998; Hwang et al., 2012; Spüler et al., 2012; Leuthardt et al., 2004; Taylor et al., 2002; Schalk et al., 2008; Moran, 2010; Brunner et al., 2011; Wang et al., 2013; Townsend and Platsko, 2016; Vansteensel et al., 2016; Nuyujukian et al., 2016; Carmena et al., 2003; Musallam et al., 2004; Santhanam et al., 2006; Hochberg et al., 2006; Ganguly et al., 2011; O’Doherty et al., 2011; Gilja et al., 2012), the performance of human clinical BCI systems is not yet high enough to support widespread adoption by people with physical limitations of speech. Here we report a high-performance intracortical BCI (iBCI) for communication, which was tested by three clinical trial participants with paralysis. The system leveraged advances in decoder design developed in prior pre-clinical and clinical studies (Gilja et al., 2015; Kao et al., 2016; Gilja et al., 2012). For all three participants, performance exceeded previous iBCIs (Bacher et al., 2015; Jarosiewicz et al., 2015) as measured by typing rate (by a factor of 1.4–4.2) and information throughput (by a factor of 2.2–4.0). This high level of performance demonstrates the potential utility of iBCIs as powerful assistive communication devices for people with limited motor function.
Clinical Trial No: NCT00912041
Intracortical microelectrodes have shown great success in enabling locked-in patients to interact with computers, robotic limbs, and their own electrically driven limbs. The recent advances have inspired world-wide enthusiasm resulting in billions of dollars invested in federal and industrial sponsorships to understanding the brain for rehabilitative applications. Additionally, private philanthropists have also demonstrated excitement in the field by investing in the use of brain interfacing technologies as a means to human augmentation. While the promise of incredible technologies is real, caution must be taken as implications regarding optimal performance and unforeseen side effects following device implantation into the brain are not fully characterized. The current study is aimed to quantify any motor deficit caused by microelectrode implantation in the motor cortex of healthy rats compared to non-implanted controls. Following electrode insertion, rats were tested on an open-field grid test to study gross motor function and a ladder test to study fine motor function. It was discovered that rats with chronically indwelling intracortical microelectrodes exhibited up to an incredible 527% increase in time to complete the fine motor task. This initial study defines the need for further and more robust behavioral testing of potential unintentional harm caused by microelectrode implantation.
Objective:
Innovations in micro-electrocorticography (µECoG) electrode array manufacturing now allow for intricate designs with smaller contact diameters and/or pitch (i.e., center-to-center contact distance) down to the sub-mm range. The aims of the present study were: (i) to investigate whether frequency ranges up to 400 Hz can be reproducibly observed in µECoG recordings and (ii) to examine how differences in topographical substructure between these frequency bands and electrode array geometries can be quantified. We also investigated, for the first time, the influence of blood vessels on signal properties and assessed the influence of cortical vasculature on topographic mapping.
Approach:
The present study employed two μECoG electrode arrays with different contact diameters and inter-contact distances, which were used to characterize neural activity from the somatosensory cortex of minipigs in a broad frequency range up to 400 Hz. The analyzed neural data were recorded in acute experiments under anesthesia during peripheral electrical stimulation.
Main results:
We observed that µECoG recordings reliably revealed multi-focal cortical somatosensory response patterns, in which response peaks were often less than 1 cm apart and would thus not have been resolvable with conventional ECoG. The response patterns differed by stimulation site and intensity, they were distinct for different frequency bands, and the results of functional mapping proved independent of cortical vascular. Our analysis of different frequency bands exhibited differences in the number of activation peaks in topographical substructures. Notably, signal strength and signal-to-noise ratios differed between the two electrode arrays, possibly due to their different sensitivity for variations in spatial patterns and signal strengths.
Significance:
Our findings that the geometry of µECoG electrode arrays can strongly influence their recording performance can help make informed decisions that maybe important in number of clinical contexts, including high-resolution brain mapping, advanced epilepsy diagnostics or brain-machine interfacing.
Options for people with severe paralysis who have lost the ability to communicate
orally are limited. We describe a method for communication in a patient with latestage
amyotrophic lateral sclerosis (ALS), involving a fully implanted brain–computer
interface that consists of subdural electrodes placed over the motor cortex
and a transmitter placed subcutaneously in the left side of the thorax. By attempting
to move the hand on the side opposite the implanted electrodes, the patient
accurately and independently controlled a computer typing program 28 weeks after
electrode placement, at the equivalent of two letters per minute. The brain–computer
interface offered autonomous communication that supplemented and at times
supplanted the patient’s eye-tracking device. (Funded by the Government of the
Netherlands and the European Union; ClinicalTrials.gov number, NCT02224469.)
The link between object perception and neural activity in visual cortical areas is a problem of fundamental importance in neuroscience. Here we show that electrical potentials from the ventral temporal cortical surface in humans contain sufficient information for spontaneous and near-instantaneous identification of a subject's perceptual state. Electrocorticographic (ECoG) arrays were placed on the subtemporal cortical surface of seven epilepsy patients. Grayscale images of faces and houses were displayed rapidly in random sequence. We developed a template projection approach to decode the continuous ECoG data stream spontaneously, predicting the occurrence, timing and type of visual stimulus. In this setting, we evaluated the independent and joint use of two well-studied features of brain signals, broadband changes in the frequency power spectrum of the potential and deflections in the raw potential trace (event-related potential; ERP). Our ability to predict both the timing of stimulus onset and the type of image was best when we used a combination of both the broadband response and ERP, suggesting that they capture different and complementary aspects of the subject's perceptual state. Specifically, we were able to predict the timing and type of 96% of all stimuli, with less than 5% false positive rate and a ~20ms error in timing.
Background:
Improved knowledge of the quality of life (QoL) of locked-in syndrome (LIS) patients have implications for managing their care, and assists clinicians in choosing the most appropriate interventions. We performed a survey of a population of LIS patients to describe the course of the QoL of LIS patients over a 6-year period and to determine the potential predictive factors of QoL changes over time.
Method:
This is a study performed over a 6-year period in patients with a LIS diagnosis. Questionnaires were sent in 2007 and 2013. The following data were recorded: i) sociodemographic data; ii) clinical data related to LIS, physical/handicap status, psychological status; iii) self-reported QoL: Anamnestic Comparative Self-Assessment (ACSA); iv) Integration in life: French Reintegration to Normal Living Index (RNLI).
Results:
Among the 67 patients included in 2007, 39 (58 %) patients returned their questionnaire in 2013. The LIS etiology was stroke in 51 individuals. The QoL of the patients was relatively satisfactory compared to populations in other severe conditions. Twenty-one (70 %) individuals reported a stable/improved QoL between 2007 and 2013. The physical/handicap statuses in 2007 and 2013 were not related to the QoL 6 years later, with the exception of one communication parameter: the individuals who used yes-no code reported significantly lower QoL levels than those who did not in 2013.
Discussion:
In opposition to a widespread opinion, LIS persons report a relatively satisfactory QoL level that stays stable over time, suggesting that life with LIS is worth living. Preservation of autonomy and communication may help them to live as normal life as possible.
Speaking is one of the most complex actions that we perform, but nearly all of us learn to do it effortlessly. Production of fluent speech requires the precise, coordinated movement of multiple articulators (for example, the lips, jaw, tongue and larynx) over rapid time scales. Here we used high-resolution, multi-electrode cortical recordings during the production of consonant-vowel syllables to determine the organization of speech sensorimotor cortex in humans. We found speech-articulator representations that are arranged somatotopically on ventral pre- and post-central gyri, and that partially overlap at individual electrodes. These representations were coordinated temporally as sequences during syllable production. Spatial patterns of cortical activity showed an emergent, population-level representation, which was organized by phonetic features. Over tens of milliseconds, the spatial patterns transitioned between distinct representations for different consonants and vowels. These results reveal the dynamic organization of speech sensorimotor cortex during the generation of multi-articulator movements that underlies our ability to speak.
The functional significance of electrical rhythms in the mammalian brain remains uncertain. In the motor cortex, the 12-20 Hz beta rhythm is known to transiently decrease in amplitude during movement, and to be altered in many motor diseases. Here we show that the activity of neuronal populations is phase-coupled with the beta rhythm on rapid timescales, and describe how the strength of this relation changes with movement. To investigate the relationship of the beta rhythm to neuronal dynamics, we measured local cortical activity using arrays of subdural electrocorticographic (ECoG) electrodes in human patients performing simple movement tasks. In addition to rhythmic brain processes, ECoG potentials also reveal a spectrally broadband motif that reflects the aggregate neural population activity beneath each electrode. During movement, the amplitude of this broadband motif follows the dynamics of individual fingers, with somatotopically specific responses for different fingers at different sites on the pre-central gyrus. The 12-20 Hz beta rhythm, in contrast, is widespread as well as spatially coherent within sulcal boundaries and decreases in amplitude across the pre- and post-central gyri in a diffuse manner that is not finger-specific. We find that the amplitude of this broadband motif is entrained on the phase of the beta rhythm, as well as rhythms at other frequencies, in peri-central cortex during fixation. During finger movement, the beta phase-entrainment is diminished or eliminated. We suggest that the beta rhythm may be more than a resting rhythm, and that this entrainment may reflect a suppressive mechanism for actively gating motor function.
Many laboratories have begun to develop brain-computer interface (BCI) systems that provide communication and control capabilities to people with severe motor disabilities. Further progress and realization of practical applications depends on systematic evaluations and comparisons of different brain signals, recording methods, processing algorithms, output formats, and operating protocols. However, the typical BCI system is designed specifically for one particular BCI method and is, therefore, not suited to the systematic studies that are essential for continued progress. In response to this problem, we have developed a documented general-purpose BCI research and development platform called BCI2000. BCI2000 can incorporate alone or in combination any brain signals, signal processing methods, output devices, and operating protocols. This report is intended to describe to investigators, biomedical engineers, and computer scientists the concepts that the BCI2000 system is based upon and gives examples of successful BCI implementations using this system. To date, we have used BCI2000 to create BCI systems for a variety of brain signals, processing methods, and applications. The data show that these systems function well in online operation and that BCI2000 satisfies the stringent real-time requirements of BCI systems. By substantially reducing labor and cost, BCI2000 facilitates the implementation of different BCI systems and other psychophysiological experiments. It is available with full documentation and free of charge for research or educational purposes and is currently being used in a variety of studies by many research groups.
Although traditionally regarded as spared, a range of oculomotor dysfunction has been recorded in patients with amyotrophic lateral sclerosis (ALS). Most frequent is ophthalmoparesis, particularly in patients with prolonged survival; however, pursuit, nystagmus, and saccadic impairments have also been reported. The apparent resistance to pathologic involvement of oculomotor (and sphincter) control pathways in most patients with ALS has prompted comparative study to establish the key pathways that underlie motor neuronal vulnerability, with the hope of generating novel therapeutic strategies. Developments in the assessment of oculomotor function, including portable eye-tracking devices, have revealed more subtle impairments in ALS in relation to phenotype, which can now be better understood through parallel elucidation of the normal cerebral oculomotor control network. Given the clinicopathologic overlap between ALS and some types of frontotemporal dementia, the study of oculomotor function has particular value in probing the variable but consistent cognitive impairment seen in ALS and that reflects frontotemporal extramotor cerebral abnormalities. By transcending the requirement to write or speak, loss of which precludes standard neuropsychological testing in some patients with advanced ALS, cognitive tests performed using only oculomotor functions offer additional potential, allowing the study of patients much later in their disease course. The study of oculomotor dysfunction holds significant promise as an additional source of much needed prognostic, monitoring, and mechanistic biomarkers for ALS.
Pathological conditions such as amyotrophic lateral sclerosis or damage to the brainstem can leave patients severely paralyzed but fully aware, in a condition known as 'locked-in syndrome'. Communication in this state is often reduced to selecting individual letters or words by arduous residual movements. More intuitive and rapid communication may be restored by directly interfacing with language areas of the cerebral cortex. We used a grid of closely spaced, nonpenetrating micro-electrodes to record local field potentials (LFPs) from the surface of face motor cortex and Wernicke's area. From these LFPs we were successful in classifying a small set of words on a trial-by-trial basis at levels well above chance. We found that the pattern of electrodes with the highest accuracy changed for each word, which supports the idea that closely spaced micro-electrodes are capable of capturing neural signals from independent neural processing assemblies. These results further support using cortical surface potentials (electrocorticography) in brain-computer interfaces. These results also show that LFPs recorded from the cortical surface (micro-electrocorticography) of language areas can be used to classify speech-related cortical rhythms and potentially restore communication to locked-in patients.
Imagery of motor movement plays an important role in learning of complex motor skills, from learning to serve in tennis to perfecting a pirouette in ballet. What and where are the neural substrates that underlie motor imagery-based learning? We measured electrocorticographic cortical surface potentials in eight human subjects during overt action and kinesthetic imagery of the same movement, focusing on power in "high frequency" (76-100 Hz) and "low frequency" (8-32 Hz) ranges. We quantitatively establish that the spatial distribution of local neuronal population activity during motor imagery mimics the spatial distribution of activity during actual motor movement. By comparing responses to electrocortical stimulation with imagery-induced cortical surface activity, we demonstrate the role of primary motor areas in movement imagery. The magnitude of imagery-induced cortical activity change was approximately 25% of that associated with actual movement. However, when subjects learned to use this imagery to control a computer cursor in a simple feedback task, the imagery-induced activity change was significantly augmented, even exceeding that of overt movement.
Author Summary
For a very long time, the measurement of the large scale potentials produced by the brain from outside of the head, using electroencephalography and magnetoencephalography, and from inside the head, using electrocorticography, has fixated on changes in specific rhythms and frequency ranges. This fixation presupposes physiologic changes where neuronal populations synchronously oscillate at specific timescales. Here, we demonstrate that there are phenomena which obey a broadband, power-law form extending across the entire frequency domain, with no special timescale. It is shown that, with local brain activity, there is an increase in power across all frequencies, and the power-law shape is conserved. Furthermore, we illustrate through simple simulation how fluctuations in this phenomenon may be linked to increases and decreases in “noise-like” patterns of activity in neuronal populations. Although power-laws have been postulated to exist in background electrical brain activity, the view that local activity can be captured by fluctuations in a broadband power-law in the power spectrum of electric potential timeseries represents a fundamentally new way of thinking about changes in the electric potential produced by the brain, and provides insight into what types of neuronal processes might produce these potentials.
Human scalp EEG studies have shown that event-related desynchronization (ERD) in the alpha (8-13 Hz) and beta (15-25 Hz) bands may be used to detect functional activation of sensorimotor cortex. However, in most previous studies somatotopy has not been examined in detail and brief, self-paced movements, focusing on the planning of motor output, have been used. We recorded electrocorticographic (ECoG) signals in five clinical subjects during a visual-motor decision task that was designed to activate the representations of different body parts in sensorimotor cortex. To focus more on execution of motor output than on its planning, subjects were instructed to make sustained isometric muscle contractions in different body parts (tongue protrusion, fist-clenching or foot dorsiflexion) in response to randomized visual stimuli depicting each action. ECoG spectral analysis utilized a mixed-effects analysis of variance model in which within-trial temporal dependencies were taken into account, and the magnitude and statistical significance of alpha and beta ERDs were mapped onto a surface rendering of each subject's brain MRI. Cortical electrical stimulation was performed in all subjects for clinical purposes, and the resulting maps of sensorimotor function were compared with those generated by ECoG spectral analysis. During the early phases of the motor responses, alpha ERD commonly occurred in a diffuse spatial pattern that was not somatotopically specific. During the late phases, the spatial pattern of alpha ERD usually became more focused and somatotopically specific. Maps of alpha ERD were closer to cortical stimulation maps when alpha ERD was sustained throughout the late phases of the motor responses. Thus, the topography of alpha ERD more resembled traditional somatotopy when its temporal profile approximated that of the motor response. The topography of beta ERD was often more discrete and somatotopically specific than that of alpha ERD, but beta ERD was often transient and sometimes absent. Sometimes, unilateral limb movement produced sustained alpha and beta ERD over bilateral sensorimotor cortices, with overlapping patterns for different body parts. The topographical spread of alpha ERD beyond expected functional-anatomical boundaries during early (and sometimes late) phases of motor responses invites a re-examination of traditional assumptions about sensorimotor functional neuroanatomy, as well as the role of alpha ERD as an index of cortical activation. We agree with others that the somatotopic representations of different body parts overlap more than previously thought. Also, unilateral limb movements may be associated with both contralateral and ipsilateral activation of sensorimotor cortex. We conjecture that alpha ERD may reflect activity within a broad synaptic network with distributed cortical representations.
In the first large study of its kind, we quantified changes in electrocorticographic signals associated with motor movement across 22 subjects with subdural electrode arrays placed for identification of seizure foci. Patients underwent a 5-7 d monitoring period with array placement, before seizure focus resection, and during this time they participated in the study. An interval-based motor-repetition task produced consistent and quantifiable spectral shifts that were mapped on a Talairach-standardized template cortex. Maps were created independently for a high-frequency band (HFB) (76-100 Hz) and a low-frequency band (LFB) (8-32 Hz) for several different movement modalities in each subject. The power in relevant electrodes consistently decreased in the LFB with movement, whereas the power in the HFB consistently increased. In addition, the HFB changes were more focal than the LFB changes. Sites of power changes corresponded to stereotactic locations in sensorimotor cortex and to the results of individual clinical electrical cortical mapping. Sensorimotor representation was found to be somatotopic, localized in stereotactic space to rolandic cortex, and typically followed the classic homunculus with limited extrarolandic representation.
Background
Approximately 20% of traumatic cervical spinal cord injuries result in tetraplegia. Neuroprosthetics are being developed to manage this condition and thus improve the lives of patients. We aimed to test the feasibility of a semi-invasive technique that uses brain signals to drive an exoskeleton.
Methods
We recruited two participants at Clinatec research centre, associated with Grenoble University Hospital, Grenoble, France, into our ongoing clinical trial. Inclusion criteria were age 18–45 years, stability of neurological deficits, a need for additional mobility expressed by the patient, ambulatory or hospitalised monitoring, registration in the French social security system, and signed informed consent. The exclusion criteria were previous brain surgery, anticoagulant treatments, neuropsychological sequelae, depression, substance dependence or misuse, and contraindications to magnetoencephalography (MEG), EEG, or MRI. One participant was excluded because of a technical problem with the implants. The remaining participant was a 28-year-old man, who had tetraplegia following a C4–C5 spinal cord injury. Two bilateral wireless epidural recorders, each with 64 electrodes, were implanted over the upper limb sensorimotor areas of the brain. Epidural electrocorticographic (ECoG) signals were processed online by an adaptive decoding algorithm to send commands to effectors (virtual avatar or exoskeleton). Throughout the 24 months of the study, the patient did various mental tasks to progressively increase the number of degrees of freedom.
Findings
Between June 12, 2017, and July 21, 2019, the patient cortically controlled a programme that simulated walking and made bimanual, multi-joint, upper-limb movements with eight degrees of freedom during various reach-and-touch tasks and wrist rotations, using a virtual avatar at home (64·0% [SD 5·1] success) or an exoskeleton in the laboratory (70·9% [11·6] success). Compared with microelectrodes, epidural ECoG is semi-invasive and has similar efficiency. The decoding models were reusable for up to approximately 7 weeks without recalibration.
Interpretation
These results showed long-term (24-month) activation of a four-limb neuroprosthetic exoskeleton by a complete brain–machine interface system using continuous, online epidural ECoG to decode brain activity in a tetraplegic patient. Up to eight degrees of freedom could be simultaneously controlled using a unique model, which was reusable without recalibration for up to about 7 weeks.
Funding
French Atomic Energy Commission, French Ministry of Health, Edmond J Safra Philanthropic Foundation, Fondation Motrice, Fondation Nanosciences, Institut Carnot, Fonds de Dotation Clinatec.
Although animal studies provided significant insights in understanding the neural basis of learning and adaptation, they often cannot dissociate between different learning mechanisms due to the lack of verbal communication. To overcome this limitation, we examined the mechanisms of learning and its limits in a human intracortical brain-machine interface (BMI) paradigm. A tetraplegic participant controlled a 2D computer cursor by modulating single-neuron activity in the anterior intraparietal area (AIP). By perturbing the neuron-to-movement mapping, the participant learned to modulate the activity of the recorded neurons to solve the perturbations by adopting a target re-aiming strategy. However, when no cognitive strategies were adequate to produce correct responses, AIP failed to adapt to perturbations. These findings suggest that learning is constrained by the pre-existing neuronal structure, although it is possible that AIP needs more training time to learn to generate novel activity patterns when cognitive re-adaptation fails to solve the perturbations.