Figure - available from: Advanced Science
This content is subject to copyright. Terms and conditions apply.
Electrode contribution during the study period. a) MRI reconstruction of the participant's brain, overlaid on top of which are the ECoG grids implanted as part of the clinical trial. Electrodes used in this study are colored in red (motor) and blue (sensory). The grey electrodes were not used in this study. b) Simulated online accuracy when the decoding model is trained with both motor and sensory electrodes, only motor electrodes, only sensory electrodes, and only the most salient electrode. Chance = 16.67% (shown as dashed line). Each box corresponds to the accuracy for n = 33 testing days (****p < 0.0001, Mann–Whitney‐Wilcoxon test two‐sided with Bonferroni correction). c) Relative contribution of each of the electrodes to the decoding results for each real‐time usage month.
Source publication
Brain‐computer interfaces (BCIs) can be used to control assistive devices by patients with neurological disorders like amyotrophic lateral sclerosis (ALS) that limit speech and movement. For assistive control, it is desirable for BCI systems to be accurate and reliable, preferably with minimal setup time. In this study, a participant with severe dy...
Similar publications
Background
Brain-computer interfaces (BCIs) can restore communication for movement- and/or speech-impaired individuals by enabling neural control of computer typing applications. Single command click detectors provide a basic yet highly functional capability.
Methods
We sought to test the performance and long-term stability of click decoding using...
Citations
... В клинически исследованных иИМК использовались внутрикорковые [9,10,, электрокортикографические (ЭкоГ) [8,[40][41][42][43][44][45][46][47][48][49] или эндоваскулярный [5,50,51] типы датчиков (рис. 1). ...
... В недавно опубликованных работах с целью реконструкции речи использовались многоканальные ЭКоГматрицы высокой плотности (128 [8,48,49] или 250 [45] электродов), позволяющие регистрировать сигналы с большой площади коры головного мозга, включая участки левой прецентральной извилины, постцентральной извилины, задних средней и нижней лобных извилин и верхней височной извилины. В работе [45] продемонстрирована иИМК система, которая с помощью подхода глубокого обучения декодирует артикуляционную активность коры головного мозга в несколько модальностей: пациент имитирует говорение, а результат распознавания выводится текстом на экран, озвучивается синхронно с движением губ и мимикой аватара лица на том же мониторе. ...
Brain-computer interfaces (BCIs) provide direct information exchange between the brain and a computer. In invasive BCIs, electrodes are placed intracranially for more accurate and faster data exchange between the brain and external devices. The primary medical objective of these technologies is to compensate for motor or speech function in patients with tetraparesis and anarthria. In recent years, the emergence of new neuroimplants for BCIs and the results demonstrated in clinical trials have led to a notable increase in interest in these systems from the scientific community, investors, and the public. This review compares different types of medical invasive BCIs, analyzes and discusses the achievements and unsolved problems of clinical application of these neurotechnologies, as well as possible consequences and risks of their wider use.
... Minimally invasive electrocorticography (ECoG) based BCI, provides a much higher signal stability than electroencephalography (EEG) based BCIs, or than highly invasive Microelectrode Array (MEA) based BCIs [5]. ECoG-BCIs showed their ability to properly decode brain activity without recalibration for several months [6], [7]. However, these studies included only one subject who was intensively trained to control the BCI. ...
... It is well known, however, that BCI control performances and motor imagery capabilities can vary significantly across subjects, 15-30% of patients even being described as BCI-illiterated or inefficient [8]. Furthermore, although studies [6], [7] showed globally stable performances over time, inter-sessions variability remained significant. ...
Comprehension of performance variabilities across subjects and sessions is crucial for real life brain-computer-interfaces (BCI) applications. This study compared three subjects that underwent implantation of minimally invasive WIMAGINE ECoG recording implants. Three training strategies to discern best achievable performance, session drift, and variability were evaluated offline using datasets recorded during real-time closed-loop BCI experiments. Results revealed distinct BCI profiles across patients, consistent with qualitative observations made during online training. These performances were correlated with two indicators computed in feature space during idle periods of BCI sessions: Euclidean distance between the current session and the session of model creation in a low-dimensional UMAP embedding, and intrinsic dimension. Between sessions distances demonstrated statistically significant correlation with models' performances, then recalibration need may be potentially anticipated from the characteristics of idle state periods. Additionally, the intrinsic dimension was significantly correlated to subjects' overall BCI capabilities. The results are consistent with pre-implantation MEG-BCI experiments, which could make it useful for patient selection.
... Speech production is a highly complex process involving an extensive and interconnected network of brain regions 2,3 . Yet, the current state-of-the-art speech BCIs are primarily focused on decoding signals from the sensorimotor cortex [4][5][6][7][8][9] . While this region has shown promise in enabling communication for patients with dysarthria or anarthria due to amyotrophic lateral sclerosis (ALS) or a brain-stem stroke, it may not cover the full intent of the user 10 nor translate to individuals with different diseases or disease progressions. ...
Speech is the fastest and most natural form of communication, which can be impaired in certain disorders. Speech brain-computer interfaces (BCIs) offer a solution by decoding brain activity into speech. Current neuroprosthetic devices focus on the motor cortex, which might not be usable in all patient populations. Fortunately, many other brain regions have been associated with the speech production process. Here, we investigate which regions are potential (alternative) targets for a speech BCI across a brain-wide distribution within a single study. The distribution includes sulci and subcortical areas, sampled with both a high temporal and a high spatial resolution. Thirty participants were recorded with intracranial electroencephalography during speech production, resulting in 3249 recorded contacts across the brain. We trained machine learning models to continuously predict speech from a brain-wide global to a single-channel local scale. Within each scale we examined a variation of selected electrode contacts based on anatomical features within participants. We found significant speech detection in both gray and white matter tissue, no significant difference between gyri and sulci at any of the analysis scales and limited contribution from subcortical areas. The best potential targets in terms of decoding accuracy and consistency are located within the depth of and surrounding the lateral fissure bilaterally, such as the (sub)central sulcus, transverse temporal gyrus (Heschls' gyrus), the supramarginal cortex and parts of the insula. These results highlight the potential benefits of extending beyond the motor cortex and reaching the sulcal depth for speech neuroprostheses.
... Brain signals Based on decoding electrical signals produced by the brain, sensed by electrodes placed on the head. Either invasively under the skull [26] or non-invasively on the scalp using EEGs [10,27]. ...
Background
Loss of communication with loved ones and carers is one of the most isolating and debilitating effects of many neurological disorders. Assistive technology (AT) supports individuals with communication, but the acceptability of AT solutions is highly variable. In this paper a novel ear based control method of AT, the concept of ’EarSwitch’, is presented. This new approach is based on detecting ear rumbling, which is the voluntary contraction of the tensor tympani muscle (TTM), resulting in observable movement of the eardrum and a dull rumbling sound. ’EarSwitch’ has the potential to be a discreet method that can complement existing AT control methods. However, only a subset of the population can ear rumble and little is known about the ability of rumbling in populations with neurological disorders.
Methods
To explore the viability of the ’EarSwitch’ concept as an AT control method we conducted in-depth online surveys with (N=1853) respondents from the general population and (N=170) respondents with self-declared neurological disorders including Motor Neurone Disease (MND) and Multiple Sclerosis (MS).This is the largest ever study to explore ear rumbling and the first to explore whether rumbling is preserved among individuals with neurological disorders. In addition, we validated rumbling, and investigated usability of the ’EarSwitch’ concept as a control input, using in-person otoscopic examination with a subset of participants.
Results
A significant proportion of the population with neurological disorders could benefit from ’EarSwitch’ controllable AT. The upper bound prevalence of the ability to rumble without accompanying movements was 55% in the general population, 38% in the neurological population, and 20% of participants with MND (N=95) reported this ability. During the validation procedure, participants achieved high accuracy in self-reporting the ability to rumble (80%) and proved concept of using the ’EarSwitch’ method to control a basic interface.
Discussion
’EarSwitch’ is a potential new AT control method control, either by itself or as a supplement to other existing methods. Results demonstrate self-reported ear rumbling is present among patients with different neurological disorders, including MND. Further research should explore how well the ability to rumble is preserved in different types and stages of neurological disorders.
... 5,6,[8][9][10][11][12] In recent years, speech BCIs have emerged as a viable path toward restoring fast, naturalistic communication for people with paralysis by instead decoding attempted speech movements. [18][19][20][21][22][23] In contrast to hand-based BCIs, speech BCIs have typically been driven by neural activity in sensorimotor cortical areas further ventral such as middle precentral gyrus (midPCG) and ventral precentral gyrus (vPCG) which are most often associated with production of orofacial movements and speech. 18,20-22 Speech BCIs far outperform cursor BCIs with regard to communication rate, 10,20 but are not as well-suited for general-purpose computer control. ...
Decoding neural activity from ventral (speech) motor cortex is known to enable high-performance speech brain-computer interface (BCI) control. It was previously unknown whether this brain area could also enable computer control via neural cursor and click, as is typically associated with dorsal (arm and hand) motor cortex. We recruited a clinical trial participant with ALS and implanted intracortical microelectrode arrays in ventral precentral gyrus (vPCG), which the participant used to operate a speech BCI in a prior study. We developed a cursor BCI driven by the participant's vPCG neural activity, and evaluated performance on a series of target selection tasks. The reported vPCG cursor BCI enabled rapidly-calibrating (40 seconds), accurate (2.90 bits per second) cursor control and click. The participant also used the BCI to control his own personal computer independently. These results suggest that placing electrodes in vPCG to optimize for speech decoding may also be a viable strategy for building a multi-modal BCI which enables both speech-based communication and computer control via cursor and click.
... The next step allowing for more complex computer control could be the use of a limited set of keywords. For example, using six-keyword navigation with commands "up", "down", "left", "right", "enter" and "back" decoded from ECoG signals has recently demonstrated reliable control of computer menus and environment at high accuracy (median accuracy of 90.59%) over several months [6]. However, scaling up the individual word decoding from ECoG has proven rather challenging due to inter-subject variability, limited amounts of data and inherent limitations of decoding from brain signals. ...
... The most relevant electrodes in both subjects were located on the dorsal part of the ventral sensorimotor cortex, with the highest weights found closest to the central sulcus. This mirrors the results of another ECoG-study for keyword decoding [6]. Interestingly, most contributing electrodes seemed to be located on the postcentral gyrus -the somatosensory area of the brain. ...
... One important difference between our study subjects and future locked-in users is that the voice onset times will not be available as ground truth for training the classifiers. As an alternative, different methods of extracting activity onset directly from brain signals have already been proposed and used [4,6]. ...
Brain-computer interfaces (BCIs) can help people with locked-in syndrome to communicate. While continuous speech decoding can be used in everyday communication, navigating a computer menu or interacting with external devices may be easier and more reliable using a small set of distinct command keywords. In this preliminary study, two able-bodied epilepsy patients, temporarily implanted with high-density electrocorticography (ECoG) electrodes, spoke 14 potential keywords out loud in Dutch. With optimized Support Vector Machines (SVMs), the maximal decoding accuracy reached was a median of 93.3% for 50 repetitions per word (practical chance level 9.6%). We also identified that a minimum of 30 repetitions was needed to achieve this result, and determined that the most relevant electrodes for decoding were on the ventral sensorimotor cortex, close to the central sulcus.
... One such limitation is the generalizability of the findings, as the intricate interplay between tumor characteristics, surgical interventions, and individual patient responses may vary widely across cases [43]. Furthermore, while direct cortical stimulation DCS and ECoG offer invaluable insights into brain function, they are invasive techniques that may not be universally applicable or preferable in all clinical scenarios [44]. ...
This study investigates the effects of occipital lobe tumors on visual processing and the role of brain-computer interface (BCI) technologies in post-surgical visual rehabilitation. Through a combination of pre-surgical functional magnetic resonance imaging (fMRI) and Diffusion Tensor Imaging (DTI), intra-operative direct cortical stimulation (DCS) and Electrocorticography (ECoG), and post-surgical BCI interventions, we provide insight into the complex dynamics between occipital lobe tumors and visual function. Our results highlight a discrepancy between clinical assessments of visual field damage and the patient's reported visual experiences, suggesting a residual functional capacity within the damaged occipital regions. Additionally, the absence of expected visual phenomena during surgery and the promising outcomes from BCI-driven rehabilitation underscore the complexity of visual processing and the potential of technology-enhanced rehabilitation strategies. This work emphasizes the need for an interdisciplinary approach in developing effective treatments for visual impairments related to brain tumors, illustrating the significant implications for neurosurgical practices and the advancement of rehabilitation sciences.
... Example devices include speech synthesizers 2 , computer cursors 3 , spellers, assistive robotic end effectors 4,5 and functional electrical stimulation devices 6,7 , along with systems that provide tactile feedback via intracortical microstimulation 8 . Recent developments have focused on the design and material composition of the implanted electrodes 9,10 as well as on improving decoding speed and accuracy 11 . The latter has mainly been driven by advances in signal processing and the application of new machine learning and deep learning algorithms, including large language models used in speech iBCIs 12,13 . ...
Implanted brain–computer interfaces (iBCIs) translate brain activity recorded intracranially into commands for virtual or physical machines to restore or rehabilitate motor, sensory or speech functions. Currently, no iBCIs have been approved by regulatory agencies for the medical device market despite being in clinical trials since 1998, with little information available about their progress and outcomes. To address this gap, we conducted a review of all identified clinical trials of iBCIs for communication, motor control or restoration of tactile perception conducted between 1998 and 2023. We summarize findings from 21 research groups worldwide and their 67 participants who received implants to understand the challenges and opportunities in the iBCI field. This analysis highlights the importance of improving participant
diversity, creating a participant registry to inform future research,
regulatory and payer approvals, investor funding and new applications, adopting governed data sharing and standards, and boosting collaborative research.
... A high sampling rate is employed to capture sudden changes, with weak signals amplified accordingly (Clément 2019;Saini and Satija 2023). Furthermore, the frequency range of interest can be extracted using high-pass and lowpass filters, while data quality is enhanced through artifact elimination techniques like Independent Component Analysis (ICA) (Shaila et al. 2023;Luo et al. 2023). ...
... At the core of BCI systems lies an algorithm that extracts features and categorizes EEG data, playing a vital role in their functioning (Schirrmeister et al. 2017). Speech Imagery in BCI offers a promising solution for speech restoration in conditions such as Amyotrophic Lateral Sclerosis (ALS) and traumatic brain injury (Luo et al. 2023). However, existing methods are limited, predominantly focusing on vowels and a few characters (Gasparini et al. 2022). ...
Numerous individuals encounter challenges in verbal communication due to various factors, including physical disabilities, neurological disorders, and strokes. In response to this pressing need, technology has actively pursued solutions to bridge the communication gap, recognizing the inherent difficulties faced in verbal communication, particularly in contexts where traditional methods may be inadequate. Electroencephalogram (EEG) has emerged as a primary non-invasive method for measuring brain activity, offering valuable insights from a cognitive neurodevelopmental perspective. It forms the basis for Brain-Computer Interfaces (BCIs) that provide a communication channel for individuals with neurological impairments, thereby empowering them to express themselves effectively. EEG-based BCIs, especially those adapted to decode imagined speech from EEG signals, represent a significant advancement in enabling individuals with speech disabilities to communicate through text or synthesized speech. By utilizing cognitive neurodevelopmental insights, researchers have been able to develop innovative approaches for interpreting EEG signals and translating them into meaningful communication outputs. To aid researchers in effectively addressing this complex challenge, this review article synthesizes key findings from state-of-the-art significant studies. It investigates into the methodologies employed by various researchers, including preprocessing techniques, feature extraction methods, and classification algorithms utilizing Deep Learning and Machine Learning approaches and their integration. Furthermore, the review outlines the potential avenues for future research, with the goal of advancing the practical implementation of EEG-based BCI systems for decoding imagined speech from a cognitive neurodevelopmental perspective.
... These insights are pivotal in directing future clinical practices to ameliorate the quality of life for those with language disorders. Another investigation into decoding speech-related neural activities through long-term implanted ECoG devices revealed that patients could reliably control external devices via BCI without the need for recalibration or retraining, thereby confirming the sustainable efficacy and stability of BCI applications [150] . ...
Neurological disorders such as Parkinson's disease, stroke, and spinal cord injury can result in impairments of motor function, consciousness, cognition, and sensory processing. Brain-Computer Interface (BCI) technology, which facilitates direct communication between the brain and external devices, emerges as an innovative key to unlocking neurological conditions, demonstrating significant promise in this context. This comprehensive review uniquely synthesizes the latest advancements in BCI research across multiple neurological disorders, offering an interdisciplinary perspective on both clinical applications and emerging technologies. We explore the progress in BCI research and its applications in addressing various neurological conditions, with a particular focus on recent clinical studies and prospective developments. Initially, the review provides an up-to-date overview of BCI technology, encompassing its classification, operational principles, and prevalent paradigms. It then critically examines specific BCI applications in movement disorders, disorders of consciousness, cognitive and mental disorders, as well as sensory disorders, highlighting novel approaches and their potential impact on patient care. This review reveals emerging trends in BCI applications, such as the integration of artificial intelligence and the development of closed-loop systems, which represent significant advancements over previous technologies. The review concludes by discussing the prospects and directions of BCI technology, underscoring the need for interdisciplinary collaboration and ethical considerations. It emphasizes the importance of prioritizing bidirectional and high-performance BCIs, areas that have been underexplored in previous reviews. Additionally, we identify crucial gaps in current research, particularly in long-term clinical efficacy and the need for standardized protocols. The role of neurosurgery in spearheading the clinical translation of BCI research is highlighted. Our comprehensive analysis presents BCI technology as an innovative key to unlocking neurological disorders, offering a transformative approach to diagnosing, treating, and rehabilitating neurological conditions, with substantial potential to enhance patients' quality of life and advance the field of neurotechnology.
