Jose del R. Millan

University of Texas at Austin | UT · Department of Electrical & Computer Engineering

Telepresence robots can support people with special needs (e.g., who cannot move) to remotely interact with people and the environment at a distance. In this application, people can communicate with the robot via alternative channels of communication, such as brain-machine interfaces, that are less accurate than the traditional mediums and allow th...

Error-related potentials (ErrP) are a prominent electroencephalogram (EEG) correlate of performance monitoring, and so crucial for learning and adapting our behavior. Although there exists an agreement that ErrP signal awareness to errors, it remains poorly understood whether they encode further information. Here we report an experiment with sixtee...

Mind-controlled wheelchairs are an intriguing assistive mobility solution applicable in complete paralysis. Despite progress in brain-machine interface (BMI) technology, its translation remains elusive. The primary objective of this study is to probe the hypothesis that BMI skill acquisition by end-users is fundamental to control a non-invasive bra...

To date, brain-computer interfaces (BCIs) have proved to play a key role in many medical applications, for example, the rehabilitation of stroke patients. For post-stroke rehabilitation, the BCIs require the EEG electrodes to precisely translate the brain signals of patients into intended movements of the paralyzed limb for months. However, the gol...

The brain mechanism of embodiment in a virtual body has grown a scientific interest recently, with a particular focus on providing optimal virtual reality (VR) experiences. Disruptions from an embodied state to a less- or non-embodied state, denominated Breaks in Embodiment (BiE), are however rarely studied despite their importance for designing in...

Task‐specificity in isolated focal dystonias is a powerful feature that may successfully be targeted with therapeutic brain–computer interfaces. While performing a symptomatic task, the patient actively modulates momentary brain activity (disorder signature) to match activity during an asymptomatic task (target signature), which is expected to tran...

The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current...

In search and rescue missions, drone operations are challenging and cognitively demanding. High levels of cognitive workload can affect rescuers' performance, leading to failure with catastrophic outcomes. To face this problem, we propose a machine learning algorithm for real-time cognitive workload monitoring to understand if a search and rescue o...

This article proposes a novel shared intelligence system for brain-machine interface (BMI) teleoperated mobile robots where user’s intention and robot’s intelligence are concurrent elements equally participating in the decision process. We designed the system to rely on policies guiding the robot’s behavior according to the current situation. W...

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signa...

In this paper, we propose a deep learning-based algorithm to improve the performance of automatic speech recognition (ASR) systems for aphasia, apraxia, and dysarthria speech by utilizing electroencephalography (EEG) features recorded synchronously with aphasia, apraxia, and dysarthria speech. We demonstrate a significant decoding performance impro...

Current non-invasive Brain Machine interfaces commonly rely on the decoding of sustained motor imagery activity (MI). This approach enables a user to control brain-actuated devices by triggering predetermined motor actions. One major drawback of such strategy is that users are not trained to stop their actions. Indeed, the termination process invol...

The last decade has seen a flowering of applications driven by brain–machine interfaces (BMIs), particularly brain-actuated robotic devices designed to restore the independence of people suffering from severe motor disabilities. This review provides an overview of the state of the art of noninvasive BMI-driven devices based on 86 studies published...

When humans perceive an erroneous action, an EEG error related potential (ErrP) is elicited as a neural response. ErrPs have been largely investigated in discrete feedback protocols, where actions are executed at discrete steps, to enable seamless brain-computer interaction. However, there are only a few studies that investigate ErrPs in continuous...

In this paper, we propose a deep learning-based algorithm to improve the performance of automatic speech recognition (ASR) systems for aphasia, apraxia, and dysarthria speech by utilizing electroencephalography (EEG) features recorded synchronously with aphasia, apraxia, and dysarthria speech. We demonstrate a significant decoding performance impro...

Adaptively increasing the difficulty level in learning was shown to be beneficial than increasing the level after some fixed time intervals. To efficiently adapt the level, we aimed at decoding the subjective difficulty level based on EEG signals. We designed a visuomotor learning task that one needed to pilot a simulated drone through a series of...

Objective: In contrast to the classical visual BCI paradigms, which adhere to a rigid trial structure and restricted user behavior, EEG-based visual recognition decoding during our daily activities remains challenging. The objective of this study is to explore the feasibility of decoding the EEG signature of visual recognition in experimental cond...

Decoding the subjective perception of task difficulty may help improve operator performance, i.e., automatically optimize the task difficulty level. Here, we aim to decode a compound of cognitive states that covaries with the task difficulty level. We designed a protocol composed of two different subtasks, flying and visual recognition, to induce d...

The mathematical properties of high-dimensional (HD) spaces show remarkable agreement with behaviors controlled by the brain. Computing with HD vectors, referred to as “hypervectors,” is a brain-inspired alternative to computing with numbers. HD computing is characterized by generality, scalability, robustness, and fast learning, making it a prime...

This work studies the class of algorithms for learning with side-information that emerges by extending generative models with embedded context-related variables. Using finite mixture models (FMMs) as the prototypical Bayesian network, we show that maximum-likelihood estimation (MLE) of parameters through expectation-maximization (EM) improves over...

Brain-machine interface (BMI) technology has rapidly matured over the last two decades, mainly thanks to the introduction of artificial intelligence (AI) methods, in particular, machine-learning algorithms. Yet, the need for subjects to learn to modulate their brain activity is a key component of successful BMI control. Blending machine and subject...

One of the most popular methods in non-invasive brain machine interfaces (BMI) rely on the decoding of sensorimotor rhythms associated to sustained motor imagery. Although motor imagery has been intensively studied, its termination was mostly neglected. Objective: Here, we provide insights in the decoding of motor imagery termination and investig...

Objective: Event Related Potentials (ERPs) reflecting cognitive response to external stimuli, are widely used in Brain Computer Interfaces (BCI). ERPs are characterized and typically decoded through a fixed set of components with particular amplitude and latency. However, the classical methods which rely on waveform features achieve a high decodin...

The human capacity to compute the likelihood that a decision is correct—known as metacognition—has proven difficult to study in isolation as it usually cooccurs with decision making. Here, we isolated postdecisional from decisional contributions to metacognition by analyzing neural correlates of confidence with multimodal imaging. Healthy volunteer...

Advances in sports sciences and neurosciences offer new opportunities to design efficient and motivating sport training tools. For instance, using NeuroFeedback (NF), athletes can learn to self-regulate specific brain rhythms and consequently improve their performances. Here, we focused on soccer goalkeepers’ Covert Visual Spatial Attention (CVSA)...

Visual attention can be spatially oriented, even in the absence of saccadic eye-movements, to facilitate the processing of incoming visual information. One behavioral proxy for this so-called covert visuospatial attention (CVSA) is the validity effect (VE): the reduction in reaction time (RT) to visual stimuli at attended locations and the increase...

Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI...

Throughout life, the central nervous system (CNS) interacts with the world and with the body by activating muscles and excreting hormones. In contrast, brain-computer interfaces (BCIs) quantify CNS activity and translate it into new artificial outputs that replace, restore, enhance, supplement, or improve the natural CNS outputs. BCIs thereby modif...

This chapter introduces the field of brain–machine interfaces (BMIs), also called brain–computer interfaces (BCIs), which has seen impressive achievements over the past few years. A BMI monitors the user’s brain activity, extracts specific features from the brain signals that reflect the intent of the subject, and translates them into actions. BMI...

Objective: Brain-computer interface (BCI) spelling is a promising communication solution for people in paralysis. Currently, BCIs suffer from imperfect decoding accuracy which calls for methods to handle spelling mistakes. Detecting error-related potentials (ErrPs) has been early identified as a potential remedy. Nevertheless, few works have studie...

Despite the growing interest in brain–machine interface (BMI)-driven neuroprostheses, the translation of the BMI output into a suitable control signal for the robotic device is often neglected. In this article, we propose a novel control approach based on dynamical systems that was explicitly designed to take into account the nature of the BMI outp...

Upper limb motor deficits in severe stroke survivors often remain unresolved over extended time periods. Novel neurotechnologies have the potential to significantly support upper limb motor restoration in severely impaired stroke individuals. Here, we review recent controlled clinical studies and reviews focusing on the mechanisms of action and eff...

In this study, we evaluated the ability to identify individual words in a binary word classification task during imagined speech, using high frequency activity (HFA; 70–150 Hz) features in the time domain. For this, we used an imagined word repetition task cued with a word perception stimulus, and followed by an overt word repetition, and compared...

Closed-loop or adaptive deep brain stimulation (DBS) for Parkinson's Disease (PD) has shown comparable clinical improvements to continuous stimulation, yet with less stimulation times and side effects. In this form of control, stimulation is driven by pathological beta oscillations recorded from the subthalamic nucleus, which have been shown to cor...

The introduction of deep learning and transfer learning techniques in fields such as computer vision allowed a leap forward in the accuracy of image classification tasks. Currently there is only limited use of such techniques in neuroscience. The challenge of using deep learning methods to successfully train models in neuroscience, lies in the comp...

Behavioral assessments of consciousness based on overt command following cannot differentiate patients with disorders of consciousness (DOC) from those who demonstrate a dissociation between intent/awareness and motor capacity: cognitive motor dissociation (CMD). We argue that delineation of peri-personal space (PPS) - the multisensory-motor space...

Cognitive information has been exploited in noninvasive Brain Computer Interface (BCI) scenarios to provide autonomous external agents with additional information. In this context, Error-related potentials (ErrPs), temporal deflections in electroencephalogram (EEG) signals when humans perceive erroneous actions, have been exploited to teach correct...

Topography of CMC during right hand movement in healthy participants. (A) CMC in the alpha (8–12 Hz) frequency band. (B) CMC in the gamma (30–40 Hz) frequency band were also increased contralaterally during movement in the healthy participant group but to a lesser extent than the beta CMC. Color scale: CMC.

Topography of beta-range CMC from the single right-sided stroke patient. The data recorded show stronger left-sided and more locally focused beta CMC. (A) Beta CMC was lower over the affected right hemisphere during left hand movements than (B) over the left hemisphere during right-handed movements. Color scale: CMC.

Motor recovery following stroke is believed to necessitate alteration in functional connectivity between cortex and muscle. Cortico-muscular coherence has been proposed as a potential biomarker for post-stroke motor deficits, enabling a quantification of recovery, as well as potentially indicating the regions of cortex involved in recovery of funct...

Movements are preceded by certain brain states that can be captured through various neuroimaging techniques. Brain-Computer Interfaces can be designed to detect the movement intention brain state during driving, which could be beneficial in improving the interaction between a smart car and its driver, by providing assistance in-line with the driver...

The human capacity to compute the likelihood that a decision is correct - known as metacognition - has proven difficult to study in isolation as it usually co-occurs with decision-making. Here, we isolated post-decisional from decisional contributions to metacognition by combining a novel paradigm with multimodal imaging. Healthy volunteers reporte...

Key properties of brain-inspired hyperdimensional (HD) computing make it a prime candidate for energy-efficient and fast learning in biosignal processing. The main challenge is however to formulate embedding methods that map biosignal measures to a binary HD space. In this paper, we explore variety of such embedding methods and examine them with a...

This article introduces the field of brain-computer interfaces (BCI), which allows the control of devices without the generation of any active motor output but directly from the decoding of the user’s brain signals. Here we review the current state of the art in the BCI field, discussing the main components of such an interface and illustrating ong...

Excessive beta oscillatory activity in the subthalamic nucleus (STN) is linked to Parkinson’s Disease (PD) motor symptoms. However, previous works have been inconsistent regarding the functional role of beta activity in untreated Parkinsonian states, questioning such role. We hypothesized that this inconsistency is due to the influence of electroph...

Non-invasive and invasive electrical neurostimulation are promising tools to better understand brain function and ultimately treat its malfunction. In current open-loop approaches, a clinician chooses a fixed set of stimulation parameters, informed by observed therapeutic benefits and previous empirical evidence. However, this procedure leads to a...

Modern cars can support their drivers by assessing and autonomously performing different driving maneuvers based on information gathered by in-car sensors. We propose that brain–machine interfaces (BMIs) can provide complementary information that can ease the interaction with intelligent cars in order to enhance the driving experience. In our appro...

Brain-computer interfaces (BCIs) based on motor imagery have been gaining attention as tools to facilitate recovery from movement disorders resulting from stroke or other causes. These BCIs can detect imagined movements that are typically required within conventional rehabilitation therapy. This information about the timing, intensity, and location...

Hand grasping is a sophisticated motor task that has received much attention by the neuroscientific community, which demonstrated how grasping activates a network involving parietal, pre-motor and motor cortices using fMRI, ECoG, LFPs and spiking activity. Yet, there is a need for a more precise spatio-temporal analysis as it is still unclear how t...

Brain-machine interfaces (BMIs) have been used to incorporate the user intention to trigger robotic devices by decoding movement onset from electroencephalography (EEG). Active neural participation is crucial to promote brain plasticity thus to enhance the opportunity of motor recovery. This study presents the decoding of lower-limb movement-relate...

To investigate whether a motor attempt EEG paradigm coupled with functional electrical stimulation can detect command following and, therefore, signs of conscious awareness in patients with disorders of consciousness, we recorded nine patients admitted to acute rehabilitation after a brain lesion. We extracted peak classification accuracy and peak...

Practical brain-computer interfaces need to overcome several challenges, including tolerance to signal variability within- and across sessions. We introduce Robust Principal Component Analysis (RPCA) as a potential approach to tackle intra-trial variability. Assuming that subjects undergo the same cognitive process or perform the same task in a sho...

Brain-computer interfaces (BCI) are used in stroke rehabilitation to translate brain signals into intended movements of the paralyzed limb. However, the efficacy and mechanisms of BCI-based therapies remain unclear. Here we show that BCI coupled to functional electrical stimulation (FES) elicits significant, clinically relevant, and lasting motor r...

Certain brain disorders resulting from brainstem infarcts, traumatic brain injury, cerebral palsy, stroke, and amyotrophic lateral sclerosis, limit verbal communication despite the patient being fully aware. People that cannot communicate due to neurological disorders would benefit from a system that can infer internal speech directly from brain si...

This paper aims at providing a preliminary description of ROS-Health, a novel framework for neurorobotics based on the middleware Robot Operating System (ROS). The increased interest in the neurorobotics field and the proliferation of several (neuro)physiological-based applications to control robotics devices made clear the importance to establish...

Author summary Noninvasive brain–computer interface (BCI) based on imagined movements can restore functions lost to disability by enabling spontaneous, direct brain control of external devices without risks associated with surgical implantation of neural interfaces. We hypothesized that, contrary to the popular trend of focusing on the machine lear...

Electrode configurations. (A) EEG channel configuration over 16 locations of the sensorimotor cortex according to the international 10–20 system. (B) EOG electrode configuration on the pilot’s right and left canthi, nasion, and forehead for the detection of ocular and facial muscle artifacts. EEG, electroencephalography; EOG, electrooculogram. (TIF...

BCI feature discriminancy for pilot P2 after artifact removal with FORCe. (A) Topographic maps of discriminancy per training month on the 16 EEG channel locations over the sensorimotor cortex monitored. Bright color indicates high discriminancy between Both Hands and Both Feet MI tasks employed by pilot P2. The discriminancy of each channel is quan...

User-training methodology details of the Cybathlon BCI race competitors. BCI, brain–computer interface. (DOCX)

BCI feature discriminancy maps for three typical BCI sessions of pilot P2 in August, September, and October after artifact removal with FORCe. Bright color indicates high discriminancy between Both Hands and Both Feet motor imagery tasks employed by pilot P2. The discriminancy of each feature (channel–frequency pair) is quantified as the Fisher sco...

BCI feature discriminancy maps per run (N) averaged for each training month. Bright color indicates high discriminancy between Both Hands and Both Feet MI tasks employed by both pilots (P1 top, P2 bottom). The discriminancy of each feature (channel-frequency pair) is quantified as the Fisher score of the EEG signal's power spectral density distribu...

BCI feature discriminancy per training modality. Topographic maps of discriminancy per training modality on the 16 EEG channel locations over the sensorimotor cortex monitored. Bright color indicates high discriminancy between Both Hands and Both Feet MI tasks employed by both pilots (P1 top, P2 bottom). The discriminancy of each channel is quantif...

BCI feature discriminancy maps per run (N) averaged for each training month. Bright color indicates high discriminancy between Both Hands and Both Feet motor imagery tasks employed by both pilots (P1 top, P2 bottom). The discriminancy of each feature (channel–frequency pair) is quantified as the Fisher score of the EEG signal's power spectral densi...

Training session information. The table presents the date of all executed training sessions for both pilots and the number and type of runs performed in each session and reported here. Asterisks indicate one or more runs have been lost due to technical failure or bad maintenance. (DOCX)

Background: Excessive beta oscillatory activity in the subthalamic nucleus (STN) is linked to Parkinson's disease and associated motor symptoms. However, the relationship between beta activity and motor symptoms has been inconsistent, which may influence the efficacy of closed-loop deep brain stimulation. Hypothesis: We hypothesized that this varia...

Competition changes the environment for athletes. The difficulty of training for such stressful events can lead to the well-known effect of “choking” under pressure, which prevents athletes from performing at their best level. To study the effect of competition on the human brain, we recorded pilot electroencephalography (EEG) data while novice sho...