Incorporating Feedback from Multiple Sensory Modalities Enhances Brain-Machine Interface Control

Department of Organismal Biology and Anatomy, University of Chicago, Chicago, Illinois 60637, USA.
The Journal of Neuroscience : The Official Journal of the Society for Neuroscience (Impact Factor: 6.75). 12/2010; 30(50):16777-87. DOI: 10.1523/JNEUROSCI.3967-10.2010
Source: PubMed

ABSTRACT The brain typically uses a rich supply of feedback from multiple sensory modalities to control movement in healthy individuals. In many individuals, these afferent pathways, as well as their efferent counterparts, are compromised by disease or injury resulting in significant impairments and reduced quality of life. Brain-machine interfaces (BMIs) offer the promise of recovered functionality to these individuals by allowing them to control a device using their thoughts. Most current BMI implementations use visual feedback for closed-loop control; however, it has been suggested that the inclusion of additional feedback modalities may lead to improvements in control. We demonstrate for the first time that kinesthetic feedback can be used together with vision to significantly improve control of a cursor driven by neural activity of the primary motor cortex (MI). Using an exoskeletal robot, the monkey's arm was moved to passively follow a cortically controlled visual cursor, thereby providing the monkey with kinesthetic information about the motion of the cursor. When visual and proprioceptive feedback were congruent, both the time to successfully reach a target decreased and the cursor paths became straighter, compared with incongruent feedback conditions. This enhanced performance was accompanied by a significant increase in the amount of movement-related information contained in the spiking activity of neurons in MI. These findings suggest that BMI control can be significantly improved in paralyzed patients with residual kinesthetic sense and provide the groundwork for augmenting cortically controlled BMIs with multiple forms of natural or surrogate sensory feedback.

1 Follower
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Neurofeedback of self-regulated brain activity in circumscribed cortical regions is used as a novel strategy to facilitate functional restoration following stroke. Basic knowledge about its impact on motor system oscillations and functional connectivity is however scarce. Specifically, a direct comparison between different feedback modalities and their neural signatures is missing. We assessed a neurofeedback training intervention of modulating β-activity in circumscribed sensorimotor regions by kinesthetic motor imagery (MI). Right-handed healthy participants received two different feedback modalities contingent to their MI-associated brain activity in a cross-over design: (I) visual feedback with a brain-computer interface (BCI) and (II) proprioceptive feedback with a brain-robot interface (BRI) orthosis attached to the right hand. High-density electroencephalography was used to examine the reactivity of the cortical motor system during the training session of each task by studying both local oscillatory power entrainment and distributed functional connectivity. Both feedback modalities activated a distributed functional connectivity network of coherent oscillations. A significantly higher skill and lower variability of self-controlled sensorimotor β-band modulation could, however, be achieved in the BRI condition. This gain in controlling regional motor oscillations was accompanied by functional coupling of remote β-band and θ-band activity in bilateral fronto-central regions and left parieto-occipital regions, respectively. The functional coupling of coherent θ-band oscillations correlated moreover with the skill of regional β-modulation thus revealing a motor learning related network. Our findings indicate that proprioceptive feedback is more suitable than visual feedback to entrain the motor network architecture during the interplay between motor imagery and feedback processing thus resulting in better volitional control of regional brain activity. Published by Elsevier Inc.
    NeuroImage 02/2015; 111. DOI:10.1016/j.neuroimage.2015.01.058 · 6.13 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Observing an action being performed and executing the same action cause similar patterns of neural activity to emerge in the primary motor cortex (MI). Previous work has shown that the neural activity evoked during action observation (AO) is informative as to both the kinematics and muscle activation patterns of the action being performed, although the neural activity recorded during action observation contains less information than the activity recorded during action execution (AE). In this study, we extend these results by comparing the representation of different kinematic variables in MI single /multi unit activity between AO and AE conditions in three rhesus macaques. We show that the representation of acceleration decreases more significantly than that of position and velocity in AO (population decoding performance for acceleration decreases more steeply, and fewer neurons in AO encode acceleration significantly as compared to AE). We discuss the relevance of these results to brain-machine interfaces that make use of neural activity during AO to initialize a mapping function between neural activity and motor commands.
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objective. Brain-computer interfaces (BCIs) are being developed to assist paralyzed people and amputees by translating neural activity into movements of a computer cursor or prosthetic limb. Here we introduce a novel BCI task paradigm, intended to help accelerate improvements to BCI systems. Through this task, we can push the performance limits of BCI systems, we can quantify more accurately how well a BCI system captures the user's intent, and we can increase the richness of the BCI movement repertoire. Approach. We have implemented an instructed path task, wherein the user must drive a cursor along a visible path. The instructed path task provides a versatile framework to increase the difficulty of the task and thereby push the limits of performance. Relative to traditional point-to-point tasks, the instructed path task allows more thorough analysis of decoding performance and greater richness of movement kinematics. Main results. We demonstrate that monkeys are able to perform the instructed path task in a closed-loop BCI setting. We further investigate how the performance under BCI control compares to native arm control, whether users can decrease their movement variability in the face of a more demanding task, and how the kinematic richness is enhanced in this task. Significance. The use of the instructed path task has the potential to accelerate the development of BCI systems and their clinical translation.
    Journal of Neural Engineering 01/2015; 12(1):016015. DOI:10.1088/1741-2560/12/1/016015 · 3.42 Impact Factor


Available from