Incorporating Feedback from Multiple Sensory Modalities Enhances Brain-Machine Interface Control

Department of Organismal Biology and Anatomy, University of Chicago, Chicago, Illinois 60637, USA.
The Journal of Neuroscience : The Official Journal of the Society for Neuroscience (Impact Factor: 6.75). 12/2010; 30(50):16777-87. DOI: 10.1523/JNEUROSCI.3967-10.2010
Source: PubMed

ABSTRACT The brain typically uses a rich supply of feedback from multiple sensory modalities to control movement in healthy individuals. In many individuals, these afferent pathways, as well as their efferent counterparts, are compromised by disease or injury resulting in significant impairments and reduced quality of life. Brain-machine interfaces (BMIs) offer the promise of recovered functionality to these individuals by allowing them to control a device using their thoughts. Most current BMI implementations use visual feedback for closed-loop control; however, it has been suggested that the inclusion of additional feedback modalities may lead to improvements in control. We demonstrate for the first time that kinesthetic feedback can be used together with vision to significantly improve control of a cursor driven by neural activity of the primary motor cortex (MI). Using an exoskeletal robot, the monkey's arm was moved to passively follow a cortically controlled visual cursor, thereby providing the monkey with kinesthetic information about the motion of the cursor. When visual and proprioceptive feedback were congruent, both the time to successfully reach a target decreased and the cursor paths became straighter, compared with incongruent feedback conditions. This enhanced performance was accompanied by a significant increase in the amount of movement-related information contained in the spiking activity of neurons in MI. These findings suggest that BMI control can be significantly improved in paralyzed patients with residual kinesthetic sense and provide the groundwork for augmenting cortically controlled BMIs with multiple forms of natural or surrogate sensory feedback.

1 Follower
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Neurofeedback of self-regulated brain activity in circumscribed cortical regions is used as a novel strategy to facilitate functional restoration following stroke. Basic knowledge about its impact on motor system oscillations and functional connectivity is however scarce. Specifically, a direct comparison between different feedback modalities and their neural signatures is missing. We assessed a neurofeedback training intervention of modulating β-activity in circumscribed sensorimotor regions by kinesthetic motor imagery (MI). Right-handed healthy participants received two different feedback modalities contingent to their MI-associated brain activity in a cross-over design: (I) visual feedback with a brain-computer interface (BCI) and (II) proprioceptive feedback with a brain-robot interface (BRI) orthosis attached to the right hand. High-density electroencephalography was used to examine the reactivity of the cortical motor system during the training session of each task by studying both local oscillatory power entrainment and distributed functional connectivity. Both feedback modalities activated a distributed functional connectivity network of coherent oscillations. A significantly higher skill and lower variability of self-controlled sensorimotor β-band modulation could, however, be achieved in the BRI condition. This gain in controlling regional motor oscillations was accompanied by functional coupling of remote β-band and θ-band activity in bilateral fronto-central regions and left parieto-occipital regions, respectively. The functional coupling of coherent θ-band oscillations correlated moreover with the skill of regional β-modulation thus revealing a motor learning related network. Our findings indicate that proprioceptive feedback is more suitable than visual feedback to entrain the motor network architecture during the interplay between motor imagery and feedback processing thus resulting in better volitional control of regional brain activity. Published by Elsevier Inc.
    NeuroImage 02/2015; 111. DOI:10.1016/j.neuroimage.2015.01.058 · 6.13 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The typical control of myoelectric interfaces, whether in laboratory settings or real-life prosthetic applications, largely relies on visual feedback because proprioceptive signals from the controlling muscles are either not available or very noisy. We conducted a set of experiments to test whether artificial proprioceptive feedback, delivered non-invasively to another limb, can improve control of a two-dimensional myoelectrically-controlled computer interface. In these experiments, participants' were required to reach a target with a visual cursor that was controlled by electromyogram signals recorded from muscles of the left hand, while they were provided with an additional proprioceptive feedback on their right arm by moving it with a robotic manipulandum. Provision of additional artificial proprioceptive feedback improved the angular accuracy of their movements when compared to using visual feedback alone but did not increase the overall accuracy quantified with the average distance between the cursor and the target. The advantages conferred by proprioception were present only when the proprioceptive feedback had similar orientation to the visual feedback in the task space and not when it was mirrored, demonstrating the importance of congruency in feedback modalities for multi-sensory integration. Our results reveal the ability of the human motor system to learn new inter-limb sensory-motor associations; the motor system can utilize task-related sensory feedback, even when it is available on a limb distinct from the one being actuated. In addition, the proposed task structure provides a flexible test paradigm by which the effectiveness of various sensory feedback and multi-sensory integration for myoelectric prosthesis control can be evaluated.
    IEEE Transactions on Neural Systems and Rehabilitation Engineering 09/2014; 23(3). DOI:10.1109/TNSRE.2014.2355856 · 2.82 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Brain-machine interfaces (BMIs) using motor cortical activity to drive an external effector like a screen cursor or a robotic arm have seen enormous success and proven their great rehabilitation potential. An emerging parallel effort is now directed to BMIs controlled by endogenous cognitive activity, also called cognitive BMIs. While more challenging, this approach opens new dimensions to the rehabilitation of cognitive disorders. In the present work, we focus on BMIs driven by visuospatial attention signals and we provide a critical review of these studies in the light of the accumulated knowledge about the psychophysics, anatomy, and neurophysiology of visual spatial attention. Importantly, we provide a unique comparative overview of the several studies, ranging from non-invasive to invasive human and non-human primates studies, that decode attention-related information from ongoing neuronal activity. We discuss these studies in the light of the challenges attention-driven cognitive BMIs have to face. In a second part of the review, we discuss past and current attention-based neurofeedback studies, describing both the covert effects of neurofeedback onto neuronal activity and its overt behavioral effects. Importantly, we compare neurofeedback studies based on the amplitude of cortical activity to studies based on the enhancement of cortical information content. Last, we discuss several lines of future research and applications for attention-driven cognitive brain-computer interfaces (BCIs), including the rehabilitation of cognitive deficits, restored communication in locked-in patients, and open-field applications for enhanced cognition in normal subjects. The core motivation of this work is the key idea that the improvement of current cognitive BMIs for therapeutic and open field applications needs to be grounded in a proper interdisciplinary understanding of the physiology of the cognitive function of interest, be it spatial attention, working memory or any other cognitive signal.
    Frontiers in Systems Neuroscience 08/2014; 8:144. DOI:10.3389/fnsys.2014.00144


Available from