Incorporating Feedback from Multiple Sensory Modalities Enhances Brain-Machine Interface Control

Department of Organismal Biology and Anatomy, University of Chicago, Chicago, Illinois 60637, USA.
The Journal of Neuroscience : The Official Journal of the Society for Neuroscience (Impact Factor: 6.34). 12/2010; 30(50):16777-87. DOI: 10.1523/JNEUROSCI.3967-10.2010
Source: PubMed


The brain typically uses a rich supply of feedback from multiple sensory modalities to control movement in healthy individuals. In many individuals, these afferent pathways, as well as their efferent counterparts, are compromised by disease or injury resulting in significant impairments and reduced quality of life. Brain-machine interfaces (BMIs) offer the promise of recovered functionality to these individuals by allowing them to control a device using their thoughts. Most current BMI implementations use visual feedback for closed-loop control; however, it has been suggested that the inclusion of additional feedback modalities may lead to improvements in control. We demonstrate for the first time that kinesthetic feedback can be used together with vision to significantly improve control of a cursor driven by neural activity of the primary motor cortex (MI). Using an exoskeletal robot, the monkey's arm was moved to passively follow a cortically controlled visual cursor, thereby providing the monkey with kinesthetic information about the motion of the cursor. When visual and proprioceptive feedback were congruent, both the time to successfully reach a target decreased and the cursor paths became straighter, compared with incongruent feedback conditions. This enhanced performance was accompanied by a significant increase in the amount of movement-related information contained in the spiking activity of neurons in MI. These findings suggest that BMI control can be significantly improved in paralyzed patients with residual kinesthetic sense and provide the groundwork for augmenting cortically controlled BMIs with multiple forms of natural or surrogate sensory feedback.

Full-text preview

Available from:
  • Source
    • "The development of sophisticated anthropomorphic robotic limbs and of methods to decode intended movements from motor areas of the brain1234567offer the possibility of restoring sensorimotor function to patients who have lost it[8]. While these are remarkable achievements, upper-limb neuroprostheses may not be clinically viable until they provide somatosensory feedback. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The dexterous manipulation of objects depends heavily on somatosensory signals from the limb. The development of anthropomorphic robotic arms and of algorithms to decode intended movements from neuronal signals has stimulated the need to restore somatosensation for use in upper-limb neuroprostheses. Without touch and proprioception, patients have difficulty controlling prosthetic limbs to a level that justifies the required invasive surgery. Intracortical microstimulation (ICMS) through chronically implanted electrode arrays has the potential to provide rich and intuitive sensory feedback. This approach to sensory restoration requires, however, that the evoked sensations remain stable over time. To investigate the stability of ICMS-evoked sensations, we measured the ability of non-human primates to detect ICMS over experimental sessions that spanned years. We found that the performance of the animals remained highly stable over time, even when they were tested with electrodes that had experienced extensive stimulation. Given the stability of the sensations that it evokes, ICMS may thus be a viable approach for sensory restoration.
    Full-text · Article · Aug 2015 · Journal of Neural Engineering
  • Source
    • "The primary somatosensory cortex (SI), for example, is anatomically and functionally connected to multiple motor cortical areas (Goldring and Ratcheson, 1972), and evidence suggests that it exercises significant influence on motor cortex during natural movement planning and execution (Avanzino et al., 2013; Hatsopoulos and Suminski, 2011; Fogassi et al., 1992; di Pellegrino et al., 1992; Shaikhouni et al., 2013). Further support for SI involvement in adaptation is provided by studies in which congruence of visual and proprioceptive feedback improved BMI performance (Suminski et al., 2010). Other sources of adaptation may come from other brain areas, particularly the posterior parietal cortex (PPC) that was shown to mediate visually-guided, on-line corrections of movement trajectories (Andersen et al., 2010; Cui and Andersen, 2007). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Neuroplasticity is key to the operation of brain machine interfaces (BMIs)-a direct communication pathway between the brain and a man-made computing device. Whereas exogenous BMIs that associate volitional control of brain activity with neurofeedback have been shown to induce long lasting plasticity, endogenous BMIs that use prolonged activity-dependent stimulation - and thus may curtail the time scale that governs natural sensorimotor integration loops - have been shown to induce short lasting plasticity. Here we summarize recent findings from studies using both categories of BMIs, and discuss the fundamental principles that may underlie their operation and the longevity of the plasticity they induce. We draw comparison to plasticity mechanisms known to mediate natural sensorimotor skill learning and discuss principles of homeostatic regulation that may constrain endogenous BMI effects in the adult mammalian brain. We propose that BMIs could be designed to facilitate structural and functional plasticity for the purpose of re-organization of target brain regions and directed augmentation of sensorimotor maps, and suggest possible avenues for future work to maximize their efficacy and viability in clinical applications. Copyright © 2015. Published by Elsevier Inc.
    Full-text · Article · May 2015 · Neurobiology of Disease
  • Source
    • "opening. This congruent incorporation of feedback from visual, haptic and proprioceptive modalities has recently been shown to enhance cortically controlled brain–machine applications in non-human primates (Suminski et al., 2010). For simplicity we refer here to this condition as MI + proprioceptive feedback as we consider this as the prominent component. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Neurofeedback of self-regulated brain activity in circumscribed cortical regions is used as a novel strategy to facilitate functional restoration following stroke. Basic knowledge about its impact on motor system oscillations and functional connectivity is however scarce. Specifically, a direct comparison between different feedback modalities and their neural signatures is missing. We assessed a neurofeedback training intervention of modulating β-activity in circumscribed sensorimotor regions by kinesthetic motor imagery (MI). Right-handed healthy participants received two different feedback modalities contingent to their MI-associated brain activity in a cross-over design: (I) visual feedback with a brain-computer interface (BCI) and (II) proprioceptive feedback with a brain-robot interface (BRI) orthosis attached to the right hand. High-density electroencephalography was used to examine the reactivity of the cortical motor system during the training session of each task by studying both local oscillatory power entrainment and distributed functional connectivity. Both feedback modalities activated a distributed functional connectivity network of coherent oscillations. A significantly higher skill and lower variability of self-controlled sensorimotor β-band modulation could, however, be achieved in the BRI condition. This gain in controlling regional motor oscillations was accompanied by functional coupling of remote β-band and θ-band activity in bilateral fronto-central regions and left parieto-occipital regions, respectively. The functional coupling of coherent θ-band oscillations correlated moreover with the skill of regional β-modulation thus revealing a motor learning related network. Our findings indicate that proprioceptive feedback is more suitable than visual feedback to entrain the motor network architecture during the interplay between motor imagery and feedback processing thus resulting in better volitional control of regional brain activity.
    Full-text · Article · Feb 2015 · NeuroImage
Show more