Incorporating Feedback from Multiple Sensory Modalities Enhances Brain-Machine Interface Control

Department of Organismal Biology and Anatomy, University of Chicago, Chicago, Illinois 60637, USA.
The Journal of Neuroscience : The Official Journal of the Society for Neuroscience (Impact Factor: 6.34). 12/2010; 30(50):16777-87. DOI: 10.1523/JNEUROSCI.3967-10.2010
Source: PubMed


The brain typically uses a rich supply of feedback from multiple sensory modalities to control movement in healthy individuals. In many individuals, these afferent pathways, as well as their efferent counterparts, are compromised by disease or injury resulting in significant impairments and reduced quality of life. Brain-machine interfaces (BMIs) offer the promise of recovered functionality to these individuals by allowing them to control a device using their thoughts. Most current BMI implementations use visual feedback for closed-loop control; however, it has been suggested that the inclusion of additional feedback modalities may lead to improvements in control. We demonstrate for the first time that kinesthetic feedback can be used together with vision to significantly improve control of a cursor driven by neural activity of the primary motor cortex (MI). Using an exoskeletal robot, the monkey's arm was moved to passively follow a cortically controlled visual cursor, thereby providing the monkey with kinesthetic information about the motion of the cursor. When visual and proprioceptive feedback were congruent, both the time to successfully reach a target decreased and the cursor paths became straighter, compared with incongruent feedback conditions. This enhanced performance was accompanied by a significant increase in the amount of movement-related information contained in the spiking activity of neurons in MI. These findings suggest that BMI control can be significantly improved in paralyzed patients with residual kinesthetic sense and provide the groundwork for augmenting cortically controlled BMIs with multiple forms of natural or surrogate sensory feedback.

1 Follower
8 Reads
  • Source
    • "The primary somatosensory cortex (SI), for example, is anatomically and functionally connected to multiple motor cortical areas (Goldring and Ratcheson, 1972), and evidence suggests that it exercises significant influence on motor cortex during natural movement planning and execution (Avanzino et al., 2013; Hatsopoulos and Suminski, 2011; Fogassi et al., 1992; di Pellegrino et al., 1992; Shaikhouni et al., 2013). Further support for SI involvement in adaptation is provided by studies in which congruence of visual and proprioceptive feedback improved BMI performance (Suminski et al., 2010). Other sources of adaptation may come from other brain areas, particularly the posterior parietal cortex (PPC) that was shown to mediate visually-guided, on-line corrections of movement trajectories (Andersen et al., 2010; Cui and Andersen, 2007). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Neuroplasticity is key to the operation of brain machine interfaces (BMIs)-a direct communication pathway between the brain and a man-made computing device. Whereas exogenous BMIs that associate volitional control of brain activity with neurofeedback have been shown to induce long lasting plasticity, endogenous BMIs that use prolonged activity-dependent stimulation - and thus may curtail the time scale that governs natural sensorimotor integration loops - have been shown to induce short lasting plasticity. Here we summarize recent findings from studies using both categories of BMIs, and discuss the fundamental principles that may underlie their operation and the longevity of the plasticity they induce. We draw comparison to plasticity mechanisms known to mediate natural sensorimotor skill learning and discuss principles of homeostatic regulation that may constrain endogenous BMI effects in the adult mammalian brain. We propose that BMIs could be designed to facilitate structural and functional plasticity for the purpose of re-organization of target brain regions and directed augmentation of sensorimotor maps, and suggest possible avenues for future work to maximize their efficacy and viability in clinical applications. Copyright © 2015. Published by Elsevier Inc.
    Neurobiology of Disease 05/2015; DOI:10.1016/j.nbd.2015.05.001 · 5.08 Impact Factor
  • Source
    • "opening. This congruent incorporation of feedback from visual, haptic and proprioceptive modalities has recently been shown to enhance cortically controlled brain–machine applications in non-human primates (Suminski et al., 2010). For simplicity we refer here to this condition as MI + proprioceptive feedback as we consider this as the prominent component. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Neurofeedback of self-regulated brain activity in circumscribed cortical regions is used as a novel strategy to facilitate functional restoration following stroke. Basic knowledge about its impact on motor system oscillations and functional connectivity is however scarce. Specifically, a direct comparison between different feedback modalities and their neural signatures is missing. We assessed a neurofeedback training intervention of modulating β-activity in circumscribed sensorimotor regions by kinesthetic motor imagery (MI). Right-handed healthy participants received two different feedback modalities contingent to their MI-associated brain activity in a cross-over design: (I) visual feedback with a brain-computer interface (BCI) and (II) proprioceptive feedback with a brain-robot interface (BRI) orthosis attached to the right hand. High-density electroencephalography was used to examine the reactivity of the cortical motor system during the training session of each task by studying both local oscillatory power entrainment and distributed functional connectivity. Both feedback modalities activated a distributed functional connectivity network of coherent oscillations. A significantly higher skill and lower variability of self-controlled sensorimotor β-band modulation could, however, be achieved in the BRI condition. This gain in controlling regional motor oscillations was accompanied by functional coupling of remote β-band and θ-band activity in bilateral fronto-central regions and left parieto-occipital regions, respectively. The functional coupling of coherent θ-band oscillations correlated moreover with the skill of regional β-modulation thus revealing a motor learning related network. Our findings indicate that proprioceptive feedback is more suitable than visual feedback to entrain the motor network architecture during the interplay between motor imagery and feedback processing thus resulting in better volitional control of regional brain activity.
    NeuroImage 02/2015; 111. DOI:10.1016/j.neuroimage.2015.01.058 · 6.36 Impact Factor
  • Source
    • "during imagined movements, demonstrating that such degradation is also present at the neural scale. Other studies have demonstrated the influence of kinesthetic feedback in ongoing M1 activity (Gaunt, et al., 2013; Herter, et al., 2009; Pruszynski, et al., 2011; Suminski, et al., 2010) suggesting the importance of trans-cortical feedback pathways for predicting optimal states(Scott, 2012) and for modulating sensory feedback accordingly (Todorov and Jordan, 2002). "
    [Show abstract] [Hide abstract]
    ABSTRACT: A major assumption of brain–machine interface research is that patients with disconnected neural pathways can still volitionally recall precise motor commands that could be decoded for naturalistic prosthetic control. However, the disconnected condition of these patients also blocks kinaesthetic feedback from the periphery, which has been shown to regulate centrally generated output responsible for accurate motor control. Here, we tested how well motor commands are generated in the absence of kinaesthetic feedback by decoding hand movements from human scalp electroencephalography in three conditions: unimpaired movement, imagined movement, and movement attempted during temporary disconnection of peripheral afferent and efferent nerves by ischemic nerve block. Our results suggest that the recall of cortical motor commands is impoverished in the absence of kinaesthetic feedback, challenging the possibility of precise naturalistic cortical prosthetic control. Hum Brain Mapp, 2014. © 2014 Wiley Periodicals, Inc.
    Human Brain Mapping 10/2014; 36(2). DOI:10.1002/hbm.22653 · 5.97 Impact Factor
Show more


8 Reads
Available from