Neuronal ensemble control of prosthetic devices by a human with tetraplegia.
ABSTRACT Neuromotor prostheses (NMPs) aim to replace or restore lost motor functions in paralysed humans by routeing movement-related signals from the brain, around damaged parts of the nervous system, to external effectors. To translate preclinical results from intact animals to a clinically useful NMP, movement signals must persist in cortex after spinal cord injury and be engaged by movement intent when sensory inputs and limb movement are long absent. Furthermore, NMPs would require that intention-driven neuronal activity be converted into a control signal that enables useful tasks. Here we show initial results for a tetraplegic human (MN) using a pilot NMP. Neuronal ensemble activity recorded through a 96-microelectrode array implanted in primary motor cortex demonstrated that intended hand motion modulates cortical spiking patterns three years after spinal cord injury. Decoders were created, providing a 'neural cursor' with which MN opened simulated e-mail and operated devices such as a television, even while conversing. Furthermore, MN used neural control to open and close a prosthetic hand, and perform rudimentary actions with a multi-jointed robotic arm. These early results suggest that NMPs based upon intracortical neuronal ensemble spiking activity could provide a valuable new neurotechnology to restore independence for humans with paralysis.
- [Show abstract] [Hide abstract]
ABSTRACT: Spinal cord injury (SCI) results in a loss of function and sensation below the level of the lesion. Neuroprosthetic technology has been developed to help restore motor and autonomic functions as well as to provide sensory feedback. This paper provides an overview of neuroprosthetic technology that aims to address the priorities for functional restoration as defined by individuals with SCI. We describe neuroprostheses that are in various stages of preclinical development, clinical testing, and commercialization including functional electrical stimulators, epidural and intraspinal microstimulation, bladder neuroprosthesis, and cortical stimulation for restoring sensation. We also discuss neural recording technologies that may provide command or feedback signals for neuroprosthetic devices. Conclusion/clinical relevance: Neuroprostheses have begun to address the priorities of individuals with SCI, although there remains room for improvement. In addition to continued technological improvements, closing the loop between the technology and the user may help provide intuitive device control with high levels of performance.The journal of spinal cord medicine 01/2013; 36(4):258-272. · 1.54 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: [MSc Thesis in Greek] Functional Cortical Connectivity studies the dynamics of information flow at the brain cortex. The sensorimotor cortex is activated in a similar fashion during actual and imaginary movement. Brain – Computer Interfaces exploit brain activity for the control of external devices. In this research we studied whether the properties of cortical functional networks differ between an actual and an imaginary movement, between movements of upper and lower extremities and between alpha and beta electroencephalographic (EEG) rhythms. Seven healthy people participated (four male, three female), performing under EEG four different motor tasks consisting of 95 trials each : a) right hand motor execution, b) right hand motor imagery, c) right foot motor execution and d) right foot motor imagery. EEG was recorded with 17 electrodes on the scalp over the sensorimotor cortex. The inverse problem was computed with Current Cortical Density. Two models of Regions Of Interest (ROIs) were defined, one consisting of 16 default regions and one consisting of 20 custom regions. Cortical functional connectivity was computed using Directed Transfer Function on both ROIs models for each subject, each motor task for alpha and beta rhythms. The results were studied with Graph Analysis, computing for each case the characteristic path length, clustering coefficient, density and small-worldness. No statistically important difference was found between functional cortical networks formed by actual and imaginary movement of the upper and lower extremities (ANOVA p<0.05). Between the functional networks formed by alpha and beta rhythms the findings consisted in a decrease in characteristic path length of 28-66.17%, a decrease in clustering coefficient of 2.09-39.76% and an increase in density of 20.18-40.65%. No statistically important difference was observed for small-worldness between alpha and beta rhythm. The properties of functional networks are similar between motor execution and imagery and could be of assistance in the future towards the enhancement of Brain-Computer Interfaces technology. We also support the conclusion that alpha and beta rhythms assume different neurophysiological roles in the sensorimotor cortex, based on the functional networks that are formed and their properties. Alpha rhythm appears to be carrier of information about the detailed evolution of a neurophysiological procedure and to cover more cortical nodes. On the contrary, beta rhythm appears to carry information about the integrated neurophysiological procedure across the sensorimotor cortex and to coordinate functional clusters. The functional networks of both alpha and beta rhythms have the properties of a small-world topology.03/2012, Degree: MSc in Medical Informatics, Supervisor: Panagiotis D. Bamidis
- [Show abstract] [Hide abstract]
ABSTRACT: The emerging field of neuroprosthetics is focused on the development of new therapeutic interventions that will be able to restore some lost neural function by selective electrical stimulation or by harnessing activity recorded from populations of neurons. As more and more patients benefit from these approaches, the interest in neural interfaces has grown significantly and a new generation of penetrating microelectrode arrays are providing unprecedented access to the neurons of the central nervous system (CNS). These microelectrodes have active tip dimensions that are similar in size to neurons and because they penetrate the nervous system, they provide selective access to these cells (within a few microns). However, the very long-term viability of chronically implanted microelectrodes and the capability of recording the same spiking activity over long time periods still remain to be established and confirmed in human studies. Here we review the main responses to acute implantation of microelectrode arrays, and emphasize that it will become essential to control the neural tissue damage induced by these intracortical microelectrodes in order to achieve the high clinical potentials accompanying this technology.Frontiers in Neuroengineering 01/2014; 7:24.
© 2006 Nature Publishing Group
Neuronal ensemble control of prosthetic
devices by a human with tetraplegia
Leigh R. Hochberg1,2,4, Mijail D. Serruya2,3, Gerhard M. Friehs5,6, Jon A. Mukand7,8, Maryam Saleh9†,
Abraham H. Caplan9, Almut Branner10, David Chen11, Richard D. Penn12& John P. Donoghue2,9
Neuromotor prostheses (NMPs) aim to replace or restore lost motor functions in paralysed humans by routeing
movement-related signals from the brain, around damaged parts of the nervous system, to external effectors. To
translate preclinical results from intact animals to a clinically useful NMP, movement signals must persist in cortex after
spinal cord injury and be engaged by movement intent when sensory inputs and limb movement are long absent.
Furthermore, NMPs would require that intention-driven neuronal activity be converted into a control signal that enables
useful tasks. Here we show initial results for a tetraplegic human (MN) using a pilot NMP. Neuronal ensemble activity
recorded through a 96-microelectrode array implanted in primary motor cortex demonstrated that intended hand motion
modulates cortical spiking patterns three years after spinal cord injury. Decoders were created, providing a ‘neural
cursor’ with which MN opened simulated e-mail and operated devices such as a television, even while conversing.
Furthermore, MN used neural control to open and close a prosthetic hand, and perform rudimentary actions with a multi-
jointed robotic arm. These early results suggest that NMPs based upon intracortical neuronal ensemble spiking activity
could provide a valuable new neurotechnology to restore independence for humans with paralysis.
Hundreds of thousands of people suffer from forms of motor
impairment in which intact movement-related areas of the brain
cannot generate movements because of damage to the spinal cord,
nerves, or muscles1. Paralysing disorders profoundly limit indepen-
dence, mobility and communication. Current assistive technologies
rely on devices for which an extant function provides a signal that
substitutes for missing actions. For example, cameras can monitor
eye movements that can be used to point a computer cursor2.
Although these surrogate devices have been available for some
time, they are typically limited in utility, cumbersome to maintain,
and disruptive of natural actions. For instance, gaze towards objects
of interest disrupts eye-based control. By contrast, an NMP is a
type of brain–computer interface (BCI) that can guide movement
by harnessing the existing neural substrate for that action—that is,
neuronal activity patterns in motor areas. An ideal NMP would
provide a safe, unobtrusive and reliable signal from the discon-
nected motor area that could restore lost function. Neurons in the
primary motor cortex (MI) arm area of monkeys, for example,
provide information about intended arm reaching trajectories3–5,
but this command signal would work for an NMP only if neural
signals persist and could be engaged by intention in paralysed
neurons, a decoder to translate ensemble firing patterns into motor
commands, and, typically, a computer gateway to engage effectors.
and in pilot trials in people with tetraparesis from spinal cord injury,
brainstem stroke, musculardystrophy, or amyotrophic lateral sclero-
sis. Currently, this system consists of a chronically implanted sensor
and external signal processors developed from preclinical animal
first in the BrainGate trial, is a 25-yr-old male (MN) who sustained a
knife wound in 2001 that transected the spinal cord between cervical
array was implanted in June 2004 into the MI arm area ‘knob’10, as
identified on pre-operative magnetic resonance imaging (MRI)
here are derived from 57 consecutive recording sessions from
14 July 2004 to 12 April 2005 (9months).
Signal quality and variety
Action potentials were readily observable on multiple electrodes,
indicating that MI neural spiking persists three years after SCI, as
suggested indirectly by functional MRI data11–14. Recorded signals
ranged from qualitatively well-isolated single neurons to mixtures of
a few different waveforms (Fig. 2a). Different waveform shapes were
identified visually, using standard time-amplitude windows, but
there was no further attempt to distinguish between well isolated
‘units’. An average of 26.9 ^ 14.2units were observed each day
(range 3–57), with mean peak-to-peak spike amplitudes of
76.4 ^ 25.0mV (mean ^ s.d., n ¼ 56 sessions) (see Supplementary
1Department of Neurology, Massachusetts General Hospital, Brigham and Women’s Hospital, and Spaulding Rehabilitation Hospital, Harvard Medical School, 55 Fruit Street,
Boston, Massachusetts 02114, USA.2Department of Neuroscience and Brain Science Program, and3Department of Engineering, Brown University, PO Box 1953, Providence,
Rhode Island 02912, USA.4Center for Restorative and Regenerative Medicine, Rehabilitation Research and Development Service, Department of Veterans Affairs, Veterans
Health Administration, 830 Chalkstone Avenue, Providence, Rhode Island 02908, USA.5Department of Clinical Neurosciences (Neurosurgery), Brown University, and
6Department of Neurosurgery, Rhode Island Hospital, 120 Dudley Street, Suite 103, Providence, Rhode Island 02905, USA.7Department of Rehabilitation Medicine, Brown
University, 593 Eddy Street, Providence, Rhode Island 02903, USA.8Sargent Rehabilitation Center, 800 Quaker Lane, Warwick, Rhode Island 02818, USA.9Cyberkinetics
Neurotechnology Systems, Inc., 100 Foxborough Boulevard–Suite 240, Foxborough, Massachusetts 02035, USA.10Cyberkinetics Neurotechnology Systems, Inc., 391 Chipeta
Way, Suite G, Salt Lake City, Utah 84108, USA.11Department of Physical Medicine and Rehabilitation, Rehabilitation Institute of Chicago, 345 E. Superior Street, 1146, Chicago,
Illinois 60611, USA.12Department of Neurosurgery, University of Chicago Hospitals, 5841 S. Maryland Avenue, MC3026, Chicago, Illinois 60637, USA. †Present address:
Graduate Program in Computational Neuroscience, University of Chicago, Chicago, Illinois 60637, USA.
Vol 442|13 July 2006|doi:10.1038/nature04970
© 2006 Nature Publishing Group
Information)8. Although details of local field potentials (LFPs) will
be described in a subsequent report, we note that LFPs, which could
be recorded simultaneously with spikes, resembled those observed in
intact monkeys (Fig. 2b)15. A notable decrease in the number of
recorded units was seen approximately 6.5months after implan-
tation and thereafter. Upon approval of a clinical protocol change at
10months, which permitted impedance measurements, we observed
a low impedance in 54 of the electrodes, consistent with a physical
short circuit to ground in the array, cable, or connector, but not
consistent with a biological event (for example, gliosis). The precise
reason for this physical change remains under investigation.
Since this report was written, we added a second trial participant
to the study, a 55-yr-old man who has had complete spinal cord
injury at C4 since 1999. Recordings were collected starting in the
seventh month after implant, after making an electrical contacts
repair in the pedestal connector. We recorded an average of
53.2 ^ 6.3units per session during trial months 7–10, again demon-
technical issue causing abrupt signal loss at most electrodes, which
may be related to the original repair, occurred at month 11 in
participant 2; the reason for this change is being evaluated.
Figure 1 | Intracortical sensor and placement, participant 1. a, The
skull. Neural signals are recorded while the pedestal is connected to the
remainder of the BrainGate system (seen in d). b, Scanning electron
micrograph of the 100-electrode sensor, 96 of which are available for neural
recording. Individual electrodes are 1-mm long and spaced 400mm apart, in
a 10 £ 10 grid. c, Pre-operative axial T1-weighted MRI of the brain of
participant1. The arm/hand ‘knob’ of therightprecentral gyrus(red arrow)
correspondsto the approximate location of the sensor implant site. Ascaled
projectionofthe4 £ 4-mmarrayontotheprecentralknobisoutlinedinred.
d, The first participant in the BrainGate trial (MN). He is sitting in a
wheelchair, mechanically ventilated through a tracheostomy. The grey box
(arrow) connected to the percutaneous pedestal contains amplifier and
signal conditioning hardware; cabling brings the amplified neural signals to
computers sitting beside the participant. He is looking at the monitor,
directing the neural cursor towards the orange square in this 16-target ‘grid’
task. A technician appears (A.H.C.) behind the participant.
Figure 2 | Electrical recordings from a sample of four electrodes.
a, Discriminated neural activity at electrodes 33, 34, 22, 95 (n ¼ 80
superimposed action potentials for each unit). On electrode 33, two
of 206 and 56mv, respectively. For electrode 34, a single unit is displayed.
Electrode 22 illustrates a low-amplitude discriminated signal. Electrode 95
shows triggered noise. Data are from trial day 90 (90 days after array
placement). b, Local field potentials during neural cursor control. In the
bottom panel, three traces of electrical recording (bandpass: 10–100Hz)
from one electrode are shown 0.5s before and 1.9s after the go cue
top of the screen. In the top panel, a Thomson multi-taper time frequency
analysis on each trial data segment was performed. This was done by sliding
a 0.3-s window every 0.05s, using a spectral resolution of 10Hz. These
power spectrograms were averaged across 20 trials to create the resulting
pseudocolour power spectral density (PSD) plot. The diagram is aligned
such that each point in the PSD plot corresponds to a time window 150ms
before and after an LFP. In the 20–30-Hz band, a decrease in power is seen
approximately 300ms after the go cue, followed by an increase in power
single trial data below.
NATURE|Vol 442|13 July 2006
© 2006 Nature Publishing Group
Modulation by intent
Imagined limb motions modulated neural firing on multipleelectro-
des, upon request, beginning at the first experimental session.
Modulation was evaluated during four consecutive sessions when
MN was asked to imagine a series of movements. This series revealed
a rich variety of firing modulations largely consistent with patterns
observed in monkey MI16. Importantly, this activity was evoked by
imagined actions in this participant with cervical spinal cord injury.
Figure 3a illustrates how certain neurons are selective for one
imagined action (hands together/apart), whereas others recorded
simultaneously are engaged by different imagined actions (elbow or
wrist). Thisdiversity includesneuronsthat firedwith imagined hand
or distal arm actions (for example, hand open/close, Fig. 3c) and
those that fired during shoulder movements that were actually
performed (see also Supplementary Fig. 1). Non-selective neurons,
active with the onset of any imagined upper extremity action
Fig. 1, neurons fired in a relatively time-locked manner upon the
request to imagine action. These results demonstrate a rich hetero-
geneity of firing patterns within a limited sample from a small
MI region. This diversity is useful in creating a flexible control
Linear filter construction
Use of MI neuronal ensemble activity as a control signal by persons
with paralysis requires a novel approach to establishing a transform
(filter function) between firing patterns and intended action. For
each session, units were used to create a filter (see Methods) to
provide a two-dimensional output signal displayed as cursor
position on a monitor. A simple linear filter algorithm, identical
to that used in intact monkeys, was used to create filters17. Unlike
most studies performed using intact monkeys, where hand
motions are known and kinematics are measured directly, we
predicted MN’s intended hand movements on the basis of
Figure 3 | Neuronal selectivity for imagined and performed movements.
a, Over an 80-s period, MN was asked to imagine performing a series of left
limb movements (which are described on the abscissa). Movement
instruction time is indicated by a vertical arrow; the go cue for each
alternating movement, presented as text on the video monitor, is indicated
by a small vertical hash mark. Spiking activity of two simultaneously
recorded units is displayed. Rasters indicate the time of each spike (thinned
for visual clarity; every third spike is shown). Normalized, integrated
firing rates (R) appear beneath each raster, as derived by the equation
R ¼ ½ðR21þnÞð12e2b=tÞ?; whereR21isthepreviousbin’s integrated firing
rate value, n ¼ the number of spikes in the current bin, b ¼ bin width, and
t ¼ time constant; bin width ¼ 50-ms window, time constant ¼ 10ms
(adapted from ref. 28); normalization is achieved by dividing by the
maximum integrated firing rate from each unit’s spike train over the time
period displayed. The top unit (channel 38) increases its firing rate (curved
simultaneously recordedunit(channel16)isactivatedmostclearlyafter the
instructiontoflex/extendthewristandto flex/extendtheelbowor movethe
shoulder anteriorly and posteriorly. All movements are imagined except for
shoulder movement, which MN actually performed. b, Go-cue-related
activity modulation for a neuron recorded simultaneously with those in a.
Each raster line is centred about the go cue, which requests that the patient
imagine a movement; the seven raster lines represent the epochs
surrounding each of the seven different movements in sequence a. The
histogram displays the total number of spikes seen in each 500-ms bin. This
neuron increased its firing rate during most imagined movement epochs,
but demonstrated poor instruction selectivity compared to the neurons
presented in a. Data are from day 161. c, Hand-instruction-related
modulation for three simultaneously recorded neurons. MN was cued to
are indicated. Each vertical tick represents one action potential (spike). An
close the hand. Data are from day 90.
NATURE|Vol 442|13 July 2006
© 2006 Nature Publishing Group
instructed actions (instruction-based algorithms have also been
reported in one study of intact monkeys18). Thus, for filter
building, MN was asked to imagine manually tracking a ‘tech-
nician’s cursor’ that was actually being moved by a technician-
operated mouse through a succession of randomly placed visual
targets (see Methods). The filter functionwas used to decode activity
and drive a ‘neural cursor’.
MI activity during neural cursor control
Features of neurons during neural cursor control resembled those
expected from MI. Neurons in MI of intact monkeys characteristi-
cally begin to modulate their firing before movement onset and
activity is tuned to hand movement direction19–21. To compare this
neural activity with MI of a human with spinal cord injury, MN
performed a step-tracking, ‘centre-out’ task using the neural cursor.
to one of four radially displaced targets (screen location: up, down,
left, right; see Supplementary Video 1). For each of six sessions, MN
performed this task by imagining hand motion (see Methods) as
after filter building without intervening practice. Timing and direc-
tional tuning features of MI neurons during imagined actions were
consistent with those observed in MI of intact non-human primates.
Figure 4 shows that spike-rate modulation occurs soon after the ‘go’
cue and that modulation varied by target location, as would
be predicted for MI if actual arm motions were performed17.
Furthermore, 66 out of 73 discriminated units (90.4%) significantly
changed their firing rate in relation to the appearance of the go cue
(Kolmogorov–Smirnov test, a ¼ 0.05, rate calculated over a sliding
1-s window, overlapping every 0.05s; 60-s data set for each con-
dition, n ¼ 3 sessions). These results indicate that, even years after
spinal cord injury and in the absence of kinaesthetic feedback and
limb movement,MIneuronscan stillbeactivelyengaged andencode
task-related information during the intention to move the limb
ordinarily controlled by that MI region.
Quality of neural cursor control
Neural cursor position was significantly correlated with technician
(x coordinate r2¼ 0.56 ^ 0.18 and y coordinate r2¼ 0.45 ^ 0.15,
n ¼ 6 sessions, Fig. 5). These correlations are similar or better than
those seen in intact monkeys when linear filters were used to predict
real-time hand position from MI neuronal ensembles5,22. The neural
cursor could be directed towards targets with a form qualitatively
similar to that seen for intact monkeys using closed-loop neural
control17,18,23. As in intact monkeys, neural cursor motion had
underlying instabilities and variable oscillatory components com-
pared to hand motions of able-bodied individuals. Continuous
neural cursor motion with the linear filter made cursor fixation at
a single location difficult to achieve.
Data from the centre-out task were used to evaluate the speed and
accuracy of cursor control, which are essential design parameters for
any future practical NMP. As shown in Fig. 6, the participant
correctly acquired 73–95% of targets (control 6.5%; n ¼ 80, paired
t-test, P , 0.0001, see Methods) when measured in a series of six
sessions (see also Supplementary Fig. 2). Performance errors
reflected both instabilities in cursor direction control and the ability
to hold at the target location. Mean time to target was 2.51 ^ 0.16s
(^s.e.m.) for successfully acquired targets. Although the best 13% of
MN’s trials were within the range consistently achieved by able-
bodied controls using acomputermouse (n ¼ 3, mean 1.06 ^ 0.08s
(^s.e.m.)), the distribution of times for MN using neural control is
skewed to longer acquisition times (Fig. 6b). Effective use of the
MN directed the cursor to randomly placed targets while attempting
to avoid obstacles in the cursor’s path (see Supplementary Video 5).
Figure 4 | Directional tuning during centre-out task. Peristimulus time
histograms show spike rates for five neurons recorded simultaneously
or bottom of the screen. Twenty trials are displayed for each target location.
Increases in activity after the go cue demonstrate movement-intention-
related modulation. Each column shows the firing of one unit in the four
directions, aligned on the cue to move. Note, for example, the time-locked
(lower right corner) and the lack of change in firing rate for upward
instruction. Changes in firing across the five neurons reveal directional
NATURE|Vol 442|13 July 2006
© 2006 Nature Publishing Group
Further information concerning spatial and temporal accuracy was
obtained from a ‘grid task’ (see Supplementary Information). As
shown in Supplementary Video 8, participant 2 also performed the
centre-out task, highlighting the ability for this second participant
with spinal cord injury to modulate his motor cortical activity
voluntarily for external device control. This participant’s neural
control, however, was generally less accurate and consistent than
MN’s; we are investigating the extent to which technical or other
factors may underlie this performance. This and other participants’
data will be reported in subsequent manuscripts.
Although he is tetraplegic, MN retains shoulder, neck and head
(see Supplementary Fig. 1). NMPs will nearly always operate in the
context of some existing movement capabilities (given the consider-
able variability of remaining sensory and motor functions in people
with CNS injury); thus, during filter building and use MN was not
asked to remain completely still, and he sometimes moved his head
action, much as such movements are known to influence activity
correlated with ordinary hand actions24. These still-intact move-
ments, however, did not appear to be essential for MN’s cursor
control, as can be appreciated particularly when he played ‘Neural
times MN moved the cursor purposefully while his head remained
stationary, and during other epochs his head moved but the cursor
motion appeared to be unrelated to head movement. We compared
neural cursor position in Supplementary Video 4 to head position
(both assessed by coordinates on a video playback monitor) and
found no consistent relationship (r2¼ 0.063). This finding is incon-
sistent with a unique causal relation between head and cursor
control. In addition, it was our repeated observation that, at least
some of the time, MN performed neural control (and open-loop)
tasks without moving his shoulder. Thus, based upon our instruc-
tions and MN’s self-reporting that he was actively imagining arm
action, we conclude that cursor motion is under the guidance of
Direct control of prosthetic devices
Continuous computercursor control could be used to provide many
valuable new outputs for a person with paralysis to carry out
Figure 5 | Reconstruction of neural cursor position during pursuit
a 5-s epoch during which MN was asked to track the technician cursor with
his neural cursor in real time. MN was able to track the general direction of
the technician cursor with the neural cursor, changing directions quickly,
whilehavingsome difficulty inoverlaying thecursorsprecisely.Trialday90.
b, x, y position control over time during one tracking trial (last 1-min epoch
of filter building). The top panel displays the x coordinate position of the
target acquisition/obstacle avoidance task. The four panels represent the
the neural cursor and illustrates the ability to avoid most obstacles and
acquire most targets within a randomly arranged field. Data are from trial
Figure 6 | Centre-out task performance. a, Target acquisition accuracy
during the centre-out task. For each of six sessions, MN acquired between
73–95% of the radially placed targets. Control targets were not present on
the monitor during task performance, but were marked as acquired if,
during post-hoc analysis of the cursor movement, the cursor had traversed
the location of one of the other three pseudo-randomly selected targets
before the correct target (see Supplementary Video 1). Data from days 72,
77, 83, 84, 86, 90 are shown. b, Time-to-target performance during centre-
target acquisitions in ,7s are shown for MN. Arrows on the abscissa
represent median times to target for each distribution. Controls’
performances (n ¼ 3 controls, 80 trials each) are collapsed into 0.2-s bins.
MN’s performance (398 trials) is collapsed into 0.5-s bins for visual clarity.
NATURE|Vol 442|13 July 2006
© 2006 Nature Publishing Group
activities of daily living. Such a control signal could be used not only
to direct computer software, but also to operate external physical
assistive devices. MN tested his ability to perform potentially useful
actions in a series of demonstrations.
and to draw an approximately circular figure using a paint program
(Supplementary Video 2). Using the neural cursor coupled to a
simple hardware interface, he adjusted the volume, channel and
power to his television. He was also able to play video games such as
devices was achieved, allowing MN to manipulate directly the
environment. In one example, neural output was coupled to a
prosthetic hand (Liberating Technologies, Inc.; see Methods) and
MN was able to open and close the hand under volitional control
(see Supplementary Video 6). Although only a one-dimensional
form of proportional control, he achieved this action after a few
trials while looking at the hand and without requiring feedback
from the cursor display. Lastly, MN used a simple multi-jointed
robotic limb to grasp an object and transport it from one location
to another (see Supplementary Video 7 and Supplementary Infor-
mation). These demonstrations illustrate transfer of control
directly to a physical device without depending on continued
viewing of computer cursor feedback, and suggest that manipu-
lation of the environment to enable activities such as self-paced
eating, as well as other goal-directed actions, could be achieved in
Notably, each of these tasks was achieved rapidly and could be
performed while the participant was conversing. Thus, the MI-based
NMP may have the property of allowing external device controlwith
little more disruption than encountered in able-bodied humans
out other motor or cognitive functions.
The research shown here from the first participant in an ongoing
pilot clinical trial provides initial evidence that a human unable to
move or sense his limbs can operate a NMP using MI neuronal
ensemble spiking activity as a control source. In addition, we
demonstrate that neural spiking remains in the MI arm area and
can be modulated by intention years after spinal cord injury. These
findings provide a number of novel insights concerning cortical
function and the impact of spinal cord injury in humans (see
Although MN used direct neural control to perform reasonably
of an able-bodied person using a manually controlled computer
mouse. A number of factors might affect control, including: (1) the
small set of randomly selected neurons recorded by this single array,
of spinal cord injury mechanisms or duration since injury; (3) the
cortical layers recorded; (4) our approach to filter building; (5) the
adaptivealgorithms or support vector machine approaches25, not yet
testedinour work); (6)attention andmotivational state during filter
building or control; and (7) the user interface. Changes in the
recorded population across days may also contribute to both varia-
bility and instability of control. Shifting ensembles may result from
small motions of the array or through other poorly understood
mechanisms. Despite these variables, it is important to note that
useful filters could be created daily from that neural population, and
that advances in knowledge and technology are likely to improve
recording and decoding. For example, cursor control might be
enhanced further by adaptive algorithms or more selective choice
of neural signals. Combinations of spiking activity and simul-
taneously recorded LFPs would provide additional signals that
might improve performance.
Efforts to optimize performance may help to identify the design
criteria for a clinically useful interface for paralysed humans. For
example, electroencephalogram (EEG)-based control systems have
been improved by limiting trial time, re-centring the cursor location
after each trial, and stopping cursor motion as soon as targets were
hit26. By contrast, our participant had to maintain cursor control at
all times without these interface enhancements. However, we found
that trial success was greater when target dwell time was decreased
(grid task). Incorporation of additional feedback (for example,
visual, auditory, somatosensory) may also be useful in enhancing
Cursor and external device control may also be improved through
learning28. MIof intact animals shows learning-related plasticity29–32,
and such plasticity undoubtedly contributes to the acquisition of
direct cortical neuronal control of an NMP. Thus, we expect that
learning will further improve control, as has been reported in
monkeys18. The circle drawing task (Supplementary Video 2), for
example, provides some evidence of improved performance with
brief practice, but a better understanding of learning will require
additional, more systematic investigation.
Other human BCIs are under development33. Foremost for com-
results with their neurotrophic (cone) electrode for three humans:
one with amyotrophic lateral sclerosis34, one with brainstem
stroke35,36, and one with mitochondrial myopathy36(see Supplemen-
tary Information). In contrast to previous studies, here we present
evidencethat corticalensemble patterns canbedecodedandused for
real-time, two-dimensional control of a computer cursor and other
external devices by a person with spinal cord injury. Scalp-based
EEG-driven BCIs also have been tested with some success in people
of paralysis26,27,37–39. Such indirect systems use a substitute for motor
been achieved in paraplegic patients using independent modulation
of scalp-recorded signals26, this system requires significant learning,
application, but notably, not surgical placement of the sensor. It
also has limited scalability to multiple functions because two-
dimensional tasks appear to engage all controllable signals. In
contrast, the direct NMP can be used during natural activities such
as speech, and requires minimal learning beyond filter creation. It is
also plausible that NMPs could be scaled so that parallel commands
could be derived simultaneously from multiple sensors each in
separate cortical regions. If achieved, relatively independent outputs
potentially could emanate bilaterally from arm and leg areas to allow
for the reanimation of paralysed limbs via functional electrical
The relative merits of EEG, transcranial40, electrocorticographic
(ECoG)41–43and intracortically based BCIs (NMPs) continue to be
explored and expanded32,33. Myriad issues including efficacy, safety
(particularly with regards to surgery), reliability, longevity, required
training and support, cosmesis, cost and availability will undoubtedly
affect BCI success; individuals with impairments in communication
and/ormobility may prioritizethesefactorsdifferentlywhenchoosing
a BCI, and it is possible that combinations of techniques will provide
reported here) depend upon the insights and assistance of trained
experts. The need for this assistance must be eliminated through
The goal of NMPs and other BCI research is to create safe, reliable
and unobtrusive neural interfaces to devices that will restore the
communication, mobility, and independence of paralysed humans.
NMPs could provide significant advances over current technologies
because they engage natural substrates for control (that is, the MI
arm/hand area), do not encumber other actions, and do not require
extensive learning. It will be important to collect results from
additional participants in order to understand the generality of
these statements. Improvements in the decoder and user interface
NATURE|Vol 442|13 July 2006
© 2006 Nature Publishing Group
are likely to provide control closer to that achieved by able-bodied
humans. The potential to use implanted sensors to ‘re-route’ neural
signals may also prove valuable in stroke and other neurological
disorders where pathways are disconnected. The present NMP
system incorporates a transcutaneous connection that tethers a
participant to a bulky cart and requires operation by a trained
technician. A wireless, implantable and miniaturized system com-
bined with automation will be required for practical use. Emerging
and available technologies appear to be sufficient to overcome these
obstacles, although the challenges of creating a fully implantable
system may be formidable. The current data provide initial proof of
concept that spiking activity from neuronal ensembles can provide a
control signal after spinal cord injury sufficient to perform at least
basic operations for a human with tetraplegia, justifying further
engineering efforts towards a human NMP.
See Supplementary Information for additional Methods.
Approval for these studies was granted by the US Food and Drug Adminis-
tration (Investigational Device Exemption) and the Rhode Island Hospital, New
England, Rehabilitation Institute of Chicago, University of Chicago, and
Spaulding Rehabilitation Hospital Institutional Review Boards. The participant
described in this report has provided permission for photographs, videos and
portions of his protected health information to be published for scientific and
educational purposes. After completion of informed consent and medical and
surgical screening procedures, the 4 £ 4-mm array of electrodes was implanted
into the motor cortex using a pneumatic insertion technique8,44. Details of the
human surgical procedure are in preparation for publication.
BrainGate system. The sensor is a 10 £ 10 array of silicon microelectrodes that
protrude1mmfrom a 4 £ 4-mm platform(Fig. 1b).At manufacture, electrodes
had an impedance of 322 ^ 138kOhm (mean ^ s.d.). The array was implanted
onto the surface of the MI arm/hand region; electrodes penetrate into the cortex
to attempt to record neurons in intermediate layers. Recorded electrical signals
pass externally through a Ti percutaneous connector, which is secured to the
skull. Cablingattached to the connectorduring recording sessions routes signals
to external amplifiers and then to a series of computers in a cart that process the
signals and convert them into an output (the neural cursor) that can be viewed
by the participant on a computer monitor (Fig. 1d). Currently, the system must
be set up and managed by an experienced technician.
Recording sessions. Research sessions were scheduled at least once per week at
the participant’s residence. Sessions could be cancelled (for example, due to
participant request or schedule conflict) or ended early at the participant’s
request. Sessions would commence with neural recording and spike discrimi-
nation, followed by filter building and structured clinical end-point (cursor
control) trials. Performance of video games and external device control demon-
filter building remained constant for any given session’s cursor control trials;
additional neural signals (detected either with alternative spike discrimination
methods, or as may have become evident during the cursor control trials) could
be used during subsequent video game or external device use.
Spike discrimination. Units were manually discriminated by a technician using
(Fig. 2) (Cyberkinetics Central Software) while the participant was at rest. We
applied no objective criteria to determine whether any particular unit (or
admixture of units) recorded on one day was the same as that recorded on a
subsequent day. Smaller amplitude signals were likely to have consisted of
admixtures of spikes from .1 neuron. These signals were nevertheless used for
Filter building. Foreach session, single and multiunit datawere used to createa
filter to generate a two-dimensional output signal. During filter building, MN
was asked to imagine moving his hand as if he were controlling a computer
mouse. Specifically, he was asked to imagine tracking a cursor on the computer
screen; this technician cursor was moved (by the technician) through a succes-
only the technician cursor and targets were visible on the screen. An initial filter
was built with the neural activity collected during this epoch, then allowing a
neural cursor to be placed on the screen. MN continued to track the technician
cursor with his neural cursor over a series of four 1-min blocks. The filter was
neural data were used from only the four neural cursor blocks.
Linear filters were constructed from a response matrix containing the firing
rate over a 1-s history for each neuron (twenty 50-ms bins), and regressing this
matrix onto technician-cursor position using a pseudoinverse technique. The
least-squares formulation comprises a closed-form solution:
u ¼ Rzf ¼ RðRTRÞ21RTk
(x,y position), T indicates the transpose of the matrix, and u is the reconstruc-
tion5. Software for displays was custom generated for this purpose and run on
standard PC computers (Orbit Micro, dual Intel Pentium 4 Xeon processors,
External devices. The robot armwas obtained from Lynxmotion, Lynx 5 Series.
The electric prosthetic hand was generously provided by Liberating Technologies.
Received 22 March; accepted 6 June 2006.
1.Jackson, A. B., Dijkers, M., Devivo, M. J. & Poczatek, R. B. A demographic
profile of new traumatic spinal cord injuries: change and stability over 30
years. Arch. Phys. Med. Rehabil. 85, 1740– -1748 (2004).
Thoumie, P. et al. Clinical and functional evaluation of a gaze controlled system
for the severely handicapped. Spinal Cord 36, 104– -109 (1998).
Humphrey, D. R., Schmidt, E. M. & Thompson, W. D. Predicting measures of
motor performance from multiple cortical spike trains. Science 170, 758– -762
Schwartz, A. B., Taylor, D. M. & Tillery, S. I. Extraction algorithms for cortical
control of arm prosthetics. Curr. Opin. Neurobiol. 11, 701– -707 (2001).
Paninski, L., Fellows, M. R., Hatsopoulos, N. G. & Donoghue, J. P.
Spatiotemporal tuning of motor cortical neurons for hand position and velocity.
J. Neurophysiol. 91, 515– -532 (2004).
Maynard, E. M., Nordhausen, C. T. & Normann, R. A. The Utah intracortical
electrode array: a recording structure for potential brain-computer interfaces.
Electroencephalogr. Clin. Neurophysiol. 102, 228– -239 (1997).
Guillory, K. S. & Normann, R. A. A 100-channel system for real time detection
and storage of extracellular spike waveforms. J. Neurosci. Methods 91, 21– -29
Suner, S., Fellows, M. R., Vargas-Irwin, C., Nakata, K. & Donoghue, J. P.
Reliability of signals from chronically implanted, silicon-based electrode array
in non-human primate primary motor cortex. IEEE Trans. Neural Syst. Rehabil.
Eng. 13, 524– -541 (2005).
Maynard, F. M. Jr et al. International standards for neurological and functional
classification of spinal cord injury. American Spinal Injury Association. Spinal
Cord 35, 266– -274 (1997).
10. Yousry, T. A. et al. Localization of the motor hand area to a knob on the
precentral gyrus. A new landmark. Brain 120, 141– -157 (1997).
11.Lotze, M., Laubis-Herrmann, U., Topka, H., Erb, M. & Grodd, W. Reorganization
in the primary motor cortex after spinal cord injury—A functional magnetic
resonance (fMRI) study. Restor. Neurol. Neurosci. 14, 183– -187 (1999).
12. Shoham, S., Halgren, E., Maynard, E. M. & Normann, R. A. Motor-cortical
activity in tetraplegics. Nature 413, 793 (2001).
13. Sabbah, P. et al. Sensorimotor cortical activity in patients with complete spinal
cord injury: a functional magnetic resonance imaging study. J. Neurotrauma 19,
53– -60 (2002).
14. Mikulis, D. J. et al. Adaptation in the motor cortex following cervical spinal cord
injury. Neurology 58, 794– -801 (2002).
15. Donoghue, J. P., Sanes, J. N., Hatsopoulos, N. G. & Gaal, G. Neural discharge
and local field potential oscillations in primate motor cortex during voluntary
movements. J. Neurophysiol. 79, 159– -173 (1998).
16. Sanes, J. N. & Donoghue, J. P. Plasticity and primary motor cortex. Annu. Rev.
Neurosci. 23, 393– -415 (2000).
17. Serruya, M. D., Hatsopoulos, N. G., Paninski, L., Fellows, M. R. & Donoghue, J. P.
Instant neural control of a movement signal. Nature 416, 141– -142 (2002).
18. Taylor, D. M., Tillery, S. I. & Schwartz, A. B. Direct cortical control of 3D
neuroprosthetic devices. Science 296, 1829– -1832 (2002).
19. Evarts, E. V. in Handbook of Physiology (ed. Brooks, V.) 1083– -1120 (Williams
and Wilkins, Baltimore, 1981).
20. Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations
between the direction of two-dimensional arm movements and cell discharge
in primate motor cortex. J. Neurosci. 2, 1527– -1537 (1982).
21. Humphrey, D. R. & Tanji, J. in Motor Control: Concepts and Issues
(eds Humphrey, D. R. & Freund, H. J.) 413– -443 (John Wiley, London, 1991).
22. Wu, W. et al. Modeling and decoding motor cortical activity using a switching
Kalman filter. IEEE Trans. Biomed. Eng. 51, 933– -942 (2004).
23. Carmena, J. M. et al. Learning to control a brain-machine interface for reaching
and grasping by primates. PLoS Biol. 1, E42 (2003).
24. Baker, J. T., Donoghue, J. P. & Sanes, J. N. Gaze direction modulates finger
movement activation patterns in human cerebral cortex. J. Neurosci. 19,
10044– -10052 (1999).
25. Olson, B. P., Si, J., Hu, J. & He, J. Closed-loop cortical control of direction using
support vector machines. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 72– -80 (2005).
26. Wolpaw, J. R. & McFarland, D. J. Control of a two-dimensional movement
signal by a noninvasive brain-computer interface in humans. Proc. Natl Acad.
Sci. USA 101, 17849– -17854 (2004).
NATURE|Vol 442|13 July 2006
© 2006 Nature Publishing Group
27. Birbaumer, N. et al. A spelling device for the paralysed. Nature 398, 297– -298
28. Fetz, E. E. & Baker, M. A. Operantly conditioned patterns on precentral unit
activity and correlated responses in adjacent cells and contralateral muscles.
J. Neurophysiol. 36, 179– -204 (1973).
29. Fetz, E. E. & Finocchio, D. V. Operant conditioning of specific patterns of neural
and muscular activity. Science 174, 431– -435 (1971).
30. Jacobs, K. M. & Donoghue, J. P. Reshaping the cortical motor map by
unmasking latent intracortical connections. Science 251, 944– -947 (1991).
31. Gandolfo, F., Li, C., Benda, B. J., Schioppa, C. P. & Bizzi, E. Cortical correlates of
learning in monkeys adapting to a new dynamical environment. Proc. Natl Acad.
Sci. USA 97, 2259– -2263 (2000).
32. Schwartz, A. B. Cortical neural prosthetics. Annu. Rev. Neurosci. 27, 487– -507
33. Hochberg, L. R. & Donoghue, J. P. Sensors for brain-computer interfaces. IEEE
Eng. Med. Biol. (in the press).
34. Kennedy, P. R. & Bakay, R. A. Restoration of neural output from a paralyzed
patient by a direct brain connection. Neuroreport 9, 1707– -1711 (1998).
35. Kennedy, P. R., Bakay, R. A., Moore, M. M., Adams, K. & Goldwaithe, J. Direct
control of a computer from the human central nervous system. IEEE Trans.
Rehabil. Eng. 8, 198– -202 (2000).
36. Kennedy, P. R., Kirby, M. T., Moore, M. M., King, B. & Mallory, A. Computer
control using human intracortical local field potentials. IEEE Trans. Neural Syst.
Rehabil. Eng. 12, 339– -344 (2004).
37. Mason, S. G., Bohringer, R., Borisoff, J. F. & Birch, G. E. Real-time control of a
video game with a direct brain– -computer interface. J. Clin. Neurophysiol. 21,
404– -408 (2004).
38. Vaughan, T. M. et al. Brain-computer interface technology: a review of the
Second International Meeting. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 94– -109
39. Muller-Putz, G. R., Scherer, R., Pfurtscheller, G. & Rupp, R. EEG-based
neuroprosthesis control: a step towards clinical practice. Neurosci. Lett. 382,
169– -174 (2005).
40. Kennedy, P. et al. Using human extra-cortical local field potentials to control a
switch. J. Neural Eng. 1, 72– -77 (2004).
41. Leuthardt, E. C., Schalk, G., Wolpaw, J. W., Ojemann, J. G. & Moran, D. W. A
brain-computer interface using electrocorticographic signals in humans.
J. Neural Eng. 1, 63– -71 (2004).
42. Graimann, B., Huggins, J. E., Levine, S. P. & Pfurtscheller, G. Toward a direct
brain interface based on human subdural recordings and wavelet-packet
analysis. IEEE Trans. Biomed. Eng. 51, 954– -962 (2004).
43. Lal, T. N., et al. in Advances in Neural Information Processing Systems
(eds Saul, L. K., Weiss, Y. & Bottou, L.) (MIT Press, Cambridge, Massachusetts,
44. Rousche, P. J. & Normann, R. A. A method for pneumatically inserting an array of
penetrating electrodes into cortical tissue. Ann. Biomed. Eng. 20, 413– -422 (1992).
Supplementary Information is linked to the online version of the paper at
Acknowledgements The authors thank J. Joseph and D. Morris for assistance;
L. Mermel for clinical planning advice; V. Zerris and M. Park for surgical
assistance; G. Polykoff for clinical trial assistance; W. Truccolo for power
spectral density analysis development; and the employees of Cyberkinetics for
device engineering, manufacturing and clinical trial design and management.
The authors also thank MN for his participation in this trial, and the nursing staff
at his assisted care facility for their assistance. The authors are grateful to
M. Serra and Sargent Rehabilitation Center, the study site, for administrative
support. The photograph of MN (Fig. 1) is copyright 2005 Rick Friedman. This
work was supported by Cyberkinetics Neurotechnology Systems, Inc.
Author Information Reprints and permissions information is available at
npg.nature.com/reprintsandpermissions. The authors declare competing
financial interests: details accompany the paper on www.nature.com/nature.
Correspondence and requests for materials should be addressed to J.P.D.
NATURE|Vol 442|13 July 2006