# Adaptive decoding for brain-machine interfaces through Bayesian parameter updates.

**ABSTRACT** Brain-machine interfaces (BMIs) transform the activity of neurons recorded in motor areas of the brain into movements of external actuators. Representation of movements by neuronal populations varies over time, during both voluntary limb movements and movements controlled through BMIs, due to motor learning, neuronal plasticity, and instability in recordings. To ensure accurate BMI performance over long time spans, BMI decoders must adapt to these changes. We propose the Bayesian regression self-training method for updating the parameters of an unscented Kalman filter decoder. This novel paradigm uses the decoder's output to periodically update its neuronal tuning model in a Bayesian linear regression. We use two previously known statistical formulations of Bayesian linear regression: a joint formulation, which allows fast and exact inference, and a factorized formulation, which allows the addition and temporary omission of neurons from updates but requires approximate variational inference. To evaluate these methods, we performed offline reconstructions and closed-loop experiments with rhesus monkeys implanted cortically with microwire electrodes. Offline reconstructions used data recorded in areas M1, S1, PMd, SMA, and PP of three monkeys while they controlled a cursor using a handheld joystick. The Bayesian regression self-training updates significantly improved the accuracy of offline reconstructions compared to the same decoder without updates. We performed 11 sessions of real-time, closed-loop experiments with a monkey implanted in areas M1 and S1. These sessions spanned 29 days. The monkey controlled the cursor using the decoder with and without updates. The updates maintained control accuracy and did not require information about monkey hand movements, assumptions about desired movements, or knowledge of the intended movement goals as training signals. These results indicate that Bayesian regression self-training can maintain BMI control accuracy over long periods, making clinical neuroprosthetics more viable.

**0**Bookmarks

**·**

**173**Views

- [Show abstract] [Hide abstract]

**ABSTRACT:**In a closed-loop brain-computer interface (BCI), adaptive decoders are used to learn parameters suited to decoding the user’s neural response. Feedback to the user provides information which permits the neural tuning to also adapt. We present an approach to model this process of co-adaptation between the encoding model of the neural signal and the decoding algorithm as a multi-agent formulation of the linear quadratic Gaussian (LQG) control problem. In simulation we characterize how decoding performance improves as the neural encoding and adaptive decoder optimize, qualitatively resembling experimentally demonstrated closed-loop improvement. We then propose a novel, modified decoder update rule which is aware of the fact that the encoder is also changing and show it can improve simulated co-adaptation dynamics. Our modeling approach offers promise for gaining insights into co-adaptation as well as improving user learning of BCI control in practical settings.Neural Information Processing Systems (NIPS); 12/2013 - [Show abstract] [Hide abstract]

**ABSTRACT:**Modern wearable robots are not yet intelligent enough to fully satisfy the demands of end-users, as they lack the sensor fusion algorithms needed to provide optimal assistance and react quickly to perturbations or changes in user intentions. Sensor fusion applications such as intention detection have been emphasized as a major challenge for both robotic orthoses and prostheses. In order to better examine the strengths and shortcomings of the field, this paper presents a review of existing sensor fusion methods for wearable robots, both stationary ones such as rehabilitation exoskeletons and portable ones such as active prostheses and full-body exoskeletons. Fusion methods are first presented as applied to individual sensing modalities (primarily electromyography, electroencephalography and mechanical sensors), and then four approaches to combining multiple modalities are presented. The strengths and weaknesses of the different methods are compared, and recommendations are made for future sensor fusion research.Robotics and Autonomous Systems 09/2014; · 1.11 Impact Factor - SourceAvailable from: Xiaoling Hu[Show abstract] [Hide abstract]

**ABSTRACT:**Successful neurological rehabilitation depends on accurate diagnosis, effective treatment, and quantitative evaluation. Neural coding, a technology for interpretation of functional and structural information of the nervous system, has contributed to the advancements in neuroimaging, brain-machine interface (BMI), and design of training devices for rehabilitation purposes. In this review, we summarized the latest breakthroughs in neuroimaging from microscale to macroscale levels with potential diagnostic applications for rehabilitation. We also reviewed the achievements in electrocorticography (ECoG) coding with both animal models and human beings for BMI design, electromyography (EMG) interpretation for interaction with external robotic systems, and robot-assisted quantitative evaluation on the progress of rehabilitation programs. Future rehabilitation would be more home-based, automatic, and self-served by patients. Further investigations and breakthroughs are mainly needed in aspects of improving the computational efficiency in neuroimaging and multichannel ECoG by selection of localized neuroinformatics, validation of the effectiveness in BMI guided rehabilitation programs, and simplification of the system operation in training devices.BioMed Research International 09/2014; · 2.71 Impact Factor

Page 1

LETTER

Communicated by Byron Yu

Adaptive Decoding for Brain-Machine Interfaces Through

Bayesian Parameter Updates

Zheng Li

zheng@cs.duke.edu

Joseph E. O’Doherty

jeo4@duke.edu

Mikhail A. Lebedev

lebedev@neuro.duke.edu

Department of Neurobiology and Center for Neuroengineering, Duke University,

Durham, NC 27710, U.S.A.

Miguel A. L. Nicolelis

nicoleli@neuro.duke.edu

Departments of Neurobiology, Biomedical Engineering, and Psychology and

Neuroscience, and Center for Neuroengineering, Duke University, Durham,

NC 27719, U.S.A.; and Edmond and Lily Safra International Institute

of Neuroscience of Natal, 59066-060 Natal, Brazil

Brain-machine interfaces (BMIs) transform the activity of neurons

recorded in motor areas of the brain into movements of external actu-

ators. Representation of movements by neuronal populations varies over

time, during both voluntary limb movements and movements controlled

through BMIs, due to motor learning, neuronal plasticity, and instability

in recordings. To ensure accurate BMI performance over long time spans,

BMI decoders must adapt to these changes. We propose the Bayesian

regression self-training method for updating the parameters of an un-

scented Kalman filter decoder. This novel paradigm uses the decoder’s

output to periodically update its neuronal tuning model in a Bayesian

linear regression. We use two previously known statistical formulations

of Bayesian linear regression: a joint formulation, which allows fast and

exact inference, and a factorized formulation, which allows the addition

and temporary omission of neurons from updates but requires approx-

imate variational inference. To evaluate these methods, we performed

offline reconstructions and closed-loop experiments with rhesus mon-

keys implanted cortically with microwire electrodes. Offline reconstruc-

tions used data recorded in areas M1, S1, PMd, SMA, and PP of three

monkeys while they controlled a cursor using a handheld joystick. The

Bayesian regression self-training updates significantly improved the ac-

curacy of offline reconstructions compared to the same decoder without

updates. We performed 11 sessions of real-time, closed-loop experiments

Neural Computation 23, 3162–3204 (2011)

c ?2011 Massachusetts Institute of Technology

Page 2

Bayesian Adaptive Decoding for Brain-Machine Interfaces 3163

with a monkey implanted in areas M1 and S1. These sessions spanned 29

days.Themonkeycontrolledthecursorusingthedecoderwithandwith-

outupdates.Theupdatesmaintainedcontrolaccuracyanddidnotrequire

informationaboutmonkeyhandmovements,assumptionsaboutdesired

movements, or knowledge of the intended movement goals as training

signals. These results indicate that Bayesian regression self-training can

maintain BMI control accuracy over long periods, making clinical neuro-

prosthetics more viable.

1 Introduction

Brain-machine interfaces (BMI) are systems that directly connect brains to

external devices, for example, prosthetic limbs for persons with paralysis

(Nicolelis, 2003). While BMI systems have to perform accurately over long

time periods, changes in the properties of recorded neuronal population

maycausesignificantalterationsinBMIoperation.Studieshaveshownthat

neuronal tuning to arm movements may change over time spans as short as

1hour(Li,Padoa-Schioppa,&Bizzi,2001;Padoa-Schioppa,Li,&Bizzi,2004;

Lebedev et al., 2005; Kim et al., 2006; Rokni, Richardson, Bizzi, & Seung,

2007; Chestek et al., 2007). Changes in neuronal tuning over time have been

also reported for BMI control (Taylor, Tillery, & Schwartz, 2002; Truccolo,

Friehs,Donoghue,&Hochberg,2008).Additionally,neuronalandrecording

instability can cause changes to the shape of recorded spike waveforms.

Thesechangesmaycausethespike-sortingsystemtoerrandleadtochanges

in the effective tuning of recorded units. For these reasons, BMI systems

needadaptivedecodingtomaintaincontrolaccuracyoverlongtimeperiods

(Kim et al., 2006; Wu and Hatsopoulos, 2008). In addition to the ability to

track neuronal variability, adaptive decoding has the benefit of exploiting

plastic changes in neuronal tuning that occur during learning and thus

facilitate coadaptive (Taylor et al., 2002) synergies between the user’s brain

and the BMI.

Early computational methods for translating recorded neuronal sig-

nals into movement commands assumed static neuronal tuning (Wessberg

et al., 2000; Serruya, Hatsopoulos, Paninski, Fellows, & Donoghue , 2002;

Carmena et al., 2003; Hochberg et al., 2006; Wu, Gao, Bienenstock,

Donoghue, & Black, 2006; Ganguly & Carmena, 2009). These methods fixed

the parameters of the neuronal decoder after an initial period of algorith-

mic training or parameter fitting. Such decoders required users to adapt to

the BMI control system by changing their neuronal modulations, and these

changes were not tracked by the decoder.

Somestudieshaveimplementedadaptiveneuronaldecoders(Sykacek&

Roberts, 2002; Eden, Frank, Barbieri, Solo, & Brown, 2004; Rickert, Braun, &

Mehring,2007;Srinivasan,Eden,Mitter,&Brown,2007;Wu&Hatsopoulos,

2008; Shpigelman, Lalazar, & Vaadia, 2009; Gilja et al., 2009; Liao et al.,

Page 3

3164 Z. Li, J. O’Doherty, M. Lebedev, and M. Nicolelis

2009). However, most of these neuronal decoders have assumed that the

desired movements—for example, the trajectory from the initial position of

the actuator to a target—were available to the update method. The update

method then adjusted the decoder parameters such that the BMI output

more closely approximated the desired movement. One approach to ensure

the availability of desired movements is to instruct the prosthetic user to

attempt prescribed movements during periodic retraining sessions.

A more versatile approach to decoder update is to shift the paradigm

from assuming a priori knowledge of desired movements to using a com-

putational method that updates the decoder’s model automatically, with

little or no external intervention. The goal of this study is to design and

validate an update method that can achieve this objective.

We propose the Bayesian regression self-training method for adaptive

decoding. This method periodically updates (every 2 minutes in our study)

the parameters of the model of neuronal tuning while decoding user inten-

tions.Duringeachupdate,theprocedurecombinesinformationfromprevi-

ous model parameters, recent neuronal activity, and outputs of the decoder

to compute new model parameters. Instead of using a priori knowledge of

the desired movements, the updates use the decoder outputs as substitutes

for the desired movements (similar to Gilja et al., 2009; Liao et al., 2009).

Bayesianlinearregression(Bishop,2006;Rencher&Schaalje,2008)between

decoderoutputsandneuronalactivity,withthepreviousparametersacting

as priors, calculates the new parameters. We chose Bayesian linear regres-

sion because it combines previous parameters and tuning changes detected

from recent neuronal activity in a probabilistic manner, so that the new

parameters are the posteriors of a Bayesian update. We also formulate a

transition model for the tuning parameters. Together, the Bayesian regres-

sion update and transition model form a batch-mode Bayesian filter for

the tuning parameters. For the decoding algorithm, we use the nth order

unscented Kalman filter (Li et al., 2009). Our decoder and parameter up-

date system simultaneously predicts the BMI user’s desired movements

and tracks tuning changes in the user’s neuronal modulation.

Unlike previous, nonprobabilistic update approaches (Wu & Hatsopou-

los, 2008; Shpigelman et al., 2009; Gilja et al., 2009), our approach auto-

matically takes into account the uncertainties of the tuning model param-

eters and the amount of useful information and noise in the new data to

compute Bayesian updates of tuning parameters. Unlike previous studies

with parameter updates at every decoding time step (Eden et al., 2004;

Rickert et al., 2007; Srinivasan et al., 2007; Wu & Hatsopoulos, 2008), our

batch-modeapproachallowstheuseoftheKalmansmoother(Rauch,Tung,

&Striebel,1965)toimprovedecoderpredictionsbeforetheiruseinupdates.

The Kalman smoother uses the model of movement change over time to

refine predictions of desired movements at a time t using predictions of de-

siredmovementsattimesaftert.Thenoncausalnatureofthispreprocessing

step means it is available only in batch-mode operations.

Page 4

Bayesian Adaptive Decoding for Brain-Machine Interfaces3165

We tested our method with rhesus monkeys that performed motor tasks

manually or through a BMI. We started with offline reconstructions of

handtrajectoriesfromneuronaldatapreviouslyrecordedinthreemonkeys.

We then conducted experiments in which one monkey used our adaptive

methodinareal-time,closed-loopBMI.Adaptingwithineachsessioninthe

offline reconstructions, our decoder was more accurate than the equivalent

unscented Kalman filter without updates. Adapting within and across ses-

sionsintheclosed-loopexperiments,ouradaptivedecodermaintainedBMI

control accuracy over 29 days, while the unscented Kalman filter without

updates degraded in accuracy. These results indicate that Bayesian regres-

sion self-training can cope with changes in neuronal tuning or neuronal

recording without the need for example movements. Bayesian regression

self-training improves the robustness of a BMI to changes in neuronal tun-

ing, making long-term BMI operation more reliable.

2 Neurophysiological Methods

2.1 Neuronal Recordings. We conducted experiments in three macaque

monkeys(Macacamulatta)implantedwithmicrowirearraysinmultiplearm

representationareasofthecortex.Allsurgicalandexperimentalprocedures

conformed to the National Research Council’s Guide for the Care and Use

of Laboratory Animals (1996) and were approved by the Duke University

AnimalCareandUseCommittee.Wetrainedthemonkeystomoveacursor

on a computer screen either using a handheld joystick or through a BMI

that generated cursor position from neuronal activity.

Figure 1A showsadiagramof electrodegeometriesand array placement

for monkeys C, G, and M. All implants were custom-made by engineers

from our lab (Lehew & Nicolelis, 2008). Monkeys C and G had similar

implants. Monkey C was implanted with four 32-microwire arrays in the

primarymotorcortex(M1),dorsalpremotorcortex(PMd),posteriorparietal

cortex (PP), and supplementary motor area (SMA) in the right hemisphere.

Monkey G was implanted with six 32-microwire electrode arrays in M1,

primary somatosensory cortex (S1), and PMd of both hemispheres. Elec-

trodes in each array were grouped into pairs and placed on a uniform grid,

with a 1 mm separation between pairs. The electrodes in each pair were

placed tightly together, and one electrode was 300 microns longer than

the other. The recording site on each electrode was at the tip. The material

anddiameteroftheelectrodesvaried,andthelongerelectrodehadequalor

greaterdiameter.MonkeyCwasimplantedwithstainlesssteelandtungsten

electrodeswithdiameters46and51microns,respectively,inareasSMAand

M1 and tungsten electrodes with a diameter of 51 microns in areas PMd

and PP. Monkey G was implanted with stainless steel electrodes measuring

40 and 63 microns in diameter.

Monkey M was implanted with four 96-microwire arrays in M1 and

S1 bilaterally in arm and leg regions so that one array corresponded to the

Page 5

3166 Z. Li, J. O’Doherty, M. Lebedev, and M. Nicolelis

Figure 1: Implants and experimental methods. (A) Implant placement and ge-

ometries. Microwire diameters are specified in microns. To reduce clutter, we

show only arrays from which data were used. (B) Primate experiment setup.

(C) Tuning model update paradigm. Updates occur periodically using the neu-

ronal data and decoder outputs generated since the previous update. (D) Three

behavioral tasks. Each task required placing the cursor over a target. In center-

out, targets were stationary. In the pursuit tasks, targets moved continuously

according to a Lissajous curve or a set of invisible, random waypoints.

corticalregionsassociatedwitheachlimb.Inthisstudy,weusedrecordings

onlyfromtherighthemispherearmarray,sincemonkeyMmanipulatedthe

joystickwithitslefthand.WithineacharrayformonkeyM,microwireswere

grouped in two 4 × 4 uniformly spaced grids of 16 triplets of electrodes.

One grid was placed over M1, and one grid was placed over S1. The grids

were separated by 2 mm, which bridged the gap of the central sulcus. The

separation between triplets of electrodes within each grid was 1 mm. The

electrodesofeachtriplethadthreedifferentlengths,staggeredat300micron

Page 6

Bayesian Adaptive Decoding for Brain-Machine Interfaces3167

intervals, and recorded from the tips. The penetration depth of each triplet

could be adjusted as the triplets could be moved individually in the depth

axis, but the depth separation of the triplets was fixed. Electrodes were

inserted into the brain 1 to 2 weeks after surgery (during which the array

base was implanted on the surface of the brain). Data used in this study

were collected while the longest electrodes in all triplets were set to 2 mm

in length. The electrodes were constructed of stainless steel. The electrode

diameters in the array were heterogeneous, with each microwire having a

diameter of 50 to 63 microns.

To extract motor parameters from neuronal activity, we sampled data

from cortical sites contralateral to the arm that held the joystick. Data from

monkey C (total 5 sessions) were recorded from M1 in 5 daily experimental

sessions, PMd in 5 sessions, SMA in 5 sessions, and PP in 1 session. Data

from monkey G (total 8 sessions) were recorded from left M1 in 8 sessions,

leftS1in7sessions,leftPMdin8sessions,andrightPMdin5sessions.Data

from monkey M (total 16 sessions) were recorded from right M1 and right

S1 in all 16 sessions. We included neurons from S1 for predictions because

prior work has found that they can be used for decoding arm movements

(Carmenaetal.,2003).Forofflinereconstructionsofhandmovementsusing

neuronaldata,weused18experimentalsessions(monkeyC:5sessions,G:8

sessions, M: sessions), each with 94 to 240 (average 143) recorded neurons.

We conducted 11 closed-loop BMI control sessions (each recording 139

neurons) with monkey M. Extracellular neuronal signals were amplified,

digitized,and bandpassfiltered using Multichannel Acquisition Processors

(Plexon, Inc.). Neuronal action potentials were triggered using thresholds

and sorted online by the Multichannel Acquisition Processors, which ran a

template matching algorithm for neuronal waveform templates set by the

experimenter.

2.2 Behavioral Tasks. In experimental sessions, monkeys sat in a pri-

mate chair in front of a large screen, mounted 1.5 m in front of them (see

Figure1B).Therecordingsystemwasconnectedtotheimplantsusinglight,

flexiblewires,asthemonkeys’headswerenotrestrained.Ananalogjoystick

with 2 degrees of freedom (left-right and forward-backward) was mounted

vertically so that the monkeys grasped the joystick tip with their hands.

The joystick was 30 cm in length and allowed a maximum linear deflection

of 12 cm. Monkey C and monkey M controlled the joystick with their left

hands, and monkey G used its right hand. An electrical resistance touch

sensor or an optical touch sensor on the joystick tip determined whether

the monkeys were holding the joystick. An LCD projector, controlled by a

computer with custom-built BMI experimental software, projected white

visual stimuli on a black background on to the screen.

During hand control, monkeys moved a cursor (a ring 1.6 cm in diame-

ter) using the joystick. The joystick to cursor gain varied between 3.2X and

6.4X, depending on session. Targets appeared as rings 16 cm to 20.8 cm

Page 7

3168Z. Li, J. O’Doherty, M. Lebedev, and M. Nicolelis

in diameter. The median speeds at which monkeys C and G moved

the joystick were approximately 3.5 cm/s to 5.5 cm/s, depending on the

session. The median speed at which monkey M moved the joystick was

approximately 0.5 cm/s to 1.1 cm/s, depending on the session. The slower

movements of monkeys M were due to lower target speeds in the behav-

ioral task. We set the target speeds lower because we found that slower

movements were decoded more accurately, which is important for model

updates when decoded trajectories are used for the training signal. All ani-

malswereovertrained(atleast1yearofexperience)inperformingthetasks

with a joystick. Monkey M had 5 months of experience with BMI control

before the closed-loop sessions.

During BMI control, we decoded the monkey’s neuronal modulations

using an unscented Kalman filter (Li et al., 2009) to control the cursor. We

fitted the unscented Kalman filter’s parameters using data recorded dur-

ing hand control and updated the filter’s parameters using the Bayesian

regression update method (experimental condition) or left them fixed (con-

trol condition). The monkey had to grasp the joystick to signal participation

(otherwisethetaskpausedandstimuliwerenotshown),butjoystickmove-

ments were not used for cursor control.

The behavioral tasks required the monkeys to place the cursor over

the target, using either the joystick or BMI. We analyzed data from three

behavioraltasks:center-out,pursuitwithLissajoustrajectories,andpursuit

with point-to-point trajectories (see Figure 1D).

In the center-out task (see Figure 1D, top), the monkeys had to hold the

cursor in a stationary center target and then move to a peripheral target

randomly placed on a fixed-radius circle around the center target. After

holding at the peripheral target and receiving a juice reward, a new trial

began with the disappearance of the peripheral target and the appearance

of the center target. The intertrial interval that followed a successful trial

was 500 ms, and the intertrial interval after a failed trial was 700 to 1000 ms.

Holdtimesvariedpersessionfrom350to1050ms.Wetreateddatacollected

from this and all other tasks as a continuous stream and did not segment

by trial or movement onset.

In the pursuit task with Lissajous trajectories (see Figure 1D, middle), a

continuously moving target followed a Lissajous curve:

x(t)=A · sin(avt + δ),

y(t)=B · sin(bvt).

(2.1a)

(2.1b)

The variables x and y are the x- and y-axis coordinates, and t is time in

seconds. We used parameters a = 3, b = 4, δ = 0.5π, v ∈ {0.1,0.15,0.2} Hz,

and A = B = 22.4 cm (in screen scale). The x- and y-axis coordinates were

uncorrelated because the temporal frequency was different for each. To

receive periodic juice rewards, the monkeys had to keep the cursor over

Page 8

Bayesian Adaptive Decoding for Brain-Machine Interfaces3169

the moving target. Rewards were larger when the cursor was closer to the

center of the target to motivate the monkeys to pursue the target accurately.

This was done by scaling the amount of juice per reward (given every 750

msofholdinginsidethetarget)bythecubeofthedifferenceoftargetradius

and distance between cursor and target center.

In the pursuit task with point-to-point trajectories (see Figure 1D, bot-

tom), the target moved smoothly from one randomly generated waypoint

to another. The waypoint positions were drawn from a uniform distri-

bution on the 76.8 cm wide by 57.6 cm tall (in screen scale) work space

and were not visible to the monkeys. We limited the target’s acceleration

(≤3.2 cm/sec, screen scale) and velocity (≤9.6 cm/sec, screen scale) to en-

sure smooth trajectories. Rewards were larger when the cursor was closer

to the center of the target to motivate the monkey to pursue the target

accurately.

2.3 Data Preprocessing. To estimate the instantaneous firing rate, we

counted spikes in 100 ms wide, nonoverlapping (i.e., spaced at 100 ms)

time bins. The joystick position was sampled at 1 KHz and downsampled

to 10 Hz. Velocity was calculated from position by two-point digital dif-

ferentiation after the downsampling. We divided position, velocity, and

binned spike counts by their standard deviations. We subtracted the means

from position, velocity, and binned spike counts during closed-loop BMI

experiments; in offline reconstructions, instead of centering kinematics and

neuronal data, we added a bias term to the neuronal tuning (observa-

tion) model. During experiments, the monkeys’ heads were not restrained,

and they were allowed to take breaks from task participation. To exclude

data recorded while the monkeys did not participate, we used two meth-

ods. First, in both closed-loop and offline experiments, we excluded data

recorded while the monkey did not hold and move the joystick. If the joy-

stick did not move for more than 2 or 3 seconds, depending on the session,

the monkey was considered not to be participating. During sessions of BMI

control, the disconnected joystick was still held and moved by the monkey

(brain control with hand movements). Second, during closed-loop experi-

ments, we found that the monkey would sometimes hold the joystick but

look away from the screen and rest. We thus excluded data recorded while

the monkey looked away from the screen from update calculations. The

experimenter watched the monkey and toggled a button in the experiment

software when the monkey looked away.

We conducted offline analyses using Matlab software (MathWorks). For

closed-loopexperiments,weimplementedtheproposedadaptivedecoding

method in C++ as part of a custom-built BMI software suite running on a

desktop computer with an Intel Core i7 processor.

2.4 Evaluation Procedures. We quantified accuracy of reconstructions

and control using the signal-to-noise ratio (SNR). SNR is commonly used in

Page 9

3170 Z. Li, J. O’Doherty, M. Lebedev, and M. Nicolelis

engineeringandwasusedinpreviousstudiesonBMIs(Sanchezetal.,2002;

Kim et al., 2003). SNR is the inverse of the normalized mean squared error,

with larger numbers indicating more accurate control. SNR is typically

reported in decibels (dB), which transforms the ratio with a logarithm to

allow arithmetic averaging. The formula for the SNR is

SNRdB= 10 × log10

?VAR

MSE

?

,

(2.2)

where VAR is the variance of the desired values and MSE is the mean

squared error of the predicted values from the desired values. For offline

reconstructions, the desired values are the hand trajectories recorded by

the joystick, and the predicted values are the outputs of the decoder. For

closed-loop control, desired values are the target trajectories of the pursuit

task, and the predicted values are the output of the decoder. In this study,

we analyzed the SNR of the position values of the trajectories. We analyzed

x- and y-coordinates separately and combined them with arithmetic aver-

aging. When we calculated SNR in 1 minute windows of a session, we used

the same signal variance (numerator), the value from the entire session,

for every window, to allow comparison among windows. When compar-

ing accuracy among conditions, we computed the SNR in decibels for each

condition and then subtracted the values.

To ease comparison to previous work, the following equation provides

the approximate correlation coefficient (Pearson’s r) corresponding to an

SNR value in dB, under two assumptions: the mean of both the desired and

predictedsignalsis0,andthevariancesofthedesiredandpredictedsignals

are equal. Without these assumptions, correlation coefficient and SNR are

generally incomparable. The equation is

r ≈ 1 −

1

10SNRdB).

2 · 10(1

(2.3)

For the derivation and details on the approximations taken, see the

appendix.

3 Computational Methods

3.1 Overview. Bayesian regression self-training updates the model

of neuronal tuning to movements used by the decoder by periodically

performing Bayesian regression on the decoder output and neuronal activ-

ity (see Figure 1C). This paradigm uses the prediction algorithm’s output

to update its parameters, hence, we call it self-training. Self-training is fea-

sible for updating neuronal tuning models because of the redundancy in

motor representation in large populations of neurons (Narayanan, Kimchi,

Page 10

Bayesian Adaptive Decoding for Brain-Machine Interfaces 3171

& Laubach, 2005). For example, when a neuron changes its tuning, the de-

coder can still use the tuning of the rest of the population to predict desired

movements. The movement information conveyed by the changed neu-

ron’s modulation is drowned out by the movement information conveyed

bytheentirepopulation.Asmallerrorduetotheoutdatedtuningmodelfor

the changed neuron will contaminate predictions, since the Kalman filter

combines information from all the neurons in a weighted average. How-

ever, if the change in tuning is small or the number of neurons that change

simultaneously is small compared to the size of the population, the error

in predictions using the entire population will be small, allowing accurate

enough predictions with which Bayesian linear regression can update the

tuning model parameters of the changed neurons.

We use a fifth-order unscented Kalman filter (Julier, Uhlmann, &

Durrant-Whyte, 1995) with a quadratic model of neuronal tuning for the

decoder (see Li et al., 2009 for implementational details). This decoder is

an extension of the well-known Kalman filter (Kalman, 1960) Bayesian esti-

mation algorithm, which has been used for BMI decoding (Wu et al., 2003,

2006; Gilja et al., 2009, 2010). It differs from previous Kalman filter decoders

byincorporatingaquadraticmodelofneuronaltuning(Moran&Schwartz,

1999;Lietal.,2009)andtheunscentedtransform(Julieretal.,1995)approxi-

mationmechanismneededtocomputeposteriorswiththenonlineartuning

model. It also allows the tuning model to capture tuning across multiple

time offsets by keeping multiple time taps of kinematic variables in its state

(five time taps in this study).

Between tuning model updates, we store the output of the unscented

Kalman filter (UKF) decoder and the neuronal activity. Before each up-

date, we preprocess the decoder output by performing Kalman smoothing

(Rauch et al., 1965). This process improves the accuracy of the estimates

by propagating information from estimates of later states to estimates of

earlier states using the movement model. For example, if the user is trying

to move a BMI-controlled cursor to a particular location, the target location

(decoded later in time) can improve the estimate of the movement trajec-

tory. This implements a probabilistic mechanism for improving the data

used for updates. This approach is similar in spirit to the heuristic methods

found to be successful in prior work: using a point in between the BMI

cursor position and target as the training signal (Shpigelman et al., 2009)

and rotating the decoded velocity vector toward the target in the training

signal (Gilja et al., 2010).

The smoothed trajectories (independent variable) and the concurrent

neuronal activity (dependent variable) are the inputs to the Bayesian linear

regression. The Bayesian linear regression uses the previous tuning model

parameters as priors and computes the posteriors on the parameters given

thestoreddata.Theupdatesoccurinthebackground(e.g.,inalow-priority

thread) while the UKF continues to decode in real time using the existing

tuning model. Once the update completes, the UKF uses the new tuning

Page 11

3172Z. Li, J. O’Doherty, M. Lebedev, and M. Nicolelis

model and storing of outputs for the next update begins. To compute the

update, we use two formulations of Bayesian linear regression. One variant

usesajointdistributionforthetuningmodelcoefficientsandallowsanalyt-

icalcalculationofthesolution.Thesecondvariantismoreflexible,allowing

temporary omission of some neurons from updates and addition of newly

discovered neurons, but it requires a factorized distribution for the tuning

model coefficients and an approximate solution using variational Bayes.

3.2 Tuning Model. We used a fifth-order UKF with two future taps and

threepasttaps,whichmeanseachspikemayrelatetothedesiredmovement

occurring within a time difference of 300 ms (Li et al., 2009). This setting

is consistent with neurophysiological data and provides accurate decoding

without requiring too many parameters. The fifth-order UKF used only

one tap of kinematics in its movement model to minimize the decoding

contribution from exploiting patterns in the movement. The tuning model

described the normalized firing rate of a neuron as a linear combination

of the position, distance from the center of the work space, velocity, and

the magnitude of velocity (speed). The magnitude terms have been found

to model neuronal activity well (Moran & Schwartz, 1999) and improve

decoder accuracy (Li et al., 2009). The tuning model is

zi,t=

5

?

k=1

⎡

⎢

⎢

⎣

⎢

⎢

⎢

⎢

⎢

⎢

⎢

hi,1,k

hi,2,k

hi,3,k

hi,4,k

hi,5,k

hi,6,k

⎤

⎥

⎥

⎦

⎥

⎥

⎥

⎥

⎥

⎥

⎥

?⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

⎢

⎢

xpost−3+k

ypost−3+k

t−3+k+ ypos2

xvelt−3+k

yvelt−3+k

xvel2

?

xpos2

t−3+k

?

t−3+k+ yvel2

t−3+k

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

⎥

⎥

.

(3.1)

Here, zi,tis the predicted binned spike count of neuron i at time t, xpost

and ypostare the x and y coordinates of the cursor at time t, and xveltand

yveltare the x and y velocities of the cursor at time t. The distance and

speed terms make the tuning model quadratic in position and velocity and

require the use of an unscented Kalman filter for prediction. hi,1,1,...,hi,6,5

are 30 scalar parameters for neuron i. We use the N × 30 matrix H to hold

the parameters for all neurons in the population, where N is the number of

neurons. The k variable iterates over the time taps in the state variables (see

Li et al., 2009, for details of the nth-order model).

The model for the noise in the quadratic tuning model is

yi,t= zi,t+ vi,t,

(3.2)

Page 12

Bayesian Adaptive Decoding for Brain-Machine Interfaces3173

where yi,tis the actual binned spike count for neuron i at time t, and vi,t

are elements of a noise vector vt, which is drawn independently from a

multivariate normal distribution with zero mean and covariance R. R is

an N × N matrix. The matrices H and R constitute the tuning model or

observation model used by the UKF. To facilitate Bayesian updates of these

two matrices, we store them as probability distributions and operate on the

distribution parameters.

3.3 Bayesian Regression. The model for the multivariate linear regres-

sion between the vector of population neuronal activity y and the vector of

kinematic features x is

yt= Hxt+ ?? ∼ N(0,R).

(3.3)

To perform a Bayesian update of the parameters, we need a formulation

for the prior and posterior distributions of H and R. It is often desirable

for the prior and posterior distributions to be from the same distribution

family and differ only in their hyperparameters (conjugacy). Conjugacy is

advantageous because it may allow analytical computation of the hyperpa-

rameters of the posterior (Bernardo & Smith, 2000). After an update, we use

the mean of the posterior distribution as the tuning model for subsequent

decoding.

We examined two formulations for the distribution on H and R, which

allow conjugacy. The first formulation represents the prior and posterior

joint probability of H and R in a normal-inverse-Wishart distribution. This

formulation allows exact, analytical calculation of the Bayesian regression

posteriors, and the formulas for computing the posteriors are well-known

(Bishop,2006;Rencher&Schaalje,2008).However,itdoesnotallowupdates

onsubsetsofneuronsortheadditionofnewlydiscoveredneurons.Thus,we

exploredanalternativeformulation.Thealternativeformulationrepresents

the tuning coefficients of each neuron as a separate multivariate normal

distribution. Analytical computation of the regression on this formulation

is not possible, and we use a variational Bayesian solution. This method

is a multiple-predicted-variable extension of the single-predicted-variable

Bayesian linear regression model presented by Bishop (2006).

3.4 Joint Formulation. The probability model for Bayesian regression

with a joint distribution for H and R is shown in Figure 2A and specified

below:

yt|xt,H,R ∼ N?Hxt,R?,

(3.4a)

{vec(H),R} ∼ NW−1?vec(μ),?−1,?,m?.

(3.4b)

Page 13

3174Z. Li, J. O’Doherty, M. Lebedev, and M. Nicolelis

Figure 2: Graphical models for (A) joint distribution Bayesian regression, and

(B)factorizeddistributionvariationalBayesianregression.Ellipsesindicateran-

dom variables, double ellipses indicate observed random variables, rectangles

indicate observed point values, and rectangles indicate repeated nodes.

xtis a column vector of length D of regression features at time t, where D

is the number of kinematic features (30 for the experiments in this study).

Features are calculated from the mean of the smoothed outputs. ytis a

column vector of length N of binned spike counts at time t. The vec()

function converts a matrix into a column vector by vertically concatenating

the columns, with the left-most column at the top. The variables μ (N × D),

? (D × D), ? (N × N), and m (scalar) are hyperparameters that describe the

normal-inverse-Wishart distribution for the tuning model parameters:

NW−1?vec(H),R|vec(μ),?−1,?,m?

= N?vec(H)|vec(μ),?−1⊗ R?W−1(R|?,m).

(3.5)

Page 14

Bayesian Adaptive Decoding for Brain-Machine Interfaces 3175

Here, ⊗ is the Kronecker product. Due to conjugacy, the posterior parame-

ters are the prior parameters updated by statistics of the neuronal activity

and kinematic features. When algebraic manipulations on the likelihood

and the prior are performed, the following update equations can be found:

?←˜? + XX?,

μ←?˜ μ˜? + YX???−1,

?←˜? + ˜ μ˜? ˜ μ?+ YY?− μ?μ?,

m← ˜ m + T.

(3.6a)

(3.6b)

(3.6c)

(3.6d)

Tildes indicate prior values for the hyperparameters. X is the D × T matrix

of features over the T time points in the update batch. Y is the N × T

matrix of binned spike counts. The prior hyperparameters ˜ μ,˜?,˜?, and ˜ m

are the posterior hyperparameters from the previous update or the initial

parameter fit (see section 3.8).

Despite the advantages of offering a fast and exact analytical solution,

this formulation has the disadvantage of linking the confidence measure

on coefficients from different neurons through the ? hyperparameter. This

means that we cannot update the coefficients of neurons separately. The

ability to update subsets of neurons is important because this allows us to

halt updating for a portion of neurons, which is useful if some neurons are

affected by transient recording noise or are stable in tuning.

Furthermore,addingnewlydiscoveredneurons tothedecoderinaprin-

cipled manner means using low initial confidence for their coefficients.

Adjusting the ? hyperparameter for new neurons is not possible since

the entire population shares this hyperparameter. While changing the new

neurons’ entries in ? affects their initial confidence, this approach does not

increase the magnitude of future updates. That is, during subsequent up-

dates, we expect the magnitude of updates for new neurons to be larger,

if all other factors are equal, since the confidence about the new neurons’

coefficients should be lower than the confidence about the coefficients of

neurons already in the model. Changing ? means that in subsequent up-

dates, all neurons are treated as if the confidence in their coefficients is

the same (see equations 3.6a and b). Alternatively, one may use separate

? terms for each neuron. Unfortunately, using separate ? terms for each

neuron breaks conjugacy and precludes analytical computation. In the next

section,wepresentanapproximatesolutionforthecasewhereeachneuron

has a separate ? hyperparameter.

3.5 FactorizedFormulation. Toallowupdatesonsubsetsofneurons,we

factorized the distribution for the tuning coefficients so that the coefficients

foreachneuronarerepresentedbyamultivariatenormaldistribution.Since

thisformulationisnolongerconjugatetothelikelihoodoflinearregression,

Page 15

3176 Z. Li, J. O’Doherty, M. Lebedev, and M. Nicolelis

we applied variational Bayes (Jordan, Ghahramani, Jaakkola, & Saul, 1999),

an approximate inference technique for hierarchical probability models.

Other studies have applied variational Bayes to decoding neuronal activity

(Ting et al., 2005, 2008; Chen, Kloosterman, Wilson, & Brown, 2010).

The variational Bayesian approach assumes a factorization of the joint

posterior of the hidden random variables (tuning parameters) given the

visible random variables (neuronal activity and smoothed decoder out-

puts). The factorization facilitates conjugacy in the links of the resulting

probability model and allows the use of an iterative, fixed-point procedure

to find the approximate posterior on the hidden variables. The fixed-point

procedurecanbethoughtofasaBayesiangeneralizationoftheexpectation-

maximization (Dempster, Laird, & Rubin, 1977) algorithm. The procedure

minimizes the Kullback-Leibler divergence between the true joint poste-

rior on the hidden variables and the factorized posterior. The steps in the

procedure do not increase the divergence, which guarantees convergence,

but potentially to a local optimum. We refer readers to works by Jordan

et al. (1999), Winn and Bishop (2005), and Bishop (2006) for more details on

variational Bayes.

We will call the variational Bayesian solution to the factorized Bayesian

regression problem variational Bayesian regression (VBR). To facilitate con-

jugacy, we separated the distributions for the tuning coefficients H and the

tuning noise covariance matrix R. The probability model for VBR is shown

in Figure 2B and specified as

yt|xt,H,R∼N(Hxt,R),

h?

(3.7a)

i∼N(μi,?−1

R∼W−1(?,m).

i),

(3.7b)

(3.7c)

Here, i is the index on neurons. hiis row i of H. μiare the mean hyperpa-

rameters for the tuning model coefficients of each neuron (column vectors

of length D). ?iare the precision hyperparameters of the tuning coefficients

foreachneuron(eachD×D).?(N×N)andm(scalar)arehyperparameters

for the tuning noise covariance R. Despite the factorization among neurons

of the tuning coefficient distributions, activity among neurons may still be

correlated due to other factors. Thus, we use the inverse-Wishart distribu-

tion to represent the distribution on a full noise covariance matrix.

3.6 VariationalApproximation. Inthefollowing,wedenotetheinferred

posterior distributions for hiand R with the Q-functions Q(hi) and Q(R),

respectively. Furthermore, we denote the product of Q(hi) as Q(H).

Variational Bayes minimizes the Kullback-Leibler divergence between

the true posterior and calculated posterior by maximizing a related lower