Content uploaded by Tetiana Aksenova
Author content
All content in this area was uploaded by Tetiana Aksenova on Nov 25, 2014
Content may be subject to copyright.
Abstract— The goal of the CLINATEC® Brain Computer
Interface (BCI) Project is to improve tetraplegic subjects’
quality of life by allowing them to interact with their
environment through the control of effectors, such as an
exoskeleton. The BCI platform is based on a wireless 64-
channel ElectroCorticoGram (ECoG) recording implant
WIMAGINE®, designed for long-term clinical application, and
a BCI software environment associated to a 4-limb exoskeleton
EMY (Enhancing MobilitY). Innovative ECoG signal decoding
algorithms will allow the control of the exoskeleton by the
subject’s brain activity. Currently, the whole BCI platform was
tested in real-time in preclinical experiments carried out in non-
human primates. In these experiments, the exoskeleton arm was
controlled by means of the decoded neuronal activity.
I. INTRODUCTION
A Brain Computer Interface (BCI) aims at providing an
alternative non-muscular communication pathway to send
commands to the external world for individuals suffering
from severe motor disabilities. Commands are generated by
means of decoding of brain neuronal activity.
ElectroEncephaloGram (EEG) recordings of neuronal
activity are widely used in BCI applications but require the
user to wear a non-ergonomic EEG helmet, necessitate daily
repositioning and recalibration, and the signal quality is not
sufficient to control effectors with large number of degrees
of freedom. Microelectrode recordings have performed
successfully in BCI experiments in the laboratory, enabling
A. Eliseyev is with CEA, LETI, CLINATEC; MINATEC Campus, 17
rue des Martyrs, 38054 Grenoble Cedex, France (corresponding author,
phone: +33-438-785-375; e-mail: eliseyev.andrey@gmail.com).
C. Mestais, G. Charvet, F. Sauter, N. Arizumi, S. Cokgungor,
T. Costecalde, M. Foerster, J. Pradal, D. Ratel, N. Tarrin, N. Torres-
Martinez, T. Aksenova, A.-L. Benabid are with CEA, LETI, CLINATEC;
MINATEC Campus, 17 rue des Martyrs, 38054 Grenoble Cedex, France (e-
mail: corinne.mestais@cea.fr, guillaume.charvet@cea.fr,
fabien.sauter@cea.fr, arizumi@gmail.com, serpil.cokgungor@cea.fr,
thomas.costecalde@cea.fr, michael.foerster@cea.fr,
louis.korczowski@gmail.com, jeremy.pradal@gmail.com,
david.ratel@cea.fr, nicolas.tarrrin@cea.fr, napoleon.torres-
martinez@cea.fr, tetiana.aksenova@cea.fr, alimlouis@sfr.fr.
J. Porcherot is with CEA, LETI, DTBS; MINATEC Campus, 17 rue des
Martyrs, 38054 Grenoble Cedex, France (e-mail: jean.porcherot@cea.fr)
N. Abroug, B. Morinière, A. Verney are with CEA LIST, DIASI/LRI;
DIGITEO Labs; rue Noetzlin, 91190 Gif sur Yvette (e-mail:
neil.abroug@cea.fr, boris.moriniere@cea.fr, alexandre.verney@cea.fr).
closed-loop control of a computer cursor and a robotic arm.
Despite encouraging successes in clinical applications [1],
microelectrode recordings have yielded limited success
outside of the laboratory, mainly due to unsolved problems
regarding the long-term robustness of the recorded signals.
Subdural or epidural ECoG electrode arrays recordings are
less sensitive to artifacts, offer higher frequency and spatial
resolution than EEG and are less invasive than
microelectrode arrays. ECoG electrode arrays are a practical
way to measure the electrical activity of the brain for BCI
purpose. Minimally invasive less traumatic epidural ECoG
chronic recording was chosen as a basic concept of the BCI
project at CLINATEC®. The goal of the project is to
improve the quality of life of tetraplegic subjects, by
allowing them to interact with their environment through the
control of effectors with multiple degrees of freedom. Most
of the studies of ECoG based BCI have been limited to
short-term experiments, and have been carried out in
laboratory conditions [2]. An ultimate daily and “out-of-the-
lab” BCI application requires longer durations of usage, and
also requires the BCI system to be controlled by the users at
their will without any external stimulus. Thanks to the
innovative wireless 64-channel ECoG recording implant
WIMAGINE® (Wireless Implantable Multi-channel
Acquisition system for Generic Interface with NEurons) [3]
as well as the original ECoG signal processing, the subject
should be able to control a 4-limbs exoskeleton EMY after
training.
Before applying the BCI platform to humans, a set of
preclinical experiments are carried out with monkeys. These
experiments allow preliminary evaluation of the system, as
well as early identification and elimination of its
shortcomings.
II. CLINATEC® BCI PLATFORM
A. ECoG recording implant WIMAGINE®
BCI clinical applications are related to the technical
challenges intrinsic to implantable medical devices [4, 5]. In
the CLINATEC® BCI project, the ECoG signals from the
subject’s brain are recorded and wirelessly transmitted to a
base station by the WIMAGINE® implant [3]. This implant
is composed of an array of 64 biocompatible electrodes, a
hermetic titanium housing which includes electronic boards,
CLINATEC® BCI platform based on the ECoG-recording implant
WIMAGINE® and the innovative signal-processing: preclinical
results
Andrey Eliseyev, Corinne Mestais-IEEE Member, Guillaume Charvet, Fabien Sauter, Neil Abroug,
Nana Arizumi, Serpil Cokgungor, Thomas Costecalde, Michael Foerster, Louis Korczowski,
Boris Morinière, Jean Porcherot, Jérémy Pradal, David Ratel, Nicolas Tarrin, Napoleon Torres-
Martinez, Alexandre Verney, Tetiana Aksenova-IEEE Member,
Alim-Louis Benabid
978-1-4244-7929-0/14/$26.00 ©2014 IEEE 1222
biocompatible antennae for wireless transmission of the data,
and a remote power supply. The general view of the implant
is represented in Fig. 1.
Figure 1: WIMAGINE® implant. A) top view, B) bottom view
The design of the WIMAGINE® implant addresses all the
constraints of a fully implantable medical device, such as
ultra-low power, miniaturization, safety and reliability. The
required tests to demonstrate the implant’s compliance to the
European Directives for Active Implantable Medical Device
(Directive 90/385/EEC and European standard EN 45502-2-
1) are in progress in certified laboratories.
The BCI Platform can include up to two WIMAGINE®
implants with a base station and a PC application (Fig. 2A).
The base station acts as a gateway between the PC
application and the WIMAGINE® implants. A headset is
used to position and maintain antennae dedicated to provide
the remote power supply to each implant and to receive the
raw ECoG data over a proprietary UHF link in the Medical
Implant Communication Service (MICS) band.
B. Software platform
CLINATEC® BCI software platform (Fig. 2B) aims at
providing the environment for coordinated operation of all
the components of the system from the signal acquisition to
the exoskeleton control. In particular, the ECoG signal is
processed using a dedicated algorithm to extract predictions
of movement which are converted into commands to control
the exoskeleton.
Figure 2: CLINATEC® BCI Platform. A) Two WIMAGINE® implants
record ECoG activity of the brain and send signals wirelessly to the base
station. B) The software platform analyzes ECoG data in real-time and
generates the commands to the external effector. C) A full-body
exoskeleton, dedicated to medical purposes, allows the subject to interact
with the surrounding environment
The software platform consists of several software
modules specially developed (in C/C++) for this application.
The ECoG recording part (based on WIMAGINE® implants)
is carried by the following software modules: Implant
Software (IS), Terminal Software (TS), Wireless Implant
Software Control Interface (WISCI) & Embedded Device
Controller (MEDOC). The Online ECoG signal decoding
part is performed by the Online Cerebral Decoder (OCD),
and the exoskeleton control part is ensured by EMY Motion
Manager (EMM), and EMY Motion Controller (EMC). The
detailed block scheme of the software system is represented
in Fig. 3.
Figure 3: Block scheme of CLINATEC® BCI software system
C. ECoG signal decoding
The neuronal signal processing (patented) approach of
CLINATEC® [6, 7] (Fig. 2B) is based on a tensor data
analysis. It allows simultaneous treatment of the signal in
several domains, namely, frequency, temporal, and spatial
(Fig. 4).
Figure 4: Time epochs of the multi-channel ECoG recording mapped to the
temporal-frequency-spatial feature space
The algorithm consists in two stages. During the first
stage (calibration), the control model is adjusted to a
particular subject. During the second stage (execution), the
model allows controlling an external effector (e.g.,
exoskeleton) through the subject’s neuronal activity. For
calibration, the high dimensional tensor-value explanatory
variables are extracted from the signal by means of
continuous wavelet transform (CWT). A decoding model is
identified using the tensor-decomposition methods. The
1223
block-wise algorithms [6, 7] allow calibration of the BCI
system in the case of high resolution of the data tensor. The
single-pass block-wise algorithm [7] provides an adaptive
learning, whereas the sparse approach [6] improves the
decoding performance. The calibrated model is integrated to
the BCI software environment.
The volatility of the wireless connection introduces
temporal loss of the signals, i.e., gaps in the data stream. To
define and to evaluate the stability of the decoding algorithm
to the loss of data, a set of computational experiments were
carried out. The goal of the experiments was to compare the
decoding performance of the algorithm using the signal
without gaps and using the same signal, contaminated by the
gaps. In particular, a known epidural-ECoG signal from the
Japanese macaques [8] was corrupted with artificial
sequences of the gaps. The sequences were generated
according to the gaps’ distribution in the wirelessly
transmitted signals and then imposed on the gaps-free data.
The empirical distribution of the gaps was approximated by a
mixture of three gaussians with the expectations 3 ms (39%),
14 ms (24%), and 60 ms (37%), respectively. In order to
have a sufficient population and to prevent any instability
from the signal processing, the number of the gaps was
increased 100 times. The decoding performance was studied
for the cases of application of different methods of gaps’
filling, namely, Zero-Order Hold, First-Order Hold,
Autoregressive Model (8th Order), Spline Cubic
Interpolation, Piecewise Polynomial Interpolation,
Sinusoidal Amplitude L1 Estimation, and Sinusoidal
Amplitude and Phase L1 Estimation [9]. In all the cases, the
algorithm has demonstrated significant robustness. The
biases of prediction (normalized averaged root mean
squares) were, correspondently, 1.4%, 1.4%, 1.4%, 3.6%,
1.4%, 1.8%, 1.3%, and 1.3%. Thus, even in the simplest case
of the filling, namely, Zero-Order Hold, the influence of the
data loss on prediction was negligible.
A particular requirement for the BCI system is stability
and robustness in the case of real-time operation. The control
system, developed by the CLINATEC® team, successfully
meets BCI requirements.
D. EMY Exoskeleton
EMY is a full-body exoskeleton, dedicated to medical
purposes and developed by the interactive robotics unit of
CEA LIST [10-12] (Fig. 2C). The current EMY architecture
features four limbs, two legs with three degrees of freedom
each, and two ABLE® anthropomorphic arm exoskeletons
with seven degrees of freedom. EMY is powered by an
external electrical source and has a deported control
electronics. The particular design of EMY’s limbs allows
accurate torque control which is achieved using a patented,
streamlined, mechanical transmission (screw-cable system)
that minimizes friction and inertia. This architecture ensures
that the current in the motor is an accurate image of the joint
torque, so there is no need for a torque/force sensor. This is
both simple and reliable while remaining energy-efficient
and cost-effective. The interface between the BCI and the
robot is achieved by the physics-simulation framework
XDE® [13]. The simulation layer allows EMY to be
controlled at different levels of complexity, like joint
motions, Cartesian motions, or Cartesian programmed
trajectories.
E. Preclinical experiments
The preclinical experiments in a monkey were carried out
in a male Macaque Rhesus. Ethical approval for them was
obtained from ComEth (IRB of the University of Grenoble,
France) in accordance with the European Communities
Council Directive of 1986 (86/609/EEC) for care of
laboratory animals.
The implant WIMAGINE® cannot be implanted on
primate’s skull since it was designed to be compatible with
human’s skull dimensions. Thus, a silicone/platinum-iridium
cortical electrode array (PMT® Corporation, Chanhassen,
USA) was implanted in the region of monkey’s left motor
cortex. It was connected to the recording electrodes of the
implant WIMAGINE® by means of a transcutaneous
connector and a specially designed test assembly. The
electrode array of the WIMAGINE® and the one used for
monkey have exactly the same pitch (4.25 mm), the same
material: platinum iridium (silicone rubber for insulation)
and the same contact area (3.14 mm2).
The setup of the experiment is represented in Fig. 5. The
monkey was trained to reach an exposed target using the
right hand. The hand movements were recorded by an optical
motion capture system Vicon (Motion Systems, Oxford,
UK). During the calibration stage, the monkey’s ECoG data
were used together with information about the hand position
to identify a prediction model. This model was applied to the
ECoG data on the second stage to generate control
commands for the arm prosthesis. Fig. 6 illustrates the
relative weights of the linear model coefficients in the spatial
and frequency domains. In the spatial domain, the weights of
different electrodes significantly vary. Application of the
sparse model identification approach [6], leads to selection
of a small subset of the most informative electrodes, namely
2, 3, 4, 8, 13. In the frequency domain, high frequencies
(from 80 Hz up to 150 Hz) have the highest contribution. At
the same time, the frequencies in the band [30, 40] Hz as
well as around 10 Hz also have a significant contribution.
The results of the motion prediction are demonstrated in
Fig. 7. The quality of the obtained prediction allows
application of the proposed approaches in humans.
Moreover, the high performance of decoding authorizes
reproducing the arm movement by the exoskeleton arm in
real-time.
Figure 5: Preclinical experiments’ setup in monkey
1224
Figure 6: The relative contribution of the elements of the spatial and
frequency domains in the identified model
Figure 7: An example of the observed (blue) and predicted (red) X-, Y-, and
Z-coordinates of the monkey’s right hand
III. DISCUSSION
CLINATEC® BCI platform is an implementation of an
innovative project aiming at the rehabilitation of persons
with severe motor handicap. It includes two ECoG
WIMAGINE® implants and the base station, BCI software
environment, ECoG signal decoding algorithms and
software, as well as the full-body exoskeleton EMY,
dedicated to medical purposes. The whole BCI platform was
tested in preclinical experiments carried out in a nonhuman
primate (Macaque Rhesus). The control model was
calibrated offline using a set of training recordings. The
performance of the decoding on a validation set provided
high correlation level between the observed and predicted
signals, varying from 0.4 to 0.8. Moreover, the application of
the informative electrodes selection method could
additionally improve the correlation level by up to 10%.
During the online experiments, a fragment of the
exoskeleton (the arm) was controlled in real-time by means
of decoded neuronal activity (decision rate DR=10 Hz). The
delay due to the signal processing, including the signal
acquisition, transmission, as well as the commands
generation, was approximately equal to 300 ms.
For clinical using, the system should be resilient to loss
of data caused by the wireless transmission of the signal.
Since the multimodal approach simultaneously processes
information from different modalities (frequency, temporal,
and spatial), it is not susceptible to the negative effects of a
temporary (up to 200 ms) absence of the signal.
The next step of the CLINATEC® project is optimization
and adjusting of all the components of the BCI system.
Additional tests in animals will be carried out for the
evaluation of BCI’s performance, as well as identification
and elimination of the possible drawbacks. For this purpose,
additional animals are scheduled to be implanted.
ACKNOWLEDGMENT
This work was performed thanks to the close
collaboration of the technical staff of CEA/LETI
CLINATEC® and DTBS. Research is supported by French
National Research Agency (ANR-Carnot Institute),
Fondation Motrice, Fondation Nanosciences, Fondation de
l’Avenir, Région Rhône-Alpes and Fondation
Philanthropique Edmond J. Safra.
REFERENCES
[1] L. R. Hochberg, D. Bacher, B. Jarosiewicz, N. Y. Masse, J. D.
Simeral, J. Vogel, et al., "Reach and grasp by people with tetraplegia
using a neurally controlled robotic arm," Nature, vol. 485, pp. 372-5,
May 17 2012.
[2] W. Wang, J. L. Collinger, A. D. Degenhart, E. C. Tyler-Kabara, A. B.
Schwartz, D. W. Moran, et al., "An electrocorticographic brain
interface in an individual with tetraplegia," PLoS One, vol. 8, p.
e55344, 2013.
[3] G. Charvet, F. Sauter-Starace, M. Foerster, D. Ratel, C. Chabrol, J.
Porcherot, et al., "WIMAGINE®: 64-channel ECoG recording implant
for human applications," in Engineering in Medicine and Biology
Society (EMBC), 2013 35th Annual International Conference of the
IEEE, 2013, pp. 2756-2759.
[4] M. Hirata, K. Matsushita, T. Suzuki, T. Yoshida, S. MORRIS, T.
YANAGISAWA, et al., "A fully-implantable wireless system for
human brain-machine interfaces using brain surface electrodes: W-
HERBS," IEICE transactions on communications, vol. 94, pp. 2448-
2453, 2011.
[5] F. Kohler, M. A. Ulloa, J. S. Ordonez, T. Stieglitz, and M. Schuettler,
"Reliability investigations and improvements of interconnection
technologies for the wireless brain-machine interface —
‘BrainCon’," in Neural Engineering (NER), 2013
6th International IEEE/EMBS Conference on, 2013, pp. 1013-1016.
[6] A. Eliseyev, C. Moro, J. Faber, A. Wyss, N. Torres, C. Mestais, et al.,
"L1-penalized N-way PLS for subset of electrodes selection in BCI
experiments," J Neural Eng, vol. 9, p. 045010, Aug 2012.
[7] A. Eliseyev and T. Aksenova, "Recursive N-way partial least squares
for brain-computer interface," PLoS One, vol. 8, p. e69962, 2013.
[8] K. Shimoda, Y. Nagasaka, Z. C. Chao, and N. Fujii, "Decoding
continuous three-dimensional hand trajectories from epidural
electrocorticographic signals in Japanese macaques," J Neural Eng,
vol. 9, p. 036015, Jun 2012.
[9] P. S. Cowpertwait and A. V. Metcalfe, Introductory time series with
R: Springer, 2009.
[10] Y. Perrot, A. Verney, B. Morinière, and P. Garrec, "EMY: Full-body
Exoskeleton," in ACM SIGGRAPH Emerging Technologies,
Anaheim, US, 2013.
[11] P. Garrec, J. Friconneau, Y. Measson, and Y. Perrot, "ABLE, an
innovative transparent exoskeleton for the upper-limb," in Intelligent
Robots and Systems, 2008. IROS 2008. IEEE/RSJ International
Conference on, 2008, pp. 1483-1488.
[12] P. Garrec, "Screw and cable actuators (scs) and their applications to
force feedback teleoperation, exoskeleton and anthropomorphic
robotics," ed: Robotics, 2010.
[13] Physics simulation software environment XDE [Online]. Available:
http://www.kalisteo.fr/lsi/en/aucune/a-propos-de-xde
1225
