Content uploaded by Louis Mayaud
Author content
All content in this area was uploaded by Louis Mayaud on Mar 05, 2019
Content may be subject to copyright.
Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=tbci20
Download by: [AGEPS Centre de Documentation] Date: 20 December 2016, At: 03:56
Brain-Computer Interfaces
ISSN: 2326-263X (Print) 2326-2621 (Online) Journal homepage: http://www.tandfonline.com/loi/tbci20
Brain-computer interface for the communication
of acute patients: a feasibility study and
a randomized controlled trial comparing
performance with healthy participants and a
traditional assistive device
Louis Mayaud, Salvador Cabanilles, Aurélien Van Langhenhove, Marco
Congedo, Alexandre Barachant, Samuel Pouplin, Sabine Filipe, Lucie
Pétégnief, Olivier Rochecouste, Eric Azabou, Caroline Hugeron, Michèle
Lejaille, David Orlikowski & Djillali Annane
To cite this article: Louis Mayaud, Salvador Cabanilles, Aurélien Van Langhenhove, Marco
Congedo, Alexandre Barachant, Samuel Pouplin, Sabine Filipe, Lucie Pétégnief, Olivier
Rochecouste, Eric Azabou, Caroline Hugeron, Michèle Lejaille, David Orlikowski & Djillali
Annane (2016) Brain-computer interface for the communication of acute patients: a
feasibility study and a randomized controlled trial comparing performance with healthy
participants and a traditional assistive device, Brain-Computer Interfaces, 3:4, 197-215, DOI:
10.1080/2326263X.2016.1254403
To link to this article: http://dx.doi.org/10.1080/2326263X.2016.1254403
© 2016 The Author(s). Published by Informa
UK Limited, trading as Taylor & Francis
Group
Published online: 16 Dec 2016.
Submit your article to this journal Article views: 21
View related articles View Crossmark data
BRAINCOMPUTER INTERFACES, 2016
VOL. 3, NO. 4, 197215
http://dx.doi.org/10.1080/2326263X.2016.1254403
Brain-computer interface for the communication of acute patients: a feasibility
study and a randomized controlled trial comparing performance with healthy
participants and a traditional assistive device
Louis Mayauda,b,c,d, Salvador Cabanillesc, Aurélien Van Langhenhovea,b,c, Marco Congedoh,
Alexandre Barachanth, Samuel Pouplina,b,c, Sabine Filipee, Lucie Pétégnieff, Olivier Rochecousteg, Eric Azaboub,c,
Caroline Hugeronc, Michèle Lejaillea, David Orlikowskia,b,c and Djillali Annanea,b,c
aINSERM, Centre d’Investigation Clinique et d’Innovation technologique (CIC-IT), UMR805, Garches, France; bINSERM, Equipes Thérapeutiques
innovantes et Technologies appliquées aux troubles neuromoteurs, U. 1179, Garches, France; cHôpital Raymond Poincaré, APHP, Garches, France;
dMensia Technologies SA, Paris, France; eDepartment DTBS, CEA/LETI, Grenoble, France; fDIXI Microtechniques Medical, Besançon, France;
gDavidson Consulting, Rennes, France; hGIPSA-Lab, CNRS, University of Grenoble-Alpes, Grenoble Institute of Technology, Grenoble, France
ABSTRACT
This study presents the outcome of the 5-year-long French national project aiming at the
development and evaluation of an eective brain-computer interface (BCI) prototype for the
communication of patients with acute motor disabilities. It presents results from two clinical studies:
a clinical feasibility study carried out partly in the intensive care unit (ICU) and the clinical evaluation
of an innovative BCI prototype. In this second study the BCI performance of patients was compared
to that of healthy volunteers and benchmarked against a traditional assistive technology (scanning
device). Altogether, 15 of 22 patients could control the BCI system with an accuracy signicantly
above the chance level. The bit-rate of the traditional assistive technology proved superior, even
though an equivalent bit-rate could be achieved using personalized parameters for the BCI. Fatigue
was found to be the primary limitation factor, which was particularly true for patients and during
the use of the BCI. A classier based on Riemannian geometry was found to contribute signicantly
to the accuracy of the BCI system. This study demonstrates that the communication of patients with
severe motor impairments can be eectively restored using an adequately designed BCI system. All
electrophysiological data are freely available at the Physionet.org platform.
Introduction
Lack of communication in the hospital may be a great
source of distress for both patients and caregivers, which
is particularly exacerbated in the acute context of inten-
sive care units (ICUs).[1] Several neurological disorders
impair communication abilities while leaving cognitive
capacities almost untouched. is has long been reported
for chronic conditions such as myopathies, spinal cord
injuries, amyotrophic lateral sclerosis (ALS), and multiple
sclerosis (MS), but is also true for acute neuropathies such
as the Guillain-Barré syndrome (GBS). Moreover, quad-
riplegic or locked-in syndrome (LIS) patients undergoing
invasive mechanical ventilation (via endotracheal tube or
tracheotomy) see their communication abilities impaired
by the sudden loss of speech.[2]
e use of computers in this context can facili-
tate communication and many technical solutions to
facilitate computer access are available.[3] In particular,
occupational therapists equip patients with ‘scanning’
systems [4–6] enabling interaction with computers or
other devices such as speech synthesizers. e assistive
technology (AT) in this case consists of a ‘click interface’
(or switch) connected to an application where a limited
number of options are visually ‘scanned’ (highlighted) for
the patient. ere are many types of available switches to
control a scanning system [7]: muscular switches, tactile
switches, pu switches, and mechanical switches, the most
common type in France, which consist of a press-down
button whose activation force, size, location (feet, near the
head, thumb), and type of activation (validation on press-
down or release) are set according to the patient’s specic
condition. Whenever a reliable switch command can be
obtained, the system is connected to a soware interface.
Typically, this consists of a grid of symbols displayed on a
KEYWORDS
Assistive technology;
communication
aids; event-related
potentials, P300 speller;
electroencephalography;
Riemannian geometry
ARTICLE HISTORY
Received 12 March 2016
Accepted 26 October 2016
© 2016 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-
nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built
upon in any way.
CONTACT Louis Mayaud louis.mayaud@gmail.com
OPEN ACCESS
198 L. MAYAUD ET AL.
path to low-cost EEG technology for the general public,
although the quality of the signal (hence the overall BCI
performance) has proved inferior to research-grade EEG
acquisition systems.[25,26] us, to date, a state-of-the-art
EEG-based BCI requires the support of a technical and
clinical team composed of highly qualied profession-
als.[27–29] A major concern for the present study is the
possibility of generating and processing visual ERPs in an
ICU environment overowing with uncontrolled sources
of noise, both acoustic (alarms, sta, mechanical ventila-
tion) and electromagnetic (bedside monitors, automated
syringes, mechanical ventilation). While EEG applications
are known to be particularly sensitive to non-controlled
environments,[16,30] to the best of our knowledge no
report exists about their use in an ICU.
P300-based BCI also have specic limitations. First of
all, since several ERPs need to be averaged to obtain a
sucient SNR, the overall bit-rate is low in practice (up
to 1 minute per letter in a P300 speller).[31,32] e signal
processing and machine learning research communities
actively address this issue by improving ERP detection
methods.[33–35] Also, the use of a P300 speller requires
constant concentration and there is still a debate on
whether some clinical populations possesses the necessary
attention span to eectively control a P300 speller, espe-
cially if the session is prolonged.[36,37] is is particu-
larly true for populations of patients admitted to the ICU
where central nervous system (CNS) depressant drugs
are usually administered as they are known to inuence
vigilance and attention.[38] It is worth noting that while
P300 speller BCI systems have traditionally been clinically
evaluated in populations of chronic patients such as those
suering from ALS,[27,28,30,39] its potential benet in
acute nervous conditions, such as GBS, have not yet been
reported.
To complete the technological transfer from ‘bench to
bedside’, BCI must gain ease of use and robustness in terms
of both algorithms and interface (signal-processing and
applications). Overcoming the aforementioned technical
challenges appears important, particularly in the context
of a better assessment of pain and cognitive function
(perhaps identication of delirium) of non-communicat-
ing patients. is would certainly result in an improved
quality of ICU stay. e Robust Brain-computer Interface
for virtual Keyboard (RoBIK) project [40] aimed at the
development of a BCI system for the communication of
patients that could be used on a daily basis in the hospital
with limited external intervention. In order to achieve this
goal a multidisciplinary approach was chosen and devel-
opments were carefully framed by clinical specications
and validation, according to a user-centered design. e
project resulted in two clinical studies:
screen, which are ashed at a regular pace (i.e. the ‘scan-
ning’) until the patient selects one. e conguration of
both the physical and the soware interface requires a
tremendous amount of time and expertise, which is one
reason why, as a matter of fact, patients with severe motor
disabilities in acute contexts such as the ICU are oen le
unequipped.
e brain-computer interface (BCI) is a longstanding
technology [8] that translates the brain electrical activ-
ity into a command for a device.[9,10] BCIs are usually
characterized by: (1) a brain activity recording modality,
(2) a paradigm that relates sensorial stimulations to some
specic brain activity, and (3) an interface connecting to
the front-end application by means of pattern-recogni-
tion techniques. Electroencephalography (EEG)-based
BCIs [11,12] have been extensively studied since EEG
is a non-invasive and portable neuroimaging modality.
In this article, we focus on BCI technology based on the
P300 event-related potential (ERP). A P300 is a positive
ERP occurring 300 to 500 ms aer the presentation of
a rare meaningful stimulus presented in a ow of many
irrelevant stimuli.[13] e repeated presentation of several
stimuli among which there is a possible target for the user
is referred to as the ‘oddball paradigm’.[13–15] e ‘P300
speller’ is currently the best-known and most widespread
ERP-based BCI communication interface. In a typical
P300 speller a regular grid of symbols is displayed on the
screen while lines and columns are ashed at random. e
participant is asked to concentrate on a target symbol. A
P300 is elicited in response to the ash of the line and the
column containing the target symbol. Since the ampli-
tude of the P300 is much smaller as compared to the EEG
background activity and EEG artifacts, the ERP is elicited
several times and averaged in order to increase the signal-
to-noise (SNR). e averaging procedure increases the
discrimination ability (target ERPs vs. non-target ERPs),
eectively resulting in a trade-o between accuracy and
speed of selection. e P300 speller represents a great
hope for severely disabled patients, especially for those
who are not able to use traditional AT.
Despite the tremendous amount of research over the
past 20years, EEG-based BCIs still suer from signicant
drawbacks such as setup time and sensitivity to noise.[16]
For instance, the placement of electrodes connected to the
scalp skin through a conductive gel is a time-consuming
process, certainly prohibitive for everyday use in both
hospitals and at the patient’s home. Moreover, caregivers
are usually not familiar with EEG technology. Attempts
to overcome this limitation include the use of caps with
pre-positioned gel-based [16,17] or dry [18–20] electrodes,
possibly interfaced with active recording systems.[21,22]
Recently, the EPOC headset [23,24] has opened the
BRAINCOMPUTER INTERFACES 199
• Study 1: Identication of the clinical and technical
limitations aecting the use of BCI applications
for the communication of acute patients in a clin-
ical setting; a questionnaire was administered to
patients and caregivers [41] in order to identify
appropriate user specications;
• Study 2: the clinical validation of a BCI system
developed with specications derived from Study I,
which consisted of
◦ An EEG headset with portable electronics
designed for convenience of use in the hospital
(Appendix 1),
◦ dedicated soware comprising a user-friendly
interface and a classication algorithm based on
Riemannian geometry (Appendix 2); the perfor-
mance of this BCI prototype on clinical samples
was compared with a state-of-the-art assistive
device for the communication of patients (scan-
ning device), in order to assess the actual clinical
value of the two technologies, and withhealthy
volunteers, in order to quantify the drop in per-
formance induced by the patients’ condition.
Finally, an oine analysis of the data collected during
these two clinical trials was carried out in order to quan-
tify the contribution of each system’s element (hardware
and soware) to the whole and to estimate the perfor-
mance of a personalized BCI. e details of this analysis
are available in Appendix 3.
Materials and methods
Table 1 summarizes the populations, hardware, soware,
algorithms, and cross-validation techniques used in each
clinical trial.
Study 1: clinical feasibility
e aim of this study was to evaluate the usability of a
state-of-the-art P300 speller BCI for communication [15]
in the challenging environment of intensive care and reha-
bilitation units.
Study design and protocol
e protocol was registered on ClinicalTrials.gov
(NCT01005524) and received clearance from local ethics
boards (‘Comité de protection des personnes’, CPP, Saint
Germain-en-Laye, 2009/10/15). Quadriplegic patients
admitted to ICUs and rehabilitation units at a tertiary-care
hospital were enrolled aer giving their informed consent.
Pregnant women, illiterate patients, patients under judi-
cial protection or without social security as well as patients
with a history of epilepsy were excluded from the study.
e protocol consisted of three P300 speller
sessions.[15]
(1) A training session was given during which
patients were instructed to spell two words
(‘OCEAN’ and ‘NUAGES’, meaning in French
‘Ocean’ and ‘Clouds’, respectively) and for
which no feedback was provided. Signals col-
lected were then used to train a state-of-the-art
pattern-recognition algorithm based on a spa-
tial filter and a classifier, detailed in the next
section.[33]
(2) e classier performance was evaluated dur-
ing an ‘online’ session during which instruc-
tions were given for three words (‘OEUFS’,
‘AVION,’ and ‘OASIS’, meaning in French ‘eggs’,
‘airplane’, and ‘oasis’, respectively). e letters
identified by the algorithm aer each sequence
of stimulations were displayed on the screen as
a feedback.
(3) Finally, an optional ‘free-spelling’ session was
oered to the patient, during which no instruc-
tion was given and patients could spell what-
ever they wished.
At the end of the session patients were requested to
rate their satisfaction with the BCI system as a commu-
nication tool by means of a visual analogue scale (VAS)
ranking from 0 (completely unsatised) to 10 (completely
satised).
Data acquisition and processing
Twenty-four silver chloride (AgCl) disk-electrodes
were connected by active-shielded coaxial cables to a
Porti32 EEG amplifier (TMSi, Twente, Netherlands)
sampling at 512Hz. Signal acquisition, processing, and
storage were performed using the open-source platform
OpenViBE.[42] Raw EEG signals were stored as GDF
Table 1.Summary of materials and methods for both studies showing the population, hardware, software, algorithms, and cross-val-
idation procedures. Abbreviations: minimum distance to mean (MDM) is a Riemannian classifier; support vector machine (SVM) is a
classifier. xDAWN is a spatial filter for event-related potentials.
Population Hardware Software Algorithms Cross-validation
Study I • Patients (P1) • TMSi • OpenViBE • xDAWN
• SVM
• 20-Fold (Training)
• Cross-session
Study 2 • Patients (P2)
• Volunteers (V2)
• RoBIK • OpenViBE
• Python 2.7
• Riemannian Potato
• MDM
• 5-Fold (Training)
• Cross-session
200 L. MAYAUD ET AL.
which were implemented as described in this section and
used for the second clinical study. Technical details can be
found in Appendices 1 and 2.
The EEG headset
e RoBIK EEG headset, shown in Figure 1 (le), has
been designed to be an eective means of recording EEG
activity usable by people not familiar with EEG technol-
ogy, while oering performance equivalent to that of a
clinical-grade EEG system. e resulting headset has silver
chloride (AgCl) electrodes mounted at the following 14
standard 10/20 locations: Po7, Fp1, Oz, Fz, C3, F4, Pz, C4,
P7, Fp2, F3, P8, Po8, and Cz. Connection to the scalp skin
is obtained by means of a cotton pad soaked with physi-
ological saline solution. e signal is sampled at 580Hz
with 12-bit resolution.
The user interface
e application and generation of visual stimulations were
handled by a dedicated application called ‘Brainmium’
(Figure 1, right). It was designed to be user-friendly so that
a caregiver with basic knowledge of informatics can oper-
ate it aer a short training. Brainmium implements state-
of-the-art P300 paradigms, using random inter-stimulus
interval (ISI) and random group ashing (dierent from
the line-column paradigm), which are meant to reduce
visual fatigue and increase discriminatory power. Details
can be found in Appendix 2.
Online artifact rejection
In this work we use a signal quality index (SQI) in order
to reject trials contaminated by excessive artifacts during
both the training and the online phase of the experiment.
e SQI is an improvement of the ‘Riemannian potato’
methods [48] based on the suggestion from Congedo.[49]
Technical details are given in Appendix 2.
format [43] including the type and location of visual stim-
ulations. is data-set is freely available on the Physionet
platform.1
As pre-processing, all EEG signals were band-pass-l-
tered between 0.01 and 30Hz by means of a Butterworth
fourth-order lter with linear phase response. e
xDAWN spatial lter [33] was trained on the training
session. xDAWN is a spatial lter specically designed
for ERP data.[44] e aim of a spatial lter in this context
is to enhance the signal of interest (ERPs) while suppress-
ing the background noise.[45] It also dramatically reduces
the dimensionality of EEG data, since a few lters suce
to summarize the signal of interest (ERPs). e spatially
ltered signal is fed to an ensemble of support vector
machine (SVM) classiers for identication of target and
non-target stimulation.[46]
Cross-validation scheme
For each participant the accuracy was assessed with a
20-fold cross-validation procedure on the training ses-
sion. A K-fold validation [47] is a technique oen used
to estimate the performance of a classication technique
(and its variability) on ‘unseen’ data. At each fold, 20%
of the data was le aside for model validation (the ‘test’
set) while the remaining 80% (the ‘training set’) was used
to design the model (the xDAWN spatial lter and the
ensemble of SVMs). Online letter-recognition accuracy
during the test session was simply obtained by applying
models derived from the training session, live, to the
online data.
Technical developments
Following this feasibility study and results from a survey
conducted with patients and caregivers,[41] we identi-
ed optimal hardware and soware BCI specications,
Figure 1.The RoBIK prototype: the RoBIK headset (left) with 14 wet EEG channels and the virtual keyboard interface ‘Brainmium’ (right).
BRAINCOMPUTER INTERFACES 201
Bedside procedure
e RoBIK prototype was setup by an occupational ther-
apist who had no specic training in electrophysiology.
e protocol consisted of the following steps:
(1) e EEG headset was set up and time was
counted from the beginning of the installation
to the beginning of step 2; the operator was
instructed to reduce the presence of power line
contaminations (50Hz) by adjusting electrode
positioning until the presence of eye blinks,
jaw muscular artifacts, and alpha waves could
be observed when the patient was instructed to
blink, clamp his jaw, and close his eyes, respec-
tively. e impedance between each electrode
and the reference was measured at the begin-
ning and end of each session and maintained
below 5 kΩ.
(2) e interface was presented to the patient and
directions for the training sessions were given.
When possible, the participant was asked to
re-formulate to make sure the task was cor-
rectly understood.
(3) e training session consisted of the spelling of
10 consecutive characters. e participant was
instructed to concentrate on the chosen letter
and count the number of times it ashed. Each
letter was ashed 20 times. e target letter was
continuously indicated in the upper section of
the screen and each new letter was notied by
printing in blue the corresponding key on the
virtual keyboard before each new sequence of
random stimulation.
(4) Data collected during this training session
were automatically passed down the processing
Online classier
e online classication was performed by means of the
Riemannian minimum distance to means (MDM) classi-
er [50] as applied to P300 data.[49] e standard method
was complemented with a new logistic decision function
as detailed in Appendix 2.
Study 2: clinical evaluation
e primary objective of this study was to evaluate the
performance of the RoBIK BCI prototype and to com-
pare its performance to that of a traditional assistive
technology. e second objective was to compare the
performance of patients using the BCI prototype to that
of healthy volunteers.
Study design and setting
e protocol was registered on ClinicalTrials.gov
(NCT01707498) and received clearance from the local
ethics board (CPP de Saint Germain-en-Laye, 2012/07/05)
and the regulatory agency for the use of a non-CE-marked
device (Agence Nationale de Sécurité des Médicaments,
ANSM, 2012/09/14, 2012-A00613–40). e clinical pro-
tocol inclusion criterion for patients was the presence
of functional quadriplegia. e inclusion criterion for
healthy participants was age greater than 18 years. Patients
were enrolled in the intensive care unit (ICU) and reha-
bilitation units at a tertiary care hospital aer giving their
informed consent. Healthy volunteers were enrolled at
the Center for Clinical Investigation and Technological
Innovation (CIC-IT) aer giving their informed consent.
Pregnant women, patients with hemodynamic instability,
illiterate participants, patients under judicial protection or
without social security, epileptic patients, and participants
showing skin/scalp sensitivity or severe visual impairment
or aged less than 18 years were excluded from the study.
e experimental design is shown in Figure 2; patients
were randomly assigned to try the RoBIK prototype or
the scanning device (described below) rst. Because there
are no well-documented learning eects in P300-based
BCIs,[51] the RoBiK prototype was evaluated only over
a single half-a-day session. Instead, the scanning speller
performance was estimated over three sessions in order to
estimate a possible learning eect.[52] Each session was
separated by at least a half-day wash-out period in order
to avoid a possible bias due to fatigue.
At the end of each session we assessed the fatigue and
satisfaction of patients by means of a VAS ranging from 0
(very tired / very unsatised) to 10 (not tired at all / very
satised). In both cases participants were asked to ll a
questionnaire to express their opinion on both techniques
(including comfort and fatigue).
Figure 2.Patients were randomly assigned to try first the RoBIK
BCI or the scanning device. The RoBIK prototype was evaluated
within half a day while the scanning device was evaluated over
three sessions in order to account for a possible learning effect.
Four different texts of equivalent difficulty were selected so that
the RoBIK device was evaluated with text 1 or 4.
202 L. MAYAUD ET AL.
Statistical analysis
For all tests, samples were tested for normality by means
of the Jarque-Bera test (signicance level at 5%) and ana-
lyzed with non-parametric or parametric tests depending
on whether the hypothesis of normal distribution of the
data was rejected or not, respectively. Tests for comparing
central location parameters (means) were chosen paired
or unpaired according to the test at hand. Dierences
between the healthy volunteers and patients (unpaired)
for the variables age, setup time, comfort, performance
during the training and online sessions were tested using
the Student t-test or the Mann Whitney U-test for nor-
mally and non-normally distributed samples, respectively.
Comparison of fatigue, setup time, and spelling accuracy
in patients (paired) using the RoBIK and the scanning sys-
tem (paired statistics) was assessed with a paired Student
t-test or a Wilcoxon test depending on whether data were
found to be normally distributed or not. Comparison of
fatigue in patients for each interface was estimated with
a repeated-measures ANOVA that uses a multivariate
framework (Hotelling T-square) to account for correlation
between measures,[53] which therefore does not need cor-
rection for sphericity.[54] Comparison of fatigue between
healthy volunteers and patients using the RoBIK interface
was estimated with the same technique considering two
independent groups of participants. For all statistical tests
the tolerance for type I error was set to 0.05. All anal-
yses were performed using Matlab (Version 8.0.0.783 -
R2012b) and toolboxes referenced.
Results
Study 1: clinical feasibility
Patients
Twelve quadriplegic patients admitted in adult medical
ICU [7] and rehabilitation units [5] met the inclusion cri-
teria and were consecutively included in the study aer
giving their informed consent. Table 1 summarizes the
sample, composed of eight men (67%) and four women,
aged from 22 to 63years old. e tolerance was good in
all patients. Four patients (33%) did not complete the pro-
tocol. Access to the occipital area was complicated by a
central catheter in the rst patient (#01). In one patient
a technical issue interrupted the inclusion (#02). One
patient fell asleep during the training session (#10). e
last patient (#11) was suspected to have eyesight problems,
although this could not fully be conrmed by medical les.
P300 speller for communication
e results of all patients included in Study 1 are presented
in Table 2. Out of 10 patients who completed the training
session, eight used the system during a test session with
pipeline for training to the MDM classication
algorithm. e data from the training sessions
were split into ve distinct training and test
sets in order to estimate the ‘real accuracy’
(percent of correctly identied characters
within 36 choices) on unseen data. e patient
was allowed to continue the protocol and pass
to the ‘online’ session if the estimated accu-
racy was above 70%, meaning that, on average,
three spelling errors every 10 characters were
tolerated, otherwise the participant was dis-
charged from the study.
(5) e ‘online’ session was equivalent to the pre-
vious one except that visual feedback was
provided to the patients by the classication
algorithm. e number of repetitions for
the online session was chosen to maximize
the estimated accuracy on the training data.
e online session was timed to last exactly
10 minutes, during which participants were
instructed to spell as many letters as possible
without corrections.
(6) Finally, patients could use the interface during
a ‘free-spelling’ session where no instruction
was given, but an output was still provided.
ese data are freely available on the Physionet
platform.2
The scanning system
An occupational therapist, as part of every day’s clinical
activity, was in charge of equipping patients with a ‘switch
interface’ (‘Buddy button’ switch, Ablenet, Roseville, MN,
USA). e nominal activation force of the switch varied
from 10 to 600 grams. Depending on the specic condi-
tion of the patient the following parameters were tuned by
the occupational therapist: activation force, size, location
(feet, near the head, thumb), type of activation (valida-
tion on press-down or release), and possible ltering of
double-clicks. Once a reliable switch command could be
obtained, the system was connected via the ‘Joycable’ USB
interface (Sensory soware, Malvern, Worcestershire,
UK) to the KEYVIT virtual keyboard (Jabbla, Ghent,
Belgium). On this interface, a regular grid of symbols
ashes lines at a regular pace (i.e. the ‘scanning’) until
the patient selects one line by activating the switch. Once
a line is selected, each element of the line is then ashed
until a second click selects the desired symbol or letter.
Aer installation of the interface, patients were asked to
spell as many characters as they could within 10 min. e
instruction text was printed big enough so that it could
be seen as displayed on an A4 sheet of paper next to the
screen.
BRAINCOMPUTER INTERFACES 203
from the study on his request. Two additional patients did
not meet the threshold of performance (cross-validated
accuracy on training session above 70%) and were dis-
charged from the study.
Comparison of patients and healthy volunteers
populations
For the RoBIK system, the setup time was 11.0 min (8.5–
15.5) for patients and 13.0 min (10.5–14.2) for healthy
volunteers (Figure 3). e dierence was not found sta-
tistically signicant (rank sum test, W = 66, p = .58).
Likewise, the experience was rated equally pleasant (rank
sum test, W = 85, p = .23), with patients reporting an
average rating of 2.5 (1.4–3.0) and healthy participants
an average rating of 1.5 (1.2–1.9). e comparison of
information transfer rate (bit-rate) between patients and
healthy controls (Figure 4) did not reveal signicant dif-
ferences (F(1,18) = 3.90, p=.07) even though a signicant
time factor (F(2,18) = 24.46, p<.001) and time-group
interaction eect was found (F(2,18) = 0.55, p< .001),
indicating that the drop in EEG discriminatory power
between the calibration and the test session was greater
in patients as compared to healthy volunteers. is might
accuracy signicantly above chance (2.6%) level (rank
sum test, W = 100, p < .001); seven could use the system
with more than half the symbols correctly identied and
an average accuracy of 84.7%. One patient had an accu-
racy of 40% and the BCI technique did not work for two
patients. Five out of eight patients (62.5%) who completed
the test session asked for the optional free spelling session
and spelled on average 10 characters with a median accu-
racy of 100.0% (72.2,100). Interestingly, the spelled words
were all the rst names of relatives. e overall satisfaction
for the technique amongst participants who completed the
training sessions was high (7.3/10).
Study 2: clinical evaluation
Participants
Table 3 summarizes demographic and other variables of
the participants.
As seen in Table 3, the patients recruited in this study
were on average 46.5years old (37.0–56.0) and the healthy
volunteers were on average 28.0years old (21.8–32.2). e
age dierence was statically signicant (rank sum test, W
= 136, p = .002). One patient was discharged prematurely
Table 2.Description of the population included in Study 1 with results: AC, access to computer; HM, voluntary head mobility; OE, oral
expression; MV, mechanical ventilation, ET, endotracheal intubation. ‘Treatments’ details drugs administered to patients during the 48 h
preceding the inclusion in the study. Training session performance is evaluated with a 20-fold cross-validation procedure on the training
data using a combination of xDAWN and SVM. It shows discrimination – area under the receiver operating curve (AUROC) – between
targets (33% of stimuli, i.e. oneone line and oneone column) and non-targets (random classifier is 50%). Testing and online performance
shows accuracy in letter selection (1 symbol in 36; chance is 2.7%). The last lines of the table summarize the distributions: categorical
variables are represented by the proportion of each category (noted in parenthesis) and continuous variables are represented by the
mean and standard deviation; whenever relevant, the sample size is indicated in parenthesis. For speller performance, brackets indicate
the average number of letters per patient. P-values are computed with a Mann-Whiteney U-test (*).
204 L. MAYAUD ET AL.
for the scanning device, a dierence that was found statis-
tically signicant (Kruskal-Wallis, H(1) = 11.48, p = .001).
e performance of the RoBIK system was signif-
icantly lower as compared to the performance of the
spelling device in the group of patients (the only group
having evaluated both techniques). e accuracy was 0.8
(0.5–0.8) and 1.0 (1.0–1.0) for the RoBIK system and
the scanning device, respectively, which was found stat-
ically signicant (Kruskal-Wallis, H(1) = 4.90, p = .027).
Likewise, the device speed for the RoBIK system was 0.5
characters per minutes (0.5–0.6) against 4.5 characters per
minute (3.5–5.0) for the scanning device (Student t-test,
t(4) = −14.78, p < .001). Naturally these translated into
indicate that the evoked potential amplitude (P300) of
patients decreases faster over time, which in turn might
relate to changes in attention. is interpretation is cor-
roborated by the results on self-reported fatigue, showing
that although no dierence between the populations can
be found (F(3,15) = 1.00, p=.33) and both samples get
tired during sessions (time factor, F(3,15) = 9.85, p=.001),
the eect is signicantly stronger in patients (interaction
factor time-fatigue, F(3,15) = 3.44, p=.049).
Comparison of RoBIK prototype with scanning device
e RoBIK system was associated in patients with a setup
time of 11.0 min (8.5–15.5) against only 3.0 min (2.0–4.2)
Table 3.Participants enrolled in Study 2 (patients and healthy volunteers). Abbreviations: ABP, arterial blood pressure – systolic (s) and
diastolic (d); bpm, beats per minute; F, female; GBS, Guillain-barré syndrome; HR, heart rate; LIS, locked-in syndrome, M, male; mmHg,
millimetres of mercury; MS, mulitple sclerosis; q, quadriplegic; s, scanning device.
Patients
Age 46 47 56 60 39 35 64 30 52 37
Gender M M F M F M m F F M
Pathology LIS GBS MS LIS LIS Q Q Q Q GBS
HR (bpm) 64 76 – 65 101 47 60 58 – 96
ABPs ABPd (mmHg) 13080 11172 – 150110 11173 5878 13060, 8646 – 8567
Randomization RoBIK S RoBIK S RoBIK S RoBIK RoBIK S S
Healthy volunteers
Age 33 50 26 31 32 28 21 21 22
Gender M F M F F M M M F
HR(bpm) 51 75 26 31 – 77 57 61 69
ABPsABPd(mmHg) 12375 11184 12369 11282 – 10882 12265 11166 13087
Figure 3.Comparison of setup time (min), self-evaluated comfort, spelling accuracy (%), and bit-rates (bit/min) between the RoBIK
system (online results) and the three sessions of the scanning device.
BRAINCOMPUTER INTERFACES 205
As reported in Table 2, we found no statistically
signicant mean dierence of performance between
mechanically ventilated patients (in the ICU, n = 7) and
non-ventilated patients (in the rehabilitation unit, n = 5)
over the dierent sessions: training (p = .46) and test (p
= .62). ese results have to be handled with care, as the
study was certainly not designed to study such an eect.
However, the individual performance reported in this
study show that neither mechanical ventilation, nor the
origin of tetraplegia, nor the use of CNS depressants may
be reliably related to failure at controlling the BCI, which
certainly is encouraging.
For three patients who could not achieve adequate
performance during the test session, several possible
explanations were identied. A technical problem on the
acquisition module of the soware degraded the system
performance for patients 2 and 3 (which was subsequently
xed on the open-source platform). is illustrates a well-
known limitation of existing BCI systems (usability of
hardware and soware components) when it comes to
their transfer to the patients’ bedside [57]; we believe that
the joint eorts of the scientic and industrial commu-
nities is gaining momentum to address these limitations.
Patient 9 reported diculties in simultaneously swallow-
ing and concentrating on the task, nally achieving low
performance. Patient 10 fell asleep during the training
session despite initially showing interest and motivation
for the protocol; this was possibly explained by painful
GBS accompanied by sleep deprivation. ere was no
obvious reason for the low performance found in patient
11 apart from a light visual impairment. is patient may
be a ‘BCI illiterate’.[58]
To the best of our knowledge, this is the rst compre-
hensive attempt to explore the use of a brain-computer
interface in an adverse clinical context. Four key factors
were explored: rst, a non-controlled clinical environ-
ment and the particularly adverse setup of the ICU (58%);
second, all ICU patients were evaluated during invasive
the signicant superiority of the scanning system over
the RoBIK system, with bit-rates of 28.3bpm (19.0–31.1)
and 1.7bpm (1.1–2.1), respectively (Student t-test, t(4) =
−12.10, p < .001).
In terms of self-reported fatigue in the patient group,
it was found signicantly greater while using the RoBIK
system, 2.7 (2.6–3.0), as compared to using the scan-
ning device, 2.0 (1.1–2.9) (F(1,29) = 49.83, p <.001).
Interestingly, the repeated-measures ANOVA did not
identify the time factor (measured before and aer inter-
vention) as signicant (F(1,29) = 1.96, p = .172), instead
suggesting a strongly signicant time-technique interac-
tion eect (F(1,29) = 16.33, p<.001), indicating that the
BCI system was associated with greater fatigability. ese
results, however, did not translate into a decreased self-re-
ported comfort, which was found equally good for the two
techniques: 4.3 (2.2–6.8) for BCI and 5.0 (4.3–6.2) for the
scanning device (Student t-test; t(7) = −0.60, p = .565).
Discussion
Study 1: clinical feasibility
Feasibility studies and evaluation of performance of
P300 speller systems so far have concentrated on rela-
tively small samples (n < 10) of late-stage ALS or acquired
brain injury patients using the system at home or in a
controlled environment.[16,27,36,55,56] A comparison
of performance between normal and severely disabled
participants showed signicantly better performance in
the healthy population,[29] which is a strong rationale
for the evaluation of this technology with a larger variety
of patients and preferably in a natural environment. A
recent study on a bigger cohort (n = 27) of ALS patients
at home mentions mechanical ventilation, but it is unclear
if patients where using the ventilators during the sessions.
[30] e Study 1 aimed at the investigation of the use of
BCI by acute patients in their real environment of use
(here, the ICU).
Figure 4.Comparison of the evolution of fatigue (left) and information rate (right) for patients (dark line) and healthy volunteers (bright
line) over three times of the BCI protocol (train, online, free).
206 L. MAYAUD ET AL.
the RoBIK prototype were specically designed to reduce
fatigue: for instance, visual fatigue was limited by the use
variable ISI, which is illustrated by Figure 5. Since the
patient population was signicantly older it is not pos-
sible to disentangle the eect of age from that of fatigue.
is certainly constitutes a signicant limitation of our
study, even though one can argue that the direction of this
eect is nonetheless unfavorable to the patient population
since fatigability is expected to increase with age. Such
a hypothesis is, however, in contrast with evidence of a
positive correlation between age and performance in a
P300 speller, which was reported in healthy volunteers
as well as in patients.[59,60] e retrospective analysis
of existing cohorts could help investigate this correlation
further.
e comparison of online performance for the use of
BCI and the scanning system in the patient population
showed that the BCI did not compare advantageously to
the traditional assistive technology. All measured indi-
cators (with the exception of self-reported comfort and
satisfaction – NS) were found to be in favor of the scan-
ning system: setup time was longer for the BCI; accuracy,
speed, and bit-rate were all found better for the scanning
device. Again, fatigue was strongly associated with the use
of the BCI system, since both the technique and the time
× technique interaction factors were signicant.
Oine analysis of clinical data
e analysis of the EEG data collected during the two clin-
ical trials described in this paper is reported in Appendix
3. e results we obtained stress the added value of the
Riemannian approach over state-of-the art classication
techniques, which was found particularly relevant for the
mechanical ventilation; third, patients with myopathy or
Guillain-Barré syndrome (42%) are populations that have
not yet been reported to use BCI; nally, this study pro-
vides an initial insight into the impact of central nervous
system (CNS) depressants on concomitant use of a P300
ERP-based BCI for communication. While this study was
clearly not dimensioned to fully explore this phenomenon,
we welcome the successful use of the BCI by some patients
under high levels of CNS depressant, which could have
primarily been thought to be prohibitive. Hence, further
research is required to exactly understand the extent to
which this kind of medication inuences the performance
of a BCI.
Study 2: clinical evaluation
e comparison of the BCI performance of healthy vol-
unteers and patients showed no statistical dierence
in the primary performance outcome (bit-rate). Our
nding adds to the small and contrasted available liter-
ature.[57,59] However, the progression of bit-rate over
time within the same day, in particular for patients, was
found signicant and corroborated the evolution of
self-reported fatigue, which increased in both groups, but
was stronger in patients. e inclusion of fatigue and time
in the analysis might in part explain the discrepancies
with the literature. We have included self-reported fatigue
in this study because we knew that the typical population
of quadriplegic patients admitted to the ICU is prone to
fatigue and because the outcome of Study 1 indicated
that fatigue was a recurrent complainr about the system.
As a consequence, we hypothesized it could constitute
an important limitation for the use of BCIs in this pop-
ulation. is was despite the fact that some elements of
Figure 5.Example of two event-related potentials (ERPs) elicited from the interface used in the pilot study (left) and the RoBIK prototype
(right); the RoBIK prototype does not show the typical steady-state visual evoked potential (SSVEP) in response to non-target stimulations
that can be seen on the non-target response (left).
BRAINCOMPUTER INTERFACES 207
hardware and soware appear to open a new era where
these limitations will be overcome.
A feasibility study was carried out in the ICU and in a
rehabilitation unit in a population of quadriplegic patients
using a traditional BCI system for communication (the
‘P300 speller’). Twelve patients were included in the study.
Among them, eight could successfully control the sys-
tem with above-chance accuracy. Performance could not
directly be related to the presence of either mechanical
ventilation or sedation. In contrast, eyesight and fatigue
were identied as possible limitation factors.
Based on this pilot phase, technical specications and
user requirements were draed for the development of
adequate bedside BCI prototypes for communication. e
hardware was a 14-channel EEG system that could be set
up in approximately 10 min. e soware was developed
with a user-friendly interface and state-of-the art pro-
cessing/classication techniques (based on Riemannian
geometry) in the back-end.
e prototype’s performance was evaluated in a clinical
trial. Its performance was compared to the performance
of traditional assistive technology (a scanning spelling
device) in patients. e comparison was carried out using
bit-rate, accuracy, setup time, comfort, and fatigue, con-
rming the superiority of the traditional AT technique
in terms of simplicity of use (setup time), cost, perfor-
mance, and – most importantly – patient fatigue. e
BCI was also evaluated in a group of healthy volunteers
to benchmark performance, showing that, if both groups
performed equally, a stronger decrease in performance
over time was observed in patients. is further strength-
ens the importance of fatigue in this population.
Finally, three oine analyses were carried out using
the data collected during phases 1 and 3:
• First, the ‘personalized’ estimated performance of
the BCI rate was derived from the training data,
pseudo-prospectively applied to the test set (online
session) and the results were compared to those of
the scanning device. e analysis conrmed that
the bit-rate is greatly improved when an optimal
and patient-specic number of repetitions is cho-
sen. e resulting performance was found compa-
rable to that of the scanning spelling device.
• Second, the individual added value of each element
of the design was quantied in an oine analysis. It
showed that the gain in ease of use generated by the
new headset came at no cost in performance. It also
conrmed the superiority of Riemannian methods
over state-of-the-art techniques for ERP detection.
However, the rejection of artifactual segments of
the data did not increase the performance above the
threshold of statistical signicance.
development of calibration-freeBCIs (also referred to as
‘cross-learning’). Conversely, the analysis did not identify
the presence of a Riemannian SQI (artifact rejection) as
a factor contributing positively to the BCI performance,
which was surprising. On the hardware side, the oine
analysis revealed that the gain in setup time oered by the
hardware prototype developed was not associated with a
noticeable drop in system performance.
Put together these innovative steps in EEG data col-
lection and analysis have the potential to dramatically
reduce the setup and calibration time. is would limit
the accumulation of fatigue prior to the actual use of the
communication interface and thereby improve overall sys-
tem performance, which was shown to be greatly aected
by fatigue. ese results are encouraging for the future use
of P300 spellers for the communication of patients. Apart
from patients for whom a reliable neural interface could
not be found (so called ‘BCI illiterates’), patients’ perfor-
mance appeared not to dier signicantly from that of
healthy participants. Most importantly, adequately chosen
parameters for the BCI interface lead to an estimated per-
formance that was not found to dier signicantly from
that of a traditional assistive technology device (t = 0.56,
permuted p-value=.33). Again, these results should be
interpreted with extreme care because this study was not
statistically powered to answer these specic questions
and because we excluded three of ten patients (from Study
2) who could not use the BCI system.
Conclusions
Some neurological disorders leave patients with baseline
cognitive status and no or little communication capacity.
LIS patients – whatever their etiology – are a well-known
illustration of such a condition. Quadriplegic patients
with invasive mechanical ventilation are also temporar-
ily deprived of communication, resulting in additional
stress for all people involved with the management of
their disease. Traditional assistive technologies for chronic
patients, such as those suering from neurodegenerative
disorders, have long been used and most severe cases are
equipped, whenever possible, with a ‘scanning’ spelling
device interfaced with simple ‘click’ contactors. However,
the use of these systems requires the intervention of an
experienced occupational therapist who must nd the
optimal set of parameters for each patient. BCIs, on the
other hand, have long been promised as a potential ‘uni-
versal’ assistive device for chronic patients such as those
with ALS. So far BCI technology has consistently shown
major limitations such aslow information transfer rate
(low spelling rate and accuracy) and the need for cum-
bersome and expensive setups for the recording of the
EEG and its processing. However, recent advances in EEG
208 L. MAYAUD ET AL.
ORCID
Louis Mayaud http://orcid.org/0000-0002-3187-8030
Marco Congedohttp://orcid.org/0000-0003-2196-0409
References
[1] Vincent JL. Communication in the ICU. Intensive Care
Med. 1997;23(10):1093–1098.
[2] Haig AJ, Katz R, Sahgal V. Mortality and complications
of the locked-in syndrome. Arch Phys Med Rehabil.
1987;68(1):24–27.
[3] Frank Lopresti EF, Mihailidis A, Kirsch N. Assistive
technology for cognitive rehabilitation: state of the art.
Neuropsychol Rehabil. 2004;14(1-2):5–39.
[4] Simpson R, Koester H, LoPresti E. Evaluation of an
adaptive row/column scanning system. Technol Disabil.
2006;18(3):127–138.
[5] Simpson RC, Koester HH. Adaptive one-switch
row-column scanning. IEEE Trans Rehabil Eng.
1999;7(4):464–473.
[6] Biswas P, Robinson P. A new screen scanning system
based on clustering screen objects. J Assist Technol.
2008;2(3):24–31.
[7] Cook AM, Polgar JM. Assistive technologies: principles
and practice. Elsevier Health Sciences; 2014.
[8] Vidal JJ. Toward direct brain-computer communication.
Annu Rev Biophys Bioeng. 1973;2:157-180. Epub
1973/01/01. doi: 10.1146/annurev.bb.02.060173.001105.
PubMed PMID: 4583653.
[9] Birbaumer N. Breaking the silence: brain?computer
interfaces (bci) for communication and motor control.
Psychophysiology. 2006;43(6):517-532. Epub 2006/11/02.
doi: 10.1111/j.1469-8986.2006.00456.x. PubMed PMID:
17076808.
[10] Wolpaw JR, Birbaumer N, Heetderks WJ, et al. Brain-
computer interface technology: a review of the rst
international meeting. IEEE Trans Rehabil Eng.
2000;8(2):164–173. Epub 2000/07/15 PubMed PMID:
10896178.
[11] Teplan M. Fundamentals of EEG measurement. Meas Sci
Re v. 2002;2(2):1–11.
[12] Wolpaw JR, McFarland DJ, Neat GW, et al. An EEG-
based brain-computer interface for cursor control.
Electroencephalogr Clin Neurophysiol. 1991;78(3):252–
259. Epub 1991/03/01 PubMed PMID: 1707798.
[13] Picton TW. e P300 wave of the human event-related
potential. J Clin Neurophysiol. 1992;9(4):456–479. Epub
1992/10/01 PubMed PMID: 1464675.
[14] Kaper M, Meinicke P, Grossekathoefer U, et al. BCI
competition 2003—data set iib: support vector machines
for the p300 Speller Paradigm. IEEE Trans Biomed Eng.
2004;51(6):1073-1076, 0018-9294.
[15] Krusienski DJ, Sellers EW, McFarland DJ, et al. Toward
enhanced P300 speller performance. J Neurosci Methods.
2008;167(1):15.
[16] Sellers EW, Vaughan TM, Wolpaw JR. A brain-computer
interface for long-term independent home use.
Amyotroph Lateral Scler. 2010;11(5):449–455.
[17] Müller K-R, Blankertz B. Toward non-invasive brain-
computer interfaces. 2006.
• Last but not least, we investigated possible model
initialization using an existing database. e ration-
ale was to reduce fatigue by removing the tiring
calibration session. e analysis revealed that the
Riemannian methods enjoy superior cross-subject
generalization, providing a good initialization that
needs to be adapted online.[44]
To conclude, this study demonstrates that caregivers
who are unfamiliar with EEG and BCIs in general can
restore some form of communication in severely disabled
patients by means of an adequately designed BCI system.
However, the population of patients who could immedi-
ately benet from it was found to be much smaller than
initially expected, since traditional assistive techniques are
remarkably eective and compare very favorably to the use
of P300-based spellers. Today, this is probably – more than
ever – due to the lack of aordable, convenient, and reli-
able EEG systems, while other technological limitations
are nally being overcome. Algorithms, for instance, are
probably reaching a form of maturity and little further
improvement can be foreseen for the coming few years; in
our opinion, adaptive techniques based on cross-subject
and cross-session initialization will play a dominant role.
Fortunately, the ongoing transition of the EEG eld from
cottage industry to a more nancially structured sector
will soon translate into better and cheaper EEG systems.
In turn, BCIs will cover the needs of an increasing pro-
portion of severely disabled patients and ultimately nd
legitimate room next to other assistive technologies on
the occupational therapist’s shelf.
Notes
1. https://physionet.org/works/P300SpellerBCIICU/.
2. https://physionet.org/works/P300SpellerBCIICU/.
3. https://github.com/alexandrebarachant/
covariancetoolbox/commit/bcccb4d750b2ad9ae6dc3
dd76e7e6633f98bc1f3.
Acknowledgements
We would like to thank the medical sta who have been in-
volved on this project, and in particular Marjorie Figère, Mar-
jorie Dezeaux, and Sandra Potier from the CICIT, Jean-Marie,
Gilles, Prof. Lofaso, and Prof. Herault from the Functional Ex-
ploration Unit, Justine Bouteille from the New Technologies
Platform (PFNT), as well as many others involved in the project.
Funding
We would like to thank the National Research Agency (ANR)
and the General Directorate for Armament (DGA) for fund-
ing this project (Project ANR- 09-TECS-013–01-RoBIK). We
would like to thank the French Association for Myopathies
(AFM) for partly funding this project.
BRAINCOMPUTER INTERFACES 209
progress in pattern recognition, image analysis, computer
vision, and applications. Springer; 2015. p. 559-66%@
3319257501.
[36] Kübler A, Furdea A, Halder S, et al. A Brain-computer
interface controlled auditory event-related potential
(P300) spelling system for locked-in patients. Ann N Y
Acad Sci. 2009;1157(1):90–100.
[37] Guo M, Xu G, Wang L, Wang J, editors. Research on
auditory BCI based on wavelet transform. Virtual
Environments Human-Computer Interfaces and
Measurement Systems (VECIMS), 2012 IEEE
International Conference on. IEEE; 2012.
[38] Vienne J, Lecciso G, Constantinescu IO, et al. Dierential
eects of sodium oxybate and baclofen on EEG, sleep,
neurobehavioral performance, and memory. Sleep.
2012;35(8):1071-83%@ 0161-8105.
[39] Mugler EM, Ruf CA, Halder S, et al. Design and
implementation of a P300-based brain-computer interface
for controlling an internet browser. IEEE Trans Neural Syst
Rehabil Eng 2010;18(6):599-609. Epub 2010/09/02. doi:
10.1109/TNSRE.2010.2068059. PubMed PMID:
20805058.
[40] Mayaud L, Filipe S, Pétégnief L, et al. Robust brain-
computer interface for virtual keyboard (RoBIK): project
results. IRBM. 2013.
[41] Mayaud L, Cabanilles S, Azabou E. Patient needs analysis
for brain-computer interfaces. In: Clerc M, Bougrain L,
Lotte F, editors. Brain computer interfaces. Great Britain:
ISTE Editions. Croydon, CR0 4YY; 2016.
[42] Renard Y, Lotte F, Gibert G, et al. OpenViBE: an open-
source soware platform to design, test, and use brain–
computer interfaces in real and virtual environments.
Presence-Teleop Virt Environ. 2010;19(1):35–53.
[43] Schlögl A. GDF-A general dataformat for biosignals.
arXiv preprint cs/0608052. 2006.
[44] Congedo M, Korczowski L, Delorme A. Spatio-temporal
common pattern: a companion method for ERP analysis
in the time domain. J Neurosci Methods. 2016;267:74-
88%@ 0165-270.
[45] Congedo M, Gouy-Pailler C, Jutten C. On the blind
source separation of human electroencephalogram
by approximate joint diagonalization of second order
statistics. Clin Neurophysiol. 2008;119(12):2677–2686.
[46] Rakotomamonjy A, Guigue V, Mallet G, et al.
Ensemble of SVMs for improving brain computer
interface P300 speller performances. Lect Notes
Comput Sc. 2005;3696:45–50. PubMed PMID:
ISI:000232193800008.
[47] Lotte F, Congedo M, Lécuyer A, et al. A review of
classication algorithms for EEG-based brain–computer
interfaces. J. Neural Eng. 2007;4.
[48] Barachant A, Andreev A, Congedo M, editors. e
Riemannian Potato: an automatic and adaptive artifact
detection method for online experiments using
Riemannian geometry. TOBI Workshop lV. 2013.
[49] Congedo M. EEG Source Analysis Université de
Grenoble; 2013.
[50] Barachant A, Bonnet S, Congedo M, et al. Multiclass
brain-computer interface classication by Riemannian
geometry. IEEE Trans Biomed Eng 2012;59(4):920-928.
Epub 2011/10/20. doi: 10.1109/TBME.2011.2172210.
PubMed PMID: 22010143.
[18] Grozea C, Voinescu CD, Fazli S. Bristle-sensors–low-cost
exible passive dry EEG electrodes for neurofeedback
and BCI applications. J Neural Eng. 2011;8(2):025008.
Epub 2011/03/26. doi: 10.1088/1741-2560/8/2/025008.
PubMed PMID: 21436526.
[19] Popescu F, Fazli S, Badower Y, et al. Single trial
classication of motor imagination using 6 dry EEG
electrodes. PLoS ONE. 2007;2(7):e637.
[20] Sellers E, Turner P, Sarnacki W, et al. A novel dry electrode
for brain-computer interface. Human-Computer
Interaction Novel Interaction Methods Tech. 2009:623-
631.
[21] Hornecker A. Dear customers, dear friends of brain
products. Brain. 2009;33.
[22] Renard Y. In the focus. Brain. 2010;37.
[23] Emotiv E. Soware Development Kit. 2010.
[24] Liu Y, Jiang X, Cao T, Wan F, Mak PU, Mak PI, et al,
editors. Implementation of SSVEP based BCI with
emotiv EPOC. Virtual Environments Human-Computer
Interfaces and Measurement Systems (VECIMS), 2012
IEEE International Conference on; 2012: IEEE.
[25] Duvinage M, Castermans T, Dutoit T, Petieau M,
Hoellinger T, De Saedeleer C, et al, editors. A P300-based
quantitative comparison between the emotiv epoc headset
and a medical eeg device. biomedical engineering/765:
telehealth/766: assistive technologies. ACTA Press; 2012.
[26] Stytsenko K, Jablonskis E, Prahm C, editors. Evaluation
of consumer EEG device emotiv epoc. MEi: CogSci
Conference 2011, Ljubljana; 2011.
[27] Sellers EW, Donchin E. A P300-based brain–computer
interface: initial tests by ALS patients. Clin Neurophysiol.
2006;117(3):538–548.
[28] Kubler A, Nijboer F, Mellinger J, et al. Patients with
ALS can use sensorimotor rhythms to operate a brain-
computer interface. Neurology. 2005;64(10):1775-1777,
0028-3878.
[29] Piccione F, Giorgi F, Tonin P, et al. P300-based brain
computer interface: reliability and performance in
healthy and paralysed participants. Clin Neurophysiol.
2006;117(3):531–537.
[30] McCane L, Vaughan TM, McFarland DJ, et al. Evaluation
of individuals with ALS for in-home use of a P300 brain-
computer interface. Soc Neurosci, Submitted. 2009.
[31] Li K, Narayan Raju VN, Sankar R, Arbel Y, Donchin E.
Advances and challenges in signal analysis for single trial
P300-BCI. foundations of augmented cognition directing
the future of adaptive systems. Springer; 2011. p. 87–94.
[32] Khan OI, Kim S-H, Rasheed T, Khan A, Kim T-S, editors.
Extraction of P300 using constrained independent
component analysis. Engineering in Medicine and
Biology Society, 2009 EMBC 2009 Annual International
Conference of the IEEE. IEEE; 2009.
[33] Rivet B, Souloumiac A, Attina V, et al. xDAWN
algorithm to enhance evoked potentials: application
to brain–computer interface. IEEE Trans Biomed Eng.
2009;56(8):2035–2043.
[34] Rakotomamonjy A, Guigue V, Mallet G, et al. Ensemble
of SVMs for improving brain computer interface P300
speller performances. Articial Neural Netw: Biol
Inspirations–ICANN 2005. 2005:45-50.
[35] Patrone M, Lecumberry F, Martín Á, Ramirez I, Seroussi
G. EEG Signal Pre-processing for the p300 speller.
210 L. MAYAUD ET AL.
[66] Congedo M. Introducing the logistic discriminant
function in electroencephalography. J Neurother.
2003;7(2):5–23.
[67] Serby H, Yom-Tov E, Inbar GF. An improved P300-
based brain-computer interface. IEEE Trans Neural Syst
Rehabil Eng. 2005;13(1):89–98.
[68] Wolpaw JR, Ramoser H, McFarland DJ, et al. EEG-
based communication: improved accuracy by response
verication. IEEE Trans Rehabil Eng. 1998;6(3):326–333.
[69] Hosmer DW Jr, Wang CY, Lin IC, et al. A computer
program for stepwise logistic regression using
maximum likelihood estimation. Comput Prog Biomed.
1978;8(2):121–134. Epub 1978/06/01 PubMed PMID:
668307.
[70] Congedo M, Barachant A, Andreev A A New generation
of brain-computer interface based on riemannian
geometry. arXiv preprint arXiv:13108115. 2013.
[71] Guger C, Krausz G, Allison BZ, et al. Comparison of
dry and gel based electrodes for P300 brain–computer
interfaces. Front Neurosci. 2012;6.
Appendix 1. Details of hardware developments
e EEG headset was designed to minimize the need for EEG
recording preparation: as a primary goal in the design phase,
skin preparation with abrasive paste, electrode positioning with
gel, and retention with tape had to be avoided. e resulting
headset is a 3D-printed polyamide structure holding 14 Ag-
Cl electrodes (Neuroservices, Evry, France) located at standard
10/20 locations: Po7, Fp1, Oz, Fz, C3, F4, Pz, C4, P7, Fp2, F3,
P8, Po8, and Cz. ese locations were selected in order to max-
imize the chance of capturing P300 ERPs using as reference
and ground the right and le mastoids, respectively, according
to previous studies by the RoBIK consortium.[61] Each elec-
trode was mounted on a polyethurane wheel connected to a
spring in order to control the pressure applied to the scalp with
the electrode. e headset was designed to meet the need of pa-
tients and safety requirements, as well as to be compliant with
the ICU environment. In particular, the headset could be used
in a lying position or in the presence of a headrest.
e headset contains an electronic component that was de-
signed to record EEG activity with high delity while maintain-
ing minimal volume and weight. An MSP430 ultra-low-power
microcontroller (Texas Instrument, Dallas, USA) was used to
control a dedicated application-specic integrated component
(ASIC) developed for this purpose: the CIrcuit for NEuronal
SIgnal Conversion (CINESIC32). Each input channel is com-
bined with an external capacitor (1.5 nF) in order to suppress
the risk of leaking current in a rst default condition, which
is essential for medical applications. e analogue channel is
composed of a fully dierential low-noise amplier, followed
by a voltage gain amplier and a programmable low-pass l-
ter. Each channel consumes about 34 μA, summing up to an
averaged consumption of 13mA at full data streaming includ-
ing baseline consumption at 3.3V. With these settings 24-hour
continuous operation can be achieved with one high-ener-
gy-density 3.6V lithium battery. Only 14 of the 32 available
channels were used in the nal prototype and each of them
was congured with a (0.5–30Hz) band-pass lter and a 60dB
voltage gain; analogue signals were digitized through a 12-bit
[51] Nijboer F, Birbaumer N, Kübler A. e inuence of
psychological state and motivation on brain–computer
interface performance in patients with amyotrophic
lateral sclerosis–a longitudinal study. Front Neurosci.
2010;4.
[52] Stewart H, Wilcock A. Improving the communication
rate for symbol based, scanning voice output device
users. Technol Disabil. 2000;13(3):141–150.
[53] Rencher X. Methods of multivariate data. Analysis. 1995.
[54] Pernet CR, Chauveau N, Gaspar C, et al. Limo
EEG: a toolbox for hierarchical linear modeling of
electroencephalographic data. Comput Intell Neurosci.
2011;2011:3.
[55] Nijboer F, Sellers E, Mellinger J, et al. A P300-based brain–
computer interface for people with amyotrophic lateral
sclerosis. Clin Neurophysiol. 2008;119(8):1909–1916.
[56] Riccio A, Leotta F, Tiripicchio S, et al. Can severe Acquired
Brain Injury users control a communication application
operated through a P300-based brain computer interface?
Age. 2011;41(46):24.
[57] Cipresso P, Carelli L, Solca F, et al. e use of P300‐based
BCIs in amyotrophic lateral sclerosis: from augmentative
and alternative communication to cognitive assessment.
Brain Behav. 2012.
[58] Guger C, Daban S, Sellers E, et al. How many people are
able to control a P300-based brain? computer interface
(BCI)? Neurosci Lett. 2009;462(1):94–98.
[59] Silvoni S, Volpato C, Cavinato M, et al. P300-based
brain–computer interface communication: evaluation
and follow-up in amyotrophic lateral sclerosis. Front
Neurosci 2009;3(60).
[60] Kübler A, Birbaumer N. Brain–computer interfaces
and com- munication in paralysis: Extinction of goal
directed thinking in completely paralysed patients? Clin
Neurophysiol. 2008;119:2658–2666.
[61] Cecotti H, Rivet B, Congedo M, et al. A robust sensor-
selection method for P300 brain–computer interfaces. J
Neural Eng. 2011;8(1):016001.
[62] Oostenveld R, Fries P, Maris E, et al. FieldTrip: open
source soware for advanced analysis of MEG, EEG,
and invasive electrophysiological data. Comput Intell
Neurosci 2011;2011:156869. Epub 2011/01/22. doi:
10.1155/2011/156869. PubMed PMID: 21253357;
PubMed Central PMCID: PMC3021840.
[63] Congedo M, Goyat M, Tarrin N, Ionescu G, Varnet L,
Rivet B, et al, editors. Brain Invaders’: a prototype of an
open-source P300-based video game working with the
OpenViBE platform. Proceedings of the 5th International
Brain-Computer Interface Conference 2011; 2011.
[64] Jin SH, Lin P, Hallett M. Abnormal reorganization
of functional cortical small-world networks in focal
hand dystonia. PLoS ONE. 2011;6(12):e28682. Epub
2011/12/17. doi: 10.1371/journal.pone.0028682.
PubMed PMID: 22174867; PubMed Central PMCID:
PMC3236757.
[65] Townsend G, LaPallo BK, Boulay CB, et al. A novel P300-
based brain–computer interface stimulus presentation
paradigm: moving beyond rows and columns Clin
Neurophysiol 2010;121(7):1109-1120. Epub 2010/03/30.
doi: 10.1016/j.clinph.2010.01.030. PubMed PMID:
20347387; PubMed Central PMCID: PMC2879474.
BRAINCOMPUTER INTERFACES 211
where
𝜆e,e=1…E
are the E eigenvalues of
Σ−1
1
Σ
2
or of
Σ−1
2
Σ
1
. Using this distance, a geometric center of mass
M
(or Fréchet
mean) of a set of covariance matrices can be estimated with a
gradient descend algorithm solving the following optimization
problem:
In other words, just as the mean in Euclidean space is the val-
ue minimizing the variance, the Riemannian center of mass
M of a set of points is the point minimizing their dispersion
(variance) in the manifold. For every covariance matrix
Σi
, we
can compute a distance δi to the center of mass of the data set
as
𝛿i
=𝛿
(
Σ
i
,M
)
. Finally, given all distances we can compute a
scalar geometric mean μ and and a scalar geometric standard
deviation σ of them using the following formula [49]:
which for Riemannian distances will approximate a symmet-
ric distribution.[49] Finally, we can then derive z-scores of the
distances as
Once μ and σ are estimated on the training data, covariance
matrices computed on new online EEG epochs are discarded
whenever their distance to the center of mass exceeds a z-score
of 2.5. e set of points on the manifold having a distance to a
center of mass less than 2.5 standard deviations forms a closed
region; in three dimensions, such a region would look like a
potato rather than a sphere, because of the non-linear nature
of the Riemannian manifold, which is why this technique was
named the ‘Riemannian Potato’.[48] During online experi-
ments, visual feedback was provided to patients so that they
could relate artifacts to specic behaviors (blinks, coughing,
head movements). is ensures that they can associate any ar-
tefactual behavior with its impact on the data quality and take
corrective actions: move less and blink at appropriate times as
much as possible. e Riemannian potato was implemented in
OpenViBE for online experiments (Study 2) and in Matlab3
for oine experiments (Study 1).
Online classier
Using the Riemannian distance [1] and the denition of
center of mass,[2] the detection of the P300 evoked potential
can be achieved by a deceptively simple classication algorithm
named the minimum distance to means (MDM).[50] Denot-
ing by+the target class of ashes, each trial
Xi
is concatenated
with a prototypical P300 evoked response
P+
(for instance, the
ensemble average estimation obtained on the training data) to
build a ‘super’ trial:
A special covariance matrix is then estimated using this su-
per trial such as
(2)
argmin
M
∑
i
𝛿2
(
M,Σi
)
(3)
𝜇
=exp
1
L
i
ln
𝛿i
,𝜎=exp
1
L
i
ln
𝛿i
𝜇
2
,
(4)
z
𝛿i
=
ln
𝛿i
𝜇
ln(𝜎).
(5)
̃
X
i=
[
P
+
X
i].
(6)
̃
Σ
i=
1
N−1
�
Xi�
Xi
T
analogue-to-digital converter (ADC) with nominal sampling
frequency of 580Hz per channel.
Raw EEG were transmitted to a eld-trip buer [62] that
was subsequently read by an OpenViBE acquisition server.[42]
Signals were then band-pass-ltered between 1 and 20Hz with
a fourth-order Butterworth with linear phase response and
decimated to 145Hz for further analysis.
Appendix 2. Details of software development
Graphical user interface
e Brainmium speller interface was designed to automate the
calibration of machine learning algorithms implemented in
OpenViBE and to provide a user-friendly interface for patients
and operators. Rather than leaving the application synchro-
nizing the received EEG data with the precise temporal occur-
rence of visual stimulations, which would be prone to delays
and jitter induced by the operating system (Windows 7 profes-
sional edition, Microso Corporation, Redmond, WA, USA),
we decided to send these stimulations through the USB di-
rectly to the EEG electronic acquisition unit to obtain accurate
timestamps for all stimulation events. Flashes had a duration
of 75ms [55] and random inter-stimulus intervals (ISI) were
drawn from an exponential distribution with mean
𝜆=0.15s
.
In previous research by our consortium, an exponential ISI was
found to reduce the cortical fatigue associated with stimulation
at xed frequency while enhancing the ERP single-trial estima-
tion.[63] In the original P300 speller paradigm symbols ash
by rows and columns. Oen detection errors arise because of
the ‘adjacency-distraction’ phenomenon,[64,65] according to
which non-target symbols in rows or columns adjacent to the
target attract the user’s attention when they ash, producing
evoked activity similar to the P300, making the detection of
the target P300 more dicult. To mitigate this eect we ash
the symbols by random groups.[63] Not only is the ‘adjacen-
cy-distraction’ eect mitigated, we also nd that the pattern of
ashing becomes totally unpredictable, which is expected to
sustain the attention of the user and to enhance the P300. More
importantly, random-group ashing allows arbitrary position-
ing of the symbols on the screen (no more need to arrange
symbols on a grid), which greatly expands the usability of the
P300 paradigm.
Online artifact rejection
e online artifact detection was initialized at the begin-
ning of each session in order to interrupt the sequence of
stimulations in the presence of excessive noise. is technique
allows a simple denition of a rejection region for incoming
data segments. e participant was instructed to stay still for
10 seconds during which a series of nsqi clean overlapping EEG
time-windows
X
i=1..n
sqi
of dimension E×S were extracted, with
S the number of samples in the time-window and E=14 the
number of electrodes. For each time-window
Xi
, a covariance
matrix
Σi
of dimension E×E was computed. Covariance ma-
trices belong to the Riemannian manifold of symmetric pos-
itive-denite matrices wherein a Riemannian metric can be
used to dene a distance δ between any two covariance matri-
ces such as [50]:
(1)
𝛿(
Σi,Σj
)
=∥ log
(
Σ−1∕2
1Σ2Σ−1∕2
1
)
∥=
[∑
e
log2𝜆e
]1∕2
,
212 L. MAYAUD ET AL.
amount of choices it oers. Bit-rates thereby gives a more rep-
resentative metric of ‘information ow’ from the participant’s
brain to the machine. e bit-rate br is dened as
where M is the number of symbols and p the spelling accuracy
(the percentage of correctly identied symbols). In this study,
M equals 36 and 73 for the RoBIK BCI and the scanning de-
vice, respectively.
e evaluation of the RoBIK prototype was designed to
demonstrate the feasibility of a well-designed BCI prototype
for communication in a clinical context. erefore, the stim-
ulation parameters (ash duration, average time between two
stimulations, and number of repetitions) were chosen to pro-
mote optimal accuracy rather than speed. e system evaluat-
ed at the bedside in this study was not optimized for bit-rate,
but for optimal data collection and robustness in the context
of a clinical feasibility investigation. is approach ensured the
collection of a large amount of data for the oine analysis. For
the BCI performance, we consider the bit-rate achieved with
the optimal number of repetitions. In order to assess the opti-
mal number of repetitions, the MDM classier described above
was tted to the training data aer artifact identication using
the aforementioned SQI index. e test set was a bootstrapped
dataset generated from the ‘online’ session of each participant
as follows: for each number of repetitions of the stimulation
k=1…50 (that is the number of times a specic letter is ashed
before a decision is made), B= 1000 groups of k target and
ve times k (5000) non-target responses were randomly se-
lected (with replacement) and averaged, so that the target to
non-target ratio was preserved. en, for each group of six re-
sponses, the predictions were obtained by assigning 1 to the
element featuring the minimum distance to the center of mass
of the Target class and 0 to the ve other elements. e seed
for the random sampling was preserved in order to evaluate
the performance of the dierent techniques on the same ran-
dom bootstraps. At the end of this procedure, 6000 predictions
were used to derive a performance index for each participant
and for each number of repetitions of the visual stimulation
(the ash). In particular, the ‘real accuracy’ was dened as the
square of the raw accuracy, since each letter is found at the in-
tersection between two groups of symbols (commonly referred
as the ‘lines’ and ‘columns’ in the traditional P300 speller ex-
periments). If predictions were to be made at random, the ‘real
accuracy’ would be 1 in 36, meaning that there is a probabil-
ity of 2.8% that the correct letter is selected by chance. With
such a denition it was arbitrarily decided that an interface
should provide a minimum ‘real accuracy’ of 70%, meaning
(1)
br
=log2(M)+p.log2p+(1−p).log2
1−p
M−1,
If the data have been previously band-pass ltered in an appro-
priate band-pass region, such a super covariance matrix con-
tains all the spatial and temporal information needed to achieve
the detection of ERPs,[70] therefore it can be used directly as a
single feature for the classication algorithm as a point on the
Riemannian manifold. For each of the two classes, Target (+)
and Non-Target (−), a Riemannian center of mass is estimated
using the data from the calibration session. e center of mass
can be understood simply as the expected covariance matrix of
a trial belonging to the corresponding class, wherein the use of
the Riemannian metric ensures that this expectation is a much
better representative as compared to the arithmetic mean. In
particular, extensive testing presented in [70] has established
that this expectation is more robust to noise and outliers. e
classication of an unseen trial is achieved by comparing the
distance of the trial to the center of mass of the Target and
Non-Target class. A classication score is appliedto unseen tri-
als according to the score function
where
𝜋
(x)=
1
1
+
exp
−(𝛼+𝛽x
)
are probabilities found by tting a
logistic regression curve with parameter α and β [66] to the set
of distances
𝛿(
Σ
+
,Σi
)
and
𝛿(
Σ
−
,Σi
)
and
Σ+
and
Σ−
are the
center of mass of the Target and Non-Target classes, respective-
ly. ese scores are averaged across repetitions, and the symbol
is assigned to the symbol at the intersection of the row and
column with highest score.
Appendix 3. Oine analysis
is oine analysis has the following objectives:
• Experiment 1: comparison of the personalized BCI with
the traditional assistive technology;
• Experiment 2: assessment of the individual contribution
of each component (RoBIK headset, artifact-detection
algorithm, classier algorithm) to the overall system
performance;
• Experiment 3: assessment of the performance of a cali-
bration-free BCI system.
Materials and methods
Elements of the materials and methods are summarized in
Table 4.
Oine analysis 1: optimal BCI compared to scanning device
e comparison of performance for communication in-
terfaces is usually assessed using the bit-rate.[67,68] e ad-
vantage of bit-rate over accuracy as a performance measure is
that it also takes into account the speed of the interface and the
(7)
s
=𝜋𝛿
(
Σ
+
,Σi
)
−𝛿
(
Σ
−
,Σi
),
Table 4.Summary of materials and methods for both studies showing the population, hardware, software, algorithms, and cross-val-
idation procedures used in each phase. Abbreviations: minimum distance to mean (MDM) is a Riemannian classifier; support vector
machine (SVM) is a classifier; stepwise linear discriminant analysis (swLDA) is a linear classifier; xDAWN is a spatial filter for event-related
potentials; P1, P2, and V2 refer to patients and volunteers included in clinical studies 1 and 2, respectively.
Population Hardware Software Algorithms Cross-validation
Exp. 1 • Patients (P2)
• Volunteers (V2)
• RoBIK • OpenViBE
• Matlab 14
• Riemannian Potato
• MDM
• 5-Fold (Training)
• Cross-session
Exp. 2–3 • Patients (P1)
• Patients (P2)
• Volunteers (V2)
• TMSi
• RoBIK
• OpenViBE
• Matlab 14
• Riemannian Potato
• MDM
• xDAWN+swLDA
• 5-Fold (Training)
• Cross-session
• Cross-participant
BRAINCOMPUTER INTERFACES 213
patients and benchmarks the performance of the RoBIK EEG
headset versus a traditional disc electrode system (TMSi am-
plier). As detailed in Figure 6, these comparisons were car-
ried out at dierent levels: with a ve-fold cross-validation
on the training set, prospectively on the online dataset, and
using cross-participant tting. For the latter, only data from
other participants wee used to t the model. e comparison
of optimal (maximum) bit-rates over all repetitions for these
conditions reveals a signicant benet of the MDM over the
xDAWN+SWLDA approach (paired t-test, t = 32.4, permuted
p < .001). is was particularly true for the cross-participant
condition where only the MDM model could generalize well
(paired t-test, t = 42.5, permuted p<.001). Other factors were
not found signicant. is seems to indicate that patients and
healthy volunteers have equivalent performance. Likewise, the
performance of the RoBIK headset did not signicantly dier
from that of the traditional EEG system, which strengthened
the rationale for the improvement in setup time (from 30 min-
utes to 10 minutes). Finally, these results also indicated that
the use of an artifact-rejection technique does not necessarily
improve the accuracy, although it results in a reduction of var-
iability, which can be noticed only with the use of a traditional
EEG recording system.
Comparison of optimal BCI to scanning device
As seen in Table 5, seven out of nine patients (77%) who
completed the Online session during Study 2 could successful-
ly use the P300 speller. For those (we excluded two additional
patients P05 and P09 who could not use the system at all), the
comparison of bit-rates did not show statistical signicance be-
tween the rst scanning session and the personalized RoBIK
performance (T = 0.56, permuted p-value=.33).
Discussion
First, the results conrm the superiority of the Riemannian
MDM approach over traditional classication based on spatial
ltering (for Online mode, line 2 on Figure 6). is nding
replicates previous reports on healthy participants.[49,50,70]
In some patients, the Riemannian MDM oers real accuracy
above 80% with as little as two repetitions, which favorably
compares with state-of-the-art techniques.[71] is supe-
riority was even more evident for the cross-subject transfer
learning (Cross-subject, line 3 on Figure 6), which is also in
line with our previous investigations.[50,70] Overall these
results suggest that the Riemannian MDM classier oers
better generalization properties as compared to spatial lters.
Interestingly, the advantages of the MDM algorithm in terms
of accuracy and generalization are accompanied by a dramat-
ic reduction of the algorithmic complexity; the Riemannian
MDM requires only the computation of centers of mass and
distances between two points and has no free parameters to
be tuned by cross-validation or heuristics, therefore it reduc-
es the risk of over-tting. In addition to this, Riemannian
methods generalize well across sessions (see Figure 6, line 2
column 2) because the Riemannian distance function is invar-
iant by any linear transformation of the data and covariance
shis observed across sessions due to movement in headset
positions or changes in electrode impedance are of this type.
As expected, the performance of the cross-participant model
was found to be signicantly lower than that of the ‘online’
mode (trained on data from a training session from the same
participant), which implies that under these conditions online
adaptation may be required to keep the performance of the
calibration-free BCI optimal.
that on the average 7 out of 10 characters should correctly be
identied. e optimal number of repetitions was chosen ac-
cordingly, corresponding bit-rates were derived as described in
Equation 1, and subsequently compared to that of the scanning
device for the same patients.
Oine analysis 2: added value of the prototype design
A second oine analysis was run to quantify the added
value of the proposed BCI prototype design. More precisely,
we used an oine study to ensure that the hardware design,
which was meant to increase ease of use, came at no cost in
performance and we similarly quantied the added value of the
Riemannian approach for artifact trial rejection and classica-
tion. To do so, oine analysis 1 was repeated with and without
online artifact trial rejection using the SQI index described
in Appendix 2. Likewise, the performance of the Riemannian
MDM classier was compared to a state-of-the-art technique
composed of the xDAWN spatial lter combined with a step-
wise linear discriminant analysis (SWLDA).[69] e compari-
son was carried out with and without the artifact trial rejection
and on both clinical datasets (Studies 1 and 2). Because the
RoBIK interface natively interrupts visual stimulations in the
presence of noise, it is expected that most target and non-target
epochs from Study 2 are clean. For this reason, only data from
Study 1 were considered to assess the benet of artifact rejec-
tion, while both datasets were used to compare the classiers.
Oine analysis 3: cross-participant-analysis
A calibration-free ERP-based BCI has been proposed by
Congedo [70] and Barachant et al. [48]; the center of mass of
the available classes is initialized using a database of previous
users and then continuously updated using the incoming data
from the online sessions of the user. In order to assess the po-
tential of such a approach, oine analysis 2 was run a second
time replacing the training set by all data available from other
participants (cross-subject learning). While oine analyses 1
and 2 used little data from the same participant, more data are
available in this oine analysis; however, they belong to dif-
ferent participants. To allow for cross-subject comparison, all
covariance matrices for the MDM models were normalized so
as to have a unit determinant as:
which derives directly from the following property of the de-
terminant:
det(cΣ)=cEdet(Σ)
, where
det (Σ)
denotes the de-
terminant of matrix
Σ
, c is a scalar, and E the size of the matrix.
Statistical analysis
e benet of the proposed BCI design was evaluated by
means of a Kruskal-Wallis test, using the technique methods
as an independent factor and the bit-rate as the dependent
variable. Using the best-performing of each method, optimal
bit-rate was identied (at 70% real accuracy) and compared to
that of the scanning system on the same patients using a paired
Student t-test. For the repeated-measures framework above, in
this manuscript we will refer to group factor, time factor, and
time-group interaction factor and related eects.
Results
Added value of signal processing and classication algo-
rithms
Figure 6 shows the inuence of several factors with respect
to real spelling accuracy: the presence of an artifact-remov-
al technique (the SQI) and the use of a traditional algorithm
(xDAWN+SWLDA) versus a Riemannian algorithm (MDM).
It also compares the performance of healthy volunteers and
(2)
Σnorm =Σ∗det(Σ)−1∕E
214 L. MAYAUD ET AL.
quired under the strict supervision of an EEG technician, thjey
were collected in an environment prone to artifacts. We have
made the data de-identied, open, and accessible to peers in
the digital supplement of this article so that the community
of researchers can investigate this and other questions further.
No dierence in performance was found between the Ro-
BIK and the TMSi headset, even though lower variability can
e use of the SQI to reject trials contaminated by artifacts
did not show clear benet in terms of bit-rate. However, Figure
5 (column 1) shows that monitoring artifacts and stopping the
operation in the presence of high noise reduce the variabili-
ty of the system, which is an important benet for real-word
operation. It should be noted that our study has been carried
out in a realistic environment. Even though the data were ac-
Table 5.Bit-rates (bits per minutes) for the clinical sample included in the clinical study of Study 2 for scanning device (session 1 only),
the RoBIK system during the online session, and the performance computed offline using the same data and algorithms but with the use
of an optimal number of repetitions (personalized). Patients 5, 8, and 9, for whom no BCI signal could be exploited, were removed from
this analysis and subsequent statistical testing.
P01 P02 P03 P04 P05 P06 P07 P08 P09 P10
Scanning 1 24.5 22.6 30.8 20.3 18.9 33.9 20.7 NA 25.1 14.5
Online RoBIK 3.1 1.7 1.7 0.2 0.0 1.4 0.1 NA 0.0 0.1
Personalized RoBIK 27.8 27.8 23.0 27.8 0.0 1.7 2.9 NA 0.0 31.5
Figure 6.Evolution of ‘real accuracy’ (% of correctly identified characters) in relation to number of flashes of each symbol under different
conditions; the three rows of plots indicate the performance in (1) cross-validated training data, (2) test-set (online) data, and (3) cross-
subject performance. The first column of plots shows the effect of artifact removal using the SQI on the data from Study 1 only. The
second column shows the impact of the classifier comparing xDAWN + stepwise linear discriminant analysis (SWLDA) and minimum
distance to mean (MDM) Riemannian classifier. The third column compares the performance of healthy volunteers and patients. The
fourth column compares the data collected with a traditional EEG system (Study 1) to that collected with the RoBIK headset (Study 2).
For all factors considered, data are collapsed across the other factors.
BRAINCOMPUTER INTERFACES 215
had numerous drawbacks (design, weight, corrosion with
saline water, mechanical electrical contact to name a few) it
shows that cheap and easy-to-use EEG systems for bedside ap-
plications are becoming a reality. e trending developments
in dry electrode systems [18,20,71] may be expected to lower
the tolerance threshold further and broaden the use of BCIs.
be observed for the TMSi headset. is supports the well-
known fact that a gel-based cap collects EEG data more con-
sistently. Nonetheless, the RoBIK headset has lower technical
specications (no true DC, 12 bits instead of 24), is therefore
cheaper, and allows a dramatic reduction in setup time (ap-
proximately three-fold). While the headset prototype tested
