Content uploaded by Roderick Murray-Smith
Author content
All content in this area was uploaded by Roderick Murray-Smith on Jun 05, 2015
Content may be subject to copyright.
THE BERLIN BRAIN-COMPUTER INTERFACE PRESENTS THE NOVEL
MENTAL TYPEWRITER HEX-O-SPELL
Benjamin Blankertz
1
, Guido Dornhege
1
, Matthias Krauledat
1,2
, Michael Schr¨oder
1
,
John Williamson
3
, Roderick Murray-Smith
3,4
, Klaus-Robert M¨uller
1,2
1
Fraunhofer FIRST (IDA), Berlin, Germany
2
Technical University Berlin, Berlin, Germany
3
University of Glasgow, Glasgow, Scotland
4
Hamilton Institute, NUI Maynooth, Ireland
E-mail: benjamin.blankertz@first.fhg.de
SUMMARY: We present a novel typewriter appli-
cation ‘Hex-o-Spell’ that is specifically tailored to
the characteristics of direct brain-to-computer inter-
action. The high bandwidth at which a user may
perceive information from the display is used in an
appealing visualization base d on hexagons. On the
other hand the control of the application is possible
at low bandwidth using only two control commands
(mental states) and is relatively stable against de-
lays and the like. The effectiveness and robustness
of the interface was demonstrated at the CeBIT 2006
(world’s largest IT fair) where two subjects oper-
ated the mental typewriter at a speed of up to 7.6
char/min. It was developed within the Berlin Brain-
Computer Interface project in cooperation with spe-
cialists for Human Computer Interaction.
INTRODUCTION
Brain-Computer Interfaces (BCIs) translate the in-
tent of a subject measured from brain signals directly
into control commands, e.g. for a computer applica-
tion or a neuroprosthesis ([3]). Although the proof-
of-concept of BCI systems was given decades ago,
several major challenges are still to be faced. One of
those challenges is to develop BCI applications which
take the specific characteristics of BCI communica-
tion into account. Apart from being prone to error
and having a rather uncontrolled variability in tim-
ing, its bandwidth is heavily unbalanced: BCI users
can perceive a high rate of information transfer from
the display, but have a low-bandwidth communica-
tion in their control actions.
The Berlin Brain-Computer Interface (BBCI) is
an EEG-based BCI system which operates on the
spatio-spectral changes during different kinds of mo-
tor imagery. It uses machine learning techniques to
adapt to the specific brain signatures of each user,
thereby achieving high quality feedback already in
the first se ssion ([1]). The mental typewriter pre-
sented here incorporates state-of-the-art knowledge
from Human Computer Interaction (HCI) and report
results of a public performance with two subjects.
METHODOLOGY
The challenge in designing a mental typewriter is to
map a small numbe r of BCI control states (typically
two) to the high number of symbols (26 letters plus
punctuation marks) while accounting for the low sig-
nal to noise ratio in the control signal. The more fluid
interaction in the BBCI system was made possible by
introducing an approach which combined probabilis-
tic data and dynamic systems theory based on our
earlier work ([2]) on mobile interfaces.
Here we take the example that the typewriter is con-
trolled by the two mental states imagined right hand
movement and imagined right foot movement. The
initial configuration is shown in the leftmost plot of
Fig. 1. Six hexagonal fields are surrounding a cir-
cle. In each of them five letters or other symbols
(including ‘<’ for backspace) are arranged. For the
selection of a symbol there is an arrow in the center of
the circle. By imagining a right hand movement the
arrow turns clockwise. An imagined foot movement
stops the rotation and the arrow starts extending.
If this imagination is performed in a longer period
the arrow touches the hexagon and thereby selects
it. Then all other hexagons are cleared and the five
symbols of the selected hexagon are moved to indi-
vidual hexagons as shown in Fig. 1. The arrow is
reset to its minimal length. Now the same proce-
dure (rotation if desired and extension of the arrow)
is repeated to select one symbol.
A language model determines the order of the sym-
bols in one hexagon depending on the context, but
this and many more important details go beyond the
scope of this note.
RESULTS
On two days in the course of the CeBIT fair 2006 in
Hannover, Germany, live demonstrations were given
with two subjects simultaneously using the BBCI
system. These demonstrations turned out to be
BBCI robustness tests par excellence. All over the
1
Figure 1: The mental typewriter ’Hex-o-Spell’. The two states classified by the BBCI system control the turning
and growing of the gray arrow respectively (see also text). Letters can thus be chosen in a two step procedure.
fair pavilion, noise sources of different kinds (electric,
acoustic,...) were potentially jeopardizing the perfor-
mance. A low air humidity made the EEG electrode
gel dry out and last but not least the subjects were
under psychological pressure to perform well for in-
stance in front of several running TV cameras or in
the presence of the German minister of research. The
preparation of the experiments started at 9:15 a.m.
and the live performance at 11 a.m. The two subjects
were either playing ‘Brain-Pong’ against each other
or writing sentences with the typewriter Hex-o-Spell.
Except for short breaks and a longer lunch break, the
subjects continued until 5 p.m. without degradation
of performance over time which is a demonstration
of great stability. The typing speed was between 2.3
and 5 char/min for one subject and between 4.6 and
7.6 char/min for the other subject. This speed was
measured for error-free, completed sentences, i.e. all
typing errors that have been committed had to be
corrected by using the backspace of the mental type-
writer.
For a B CI driven typewriter not operating on evoked
potentials this is a world class spelling speed, espe-
cially taking into account the environment and the
fact that the subjects did not train the usage of the
BBCI typewriter interface: the subjects used the
typewriter application only twice before.
DISCUSSION
The prospective value of BCI research for rehabil-
itation is well known. In light of the work pre-
sented here we would advocate a further point. BCI
provides stimulation to HCI researchers as an ex-
treme example of the sort of interaction which is
becoming more common: interaction with ‘uncon-
ventional’ computers in mobile phones, or with de-
vices embedded in the environment. These have a
number of shared attributes: high-dimensional, noisy
inputs, which describe intrinsically low-dimensional
content; data with content at multiple time-scales;
and a significant uncontrolled variability. The mis-
match in the bandwidth between the display and
control channels (as explained in the introduction)
and the slow, frustrating error correction motivate
a more ‘negotiated’ style of interaction, where com-
mitments are withheld until appropriate levels of ev-
idence have been accumulated (i.e. the entropy of the
beliefs inferred from the behavior of the joint human-
computer system should change smoothly, limited by
the maximum input bandwidth). The dynamics of a
cursor, given such noisy inputs, should be stabilized
by controllers which infer potential actions, as well
as the structure of the variability in the sensed data.
Hex-o-Spell demonstrates the potential of such in-
telligent stabilising dynamics in a noisy, but richly-
sensed medium. The results suggest that the ap-
proach is a fruitful one, and one which leaving open
the potential for incorporating sophisticated models
without ad hoc modifications.
ACKNOWLEDGEMENTS
This work was supported in part by a grant
of the BMBF (FKZ 01IBE01A), by the SFI
(00/PI.1/C067), and by the IST Programme of the
EU under the PASCAL NoE (IST-2002-506778).
REFERENCES
[1] Blankertz B, Dornhege G, Krauledat M, M¨uller K-
R, Kunzmann V, Losch F, Curio G, The Berlin Brain-
Computer Interface: EEG-based communication without
subject training, IEEE Trans. Neural Sys. Rehab. Eng.
(2006), accepted.
[2] Williamson J, Murray-Smith R, Dynamics and proba-
bilistic text entry, Proc. of the Hamilton Summer School
on Switching and Learning in Feedback systems (Murray-
Smith R and Shorten R, eds.), LNCS vol. 3355, 2005,
pp. 333–342.
[3] Wolpaw JR, Birbaumer N, McFarland DJ, Pfurt-
scheller G, Vaughan TM, Brain-computer interfaces for
communication and control, Clin. Neurophysiol. 113
(2002), 767–791.
2
