Conference PaperPDF Available

EEGlass: An EEG-Eyeware Prototype for Ubiquitous Brain-Computer Interaction

Authors:

Abstract and Figures

Contemporary Head-Mounted Displays (HMDs) are progressively becoming socially acceptable by approaching the size and design of normal eyewear. Apart from the exciting interaction design prospects, HMDs bear significant potential in hosting an array of physiological sensors very adjacent to the human skull. As a proof of concept, we illustrate EEGlass, an early wearable prototype comprised of plastic eyewear frames for approximating the form factor of a modern HMD. EEGlass is equipped with an Open-BCI board and a set of EEG electrodes at the contact points with the skull for unobtrusively collecting data related to the activity of the human brain. We tested our prototype with 1 participant performing cognitive and sensorimotor tasks while wearing an established Electroencephalography (EEG) device for obtaining a baseline. Our preliminary results showcase that EEGlass is capable of accurately capturing resting state, detect motor-action and Electrooculographic (EOG) artifacts. Further experimentation is required, but our early trials with EEGlass are promising in that HMDs could serve as a springboard for moving EEG outside of the lab and in our everyday life, facilitating the design of neuroadaptive systems.
Content may be subject to copyright.
EEGlass: An EEG-Eyeware Prototype for Ubiquitous
Brain-Computer Interaction
Athanasios Vourvopoulos
Department of Bioengineering
Institute for Systems and Robotics
Instituto Superior Técnico
Universidade de Lisboa
Lisboa, Portugal
athanasios.vourvopoulos@tecnico.ulisboa.pt
Evangelos Niforatos
Department of Computer Science
Norwegian University of Science and
Technology (NTNU)
Trondheim, Norway
evangelos.niforatos@ntnu.no
Michail Giannakos
Department of Computer Science
Norwegian University of Science and
Technology (NTNU)
Trondheim, Norway
michail.giannakos@ntnu.no
ABSTRACT
Contemporary Head-Mounted Displays (HMDs) are progressively
becoming socially acceptable by approaching the size and design
of normal eyewear. Apart from the exciting interaction design
prospects, HMDs bear signicant potential in hosting an array
of physiological sensors very adjacent to the human skull. As a
proof of concept, we illustrate EEGlass, an early wearable proto-
type comprised of plastic eyewear frames for approximating the
form factor of a modern HMD. EEGlass is equipped with an Open-
BCI board and a set of EEG electrodes at the contact points with
the skull for unobtrusively collecting data related to the activity
of the human brain. We tested our prototype with 1 participant
performing cognitive and sensorimotor tasks while wearing an
established Electroencephalography (EEG) device for obtaining a
baseline. Our preliminary results showcase that EEGlass is capa-
ble of accurately capturing resting state, detect motor-action and
Electrooculographic (EOG) artifacts. Further experimentation is
required, but our early trials with EEGlass are promising in that
HMDs could serve as a springboard for moving EEG outside of the
lab and in our everyday life, facilitating the design of neuroadaptive
systems.
CCS CONCEPTS
Hardware Neural systems.
KEYWORDS
Head-Mounted Displays, Electroencephalography, Brain-Computer
Interfaces, Neuroadaptive Systems
ACM Reference Format:
Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos.
2019. EEGlass: An EEG-Eyeware Prototype for Ubiquitous Brain-Computer
Interaction. In Adjunct Proceedings of the 2019 ACM International Joint
Conference on Pervasive and Ubiquitous Computing and the 2019 International
Symposium on Wearable Computers (UbiComp/ISWC ’19 Adjunct), September
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom
©2019 Association for Computing Machinery.
ACM ISBN 978-1-4503-6869-8/19/09. . . $15.00
https://doi.org/10.1145/3341162.3348383
9–13, 2019, London, United Kingdom. ACM, New York, NY, USA, 6 pages.
https://doi.org/10.1145/3341162.3348383
1 STATE OF THE ART
In the year 2022, over 80 million Head-Mounted Display (HMD)
units are expected to ship, up from over 34 million units estimated
to be sold in 2019
1
. This predicted HMD proliferation is propelled by
recent advances in optics and hardware miniaturization, rendering
eyewear the next frontier for Wearable and Ubiquitous Comput-
ing (e.g., Google Glass Enterprise Edition
2
, Microsoft HoloLens 2
3
,
MagicLeap One
4
, Focals by North
5
and Vuzix Blade
6
). Undoubtedly,
an HMD uptake will actualize the vision of Augmented Reality
(AR), disrupting the way we consume information and execute our
daily chores [
3
]. From revisiting the groceries list to navigating
to our destination, a plethora of daily tasks will be reshaped by
AR and the unique form factor of HMDs. Except for the immense
opportunities in interaction design, the HMD form factor promises
an unprecedented contact potential with the human skull: the shell
of our brain where the higher cognitive and perceptual processes
reside.
Evidently, this “skull-contact” potential that eyewear bears could
not go unnoticed. Various commercial eyewear products claim to
utilize EEG (or EOG – electrooculography) and the contact points
with the skull to provide so-called “neurofeedback”. For example,
Narbis sunglasses
7
utilize three electrodes, two at the back of the
temples of the device touching the left and right mastoids, and
one at the tip of a protruding arm that touches the top of the
skull, for tracking concentration. When concentration is low, the
Narbis electrochromatic lenses start darkening for inviting the
user to focus more. Lowdown Focus by Smith
8
employs a more
socially acceptable design equipping a typical pair of sunglasses
with silicon electrodes at the edges of the temples that touch the
left and right mastoids. A companion mobile application connects
to the Lowdown Focus sunglasses for collecting the readings so
that one can track and increase one’s concentration levels. JINS
1
https://www.gartner.com/en/newsroom/press-releases/2018-11-29- gartner-says-
worldwide-wearable- device-sales-to- grow-
2https://www.google.com/glass/start/
3https://www.microsoft.com/en-us/hololens
4https://www.magicleap.com/magic-leap-one
5https://www.bynorth.com
6https://www.vuzix.com/products/blade-smart-glasses
7http://narbis.com/company/
8https://www.smithoptics.com/us/lowdownfocus
UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos
Figure 1: The EEGlass prototype worn by a user.
MEME
9
is perhaps the most prominent eyewear that employs near-
skull contact for providing neurofeedback. JINS MEME utilizes
two electrodes embedded in the bridge of the eyewear that touch
the nasal bone to detect concentration levels by measuring the
duration and the number of eye blinks via EOG [
20
]. More recently,
a US patent was published about the design of an eyewear frame
that features a exible protrusion for holding the EEG electrodes
in contact with the skull in dierent positions [
5
]. Beyond the
strong commercialization interest, the combination of eyewear
with EEG yields interesting niche applications in the domain of
health research and Human-Computer Interaction (HCI). e-Glass
is an EEG-enabled eyewear that employs an OpenBCI board and
a set of electrodes across the inner side of the frame for detecting
epileptic seizures [
19
]. PhysioHMD prototype adopts a bulky “mask”
design that encases a portion of the face for hosting a wide range
of physiological sensors, including EEG electrodes, capturing also
facial expressions [
2
]. PhysioHMD is intended as a platform that
informs the design of AR and Virtual Reality (VR) experiences.
On one hand, commercial approaches that combine modern eye-
wear with EEG (and EOG) are in general socially acceptable, but also
rather limited in providing high-level information about brain activ-
ity (e.g., daily concentration levels), typically via a dedicated mobile
application. Moreover, commercial “neurofeedback” products are
considered “black-box” systems that utilize proprietary hardware
and software. On the other hand, experimental EEG-eyewear proto-
types produce ne-grained information about brain activity, while
utilizing open hardware and software solutions. However, such
prototypes are by default too dorky to wear outside a research lab,
and cumbersome to use for extended periods of time. The EEGlass
prototype attempts to fuse the social acceptability and increased
9https://jins-meme.com/en/
Last accessed on July 3, 2019.
8
Figure 2: The 10-10 system of electrode placement topology
for the EEGlass and Enobio 8.
“wearability” of commercial EEG-eyewear with the high informa-
tion granularity and openness of experimental EEG-eyewear (see
Figure 1). The outcomes of such a successful fusion remain tenta-
tive but can potentially nurture existing and envisioned cognitive
systems [
13
], democratize EEG, and eventually pave the way for
touch-less input.
2 BACKGROUND
The human brain is an electrochemical system that generates a com-
bination of dynamic biosignals or action potentials. EEG is the most
common brain signal acquisition technique established almost a
century ago [
1
]. Non-invasive EEG utilizes scalp-contact electrodes
for capturing the combined electrical activity of populations of
excitable cells known as neurons. When neurons activate, they
produce electromagnetic elds of discrete potential patterns, dis-
tinguished by dierent wave oscillations in the frequency domain.
These patterns are ascribed to dierent states of mental activity
and are identied by the wave oscillations they cause in the fre-
quency domain, known as EEG bands or rhythms [
4
]. EEG rhythms
are divided into dierent frequency ranges including Delta (1–4
Hz), Theta (4–8 Hz), Alpha (8–13 Hz), Beta (13–30 Hz) and Gamma
(25–90 Hz) [
11
], and each rhythm or combination of rhythmic ac-
tivity is linked to dierent mental states. For example, rhythms
in the Alpha and Beta frequency bands are functionally related
to major sensorimotor systems, which activate primarily through
motor preparation or execution [
6
]. Alpha and Theta oscillations
are known to reect cognitive and memory performance [
9
], and
Theta was shown by early EEG studies to be closely connected to
problem-solving, perceptual processing and learning [
18
]. Delta
rhythm is related to concentration, attention and internal process-
ing [
8
], whereas Gamma has been linked to consciousness and sense
of self, and can be volitionally modulated during meditation [
10
].
EEGlass: An EEG-Eyeware Prototype for Ubiquitous Brain-Computer Interaction UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom
Figure 3: The EEGlass electrode contact points and the re-
spective OpenBCI channels.
Interpreting cognitive states or motor intentions from dierent
EEG rhythms is a complex process and is impossible to associate a
single frequency range, or cortical location, to a brain function.
Nowadays, measuring oscillatory brain activity with EEG is
utilized for linking the human brain with computers via Brain-
Computer Interfaces (BCIs). BCIs have been successfully utilized in
the medical domain where BCIs enable amputees to gain control
over prosthetic limbs [
15
]. Other BCI application areas include mon-
itoring user’s cognitive states (e.g., attention levels and workload),
gaming and rehabilitation. Thus, BCIs and wearable technologies
such as HMDs, oer a unique opportunity to measure user needs
over time and in real-life settings, informing how critical software
aspects (e.g., interface) should respond or adapt. The potential of
EEGlass in monitoring surreptitiously and in real-time user’s phys-
iological and cognitive states, renders it an ideal BCI for facilitating
interaction with neuroadaptive systems.
3 EEGLASS PROTOTYPE
The EEGlass prototype is comprised of plastic eyewear frames that
can be tted with a Google Glass HMD. We opted for this type of
eyewear for: (1) low cost, (2) availability, (3) good tting and (4)
modern HMD resemblance. In fact, the selected eyewear frames
follow the trend in the eld of HMDs: hardware miniaturization
and social acceptability [
12
]. The EEG system that we embedded
in the frames is Cyton Biosensing Board by OpenBCI (OpenBCI,
NY, USA). OpenBCI is a popular and aordable open hardware
and software platform for the collection and analysis of biosignals
such as EEG, EMG (Electromyography), ECG (Electrocardiography)
and others, inspired by the grassroots movement of DIY (“Do It
Yourself”) [
21
]. The Cyton board encompasses 8 biopotential input
channels (for hosting up to 8 electrodes), a 3-axes accelerometer,
local storage, wireless communication modules, while being fully
programmable and Arduino compatible. Evidently, the EEGlass
electrode topology is restricted by the eyewear form factor and at
the contact points with the skull. Thus, EEGlass utilizes 3 electrodes
(plus 2 for reference and ground) based on the 10-10 system (see
Figure 2) for measuring brain activity: 1 electrode placed inwards
at the top of the eyewear bridge touching the skull at glabella,
and 2 more electrodes at the inner side of the eyewear temples,
touching the left and right mastoids, behind the left and right ears,
Figure 4: A user performing a motor task while wearing both
EEGlass and Enobio 8 EEG systems.
respectively (see Figure 3). The reference and ground electrodes
are placed at the inner part of the eyewear bridge, touching the left
and right sides of the nasal bone, respectively (see Figure 3).
4 FIRST TRIALS AND EARLY RESULTS
In this work, we explore if the low spatial resolution of a “skull-
peripheral” electrode topology, imposed by the form factor of a
modern HMD, can approximate the accuracy of a typical electrode
topology utilized in EEG studies. To this end, we used Enobio 8
(Neuroelectrics, Barcelona, Spain) EEG system for forming a base-
line. Enobio 8 is a wireless, 8-channel, EEG system with a 3-axes
accelerometer for the recording and visualization of 24-bit EEG
data at 500 Hz. The spatial distribution of electrodes for our trials
followed the 10-10 system conguration, with electrodes placed
over the frontal area (Fpz), central (C3, C4), and parietal (Pz) (see
Figure 2). The electrodes were referenced and grounded to the right
ear lobe, and the electrode impedance was kept at
<
20
k
. Both the
Enobio 8 and the OpenBCI (embedded in the EEGlass) EEG systems
were connected via Bluetooth to a dedicated desktop computer for
raw signal acquisition and processing.
The EEG acquisition session began with a 4-minute period for
acquiring resting state data, and the motor-action session following
next. The resting state data was acquired during alternating 1-
minute periods with eyes open and eyes closed. The subject was
instructed to remain silent while either xating his eye-gaze on a
white cross displayed on a computer screen, or when having his
eyes closed. In the motor-action session, we employed the Graz-
BCI paradigm [
16
] to display a random sequence of directional left
and right arrows on a computer screen (see Figure 4). When an
arrow appeared, the user responded to the stimulus by performing
a motor-action with the corresponding hand. The motor-action
session was congured to acquire data in 24 blocks (epochs) per
class (left and right hand arrow).
UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos
(a) EEGlass resting state. (b) Enobio 8 resting state.
Figure 5: Resting states for EEGlass and Enobio 8. Both EEG systems detect a clear Alpha rhythm peak at 8-12 Hz.
For both systems, we used the OpenVibe acquisition servers for
simultaneous EEG signal acquisition [
17
]. Next, we used the Open-
Vibe designer for obtaining the raw EEG streams from both servers
and synchronizing them with the stimuli, before storing them in a
.gdf le. We processed the acquired EEG signals in MATLAB
®
(The
MathWorks, MA, USA) with the EEGLAB toolbox [
7
]: after import-
ing the data and the channel information, we applied a high-pass
lter at 1 Hz to remove the “baseline drift” followed by line-noise
and harmonics removal at 50 Hz. Then, we used Welch’s method
[
22
] for Power Spectral Density (PSD) of the power spectrum to
compute the average spectral power across the following frequency
bands during resting state: Delta (1–4 Hz), Theta (4–7 Hz), Alpha
(8–12 Hz), and Beta (12–30 Hz). The event-related synchroniza-
tion/desynchronization (ERS/ERD) was extracted following the
standard ERS/ERD method [
14
] across the Alpha band power (8–12
Hz) and the Beta band power (12–30 Hz) over C3 and C4 electrode
locations for the Enobio 8 system, and TP9, TP10 for the OpenBCI,
respectively. We calculated the ERD by using the following formula:
ERD =
(PowerM ot or Ac ti vit yPowerB ase l ine )
PowerBase li ne
100 (1)
Early results from comparing EEG signals acquired via EEGlass
with Enobio 8 indicate that EEGlass captures very closely the band
power of Enobio 8, but also the the decrease of oscillatory activity
Table 1: Resting state and EEG rhythms in µV2/Hz recorded
by EEGlass and Enobio 8 EEG systems.
System Delta Theta Alpha Beta
EEGlass 21.279 4.775 12.992 0.431
Enobio 8 4.984 2.196 13.397 0.2993
(ERD) during the motor-action, as we anticipated. Figure 5 shows
that despite the fundamentally dierent electrode topology of EE-
Glass, both EEGlass and Enobio detect a clear Alpha peak (8–12
Hz) in the electrical activity of the brain during resting state. Ta-
ble 1 summarizes the recorded EEG rhythms during resting state
for both EEGlass and Enobio 8 EEG systems. For investigating if
brain activity linked to motor-action diered substantially between
the two EEG systems, we compared the average ERD between lat-
eral electrodes for both systems:
(TP
9
,C
3
)
for right-hand, and
(CP
10
,C
4
)
for left-hand movement. Dependent samples t-tests
displayed signicant dierences in the average ERD between both
pairs of lateral electrodes (
tT P 9|C3(
199
)=
10
.
214
,p< .
001 and
tC P 10|C4(
199
),p< .
001), as shown in Figure 6 and summarized in
Table 2. This indicates that the captured brain activity related to
upper limb motor-action diered signicantly between the EEGlass
and Enobio 8 EEG systems. Moreover, Figure 7 showcases that
EEGlass was able to detect basic EOG activity related to eye move-
ment in 4 primary directions: up, down, left and right. Although
the current electrode setup can capture eye-movement with only 1
degree-of-freedom (DoF), it can also detect saccadic eye movement
and eye blinks.
Table 2: Average desynchronization (% ERD) between 8–24
Hz per electrode and hand for both EEGlass and Enobio 8
EEG systems.
EEGlass Enobio 8
Electrode TP9 TP10 C3 C4
Movement Right Left Right Left
Mean -5.058 -3.065 -20.272 -18.899
SD 11.6383 12.273 19.048 22.47
EEGlass: An EEG-Eyeware Prototype for Ubiquitous Brain-Computer Interaction UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom
ERD (%)
(a) Left hand ERD as captured by CP10 (EEGlass) and C4 (Enobio 8) electrodes.
ERD (%)
(b) Right hand ERD as captured by TP9 (EEGlass) and C3 (Enobio 8) electrodes.
Figure 6: Average motor-action (ERD) for lateral electrode pairs of both EEGlass and Enobio 8 EEG Systems. Signicant dier-
ences (p< .001) were found between both electrode pairs for both right and left hand motor-action.
5 DISCUSSION AND FUTURE WORK
Our preliminary results serve as a proof of concept for piggybacking
EEG on eyewear and HMDs, yet in a cost-eective, unobtrusive
and socially acceptable fashion. Trials with 1 participant indicate
that the EEGlass is capable of capturing brain activity manifested
in two modes of resting state: (a) eyes open and focused on a target,
and (b) eyes closed. In fact, brain activity recorded during resting
state with EEGlass demonstrates similar variations in frequency
and amplitude to when recorded with an established EEG system
such as Enobio 8. However, recorded brain activity linked to upper
limb motor-action and captured with EEGlass, displayed signicant
dierences when compared to that captured with Enobio 8, as
anticipated due to the localized motor activity of the sensorimotor
cortices under C3, C4 electrodes. Yet, EEGlass managed to capture
ERD from its TP9 and TP10 electrodes, though with less power than
Enobio 8 due to the low spatial resolution of the EEG signals on the
scalp, relying on signal propagation to reach the remotely located
EEGlass electrodes. Moreover, EEGlass was able to detect subtle
eye movements in 4 basic directions, displaying an eye-tracking
potential particularly useful for navigating in heads-up interfaces.
Undoubtedly, low sample size (N=1) and stationary experimental
settings are signicant limitations that we will address in followup
studies. However, human skull and brain anatomy is universally
homogeneous, and the eyewear/HMD form factor ensures a rather
stable electrode contact, only somewhat inuenced by movement.
In future iterations, we will utilize prominent machine learning
techniques for training algorithms to match input from EEGlass
to that of established EEG systems, and test our prototype in user
studies with actual HMDs. We believe a merger between EEG and
HMDs bears an unprecedented potential to revolutionize human-
machine interaction, facilitating touch-less input and greatly in-
creasing human-machine communication throughput.
6 ACKNOWLEDGEMENTS
The authors acknowledge the nancial support of the Swiss Na-
tional Science Foundation (SNSF) under grant number: 184146
We would also like to thank Ph.D. student Octavio Marin Pardo for
helping with the data acquisition.
REFERENCES
[1]
Hans Berger. 1933. Über das elektrenkephalogramm des menschen. European
archives of psychiatry and clinical neuroscience 98, 1 (1933), 231–254.
[2]
Guillermo Bernal, Tao Yang, Abhinandan Jain, and Pattie Maes. 2018. PhysioHMD:
a conformable, modular toolkit for collecting physiological data from head-
mounted displays. In Proceedings of the 2018 ACM International Symposium on
Wearable Computers. ACM, 160–167.
[3]
Mark Billinghurst and Thad Starner. 1999. Wearable devices:new ways to manage
information. Computer 32, 1 (1999), 57–64.
[4]
György Buzsáki and Andreas Draguhn. 2004. Neuronal oscillations in cortical
networks. science 304, 5679 (2004), 1926–1929.
[5]
Robert A Connor. 2018. EEG glasses (electroencephalographic eyewear). US
Patent 9,968,297.
[6]
Nathan E Crone, Diana L Miglioretti, Barry Gordon, and Ronald P Lesser. 1998.
Functional mapping of human sensorimotor cortex with electrocorticographic
spectral analysis. II. Event-related synchronization in the gamma band. Brain: a
journal of neurology 121, 12 (1998), 2301–2315.
[7]
Arnaud Delorme and Scott Makeig. 2004. EEGLAB: an open source toolbox for
analysis of single-trial EEG dynamics including independent component analysis.
Journal of neuroscience methods 134, 1 (2004), 9–21.
[8]
Thalía Harmony, Thalía Fernández, Juan Silva, Jorge Bernal, Lourdes Díaz-Comas,
Alfonso Reyes, Erzsébet Marosi, Mario Rodríguez, and Miguel Rodríguez. 1996.
EEG delta activity: an indicator of attention to internal processing during perfor-
mance of mental tasks. International journal of psychophysiology 24, 1-2 (1996),
161–171.
[9]
Wolfgang Klimesch. 1999. EEG alpha and theta oscillations reect cognitive and
memory performance: a review and analysis. Brain research reviews 29, 2-3 (1999),
169–195.
[10]
Dietrich Lehmann, PL Faber, Peter Achermann, Daniel Jeanmonod, Lorena RR Gi-
anotti, and Diego Pizzagalli. 2001. Brain sources of EEG gamma frequency during
volitionally meditation-induced, altered states of consciousness, and experience
of the self. Psychiatry Research: Neuroimaging 108, 2 (2001), 111–121.
[11]
Ernst Niedermeyer and FH Lopes da Silva. 2005. Electroencephalography: basic
principles, clinical applications, and related elds. Lippincott Williams & Wilkins.
[12]
Evangelos Niforatos and Mélodie Vidal. 2019. Eects of a Monocular Laser-
Based Head-Mounted Display on Human Night Vision. In Proceedings of the 10th
Augmented Human International Conference 2019. ACM, 31.
UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos
Nz
Figure 7: Electrooculographic (EOG) activity (µV/ms) recorded by EEGlass and linked to eye movement in 4 basic directions.
[13]
Evangelos Niforatos, Athanasios Vourvopoulos, and Marc Langheinrich. 2017.
Amplifying human cognition: bridging the cognitive gap between human and
machine. In Proceedings of the 2017 ACM International Joint Conference on Per-
vasive and Ubiquitous Computing and Proceedings of the 2017 ACM International
Symposium on Wearable Computers. ACM, 673–680.
[14]
Gert Pfurtscheller and A Aranibar. 1979. Evaluation of event-related desyn-
chronization (ERD) preceding and following voluntary self-paced movement.
Electroencephalography and clinical neurophysiology 46, 2 (1979), 138–146.
[15]
Gert Pfurtscheller and Christa Neuper. 2001. Motor imagery and direct brain-
computer communication. Proc. IEEE 89, 7 (2001), 1123–1134.
[16]
Gert Pfurtscheller, Christa Neuper, GR Muller, Bernhard Obermaier, Gunter
Krausz, A Schlogl, Reinhold Scherer, Bernhard Graimann, Claudia Keinrath,
Dimitris Skliris, et al
.
2003. Graz-BCI: state of the art and clinical applications.
IEEE Transactions on neural systems and rehabilitation engineering 11, 2 (2003),
1–4.
[17]
Yann Renard, Fabien Lotte, Guillaume Gibert, Marco Congedo, Emmanuel Maby,
Vincent Delannoy, Olivier Bertrand, and Anatole Lécuyer. 2010. Openvibe: An
open-source software platform to design, test, and use brain–computer interfaces
in real and virtual environments. Presence: teleoperators and virtual environments
19, 1 (2010), 35–53.
[18]
Daniel L Schacter. 1977. EEG theta waves and psychological phenomena: A
review and analysis. Biological psychology 5, 1 (1977), 47–82.
[19]
Dionisije Sopic, Amir Aminifar, and David Atienza. 2018. e-glass: A wearable
system for real-time detection of epileptic seizures. In 2018 IEEE International
Symposium on Circuits and Systems (ISCAS). IEEE, 1–5.
[20]
Yuji Uema and Kazutaka Inoue. 2017. JINS MEME algorithm for estimation and
tracking of concentration of users. In Proceedings of the 2017 ACM International
Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the
2017 ACM International Symposium on Wearable Computers. ACM, 297–300.
[21]
Athanasios Vourvopoulos and Sergi Bermudez i Badia. 2016. Usability and Cost-
eectiveness in Brain-Computer Interaction: Is it User Throughput or Technology
Related?. In Proceedings of the 7th Augmented Human International Conference
2016. ACM, 19.
[22]
Peter Welch. 1967. The use of fast Fourier transform for the estimation of power
spectra: a method based on time averaging over short, modied periodograms.
IEEE Transactions on audio and electroacoustics 15, 2 (1967), 70–73.
... Finally, the Papers field provides the list of reviewed works employing a specific device. A total of 13 of 84 papers [86,123,128,134,138,143,154,[158][159][160][161][162][163] do not appear in this column, having that the authors design and use custom EEG headsets. Finally, Khan et al. [135] and Vourvopoulos et al. [162] compare their custom devices with Emotiv EPOC+ and Enobio 8, respectively. ...
... A total of 13 of 84 papers [86,123,128,134,138,143,154,[158][159][160][161][162][163] do not appear in this column, having that the authors design and use custom EEG headsets. Finally, Khan et al. [135] and Vourvopoulos et al. [162] compare their custom devices with Emotiv EPOC+ and Enobio 8, respectively. ...
... A total of 14 of 84 works prefer reducing the number of electrodes provided by the device producers [88,92,104,106,110,126,142,162] and especially make a selection on the channel placed over the central cortical area. In particular, only the C{3,4} [139,147], Cz [92,143,148], and C{1,2} [102] electrodes are considered. ...
Article
Full-text available
In recent decades, the automatic recognition and interpretation of brain waves acquired by electroencephalographic (EEG) technologies have undergone remarkable growth, leading to a consequent rapid development of brain–computer interfaces (BCIs). EEG-based BCIs are non-invasive systems that allow communication between a human being and an external device interpreting brain activity directly. Thanks to the advances in neurotechnologies, and especially in the field of wearable devices, BCIs are now also employed outside medical and clinical applications. Within this context, this paper proposes a systematic review of EEG-based BCIs, focusing on one of the most promising paradigms based on motor imagery (MI) and limiting the analysis to applications that adopt wearable devices. This review aims to evaluate the maturity levels of these systems, both from the technological and computational points of view. The selection of papers has been performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), leading to 84 publications considered in the last ten years (from 2012 to 2022). Besides technological and computational aspects, this review also aims to systematically list experimental paradigms and available datasets in order to identify benchmarks and guidelines for the development of new applications and computational models.
... Finally, the Papers field provides the list of reviewed works employing a specific device. Thirteen on 84 papers [82,119,124,130,134,139,150,[154][155][156][157][158][159] do not appear in this column, having that the authors design and use custom EEG headsets. Finally, Khan et al. [131] and Vourvopoulos et al. [158] compare their custom devices with Emotiv EPOC+ and Enobio 8, respectively. ...
... Thirteen on 84 papers [82,119,124,130,134,139,150,[154][155][156][157][158][159] do not appear in this column, having that the authors design and use custom EEG headsets. Finally, Khan et al. [131] and Vourvopoulos et al. [158] compare their custom devices with Emotiv EPOC+ and Enobio 8, respectively. ...
... After having given a brief overview of the synthetic information regarding EEG related devices, it is possible to move to a deeper analysis of how these technologies have been effectively used by the authors according to the table present in the supplementary materials. Fourteen works on 84 prefer reducing the number of electrodes provided by the device producers [84,88,100,102,106,122,138,158] and especially make a selection on the channel placed over the central cortical area, in particular considering only the C{3,4} [135,143], Cz [88,139,144] and C{1,2} [98] electrodes. Moreover, some of the authors clearly specifies the sampling rate used for signal acquisition that goes from 125Hz [109], 128Hz [87,106,118,129,131,135], 250Hz [94,95,108,141,145], 256Hz [131], 500Hz [84,88,89,93,99,113,133,138,143,144,148,160,161] to 512Hz [120]. ...
Preprint
Full-text available
In the last decades, the automatic recognition and interpretation of brain waves acquired by electroencephalographic (EEG) technologies have undergone remarkable growth, leading to a consequent rapid development of Brain Computer Interfaces (BCIs). EEG-based BCIs are non-invasive systems that allow communication between a human being and an external device interpreting brain activity directly. Thanks to the advances in neurotechnologies, and especially in the field of wearable devices, BCIs are now also employed outside medical and clinical applications. Within this context, this paper proposes a systematic review of EEG-based BCIs, focusing on one of the most promising paradigms based on Motor Imagery (MI) and limiting the analysis to applications that adopt wearable devices. This review aims to evaluate the maturity levels of these systems, both from the technological and computational points of view. The selection of papers has been performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), leading to 84 publications considered in the last ten years (from 2012 to 2022). Besides technological and computational aspects, this review also aims at systematically list experimental paradigms and available datasets in order to identify benchmarks and guidelines for the development of new applications and computational models.
... Recent advances in microfabrication and print technologies enabled new ways to design micro-patterned electrodes (gold [1,30] and graphene [11]). Due to the development of these technologies and 3D printing, designing wearable platforms has become possible, such as eyeglass types [5,[31][32][33][34][35][36][37][38][39], face mask types [7,[40][41][42], ear plug types [43][44][45][46][47], and headband types [48][49][50][51][52][53][54][55][56][57][58] for various applications. Previously, various controllers for an HMI such as wheelchairs [1,4,51,52], drones [11,59], game interfaces [5,36,47,60,61], and virtual keyboards [34,38,51,62] were created by using only an EOG signal. ...
... For example, wet Ag/AgCl electrodes are used for the analysis of various activity recognition fields [65,81,82] or HMI controllers [37,60,62,83]. Dry flat Ag/AgCl electrodes are used on various wearable platforms such as eyeglasses [35], head caps [55], and goggles [34]. From the perspective of wet Ag/AgCl electrodes (Figure 2c), conductive gel dehydration results in electrode performance degradation over time. ...
Article
Full-text available
Eye movements show primary responses that reflect humans' voluntary intention and conscious selection. Because visual perception is one of the fundamental sensory interactions in the brain, eye movements contain critical information regarding physical/psychological health, perception, intention, and preference. With the advancement of wearable device technologies, the performance of monitoring eye tracking has been significantly improved. It also has led to myriad applications for assisting and augmenting human activities. Among them, electrooculograms, measured by skin-mounted electrodes, have been widely used to track eye motions accurately. In addition, eye trackers that detect reflected optical signals offer alternative ways without using wearable sensors. This paper outlines a systematic summary of the latest research on various materials, sensors, and integrated systems for monitoring eye movements and enabling human-machine interfaces. Specifically, we summarize recent developments in soft materials, biocompatible materials, manufacturing methods, sensor functions, systems' performances, and their applications in eye tracking. Finally, we discuss the remaining challenges and suggest research directions for future studies.
... The traditional way of acquiring signals has been through gel-electrodes that are placed on the body. In addition to the use of traditional wearables such as smartwatches and fitness trackers, recent advances in fabrication and electronics have led to the integration of bio-sensing electrodes in other devices such as eye-glasses [101], VR headmounted displays [102], and textiles [97]. ...
Article
Full-text available
Data generated from sources such as wearable sensors, medical imaging, personal health records, pathology records, and public health organizations have resulted in a massive information increase in the medical sciences over the last decade. Advances in computational hardware, such as cloud computing, Graphical Processing Units (GPUs), and Tensor Processing Units (TPUs), provide the means to utilize these data. Consequently, many Artificial Intelligence (AI)-based methods have been developed to infer from large healthcare data. Here, we present an overview of recent progress in artificial intelligence and biosensors in medical and life sciences. We discuss the role of machine learning in medical imaging, precision medicine, and biosensors for the Internet of Things (IoT). We review the most recent advancements in wearable biosensing technologies that use AI to assist in monitoring bodily electro-physiological and electro-chemical signals and disease diagnosis, demonstrating the trend towards personalized medicine with highly effective, inexpensive, and precise point-of-care treatment. Furthermore, an overview of the advances in computing technologies, such as accelerated artificial intelligence, edge computing, and federated learning for medical data, are also documented. Finally, we investigate challenges in data-driven AI approaches, the potential issues that biosensors and IoT-based healthcare generate, and the distribution shifts that occur among different data modalities, concluding with an overview of future prospects.
... Vourvopoulos et al. [19] work on the EEGlass prototype for an interactive brain-computer interface; this prototype consists of wearable glasses with attached electrodes for recording brain activities. The essential signal filtration for the artifact removal and removal of high band-pass signals are applied so the only needed bandwidth signals can be used in the learning and testing process [20]. ...
... Digital interactive interfaces are widely used in driving environment (Schmidt et al., 2010;Kataria et al., 2015;Costa et al., 2017;Wang et al., 2021;He et al., 2022), industrial production environment (Gamecho et al., 2015;Diego-Mas et al., 2017), medical diagnosis environment (Erol Barkana and Açık, 2014;Açık et al., 2016;Meena et al., 2018), leisure and entertainment environment (Lu and Jia, 2014;Kanaan and Moacdieh, 2021), etc. Especially with the development of intelligent technology, many physical interactive interfaces have been gradually transformed into digital interactive interfaces (Shneiderman, 1997), which, together with touch screen (Jansen and Dragicevic, 2013), eye-movement control technology (Meena et al., 2018;Bozomitu et al., 2019), electroencephalographic (EEG) control technology (Vourvopoulos et al., 2019;Si-Mohammed et al., 2020), voice control technology (Hogan et al., 2017), motion capture technology (Kirsh, 2013), ...
Article
Full-text available
With the development of various intelligent technologies, the application of interactive interfaces is becoming more and more widespread, and the related researches conducted for interactive interfaces are also increasing. The purpose of this study was to explore the influence of icon layout location, icon graphic type, and icon layout method on users' searching performance in interactive interfaces through eye-tracking technology. Participants were asked to perform search tasks for the search target (facet icon or linear icon) on each image. Thus, each trial consisted of a search task on a given image. In total, each participant had 36 trials to complete. Searching time, fixation duration, and fixation count were collected to evaluate the searching performance of participants. Results showed that when faced with familiar icons, whether the graphic type of icons was facet or linear did not affect the user's experience, but when other factors of the interaction interface changed, facet icons provided a more stable experience for users. And compared to the rectangular layout, the circular layout method provided a more stable experience for users when the location of icons in the interactive interface changed, but icons located in the top half of the interactive interface were easier to find than those located in the bottom half, regardless of whether the layout was circular or rectangular. These results could be used in the layout and icon design of the interactive interfaces to facilitate their optimization.
Article
Full-text available
Recent advances in wearable technologies have enabled ways for people to interact with external devices, known as human-machine interfaces (HMIs). Among them, electrooculography (EOG), measured by wearable devices, is used for eye movement-enabled HMI. Most prior studies have utilized conventional gel electrodes for EOG recording. However, the gel is problematic due to skin irritation, while separate bulky electronics cause motion artifacts. Here, we introduce a low-profile, headband-type, soft wearable electronic system with embedded stretchable electrodes, and a flexible wireless circuit to detect EOG signals for persistent HMIs. The headband with dry electrodes is printed with flexible thermoplastic polyurethane. Nanomembrane electrodes are prepared by thin-film deposition and laser cutting techniques. A set of signal processing data from dry electrodes demonstrate successful real-time classification of eye motions, including blink, up, down, left, and right. Our study shows that the convolutional neural network performs exceptionally well compared to other machine learning methods, showing 98.3% accuracy with six classes: the highest performance till date in EOG classification with only four electrodes. Collectively, the real-time demonstration of continuous wireless control of a two-wheeled radio-controlled car captures the potential of the bioelectronic system and the algorithm for targeting various HMI and virtual reality applications.
Conference Paper
Full-text available
In the last few years, Brain-Computer Interfaces(BCI’s) have progressed as an emerging research area in the domain of neurotechnology. This is primarily due to the introduction of low-cost electroencephalographic (EEG) systems that render BCI technology accessible for non-medical research, but also due to the advancements of signal processing and machine learning methods. Consequently, BCI’s can now provide a wide new range of possibilities in the way users interact with a computer system (e.g., neuroadaptive interfaces). However, major challenges must still be addressed for BCI systems to mature into an established communication medium for effective brain–computer interaction. One of the major challenges involves the easy integration of real-time processing pipelines with portable EEG systems for out-of-the-lab use. To date, despite the amount of options current open-source tools provide, most toolboxes focus mainly in extending the processing and classification methods but lack on the ability to provide an easy-to-design yet extensible architecture for ubiquitous use. Here, we present NeuXus, a modular Python toolbox for real-time biosignal processing and pipeline design. NeuXus is open-source and platform independent, providing high-level implementation of processing pipelines for easy BCI design and deployment, including for simultaneous EEG-fMRI acquisitions through real-time gradient artifact correction.
Conference Paper
Full-text available
Head-mounted displays (HMDs) are expected to dominate the market of wearable electronics in the next 5 years. This foreseen proliferation of HMDs yields a plethora of design opportunities for revolutionizing everyday life via novel use cases, but also generates a considerable number of substantial safety implications. In this work, we systematically investigated the effect of a novel monocular laser-based HMD on the ability of our participants to see in low ambient light conditions in lab settings. We recruited a total of 19 participants in two studies and performed a series of established vision tests while using the newly available Focals by North HMD. We tested our participants' night vision after being exposed to different levels of laser luminous power and laser colors while using Focals, either with one or both eyes open. Our results showcase that the image perceived by the non-exposed eye compensates for the loss of contrast sensitivity observed in the image perceived by the laser-exposed eye. This indicates that monocular laser-based HMDs, such as Focals, permit dark adaptation to occur naturally for the non-exposed eye.
Conference Paper
Full-text available
Evolution has always been the main driving force of change for both the human body and brain. Presently, in the Information era, our cognitive capacities cannot simply rely on natural evolution to keep up with the immense advancements in the field of Ubiquitous technologies , which remain largely uninformed about our cognitive states. As a result, a so-called " cognitive gap " is forming between the human (users) and the machine (systems) preventing us from fully harnessing the benefits of modern technologies. We argue that a " cogni-tive information layer " , placed in-between human and machine, could bridge that gap, informing the machine side about aspects of our cognition in real time (e.g., attention levels). In this position paper, we present our vision for such a software architecture, we describe how it could serve as a framework for designing and developing cognition-aware applications, and we show-case some application scenarios as a roadmap towards human-machine convergence and symbiosis.
Conference Paper
Full-text available
In recent years, Brain-Computer Interfaces (BCIs) have been steadily gaining ground in the market, used either as an implicit or explicit input method in computers for accessibility, entertainment or rehabilitation. Past research in BCI has heavily neglected the human aspect in the loop, focusing mostly in the machine layer. Further, due to the high cost of current BCI systems, many studies rely on low-cost and low-quality equipment with difficulties to provide significant advancements in physiological computing. Open-Source projects are offered as alternatives to expensive medical equipment. Nevertheless, the effectiveness of such systems over their cost is still unclear, and whether they can deliver the same level of experience as their more expensive counterparts. In this paper, we demonstrate that effective BCI interaction in a Motor-Imagery BCI paradigm can be accomplished without requiring high-end/high-cost devices, by analyzing and comparing EEG systems ranging from open source devices to medically certified systems.
Conference Paper
Virtual and augmented reality headsets are unique as they have access to our facial area: an area that presents an excellent opportunity for always-available input and insight into the user's state. Their position on the face makes it possible to capture bio-signals as well as facial expressions. This paper introduces the PhysioHMD, a software and hardware modular interface built for collecting affect and physiological data from users wearing a head-mounted display. The PhysioHMD platform is a flexible architecture enables researchers and developers to aggregate and interprets signals in real-time, and use those to develop novel, personalized interactions and evaluate virtual experiences. Offering an interface that is not only easy to extend but also is complemented by a suite of tools for testing and analysis. We hope that PhysioHMD can become a universal, publicly available testbed for VR and AR researchers.
Conference Paper
Activity tracking using a wearable device is an emerging research field. Large-scale studies on activity tracking performed with eyewear-type wearable devices remains a challenging area owing to the negative effect such devices have on users' looks. To cope with this challenge, JINS Inc., an eyewear retailer in Japan, has developed a state-of-the-art smart eyewear called JINS MEME. The design of JINS MEME is almost the same as that of Wellington-type eyeglasses so that people can use it in their daily lives. JINS MEME is equipped with sensors to detect a user's eye movement and head motion. In addition to these functions of JINS MEME, JINS developed an application to measure concentration levels of users. In this demonstration, users will experience wearing the JINS MEME glasses and their concentration will be measured while they perform a certain task at our booth.
Article
The Graz Brain-Computer Interface (BCI) analyses and classifies the dynamics of oscillatory EEG components during motor imagery. At this time a patient controls the closing and opening of a hand orthosis with a BCI and students are able to write error free with a spelling rate of 1 character/minute.
Chapter
Established in 1982 as the leading reference on electroencephalography, Drs. Niedermeyer's and Lopes da Silva's text is now in its thoroughly updated Fifth Edition. An international group of experts provides comprehensive coverage of the neurophysiologic and technical aspects of EEG, evoked potentials, and magnetoencephalography, as well as the clinical applications of these studies in neonates, infants, children, adults, and older adults. This edition includes digital EEG and advances in areas such as neurocognition. Three new chapters cover the topics of Ultra-Fast EEG Frequencies, Ultra-Slow Activity, and Cortico-Muscular Coherence. Hundreds of EEG tracings and other illustrations complement the text.