Conference PaperPDF Available

EEGlass: An EEG-Eyeware Prototype for Ubiquitous Brain-Computer Interaction

Authors:

Abstract and Figures

Contemporary Head-Mounted Displays (HMDs) are progressively becoming socially acceptable by approaching the size and design of normal eyewear. Apart from the exciting interaction design prospects, HMDs bear significant potential in hosting an array of physiological sensors very adjacent to the human skull. As a proof of concept, we illustrate EEGlass, an early wearable prototype comprised of plastic eyewear frames for approximating the form factor of a modern HMD. EEGlass is equipped with an Open-BCI board and a set of EEG electrodes at the contact points with the skull for unobtrusively collecting data related to the activity of the human brain. We tested our prototype with 1 participant performing cognitive and sensorimotor tasks while wearing an established Electroencephalography (EEG) device for obtaining a baseline. Our preliminary results showcase that EEGlass is capable of accurately capturing resting state, detect motor-action and Electrooculographic (EOG) artifacts. Further experimentation is required, but our early trials with EEGlass are promising in that HMDs could serve as a springboard for moving EEG outside of the lab and in our everyday life, facilitating the design of neuroadaptive systems.
Content may be subject to copyright.
EEGlass: An EEG-Eyeware Prototype for Ubiquitous
Brain-Computer Interaction
Athanasios Vourvopoulos
Department of Bioengineering
Institute for Systems and Robotics
Instituto Superior Técnico
Universidade de Lisboa
Lisboa, Portugal
athanasios.vourvopoulos@tecnico.ulisboa.pt
Evangelos Niforatos
Department of Computer Science
Norwegian University of Science and
Technology (NTNU)
Trondheim, Norway
evangelos.niforatos@ntnu.no
Michail Giannakos
Department of Computer Science
Norwegian University of Science and
Technology (NTNU)
Trondheim, Norway
michail.giannakos@ntnu.no
ABSTRACT
Contemporary Head-Mounted Displays (HMDs) are progressively
becoming socially acceptable by approaching the size and design
of normal eyewear. Apart from the exciting interaction design
prospects, HMDs bear signicant potential in hosting an array
of physiological sensors very adjacent to the human skull. As a
proof of concept, we illustrate EEGlass, an early wearable proto-
type comprised of plastic eyewear frames for approximating the
form factor of a modern HMD. EEGlass is equipped with an Open-
BCI board and a set of EEG electrodes at the contact points with
the skull for unobtrusively collecting data related to the activity
of the human brain. We tested our prototype with 1 participant
performing cognitive and sensorimotor tasks while wearing an
established Electroencephalography (EEG) device for obtaining a
baseline. Our preliminary results showcase that EEGlass is capa-
ble of accurately capturing resting state, detect motor-action and
Electrooculographic (EOG) artifacts. Further experimentation is
required, but our early trials with EEGlass are promising in that
HMDs could serve as a springboard for moving EEG outside of the
lab and in our everyday life, facilitating the design of neuroadaptive
systems.
CCS CONCEPTS
Hardware Neural systems.
KEYWORDS
Head-Mounted Displays, Electroencephalography, Brain-Computer
Interfaces, Neuroadaptive Systems
ACM Reference Format:
Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos.
2019. EEGlass: An EEG-Eyeware Prototype for Ubiquitous Brain-Computer
Interaction. In Adjunct Proceedings of the 2019 ACM International Joint
Conference on Pervasive and Ubiquitous Computing and the 2019 International
Symposium on Wearable Computers (UbiComp/ISWC ’19 Adjunct), September
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom
©2019 Association for Computing Machinery.
ACM ISBN 978-1-4503-6869-8/19/09. . . $15.00
https://doi.org/10.1145/3341162.3348383
9–13, 2019, London, United Kingdom. ACM, New York, NY, USA, 6 pages.
https://doi.org/10.1145/3341162.3348383
1 STATE OF THE ART
In the year 2022, over 80 million Head-Mounted Display (HMD)
units are expected to ship, up from over 34 million units estimated
to be sold in 2019
1
. This predicted HMD proliferation is propelled by
recent advances in optics and hardware miniaturization, rendering
eyewear the next frontier for Wearable and Ubiquitous Comput-
ing (e.g., Google Glass Enterprise Edition
2
, Microsoft HoloLens 2
3
,
MagicLeap One
4
, Focals by North
5
and Vuzix Blade
6
). Undoubtedly,
an HMD uptake will actualize the vision of Augmented Reality
(AR), disrupting the way we consume information and execute our
daily chores [
3
]. From revisiting the groceries list to navigating
to our destination, a plethora of daily tasks will be reshaped by
AR and the unique form factor of HMDs. Except for the immense
opportunities in interaction design, the HMD form factor promises
an unprecedented contact potential with the human skull: the shell
of our brain where the higher cognitive and perceptual processes
reside.
Evidently, this “skull-contact” potential that eyewear bears could
not go unnoticed. Various commercial eyewear products claim to
utilize EEG (or EOG – electrooculography) and the contact points
with the skull to provide so-called “neurofeedback”. For example,
Narbis sunglasses
7
utilize three electrodes, two at the back of the
temples of the device touching the left and right mastoids, and
one at the tip of a protruding arm that touches the top of the
skull, for tracking concentration. When concentration is low, the
Narbis electrochromatic lenses start darkening for inviting the
user to focus more. Lowdown Focus by Smith
8
employs a more
socially acceptable design equipping a typical pair of sunglasses
with silicon electrodes at the edges of the temples that touch the
left and right mastoids. A companion mobile application connects
to the Lowdown Focus sunglasses for collecting the readings so
that one can track and increase one’s concentration levels. JINS
1
https://www.gartner.com/en/newsroom/press-releases/2018-11-29- gartner-says-
worldwide-wearable- device-sales-to- grow-
2https://www.google.com/glass/start/
3https://www.microsoft.com/en-us/hololens
4https://www.magicleap.com/magic-leap-one
5https://www.bynorth.com
6https://www.vuzix.com/products/blade-smart-glasses
7http://narbis.com/company/
8https://www.smithoptics.com/us/lowdownfocus
UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos
Figure 1: The EEGlass prototype worn by a user.
MEME
9
is perhaps the most prominent eyewear that employs near-
skull contact for providing neurofeedback. JINS MEME utilizes
two electrodes embedded in the bridge of the eyewear that touch
the nasal bone to detect concentration levels by measuring the
duration and the number of eye blinks via EOG [
20
]. More recently,
a US patent was published about the design of an eyewear frame
that features a exible protrusion for holding the EEG electrodes
in contact with the skull in dierent positions [
5
]. Beyond the
strong commercialization interest, the combination of eyewear
with EEG yields interesting niche applications in the domain of
health research and Human-Computer Interaction (HCI). e-Glass
is an EEG-enabled eyewear that employs an OpenBCI board and
a set of electrodes across the inner side of the frame for detecting
epileptic seizures [
19
]. PhysioHMD prototype adopts a bulky “mask”
design that encases a portion of the face for hosting a wide range
of physiological sensors, including EEG electrodes, capturing also
facial expressions [
2
]. PhysioHMD is intended as a platform that
informs the design of AR and Virtual Reality (VR) experiences.
On one hand, commercial approaches that combine modern eye-
wear with EEG (and EOG) are in general socially acceptable, but also
rather limited in providing high-level information about brain activ-
ity (e.g., daily concentration levels), typically via a dedicated mobile
application. Moreover, commercial “neurofeedback” products are
considered “black-box” systems that utilize proprietary hardware
and software. On the other hand, experimental EEG-eyewear proto-
types produce ne-grained information about brain activity, while
utilizing open hardware and software solutions. However, such
prototypes are by default too dorky to wear outside a research lab,
and cumbersome to use for extended periods of time. The EEGlass
prototype attempts to fuse the social acceptability and increased
9https://jins-meme.com/en/
Last accessed on July 3, 2019.
8
Figure 2: The 10-10 system of electrode placement topology
for the EEGlass and Enobio 8.
“wearability” of commercial EEG-eyewear with the high informa-
tion granularity and openness of experimental EEG-eyewear (see
Figure 1). The outcomes of such a successful fusion remain tenta-
tive but can potentially nurture existing and envisioned cognitive
systems [
13
], democratize EEG, and eventually pave the way for
touch-less input.
2 BACKGROUND
The human brain is an electrochemical system that generates a com-
bination of dynamic biosignals or action potentials. EEG is the most
common brain signal acquisition technique established almost a
century ago [
1
]. Non-invasive EEG utilizes scalp-contact electrodes
for capturing the combined electrical activity of populations of
excitable cells known as neurons. When neurons activate, they
produce electromagnetic elds of discrete potential patterns, dis-
tinguished by dierent wave oscillations in the frequency domain.
These patterns are ascribed to dierent states of mental activity
and are identied by the wave oscillations they cause in the fre-
quency domain, known as EEG bands or rhythms [
4
]. EEG rhythms
are divided into dierent frequency ranges including Delta (1–4
Hz), Theta (4–8 Hz), Alpha (8–13 Hz), Beta (13–30 Hz) and Gamma
(25–90 Hz) [
11
], and each rhythm or combination of rhythmic ac-
tivity is linked to dierent mental states. For example, rhythms
in the Alpha and Beta frequency bands are functionally related
to major sensorimotor systems, which activate primarily through
motor preparation or execution [
6
]. Alpha and Theta oscillations
are known to reect cognitive and memory performance [
9
], and
Theta was shown by early EEG studies to be closely connected to
problem-solving, perceptual processing and learning [
18
]. Delta
rhythm is related to concentration, attention and internal process-
ing [
8
], whereas Gamma has been linked to consciousness and sense
of self, and can be volitionally modulated during meditation [
10
].
EEGlass: An EEG-Eyeware Prototype for Ubiquitous Brain-Computer Interaction UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom
Figure 3: The EEGlass electrode contact points and the re-
spective OpenBCI channels.
Interpreting cognitive states or motor intentions from dierent
EEG rhythms is a complex process and is impossible to associate a
single frequency range, or cortical location, to a brain function.
Nowadays, measuring oscillatory brain activity with EEG is
utilized for linking the human brain with computers via Brain-
Computer Interfaces (BCIs). BCIs have been successfully utilized in
the medical domain where BCIs enable amputees to gain control
over prosthetic limbs [
15
]. Other BCI application areas include mon-
itoring user’s cognitive states (e.g., attention levels and workload),
gaming and rehabilitation. Thus, BCIs and wearable technologies
such as HMDs, oer a unique opportunity to measure user needs
over time and in real-life settings, informing how critical software
aspects (e.g., interface) should respond or adapt. The potential of
EEGlass in monitoring surreptitiously and in real-time user’s phys-
iological and cognitive states, renders it an ideal BCI for facilitating
interaction with neuroadaptive systems.
3 EEGLASS PROTOTYPE
The EEGlass prototype is comprised of plastic eyewear frames that
can be tted with a Google Glass HMD. We opted for this type of
eyewear for: (1) low cost, (2) availability, (3) good tting and (4)
modern HMD resemblance. In fact, the selected eyewear frames
follow the trend in the eld of HMDs: hardware miniaturization
and social acceptability [
12
]. The EEG system that we embedded
in the frames is Cyton Biosensing Board by OpenBCI (OpenBCI,
NY, USA). OpenBCI is a popular and aordable open hardware
and software platform for the collection and analysis of biosignals
such as EEG, EMG (Electromyography), ECG (Electrocardiography)
and others, inspired by the grassroots movement of DIY (“Do It
Yourself”) [
21
]. The Cyton board encompasses 8 biopotential input
channels (for hosting up to 8 electrodes), a 3-axes accelerometer,
local storage, wireless communication modules, while being fully
programmable and Arduino compatible. Evidently, the EEGlass
electrode topology is restricted by the eyewear form factor and at
the contact points with the skull. Thus, EEGlass utilizes 3 electrodes
(plus 2 for reference and ground) based on the 10-10 system (see
Figure 2) for measuring brain activity: 1 electrode placed inwards
at the top of the eyewear bridge touching the skull at glabella,
and 2 more electrodes at the inner side of the eyewear temples,
touching the left and right mastoids, behind the left and right ears,
Figure 4: A user performing a motor task while wearing both
EEGlass and Enobio 8 EEG systems.
respectively (see Figure 3). The reference and ground electrodes
are placed at the inner part of the eyewear bridge, touching the left
and right sides of the nasal bone, respectively (see Figure 3).
4 FIRST TRIALS AND EARLY RESULTS
In this work, we explore if the low spatial resolution of a “skull-
peripheral” electrode topology, imposed by the form factor of a
modern HMD, can approximate the accuracy of a typical electrode
topology utilized in EEG studies. To this end, we used Enobio 8
(Neuroelectrics, Barcelona, Spain) EEG system for forming a base-
line. Enobio 8 is a wireless, 8-channel, EEG system with a 3-axes
accelerometer for the recording and visualization of 24-bit EEG
data at 500 Hz. The spatial distribution of electrodes for our trials
followed the 10-10 system conguration, with electrodes placed
over the frontal area (Fpz), central (C3, C4), and parietal (Pz) (see
Figure 2). The electrodes were referenced and grounded to the right
ear lobe, and the electrode impedance was kept at
<
20
k
. Both the
Enobio 8 and the OpenBCI (embedded in the EEGlass) EEG systems
were connected via Bluetooth to a dedicated desktop computer for
raw signal acquisition and processing.
The EEG acquisition session began with a 4-minute period for
acquiring resting state data, and the motor-action session following
next. The resting state data was acquired during alternating 1-
minute periods with eyes open and eyes closed. The subject was
instructed to remain silent while either xating his eye-gaze on a
white cross displayed on a computer screen, or when having his
eyes closed. In the motor-action session, we employed the Graz-
BCI paradigm [
16
] to display a random sequence of directional left
and right arrows on a computer screen (see Figure 4). When an
arrow appeared, the user responded to the stimulus by performing
a motor-action with the corresponding hand. The motor-action
session was congured to acquire data in 24 blocks (epochs) per
class (left and right hand arrow).
UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos
(a) EEGlass resting state. (b) Enobio 8 resting state.
Figure 5: Resting states for EEGlass and Enobio 8. Both EEG systems detect a clear Alpha rhythm peak at 8-12 Hz.
For both systems, we used the OpenVibe acquisition servers for
simultaneous EEG signal acquisition [
17
]. Next, we used the Open-
Vibe designer for obtaining the raw EEG streams from both servers
and synchronizing them with the stimuli, before storing them in a
.gdf le. We processed the acquired EEG signals in MATLAB
®
(The
MathWorks, MA, USA) with the EEGLAB toolbox [
7
]: after import-
ing the data and the channel information, we applied a high-pass
lter at 1 Hz to remove the “baseline drift” followed by line-noise
and harmonics removal at 50 Hz. Then, we used Welch’s method
[
22
] for Power Spectral Density (PSD) of the power spectrum to
compute the average spectral power across the following frequency
bands during resting state: Delta (1–4 Hz), Theta (4–7 Hz), Alpha
(8–12 Hz), and Beta (12–30 Hz). The event-related synchroniza-
tion/desynchronization (ERS/ERD) was extracted following the
standard ERS/ERD method [
14
] across the Alpha band power (8–12
Hz) and the Beta band power (12–30 Hz) over C3 and C4 electrode
locations for the Enobio 8 system, and TP9, TP10 for the OpenBCI,
respectively. We calculated the ERD by using the following formula:
ERD =
(PowerM ot or Ac ti vit yPowerB ase l ine )
PowerBase li ne
100 (1)
Early results from comparing EEG signals acquired via EEGlass
with Enobio 8 indicate that EEGlass captures very closely the band
power of Enobio 8, but also the the decrease of oscillatory activity
Table 1: Resting state and EEG rhythms in µV2/Hz recorded
by EEGlass and Enobio 8 EEG systems.
System Delta Theta Alpha Beta
EEGlass 21.279 4.775 12.992 0.431
Enobio 8 4.984 2.196 13.397 0.2993
(ERD) during the motor-action, as we anticipated. Figure 5 shows
that despite the fundamentally dierent electrode topology of EE-
Glass, both EEGlass and Enobio detect a clear Alpha peak (8–12
Hz) in the electrical activity of the brain during resting state. Ta-
ble 1 summarizes the recorded EEG rhythms during resting state
for both EEGlass and Enobio 8 EEG systems. For investigating if
brain activity linked to motor-action diered substantially between
the two EEG systems, we compared the average ERD between lat-
eral electrodes for both systems:
(TP
9
,C
3
)
for right-hand, and
(CP
10
,C
4
)
for left-hand movement. Dependent samples t-tests
displayed signicant dierences in the average ERD between both
pairs of lateral electrodes (
tT P 9|C3(
199
)=
10
.
214
,p< .
001 and
tC P 10|C4(
199
),p< .
001), as shown in Figure 6 and summarized in
Table 2. This indicates that the captured brain activity related to
upper limb motor-action diered signicantly between the EEGlass
and Enobio 8 EEG systems. Moreover, Figure 7 showcases that
EEGlass was able to detect basic EOG activity related to eye move-
ment in 4 primary directions: up, down, left and right. Although
the current electrode setup can capture eye-movement with only 1
degree-of-freedom (DoF), it can also detect saccadic eye movement
and eye blinks.
Table 2: Average desynchronization (% ERD) between 8–24
Hz per electrode and hand for both EEGlass and Enobio 8
EEG systems.
EEGlass Enobio 8
Electrode TP9 TP10 C3 C4
Movement Right Left Right Left
Mean -5.058 -3.065 -20.272 -18.899
SD 11.6383 12.273 19.048 22.47
EEGlass: An EEG-Eyeware Prototype for Ubiquitous Brain-Computer Interaction UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom
ERD (%)
(a) Left hand ERD as captured by CP10 (EEGlass) and C4 (Enobio 8) electrodes.
ERD (%)
(b) Right hand ERD as captured by TP9 (EEGlass) and C3 (Enobio 8) electrodes.
Figure 6: Average motor-action (ERD) for lateral electrode pairs of both EEGlass and Enobio 8 EEG Systems. Signicant dier-
ences (p< .001) were found between both electrode pairs for both right and left hand motor-action.
5 DISCUSSION AND FUTURE WORK
Our preliminary results serve as a proof of concept for piggybacking
EEG on eyewear and HMDs, yet in a cost-eective, unobtrusive
and socially acceptable fashion. Trials with 1 participant indicate
that the EEGlass is capable of capturing brain activity manifested
in two modes of resting state: (a) eyes open and focused on a target,
and (b) eyes closed. In fact, brain activity recorded during resting
state with EEGlass demonstrates similar variations in frequency
and amplitude to when recorded with an established EEG system
such as Enobio 8. However, recorded brain activity linked to upper
limb motor-action and captured with EEGlass, displayed signicant
dierences when compared to that captured with Enobio 8, as
anticipated due to the localized motor activity of the sensorimotor
cortices under C3, C4 electrodes. Yet, EEGlass managed to capture
ERD from its TP9 and TP10 electrodes, though with less power than
Enobio 8 due to the low spatial resolution of the EEG signals on the
scalp, relying on signal propagation to reach the remotely located
EEGlass electrodes. Moreover, EEGlass was able to detect subtle
eye movements in 4 basic directions, displaying an eye-tracking
potential particularly useful for navigating in heads-up interfaces.
Undoubtedly, low sample size (N=1) and stationary experimental
settings are signicant limitations that we will address in followup
studies. However, human skull and brain anatomy is universally
homogeneous, and the eyewear/HMD form factor ensures a rather
stable electrode contact, only somewhat inuenced by movement.
In future iterations, we will utilize prominent machine learning
techniques for training algorithms to match input from EEGlass
to that of established EEG systems, and test our prototype in user
studies with actual HMDs. We believe a merger between EEG and
HMDs bears an unprecedented potential to revolutionize human-
machine interaction, facilitating touch-less input and greatly in-
creasing human-machine communication throughput.
6 ACKNOWLEDGEMENTS
The authors acknowledge the nancial support of the Swiss Na-
tional Science Foundation (SNSF) under grant number: 184146
We would also like to thank Ph.D. student Octavio Marin Pardo for
helping with the data acquisition.
REFERENCES
[1]
Hans Berger. 1933. Über das elektrenkephalogramm des menschen. European
archives of psychiatry and clinical neuroscience 98, 1 (1933), 231–254.
[2]
Guillermo Bernal, Tao Yang, Abhinandan Jain, and Pattie Maes. 2018. PhysioHMD:
a conformable, modular toolkit for collecting physiological data from head-
mounted displays. In Proceedings of the 2018 ACM International Symposium on
Wearable Computers. ACM, 160–167.
[3]
Mark Billinghurst and Thad Starner. 1999. Wearable devices:new ways to manage
information. Computer 32, 1 (1999), 57–64.
[4]
György Buzsáki and Andreas Draguhn. 2004. Neuronal oscillations in cortical
networks. science 304, 5679 (2004), 1926–1929.
[5]
Robert A Connor. 2018. EEG glasses (electroencephalographic eyewear). US
Patent 9,968,297.
[6]
Nathan E Crone, Diana L Miglioretti, Barry Gordon, and Ronald P Lesser. 1998.
Functional mapping of human sensorimotor cortex with electrocorticographic
spectral analysis. II. Event-related synchronization in the gamma band. Brain: a
journal of neurology 121, 12 (1998), 2301–2315.
[7]
Arnaud Delorme and Scott Makeig. 2004. EEGLAB: an open source toolbox for
analysis of single-trial EEG dynamics including independent component analysis.
Journal of neuroscience methods 134, 1 (2004), 9–21.
[8]
Thalía Harmony, Thalía Fernández, Juan Silva, Jorge Bernal, Lourdes Díaz-Comas,
Alfonso Reyes, Erzsébet Marosi, Mario Rodríguez, and Miguel Rodríguez. 1996.
EEG delta activity: an indicator of attention to internal processing during perfor-
mance of mental tasks. International journal of psychophysiology 24, 1-2 (1996),
161–171.
[9]
Wolfgang Klimesch. 1999. EEG alpha and theta oscillations reect cognitive and
memory performance: a review and analysis. Brain research reviews 29, 2-3 (1999),
169–195.
[10]
Dietrich Lehmann, PL Faber, Peter Achermann, Daniel Jeanmonod, Lorena RR Gi-
anotti, and Diego Pizzagalli. 2001. Brain sources of EEG gamma frequency during
volitionally meditation-induced, altered states of consciousness, and experience
of the self. Psychiatry Research: Neuroimaging 108, 2 (2001), 111–121.
[11]
Ernst Niedermeyer and FH Lopes da Silva. 2005. Electroencephalography: basic
principles, clinical applications, and related elds. Lippincott Williams & Wilkins.
[12]
Evangelos Niforatos and Mélodie Vidal. 2019. Eects of a Monocular Laser-
Based Head-Mounted Display on Human Night Vision. In Proceedings of the 10th
Augmented Human International Conference 2019. ACM, 31.
UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos
Nz
Figure 7: Electrooculographic (EOG) activity (µV/ms) recorded by EEGlass and linked to eye movement in 4 basic directions.
[13]
Evangelos Niforatos, Athanasios Vourvopoulos, and Marc Langheinrich. 2017.
Amplifying human cognition: bridging the cognitive gap between human and
machine. In Proceedings of the 2017 ACM International Joint Conference on Per-
vasive and Ubiquitous Computing and Proceedings of the 2017 ACM International
Symposium on Wearable Computers. ACM, 673–680.
[14]
Gert Pfurtscheller and A Aranibar. 1979. Evaluation of event-related desyn-
chronization (ERD) preceding and following voluntary self-paced movement.
Electroencephalography and clinical neurophysiology 46, 2 (1979), 138–146.
[15]
Gert Pfurtscheller and Christa Neuper. 2001. Motor imagery and direct brain-
computer communication. Proc. IEEE 89, 7 (2001), 1123–1134.
[16]
Gert Pfurtscheller, Christa Neuper, GR Muller, Bernhard Obermaier, Gunter
Krausz, A Schlogl, Reinhold Scherer, Bernhard Graimann, Claudia Keinrath,
Dimitris Skliris, et al
.
2003. Graz-BCI: state of the art and clinical applications.
IEEE Transactions on neural systems and rehabilitation engineering 11, 2 (2003),
1–4.
[17]
Yann Renard, Fabien Lotte, Guillaume Gibert, Marco Congedo, Emmanuel Maby,
Vincent Delannoy, Olivier Bertrand, and Anatole Lécuyer. 2010. Openvibe: An
open-source software platform to design, test, and use brain–computer interfaces
in real and virtual environments. Presence: teleoperators and virtual environments
19, 1 (2010), 35–53.
[18]
Daniel L Schacter. 1977. EEG theta waves and psychological phenomena: A
review and analysis. Biological psychology 5, 1 (1977), 47–82.
[19]
Dionisije Sopic, Amir Aminifar, and David Atienza. 2018. e-glass: A wearable
system for real-time detection of epileptic seizures. In 2018 IEEE International
Symposium on Circuits and Systems (ISCAS). IEEE, 1–5.
[20]
Yuji Uema and Kazutaka Inoue. 2017. JINS MEME algorithm for estimation and
tracking of concentration of users. In Proceedings of the 2017 ACM International
Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the
2017 ACM International Symposium on Wearable Computers. ACM, 297–300.
[21]
Athanasios Vourvopoulos and Sergi Bermudez i Badia. 2016. Usability and Cost-
eectiveness in Brain-Computer Interaction: Is it User Throughput or Technology
Related?. In Proceedings of the 7th Augmented Human International Conference
2016. ACM, 19.
[22]
Peter Welch. 1967. The use of fast Fourier transform for the estimation of power
spectra: a method based on time averaging over short, modied periodograms.
IEEE Transactions on audio and electroacoustics 15, 2 (1967), 70–73.
... As all these signals alone might lack specificity in detecting user state (such as boredom, stress, etc.), hybrid methods have been proposed fusing between different signal types. These methods have been demonstrated, e.g., for combining eyetracking and EEG (Krol et al., 2017;Vourvopoulos et al., 2019). Moreover, integration of electromyographic (EMG) activity has been shown to result in a better and more stable BMI performance in motor tasks (Leeb et al., 2011). ...
... Using further sensors in fusion with EEG data can increase error detection substantially, especially when combined with double-step decoding as an additional validation method. Such fusion has been successfully tested showing improved detection of EEG events when measured together with eye signals (Vourvopoulos et al., 2019). When combined with machine learning, this sensor fusion will also be faster, allowing the detection of faster physiological signals and the prediction of possible errors that are anticipated to occur based on prior decisions during the HRC. ...
... In addition, fusion allows a crossmodal reconstruction module to learn dependencies between simultaneously recorded data streams and based on that reconstruct signal data in case one modality is missing. Compared to uni-modal classification, this has been demonstrated to provide a more robust output by the decoder (Vourvopoulos et al., 2019;Rahate et al., 2022). ...
Article
Full-text available
Human activity recognition (HAR) and brain-machine interface (BMI) are two emerging technologies that can enhance human-robot collaboration (HRC) in domains such as industry or healthcare. HAR uses sensors or cameras to capture and analyze the movements and actions of humans, while BMI uses human brain signals to decode action intentions. Both technologies face challenges impacting accuracy, reliability, and usability. In this article, we review the state-of-the-art techniques and methods for HAR and BMI and highlight their strengths and limitations. We then propose a hybrid framework that fuses HAR and BMI data, which can integrate the complementary information from the brain and body motion signals and improve the performance of human state decoding. We also discuss our hybrid method’s potential benefits and implications for HRC.
... Finally, the Papers field provides the list of reviewed works employing a specific device. A total of 13 of 84 papers [86,123,128,134,138,143,154,[158][159][160][161][162][163] do not appear in this column, having that the authors design and use custom EEG headsets. Finally, Khan et al. [135] and Vourvopoulos et al. [162] compare their custom devices with Emotiv EPOC+ and Enobio 8, respectively. ...
... A total of 13 of 84 papers [86,123,128,134,138,143,154,[158][159][160][161][162][163] do not appear in this column, having that the authors design and use custom EEG headsets. Finally, Khan et al. [135] and Vourvopoulos et al. [162] compare their custom devices with Emotiv EPOC+ and Enobio 8, respectively. ...
... A total of 14 of 84 works prefer reducing the number of electrodes provided by the device producers [88,92,104,106,110,126,142,162] and especially make a selection on the channel placed over the central cortical area. In particular, only the C{3,4} [139,147], Cz [92,143,148], and C{1,2} [102] electrodes are considered. ...
Article
Full-text available
In recent decades, the automatic recognition and interpretation of brain waves acquired by electroencephalographic (EEG) technologies have undergone remarkable growth, leading to a consequent rapid development of brain–computer interfaces (BCIs). EEG-based BCIs are non-invasive systems that allow communication between a human being and an external device interpreting brain activity directly. Thanks to the advances in neurotechnologies, and especially in the field of wearable devices, BCIs are now also employed outside medical and clinical applications. Within this context, this paper proposes a systematic review of EEG-based BCIs, focusing on one of the most promising paradigms based on motor imagery (MI) and limiting the analysis to applications that adopt wearable devices. This review aims to evaluate the maturity levels of these systems, both from the technological and computational points of view. The selection of papers has been performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), leading to 84 publications considered in the last ten years (from 2012 to 2022). Besides technological and computational aspects, this review also aims to systematically list experimental paradigms and available datasets in order to identify benchmarks and guidelines for the development of new applications and computational models.
... The traditional way of acquiring signals has been through gel-electrodes that are placed on the body. In addition to the use of traditional wearables such as smartwatches and fitness trackers, recent advances in fabrication and electronics have led to the integration of bio-sensing electrodes in other devices such as eye-glasses [101], VR headmounted displays [102], and textiles [97]. ...
Article
Full-text available
Data generated from sources such as wearable sensors, medical imaging, personal health records, pathology records, and public health organizations have resulted in a massive information increase in the medical sciences over the last decade. Advances in computational hardware, such as cloud computing, Graphical Processing Units (GPUs), and Tensor Processing Units (TPUs), provide the means to utilize these data. Consequently, many Artificial Intelligence (AI)-based methods have been developed to infer from large healthcare data. Here, we present an overview of recent progress in artificial intelligence and biosensors in medical and life sciences. We discuss the role of machine learning in medical imaging, precision medicine, and biosensors for the Internet of Things (IoT). We review the most recent advancements in wearable biosensing technologies that use AI to assist in monitoring bodily electro-physiological and electro-chemical signals and disease diagnosis, demonstrating the trend towards personalized medicine with highly effective, inexpensive, and precise point-of-care treatment. Furthermore, an overview of the advances in computing technologies, such as accelerated artificial intelligence, edge computing, and federated learning for medical data, are also documented. Finally, we investigate challenges in data-driven AI approaches, the potential issues that biosensors and IoT-based healthcare generate, and the distribution shifts that occur among different data modalities, concluding with an overview of future prospects.
... For example, SOTA deep neural network (DNN) models enable the cross-referencing of language and image information. On the other hand, human capability can be augmented by wearable technology [43,50,53]. For example, in Industry 4.0, augmented reality (AR) glasses are potentially used in activities such as production, quality control, safety management, etc. [37]. ...
Article
Full-text available
Background: In the modern era, the complexity and increase in healthcare data will drive us towards the growing use of artificial intelligence. For optimal progress in precision medicine, a thorough examination of comprehensive patient data alongside various and extensive factors is essential to differentiate between sick and relatively healthy individuals. Artificial intelligence has the potential to enhance patient care and facilitate easier decision-making for healthcare professionals through advanced computations and inferences, enabling the system to reason, learn, and ultimately streamline medical decision-making. Artificial intelligence is currently being employed by healthcare providers and life sciences companies in various domains. In this article, we will review recent advancements in the application of artificial intelligence in the fields of medicine, pharmaceuticals, and genomics. Additionally, we will discuss the role of machine learning in medical imaging, precision medicine, and biosensors. The article will also explore some advances in biosensor technologies that utilize artificial intelligence to assist in monitoring electro-physiological and electrochemical signals of the body and diagnosing diseases. These advancements indicate a trend towards personalized medicine, which is both highly effective, cost-effective, and precise in the point of care. Researchers, with access to a wide range of datasets and modern computational techniques such as machine learning (ML) and deep learning (DL), can usher in a new era of genomics and effective drug discovery. Deep learning, using algorithms to create an artificial neural network (ANN), can autonomously learn and make decisions, mimicking the human brain.
Article
Neurofeedback refers to the process of feeding a sensory representation of brain activity back to users in real time to improve a particular brain function, e.g., their focus and/or attention on a particular task. This study addressed the notable lack of research on methods used to visualize EEG data and their effects on the immersive quality of VR. We developed an algorithm to quantify focus, yielding a focus score. A pre-study with twenty participants confirmed its effectiveness in distinguishing between focused and relaxed mental states. Subsequently, we used this focus score to prototype a VR experience system visualizing the focus score in preconfigured manners, which was utilized in an exploratory study to assess the impact of different neurofeedback visualization methods on user engagement and focus in VR. Among all the visualization methods evaluated, the environmental scheme stood out due to its superior usability during task execution, its ability to evoke positive emotions through the visualization of objects or scenes, and its minimal deviation from user expectations. Additionally, we explored design guidelines based on collected results for future research to further refine the visualization scheme, ensuring effective integration of the focus score within the VR environment. These enhancements are crucial for designing neurofeedback visualization schemes that aim to boost participant focus in VR settings, offering significant insights into the optimization of such technologies.
Conference Paper
Full-text available
Head-mounted displays (HMDs) are expected to dominate the market of wearable electronics in the next 5 years. This foreseen proliferation of HMDs yields a plethora of design opportunities for revolutionizing everyday life via novel use cases, but also generates a considerable number of substantial safety implications. In this work, we systematically investigated the effect of a novel monocular laser-based HMD on the ability of our participants to see in low ambient light conditions in lab settings. We recruited a total of 19 participants in two studies and performed a series of established vision tests while using the newly available Focals by North HMD. We tested our participants' night vision after being exposed to different levels of laser luminous power and laser colors while using Focals, either with one or both eyes open. Our results showcase that the image perceived by the non-exposed eye compensates for the loss of contrast sensitivity observed in the image perceived by the laser-exposed eye. This indicates that monocular laser-based HMDs, such as Focals, permit dark adaptation to occur naturally for the non-exposed eye.
Conference Paper
Full-text available
Evolution has always been the main driving force of change for both the human body and brain. Presently, in the Information era, our cognitive capacities cannot simply rely on natural evolution to keep up with the immense advancements in the field of Ubiquitous technologies , which remain largely uninformed about our cognitive states. As a result, a so-called " cognitive gap " is forming between the human (users) and the machine (systems) preventing us from fully harnessing the benefits of modern technologies. We argue that a " cogni-tive information layer " , placed in-between human and machine, could bridge that gap, informing the machine side about aspects of our cognition in real time (e.g., attention levels). In this position paper, we present our vision for such a software architecture, we describe how it could serve as a framework for designing and developing cognition-aware applications, and we show-case some application scenarios as a roadmap towards human-machine convergence and symbiosis.
Conference Paper
Full-text available
In recent years, Brain-Computer Interfaces (BCIs) have been steadily gaining ground in the market, used either as an implicit or explicit input method in computers for accessibility, entertainment or rehabilitation. Past research in BCI has heavily neglected the human aspect in the loop, focusing mostly in the machine layer. Further, due to the high cost of current BCI systems, many studies rely on low-cost and low-quality equipment with difficulties to provide significant advancements in physiological computing. Open-Source projects are offered as alternatives to expensive medical equipment. Nevertheless, the effectiveness of such systems over their cost is still unclear, and whether they can deliver the same level of experience as their more expensive counterparts. In this paper, we demonstrate that effective BCI interaction in a Motor-Imagery BCI paradigm can be accomplished without requiring high-end/high-cost devices, by analyzing and comparing EEG systems ranging from open source devices to medically certified systems.
Conference Paper
Virtual and augmented reality headsets are unique as they have access to our facial area: an area that presents an excellent opportunity for always-available input and insight into the user's state. Their position on the face makes it possible to capture bio-signals as well as facial expressions. This paper introduces the PhysioHMD, a software and hardware modular interface built for collecting affect and physiological data from users wearing a head-mounted display. The PhysioHMD platform is a flexible architecture enables researchers and developers to aggregate and interprets signals in real-time, and use those to develop novel, personalized interactions and evaluate virtual experiences. Offering an interface that is not only easy to extend but also is complemented by a suite of tools for testing and analysis. We hope that PhysioHMD can become a universal, publicly available testbed for VR and AR researchers.
Conference Paper
Activity tracking using a wearable device is an emerging research field. Large-scale studies on activity tracking performed with eyewear-type wearable devices remains a challenging area owing to the negative effect such devices have on users' looks. To cope with this challenge, JINS Inc., an eyewear retailer in Japan, has developed a state-of-the-art smart eyewear called JINS MEME. The design of JINS MEME is almost the same as that of Wellington-type eyeglasses so that people can use it in their daily lives. JINS MEME is equipped with sensors to detect a user's eye movement and head motion. In addition to these functions of JINS MEME, JINS developed an application to measure concentration levels of users. In this demonstration, users will experience wearing the JINS MEME glasses and their concentration will be measured while they perform a certain task at our booth.
Article
The Graz Brain-Computer Interface (BCI) analyses and classifies the dynamics of oscillatory EEG components during motor imagery. At this time a patient controls the closing and opening of a hand orthosis with a BCI and students are able to write error free with a spelling rate of 1 character/minute.
Chapter
Established in 1982 as the leading reference on electroencephalography, Drs. Niedermeyer's and Lopes da Silva's text is now in its thoroughly updated Fifth Edition. An international group of experts provides comprehensive coverage of the neurophysiologic and technical aspects of EEG, evoked potentials, and magnetoencephalography, as well as the clinical applications of these studies in neonates, infants, children, adults, and older adults. This edition includes digital EEG and advances in areas such as neurocognition. Three new chapters cover the topics of Ultra-Fast EEG Frequencies, Ultra-Slow Activity, and Cortico-Muscular Coherence. Hundreds of EEG tracings and other illustrations complement the text.