Conference PaperPDF Available

Abstract and Figures

Previous studies that involve measuring EEG, or electroencephalograms, have mainly been experimentally-driven projects; for instance, EEG has long been used in research to help identify and elucidate our understanding of many neuroscientific, cognitive, and clinical issues (e.g., sleep, seizures, memory). However, advances in technology have made EEG more accessible to the population. This opens up lines for EEG to provide more information about brain activity in everyday life, rather than in a laboratory setting. To take advantage of the technological advances that have allowed for this, we introduce the Brain-EE system, a method for evaluating user engaged enjoyment that uses a commercially available EEG tool (Muse). During testing, fifteen participants engaged in two tasks (playing two different video games via tablet), and their EEG data were recorded. The Brain-EE system supported much of the previous literature on enjoyment; increases in frontal theta activity strongly and reliably predicted which game each individual participant preferred. We hope to develop the Brain-EE system further in order to contribute to a wide variety of applications (e.g., usability testing, clinical or experimental applications, evaluation methods, etc.).
Content may be subject to copyright.
Brain-EE: Brain Enjoyment Evaluation using Commercial
EEG Headband
Maher Abujelala
HERACLEIA Human-Centered Computing Laboratory
Department of Computer Science and Engineering
University of Texas at Arlington
maher.abujelala@mavs.uta.edu
Aayush Sharma
HERACLEIA Human-Centered Computing Laboratory
Department of Computer Science and Engineering
University of Texas at Arlington
aayush.sharma@mavs.uta.edu
Cheryl Abellanoza
HERACLEIA Human-Centered Computing Laboratory
Department of Computer Science and Engineering
Department of Psychology
University of Texas at Arlington
cheryl.abellanoza@mavs.uta.edu
Fillia Makedon
HERACLEIA Human-Centered Computing Laboratory
Department of Computer Science and Engineering
University of Texas at Arlington
makedon@uta.edu
ABSTRACT
Previous studies that involve measuring EEG, or
electroencephalograms, have mainly been experimentally-driven
projects; for instance, EEG has long been used in research to help
identify and elucidate our understanding of many neuroscientific,
cognitive, and clinical issues (e.g., sleep, seizures, memory).
However, advances in technology have made EEG more
accessible to the population. This opens up lines for EEG to
provide more information about brain activity in everyday life,
rather than in a laboratory setting. To take advantage of the
technological advances that have allowed for this, we introduce
the Brain-EE system, a method for evaluating user engaged
enjoyment that uses a commercially available EEG tool (Muse).
During testing, fifteen participants engaged in two tasks (playing
two different video games via tablet), and their EEG data were
recorded. The Brain-EE system supported much of the previous
literature on enjoyment; increases in frontal theta activity strongly
and reliably predicted which game each individual participant
preferred. We hope to develop the Brain-EE system further in
order to contribute to a wide variety of applications (e.g., usability
testing, clinical or experimental applications, evaluation methods,
etc.).
CCS Concepts
Human-centered computing~Interaction design theory,
concepts and paradigmsHuman-centered computing~User
studies Human-centered computing~HCI theory, concepts and
models Theory of computation~Active learningComputing
methodologies~Machine learning algorithms Mathematics of
computing~Statistical paradigms
Keywords
EEG; Headband; Enjoyment; Muse; Neurofeedback; BCI
1.!INTRODUCTION
Brain-Computer Interface (BCI) describes the neurofeedback
interface that allows users to communicate and express their
thoughts and feelings without talking or moving. Although BCI
applications are still very limited, the focus of BCI has expanded
over the recent years from helping diagnose mental disorders to
enhance human life in different aspects. For instance, BCI can
provide people with severe motor disabilities a new way to control
augmentative technologies, and help others relief, relax and
meditate. Since BCIs have access to the brain signals and can
estimate the human emotions, the interaction between human and
machine can be improved and adapting to the human preference
[16].
In this paper, we present the Brain-EE system, which can predict
engaged enjoyment when performing different tasks. We start
this paper with a short background of EEG signals used in BCI
systems (section 2). In section 3, we share related work of BCI
systems designed for daily life use. A detailed description of the
Brain-EE system is included in section 4. In section 5 and 6, we
discuss our findings and outline future steps for the Brain-EE
system.
2.!BACKGROUND OF EEG
Electroencephalogram (EEG) is a measure of the electrical
activity that typically occurs around the scalp as a result of brain
activity (i.e., cortical activity). There are five EEG frequencies
that are usually denoted in research: delta (1-4 Hz), theta (5-8 Hz),
alpha (9-13 Hz), beta (12-30 Hz), and gamma (30-50Hz) [18].
EEG can typically be seen in neuroscientific research, where
different activity patterns can reflect several types of phenomena.
For example, EEG is one of the most common tools used in the
study of sleep [2]. EEG studies often serve as important tools in
memory research [6]. Many EEG studies are done in conjunction
with other brain imaging techniques (like functional magnetic
resonance imaging, or fMRI) to identify the presence of abnormal
brain activity, such as with seizures or stroke [18] [19]. Some
studies use EEG to measure mood states; for instance, it has been
shown that increased cortical alpha and beta activity can reflect
stress during activities [1] [12] [13]. Also, frontal alpha activity
and increased theta correlations, but no change in frontal beta
activity, has been connected to measuring engaged enjoyment
during different types of activities [7] [9] [17].
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. Copyrights
for components of this work owned by others than ACM must be
honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee. Request permissions from
Permissions@acm.org.
PETRA’16, June 29 July 01, 2016, Corfu Island, Greece
© 2016 ACM. ISBN 978-1-4503-4337-4/16/06…$15.00
DOI: http://dx.doi.org/10.1145/2910674.2910691
Though EEG seems to have considerable versatility in helping
people understand how their brain activity may correlate with
certain states or activities, formal EEG methods in research
laboratories can be expensive and difficult to set up. For example,
formal EEG laboratories often use EEG caps that have many
recording sites (i.e., electrodes) that require the placement of gel
or paste on the scalp in order to ensure good connectivity for clean
recordings. Researchers require advanced knowledge of statistical
and practical techniques for running experimental studies. Data
analysis may need complex methods and special computing
requirements. These studies also often take a long amount of time
as they move from early stages (preparation, IRB review) to later
stages (data collection/screening/cleaning/analysis and formal
write-up). Finally, as with any experimental study, applicability
to everyday life is limited; the environment of the formal
laboratory often does not reflect the environment in which people
live their lives.
Lately, the consumer market has attempted to make use of the
great advances discovered using EEG in the last couple of decades
for a variety of uses. As such, EEG may be a useful tool for
measuring several types of user states, and current
experimentation and methodology are allowing for the expansion
of EEG use outside of the laboratory and into everyday life.
3.!LIFESTYLE EEG INSTRUMENTS AND
APPLICATIONS
The advances of sensor technology have led new cost- and power-
efficient devices to the market, making them available for a larger
population of developers and researchers. Nowadays, many
portable EEG devices are consumer-grade, low-cost devices that
are targeted for lifestyle applications. These products also often
rebrand EEG data with a simpler, easily-understood term:
neurofeedback, or NFB.
There are many companies that produce these low-cost NFB
devices. Vendors like InteraXon, NeuroSky, Emotiv, Melon, and
Versus provide some of these off-the-shelf, inexpensive devices
for consumers. The number of electrodes on these devices is
limited (i.e., 2-14 electrodes) compared to the clinical grade
devices (i.e., 16-32), their resolution is low, and the electrodes are
usually focused on a specific portion of the brain; however, these
devices are still appropriate for specific applications.
Many of these portable EEG devices are simple to set up; they
connect via Bluetooth to a smartphone, a computer, or a
microcontroller, where data can be analyzed directly. They use
dry electrodes that do not require intensive preparation or clean-
up, and these electrodes connect to the skin without the need for
any gel or paste. These portable, cost-effective devices may also
be used with related available EEG tools and open-source
platforms (e.g., OpenEEG, BCI2000, OpenViBE, and MuLES).
These changes have helped evolve EEG applications in both novel
and established fields [3]. For example, [10] has designed a
custom-made driver drowsiness detection system that uses an
open-source hardware platform with dry electrodes to send
drowsiness level to the driver’s smartwatch, and alert him/her
when it is unsafe to drive. As such, these changes allow vendors
to provide research tools that make their devices applicable to
newer, lesser explored methods [5].
One of these uses is detecting the psychological human state,
including but not limited to: happiness, enjoyment, pain, or
concentration. In [8], the Muse headband, from InteraXon, was
used to detect cold-induced pain using self-calibrating protocols
and various classification algorithms. Similarly, [11] has shown
the possibility of using the Muse headband to measure
concentration and relaxation.
Necomimi Brainwave Cat Ears, from NeuroSky, is a toy that uses
EEG signals from the frontal lobe of the brain to present four
mind states in real-time including: high relaxation,
focus/relaxation, high focus, and high interest [15]. The pose of
the friendly-looking ears on the headband changes according to
the current state of mind. Also, [14] mimics the concept of light
bulb/idea metaphor: mounted on the helmet is a light bulb that
illuminates when the sensors detect an increase in thinking and
focus.
Given that human emotions are traditionally qualitative and are
very difficult to estimate, these devices can supplement other
measures of emotions, like self-assessment (e.g., through Likert or
dichotomous scales) or psychological examinations. In [4],
researchers used activities that tested attention and executive
functioning (i.e., the Stroop test [21] and the Towers of Hanoi
[20]) as well as a questionnaire to assess the suitability of the
NeuroSky headset for measuring meditation and attention levels.
Further, in [17], game boredom and flow were estimated in
personalized training using Support Vector Machine (SVM)
analysis while playing different levels of Tetris games.
Based on this new line of research, we introduce an idea to use
one such consumer-grade EEG device (Muse) to measure user
enjoyment. Our system, the Brain-EE system, aims to measure
engaged enjoyment in such a way that we can predict participant’s
preferences for activities based on their EEG data. In the next
section, we explain the methods we used to evaluate user
enjoyment.
4.!SYSTEM DESCRIPTION
The Brain-EE system aims to measure participant enjoyment
through EEG activity as it differs during two games. To measure
EEG, Brain-EE uses the commercially available product, Muse.
The two games that were played by the users, and for which users
would rate their enjoyment, were Piano Tiles 2 and Soccer 2016.
Both games were viewed and played in tablet platform. To create
the user interface and to analyze the data, we used MATLAB.
Figure 1 gives a general overview of the system setup. The
following subsections go into greater detail about our methods.
Figure 1: System Overview
4.1!Participants
Fifteen total healthy participants were used in this study. All were
University of Texas at Arlington students (13 males, 2 female)
aged 20-35.
4.2!Equipment and Software
4.2.1!Hardware
The EEG device used in this experiment is the Muse headband
from InteraXan. It is an off-the shelf, low-cost EEG headband,
and it has been used in multiple studies [3, 5, 8, 11, 14]. It has
seven EEG electrodes (4 Channels and 3 References) (please refer
to Figure 2) and 3-axis accelerometer. The sample rate of the used
EEG signals is 10 Hz, whereas the sample rate of the
accelerometer is 50 Hz. All data are transmitted via Bluetooth to
a laptop running Mac OS Yosemite 10.10.5. Also, for this study,
Relax Melodies were used during baseline data collection
(“Relax” phase), and the games used in testing (“Game 1” and
“Game 2” phases) were run on Asus tablet running Android OS
4.0 .
4.2.2!Software
For our experiment, we used two different games that were
comparable in skill level (i.e., using a touchpad screen to
complete the game) but were polarizing in likability; specifically,
we used Soccer 2016 (less likability) and Piano Tiles 2 (more
likability). With the Muse headband, InteraXon provides Muse
SDK (i.e., Muse IO, MusePlayer, MuseLab). In our experiment,
we take advantage of MuseIO, which connects with Muse via
Bluetooth and acquires EEG data, accelerometer data, and other
data (e.g., jaw clenches, eye blinks). The MuseIO streams these
data back to a specific port on the PC in the form of OSC (open
sound control) messages. To retrieve the data, we used
MATLAB’s Instrumentation Control Toolbox, which lets
MATLAB connect directly to that specified port using a TCP/IP
protocol. Once the data are received in MATLAB, our MATLAB
GUI application filters, processes, and analyzes the data.
4.3!Interactive User Interface
Our interface is user-friendly, clear, and easy to navigate. First,
the GUI provides step-by-step instruction for proper completion
of setup and data collection. Also, it plots real-time graphs for
accelerometer data to check if data is fetched in real time, as well
as the is_goodsignals provided by Muse that show whether the
EEG sensors on the headband are well-connected to the
participant’s head. In our experiment, data were collected in
intervals of 120 seconds, with each collection phase terminating
automatically. Data are analyzed using an algorithm (discussed in
detail Section 4.6), and the final result of the analysis is shown
with a picture of the game that the participant is predicted to
prefer.
4.4!Procedure
In our experiment, after giving consent to participate in the study,
participants answered relevant demographic questions (e.g. age,
gender). Then, participants were fitted with the Muse headset.
Connectivity for all four channels (via “is_good” signals) was
displayed; proper connectivity was checked before proceeding to
the experimental phases. After proper connection with the Muse
was ensured, the baseline phase (“Relax”) was established.
During this time, participants were asked to close their eyes and
listen to calming white noise for 120 seconds, while their EEG
data were recorded. After this, participants were asked to fill out
a general mood survey. Next, the two active phases (“Game 1”
and “Game 2”) were established. During each of these phases, the
participants were asked to play one of two games (Soccer 2016 or
Piano Tiles 2) for 120 seconds, while their EEG data were
recorded. After Game 1 and Game 2 phases, participants were
asked to fill out general mood surveys and to rate the games. For a
walkthrough of the protocol, as well as an example of the user
interface, please refer to our video at
https://youtu.be/brFZ93Omq5U.
4.5!Data Collection
The EEG signals are obtained from the four Muse input
electrodes. Two input electrodes are located on the forehead, and
one input electrode above each ear as shown in blue in Figure 3.
Three reference electrodes are located in the middle between the
two input electrodes on the forehead as shown in green in Figure
3. These EEG signals are represented as the absolute band power
for the 5 standard frequencies commonly used in EEG
applications (delta, theta, alpha, beta, and gamma). Due to the
frequent disconnection of electrodes while moving the head, the
EEG signals are filtered based on the is_goodvariable provided
by the Muse. In case any electrode is not properly connected
during the experiment, all the recorded data during the period of
disconnection is omitted from analysis.
Figure 2: Muse headband.
Figure 3: Muse electrode locations by 10-
20 International Standards.
4.6!Classification Methods and Results
EEG data analyses, particularly those involved in experimental
literature, often require elaborate filtering and preprocessing
methods in order to make group comparisons (t-tests). However,
brain activity and structures can vary drastically from person to
person; this means that it can be hard to apply findings to the
general populations [8]. Our goal is to apply a classification
method that should work across these differences and for all
healthy people. Therefore, we did not use the actual EEG data
directly in our classification method. Instead, we used the t-test
results.
For each frequency band (e.g., alpha) we used the average of the 4
channels. By doing that, we had five arrays of frequency bands
data for each phase of the experiment (i.e., “Relax”, “Game 1”,
and “Game 2”). During analysis, we filtered out data that
reflected poor connectivity. Since this filtering process necessarily
varied by individuals, there would sometimes be unequal amounts
of data points for different phases of the experiment for different
participants. To ensure that we had an equal amount of data
points across phases and across participants, we omitted data
based on the start and end times of each of the phases of
experiments, and we ignored remainder data (for example, if the
Relax phase had 800 data points, and Game 1 had 1000 data
points, we would only use the first 800 data points of Game 1).
Then, we ran a t-test comparing Game 1 data and Relax phase
data, as well as a t-test comparing Game 2 data and Relax phase
data. By doing this, the data of each participant is simplified and
summarized in t-test results that represent the changes in EEG
between Relax vs. Game 1, as well as between Relax vs. Game 2
(one result each for delta, theta, alpha, beta, and gamma
frequencies, for each of the two comparisons).
Finally, the results of the t-tests were used to train a linear
regression to be able to predict which out of two options (Game 1
or Game 2) the participant engaged with more. After using the
data of the first 10 participants to train the linear regression, the
linear regression was run for the last 5 participants. Equation 1 is
the result of the linear regression, and it evaluates both of the
games based on the t-test results of all the frequency bands. The
game that yields a higher ! value is the game that is predicted to
be the game that the participant reports enjoying the most. When
this equation was tested on the last 5 participants, it resulted in
100% accuracy in identifying which games they preferred.
! " #$%$&'( # $%$()&* + $%$,'&- # $%$$.,/ + $%$$$01
# $%$$),2333333333333333333333333333333333333333333333333333333333456
4.7!Analysis of Results
In Section 2, we discussed that other studies have found that
frontal alpha activity and increased theta correlations, but no
change in frontal beta activity, has been connected to measuring
engaged enjoyment during different types of activities [7] [9] [17].
We find that our results in this experiment partly agrees with these
studies. Table 1 includes the coefficients of the linear regression
equation, and their multiplication with the average of t-tests for
each of the frequency bands for the training data that correspond
to either enjoying or not enjoying the game. In the calculation of
the coefficients of Equation 1, the ! in the linear regression is set
to 1 for the games that participants enjoyed and to 0 for the games
that the participants did not enjoy. With that in mind, we can see
that in the highlighted section of Table 1, where coefficients are
multiplied with the average t-test results of the participants, that
theta is significantly higher for the game that they enjoyed, and in
the algorithm, has a positive coefficient. This suggests that the
increase in theta reflects more enjoyment. Contrastingly, alpha
and delta are higher for the game that they enjoyed, but in the
algorithm, they have negative coefficients. This contradictory
result suggests that alpha and delta are less related with the
prediction of enjoyment. Finally, though beta and gamma change
very slightly, their coefficients seem to have little impact in
predicting enjoyment. Despite the fact that these remarks may not
apply to all participants, they hold potential for a significant
influence on enjoyment research.
Table 1: Summary of the t-test averages across all the
frequency bands.
t-test!of
*
-
/
1
2
Coeff.(C)
-0.0136
0.0256
-0.0072
0.0009
-0.0032
Enjoy (E)
34.49
60.97
29.33
64.01
23.01
No Enjoy
1.73
16.26
5.54
47.67
28.14
C×E
-0.4691
1.5608
-0.2112
0.0576
-0.0736
C×NE
-0.0235
0.4162
-0.0399
0.0429
-0.0900
Coeff. (C)represents the coefficients of the linear regression
equation. Enjoy (E)represents the average of t-tests of the
frequency bands corresponding to enjoying a game. No Enjoy
(NE)represents the average of t-tests of the frequency bands
corresponding to not enjoying a game. “C×E” and “C×NE”
represent the simple multiplication of the coefficients and the
corresponding t-test averages for enjoyment and no enjoyment,
respectively.
5.!DISCUSSION AND FUTURE WORK
Initial statistical analyses helped with the refinement of the Brain-
EE system. First, group data for rest vs. active conditions
matched what was expected: no changes in beta but increases in
frontal alpha (t(13774) = 27.22, p < .001) and theta (t(13774) =
62.69, p < .001) activity, with theta being the best predictor of
whether users reported liking vs. not liking a game (χ2(3) =
4912.79, p < .001, accounting for 67.8% of the variance).
However, early variability of the results (especially with regarded
to alpha and beta), as well as a desire to look for individual data
differences rather than group differences, led to the development
of the algorithm. This early variability in the EEG data might
suggest differences in the magnitude of enjoyment. For instance,
while one participant reported “yes” when asked if he liked the
game, he rated the game a 6/10, which may suggest he did not
strongly like the game. In the development stages of Brain-EE,
the purpose of the study was to detect enjoyment, rather than to
evaluate the participant’s preference between two tasks.
However, when the same method (equation 1) was used to detect
enjoyment, the accuracy of the results hovered around 60%. One
of the reason behind this low accuracy might be that participants
sometimes have difficulties deciding whether they like a task or
not, but it is easier for them to decide which of two tasks they
prefer. Also, further testing can include analysis of questionnaire
responses in conjunction with EEG data to see how user
perceptions correlate with their brain activity. In particular, data
can be grouped based on demographic information; for instance,
further research can see how more experienced gamers might
show a different trend in engagement and preference than less
experienced gamers.
Future work includes expanding the usability of the Brain-EE
system for other purposes. For example, the Brain-EE system can
be utilized in usability testing to see how participants may be
more engaged with different web designs, therapy games, movies,
or advertisements. Finally, we plan to provide the data and the
Brain-EE system as an open-source dataset to be used and
evaluated by other researchers.
6.!CONCLUSION
Currently, the Brain-EE system is able to measure EEG activity
that reflects how much a user is engaged in a given task, which
then allows for the comparison of which out of two given tasks
are preferred by the user. The Brain-EE system supported much
of the previous literature on enjoyment; increases in frontal theta
activity strongly and reliably predicted which game each
individual participant preferred. The Brain-EE system can be
adapted to measure user engagement and enjoyment for other
stimuli (e.g., usability testing for different products), or to
measure other mood states (e.g., stress). It is the Brain-EE team’s
aim to refine our system with the hopes of increasing adaptability
for a wide array of future uses, thereby giving more objective and
concrete methods of measuring subjective phenomena.
7.!ACKNOWLEDGMENTS
This work is supported in part by the National Science Foundation
under award number 1041637. Any opinions, findings, and
conclusions or recommendations expressed in this publication are
those of the authors and do not necessarily reflect the views of the
National Science Foundation.
8.!REFERENCES
[1] Alonso, J.F., Romero, S., Ballester, M.R., Antonijoan,
R.M. and Mañanas, M.A. 2015. Stress assessment based
on EEG univariate features and functional connectivity
measures. Physiological Measurement. 36, 7 (2015),
135165.
[2] Campbell, I.G. EEG Recording and Analysis for Sleep
Research.
[3] Cassani, R., Banville, H. and Falk, T.H. 2015. MuLES#:
An Open Source EEG Acquisition and Streaming Server
for Quick and Simple Prototyping and Recording.
(2015), 912.
[4] Crowley, K., Sliney, A., Pitt, I. and Murphy, D. 2010.
Evaluating a brain-computer interface to categorise
human emotional response. Proceedings - 10th IEEE
International Conference on Advanced Learning
Technologies, ICALT 2010. (2010), 276278.
[5] Galway, L., Mccullagh, P., Lightbody, G., Brennan, C.
and Trainor, D. 2015. The Potential of the Brain-
Computer Interface for Learning#: A Technology Review.
14th International Conference on Computer and
Information Technology; Ubiquitous Computing and
Communications. (2015), 16.
[6] Greenberg, J.A., Burke, J.F., Haque, R., Kahana, M.J.
and Zaghloul, K.A. 2015. Decreases in theta and
increases in high frequency activity underlie associative
memory encoding. NeuroImage. 114, (2015), 257263.
[7] Hurless, N., Mekic, A., Peña, S., Humphries, E., Gentry,
H. and Nichols, D.F. 2013. Music genre preference and
tempo alter alpha and beta waves in human non-
musicians. Impulse: The Premier Undergraduate
Neuroscience Journal. (2013), 111.
[8] Karydis, T., Aguiar, F., Foster, S.L. and Mershin, A.
2015. Performance Characterization of Self-calibrating
Protocols for Wearable EEG Applications. Proceedings
of the 8th ACM International Conference on PErvasive
Technologies Related to Assistive Environments.
February 2016 (2015), 38:138:7.
[9] Lee, Y.Y. and Hsieh, S. 2014. Classifying different
emotional states by means of eegbased functional
connectivity patterns. PLoS ONE. 9, 4 (2014).
[10] Li, G., Lee, B. and Chung, W. 2015. Smartwatch-Based
Wearable EEG System for Driver Drowsiness Detection.
15, 12 (2015), 71697180.
[11] Li, Z., Xu, J. and Zhu, T. Prediction of Brain States of
Concentration and Relaxation in Real Time with Portable
Electroencephalographs. 118.
[12] Luijcks, R., Hermens, H.J., Bodar, L., Vossen, C.J., Van
Os, J. and Lousberg, R. 2014. Experimentally induced
stress validated by EMG activity. PloS one. 9, 4 (2014),
e95215.
[13] Luijcks, R., Vossen, C.J., Hermens, H.J., van Os, J. and
Lousberg, R. 2015. The Influence of Perceived Stress on
Cortical Reactivity: A Proof-Of-Principle Study. Plos
One. 10, 6 (2015), e0129220.
[14] Mann, S., Janzen, R., Wu, H., Lu, M.H. and Guleria, N.
2015. Bright ideas: A wearable interactive
“Inventometer” (brainwave-based idea display). 2015
IEEE Games Entertainment Media Conference (GEM).
(2015), 18.
[15] Necomimi - Brainwave Cat Ears:
http://www.necomimi.com/.
[16] Picard, R.W. 1999. Affective Computing for HCI. Procs.
8th HCI International on Human-Computer Interaction:
Ergonomics and User Interfaces. (1999), 829833.
[17] Plotnikov, A., Stakheika, N., De Gloria, A., Schatten, C.,
Bellotti, F., Berta, R., Fiorini, C. and Ansovini, F. 2012.
Exploiting real-time EEG analysis for assessing flow in
games. Proceedings of the 12th IEEE International
Conference on Advanced Learning Technologies, ICALT
2012. (2012), 688689.
[18] Rabiller, G., He, J.W., Nishijima, Y., Wong, A. and Liu,
J. 2015. Perturbation of brain oscillations after ischemic
stroke: A potential biomarker for post-stroke function
and therapy. International Journal of Molecular
Sciences. 16, 10 (2015), 2560525640.
[19] Safi-Harb, M., Proulx, S., Von Ellenrieder, N. and
Gotman, J. 2015. Advantages and disadvantages of a fast
fMRI sequence in the context of EEGfMRI
investigation of epilepsy patients: A realistic simulation
study. NeuroImage. 119, (2015), 2032.
[20] Schmidtke, K. 2010. Tower of Hanoi Problem. Corsini
Encyclopedia of Psychology. (2010).
[21] Stroop, J.R. 1935. Studies of interference in serial verbal
reactions. Journal of Experimental Psychology. 18, 6
(1935), 643662.
... The study achieves high classification accuracy with a reduced feature set using classifiers such as Bayesian Networks, Support Vector Machines, and Random Forests. On the other hand, [6], explore EEG data in everyday scenarios to evaluate user engagement and enjoyment in tablet-based video games. They find that frontal theta activity is a strong predictor of game preference. ...
... Every section in the table addresses an alternate report or examination, exhibiting the presentation of different characterization calculations. [6], achieved an accuracy of 87.62%, demonstrating the model's proficiency in pattern extraction; and a hybrid model combining RF and Deep Neural Network, [7], achieved accuracies of up to 97.89% for RF and 94.89% for DNN. These comparative insights demonstrate the advanced performance and potential of our chosen models, particularly RF. ...
Article
Full-text available
This article delves into using machine learning algorithms for emotion classification via EEG brain signals. The goal is to discover an accurate model beyond traditional methods, necessitating AI for classifying emotional EEG signals. This study, motivated by the complex link between emotions and neural activity, employs Random Forest, Support Vector Machines, and K-Nearest Neighbors. Notably, Random Forest achieves 99% accuracy, SVM 98%, and KNN 94%. These impressive results, backed by performance metrics like confusion matrices, reveal each model’s effectiveness in emotion classification. The dataset, rich in varied emotional stimuli and EEG placements, provides a robust foundation for detailed analysis. This research underscores significant applications in affective computing and mental health, offering a promising path to understanding the intricate relationship between EEG signals and human emotions.
... A Muse 2 brain sensing headband [1] (Interaxon Inc.) was utilized for EEG measurements in each lighting condition for each temperature-specific session, for a total of nine recordings for each subject. As already mentioned, the EEG data were recorded from four channels, strategically positioned based on the main hypothesized brain regions involved in processing temperature and light variations. ...
Article
Full-text available
Understanding the neural responses to indoor characteristics like temperature and light is crucial for comprehending how the physical environment influences the human brain. Our study introduces an innovative approach using entropy analysis, specifically, approximate entropy (ApEn), applied to electroencephalographic (EEG) signals to investigate neural responses to temperature and light variations in indoor environments. By strategically placing electrodes over specific brain regions linked to temperature and light processing, we show how ApEn can be influenced by indoor factors. We also integrate heart indices from a multi-sensor bracelet to create a machine learning classifier for temperature conditions. Results showed that in anterior frontal and temporoparietal areas, neutral temperature conditions yield higher ApEn values. The anterior frontal area showed a trend of gradually decreasing ApEn values from neutral to warm conditions, with cold being in an intermediate position. There was a significant interaction between light and site factors, only evident in the temporoparietal region. Here, the neutral light condition had higher ApEn values compared to blue and red light conditions. Positive correlations between anterior frontal ApEn and thermal comfort scores suggest a link between entropy and perceived thermal comfort. Our quadratic SVM classifier, incorporating entropy and heart features, demonstrates strong performance (until 90% in terms of AUC, accuracy, sensitivity, and specificity) in classifying temperature sensations. This study offers insights into neural responses to indoor factors and presents a novel approach for temperature classification using EEG entropy and heart features.
... The present study, however, did not find alpha power, or any other EEG metric, to be a significant predictor of subjective cognitive load. This contradicts the current theory of the existance of neural correlates of cognitive load; in regards to the trend of the results however, an increase in attention, and thus alpha power, may have been observed due to enjoyment of the task, though evidence for this in the literature is mixed [62], [63]. ...
Preprint
Understanding the interaction between cognitive load and features of video game play is important for the accurate measurement and application of this psychological construct in both research and industry scenarios to enhance the player experience. A challenge within this domain is the use of measurements developed in different areas in video games without first validating or testing the reliability of these tools. We present a holistic evaluation of methods used to measure cognitive load during naturalistic gameplay of a commercially available sandbox game. This study measures electroencephalography, electromyography, heart rate, heart rate variability, electrodermal activity, and eye blink rate from players during a building task in Minecraft whilst being assisted by a virtual companion. Various models reveal that a number of physiological measures can be used as a proxy of subjective cognitive load measurement, providing further insight into the player experience during gameplay. Our analysis also reveals some discriminant validity of the subjective measurement used. These results can help inform the choice of sensor in evaluating video game features, or when needing to accurately measure cognitive load for an adaptive situation or high-risk scenario.
... The data quality of the Muse 2 has been validated in research to collect EEG in various conditions, including frontal alpha analysis (Cannard et al., 2021) and event-related potentials (Krigolson et al., 2017(Krigolson et al., , 2021. Muse EEG headsets have even been used to diagnose stroke (Wilkinson et al., 2020) and characterize population dynamics (Hashemi et al., 2016), as well as to evaluate video game enjoyment (Abujelala et al., 2016), among other applications, serving to broadly validate the instrument for neuroscience research. Nevertheless, we acknowledge that some studies highlight differences in results from medical-grade EEG equipment as regards resting state analysis (Ratti et al., 2017) and ambulatory measurements (Przegalinska et al., 2017). ...
Article
Full-text available
Virtual reality (VR) has been used to induce out-of-body experience (OBE), but the construct validity of prior methods has been criticized. The current research evaluated a new VR paradigm to induce OBE that included implementation of a brain–computer interface (BCI), affording a means of neuromodulation. Thirty volunteers were tested in a VR paradigm that compared subjective report and electroencephalography (EEG) oscillations at frontal (AF7 and AF8) and right temporoparietal (CP6) regions to two personalized 360° videos, one of which simulated an in-body experience (IBE) and the other an OBE. Real-time analyses of frontal EEG alpha power determined the speed, brightness, and color saturation of the videos, thus providing neurofeedback as a means for invoking a sense of mental agency over the perceived movement of the spatial location of consciousness without bodily movement. As compared with response to the IBE in VR, participants reported that they had a greater sense of being in an elevated position above and outside of their bodies, had an experience of looking down at themselves from above, had experiences of floating and having an invisible body, and felt greater awe during the OBE. Participants also reported that increasing EEG alpha power was not easy, implying variability in perceived control or agency during the OBE. Theta oscillations were lower during the OBE than the IBE, while the difference in alpha and beta oscillations across electrodes varied between the OBE and IBE. The current research provides proof of concept of a promising new paradigm that combines VR and neuromodulation in the induction of OBE (VR-NIOBE). Further psychophysiological research and dismantling studies of the paradigm are warranted.
... EEG data are currently collected using wearable devices such as the Muse EEG headband (Saganowski et al., 2020) and Cognionics (Casciola et al., 2021), which makes it easier for researchers to get a brain response to the emotions stimulated by the material (e.g., video, figure) shown. Wearable EEGs are considered low-cost, non-invasive devices for monitoring brain activity during daily activities (Abujelala et al., 2016;Garcia-Moreno et al., 2020). As a result, the budget required is relatively small compared to the use of complex medical EEGs, and the user does not experience any restrictions on their freedom of movement. ...
Article
Full-text available
There is a lot of research being done on detecting human emotions. Emotion detection models are developed based on physiological data. With the development of low-cost wearable devices that measure human physiological data such as brain activity, heart rate, and skin conductivity, this research can be conducted in developing countries like Southeast Asia. However, as far as the author's research is concerned, a literature review has yet to be found on how this research on emotion detection was carried out in Southeast Asia. Therefore, this study aimed to conduct a systematic review of emotion detection research in Southeast Asia, focusing on the selection of physiological data, classification methods, and how the experiment was conducted according to the number of participants and duration. Using PRISMA guidelines, 22 SCOPUS-indexed journal articles and proceedings were reviewed. The review found that physiological data were dominated by brain activity data with the Muse Headband, followed by heart rate and skin conductivity collected with various wristbands, from around 5-31 participants, for 8 minutes to 7 weeks. Classification analysis applies machine learning, deep learning, and traditional statistics. The experiments were conducted primarily in sitting and standing positions, conditioned environments (for developing research), and unconditioned environments (applied research). This review concluded that future research opportunities exist regarding other data types, data labeling methods, and broader applications. These reviews will contribute to the enrichment of ideas and the development of emotion recognition research in Southeast Asian countries in the future.
... EEG signals are frequently used in biomedical engineering and neurological science research due to their non-invasiveness and low cost. They have previously been utilized for the categorization of emotions [29], stress detection [19], body movement detection [38], and assessment purposes [4]. In the recent past, several EEG-based approaches have been proposed for detecting cognitive fatigue. ...
... Researchers provided evidence that Muse is an effective portable tool for continuous recording EEG data [39,40], applicable also outside of its designed functionality (meditation and training device). In the field of thermal comfort evaluation, the authors validated the usage of this portable device for EEG measurements, assessing its capabilities to discriminate different human thermal sensation and its low invasiveness for the participants, during experimental session [41,42]. ...
Article
Full-text available
This paper presents a methodology for the application of electroencephalographic (EEG) Entropy measurements for indoor thermal comfort estimation. Wearables have been demonstrated to be capable of providing accurate physiological measurements to interpret individual thermal responses. Several studies demonstrated the correlation between the EEG Power Spectrum Density (PSD) variation and the subjects' responses exposed to different ambient temperatures. We present a complementary approach based on Approximate Entropy (ApEn) of EEG as a measure for the predictability of EEG series in describing the human thermal condition. We analysed the ApEn of EEG signals acquired from 24 subjects, exposed to three different temperatures (cold: 16°C; neutral: 25°C; warm: 33°C) in a controlled environment, by 4-channels wearable EEG sensors (256 Hz sampling frequency). Statistical analysis showed for both anterior frontal and temporoparietal sites significant differences between neutral, cold, and warm conditions, with a higher value of ApEn in the neutral one. In the anterior frontal area, there was a significative trend of ApEn with smaller values from the neutral to the warm condition, with the cold intermediate. The outcome opens the scenario up to innovative measurement systems, based on wearable EEG devices, for the application of personal comfort models to indoor environmental monitoring and control.
Chapter
Emotion recognition from EEG signals is a major field of research in cognitive computing. The major challenges involved in the task are extracting meaningful features from the signals and building an accurate model. Artificial intelligence (AI) algorithms are used by specialists in particular domains to generate decision-support systems. In recent times, researchers studying brain–computer interfaces (BCIs) have employed deep learning (DL) algorithms to interpret and analyze neural data. It has additionally revealed a possible problem with low transparency because of the complex structure of the algorithms. To improve the interpretability of AI algorithms and their decision-making procedures, explainable artificial intelligence (XAI) presents itself as a viable solution. This study emphasizes the application of deep learning models LSTM (Long Short-Term Memory) and RNN (Recurrent Neural Network) for prediction, while explaining the model prediction outcome and analyzing the feature importance for each feature through different XAI methods. Specifically, the LIME (Local Interpretable Model Agnostic Explanation), SHAP (Shapley Additive exPlanations), ELI5 (Explain Like I’m 5), and Partial Dependence Plot (PDP) techniques are utilized to describe the fundamental features that contribute to the classification results. The efficacy of the aforementioned XAI methods is validated on the publicly available EEG Brainwave Dataset: Feeling Emotions. The overall objective of this research is to strengthen trust and interpret the challenging neural network models in the real-world scenarios by bridging the gap between human interpretability and different ML and DL algorithms.
Conference Paper
Full-text available
The light-bulb idea metaphor (a light bulb above someone's head that appears or switches on when they think of an idea) is widespread in the literature and popular culture. But, to the best of our knowledge, nobody has ever built an actual device that implements this function. Therefore we present the "Inventometer", a fun and playful wearable device that measures and displays epiphanometric data to a real light bulb, so its wearer and others in the environment are alerted to "aha! moments" ("eureka moments"), epiphanies, inventions, and idea formation in the brain/mind of the wearer. It senses brainwave signals indicative of epiphanies, and indicates to others a continuously varying epiphanometric quantity by adjusting the light output of the bulb. It uses simple machine learning on EEG (electroencephalogaph) brainwave signals to automatically detect and quantify the novelty of ideas formed in the brain. Long exposure photographs — made while one or more Inventometer wearers walk around a room — result in a Phenomenal Augmented Reality (Augmented Reality of or pertaining to physical phenomena and phenomenology). In particular, these "epiphanographs", over time, indicate what sorts of things in an environment tended to stimulate epiphanies and to what degree — epiphanogrammetry as a possible new field of study. Brain Games are designed so multiple people can compete using their minds much like people competing using their bodies (e.g. arm wrestling). The "brightest" people in a room become visible by way of their epiphanographs. But with their wearable display bulbs glowing brightly, they also serve as a distraction to each other, thus introducing a richly complex and competitive collective biofeedback gaming space. The device is meant to appeal to people of all ages, from children to university students, and thus forms a good teaching for making skills, science, and engineering (e.g. power electronics, 3D printing, etc.). It is also an ongoing research project.
Article
Full-text available
Therapeutic Neurofeedback (NFB) using real-time electroencephalography (EEG) data works by reinforcing desired brainwave patterns. Although EEG is a well-established diagnostic tool and EEG-NFB shows great promise for enhancing cognitive performance and treating neurological disorders, proof of its efficacy has been limited. Here we characterize a novel Self-Calibrating Protocol (SCP) method coupled to five standard machine learning algorithms to classify brain states corresponding to the experience of "pain" or "no pain". Our results indicate that commercially available, wearable EEG sensors provide sufficient data fidelity to robustly differentiate the two "perceptually opposite" brain states. Crucially, use of SCP allows us for the first time to bypass the pitfalls associated with trying to force an individual's brain wave patterns to match "normed" target patterns obtained over population averages. These are necessary steps towards personalized NFB therapies and bespoke Brain-Computer Interfaces and brain training suitable to a wide variety of individual needs.
Article
Full-text available
Brain waves resonate from the generators of electrical current and propagate across brain regions with oscillation frequencies ranging from 0.05 to 500 Hz. The commonly observed oscillatory waves recorded by an electroencephalogram (EEG) in normal adult humans can be grouped into five main categories according to the frequency and amplitude, namely δ (1-4 Hz, 20-200 μV), θ (4-8 Hz, 10 μV), α (8-12 Hz, 20-200 μV), β (12-30 Hz, 5-10 μV), and γ (30-80 Hz, low amplitude). Emerging evidence from experimental and human studies suggests that groups of function and behavior seem to be specifically associated with the presence of each oscillation band, although the complex relationship between oscillation frequency and function, as well as the interaction between brain oscillations, are far from clear. Changes of brain oscillation patterns have long been implicated in the diseases of the central nervous system including ischemic stroke, in which the reduction of cerebral blood flow as well as the progression of tissue damage have direct spatiotemporal effects on the power of several oscillatory bands and their interactions. This review summarizes the current knowledge in behavior and function associated with each brain oscillation, and also in the specific changes in brain electrical activities that correspond to the molecular events and functional alterations observed after experimental and human stroke. We provide the basis of the generations of brain oscillations and potential cellular and molecular mechanisms underlying stroke-induced perturbation. We will also discuss the implications of using brain oscillation patterns as biomarkers for the prediction of stroke outcome and therapeutic efficacy.
Article
Full-text available
Driver drowsiness is a major cause of mortality in traffic accidents worldwide. Many physiological signals have been proposed to detect driver drowsiness. Among these signals, an electroencephalographic (EEG) signal, which reflects the brain activities, is more directly related to drowsiness. Thus, many EEG-based driver drowsiness detection (DDD) models gained more and more attention in recent years. However, one limitation of these studies is that these models merely estimate discrete labels and, thus, did not allow for estimating the relative severity of driver drowsiness. This paper proposes a support vector machine-based posterior probabilistic model (SVMPPM) for DDD, aimed at transforming the drowsiness level to any value of instead of discrete labels. A fully wearable EEG system which consists of a Bluetooth-enabled EEG headband and a commercial smartwatch was used to evaluate the proposed model in a real-time way. Twenty subjects who participated in a 1-h monotonous driving simulation experiment were used to develop this model with fifteen subjects for a building model and five subjects for a testing model. According to a video-based reference, the proposed system obtained an accuracy of 91.25% for an alert group (73 out of 80 data sets), 83.78% for an early-warning group (93 out of 111 data sets), and 91.92% for a full-warning group (91 out of 99 data sets). These results indicate that the combination of the proposed SVMPPM, the EEG headband, and the wrist-worn smart device constitutes an effective, simple, and inexpensive wearable solution for DDD.
Article
Full-text available
In our research, we attempt to help people recognize their brain state of concentration or relaxation more conveniently and in real time. Considering the inconvenience of wearing traditional multiple electrode electroencephalographs, we choose Muse to collect data which is a portable headband launched lately with a number of useful functions and channels and it is much easier for the public to use. Besides, traditional online analysis did not focus on the synchronism between users and computers and the time delay problem did exist. To solve the problem, by building the Analytic Hierarchy Model, we choose the two gamma wave channels of F7 and F8 as the data source instead of using both beta and alpha channels traditionally.Using the Common Space Pattern algorithm and the Support Vector Machine model, the channels we choose have a higher recognition accuracy rate and smaller amount of data to be dealt with by the computer than the traditional ones. Furthermore, we make use of the Feedforward Neural Network Model to predict subjects'brain states in half a second. Finally, we design a plane program in Python where a plane can be controlled to go up or down when users concentrate or relax. The SVM model and the Feedforward Neural Network model have both been tested by 12 subjects and they give an evaluation ranging from 1 to 10 points. The former gets 7.58 points while the latter gets 8.83, which proves that the time delay problem is improved once more.
Article
Full-text available
The aim of this study was to investigate how perceived stress may affect electroencephalographical (EEG) activity in a stress paradigm in a sample of 76 healthy participants. EEG activity was analyzed using multilevel modeling, allowing estimation of nested effects (EEG time segments within subjects). The stress paradigm consisted of a 3-minute pre-stimulus stress period and a 2-minute post-stimulus phase. At t=3 minutes, a single electrical stimulus was delivered. Participants were unaware of the precise moment of stimulus delivery and its intensity level. In the EEG time course of alpha activity, a stronger increase was observed during the post-stimulus period as compared to the pre-stimulus period. An opposite time course effect was apparent for gamma activity. Both effects were in line with a priori expectations and support the validity of this experimental EEG-stress paradigm. Secondly, we investigated whether interaction effects of stress and coping, as measured with the Perceived Stress Scale-10 questionnaire (PSS-10), could be demonstrated. A higher perceived stress score was accompanied by a greater increase in delta- and theta-activity during the post-stimulus phase, compared to low scores. In contrast, low coping capacity was associated with a stronger decrease in slow beta, fast beta and gamma activity during the post-stimulus phase. The results of the present article may be interpreted as proof-of-principle that EEG stress-related activity depends on the level of subjectively reported perceived stress. The inclusion of psychosocial variables measuring coping styles as well as stress-related personality aspects permits further examination of the interconnection between mind and body and may inform on the process of transformation from acute to chronic stress.
Conference Paper
Recent advances in the tracking and quantification of pain using consumer-grade wearable EEG headbands, such as Muse [8] and Neurosky [12], coupled to efficient machine learning [9], pave the way towards applying Self-Calibrating Protocols (SCP) [10] and Dynamic Background Reduction (DBR) [11] principles to basic research, while empowering new applications. In the cases of neurological conditions and chronic pain management, SCP is of particular interest during the early diagnostic process as well as an aid in personalizing intervention strategies. In this paper, we outline a framework based on SCP, to design machine learning systems that completely bypass the pitfalls of using normed neurophysiological states for diagnostics. This effort targets short-term practical development of personalized early diagnostics and treatment strategies and has longer-term implications for Brain-Computer Interface (BCI) and Human-Computer Interaction (HCI) methodologies.
Conference Paper
The past few years have seen the availability of consumer electroencephalography (EEG) devices increase significantly. These devices, usually with a compact form factor and affordable price, now allow researchers and enthusiasts to use EEG in various new contexts and environments. However, the many consumer headsets often require extensive programming experience to interface with their respective drivers; moreover, standardization of the access and recording of EEG data across the devices still remains to be done. Consequently, a tool is needed to facilitate the recording and streaming of EEG data from consumer headsets that can easily be interfaced with different programming languages and software, and that allows interchangeability between devices. This paper describes the open source MuSAE Lab EEG Server (MuLES), an EEG acquisition and streaming server that aims at creating a standard interface for portable EEG headsets, in order to accelerate the development of brain-computer interfaces (BCIs) and of general EEG use in novel contexts. In addition to the EEG server interface which currently supports five different consumer devices and session playback, clients are developed for use on different platforms and in various programming languages, making prototyping and recording a quick and simple task. To validate the functionality and usability of the EEG server, a use case highlighting its main features is presented. The developed tool simplifies the acquisition and recording of EEG data from portable consumer devices by providing a single efficient interface, with applications in areas such as basic and behavioural research, prototyping, neurogaming, live performance and arts.
Article
EEG-fMRI is an established technique to allow mapping BOLD changes in response to interictal discharges recorded in the EEG of epilepsy patients. Traditional fMRI experiments rely on an echo planar imaging (EPI) sequence with a time resolution given by its time-to-repetition (TR) of ~2s. Recently, multiple fast fMRI sequences have been developed to get around the limited temporal resolution of the EPI sequence, and achieved a TR in the 100 ms range or lower. One such sequence is called Magnetic Resonance EncephaloGraphy (MREG). Its high temporal resolution should offer increased detection sensitivity and statistical power in the context of epilepsy studies and in fMRI experiments in general. The aim of this work was to investigate the advantages and disadvantages offered by MREG. This was done by superimposing artificial event-related BOLD responses on EPI and MREG background signals, from 5 epileptic patients, that were free of epileptic discharges (spikes) on simultaneously recorded EEG. These functional datasets simulated different spiking rates and hemodynamic response amplitudes, and were analysed with the commonly used General Linear Model (GLM) with the canonical hemodynamic response function (HRF) as a fixed model of the response. Robustness to violation of the assumptions of the GLM was additionally assessed with similar simulations using variable spike-to-spike response amplitudes and 8 non-canonical HRFs. Consistent with previous work, MREG yields higher maximum statistical t-values than EPI, but our simulations showed these statistics to be inflated, as the false positive rate at a standard threshold was high. At thresholds set to appropriately control specificity, EPI showed better true positive rate and larger cluster size than MREG. However, the lack of an appropriate calibration of the amplitude of the responses across the sequences precludes definitive judgment on their relative sensitivity. In addition, we show that a mismatch between the assumed and actual HRF impairs more MREG detection performance, but that EPI is more affected by non-modeled spike-to-spike variations of response amplitude. Filtering-out physiological noise, which is not aliased at the fast sampling rate of MREG, and the modeling of temporal autocorrelation are advantageous in increasing the detection power of MREG. This simulation study 1) warrants care when interpreting statistical t-values from fast fMRI sequences, 2) proposes thresholds for valid inferences and processing methods for maximal sensitivities, and 3) demonstrates the relative robustness/susceptibility of MREG and EPI to violation of the GLM's assumptions. Copyright © 2015. Published by Elsevier Inc.