Available via license: CC BY 4.0
Content may be subject to copyright.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
1
Gaming Beyond the Novelty-Effect of Immersive
Virtual Reality for Physical Rehabilitation
Aviv Elor, Michael Powell, Evanjelin Mahmoodi, Mircea Teodorescu, Sri Kurniawan
University of California - Santa Cruz
Jack Baskin School of Engineering
Santa Cruz, California, United States
{aelor,mopowell,emahmoodi,mteodore,skurnia}@ucsc.edu
Abstract—Immersive Virtual Reality (iVR) Head-Mounted Dis-
play (HMD) systems paired with serious exercise games can
positively augment physical rehabilitation process from both
engagement and analytics perspectives. This paper presents a
serious game for iVR HMD based long term upper-extremity
exercise. We demonstrate the capabilities of our game through
a case study with five users recovering from upper-extremity in-
juries. We examine how our program maintains engagement and
motivation over eight weeks, where users completed bi-weekly
prescribed movements framed as protecting a virtual butterfly.
We assess user experiences through a mixture of biomarkers
from brainwave, heart rate, and galvanic skin response recorded
at runtime as well as motion capture and behavioral game data.
Our results suggest that the iVR game was an effective medium in
inducing high compliance, physical performance, and biometric
changes even with increasing difficulty beyond the novelty effect
period. We conclude with considerations of future work for iVR
physical therapy games that adapt to biometric response.
Index Terms—Serious Games, Immersive Virtual Reality, Phys-
ical Rehabilitation, User Studies, Head-Mounted Display, Games
for Health
I. INT ROD UC TI ON
With the mass commercial adoption of immersive Virtual
Reality (iVR) based Head-Mounted Display systems (HMDs)
and over 200 million headsets sold since 2016, iVR systems
have almost become a common household gadget [1]. On a
parallel track, rehabilitation research, including physical and
cognitive work that incorporates VR based interventions, has
been on the rise due to the ability to create programmable
immersive experiences that can directly influence human be-
havior [2]–[4]. The ability to run conventional therapy in a
virtual environment can be paired with high-fidelity motion
capture, telepresence capabilities, and accessible experiences
[5], [6]. Through serious games, immersive environments with
commercial HMDs can be programmed to translate therapeutic
goals into game mechanics, making the therapies more enjoy-
able [7]–[12].
It has been shown that iVR can be successfully used for
treating Post Traumatic Stress Disorder [13], Borderline Per-
sonality Disorder [14], various phobias [4], [15], schizophrenia
[16], and others. The detachment from reality and immersion
in a virtual world can reduce discomfort, even as far as
minimizing pain when compared to clinical analgesic treat-
ments [17]. Strong immersive stimuli through a VR system
together with the ability to combine presence and emotion
in a virtual world is key to influencing user behavior [13].
Fig. 1. A user plays Project Butterfly during their bi-weekly exercise session:
a) The user physically matches the butterfly’s position to protect it while also
catching crystals to infer the motion path. b) The user’s view of the game
through the Vive HMD. Score is visible on the user’s bubble shield, with time
and repetitions visible for the evaluator interface. c) The scene view of the
user in unity. The bottom half of the figure depicts sensor peripherals utilized
during Project Butterfly user testing.
However, quantifying this success is often difficult due to
system constraints and a lack of computational power [18].
Similarly to iVR, research in biometric sensing has seen
explosive growth over the past decade. We argue that biofeed-
back may serve as a versatile tool to quantify the success of an
iVR based physical therapy experience. Brain-Computer Inter-
faces incorporating Electroencephalogram (EEG) devices have
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
2
become more affordable and user-friendly – they come with
computational techniques for understanding user engagement
and intent in the fields of medical, entertainment, education,
gaming, and more [19]. The analysis of different brainwave
frequencies has been correlated to different psychological
functions, such as the 8-13 Hz Alpha band relating to stress
[20], the 13-32 Hz Beta band relating to focus [21], [22], the
0.5-4 Hz Delta band relating to awareness [23], [24], the 4-8
Hz Theta band relating to sensorimotor processing [25], [26],
and the Gamma band of 32-100 Hz related to cognition [27],
[28].
Researchers are combining these interfaces with other forms
of multi-modal biometric data collection such as Galvanic Skin
Response (GSR) and Heart Rate (HR) to increase affective
inference. Through measuring the changes in skin resistance,
GSR has been linked to emotional arousal [29]. Quantifying
the intensity of GSR may enable researchers to record emo-
tional response with stimuli such as music, television, and
gaming [30], [31]. Additionally, it has been shown that the
accuracy of the non-invasive prediction of the psychological
response can be increased by using data fusion algorithms
that integrate GSR, HR and EEG readings [32]–[35]. These
biometric measures paired with iVR therapy may yield much
potential for understanding physical exercise experiences, but
are not often incorporated in the traditional physical therapy
regimen.
Traditional outpatient physical therapy usually involves clin-
ical visits where patients perform exercises, receive evalua-
tions, and undergo in-person manual therapy interventions.
Better clinical outcomes are often associated with a well estab-
lished schedule of at-home exercises following the clinical vis-
its [36]. Many meta-analyses and reviews have suggested that
virtual reality-based rehabilitation techniques, some with bio-
metric sensing [37]–[39], could outperform traditional physical
therapy [40]–[42]. Some of these studies have shown that
iVR therapy can be effective for weeks or months [43]–[45].
However, although there is mounting evidence that VR HMD-
based rehabilitation systems could benefit from integration
with with biometric, physical, and user-reported analysis, there
is a lack of studies with extended period investigating this
phenomenon.
The study reported in this paper aims to answer the question
of: Can an iVR HMD experience maintain engagement beyond
the novelty period and show continued rehabilitative improve-
ment using multi-modal analysis when used as a physical ther-
apy environment? To answer this question, we expand upon
an iVR serious game for controlled physical exercise. The
purpose of the updated design is to investigate improvements
in physical performance using an iVR system by following
protocols that are similar to conventional physical rehabilita-
tion. Three outpatient Doctoral Physical Therapists, with over
40 years of combined professional experience, helped design
the protocols used in this study to match exercises used in
clinical settings. Through these three consultants, we learned
that the principles of functional shoulder rehabilitation for late-
phase recovery usually extend from 6 to 12 weeks of treatment
to “(1) restore full range of motion and flexibility... and (2)
increase strength, power, and endurance with exercises that
stress core-based muscle synergy” [46], [47]. In this study,
we extend these principles to stimulate range of motion in the
first four weeks and increase strength in the last four weeks.
Specifically, the contributions of this study are:
1) A demonstration that our iVR HMD based serious game
system can be effective for physical rehabilitation.
2) An examination of methods towards maintaining en-
gagement and motivation over extended period of time.
3) An assessment of the feasibility of using biometrics to
complement the iVR game.
In reflecting on Rego et al.’s taxonomy of serious games for
rehabilitation, we present a two-month user study on a motor
game that utilizes motion-tracking for a head-mounted display
based 3D single-player action experience with adaptability,
progress monitoring, performance feedback, and clinic-to-
home portability [48]. We define serious games as games
whose purposes are beyond entertainment only [48]. We
define iVR-based serious games as immersive experiences that
incorporate 3D user interaction and motion capture, often with
an HMD. To the best of our knowledge, this study is one of the
first to leverage an immersive VR HMD based serious game
for an extended physical therapy period and that examines
multi-modal biometric feedback and physical performance.
II. SY ST EM DE SI GN
Our physical therapy system is based on a game called
“Project Butterfly” (PBF) that was proposed by Elor et al
[49], [50]. Previously, PBF explored the feasibility of a virtual
reality enhanced exo-skeleton for post-stroke and elderly as-
sistance through two exercises, but was not designed or tested
for upper extremity physical therapy over a extended period
of time with varying custom exercise movements as reported
in this study. The game was built using the Unity 2018.2.11f1
Game Engine with SteamVR and incorporates the HTC Vive
Pro 2018 (Vive). Vive uses outside-in tracking through a
constellation of “lighthouse” laser systems for pose collection
in a 3D 4x4m space [1]. It has been verified in previous studies
to analyze therapeutic gamification [51], [52], postural analysis
[53], and accuracy for research data collection [54].
To explore the effects of extended upper-limb rehabilitation,
we heavily modified PBF to create a new gaming experience.
These improvements were designed and suggested by the
collaborating therapists to make the game customizable and
aid best in the rehabilitation process. The updated goal of
the PBF is to safeguard a virtually flying butterfly from
adverse weather and flying projectiles using a translucent
protective “bubble shield” that is controlled by the player
through the Vive Hand Controller. The flying pattern of the
butterfly recreates the rehabilitation exercises, and the game
was modified to allow dynamic adjustment of movements
through therapist-led motion capture, enabling the dynamic
control of pace and position for the prescribed exercise.
Specifically, using the HTC Vive Controllers in PBF, therapists
can record motion paths by entering a custom unity scene
and performing exercises by holding the controller trigger
while performing the desired movement. Movement paths
are remotely uploaded into the unity environment through
comma-separated value format. The 3D motion vector path
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
3
is then previewed, confirmed, and named by the therapist,
which is then adjusted to user arm length during gameplay.
The user is required to follow the path of the butterfly with
a 0.1 meters error, where they are awarded a score point
(accompanied by audio and haptic feedback) for each half a
second they successfully protect the butterfly. This includes a
scoring system for the user to engage in self-competition.
Gameplay sessions were updated to provide feedback to
both the user and the therapist by collecting EEG, GSR, HR,
motion capture, and player behavior datum. Each exercise is
prescribed by our collaborating physical therapists to establish
range of motion in the first stage of rehabilitation and complex
movement in the last stage. The user starts following the
path by activating an animation with the controller placed on
the stationary butterfly for three seconds at the start of the
game. Collectible game projectiles were added in the shape of
crystals, and spawn from a distance while veering towards the
butterfly as a motion path indicator for the user to follow their
exercise. The gameplay of the new PBF version is shown in
Figure 1.
Additionally, the updated project applies concepts from
Self-Determination Theory (SDT [55]) to intrinsically moti-
vate users to become the guardian of a butterfly (relatedness),
gain mastery of controlling a virtual shield in following
the butterfly’s path (competence), and be in full control of
their upper extremity movement with progressive difficulty
(autonomy). Previous studies have examined motivation in
exergames through gamification (challenge and score) [56],
self-competition and virtual trainers [57], music [58], sensory
feedback [59], rhythmic clapping [60]. PBF differs from these
exergames in a gameplay mechanic aimed towards relatedness:
enacting a helping behavior on the user to protect the butterfly,
building a gradual sense of attachment and belonging to the
butterfly through becoming it’s guardian. Research has shown
that helping behavior can provide a variety of material, social,
and self-rewards such as mood enhancement to the helper
[61]. Subsequently, the care-helping relationship was found to
be strong mediator for stimulating planned, long-term helping
behaviors in user participation [62]. Through iVR, we extend
these concepts into an immersive environment. Given that
iVR therapeutic intervention’s success is often attributed to
the influence of immersion on users in terms of enhancing
the relationship between presence and emotion [4], [13], the
system proposed in this paper provides a mixture of visual,
audio, and haptic feedback for upper body movements helping
the user to overcome the adversity of physical task-based
objectives.
III. USE R STU DY
The methods used in the study reported in this paper
received Institutional Review Board’s approval from the Uni-
versity of California - Santa Cruz Office of Research Com-
pliance Administration. Five college students recovering from
upper limb injuries (including shoulder dislocation, shoulder
impingement, and rotator cuff tears) consented to volunteer.
These participants consisted of one female and four males
ranging from 21 to 28 years old. Additionally, the participants
were no longer performing in-person physical therapies, nor
continuing at-home therapy exercises, thereby mimicking our
target population. These users agreed to volunteer for two
weekly sessions for eight weeks, although, as the consent form
stated, they were free to drop out of the study at any time.
Participants were offered a $50 USD gift card upon completing
the study (or were paid about $3.12 USD per session). All
users completed all sessions of the study.
Fig. 2. Movements tested in the Foundation and Challenge protocols. The
dotted line shows the butterfly’s path, while the arrows indicate the axis of
rotation for relevant movements.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
4
A. Data Collection
Figure 1 shows the experimental setup. Through Vive and
the Unity Game Engine, motion capture and behavior game
data were recorded and stored during runtime at 90 Hz using
a data exportation method developed in previous studies by
Elor et al. [11], [49]. Motions were captured through utilizing
the HTC Vive Pro 2018’s outside-in tracking algorithm by
a constellation of “lighthouse” laser systems [1]. Runtime
recording of brainwave (EEG), GSR, and HR data was also
performed to complement the Vive data. EEG was sensed
from the pre-frontal cortex (TP9/10, AF1/7 locations) and
recorded using InteraXon Muse 2, a commercially available
brain sensing headband [63]. While Muse is a relatively new
EEG commercial device, it has successfully been used in
other studies to infer mental state, analyze event potentials,
and record biofeedback [64]–[66]. For GSR, we utilized the
Neulog GSR logger sensor NUL-217 [67] to measure the
skin’s conductivity between the fingers of the non-dominant
hand. HR was recorded through the LED optical sensor Polar
OH1 [68]. Every sensor was chosen with accessibility and
cost as a factor. These devices are easy to set up for a
user at home, with dry contact after wiping with saline to
reduce the costs of any EEG gels or sticktrodes for HR and
GSR that are usually required when using many clinical-grade
sensors. Sensor locations and equipment setup used for each
user testing session are seen in Figure 1.
At the end of each testing session, users were asked a series
of Likert scale questions regarding their rehabilitation session
experience with PBF. Survey questionnaires were inspired
from a Jennett et al. survey, which measures immersion
in games [69]. The survey was implemented to focus on
behavioral engagement and a self-reported emotional response
between multiple exercise sessions over many weeks. Reflect-
ing on Doherty and Doherty’s review of engagement in human-
computer interaction, we define engagement assessment as a
behavioral understanding of whether a user desires to and can
effectively use a system [70]. For understanding emotional
response, we utilized the circumplex model of affect to assess
self-reported emotions from users between exercise sessions
[71], [72]. The goals of these surveys were to understand
if users would self-perceive PBF as engaging, immersive,
and positive throughout the two months as the users become
acclimated to the game during rehabilitation. An exit interview
was also conducted at the end of the two months to better
understand participants’ subjective feedback.
B. Protocol
The study followed each of the five users during their pre-
scribed rehabilitative exercises through two protocol phases:
Foundation and Challenge. The Foundation Protocol instigated
basic motion primitives that emphasized recovery of Range of
Motion (ROM) and shoulder strength through three simple
exercises: Forward Arm Raise [FAR], Side Arm Raise [SAR],
and Shoulder Rotation [SR]. The goal of the Foundation Proto-
col was to acclimate the user into performing physical therapy
in iVR with minimal weight resistance to reduce the possibility
of injury as recommended by our collaborating therapists,
and see if the user could maintain compliance over an ex-
Game Performance
12345
Foundation Session
90
92
94
96
98
Compliance Rate [%]
Foundation Protocol
12345
Challenge Session
90
92
94
96
98
Compliance Rate [%]
Challenge Protocol
12345
Foundation Session
20
25
30
Weak Displacement [movement/m]
12345
Challenge Session
20
25
30
Weak Displacement [movement/m]
12345
Foundation Session
0
2
4
6
8
10
Weight Resistance [lbs]
12345
Challenge Session
0
2
4
6
8
10
Weight Resistance [lbs]
Fig. 3. Game performance between Foundation Protocol (in red of 225
recorded exercises) and Challenge Protocol (in green of 350 recorded exer-
cises). Row one shows compliance, where compliance is defined as the total
time protecting the butterfly over the game’s total time. Row two shows the
mean upper-limb displacement between all exercises required in that session.
Row three indicates the mean weight used between all exercises of that
session. Error bars indicate standard error (note the Foundation Protocol had
less variability between users, so error bars appear substantially smaller than
Challenge Protocol due to shared scale).
Physiological Response
1 2 3 4 5
Foundation Session
8
10
12
HR [bpm]
Foundation Protocol Change
1 2 3 4 5
Challenge Session
8
10
12
HR [bpm]
Challenge Protocol Change
1 2 3 4 5
Foundation Session
1
1.2
1.4
1.6
GSR [uS]
1 2 3 4 5
Challenge Session
1
1.2
1.4
1.6
GSR [uS]
Fig. 4. Physiological HR and GSR responses from gameplay are shown. Row
one illustrates mean change from resting state of heart rate. Row two illustrates
mean change from resting state of galvanic skin response. Biometric change
is calculated as the offset between gameplay biometrics against resting-state
biometrics. Error bars indicate standard error.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
5
tended period of iVR exposure. The Challenge Protocol added
four complex motions that further pushed ROM recovery
while including the original Foundation Protocol’s exercises
using increasingly higher weights. The goal of the Challenge
Protocol was to challenge the user with continuing weight
increases and complex movements to investigate the effects of
more complex exercises on compliance and engagement. The
complex movement exercises added were External Rotation
[EXR], Abducted Rotation [ABR], Mixed Press [MXDPR],
and Mixed Circles [MXDPC]. These movements and their
protocol inclusions can be seen in Figure 2.
Each protocol consisted of five sessions, excluding the
initial tutorial session. Two evaluators were always present
to monitor the user for irregular activity and record qualitative
observations. Exercises were played in circuit rotation and can
be seen in Figure 2. Weighted straps were used for the motions
FAR, SAR, and SR and the weights were gradually increased
throughout the study to match the participant’s capability.
The protocol for increasing weight was determined with our
collaborating therapists to minimize the chance of injury.
Participants usually started at 1-2 pounds and weights were
increased by 1-1.5 pounds after comfortably completing two
rounds of gameplay. The order of the movement was kept
consistent to monitor biometric results for examination of data
between sessions, as illustrated in Figure 2.
Users performed three sets of the exercises for the Foun-
dation Protocol and two sets of the seven exercises during
the Challenge Protocol. A break of approximately 90 seconds
was given between each exercise. The user then performed 60
seconds of the prescribed movements with biofeedback, game
data, and video recorded. The entire sessions lasted forty-
five minutes to an hour, including post-user surveys. For our
analysis, we examined the change from the resting baseline
to the gameplay data. A biometric baseline was recorded
at the beginning of each session where the user wore all
of the sensors shown in Figure 1 while sitting in a relaxed
state with the main screen of PBF displayed on the Vive.
The first session, dubbed session “zero”, of each protocol
phase consisted of a basic range of motion test and tutorial
gameplay where the participant performed all new exercises
of the protocol. By playing the games on this initial session, we
hoped to limit the novelty effect. It should be noted that data
was not collected on these tutorial sessions, as the protocol
was unique to each individual to answer questions and help
with acclimation to the iVR environment.
IV. RES ULTS
Results from session data were post-processed using the
Mathworks Matlab 2018b Statistics and Machine Learning
Toolbox [73]. We examined user performance between every
session for mean and standard error. In total, we collected
225 session exercises for the Foundation protocol and 350
session exercises for the Challenge protocol for every data
type. Biometric signals were normalized from each user’s
baseline resting state to examine the changes induced by
gameplay.
The game performance data shows general improvements in
weight resistance over time with maintenance of compliance
Neural Response
12345
Foundation Session
0.05
0.1
0.15
0.2
Stress [ bels]
Foundation Protocol Change
12345
Challenge Session
0.05
0.1
0.15
0.2
Stress [ bels]
Challenge Protocol Change
12345
Foundation Session
0.1
0.2
0.3
Focus [ bels]
12345
Challenge Session
0.1
0.2
0.3
Focus [ bels]
12345
Foundation Session
0.1
0.2
0.3
0.4
0.5
Awareness [ bels]
12345
Challenge Session
0.1
0.2
0.3
0.4
0.5
Awareness [ bels]
12345
Foundation Session
0
0.1
0.2
Motor [ bels]
12345
Challenge Session
0
0.1
0.2
Motor [ bels]
12345
Foundation Session
0.2
0.3
0.4
0.5
Cognition [ bels]
12345
Challenge Session
0.2
0.3
0.4
0.5
Cognition [ bels]
Fig. 5. EEG responses between Foundation Protocol (in red of 225 recorded
exercises) and Challenge Protocol (in green of 350 recorded exercises). Rows
1-5 show Alpha, Beta, Delta, Theta, and Gamma bands resting state change
respectively. Error bars indicate standard error.
Facial Movement Response
12345
Foundation Session
-0.6
-0.4
-0.2
0
0.2
Blinks [per s]
Foundation Protocol Change
12345
Challenge Session
-0.6
-0.4
-0.2
0
0.2
Blinks [per s]
Challenge Protocol Change
12345
Foundation Session
0
0.02
0.04
Jaw Clenches [per s]
12345
Challenge Session
0
0.02
0.04
Jaw Clenches [per s]
Fig. 6. Facial muscle movements recorded with Muse between Foundation
Protocol (in red of 225 recorded exercises) and Challenge Protocol (in green
of 350 recorded exercises). Row one shows the mean resting state change
of blinks per second. Row two shows the mean change of jaw clenches per
second from resting state.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
6
Engagement Survey Response
Fig. 7. Survey responses on engagement from 5 subjects, with 1=strongly disagree and 5=strongly agree.
and exercise movement, as shown in Figure 3. On average,
users were able to handle more weight resistance per exercise
than the initial session, and while the compliance remains
almost constant in the Foundation protocol, it increases signifi-
cantly in the Challenge protocol. In both protocols, users were
able to perform the same movements with a gradual weight
increase.
For physiological performance, PBF was able to record and
monitor elevated HR and GSR measurements when compared
to resting-state for all sessions of each protocol. Figure 4
shows the changes from resting baselines and indicates that
PBF always induced an elevated HR (indicating physical
engagement) and stimulated GSR by 1uS or higher (indicating
induced arousal). In the Foundation Protocol, users maintained
a constant level of increased physical activity with a slow de-
cline of arousal. In the Challenge Protocol, users had increased
intensity of physiological activity with a considerable decline
of arousal that eventually stabilized.
For brainwave response, neural activities were measures
at all sessions and all protocols, as shown in Figure 5. All
wavebands were found to be at a positive increase from
resting-state change which indicates that Alpha, Beta, Delta,
Theta, and Gamma waves were elevated during PBF usage. In
the Foundation Protocol, all brainwave responses from users
generally increased in the middle of the sessions and began
declining towards the last sessions. In the Challenge Protocol,
all brainwave activities had a more substantial initial session
than the Foundation Protocol and generally declined overtime
to nearly the same level as the Foundation Protocol’s last
session.
Additionally, Muse [63] holds the capability to detect facial
muscle movements to determine a Boolean response of eye
blinks and jaw clenches. This data was recorded during
runtime gameplay, and converted to facial movements per
second based on changes from the baseline, as seen in Figure
6. While playing PBF, users in the Foundation Protocol tended
to blink less than their resting state for every session (with
the exception of Session #3). Jaw clenches do not vary much
between sessions. In the Challenge Protocol, users tended
to blink and clench their jaw much more between every
session than their baseline resting state. Unlike the Foundation
Protocol, these blinks were always at a positive increase when
compared to resting state, except for the first session, and were
more rapid. Lastly, jaw clenches tended to decline as time
progressed between sessions.
For user’s self-reported responses, the qualitative survey
questions can be seen in Figure 7 (engagement based) and
Figure 8 (emotion based). For both protocols on each session,
Emotion Survey Responses
Fig. 8. The self reported emotions ratios felt by users from post-gameplay
survey.
most users agreed that the game remained more engaging than
their traditional therapy routine and that the game provided a
distraction for them during their exercise, as shown in Figure
7. Similarly, the majority of users reported a positive range
of emotions for each exercise ranging from Happy/Joyful,
Excited/Motivated, and Relaxed, as shown in Figure 8. Q3-
4 show the largest differences in survey responses between
protocols. Specifically on Q4, the Foundation Protocol saw
a transition from unanimous disagreement with noticing “the
outside world while playing the game” to a greater majority of
neutral as time progressed. The Challenge Protocol was inverse
to this effect, where users eventually became unanimous in
disagreeing that they could notice the outside world during
gameplay. In essence, this suggests that users were much
more engaged in the game during the last two sessions of
the Challenge protocol.
V. DISCUSSI ON
Through analyzing the data from our two months study, we
observed the following phenomena:
PBF was able to elicit rehabilitative responses similar to
traditional therapy, including increases in muscle’s strength,
control and flexibility. The results suggest that across all users,
their resistance successfully increased throughout the study, as
evidenced by the weight increments that the users were able
to cope with. Heart rate increased for both protocols, which
we concluded were due to the increased weights that require
additional muscular efforts. Compliance improved more during
the Challenge Protocol than the Foundation Protocol, which
may suggest that users that were challenged with the complex
movements followed the protocol more carefully than asked to
perform simpler movements. During the exit interview, users
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
7
perceived that they gained significant strength and stability
through playing the game. They felt they would have been
unable to play the game at the beginning of the study using
their final session’s weights, and yet, users were able to
perform those exercises with those weights.
Users can remain engaged in physical therapy using PBF
and HTC Vive beyond the novelty effect period. One of the
dangers of long-term therapy is when users get bored and
lose interest in the exercises. We did not observe any decrease
of interest and engagement beyond the novelty effect (when
users were still new at iVR games). The greatest changes in all
the brainwave bands (which are often associated with levels of
stress [20], focus [21], [22], awareness [23], [24], motor [25],
[26], and cognition [27], [28]) were seen in the transition from
the Foundation Protocol to the Challenge Protocol. This sharp
increase in all bands suggests that the additional exercises were
able to engage the users considerably. Additionally, blinks
were the lowest for each protocol’s first session, indicating
the user may have been more focused during these sessions
[74]. This could mean that creating new types of movements
after the user has become accustomed to a set of exercises
can stimulate users to remain interested. When the Challenge
Protocol was introduced, jaw clenches were increased from
baseline, possibly indicating greater effort of the participant
[75]. The survey responses also showed that users felt engaged
by each protocol. In Q1 (Figure 7), users compared gameplay
exercises to their traditional therapy. GSR responses declined
over time in both protocols, and we speculate that this is
most likely due to users becoming more acclimated to the
game over time and thus causing drop in arousal. We should
note that at the end of each protocol, GSR’ level stabilizes,
indicating a steady state arousal. The survey results suggests
that PBF was more successful in enabling engagement of
physical therapy than traditional interventions. Questions Q2-4
also demonstrated that the user felt present in the game world,
possibly indicating a successful immersion. Additionally, users
stated that they enjoyed the first few sessions, but started
to lose enthusiasm as they felt the Foundation Protocol was
too repetitive and straightforward. With the introduction of
new and more complex movements that could not be easily
memorized in Challenge Protocol, the users were excited once
again to play the game. Some users stated that the more
complex movements kept them engaged rather than “zoning
out” as they did during the simple movements over time.
iVR games have the potential as a long-term physical
therapy tool that can be used at home. PBF was successful
in inducing rehabilitative response while maintaining extended
engagement. The basic version of PBF only requires the Vive
headset and none of the biosensors for remote usage. We argue
that this suggests a low-cost solution for rehabilitation exercise
compared to traditional long term physical therapy sessions
that require users to visit a clinic.
It appears that the differences in the difficulty levels and
goals between the two protocols induced noticeable change
in the different brainwave measures. The most substantial
changes were seen in the transition from Foundation Proto-
col to Challenge protocol, where new and more challenging
games were added to the already existing games. From the
context of Alpha band power (often associated with stress),
these results may indicate that Challenge Protocol has much
greater difficulty than Foundation Protocol, and induced a
higher amount of stress when transitioning to new and more
challenging exercises. It appears that Beta (often associated
with focus) spiked higher when difficulty was increased, which
was expected as users must focus harder when the game
became more difficult. One take away message from these two
findings is that, if we are to design games whose difficulty
levels adapt to users’ biometric changes, there should be a
careful consideration to design how sharply difficulty levels
increase, as increased levels of difficulty induces higher focus,
but also higher stress.
More work needs to be done in establishing dynamic pro-
gression of difficulty, utilizing biomarkers, and testing more
users. Progression during the game was an essential aspect
for all users. They related adding new movements to the
game to be like “unlocking a new level.” All participants
stated that they would like to have more levels to advance
through and clear goals for each level. This would help keep
things dynamic and avoid boredom due to repetition. Three of
the five participants would recommend this game to a friend
in its current version, while the last two stated they would
recommend the game if more levels, progression, and goals
were incorporated into the game.
It should be noted that future research should explore a
more significant number of users and VR experiences to
understand the long-term effects and user response of iVR
physical rehabilitation gaming. As more immersive virtual
environments are crafted for physical rehabilitation, there is
a need to establish how such a system can be tuned to the
user’s biometrics to induce a desirable range of activity and
understand how this will compare to conventional physical
rehabilitation. In this study, seven different movements and
one virtual environment were explored for upper extremity
physical rehabilitation. More motions and varied experiences
should be investigated to examine the game design, difficulty,
and adaptation to iVR stimuli. We are also mindful that
there were only five users that we followed for two months;
however, we believe that this study is an important step
towards gathering insights for future studies.
VI. CO NC LU SI ON
This study explored the effects of an immersive Virtual
Reality HMD gamified upper-extremity physical therapy that
record both physical and biometric responses over the course
of two months. To provide a more engaging experience, we
designed the study so that users completed their prescribed
therapeutic movements by protecting a virtual butterfly in a
dynamic and adaptive virtual environment. Two rehabilitative
goals were set in the study: recovery of foundational move-
ments and progressing with more complex motions. The study
results suggest that movement improvements over time can
be quantitatively assessed through game logs. The study also
concluded that the biometric responses can complement game
data and provide a richer insight on user engagement. These
findings may indicate that long term immersive Virtual Reality
physical rehabilitation is feasible.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
8
In the future, we aim to expand PBF’s capabilities for
home health and to run larger trials for comparison with
conventional therapy methods. We would like to dive deeper
into the effects of immersive physio-rehabilitation through
controlled trials to understand how user-perceived confidence
and difficulty influences the recovery journey. Additionally,
virtual environments, such as PBF, provide an opportunity to
explore runtime biofeedback and adaptive difficulty with emo-
tion classification, which we also plan to investigate Motion
capture data with biomechanical simulation may be utilized to
estimate muscle forces for understanding biased movements
and how to best prescribe rehabilitation towards addressing
weaknesses. We plan to run biomechanical simulation for this
estimation The creation of an adaptive, personalized physical
therapy game that adjusts to the user’s mental and physical
state in runtime may yield immense potential. Subsequently,
there are far more butterflies to follow on the road ahead.
ACK NOW LE DG ME NT
This material is based upon work supported by the National
Science Foundation under Grant No #1521532. Any opinions,
findings, and conclusions or recommendations expressed in
this material are those of the author(s) and do not necessarily
reflect the views of the National Science Foundation. The
authors thank the participants of this study, without their time
and effort, none of this would be possible.
REF ER EN CE S
[1] M. Beccue and C. Wheelock, “Research report: Virtual
reality for consumer markets,” Q4 2016. [Online]. Available:
https://www.tractica.com/research/virtual-reality-for-consumer-markets/
[2] R. Skarbez, F. P. Brooks, Jr, and M. C. Whitton, “A survey of presence
and related concepts,” ACM Computing Surveys (CSUR), vol. 50, no. 6,
pp. 1–39, 2017.
[3] M. Slater, “Place illusion and plausibility can lead to realistic behaviour
in immersive virtual environments,” Philosophical Transactions of the
Royal Society B: Biological Sciences, vol. 364, no. 1535, pp. 3549–3557,
2009.
[4] J. Diemer, G. W. Alpers, H. M. Peperkorn, Y. Shiban, and
A. M¨
uhlberger, “The impact of perception and presence on emotional
reactions: a review of research in virtual reality,” Frontiers in psychology,
vol. 6, 2015.
[5] K. R. Lohse, C. G. Hilderman, K. L. Cheung, S. Tatla, and H. M. Van der
Loos, “Virtual reality therapy for adults post-stroke: a systematic review
and meta-analysis exploring virtual environments and commercial games
in therapy,” PloS one, vol. 9, no. 3, p. e93318, 2014.
[6] A. Elor and S. Kurniawan, “The Ultimate Display for Physical Reha-
bilitation: A Bridging Review on Immersive Virtual Reality,” Frontiers
in Virtual Reality, vol. 1, p. 585993, Nov. 2020. [Online]. Available:
https://www.frontiersin.org/articles/10.3389/frvir.2020.585993/full
[7] S. C. Gobron, N. Zannini, N. Wenk, C. Schmitt, Y. Charrotton,
A. Fauquex, M. Lauria, F. Degache, and R. Frischknecht, “Serious games
for rehabilitation using head-mounted display and haptic devices,” in
International Conference on Augmented and Virtual Reality. Springer,
2015, pp. 199–219.
[8] E. P. Cherniack, “Not just fun and games: applications of virtual reality
in the identification and rehabilitation of cognitive disorders of the
elderly,” Disability and rehabilitation: Assistive technology, vol. 6, no. 4,
pp. 283–289, 2011.
[9] S. Finkelstein, A. Nickel, Z. Lipps, T. Barnes, Z. Wartell, and E. A.
Suma, “Astrojumper: Motivating exercise with an immersive virtual
reality exergame,” Presence: Teleoperators and Virtual Environments,
vol. 20, no. 1, pp. 78–92, 2011.
[10] A. Elor, S. Kurniawan, and M. Teodorescu, “Towards an Immersive
Virtual Reality Game for Smarter Post-Stroke Rehabilitation,” in 2018
IEEE International Conference on Smart Computing (SMARTCOMP).
Taormina: IEEE, Jun. 2018, pp. 219–225. [Online]. Available:
https://ieeexplore.ieee.org/document/8421352/
[11] A. Elor, M. Teodorescu, and S. Kurniawan, “Project Star
Catcher: A Novel Immersive Virtual Reality Experience for
Upper Limb Rehabilitation,” ACM Transactions on Accessible
Computing, vol. 11, no. 4, pp. 1–25, Nov. 2018. [Online]. Available:
https://dl.acm.org/doi/10.1145/3265755
[12] A. Elor, M. Powell, E. Mahmoodi, N. Hawthorne, M. Teodorescu,
and S. Kurniawan, “On Shooting Stars: Comparing CAVE and HMD
Immersive Virtual Reality Exergaming for Adults with Mixed Ability,”
ACM Transactions on Computing for Healthcare, vol. 1, no. 4, pp. 1–22,
Dec. 2020. [Online]. Available: https://dl.acm.org/doi/10.1145/3396249
[13] N. Morina, H. Ijntema, K. Meyerbr ¨
oker, and P. M. Emmelkamp, “Can
virtual reality exposure therapy gains be generalized to real-life? a meta-
analysis of studies applying behavioral assessments,” Behaviour research
and therapy, vol. 74, pp. 18–24, 2015.
[14] M. V. Nararro-Haro, H. G. Hoffman, A. Garcia-Palacios, M. Sampaio,
W. Alhalabi, K. Hall, and M. Linehan, “The use of virtual reality to
facilitate mindfulness skills training in dialectical behavioral therapy for
borderline personality disorder: a case study,” Frontiers in psychology,
vol. 7, 2016.
[15] Y. Shiban, I. Schelhorn, P. Pauli, and A. M¨
uhlberger, “Effect of com-
bined multiple contexts and multiple stimuli exposure in spider phobia:
a randomized clinical trial in virtual reality,” Behaviour research and
therapy, vol. 71, pp. 45–53, 2015.
[16] M. Rus-Calafell, J. Guti´
errez-Maldonado, and J. Ribas-Sabat´
e, “A virtual
reality-integrated program for improving social skills in patients with
schizophrenia: a pilot study,” Journal of behavior therapy and experi-
mental psychiatry, vol. 45, no. 1, pp. 81–89, 2014.
[17] D. Gromala, X. Tong, A. Choo, M. Karamnejad, and C. D. Shaw,
“The virtual meditative walk: virtual reality therapy for chronic pain
management,” in Proceedings of the 33rd Annual ACM Conference on
Human Factors in Computing Systems. ACM, 2015, pp. 521–524.
[18] P. J. Costello, Health and safety issues associated with virtual reality:
a review of current literature. Advisory Group on Computer Graphics,
1997.
[19] A. Al-Nafjan, M. Hosny, Y. Al-Ohali, and A. Al-Wabil, “Review
and classification of emotion recognition based on eeg brain-computer
interface system research: a systematic review,” Applied Sciences, vol. 7,
no. 12, p. 1239, 2017.
[20] J. J. Foster, D. W. Sutterer, J. T. Serences, E. K. Vogel, and E. Awh,
“Alpha-band oscillations enable spatially and temporally resolved track-
ing of covert spatial attention,” Psychological science, vol. 28, no. 7,
pp. 929–941, 2017.
[21] M. Rangaswamy, B. Porjesz, D. B. Chorlian, K. Wang, K. A. Jones,
L. O. Bauer, J. Rohrbaugh, S. J. O’connor, S. Kuperman, T. Reich et al.,
“Beta power in the eeg of alcoholics,” Biological psychiatry, vol. 52,
no. 8, pp. 831–842, 2002.
[22] J. Baumeister, T. Barthel, K.-R. Geiss, and M. Weiss, “Influence of
phosphatidylserine on cognitive performance and cortical activity after
induced stress,” Nutritional neuroscience, vol. 11, no. 3, pp. 103–110,
2008.
[23] J. A. Hobson and E. F. Pace-Schott, “The cognitive neuroscience of
sleep: neuronal systems, consciousness and learning,” Nature Reviews
Neuroscience, vol. 3, no. 9, p. 679, 2002.
[24] F. Brigo, “Intermittent rhythmic delta activity patterns,” Epilepsy &
Behavior, vol. 20, no. 2, pp. 254–256, 2011.
[25] J. O‘Keefe and N. Burgess, “Theta activity, virtual navigation and the
human hippocampus,” Trends in cognitive sciences, vol. 3, no. 11, pp.
403–406, 1999.
[26] M. E. Hasselmo and H. Eichenbaum, “Hippocampal mechanisms for
the context-dependent retrieval of episodes,” Neural networks, vol. 18,
no. 9, pp. 1172–1190, 2005.
[27] J. R. Hughes, “Gamma, fast, and ultrafast waves of the brain: their
relationships with epilepsy and behavior,” Epilepsy & Behavior, vol. 13,
no. 1, pp. 25–31, 2008.
[28] S. O‘Nuallain, “Zero power and selflessness: What meditation and
conscious perception have in common,” Journal of Cognitive Sciences,
vol. 4, no. 2, pp. 46–64, 2009.
[29] W. Boucsein, Electrodermal activity. Springer Science & Business
Media, 2012.
[30] R. J. Rajae-Joordens, “Measuring experiences in gaming and tv appli-
cations,” in Probing experience. Springer, 2008, pp. 77–90.
[31] V. N. Salimpoor, M. Benovoy, G. Longo, J. R. Cooperstock, and R. J.
Zatorre, “The rewarding aspects of music listening are related to degree
of emotional arousal,” PloS one, vol. 4, no. 10, p. e7487, 2009.
[32] A. Goshvarpour, A. Abbasi, and A. Goshvarpour, “An accurate emotion
recognition system using ecg and gsr signals and matching pursuit
method,” biomedical journal, vol. 40, no. 6, pp. 355–368, 2017.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2021.3069445, IEEE
Transactions on Games
9
[33] M. Liu, D. Fan, X. Zhang, and X. Gong, “Human emotion recognition
based on galvanic skin response signal feature selection and svm,” in
2016 International Conference on Smart City and Systems Engineering
(ICSCSE). IEEE, 2016, pp. 157–160.
[34] M. S. Cameirao, I. B. S. Berm ´
udez, E. Duarte Oller, and P. F. Verschure,
“The rehabilitation gaming system: a review,” Stud Health Technol
Inform, vol. 145, no. 6, 2009.
[35] R. Soares, E. Siqueira, M. Miura, T. Silva, and C. Castanho, “Biofeed-
back sensors in game telemetry research,” Simp´
osio Brasileiro de Jogos
e Entretenimento Digital, pp. 81–89, 2016.
[36] D. Deutscher, S. D. Horn, R. Dickstein, D. L. Hart, R. J. Smout,
M. Gutvirtz, and I. Ariel, “Associations between treatment processes,
patient characteristics, and outcomes in outpatient physical therapy
practice,” Archives of physical medicine and rehabilitation, vol. 90,
no. 8, pp. 1349–1363, 2009.
[37] R. S. Calabr `
o, A. Naro, M. Russo, A. Leo, R. De Luca, T. Balletta,
A. Buda, G. La Rosa, A. Bramanti, and P. Bramanti, “The role of
virtual reality in improving motor performance as revealed by eeg: a
randomized clinical trial,” Journal of neuroengineering and rehabilita-
tion, vol. 14, no. 1, p. 53, 2017.
[38] M. D. Wiederhold, M. Crisci, V. Patel, M. Nonaka, and B. K. Wieder-
hold, “Physiological monitoring during augmented reality exercise con-
firms advantages to health and well-being,” Cyberpsychology, Behavior,
and Social Networking, vol. 22, no. 2, pp. 122–126, 2019.
[39] U. Qidwai, M. Ajimsha, and M. Shakir, “The role of eeg and emg
combined virtual reality gaming system in facial palsy rehabilitation-
a case report,” Journal of bodywork and movement therapies, vol. 23,
no. 2, pp. 425–431, 2019.
[40] S. Ahn and S. Hwang, “Virtual rehabilitation of upper extremity function
and independence for stoke: a meta-analysis,” Journal of exercise
rehabilitation, vol. 15, no. 3, p. 358, 2019.
[41] A. I. Corregidor-S´
anchez, A. Segura-Fragoso, M. Rodr´
ıguez-Hern´
andez,
J. J. Criado-Alvarez, G.-G. Jaime, and B. Polonio-L´
opez, “Can ex-
ergames contribute to improving walking capacity in older adults? a
systematic review and meta-analysis,” Maturitas, 2019.
[42] M. C. Howard, “A meta-analysis and systematic literature review of
virtual reality rehabilitation programs,” Computers in Human Behavior,
vol. 70, pp. 317–327, 2017.
[43] I. Brunner, J. S. Skouen, H. Hofstad, J. Aßmus, F. Becker, A.-M.
Sanders, H. Pallesen, L. Q. Kristensen, M. Michielsen, L. Thijs et al.,
“Virtual reality training for upper extremity in subacute stroke (virtues):
a multicenter rct,” Neurology, vol. 89, no. 24, pp. 2413–2421, 2017.
[44] S. R. S. BPT and N. Giladi, “Improved mobility and reduced fall risk
in older adults after five weeks of virtual reality training,” Journal of
Alternative Medicine Research, vol. 9, no. 2, p. 171, 2017.
[45] P. Standen, K. Threapleton, A. Richardson, L. Connell, D. Brown,
S. Battersby, F. Platts, and A. Burton, “A low cost virtual reality system
for home based rehabilitation of the arm following stroke: a randomised
controlled feasibility trial,” Clinical rehabilitation, vol. 31, no. 3, pp.
340–350, 2017.
[46] W. B. Kibler, J. McMULLEN, and T. Uhl, “Shoulder rehabilitation
strategies, guidelines, and practice,” Orthopedic Clinics, vol. 32, no. 3,
pp. 527–538, 2001.
[47] W. B. Kibler, “Shoulder rehabilitation: principles and practice.” Medicine
and science in sports and exercise, vol. 30, no. 4 Suppl, pp. S40–50,
1998.
[48] P. Rego, P. M. Moreira, and L. P. Reis, “Serious games for rehabilitation:
A survey and a classification towards a taxonomy,” in 5th Iberian
conference on information systems and technologies. IEEE, 2010, pp.
1–6.
[49] A. Elor, S. Lessard, M. Teodorescu, and S. Kurniawan, “Project
Butterfly: Synergizing Immersive Virtual Reality with Actuated
Soft Exosuit for Upper-Extremity Rehabilitation,” in 2019 IEEE
Conference on Virtual Reality and 3D User Interfaces (VR).
Osaka, Japan: IEEE, Mar. 2019, pp. 1448–1456. [Online]. Available:
https://ieeexplore.ieee.org/document/8798014/
[50] A. Elor and S. Kurniawan, “Deep Reinforcement Learning in Immersive
Virtual Reality Exergame for Agent Movement Guidance,” in 2020
IEEE 8th International Conference on Serious Games and Applications
for Health (SeGAH). Vancouver, BC, Canada: IEEE, Aug. 2020, pp.
1–7. [Online]. Available: https://ieeexplore.ieee.org/document/9201901/
[51] A. Borrego, J. Latorre, M. Alca ˜
niz, and R. Llorens, “Comparison of
oculus rift and htc vive: feasibility for virtual reality-based exploration,
navigation, exergaming, and rehabilitation,” Games for health journal,
vol. 7, no. 3, pp. 151–156, 2018.
[52] S. M. Palaniappan and B. S. Duerstock, “Developing rehabilitation
practices using virtual reality exergaming,” in 2018 IEEE International
Symposium on Signal Processing and Information Technology (ISSPIT).
IEEE, 2018, pp. 090–094.
[53] F. Soffel, M. Zank, and A. Kunz, “Postural stability analysis in virtual
reality using the htc vive,” in Proceedings of the 22nd ACM Conference
on Virtual Reality Software and Technology. ACM, 2016, pp. 351–352.
[54] D. C. Niehorster, L. Li, and M. Lappe, “The accuracy and precision of
position and orientation tracking in the htc vive virtual reality system for
scientific research,” i-Perception, vol. 8, no. 3, p. 2041669517708205,
2017.
[55] E. L. Deci and R. M. Ryan, “Self-determination theory.” 2012.
[56] L. A. Shaw, B. C. W¨
unsche, C. Lutteroth, S. Marks, J. Buckley, and
P. Corballis, “Development and evaluation of an exercycle game using
immersive technologies,” 2015.
[57] L. A. Shaw, J. Buckley, P. M. Corballis, C. Lutteroth, and B. C.
W¨
unsche, “Competition and cooperation with virtual players in an
exergame,” PeerJ Computer Science, vol. 2, p. e92, 2016.
[58] A. Keesing, M. Ooi, O. Wu, X. Ye, L. Shaw, and B. C. W¨
unsche, “Hiit
with hits: Using music and gameplay to induce hiit in exergames,” in
Proceedings of the Australasian Computer Science Week Multiconfer-
ence, 2019, pp. 1–10.
[59] L. A. Shaw, B. C. Wuensche, C. Lutteroth, J. Buckley, and P. Corballis,
“Evaluating sensory feedback for immersion in exergames,” in Proceed-
ings of the Australasian Computer Science Week Multiconference, 2017,
pp. 1–6.
[60] J. C. Haller, Y. H. Jang, J. Haller, L. Shaw, and B. C. W¨
unsche, “Hiit the
road: Using virtual spectator feedback in hiit-based exergaming,” in Pro-
ceedings of the Australasian Computer Science Week Multiconference,
2019, pp. 1–9.
[61] C. D. Batson, P. A. Van Lange, N. Ahmad, and D. A. Lishner, “Altruism
and helping behavior,” The Sage handbook of social psychology, vol. 1,
pp. 279–295, 2003.
[62] M. O. Wilhelm and R. Bekkers, “Helping behavior, dispositional em-
pathic concern, and the principle of care,” Social Psychology Quarterly,
vol. 73, no. 1, pp. 11–32, 2010.
[63] InteraXon, “Featured research with muse,” Tech.
Rep., 29.06.2019, https://choosemuse.com/muse-
research/developer.choosemuse.com/tools/available-data.
[64] N. Kovacevic, P. Ritter, W. Tays, S. Moreno, and A. R. McIntosh,
“‘my virtual dream’: Collective neurofeedback in an immersive art
environment,” PloS one, vol. 10, no. 7, p. e0130129, 2015.
[65] O. E. Krigolson, C. C. Williams, A. Norton, C. D. Hassall, and F. L.
Colino, “Choosing muse: Validation of a low-cost, portable eeg system
for erp research,” Frontiers in neuroscience, vol. 11, p. 109, 2017.
[66] S. Bhayee, P. Tomaszewski, D. H. Lee, G. Moffat, L. Pino, S. Moreno,
and N. A. Farb, “Attentional and affective consequences of technology
supported mindfulness training: a randomised, active control, efficacy
trial,” BMC psychology, vol. 4, no. 1, p. 60, 2016.
[67] Neulog, “Gsr logger sensor nul-217,” Tech. Rep., 29.06.2019,
https://neulog.com/gsr/.
[68] Polar, “Polar oh1: optical heart rate sensor,” Tech. Rep., 29.06.2019,
https://www.polar.com/us-en/products/accessories/oh1-optical-heart-
rate-sensor.
[69] C. Jennett, A. L. Cox, P. Cairns, S. Dhoparee, A. Epps, T. Tijs, and
A. Walton, “Measuring and defining the experience of immersion in
games,” International journal of human-computer studies, vol. 66, no. 9,
pp. 641–661, 2008.
[70] K. Doherty and G. Doherty, “Engagement in hci: conception, theory
and measurement,” ACM Computing Surveys (CSUR), vol. 51, no. 5,
pp. 1–39, 2018.
[71] J. A. Russell, “A circumplex model of affect.” Journal of personality
and social psychology, vol. 39, no. 6, p. 1161, 1980.
[72] P. Ekman, “Basic emotions,” Handbook of cognition and emotion,
vol. 98, no. 45-60, p. 16, 1999.
[73] D. J. Higham and N. J. Higham, MATLAB guide. Siam, 2016, vol.
150.
[74] M. Rosenfield, S. Jahan, K. Nunez, and K. Chan, “Cognitive demand,
digital screens and blink rate,” Computers in Human Behavior, vol. 51,
pp. 403–406, 2015.
[75] D.-H. Huang, S.-W. Chou, Y.-L. Chen, and W.-K. Chiou, “Frowning
and jaw clenching muscle activity reflects the perception of effort during
incremental workload cycling,” Journal of sports science & medicine,
vol. 13, no. 4, p. 921, 2014.
