Content uploaded by Sebastiano Massaro
Author content
All content in this area was uploaded by Sebastiano Massaro on Oct 31, 2018
Content may be subject to copyright.
1
Heart Rate Variability Analysis and Performance during a Repeated Mental
Workload Task
R.Castaldo1, L. Montesinos1, S. Wan1, A. Serban2, S. Massaro3 and L. Pecchia1
1 University of Warwick, School of Engineering, Coventry, UK
2 Virginia Commonwealth University, School of Business, Richmond, Virginia, USA
3University of Warwick, Warwick Business School & GRP in Behavioral Science, Coventry, UK
Abstract— We designed and conducted an experiment using
a repetitive task to investigate associations between mental
workload, performance, and Heart Rate Variability (HRV) fea-
tures across repetitions. According to the literature, we define
mental workload as the interaction between a person and a task
that causes task demands to exceed the person’s capacity to de-
liver. Mental workload was triggered by the use of a highly-
paced video game repeated over time. Before engaging with the
task, each subject was assessed in controlled condition (i.e., re-
laxing period) for a short time. Short term HRV features varia-
tions between the baseline (i.e., control situation) and each re-
petitive gaming session (i.e., mental task) were explored. The
results show that HRV dynamics diminish with repetitions,
while performance increases. Importantly, this suggests that
HRV features can be well correlated with performance. Overall,
this study advances the use of HRV analysis in the behavioral
sciences at large, allowing the design of flexible neurophysiolog-
ical lab-based experiments. Thus, it also opens the way to future
autonomic behavioral neuroscience research.
Keywords— HRV, behavioral science, mental workload,
mental stress, performance
I.INTRODUCTION
Several physiological signals (e.g., ECG, EMG, and skin
conductance) have been increasingly used in autonomic neu-
roscience and behavioral science to assess mental workload
and attention. Among them, cardiac activity and Heart Rate
Variability (HRV) are some of the most investigated signals
and methodologies used, respectively. Indeed, research has
argued that HRV offers an index of the autonomic nervous
system (ANS) activity, useful to investigate valence, arousal,
attention, cognitive effort, and stress [1]. For instance, re-
search shows that HRV is reduced as mental workload and
attentional demands increase [2]. At the same time, this
method allows overcoming sensitivity issues of self-reported
approaches (e.g., surveys) in capturing behavioral constructs
[3]. Added to this, HRV analysis is non-invasive, wearable,
relatively inexpensive, and enables long-term registration,
thus being applicable in the laboratory and easily transferable
into real-life settings. Leveraging these benefits, HRV has
been often used as indicator of mental stress and workload
[4].
In this study, we sought to investigate to what extent
mental workload can be assessed by measuring HRV varia-
tions; we also sought to understand if performance is corre-
lated with changes in HRV features during repeated tasks.
Mental workload reflects the mental demand and physiolog-
ical stress experienced by a subject during a task [5]. Here,
mental workload was triggered by repetitive sessions of a
highly paced video game. Indeed, video games have been
shown to be good settings to trigger mental workload [6].
Put it in other words, this study sought to understand if
short term HRV analysis can be a reliable indicator of mental
workload inducted by a demanding task, and if performance
is associated with changes in HRV features among repeated
tasks.
To date, only a few studies [7-10] have attempted to tackle
this problem using ECG recordings. For one, Henning et al.
[8] show that reduced mental load is associated with slower
heart rate (HR) [7]. For another, Lehrer et al. [9] investigate
performance of air-pilots, showing that it is negatively corre-
lated with the low-frequency/high-frequency ratio (LF/HF)
and with mean HR. Finally, Fairclough et al. [10] investigate
trends between physiological signals and performance by us-
ing subtasks with increasing demands, and showing that the
mean of inter-times among consecutive normal beats
(MeanNN) and the vagal tone are lower during high demand-
ing mental tasks. Yet, these studies were generally carried out
on only a few subjects, lacking sufficient statistical power.
In the present experiment we tackle this gaps and investi-
gate how mental workload affects HRV over repeated video
gaming sessions in 54 subjects. As a secondary outcome, we
also aim to demonstrate to what extent HRV features are cor-
related with performance over time in the context of our re-
peated task.
II.METHODS AND MATERIAL
A.Study Participants
69 healthy subjects (32 females; age: 26±2.2 years) with
no history of heart disease, systemic hypertension, or other
conditions potentially influencing HRV analysis were en-
rolled in this study. They were not obese and did not take
medication, drugs, or alcohol in the 12 hours preceding the
2
experiment. All subjects were right handed with normal vi-
sion, frequent computers users (i.e., daily), and skilled at op-
erating mouse and keyboard. None of the subjects included
in our current analysis was familiar with video games. The
Biomedical and Scientific Research Ethics Committee of the
University of Warwick approved this study (ref. REGO-
2014-656 AMO1), assuring anonymity and no side effects or
possible disadvantages for the subjects. All subjects were
carefully instructed and informed consent was acquired prior
to the experiment. Subjects were compensated with a fixed
show-up fee.
B.Protocol
The study was conducted in the Behavioral Science La-
boratory of the Warwick Business School under controlled
conditions: a quiet room, at a comfortable temperature, at the
same hour in the morning. They were instructed to sit com-
fortably on an armchair and were assisted in wearing a Bi-
oPatch™ M3 device (Zephyr, Annapolis, USA) to record the
ECG.
Continuous ECG recording was performed during base-
line (i.e., control condition) and mental task (i.e., four video
gaming sessions). A 60 s break was taken between sessions.
The baseline session was recorded for 5 minutes in which the
participants were asked to complete a computer-based survey
on demographic and anthropometric information (i.e. age,
gender weight, height, and handedness) (Fig 1). Before the
video game started, the researchers presented each partici-
pant with a brief introduction on the game’s instructions. The
video game was administered repetitively on a 24’’ PC mon-
itor for 5 nominal minutes per session.
Fig. 1 Study protocol
C.Video Game
A highly-paced one-person shooter video game containing
some violent content (i.e., war scenes and gun fighting),
ranked as suitable to people above 16 years, was used as trig-
ger of mental workload. Highly-paced video games have
shown to be a reliable method to elicit mental workload [6].
The video game was played as a repetitive task (i.e., four ses-
sions). This was done in order to investigate the impact of
mental workload on HRV, as well as any correlation between
HRV and performance over time. The main challenge posed
to participants during the sessions was to complete a mission
by killing as many enemies as possible. The performance for
each game session was computed directly by the video game
as the number of enemies killed per session.
D.HRV Analysis
The RR interval time-series was extracted from ECG rec-
ords using an automatic QRS detector, WQRS, available in
the PhysioNet’s toolkit [11]. QRS review and correction was
performed using PhysioNet’s WAVE. The fraction of total
RR intervals labelled as normal-to-normal (NN) intervals
was computed as NN/RR ratio. NN/RR ratio was then used
to measure the reliability of the data. Records with NN/RR
ratio less than 90% threshold were excluded from the analy-
sis. HRV analysis was performed on 5 min excerpts using
Kubios (version 2.2) [12]. Time and frequency-domain fea-
tures were analyzed according to international guidelines
[13], while non-linear measures were analyzed as described
in [14]. Frequency domain features were extracted from
power spectrum estimated with autoregressive (AR) model
methods [12]. Finally, 20 HRV features were examined (Ta-
ble 1). Table 1. HRV features analyzed in the study
HRV
Measures
Units
Description
MeanNN
[ms]
The mean of RR intervals
StdNN
[ms]
Standard deviation of RR intervals
RMSSD
[ms]
Square root of the mean squared differences be-
tween successive RR intervals
NN50
-
Number of successive RR interval pairs that differ
more than 50 s
pNN50
[%]
NN50 divided by the total number of RR intervals
Absolute
power
[ms2]
Absolute powers of LF(0.04-0.15Hz) and HF
bands( 0.15-0.4 Hz)
LF/HF
-
Ratio between LF and HF band powers
sd1, sd2
[ms]
The standard deviation of the Poincare’ plot per-
pendicular to (SD1) and along (SD2)the line-of-
identity
Apen
-
Approximate Entropy
Sampen
-
Sample entropy
d2
-
Correlation dimension
dfa1,
dfa2
-
Detrented fluctuation analysis: Short term and Long
term fluctuation slope
RPlmean
[beats]
Recurrence plot analysis: Mean line length
RPlmax
[beats]
Recurrence plot analysis: Maximum line length
REC
[%]
Recurrence rate
RPadet
[%]
Recurrence plot analysis: Determinism
ShanEn
-
Shannon entropy
E.Statistical, Trend and Correlation Analysis
Because HRV features were found non-normally distrib-
uted, Median (MD), Median Absolute Deviation (MAD) and
interquartile range (IQR) (i.e., non-parametric descriptors)
were computed for each repetition. The non-parametric Wil-
coxon Signed-Rank Test was used to appreciate statistical
differences of HRV features variation between the baseline
3
(i.e., control situation) and each repetitive task (i.e., video
game sessions). Comparisons among sessions were per-
formed using a Kruskal-Wallis test.
The trends of MD for HRV features were also reported
using the convention proposed in [4]:
two arrows, ↓↓ (or ↑↑), were used to report a signifi-
cant (p<0.05) decrease (or increase) of one feature
during the game session;
one arrow was used for non-significant variations: ↓
(or ↑) indicated a non-significant (p>.05) decrease (or
increase) of a measure during the game session.
Moreover, for exploratory purposes, the MD was plotted
along the performance per game session (i.e., number of
points) per each HRV feature. MD and performance were
normalized to their maximum and the trends between HRV
features and performance investigated. Spearman’s correla-
tion coefficients (rho) between each HRV feature and perfor-
mance over the four sessions were computed. A HRV feature
and performance were considered highly correlated if rho re-
sulted greater than 0.7 and significant if the associated p-
value was less than 0.05.
III.RESULTS AND DISCUSSION
Data for 54 participants were included in the current study.
ECG signals for 15 participants were discarded after quality
check (i.e., NN/RR ratio < 90%).
Table 2 shows the p-value and HRV trend between the re-
peated video gaming sessions (i.e., mental workload) and the
baseline (i.e., control condition). Seven HRV features
changed significantly between session 1 and the baseline (see
column 2: GS1 vs. B). In particular, MeanNN, SdNN, LF,
sd2 and dfa1 decreased significantly, while Apen and
Sampen increased significantly. As shown in the other col-
umns of Table 2, in GS2-4 more HRV features changed sig-
nificantly relatively to the baseline (see HRV features 8, 10,
and 11 respectively).
All the HRV features maintained similar trends over the
four sessions, reflecting consistency of HRV fluctuation as
the task progressed. Yet, the number of non-linear HRV fea-
tures showing to be significantly different increased moving
from session 1 to session 4. Indeed, non-linear HRV features
4, 5, 7 and 9 changed in game 1, 2, 3 and 4 respectively. This
overall reflects an increasingly depressed HRV during each
repetition, suggesting an increase in stress [4]. By comparing
within sessions, dfa2 and RPadet were found significantly
different between session 1 and 4, while ShanEn between
session 3 and 4. Previous studies [15-17], showed that as one
task’s demand increases, HRV becomes more depressed.
Added to this, here we found that also on repetitive tasks,
there is a depressed HRV over sessions, which is possibly
due to an increase in mental workload.
Fig. 2 shows the correlation coefficients between normal-
ized HRV features and performance over the four game ses-
sions. Fig. 2 shows that performance increased over sessions
overall. We reasoned that this occurred because players tend
to perform better once become more familiar with the video
game; this finding is also in agreement with previous studies
showing similar rates at which people improve through prac-
tice [18]. Noteworthy, eight HRV features presented high
correlation with performance over the four sessions. Aside
from MeanNN, all the remaining HRV features showed a
negative correlation with performance over sessions. Yet, the
p-value of those correlations was higher than 0.05, possibly
because of the limited amount of repetitions.
IV.CONCLUSIONS
In this study, we investigated the effects of mental work-
load induced by a repetitive task on HRV features. We found
that over repetitions, a depressed HRV accentuates as repeti-
tions progress. This is likely correlated with the increase in
the mental workload. Moreover, we found a correlation be-
tween task performance and key HRV features.
All in all, this study opens novel opportunities for research
at the interface between HRV analysis and behavioral science
Table 2. HRV Analysis
HRV
Features
GS1 vs B
GS2 vs B
GS3 vs B
GS4 vs B
p-val
Tr
p-val
Tr
p-val
Tr
p-val
Tr
MeanNN
0.029
↓↓
0.121
↓
0.405
↓
0.608
↓
StdNN
0.002
↓↓
0.000
↓↓
0.000
↓↓
0.000
↓↓
RMSSD
0.144
↓
0.080
↓
0.119
↓
0.227
↓
NN50
0.277
↓
0.077
↓
0.140
↓
0.190
↓
pNN50
0.177
↓
0.058
↓
0.123
↓
0.175
↓
LF
0.001
↓↓
0.001
↓↓
0.000
↓↓
0.000
↓↓
HF
0.053
↓
0.009
↓↓
0.006
↓↓
0.004
↓↓
LF/HF
0.412
↓
0.706
↓
0.724
↓
0.080
↓
sd1
0.144
↓
0.080
↓
0.119
↓
0.227
↓
sd2
0.001
↓↓
0.000
↓↓
0.000
↓↓
0.000
↓↓
Apen
0.008
↑↑
0.001
↑↑
0.001
↑↑
0.003
↑↑
Sampen
0.018
↑↑
0.000
↑↑
0.000
↑↑
0.000
↑↑
d2
0.747
↓
0.517
↓
0.118
↓
0.270
↓
dfa1
0.026
↓↓
0.115
↓
0.102
↓
0.010
↓↓
dfa2
0.052
↓
0.011
↓↓
0.028
↓↓
0.007a
↓↓
RPlmean
0.196
↓
0.076
↓
0.093
↓
0.005
↓↓
RPlmax
0.103
↓
0.236
↓
0.045
↓↓
0.076
↓
REC
0.078
↓
0.059
↓
0.027
↓↓
0.003
↓↓
RPadet
0.060
↓
0.012
↓↓
0.002
↓↓
0.000a
↓↓
ShanEn
0.158
↓
0.107
↓
0.114
↓
0.006b
↓↓
Tr: Trend; p-val: p-value; GS1-4: Game Sessions 1-4; B: Baseline; ↓↓ (↑↑):
significantly lower (higher) under mental workload (p<.05);↓(↑): lower (higher)
under mental workload (p>.05); a significant different between GS1 and GS4; b
significant different between GS3 and GS4.
4
in both lab-based and real-life settings. This also is poten-
tially able to foster the autonomic neuroscience research
agenda.
ACKNOWLEDGMENT
We acknowledge funding from the Global Research Priority
in Behavioural Science of the University of Warwick.
CONFLICT OF INTEREST
The authors declare no conflicts of interest.
REFERENCES
[1] S. Massaro and L. Pecchia, "Heart Rate Variability (HRV) Analysis A
Methodology for Organizational Neuroscience," Organizational
Research Methods, p. 1094428116681072, 2016.
[2] G. N. Yannakakis, H. P. Martinez, and M. Garbarino,
"Psychophysiology in games," in Emotion in Games, ed:
Springer, 2016, pp. 119-137.
[3] J. M. Kivikangas, G. Chanel, B. Cowley, I. Ekman, M. Salminen, S.
Järvelä, et al., "A review of the use of psychophysiological
methods in game research," journal of gaming & virtual worlds,
vol. 3, pp. 181-199, 2011.
[4] R. Castaldo, P. Melillo, U. Bracale, M. Caserta, M. Triassi, and L.
Pecchia, "Acute mental stress assessment via short term HRV
analysis in healthy adults: A systematic review with meta-
analysis," Biomedical Signal Processing and Control, vol. 18,
pp. 370-377, 2015.
[5] P. S. Tsang and M. A. Vidulich, "Mental workload and situation
awareness," Handbook of Human Factors and Ergonomics,
Third Edition, pp. 243-268, 2006.
[6] S. Hébert, R. Béland, O. Dionne-Fournelle, M. Crête, and S. J.
Lupien, "Physiological stress response to video-game playing:
the contribution of built-in music," Life sciences, vol. 76, pp.
2371-2380, 2005.
[7] R. A. Henning, W. Boucsein, and M. C. Gil, "Social–physiological
compliance as a determinant of team performance,"
International Journal of Psychophysiology, vol. 40, pp. 221-
232, 2001.
[8] R. A. Henning and S. L. Sauter, "Work-physiological synchronization
as a determinant of performance in repetitive computer work,"
Biological psychology, vol. 42, pp. 269-286, 1996.
[9] P. Lehrer, M. Karavidas, S.-E. Lu, E. Vaschillo, B. Vaschillo, and A.
Cheng, "Cardiac data increase association between self-report
and both expert ratings of task load and task performance in
flight simulator tasks: An exploratory study," International
Journal of Psychophysiology, vol. 76, pp. 80-87, 2010.
[10] S. H. Fairclough, L. Venables, and A. Tattersall, "The influence of
task demand and learning on the psychophysiological
response," International Journal of Psychophysiology, vol. 56,
pp. 171-184, 2005.
[11] A. L. Goldberger, L. A. Amaral, L. Glass, J. M. Hausdorff, P. C.
Ivanov, R. G. Mark, et al., "Physiobank, physiotoolkit, and
physionet components of a new research resource for complex
physiologic signals," Circulation, vol. 101, pp. e215-e220,
2000.
[12] M. P. Tarvainen and J.-P. Niskanen, "Kubios HRV User's Guide,"
Biosignal Analysis and Medical Imaging Group (BSAMIG),
Department of Physics University of Kuopio, 2013.
[13] T. Force, "Heart rate variability guidelines: Standards of
measurement, physiological interpretation, and clinical use,"
European Heart Journal, vol. 17, pp. 354-381, 1996.
[14] P. Melillo, M. Bracale, and L. Pecchia, "Nonlinear Heart Rate
Variability features for real-life stress detection. Case study:
students under stress due to university examination,"
BioMedical Engineering OnLine, vol. 10, pp. 1-13, 2011/11/07
2011.
[15] B. Cinaz, R. La Marca, B. Arnrich, and G. Tröster, "Monitoring of
mental workload levels," in Proceedings of IADIS eHealth
conference, 2010, pp. 189-193.
[16] B. Cinaz, B. Arnrich, R. La Marca, and G. Tröster, "Monitoring of
mental workload levels during an everyday life office-work
scenario," Personal and ubiquitous computing, vol. 17, pp. 229-
239, 2013.
[17] J. Paxion, E. Galy, and C. Berthelon, "Mental workload and driving,"
Frontiers in psychology, vol. 5, 2014.
[18] F. E. Ritter and L. J. Schooler, "The learning curve," International
encyclopedia of the social and behavioral sciences, vol. 13, pp.
8602-8605, 2001.
Fig. 2 Trend Analysis and Correlation Coefficients of HRV
features and performance.
