ArticlePDF Available

Abstract and Figures

Virtual Reality (VR) is increasingly being used in combination with psycho-physiological measures to improve assessment of distress in mental health research and therapy. However, the analysis and interpretation of multiple physiological measures is time consuming and requires specific skills, which are not available to most clinicians. To address this issue, we designed and developed a Decision Support System (DSS) for automatic classification of stress levels during exposure to VR environments. The DSS integrates different biosensor data (ECG, breathing rate, EEG) and behavioral data (body gestures correlated with stress), following a training process in which self-rated and clinical-rated stress levels are used as ground truth. Detected stress events for each VR session are reported to the therapist as an aggregated value (ranging from 0 to 1) and graphically displayed on a diagram accessible by the therapist through a web-based interface.
Content may be subject to copyright.
A Decision Support System for Real-Time
Stress Detection During Virtual Reality
Andrea GAGGIOLIa,b, Pietro CIPRESSOa, Silvia SERINOa, Giovanni PIOGGIAc,
Gennaro TARTARISCOc, Giovanni BALDUSc, Daniele CORDAc, Marcello FERROd ,
Nicola CARBONAROe, Alessandro TOGNETTIe,f, Danilo DE ROSSIf, Dimitris
GIAKOUMISg, Dimitrios TZOVARASg, Alejandro RIERAh, Giuseppe RIVAa,b
aATN-P Lab., Istituto Auxologico Italiano, Milan, Italy
bDepartment of Psychology, Università Cattolica del Sacro Cuore, Milan, Italy
cNational Research Council of Italy (CNR), Institute of Clinical Physiology (IFC), Italy
d“Antonio Zampolli” Institute for Computational Linguistics (ILC), Italy
eResearch Center "E.Piaggio", University of Pisa, Via Diotisalvi 2, Pisa, Italy
fInformation Engineering Department, University of Pisa, Via Caruso 2, Pisa, Italy
gInformatics and Telematics Institute, Centre for Research and Technology
Hellas (CERTH-ITI), Thermi, Thessaloniki, Greece
hStarlab Barcelona SL, Barcelona, Spain
Abstract. Virtual Reality (VR) is increasingly being used in combination with
psycho-physiological measures to improve assessment of distress in mental health
research and therapy. However, the analysis and interpretation of multiple
physiological measures is time consuming and requires specific skills, which are
not available to most clinicians. To address this issue, we designed and developed
a Decision Support System (DSS) for automatic classification of stress levels
during exposure to VR environments. The DSS integrates different biosensor data
(ECG, breathing rate, EEG) and behavioral data (body gestures correlated with
stress), following a training process in which self-rated and clinical-rated stress
levels are used as ground truth. Detected stress events for each VR session are
reported to the therapist as an aggregated value (ranging from 0 to 1) and
graphically displayed on a diagram accessible by the therapist through a web-
based interface.
Keywords. Psychological Stress, Psychophysiology, Virtual Reality, Decision
Support System, Biosensors.
In recent years, there has been growing interest towards the use of Virtual Reality (VR)
in mental health research and practice. In particular, an emerging application of this
approach concerns the assessment and management of psychological stress. At this
purpose, one of the most investigated cognitive-behavioral techniques is the Stress
Inoculation Training (SIT; [1]). SIT is implemented through gradual and repeated
Studies in Health Technology and
Informatics, Volume 196
IOS Press, 2014
© 2014 The authors and IOS Press. All rights reserved.
exposure to events, which have been previously identified as potential stressors. The
key idea underlying this technique is that “inoculating” the stressor in combination
with the acquisition of effective coping skills can prepare the patient to face similar
situations in daily life. VR is thought to further enhance the efficacy this process by
exposing the patient to realistic simulations of typical stressful situations [2].
A key issue in the application of VR in the SIT approach is how to accurately
evaluate the stress response during the exposure to the simulated stressor. The use of
biosensors for monitoring physiological and behavioral correlates of stress has been
proposed as a potential solution to this need [3; 4]. However, the analysis and
interpretation of multiple psychophysiological measures is time-consuming and
requires specific technical skills, which are often not often available to clinicians. In the
following, we present a Decision Support System (DSS) that has been designed to
assist the therapist/researcher in the assessment of stress levels during virtual exposure.
The specific goal of the DSS is to evaluate the psychological state of each patient by
analyzing previously acquired knowledge, such as patient’s physiological and
behavioral profile, and current sensory data. For each user, the DSS implements a
personal classifier that uses machine-learning techniques. Once trained, the DSS
analyzes the new incoming multi-sensors data acquired from the user and infers his/her
stress levels.
1. System Architecture
The system’s architecture consists of three main components: a) VR platform; b)
biosensors module; c) analysis and decision module.
1.1. VR Platform and Virtual Environments
The VR platform integrating the decision support system is NeuroVR-2 [5]
( A set of virtual environments for stress inoculation was
developed for this VR platform, targeting two highly-stressed professionals: teachers
and nurses. Virtual stressful situations were developed following design guidelines
collected in focus groups and in-depth interviews, which involved representative
samples of these professionals.
1.2. Biosensors Module
As concerns the biosensors module, the system currently supports the following
physiological and behavioral data acquisition platforms:
ECG and breathing (through a custom-designed Personal Biomonitoring
System described elsewhere [6])
EEG, through the platform “Enobio”[8]
Gesture-recognition system based on Microsoft Kinect, which is able to
automatically detect stress-related gestures (through the platform “CBAR”
A.Gaggioliet al./ADSSforReal-Time Stress Detection During Virtual Reality Exposure 115
These platforms feature minimally invasive equipment and support wireless
connectivity: two essential requirements for ensuring patient’s comfort during VR
1.2.1. Personal Biomonitoring System (PBS)
The PBS uses wireless sensors for ECG, activity monitoring (including posture) and
breathing rate [6]. The system consists of a chest band in which sensor modules and
electrodes are integrated; these components have been designed to ensure minimal
discomfort of the user. Acquired data are pre-processed on a wearable platform with
the purpose of extracting features to be transmitted to the central database; this
approach avoids using large sets of raw data and allows increasing battery life of the
wearable platform, as effect of reduced synchronization and processing times. For the
breathing rate, the sensors – integrated in a flexible band - were selected for accuracy in
chest movement and reduced sensitivity to artifacts. A piezoelectric PVDF transducer
was used to measure mechanical forces associated with chest movements [7].
1.2.2. Electroencephalogram System “Enobio”
Enobio is a wearable, modular and wireless electrophysiology sensor system for
monitoring brain activity (EEG) [8]. In the Enobio platform (Fig. 1), a feature extractor
computes the Fourier Transform and extracts the frequency power of several bands in
real time. In order to identify relevant EEG features for detecting stress in VR, a study
was undertaken [9]. The experimental setup consisted having the subject performing
different tasks designed to induce psychological stress (i.e. exposing the participant to a
fake blood sample). During the execution of these tasks, EEG was recorded with
Biosemi 32 channel. For every single pair of symmetrical EEG channels, alpha
asymmetry and the alpha/beta ratio (for each channel in this case) were extracted.
Alpha asymmetry is related with the valence dimension of emotions [10] while the
beta/alpha ratio is related with the arousal dimension of emotions [11]. Statistical
analysis of data collected from 12 participants showed that both beta-alpha ratio and
alpha asymmetry were correlated to stress levels on each task [9]. An interesting
feature of the Enobio system is its real-time capability. Not only all data processing
steps, but also the fusion stages can be performed in real time, allowing continuous
monitoring of stress levels during VR exposure.
Figure 1. The wireless EEG system “Enobio”
A.Gaggioliet al./ADSSforReal-Time Stress Detection During Virtual Reality Exposure116
1.2.3. Gesture Recognition System “CBAR”
A further module integrated in the VR platform is the “Camera and accelerometer
based Activity Recognition” (CBAR) system. Its purpose is to monitor in real-time the
subject’s activities through a video and an accelerometer modality, extracting
behavioral features that correlate with stress. The CBAR video modality is based on a
low-cost camera (Kinect), monitoring the subject as shown in Fig. 2(a). From the video
images sequence, Motion History Images (MHIs) [12] are extracted (Fig. 2(b)),
allowing the calculation of parameters related to upper-body activity. The
accelerometer modality is based on two tri-axial accelerometers placed at the subject’s
knees, for monitoring lower body activity. Implementation details of both modalities
can be found in [13]. In the clinical setting, the CBAR extracts in real-time a set of
behavioral features from both modalities and provides them to the VR platform. These
features express qualitative aspects of the subject’s movement, such as the average
upper body activity level and its deviation calculated through the non-black portion
either of all MHIs or only of MHIs that signify “increased activity”, the frequency of
occurrence of specific gestures (e.g. hand on head) and foot trembling. These features
showed significant correlates to psychological stress in the experimental evaluation of
[13], being capable to enhance automatic stress detection that is more typically based
on biosignals monitoring. Therefore, within the Interstress VR platform, the behavioral
features extracted from the CBAR augment the automatic stress monitoring process so
as to increase its effectiveness, whereas, in a more general view, they augment the
platform and the clinician with objective information regarding the patient’s
behavioural correlates of stress.
Figure 2. The Camera and accelerometer –based Activity Recognition CBAR
1.3. Central Database and Decision Module
Biosensor and activity data collected during VR sessions are filtered, pre-processed in
order to extract specific stress-related features and sent to a central database, where
they are linked by means of the timestamp information. The extracted features are
handled by the analysis and decision module (DSS), which allows classifying the
current stress level, after previous knowledge acquired during a training phase. The
DSS application (installed and running on a remote machine in respect to the database),
A.Gaggioliet al./ADSSforReal-Time Stress Detection During Virtual Reality Exposure 117
works at regular intervals, to query the remote database and to download data related to
all users and their VR sessions. In order to enforce the data security, the database is
indirectly accessed by the DSS using the WSDL layer interface exposed by the
INTERSTRESS architecture. The DSS is realized on a knowledge basis, using Fuzzy
Logic (FL), artificial neural network (ANN) and pattern classification algorithms. The
FL rule-based algorithm consists of three steps: fuzzification, inference and
defuzzification (Fig.3). The fuzzification converts the input from continuous values to
linguistic variables through the definition of membership functions. The inference
engine applies a set of fuzzy rules to generate linguistic values as output. In the final,
defuzzification step, the linguistic variables are converted to continuous values (real
outputs of the system) [14]. We developed membership functions and fuzzy rules for
each parameter considered. The fuzzified features extracted from bio- and activity
sensors are used as input by the classifier, a Kohonen's self-organizing ANN based on
unsupervised learning [15].
Figure 3. Architecture of the Decision Support System.
As shown in Fig. 4, data processing in the decision module consists of a training
phase and a test phase. In the training phase, the self-organizing map is trained to adapt
itself to classify the given inputs. The loaded features, along with self- and clinical-
reported stress levels, form the training set. For this purpose, the following
psychological questionnaires are used immediately after virtual exposure: i) Visual
Analogue Scale for Anxiety (VAS-A, self-rated by the patient); ii) State Anxiety
Inventory Y1 (STAI-Y1, self-rated); iii) Cognitive Behavioral Checklist (clinical-rated).
In this way, synaptic weights of networks internal connections are modified, with the
help of a learning algorithm, in order to force the output to minimize the error with the
presented example (the stress score obtained from these surveys). In the test phase, the
fuzzified features are given as an input to the network. The ANN, adequately trained, is
able to classify the given input in order to present a consequent output value. The value
obtained (output of ANN), is the inferred stress level. When the decision module infers
a new stress level value, this is uploaded again to the remote database (Fig. 2). Patient’s
A.Gaggioliet al./ADSSforReal-Time Stress Detection During Virtual Reality Exposure118
stress levels based on DSS reports values are then graphically displayed on a diagram,
which is accessible by the therapist through a Web-based interface.
Figure 4. The Decision Support System process, from data acquisition to classification of global stress level.
2. Conclusions and Discussion
Here, we have described a decision support system that assists the therapist in
analyzing and interpreting patient’s stress levels during VR exposure. The DSS is
based on multi-parametric measures physiological and behavioral measures collected
from different wireless biosensor modules. This strategy allows a closed-loop approach
actually missing in current strategies to the evaluation and treatment of psychological
stress [16]: a) the assessment is conducted continuously throughout the virtual
experiences: it enables tracking of the individual’s psycho-physiological status over
time in the context of a realistic stressful situation; b) the information provided by the
DSS is constantly used by the therapist to improve both the appraisal and the coping
skills of the patient. To test and validate the system, we have recently started a
controlled clinical trial, approved by the US database (Management
and Treatment of Stress-related Disorders - INTERSTRESS NCT01683617).
3. Acknowledgments
This work was supported by the European funded project ‘‘Interstress-Interreality in
the management and treatment of stress-related disorders’’, FP7-247685.
[1] D. Meichenbaum and R. Novaco, Stress inoculation: a preventative approach, Issues Ment Health Nurs
7 (1985), 419-435.
A.Gaggioliet al./ADSSforReal-Time Stress Detection During Virtual Reality Exposure 119
[2] M.C. Stetz, M.L. Thomas, M.B. Russo, T.A. Stetz, R.M. Wildzunas, J.J. McDonald, B.K. Wiederhold,
and J.A. Romano, Jr., Stress, mental health, and cognition: a brief review of relationships and
countermeasures, Aviat Space Environ Med 78 (2007), 252-260.
[3] S. Popovic, M. Horvat, D. Kukolja, B. Dropuljic, and K. Cosic, Stress inoculation training supported by
physiology-driven adaptive virtual reality stimulation, Stud Health Technol Inform 144 (2009), 50-54.
[4] G. Riva, S. Raspelli, D. Algeri, F. Pallavicini, A. Gorini, B.K. Wiederhold, and A. Gaggioli, Interreality
in practice: bridging virtual and real worlds in the treatment of posttraumatic stress disorders,
Cyberpsychol Behav Soc Netw 13(1) (2010), 55-65.
[5] G. Riva, A. Gaggioli, A. Grassi, S. Raspelli, P. Cipresso, F. Pallavicini, C. Vigna, A. Gagliati, S. Gasco,
and G. Donvito, NeuroVR 2--a free virtual reality platform for the assessment and treatment in
behavioral health care, Stud Health Technol Inform 163 (2011), 493-495.
[6] N. Carbonaro, G. Anania, G. Dalle Mura, M. Tesconi, A. Tognetti, G. Zupone, and D. De Rossi,
Wearable biomonitoring system for stress management: A preliminary study on robust ECG signal
processing, in: The 12th IEEE International Symposium on a World of Wireless, Mobile and
Multimedia Networks (2011), 1-6.
[7] N. Carbonaro, A. Tognetti, G. Anania, D. De Rossi, P. Cipresso, A. Gaggioli, G. Riva, "A mobile
biosensor to detect cardiorespiratory activity for stress tracking," Pervasive Computing Technologies
for Healthcare (PervasiveHealth), 2013 7th International Conference on , vol., no., pp.440,445, 5-8
May 2013
[8] G. Ruffini, S. Dunne, E. Farres, I. Cester, P.C. Watts, S.P. Silva, C. Grau, L. Fuentemilla, J. Marco-
Pallares, and B. Vandecasteele, Enobio dry electrophysiology electrode; first human trial plus wireless
electrode system, Conf Proc IEEE Eng Med Biol Soc (2007), 6690-6694.
[9] A. Riera, A. Soria-Frisch, A. Albajes-Eizagirre, P. Cipresso, C. Grau, S. Dunne, G. Ruffini, Electro-
physiological data fusion for stress detection, Stud Health Technol Inform 181 (2012), 228-32.
[10] R.S. Lewis, N.Y. Weekes, T.H. Wang, The effect of a naturalistic stressor on frontal EEG asymmetry,
stress, and health, Biol Psychol 75(3) (2007) 239-47.
[11] D.O. Bos, EEG-based Emotion Recognition. Emotion 57 (2006), 1798-806.
[12] A.F. Bobick, J.W. Davis, The recognition of human movement using temporal templates, IEEE
Transactions on Pattern Analysis and Machine Intelligence 23 (2001) 257–267.
[13] D. Giakoumis, A. Drosou, P. Cipresso, D. Tzovaras, G. Hassapis, et al., Using Activity-Related
Behavioural Features towards More Effective Automatic Stress Detection, PLoS ONE 7(9) (2012):
e43571. doi:10.1371/journal.pone.0043571
[14] B. Kosko, Fuzzy thinking: the new science of fuzzy logic, Hyperion, New York, 1993.
[15] T. Kohonen, Self-Organizing Maps, Springer Verlag, Berlin, 1995.
[16] S. Serino, S. Triberti, D. Villani, P. Cipresso, A. Gaggioli, G. Riva, Toward a validation of cyber-
interventions for stress disorders based on stress inoculation training: a systematic review, Virtual
Reality (in press) DOI 10.1007/s10055-013-0237-6.
A.Gaggioliet al./ A DSS for Real-Time Stress Detection During Virtual Reality Exposure120
... The time sequence of psychological stress was studied through evoked potentials in an effectively designed stress-stimulation model [6]. In the perspective of objective identification and differentiation of stress from other settings by EEG-related approaches, many computational techniques have been applied, such as support vector machine [7,8], K-nearest neighbors (KNN) [9,10], artificial neural networks (ANNs) [11,12] and random forest [13]. ...
... EEG signals have abundant features about the brain dynamics. Analysis in the frequency domain of EEG signals splits the data in various frequency bands, e.g., delta (1-4 Hz), theta (4-8 Hz), alpha (8)(9)(10)(11)(12), beta (12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30) and gamma (30-40 Hz). These frequency bands describe many cognitive behaviors. ...
... The average classification accuracy was about 95% using K-ELM and features extracted from physiological signals. Similar approach is presented in [93], where the authors describe a Decision Support System (DSS) for automatic classification of stress levels during exposure to VR environments. Proposed system integrates ECG, RIP and EEG sensors as well as behavioral data (body gestures tracking with Kinect). ...
Full-text available
The goal of this research is to integrate Virtual Reality (VR) with the bilateral stimulation used in EMDR as a tool to relieve stress. We created a 15 minutes relaxation training program for adults in a virtual, relaxing environment in form of a walk in the woods. The target platform for the tool is HTC Vive, however it can be easily ported to other VR platforms. An integral part of this tool is a set of sensors, which serves as physiological measures to evaluate the effectiveness of such system. What is more, the system integrate visual (passing sphere), auditory (surround sound) and tactile signals (vibration of controllers). A pilot treatment programme, incorporating the above mentioned VR system, was carried out. Experimental group consisting of 28 healthy adult volunteers (office workers), participated in three different sessions of relaxation training. Before starting, baseline features such as subjectively perceived stress, mood, heart rate, galvanic skin response and muscle response were registered. The monitoring of physiological indicators is continued during the training session and one minute after its completion. Before and after the session, volunteers were asked to re-fill questionnaires regarding the current stress level and mood. The obtained results were analyzed in terms of variability over time: before, during and after the session.
... This approach does not exploit data modeling and learning techniques, so knowledge elicitation represents a major drawback. Gaggioli et al. [83] have developed a DSS for real-time stress detection to track users experienced stress during Stress Inoculation Training (SIT) procedures by exploiting the subject's exposure to virtual reality (VR) environments. The DSS integrates ECG, EEG, breathing rate (BR), and body gestures collected using multiple sensing platforms in order to extract several features, and it is built by using fuzzy logic in conjunction with supervised SOMs. ...
Full-text available
Personalised training of motor and cognitive abilities is fundamental to help older people maintain a good quality of life, especially in case of frailty conditions. However, the training activity can increase the stress level, especially in persons affected by a chronic stress condition. Wearable technologies and m-health solutions can support the person, the medical specialist, and long-term care facilities to efficiently implement personalised therapy solutions by monitoring the stress level of each subject during the motor and cognitive training. In this paper we present a comprehensive work on this topic, starting from a pilot study involving a group of frail older adults suffering from Mild Cognitive Impairment (MCI) who actively participated in cognitive and motor rehabilitation sessions equipped with wearable physiological sensors and a mobile application for physiological monitoring. We analyse the collected data to investigate the stress response of frail older subjects during the therapy, and how the cognitive training is positively affected by physical exercise. Then, we evaluated a stress detection system based on several machine learning algorithms in order to highlight their performances on the real dataset we collected. However, stress detection algorithms generally provide only the identification of a stressful/non stressful event, which is not sufficient to personalise the therapy. Therefore, we propose a mobile system architecture for online stress monitoring able to infer the stress level during a session. The obtained result is then used as input for a Decision Support System (DSS) in order to support the medical user in the definition of a personalised therapy for frail older adults.
... Among others, the models will encode user behavior information derived from unobtrusive user activity monitoring, as well as user affect recognition. User activity monitoring will be based on unobtrusive sensing approaches; indicatively, action recognition based on the accelerometers of the user's smart-watch [17], audio signals processing [18], as well as further sources, related e.g. to biosignals processing for user emotion recognition [19,20,21]. The VUMs then will be used to represent users in automatic simulations to support decision making processes [22]. ...
Conference Paper
Full-text available
Along with population ageing comes the increasingly intensified phenomenon of a shrinking and ageing workforce. Novel solutions are needed so as to help ageing workers maintain workability and productivity, along with a balance between work and personal life, which supports them into good quality of life, active and healthy ageing. In this line, the "[email protected]" project, initiated by the European Union, develops a novel ICT-based, personalized system to support ageing workers (aged 50+) into designing fit for purpose work environments and managing flexibly their evolving needs. On top of personalized, dynamically adapted worker and workplace models, computational intelligence will assess user specificities and needs i.r.t. work conditions, both in terms of ergonomics, health and safety issues and task assignments. Recommendations will then be provided both to the worker and company, under strict privacy restrictions, on how the working conditions must adapt. The worker models will be populated by unobtrusive worker sensing, both at work, at home and on the move. To foster workability and productivity, personalized, intuitive, age-friendly productivity, co-design enhancement tools will be developed, including ones for AR/VR-based context-awareness and telepresence, lifelong learning and knowledge sharing. On top of these, a novel Ambient Virtual Coach (AVC) will encompass an empathic mirroring avatar for subtle notifications provision, an adaptive Visual Analytics - based personal dashboard, and a reward-based motivation system targeting positive and balanced worker behavior at work and personal life, towards a novel paradigm of ambient support into workability and well-being. The integrated system will be developed by user-centered design and will be evaluated at two pilot sites, related to core Industry 4.0 processes of mining and machines production.
Full-text available
Virtual reality (VR) is a potentially powerful technology for enhancing assessment in mental health. At any time or place, individuals can be transported into immersive and interactive virtual worlds that are fully controlled by the researcher or clinician. This capability is central to recent interest in how VR might be harnessed in both treatment and assessment of mental health conditions. The current review provides a summary of the advantages of using VR for assessment in mental health, focusing on increasing ecological validity of highly controlled environments, enhancing personalization and engagement, and capturing real-time, automated data in real-world contexts. Considerations for the implementation of VR in research and clinical settings are discussed, including current issues with cost and access, developing evidence base, technical challenges, and ethical implications. The opportunities and challenges of VR are important to understand as researchers and clinicians look to harness this technology to improve mental health outcomes. .
Full-text available
This issue gathers all the articles presented at the scientific conference ConVRgence (VRIC) which took place during Laval Virtual World 2020 (April 22-24).
Stress is inevitably experienced by almost every person at some stage of their life. A reliable and accurate measurement of stress can give an estimate of an individual's stress burden. It is necessary to take essential steps to relieve the burden and regain control for better health. Listening to music is a way that can help in breaking the hold of stress. This study examines the effect of music tracks in English and Urdu language on human stress level using brain signals. Twenty-seven subjects including 14 males and 13 females having Urdu as their first language, with ages ranging from 20 to 35 years, voluntarily participated in the study. The electroencephalograph (EEG) signals of the participants are recorded, while listening to different music tracks by using a four-channel MUSE headband. Participants are asked to subjectively report their stress level using the state and trait anxiety questionnaire. The English music tracks used in this study are categorized into four genres i.e., rock, metal, electronic, and rap. The Urdu music tracks consist of five genres i.e., famous, patriotic, melodious, qawali, and ghazal. Five groups of features including absolute power, relative power, coherence, phase lag, and amplitude asymmetry are extracted from the preprocessed EEG signals of four channels and five bands, which are used by the classifier for stress classification. Four classifier algorithms namely sequential minimal optimization, stochastic decent gradient, logistic regression (LR), and multilayer perceptron are used to classify the subject's stress level into two and three classes. It is observed that LR performs well in identifying stress with the highest reported accuracy of 98.76% and 95.06% for two- and three-level classification respectively. For understanding gender, language, and genre related discriminations in stress, a t-test and one-way analysis of variance is used. It is evident from results that English music tracks have more influence on stress level reduction as compared to Urdu music tracks. Among the genres of both languages, a noticeable difference is not found. Moreover, significant difference is found in the scores reported by females as compared to males. This indicates that the stress behavior of females is more sensitive to music as compared to males.
Full-text available
Here we introduce the design and preliminary validation of a general-purpose architecture for affective-driven procedural content generation in Virtual Reality (VR) applications in mental health and wellbeing. The architecture supports seven commercial physiological sensing technologies and can be deployed in immersive and non-immersive VR systems. To demonstrate the concept, we developed the "The Emotional Labyrinth", a non-linear scenario in which navigation in a procedurally-generated 3D maze is entirely decided by the user, and whose features are dynamically adapted according to a set of emotional states. During navigation, affective states are dynamically represented through pictures, music, and animated visual metaphors chosen to represent and induce affective states. The underlying hypothesis is that exposing users to multimodal representations of their affective states can create a feedback loop that supports emotional self-awareness and fosters more effective emotional regulation strategies. We carried out a first study to (i) assess the effectiveness of the selected metaphors in inducing target emotions, and (ii) identify relevant psycho-physiological markers of the emotional experience generated by the labyrinth. Results show that the Emotional Labyrinth is overall a pleasant experience in which the proposed procedural content generation can induce distinctive psycho-physiological patterns, generally coherent with the meaning of the metaphors used in the labyrinth design. Further, collected psycho-physiological responses such as electrocardiography, respiration, electrodermal activity, and electromyography are used to generate computational models of users' reported experience. These models enable the future implementation of the closed loop mechanism to adapt the Labyrinth procedurally to the users' affective state.
Conference Paper
Full-text available
Stress is an increasingly recognized phenomenon that has negative effects on growing numbers of people. Stress assessment is a complex issue, but different studies have shown that monitoring user psychophysiological parameter during daily life can be greatly helpful in stress evaluation. In this study a wearable biosensor platform able to collect physiological and behavioral parameters is reported. The developed wearable platform, in terms of hardware and processing algorithms, is described. Moreover the use of this wearable biosensor platform in combination with advanced simulation technologies, such as virtual reality offer interesting opportunities for innovative personal health-care solutions to stress. A recently founded European project, "INTERSTRESS - Interreality in the management and treatment of stress-related disorders," will take into account these relevant aspects.
Full-text available
New advanced technologies have recently emerged as a potentially effective way for delivering stress management techniques. Specifically, the stress inoculation training (SIT) represents a validated approach to manage stress in several settings, and research is growing related to this clinical protocol combined with advanced technologies. This review aims to outline the state of the art of cyber-interventions based on SIT methodology (cyberSIT). In the current review, we deeply analyzed and discussed three aspects of the selected studies: (1) the type of technological devices used for delivering cyber-SIT; (2) the sampling strategies; (3) and the stress-related measures for assessing the effectiveness of cyber-SIT. The results of this systematic review suggest the potential efficacy of cyber-SIT for managing psychological stress in several settings. Considering cyber-SIT for psychological stress, controlled trials testing a greater number of participants are needed. Other future challenges include adopting better inclusion/exclusion criteria, standardized outcome measures, and different conditions for comparing the effect and/or the integration of different technological devices. In conclusion, as the cyber-SIT may play an important role in the future clinical psychology, it is crucial to enhance the validation of this approach from a methodological point of view.
Full-text available
Making the computer more empathic to the user is one of the aspects of aective computing. With EEG-based emotion recognition, the computer can actually take a look inside the user's head to observe their mental state. This paper describes a research project conducted to rec- ognize emotion from brain signals measured with the BraIn- quiry EEG PET device. Firstly, literature research has been performed to establish a suitable approach and determine optimal placement of a limited number of electrodes for emo- tion recognition. The second step consisted of putting this information into practice with the BraInquiry apparel dur- ing the experimental phase, which finally was analyzed to see what results could be achieved.
Full-text available
This paper introduces activity-related behavioural features that can be automatically extracted from a computer system, with the aim to increase the effectiveness of automatic stress detection. The proposed features are based on processing of appropriate video and accelerometer recordings taken from the monitored subjects. For the purposes of the present study, an experiment was conducted that utilized a stress-induction protocol based on the stroop colour word test. Video, accelerometer and biosignal (Electrocardiogram and Galvanic Skin Response) recordings were collected from nineteen participants. Then, an explorative study was conducted by following a methodology mainly based on spatiotemporal descriptors (Motion History Images) that are extracted from video sequences. A large set of activity-related behavioural features, potentially useful for automatic stress detection, were proposed and examined. Experimental evaluation showed that several of these behavioural features significantly correlate to self-reported stress. Moreover, it was found that the use of the proposed features can significantly enhance the performance of typical automatic stress detection systems, commonly based on biosignal processing.
Conference Paper
Full-text available
In this work we describe the performance evaluation of a system for stress detection. The analysed data is acquired by following an experimental protocol designed to induce cognitive stress to the subjects. The experimental set-up included the recording of electroencephalography (EEG) and facial (corrugator and zygomatic) electromyography (EMG). In a preliminary analysis we are able to correlate EEG features (alpha asymmetry and alpha/beta ratio using only 3 channels) with the stress level of the subjects statistically (by using averages over subjects) but also on a subject-to-subject basis by using computational intelligence techniques reaching classification rates up to 79% when classifying 3 minutes takes. On a second step, we apply fusion techniques to the overall multi-modal feature set fusing the formerly mentioned EEG features with EMG energy. We show that the results improve significantly providing a more robust stress index every second. Given the achieved performance the system described in this work can be successfully applied for stress therapy when combined with virtual reality.
Conference Paper
Stress is an increasingly recognized phenomenon that has negative effects on growing numbers of people. Stress assessment is a complex issue, but different studies have shown that monitoring user psychophysiological parameter during daily life can be greatly helpful in stress evaluation. In this study a wearable biosensor platform able to collect physiological and behavioral parameters is reported. The developed wearable platform, in terms of hardware and processing algorithms, is described. Moreover the use of this wearable biosensor platform in combination with advanced simulation technologies, such as virtual reality offer interesting opportunities for innovative personal health-care solutions to stress. A recently founded European project, "INTERSTRESS - Interreality in the management and treatment of stress-related disorders," will take into account these relevant aspects.
Conference Paper
There is a close correlation between stress and health risk factors such as poor immune function and cardio- vascular problems. Various researches showed that long-term exposure to stress and its related diseases are responsible of dramatic increase of mortality in the Western Countries. In this context, the European Collaborative Project INTERSTRESS is aimed at designing and developing advanced simulation and sensing technologies for the assessment and treatment of psychological stress, based on mobile biosensors. In this paper a wearable system able to implement the acquisition and the real-time elaboration of the ECG signal for stress management purposes will be described. A novel and robust algorithm for QRS complex detection has been developed. Robust QRS detection is fundamental to evaluate Heart Rate and Heart Rate Variability that are relevant parameters used as quantitative marker related to mental stress. In comparison to existing solutions the realized algorithm presents many advantages: an adaptive optimal filtering technique that avoids the use of thresholds and empirical rules for R peaks detection, low computational cost for real time elaboration and good tollerance with noisy ECG signal. Keywords-kalman filter, QRS detection, Heart Rate, ECG signal.