Conference PaperPDF Available

Comparing affective responses to standardized pictures and videos: A study report


Abstract and Figures

Multimedia documents such as text, images, sounds or videos elicit emotional responses of different polarity and intensity in exposed human subjects. These stimuli are stored in affective multimedia databases. The problem of emotion processing is an important issue in Human-Computer Interaction and different interdisciplinary studies particularly those related to psychology and neuroscience. Accurate prediction of users' attention and emotion has many practical applications such as the development of affective computer interfaces, multifaceted search engines, video- on-demand, Internet communication and video games. To this regard we present results of a study with N=10 participants to investigate the capability of standardized affective multimedia databases in stimulation of emotion. Each participant was exposed to picture and video stimuli with previously determined semantics and emotion. During exposure participants' physiological signals were recorded and estimated for emotion in an off-line analysis. Participants reported their emotion states after each exposure session. The a posteriori and a priori emotion values were compared. The experiment showed, among other reported results, that carefully designed video sequences induce a stronger and more accurate emotional reaction than pictures. Individual participants' differences greatly influence the intensity and polarity of experienced emotion.
Content may be subject to copyright.
Comparing affective responses to standardized pictures and videos: A study
Marko Horvat1, Davor Kukolja2 and Dragutin Ivanec3
1Polytechnic of Zagreb, Department of Computer Science and Information Technology
2University of Zagreb, Faculty of Electrical Engineering and Computing, Department of Electric Machines, Drives and
3University of Zagreb, Faculty of Humanities and Social Sciences, Department of Psychology
Cite as: M. Horvat, D. Kukolja, and D. Ivanec, “Comparing affective responses to standardized pictures and videos: A study report,” In
MIPRO, 2015 Proceedings of the 38th International Convention, IEEE, pp. 1394 1398, May 2015.
Abstract - Multimedia documents such as text, images,
sounds or videos elicit emotional responses of different
polarity and intensity in exposed human subjects. These
stimuli are stored in affective multimedia databases. The
problem of emotion processing is an important issue in
Human-Computer Interaction and different interdisciplinary
studies particularly those related to psychology and
neuroscience. Accurate prediction of users’ attention and
emotion has many practical applications such as the
development of affective computer interfaces, multifaceted
search engines, video-on-demand, Internet communication
and video games. To this regard we present results of a study
with N=10 participants to investigate the capability of
standardized affective multimedia databases in stimulation of
emotion. Each participant was exposed to picture and video
stimuli with previously determined semantics and emotion.
During exposure participants’ physiological signals were
recorded and estimated for emotion in an off-line analysis.
Participants reported their emotion states after each exposure
session. The a posteriori and a priori emotion values were
compared. The experiment showed, among other reported
results, that carefully designed video sequences induce a
stronger and more accurate emotional reaction than pictures.
Individual participants’ differences greatly influence the
intensity and polarity of experienced emotion.
Any multimedia file can generate positive, negative or
neutral emotions of varying intensity and duration [1]. By
observing still images, films, printed text or listening to
sounds, music and voices emotional states of affected
subjects may be modulated [2] [3]. This spontaneous
cognitive process is an important research topic in
psychology, neuroscience and cognitive sciences but also
in many interdisciplinary domains like Affective
Computing and Human-Computer Interaction (HCI).
Multimedia documents with a priori annotated semantic
and emotion content are stored in affective multimedia
databases and are intended for inducing or stimulating
emotions in exposed subjects. Because of their purpose
such multimedia documents are also referred to as stimuli.
Affective multimedia databases are standardized which
allows them to be used in a controllable and predictable
manner: the emotion elicitation results can be measured,
replicated and validated by different research teams [4] [5].
Combined with immersive and unobtrusive visualization
hardware in low-interference ambient affective multimedia
databases provide a simple, low-cost and efficient means to
investigate a wide range of emotional reactions [6] [7].
Compared to static pictures and sound, video is more
powerful format for elicitation of emotional states because
it can seamlessly and concurrently stimulates visual and
auditory senses thereby multiplying their individual
impacts through psychophysiological and neurological
underlying mechanisms [8] [9]. It has been repetitively
experimentally demonstrated that if careful attention is
paid to video editing, i.e. the temporal and contextual
alignment of multimedia stream relative to personal
cognitions of targeted subjects, it is possible to achieve
more intense and accurate stimulation of emotional states
and related phenomena such as attention, anxiety and stress
[7]. In practical terms affective video databases are much
more useful tools than picture of sound databases.
However, today very few affective video databases exist
while the most prevalent are picture databases. Therefore,
it is important to explore the possibility and scope of using
documents from existing picture and sound databases to
construct successful video sequences for fast, accurate and
strong stimulation of emotional states. This goal was
behind the motivation for the described study.
The remainder of this paper is organized as follows;
Section 2 provides background information about the
experimental study and its setup. Section 3 brings forward
the results of the study which are discussed in Section 4.
Finally, Section 5 concludes the paper and outlines future
work into this subject.
The study was performed at University of Zagreb,
Faculty of Electrical Engineering and Computing in
cooperation with experts from Department of Psychology,
Faculty of Humanities and Social Sciences. A
homogeneous group of N=10 college students (4 males, 6
women) with an average age 23.8 years (std = 4.35)
participated in the experiment.
Each participant was stimulated with videos and still
images taken from the International Affective Picture
System (IAPS) [4] and the International Affective Digital
Sounds System (IADS) [5]. IAPS and IADS are two of the
most cited databases in the area of affective stimulation.
These databases were created with three goals in mind: i)
better experimental control of emotional stimuli, ii)
increasing the ability of cross-study comparisons of results,
and iii) facilitating direct replication of undertaken studies
[10]. In this experiment a picture from IAPS and a sound
from IADS were combined to make one video-clip. Same
IAPS pictures were also used as still image stimuli without
sounds. Some of the pictures used to construct the emotion
elicitation sequences are shown in Fig. 1.
Fig. 1. A sample of IAPS pictures used as emotion eliciting video
clips and images. Neutral (left column), fear (middle) and
happiness dominant emotion stimuli (right).
The dominant emotions purposely elicitated in the
experiment were happiness and fear. The stimuli were
selected using published research on emotion norms in
IAPS and IADS [11] [12] as the most powerful images and
sounds for simulation of the two targeted emotions. Firstly,
using 200 images and 100 sounds were selected and ranked
based on their intensity of happiness and fear emotion
norms [11] [12]. Secondly, the sorted list was thoroughly
examined by psychology experts and 20 optimal pictures
and 20 sounds were manually selected for inclusion in the
elicitation sequences. These stimuli were considered the
most likely to induce happiness and fear in the
participants’ population.
Each participant was exposed to four emotion elicitation
sequences in two separate sessions or series. Each session
consisted of one happiness sequence and one fear inducing
sequence, and also of one video sequence and one still
image sequence. A single sequence was made up from 10
images or 10 video-clips (Fig. 2). Therefore, in total each
participant was exposed to 20 images and 20 video-clips.
The length of each stimulus was exactly 15 seconds after
which the participant was shown a blank neutral screen and
had to write down his affective judgments in a self-
assessment questionnaire (SAQ). The expression of
subjects’ ratings was not time restricted after which the
participant could resume the sequence by himself (i.e. with
a mouse click). Immediately before the start of the
experiment each participant was separately introduced to
the stimulation protocol with a neutral sequence. The
neutral sequence consisted of one low arousal and valence
picture (N Pic) and one video-clip without dominant
emotions (N Video). All stimulation sequences are
available by contacting the first author.
Half of the participants were first exposed to happiness
sequences, and then fear sequences, and also a half of the
participants first watched still images and then videos. To
prevent the unwanted drift of physiological signals (cardiac
and respiratory) before nonneutral sequences participants
were exposed to a neutral stimulus until their baseline
response was established [13]. The neutral blank screen
only showed teal color which − according to [14] − has an
optimal ratio of stimulating positive and negative
The participants’ emotional responses were recorded by
two methods: 1) self-assessment responses i 2) real-time
monitoring of physiological signals. After each exposure to
a stimulus participants filled out a self-assessment
questionnaire. Each report was unique to a specific
stimulus and participants could not see its contents before
the stimulus has finished. The test contained following
instructions: 1) “Evaluate the type and intensity of
emotions” for each of the emotional norms (“happiness”,
‘sadness”, “disgust”, “fear” and “some other emotion” if
none of previous) on a scale with values 0 − 9, where 0
represented “None”, 9 “Extremely” while 5 was a neutral
value; and 2) “Evaluate pleasure and arousal” with values -
4 4 where value -4 was labeled “Extremely unpleasant”
and “Extremely calming”, and 4 “Extremely pleasant” and
“Extremely arousing”. Value 0 indicated a neutral
sensation of valence or arousal. The report was validated
during the preparations for the experiment.
The monitored physiological signals were skin
conductance, electrocardiogram (ECG), respiration and
skin temperature with a sampling frequency of 1250 Hz.
For the acquisition of signals we used BIOPAC MP150
with AcqKnowledge software. The system was
synchronized with SuperLab tool for presentation of
stimuli to the participants. Emotional states were estimated
off-line, with varying levels of certainty, from the recorded
signals using previously developed MATLAB software
[15] [16]. Before starting the experiment, each participant
read the instructions, filled introductory questionnaire and
signed informed consent agreement. Additional help, if
necessary, was provided by the trained lab assistant who
also placed physiological sensors on the participant’s body.
Fig. 2. The timeline of emotion stimulation paradigm. The order of dominant emotion (happiness/fear) and multimedia format
(picture/video) were randomized.
Participants were seated in a separate part of the lab, 60
90 cm before a 19" 4:3 high-definition LCD computer monitor and wore ergonomic headphones. The supervisor
station was equipped with a separate computer monitor
where the experiment was controlled and participants’
psychological signals were monitored in real-time [16].
The experimenter and participants were physically
separated by a solid artificial wall and had no mutual
visual or auditory contact. Additionally, participants did
not experience sensory distractions during the experiment.
The implemented procedures and the experimental layout
were compatible with a setup of a common
psychophysiological laboratory [17].
Fig. 3. The layout of participants’ station. A person is observing
LCD screen with audio headphones and wearing skin
conductance, electrocardiogram, respiration and skin temperature
contact sensors connected to BIOPAC MP150 system. Self-
assessment questionnaire is on the desk. Supervisor is seated on
the other side of the wall barrier.
A rest period followed each exposure session during
which participants relaxed. This was verified by examining
the physiological signal parameters that were visualized in
real time. The exposure could resume only after the
baseline signal levels were reestablished.
The potential of evoking emotional reactions using
video clips − constructed from IAPS pictures and IADS
sounds and IAPS pictures was evaluated under equal
conditions. Emotional dimensions pleasure and arousal
were rated on a scale 1 9, and emotion norms (i.e.
discrete emotions) on a scale 1 10. A lower value (1 − 3)
in both scales implies a lesser experience and a higher
value (7 10) a more intense experience of the particular
emotional dimension or norm, respectively. Also, higher
frequency of ratings in the upper part of the scale (response
> 5) or the highest attainable values (9 and 10) signifies a
more intense and powerful stimulation. The frequencies of
responses are displayed as frequency distribution diagrams.
The aggregated results are shown in Fig. 4.
Results based on the analysis of participants’ self-
assessments indicate that the most pronounced reported
difference is in arousal emotion dimension. Indeed, videos
relative to pictures can more frequently stimulate higher
arousal in sequences with dominant happiness and fear.
Additionally, video sequences often stimulated higher
levels of pleasure in happiness sequences, and lower levels
of pleasure in fear sequences, but this distinction is less
pronounced than with arousal.
From Fig. 4 it is evident that video sequences were more
powerful in stimulation of both emotional dimensions than
picture sequences. Happiness-dominant video sequences
more often elicited higher levels of happiness basic
emotion than happiness-dominant picture sequences.
Similarly, fear-dominant video sequences provoked more
above average fear ratings than fear-dominant picture
sequences. Although frequency distribution differences in
basic emotions are present, they are less obvious than the
spread in emotional dimensions, especially arousal. In
general, the emotion provoking potential of video is more
apparent in emotional valence and arousal than in specific
emotions happiness, sadness, disgust and fear. Due to
reported low stimulation of other emotions except fear and
happiness it may be concluded that the sequences were
emotionally accurate. This is particularly evident in very
low levels of disgust and sadness in happiness sequences
and even in fear sequences. In overall video sequences
provoked a “cleaner” and more powerful affective
response, i.e. with lower reported intensities of emotions
different from those targeted, than picture sequences which
corresponds well with findings from previous studies [13]
However, because of relatively small number of
participants the results analysis is strongly influenced with
noticable differences between individual reports. There is a
significant variability in the intensity of provoked emotion,
polarity (i.e. positive or negative) and discrete category
among some participants. Such discrepancies are present in
video and picture sequences.
Unfortunately, due to objective reasons it was not
possible to include more participants in the study.
Subsequently, an independent stimulation protocol could
not be implemented and the same visual stimuli had to be
used in video and image sessions. If the number of
participants was significantly larger different stimuli could
be used in videos and images.
Based on the collected results it can be expected that
multimedia sequences, carefully prepared for a particular
group of participants, will be able to provoke targeted
emotional states with the desired intensity. However,
construction of optimal sequences proved to be difficult
because IAPS, and particularly, IADS databases do not
have a wide selection of stimuli with accentuated specific
basic emotions. A better choice of provoking visual stimuli
is clearly needed which encourages construction of more
affective multimedia databases, annotated both with
emotional dimensions and discrete emotions, and having a
large semantic space. This in turn necessitates
development of powerful tools for multimedia stimuli
retrieval which can efficiently perform multifaceted search
in such databases, along several emotional, semantic and
contextual data dimensions, thus assisting researchers in
finding optimal stimuli for personalized emotion elicitation
sequences [19].
Fig. 4. Frequency distribution of N=10 participants ratings after elicitation with IAPS and IADS video clips (“Video”) and IAPS
pictures (“Pictures”). Reported emotional dimensions arousal and pleasure (upper and middle rows), discrete emotions happiness and
fear (bottom row).
Emotional reactions can be induced by virtually any
multimedia format: films, pictures, sounds, voice and even
text. Participants’ responses depend not only on stimuli
content, properties and type, but also on a number of
intrinsic and extrinsic factors which may be difficult to
control in a practical experiment [13] [17]. For example,
participants motivation, attitude, character traits, beliefs,
past experiences and the experimental environment (i.e. the
setup of the laboratory and the experimental protocol) play
an extremely important role in formation of emotion.
Therefore, a comparison of emotional reactions induced by
pictures and videos may be regarded as a hands-on tutorial
for researchers as to which multimedia stimuli properties
can lead to more accurate, precise and faster elicitation.
The study showed that standardized picture and sound
databases can be joined together and used as videos for
elicitation of high-arousal emotional states. Deliberate and
practical stimulation of discrete emotions happiness and
fear is attainable but it is more difficult and prone to error,
especially happiness. The least successful was stimulation
of very positive, i.e. high valence, emotional states.
We hope that the presented study could be used in
design of emotion elicitation protocols as well as future
affective multimedia databases. Additionally, the study’s
results may help researchers to find the optimal multimedia
format for elicitation of emotion even if appropriate video
stimuli are not available.
[1] S. Brave and C. Nass, Emotion in human-computer
interaction,” The human-computer interaction handbook:
fundamentals, evolving technologies and emerging
applications, CRC Press, Taylor & Francis, Florida, pp.
81−96, 2002.
[2] J. A. Coan and J. J. B. Allen, The Handbook of Emotion
Elicitation and Assessment,” Oxford University Press
Series in Affective Science, Oxford University Press, USA,
[3] D. Grandjean, D. Sander, and K. R. Scherer, Conscious
emotional experience emerges as a function of multilevel,
appraisal-driven response synchronization,” Consciousness
and Cognition, vol. 17, pp. 48495, 2008.
[4] P. J. Lang, M. M. Bradley, and B. N. Cuthbert,
“International affective picture system (IAPS): Affective
ratings of pictures and instruction manual,” Technical
Report A−8, University of Florida, Gainesville, FL, 2008.
[5] P. J. Lang and M. M. Bradley, “The International Affective
Digitized Sounds (2nd Edition; IADS-2): affective ratings
of sounds and instruction manual,” Technical report B-3,
University of Florida, Gainesville, FL, 2007.
[6] J. J. Gross and R. W. Levenson, Emotion elicitation using
films,” Cognition & Emotion, vol. 9(1), pp. 87108, 1995.
[7] D. Villani and G. Riva, Does interactive media enhance
the management of stress? Suggestions from a controlled
study,” Cyberpsychology, Behavior, and Social
Networking, vol. 15(1), pp. 2430, 2012.
[8] J. T. Cacioppo, G. G. Berntson, J. T. Larsen, K. M.
Poehlmann, and T. A. Ito, The psychophysiology of
emotion,” Handbook of emotions, vol. 2, 173191, 2000.
[9] K. C. Berridge and M. L. Kringelbach, “Neuroscience of
Affect: Brain Mechanisms of Pleasure and Displeasure,”
Current opinion in neurobiology, vol. 23(3), pp. 294303,
[10] M. M. Bradley and P. J. Lang, Measuring emotion:
Behavior, feeling and physiology,” Cognitive neuroscience
of emotion, Oxford University Press, New York, pp. 242
276, 2000.
[11] T. M. Libkuman, H. Otani, R. Kern, S. G. Viger, and N.
Novak, Multidimensional normative ratings for the
international affective picture system,” Behavior Research
Methods, vol. 39(2), pp. 326−334, 2007.
[12] R. A. Stevenson and T. W. James, Affective auditory
stimuli: Characterization of the International Affective
Digitized Sounds (IADS) by discrete emotional
categories,” Behavior Research Methods, vol. 40(1), pp.
315−321, 2008.
[13] J. Rottenberg, R. D. Ray, and J. J. Gross, Emotion
elicitation using films,” Handbook of emotion elicitation
and assessment, Oxford University Press, New York, pp.
9−28, 2007.
[14] K. A. Y. A. NAz and H. Epps, Relationship between color
and emotion: a study of college students,” College Student
Journal, vol. 38(3), pp. 396−405, 2004.
[15] D. Kukolja, S. Popović, B. Dropuljić, M. Horvat, and K.
Ćosić, Real-time emotional state estimator for adaptive
virtual reality stimulation,” Foundations of Augmented
Cognition, Neuroergonomics and Operational
Neuroscience, Springer Berlin Heidelberg, pp. 175184,
[16] D. Kukolja, S. Popović, M. Horvat, B. Kovač, and K.
Ćosić, Comparative analysis of emotion estimation
methods based on physiological measurements for real-
time applications,” International journal of human-
computer studies, vol. 72(10), pp. 717−727, 2014.
[17] J. J. Curtin, D. L. Lozano, and J. J. Allen, The
psychophysiological laboratory,” Handbook of emotion
elicitation and assessment, Oxford University Press, New
York, pp. 398−425, 2007.
[18] C. G. Courtney, M. E. Dawson, A. M. Schell, A. Iyer, and
T. D. Parsons, Better than the real thing: Eliciting fear
with moving and static computer-generated stimuli,”
International Journal of Psychophysiology, vol. 78(2), pp.
107114, 2010.
[19] M. Horvat, N. Bogunović, and K. Ćosić, “STIMONT: a
core ontology for multimedia stimuli description,”
Multimedia Tools and Applications, vol. 73(3), pp.
1103−1127, 2014.
... A common denominator in research studying the neural basis of disgust is that it mostly relies on pictures from for example the International Affective Picture System (IAPS) or on movie clips to induce the aversive and repulsive response in participants (19,20). Although it is conceivable that participants over-report successful emotion induction as a form of obsequiousness bias (21,22), using pictures and videos for emotion induction has been shown to reliably trigger affective states (23). Nonetheless, there is still a marked difference in experiencing a threatening or disgusting stimulus only as a visual experience vs. experiencing said stimulus in real life, where the interaction with stimuli from the environment is usually involving multimodal sensory inputs. ...
Full-text available
Emotion induction in psychological and neuroscientific research has been mostly done by presenting participants with picture or film material. However, it is debatable whether this passive approach to emotion induction results in an affective state comparable to real-life emotions, and if the neural correlates of emotion processing are ecologically valid. To investigate the appropriateness of pictures for the induction of emotions, we presented 56 participants in a within-subjects design with naturalistic disgusting and neutral stimuli as well as with pictures of said stimulus material while recording continuous EEG data. We calculated asymmetry indices (AIs) for alpha power as an index of emotion processing and emotion regulation at the F3/4, F5/6, F7/8, and O1/2 electrode pairs. Participants reported higher disgust ratings for disgusting naturalistic compared to disgusting pictorial stimuli. Investigating changes in the EEG signal in participants with a pronounced disgust response (n = 38), we found smaller AIs for naturalistic stimuli compared to pictures. Moreover, in this disgusted sub-sample, there were smaller AIs in response to naturalistic disgusting stimuli compared to pictorial disgusting and neutral stimuli at the O1/2 electrode pair indicating stronger activation of the right relative to the left hemisphere by naturalistic stimuli. As the right hemisphere has been shown to display dominance in processing negative and withdrawal-associated emotions, this might indicate that naturalistic stimuli are more appropriate for the induction of emotions than picture stimuli. To improve the validity of results from emotion induction, future research should incorporate stimulus material that is as naturalistic as possible.
... Continuing with the implementation of the e-learning intelligent agent, it will be necessary to precisely define a rigorous experimentation protocol. Especially important will be to determine a set of rules that control cognition and behavior of homogenous groups and individuals in classrooms, in particular emotional responses to standardized images and video-clips [16]. We expect that establishing the ground-truth will greatly assist in personalization and a more effective learning. ...
Conference Paper
Full-text available
Because of the global COVID-19 pandemic, online learning has become the dominant teaching method. Moreover, a wide range of e-learning pedagogies are rapidly gaining importance, and in some cases emerging as the preferred approach in education over the traditional methods and techniques of classroom teaching. However much has to be done to efficiently assess student engagement and the learning curve. In this regard, we have proposed construction of an intelligent agent for personalized and adaptive assessment of learning performance based on methods for automated estimation of attention and emotion. We report on the first progress towards the development of the intelligent agent. Three classifiers were used in parallel to detect information about the progress of student engagement. Object detection in video is accomplished with YOLOv3, emotion detection from facial expressions using PAZ software library, and detection of head, arms, and upper-body orientation and position with OpenPose system. NimStim facial expression database, WIDER Attribute Dataset, and UPNA Head Pose Database were used for experimental validation of the individual classifiers. Our system attained the highest precision and recall of 79.13% and 94.15%, respectively, and the highest success rate of 59.56% in recognition of 6 discrete emotions from facial expressions.
... The aim of this paper is to evaluate the feasibility of detecting emotional reactions induced in virtual reality (VR) using a commercial EEG system. Previous studies have clearly demonstrated that elicitation of specific emotions with films, videos and pictures is challenging but possible [1][2] [3]. High stimuli realism, sufficient personal relevance, and strong engagement have been suggested as the most significant factors contributing to immersion and presence leading to a successful elicitation [1] [4][5] [6]. ...
Conference Paper
Full-text available
We report on a pilot study involving emotion elicitation in virtual reality (VR) and assessment of emotional responses with a consumer-grade EEG device. The stimulation used HTC Vive VR system showing pictures from NAPS database within a specifically designed virtual environment. The stimulation consisted of two distinct sequences with 10 pictures of happiness and 10 pictures of fear. Each picture was contained in a separate virtual room that the participants traveled through along a preset path. The estimation employed EMOTIV EPOC+ 14-channel EEG headset and a custom-developed application. The software wirelessly received EEG signals from alpha, beta low, beta high, gamma and theta bands, time-stamped them and dynamically stored in a relational database for subsequent analysis. Our preliminary results show that statistically significant correlations between valence and arousal ratings of pictures and EEG bands are present but highly personalized. Simultaneous correct placement of VR and EEG headsets is demanding and precise localization of electrodes is difficult. In fact, if emotion estimation is not strictly necessary we recommend using devices with fewer electrodes. Nevertheless, we found the EEG to be effective. By acknowledging its limitations, and using the headset in the correct context, experiments involving emotions may be significantly amended.
... Također, iz cjelokupne terapije se izbacuje utjecaj liječnika koji ne utječe direktno na izlječenje već postaje samo još jedan instrument u razumijevanju pacijentovih procesa [3] [4]. S druge strane, postoji velik broj navoda literature i dokumentiranih slučajeva koji podržavaju tezu o djelotvornosti obitelji biofeedback terapija koje koriste različite fiziološke signale (npr. [5] [6] [7]). Srčani ritam nastaje kao oblik spontane električne aktivnosti u srčanom organu. ...
Full-text available
An increasing number of psychological problems caused by stress impose the necessity of research into the application of new technologies in the prevention and treatment of mental disorders. The use of smartwatches with built-in physiological sensors allows for the detection of cardiac rhythm and potentially some related therapeutic procedures. Authors conducted heart rate measurements using two different smart mobile devices. The devices have been installed with their own built-in Android Wear heart rate measurement application, as well as two commercial applications of the same purpose. The accuracy of all measurements was controlled by a professional psychophysiological measuring device. The obtained data were statistically processed, measures of central tendency were determined and the obtained features were compared. It may be concluded that some smartwatches can even now be used for basic forms of biofeedback therapy. Acquisition of traditional professional equipment is not always necessary. The proprietary mobile application does not differ significantly in the measurement accuracy from commercial. With further development of mobile technologies, we can expect that in the near future, the capabilities of wearable devices for physiology monitoring will be greatly increased.
Full-text available
1. Summary Most studies on emotion processing rely on the presentation of emotional images or films. However, this methodology lacks ecological validity, limiting the extent to which findings can generalize to emotion processing in the wild. More realistic paradigms using Virtual Reality (VR) may be better suited to investigate authentic emotional states and their neuronal correlates. This preregistered study examines the neuronal underpinnings of naturalistic fear, measured using mobile electroencephalography (EEG). Seventy-five healthy participants entered a simulation in which they walked across a virtual plank which extended from the side of a skyscraper – either 80 stories up (the negative condition) or at street level (the neutral condition). Subjective ratings showed that the negative condition induced feelings of fear and presence. Following the VR experience, subjects passively viewed negative and neutral images from the International Affective Picture system (IAPS) outside of VR. We compared frontal alpha asymmetry between the plank and IAPS task and across valence of the conditions. Asymmetry indices (AI) in the plank task revealed greater right-hemispheric lateralization during the negative VR condition, relative to the neutral VR condition and to IAPS viewing. Within the IAPS task, no significant asymmetries were detected, though AIs in the VR task and in the IAPS task were negatively correlated suggesting that stronger right-hemispheric activation in the VR task was associated with stronger left-hemispheric activation during the IAPS task. In summary, our findings indicate that immersive technologies such as VR can advance emotion research by providing more ecologically valid ways to induce emotion.
Previous studies have noticed that depression, neuroticism, extraversion, and mood can leave linguistic fingerprints, particularly on pronoun use. The first aim of the present study was to examine the linguistic associations and impacts of these psychological constructs among Iranian native speakers of Farsi. Secondly, the linguistic correlates of depression, neuroticism, and extraversion were investigated in English, as a foreign language. For these goals, 220 Iranian adults (58.2% female, Mage = 25.2; SD = 5.19) participated and were assigned to four different groups (positive, neutral, and negative Farsi mood groups and a neutral English group). As expected, depression correlated with I-talk in Farsi (r = 0.217, p < 0.05). It was also associated with more negative emotion words (r = 0.355, p < 0.05), less positive emotion words (r = 0.421, p < 0.05), and less we-talk (r = 0.22, p < 0.05). Nonetheless, the results were not supportive of the association between I-talk and neuroticism or extraversion. Consistent with former observations, induced negative mood decreased self-referential language. The English responses showed that speaking in one's foreign versus native language can strongly diminish the linguistic effects of the psychological constructs.
Conference Paper
Full-text available
Sequences of multimedia documents are successfully used in laboratory settings and in practice to deliberately elicit specific emotional reactions. To ensure a successful experiment the emotion provoking stimuli must be selected carefully and have a specific order in which they are presented to the participants. Temporal aspect – duration of individual stimuli within sequences, duration of whole sequences and pauses between stimuli and sequences – must also be chosen with great care. Construction of effective sequences is a delicate and time consuming activity which requires significant group manual effort from domain experts. To facilitate this task we propose a new ontology called StimSeqOnt for formal description of stimuli sequences. The ontology is written in OWL DL language and provides formal and sufficiently expressive representation of affective concepts, high-level semantics, stimuli documents, multimedia formats and repositories used. In StimSeqOnt all relevant metadata about stimuli sequences may be stored as formal concepts. If available, elicited physiological data of previously exposed participants are available for comparison thereby enabling prediction of emotional responses. The StimSeqOnt is designed in compliance with ontology guidelines to facilitate sharing and reuse of expert knowledge.
Full-text available
1. ABSTRACT The aim of this study was to investigate if physiological changes that take place during planning and telling a lie are the same ones that take place during planning and telling a speech at thermographic level. For this, we run two experiments based on social stress task (Trier Social Test-TST) and lie detection. The main result was that the simulation of social stress did not produce real stress while lying produced real stress, with significant changes in the temperature of the nose and hands. 2. RESUMEN El propósito de este estudio fue investigar si los cambios fisiológicos que se producen a la hora de planificar y decir una mentira son los mismos que se producen a la hora de planificar y decir un discurso a nivel termográfico. Para ello se llevaron a cabo dos experimentos basados en tareas de estrés social y detección de mentiras. El resultado principal fue que la simulación del estrés social no produjo verdadero estrés mientras que mentir sí lo produce, observándose cambios significativos en la temperatura de la nariz y las manos a nivel termográfico durante la realización de las tareas. Palabras clave: termografía, estrés social, estrés físico, planificar discurso, ejecutar discurso, planificar mentira, ejecutar mentira. 3. INTRODUCCIÓN Las respuestas al estrés y la ansiedad han sido objeto de estudio a través de diferentes protocolos de estrés psicológico a lo largo de los últimos años. El Test Social de Estrés de Trier (TSST) es el protocolo estandarizado más útil y apropiado para los estudios de reactividad de la hormona del estrés (Birkett, 2011). Este protocolo ha sido de gran utilidad para observar los cambios que se producen en la frecuencia cardíaca, la presión arterial y las respuestas a nivel endocrino que tienen que ver con el estrés.
Full-text available
Affective multimedia documents such as images, sounds or videos elicit emotional responses in exposed human subjects. These stimuli are stored in affective multimedia databases and successfully used for a wide variety of research in psychology and neuroscience in areas related to attention and emotion processing. Although important all affective multimedia databases have numerous deficiencies which impair their applicability. These problems, which are brought forward in the paper, result in low recall and precision of multimedia stimuli retrieval which makes creating emotion elicitation procedures difficult and labor-intensive. To address these issues a new core ontology STIMONT is introduced. The STIMONT is written in OWL-DL formalism and extends W3C EmotionML format with an expressive and formal representation of affective concepts, high-level semantics, stimuli document metadata and the elicited physiology. The advantages of ontology in description of affective multimedia stimuli are demonstrated in a document retrieval experiment and compared against contemporary keyword-based querying methods. Also, a software tool Intelligent Stimulus Generator for retrieval of affective multimedia and construction of stimuli sequences is presented.
In order to improve intelligent Human-Computer Interaction it is important to create a personalized adaptive emotion estimator that is able to learn over time emotional response idiosyncrasies of individual person and thus enhance estimation accuracy. This paper, with the aim of identifying preferable methods for such a concept, presents an experiment-based comparative study of seven feature reduction and seven machine learning methods commonly used for emotion estimation based on physiological signals. The analysis was performed on data obtained in an emotion elicitation experiment involving 14 participants. Specific discrete emotions were targeted with stimuli from the International Affective Picture System database. The experiment was necessary to achieve the uniformity in the various aspects of emotion elicitation, data processing, feature calculation, self-reporting procedures and estimation evaluation, in order to avoid inconsistency problems that arise when results from studies that use different emotion-related databases are mutually compared. The results of the performed experiment indicate that the combination of a multilayer perceptron (MLP) with sequential floating forward selection (SFFS) exhibited the highest accuracy in discrete emotion classification based on physiological features calculated from ECG, respiration, skin conductance and skin temperature. Using leave-one-session-out crossvalidation method, 60.3% accuracy in classification of 5 discrete emotions (sadness, disgust, fear, happiness and neutral) was obtained. In order to identify which methods may be the most suitable for real-time estimator adaptation, execution and learning times of emotion estimators were also comparatively analyzed. Based on this analysis, preferred feature reduction method for real-time estimator adaptation was minimum redundancy – maximum relevance (mRMR), which was the fastest approach in terms of combined execution and learning time, as well as the second best in accuracy, after SFFS. In combination with mRMR, highest accuracies were achieved by k-nearest neighbor (kNN) and MLP with negligible difference (50.33% versus 50.54%); however, mRMR+kNN is preferable option for real-time estimator adaptation due to considerably lower combined execution and learning time of kNN versus MLP.
Researchers interested in emotion have long struggled with the problem of how to elicit emotional responses in the laboratory. In this article, we summarise five years of work to develop a set of films that reliably elicit each of eight emotional states (amusement, anger, contentment, disgust, fear, neutral, sadness, and surprise). After evaluating over 250 films, we showed selected film clips to an ethnically diverse sample of 494 English-speaking subjects. We then chose the two best films for each of the eight target emotions based on the intensity and discreteness of subjects' responses to each film. We found that our set of 16 films successfully elicited amusement, anger, contentment. disgust, sadness, surprise, a relatively neutral state, and, to a lesser extent, fear. We compare this set of films with another set recently described by Philippot (1993), and indicate that detailed instructions for creating our set of film stimuli will be provided on request.
Affective neuroscience aims to understand how affect (pleasure or displeasure) is created by brains. Progress is aided by recognizing that affect has both objective and subjective features. Those dual aspects reflect that affective reactions are generated by neural mechanisms, selected in evolution based on their real (objective) consequences for genetic fitness. We review evidence for neural representation of pleasure in the brain (gained largely from neuroimaging studies), and evidence for the causal generation of pleasure (gained largely from brain manipulation studies). We suggest that representation and causation may actually reflect somewhat separable neuropsychological functions. Representation reaches an apex in limbic regions of prefrontal cortex, especially orbitofrontal cortex, influencing decisions and affective regulation. Causation of core pleasure or 'liking' reactions is much more subcortically weighted, and sometimes surprisingly localized. Pleasure 'liking' is especially generated by restricted hedonic hotspot circuits in nucleus accumbens (NAc) and ventral pallidum. Another example of localized valence generation, beyond hedonic hotspots, is an affective keyboard mechanism in NAc for releasing intense motivations such as either positively valenced desire and/or negatively valenced dread.