ArticlePDF Available

Crossmodal transfer of emotion by music

Authors:

Abstract and Figures

Music is one of the most powerful elicitors of subjective emotion, yet it is not clear whether emotions elicited by music are similar to emotions elicited by visual stimuli. This leads to an open question: can music-elicited emotion be transferred to and/or influence subsequent vision-elicited emotional processing? Here we addressed this question by investigating processing of emotional faces (neutral, happy and sad) primed by short excerpts of musical stimuli (happy and sad). Our behavioural experiment showed a significant effect of musical priming: prior listening to a happy (sad) music enhanced the perceived happiness (sadness) of a face irrespective of facial emotion. Further, this musical priming-induced effect was largest for neutral face. Our electrophysiological experiment showed that such crossmodal priming effects were manifested by event related brain potential components at a very early (within 100 ms post-stimulus) stages of neuronal information processing. Altogether, these results offer new insight into the crossmodal nature of music and its ability to transfer emotion to visual modality.
Content may be subject to copyright.
GOLDSMITHS Research Online
Article (refereed)
Logeswaran, Nidhya and Bhattacharya, Joydeep
Crossmodal transfer of emotion by music
Originally published in Neuroscience LettersThe publisher's version is
available at: http://dx.doi.org/10.1016/j.neulet.2009.03.044
You may cite this version as: Logeswaran, Nidhya and Bhattacharya,
Joydeep, 2009. Crossmodal transfer of emotion by music. Neuroscience
Letters, 455 (2). pp. 129-133. ISSN 03043940 [Article]: Goldsmiths Research
Online.
Available at: http://eprints.gold.ac.uk/4213/
This document is the author’s final manuscript version of the journal article,
incorporating any revisions agreed during peer review. Some differences
between this version and the publisher’s version remain. You are advised to
consult the publisher’s version if you wish to cite from it.
Copyright © and Moral Rights for the papers on this site are retained by the
individual authors and/or other copyright owners.
http://eprints-gro.goldsmiths.ac.uk
Contact Goldsmiths Research Online at: lib-eprints@gold.ac.uk
This article appeared in a journal published by Elsevier. The attached
copy is furnished to the author for internal non-commercial research
and education use, including for instruction at the authors institution
and sharing with colleagues.
Other uses, including reproduction and distribution, or selling or
licensing copies, or posting to personal, institutional or third party
websites are prohibited.
In most cases authors are permitted to post their version of the
article (e.g. in Word or Tex form) to their personal website or
institutional repository. Authors requiring further information
regarding Elsevier’s archiving and manuscript policies are
encouraged to visit:
http://www.elsevier.com/copyright
Author's personal copy
Neuroscience Letters 455 (2009) 129–133
Contents lists available at ScienceDirect
Neuroscience Letters
journal homepage: www.elsevier.com/locate/neulet
Crossmodal transfer of emotion by music
Nidhya Logeswarana, Joydeep Bhattacharyaa,b,
aDepartment of Psychology, Goldsmiths College, University of London, London SE14 6NW, United Kingdom
bCommission for Scientific Visualization, Austrian Academy of Sciences, Vienna A1220, Austria
article info
Article history:
Received 31 October 2008
Received in revised form 3 March 2009
Accepted 11 March 2009
Keywords:
Music
Face
Emotion
Priming
ERP
abstract
Music is one of the most powerful elicitors of subjective emotion, yet it is not clear whether emotions
elicited by music are similar to emotions elicited by visual stimuli. This leads to an open question: can
music-elicited emotion be transferred to and/or influence subsequent vision-elicited emotional process-
ing? Here we addressed this question by investigating processing of emotional faces (neutral, happy and
sad) primed by short excerpts of musical stimuli (happy and sad). Our behavioural experiment showed
a significant effect of musical priming: prior listening to a happy (sad) music enhanced the perceived
happiness (sadness) of a face irrespective of facial emotion. Further, this musical priming-induced effect
was largest for neutral face. Our electrophysiological experiment showed that such crossmodal priming
effects were manifested by event related brain potential components at a very early (within 100 ms post-
stimulus) stages of neuronal information processing. Altogether, these results offer new insight into the
crossmodal nature of music and its ability to transfer emotion to visual modality.
© 2009 Elsevier Ireland Ltd. All rights reserved.
Music is often considered as the language of emotion and one of
the oldest held views is that music arises principally from human
communication—a performer delivers some message to a receptive
listener. This message is supposed to be an emotional one and this
emotional communication is postulated to be the principal pur-
pose of music [20]. In an extensive review of music performance
[9], the analysis of communication accuracy showed that profes-
sional music performers are able to communicate basic emotions
(e.g., happy, sad, anger) to listeners with an accuracy almost as
high as in facial and vocal expression of emotions. Further, there
is considerable empirical evidence supporting the statement that
emotion is an integral part of a musical experience (see Ref. [10] for
a review).
But are musically induced emotions similar to other emotional
experiences [23]? An early EEG study [4] demonstrated a character-
istic difference in cortical brain activationpatterns: positive musical
excerpts produced a more pronounced lateralisation towards the
left fronto-temporal cortices, whereas negative musical excerpts
produced a right fronto-temporal activation pattern. This early
result is supported by recent studies showing that left frontal areas
are involved with the processing of positive music and right frontal
areas with the negative music [1,8,16]. Similar frontal asymmetry is
well reported for the processing of affective visual stimuli [2,3].
Corresponding author at: Department of Psychology, Goldsmiths College, Uni-
versity of London, New Cross, London SE14 6NW, United Kingdom.
Tel.: +44 2079197334; fax: +44 2079197873.
E-mail address: j.bhattacharya@gold.ac.uk (J. Bhattacharya).
Therefore, it is reasonable to infer that there are some overlaps
between musical emotions and visual emotions.
But can these musically induced emotions arising through the
auditory channel influence our interpretation of emotions aris-
ing through other sensory channels (i.e. visual)? Research on
crossmodal integration of auditory and visual emotions [5] shows
that rating of affective information in one sensory modality can
be biased towards the direction of the emotional valence of
information in another sensory modality. Event-related-potential
(ERP) studies presenting emotionally congruent and incongru-
ent face–voice pairs reveal early ERP effects (N1, P2 components)
for congruent face–voice pairs, suggesting an early interaction
between auditory and visual emotional stimuli [15,22]. Therefore,
musical emotion can interact with visual emotion for simultaneous
music and visual processing.
But can musical emotion interact with or even influence the
visual emotion for non-simultaneous processes? In other words,
can music be used as an affective priming stimulus which could
systematically influence the emotional processes of target visual
stimuli? Music was earlier used as a priming stimulus in semantic
context [12,21]. To the best of our knowledge, the current study is
the first to address this issue in a crossmodal context by using both
behavioural and ERP experiments.
We performed two separate experiments – (i) behavioural and
(ii) electrophysiological (EEG) – on a total of 46 adult human par-
ticipants. Thirty participants (15 males and 15 females, mean age
26.1±4.31 years) took part in (i) without any cash incentive, and
sixteen participants (8 males and 8 females, mean age 27.5±5.88
years) took part in (ii) against a small cash incentive. All partici-
0304-3940/$ – see front matter © 2009 Elsevier Ireland Ltd. All rights reserved.
doi:10.1016/j.neulet.2009.03.044
Author's personal copy
130 N. Logeswaran, J. Bhattacharya / Neuroscience Letters 455 (2009) 129–133
Fig. 1. (a) Emotion ratings of happy, sad and neutral faces, regardless of musical primes. (b) Ratings for six individual conditions: happy, sad and neutral faces primed by
happy or sad musical excerpts. (c) Difference (happy sad) in ratings for three facial emotions. Note that the largest effect was found for neutral facial emotion.
pants were healthy right-handed university students, had normal
hearing, normal or corrected-to-normal vision, and had no special
musical expertise or musical education. The study was conducted
in accordance with the Declaration of Helsinki, and was approved
by the Internal Ethics Committee at Goldsmiths College, University
of London. All participants gave informed written consent before
both experiments.
All musical stimuli were taken from a previous study [1]. Briefly,
there were 120 instrumental musical excerpts belonging to two
emotional categories: happy and sad. Each piece was played for
15s with both beginning and end faded in and out, respectively, to
minimize surprise. The visual stimuli were faces of 40 different indi-
viduals with each individual showing threetypes of facial emotions:
happy, sad and neutral (http://www.macbrain.org/stim/faces.htm).
There were 90 trials equally divided into six possible conditions
(2 musical emotions ×3 facial emotions). Each trial lasted for 16s,
where a 15-s musical excerpt was followed by a facial stimulus pre-
sented for 1 s. At the end of each trial, participants were required
to rate the facial emotion on a 7-point scale: 1= extremely sad,
2 = moderately sad, 3 = slightly sad, 4 = neutral, 5 = slightly happy,
6 = moderately happy, and 7 = extremely happy. Participants were
told to try and to feel the emotion of the musical stimuli and
to rely mainly on their feelings while they rated the facial stim-
uli.
The EEG study followed a similar procedure to the behavioural
study but with the following exceptions. There were 120 trials with
20 trials for each condition. Further, instead of an explicit emo-
tional evaluation, the participants were asked to press a button
whenever a female face was shown. This minimized the explicit
components of emotional processing, and the remaining differ-
ences, if any,would reflect the implicitness of emotional processing.
Trials were randomized within each block and across participants.
EEG signals were recorded from 28 Ag/AgCl scalp electrodes (Fp1,
Fp2, F3, F4, F7, F8, Fz, FC3, FC4, FCz, C5, C6, C3, C4, Cz, CP5, CP6, CP3,
CP4, CPz, P7, P8, P3, P4, Pz, O1, O2, Oz) according to the Interna-
tional 10/20-system. Horizontal and vertical eye movements were
recorded from electrodes placed around the eyes. Impedances were
kept below 5 k. All scalp electrodes were algebraically referenced
to the average of two earlobes. The sampling frequency was 500 Hz.
Perceived emotional ratings were assessed using factorial
repeated-measures analysis of variance (ANOVA) with the factors,
musical emotion (two levels: happy and sad) and facial emotion
(three levels: happy, sad and neutral). Further post hoc compar-
isons between pairs of specific conditions were carried out by using
Fig. 2. Grand average ERP profiles at 13 selected electrodes (see the electrode locations on the top) during processing of neutral facial stimuli primed by either happy music
(thick line) or sad music (thin line).
Author's personal copy
N. Logeswaran, J. Bhattacharya / Neuroscience Letters 455 (2009) 129–133 131
paired t-tests. Greenhouse–Geisser correction wasapplie d toadjust
for degrees of freedom.
The data pre-processing was carried out by EEGLAB [6]. Stim-
ulus epochs of 1500ms starting 500 ms before the face onset
were generated offline from the continuous data. Extracted epochs
were subsequently checked for artefacts by visual inspection. In
order to correct for ocular artefacts including eye blinks, Inde-
pendent Component Analysis was performed on the remaining
epochs. ERP was obtained by averaging artefact free epochs for
each of the six conditions. A series of 2 ×2 factorial repeated-
measures ANOVAs with factors, priming (happy vs. sad) and region
(as selected after scalp maps), were conducted on mean ERP
amplitudes at specific regions of interest (ROI) between any two
conditions with identical facial emotion but with different musical
primes. Temporal regions of interest were selected on the basis of
global field power (GFP) [13] which quantifies the instantaneous
global activity across the spatial potential fields. Spatial regions
of interest were selected on the basis of scalp maps of mean ERP
amplitudes at the pre-selected temporal region of interest. Across
statistical comparison, we found that spatial regions of interest
consisted of two levels—anterior and posterior. Instead of a data-
blind procedure of selecting region of interests on an ad-hoc basis,
this data-driven method selected only a few regions of interests,
thereby minimizing the error variance and maximizing the effect
size.
Fig. 1(a) shows the mean ratings of happy, neutral and sad
faces regardless of the type of musical primes. It was clear that
the facial emotions were rightly rated and classified by our par-
ticipants. Mean ratings for six conditions, separately, are shown
in Fig. 1(b). The happy faces when primed by happy music were
rated (6.13 ±0.36) higher (i.e. more happy) than when primed
by sad music (5.56 ±0.35), and the sad faces when primed by
sad music were rated (1.87±0.30) lower (more sad) than when
primed by happy music (2.44 ±0.30). Further, when the neutral
faces were primed by happy music, the rating (4.68±0.33) was
much higher than when it was primed by sad music (3.37±0.54).
Therefore, the differential effects of priming (happysad) were
similar for happy and sad faces (mean difference rating of 0.57)
but was almost doubled (mean difference rating of 1.31) for
neutral face (Fig. 1(c)). Repeated-measures ANOVA showed that
there were highly significant main effects for musical emotion
(F(1,29)= 103.29, p< 0.001) and facial emotion (F(2,28) = 1358.89,
p< 0.001). The music ×faces interaction effect was also found to
be highly significant (F(2,58) = 34.37, p< 0.001). These results show
that the effect of musical priming was largest for emotionally neu-
tral target stimuli.
ERPs were always compared between two conditions with same
facial emotion but with different priming, and same analysis pro-
cedure was applied for all three types of facial emotions. Since the
behaviour study showed largest effect for neutral faces, we strate-
gically emphasized the results for neutral faces in details as follows,
and the results for other two types of facial emotions will be briefly
mentioned.
Visual inspections revealed that ERP profiles for neutral faces
primed by happy music were markedlydif ferent from those primed
by sad music (Fig. 2). Enhanced N1 component was seen across all
frontal and central electrodes for happy, as opposed to sad, musi-
cal primes. The classical N170 face component was exhibited in
occipital and parietal (P7, P8, not shown) regions bilaterally for
both priming conditions. Between 180 and 250 ms, an increased
positivity (or reduced negativity) for happy primes as compared to
sad primes was noticed in frontal, central. At a later stage (300 ms
onward), posterior positivity was observed for both conditions. GFP
values were plotted in Fig. 3 (top panel) and the two profiles were
separated as early as from 50ms till 150ms, and then again for the
time period 190–210ms. Scalp maps at these time windows were
Fig. 3. Global field power values for three emotional facial stimuli: neutral face
(upper panel), happy face (middle panel), and sad face (lower panel). For each emo-
tion type, two types of musical priming, happy and sad, were shown in thick and
in thin lines, respectively. The high GFP values correspond to pronounced potential
fields with high peaks (both positive and negative)and steep gradients, whereas low
GFP values correspond to flat potential fields. Foreach facial emotion, time windows
where the two profiles showed maximal separation between two musical priming
were used for successive statistical analysis.
displayed in Fig. 4 for both conditions and their differences. As com-
pared to negative musical prime, positive musical prime showed
an enhanced negativity in frontal and central brain regions during
50–150ms and enhanced positivity during 190–210 ms in similar
anterior brain regions. For the N1 time window (50–150 ms), statis-
tical analysis showed a significant effect of priming (F(1,15)= 5.35,
p= 0.03), and a priming ×region interaction effect (F(1,15)= 8.62,
p= 0.01). For the later time window (190–210 ms), a significant
priming effect (F(1,15) = 4.66, p= 0.047) was found.
GFP plots for other two types of facial emotions were shown in
Fig. 3 (middle and lower panels). For happy facial stimuli, differ-
ences between happy and sad musical primes were found between
0–50 and 160–210ms. Statistical analysis of mean ERP ampli-
tudes showed a near significant priming effect during 160–210 ms
(F(1,15) = 4.33, p= 0.06) and an almost significant priming×region
interaction effect between 160 and 210 ms (F(1,15) = 4.16, p= 0.06).
For sad facial stimuli, the early (0–50ms) difference between
the two priming conditions was also found, but the later differ-
ences were found during 430–450 ms. Statistical analysis of mean
ERP amplitudes showed significant region effects, F(1,15)= 7.54,
p= 0.015 and F(1,15) = 32.29, p<0.001, for both time windows,
Author's personal copy
132 N. Logeswaran, J. Bhattacharya / Neuroscience Letters 455 (2009) 129–133
Fig. 4. The scalp distribution of mean ERP amplitudes at two different time windows (50–150 and 190–210ms) for target neutral faces primed by happy (left column) and
sad (middle column) musical stimuli. The right column shows the same but for the difference potentials (sadhappy). As compared to sad musical prime, happy musical
prime produced enhanced negativity at the first time window and enhanced positivity at the second time window, both over anterior brain regions.
0–50 ms and 430–450 ms, respectively, and a near significant effect
of priming (F(1,15) = 4.30, p= 0.06) was found at the later time win-
dow.
Our behavioural experiment confirmed that emotional rating
of the target facial stimuli could be biased towards the direction
of the emotional valence of the musical primes. Earlier research
[5,22] reported that emotions in auditory stimuli interactwith emo-
tions in visual stimuli for simultaneously presented auditory–visual
stimuli. But our result extends it further by showing that such
interaction could also occur for non-simultaneous processing, i.e.
when the emotional auditory stimuli precede the emotional visual
stimuli. In other words, priming musical stimuli can systematically
influence the perceived emotional contents in target visual stimuli.
Music was earlier used in semantic priming paradigms. Using chord
sequences, either consonant or dissonant, as priming stimuli, it was
shown [21] that target words are faster recognized for emotionally
congruent chord–word pairs than for incongruent ones. Further,
when target words are preceded by semantically unrelated musical
primes, N400, an ERP component reflecting contextual integration,
effect is reported [12]. Altogether, this suggests that music has an
excellent potential to be used as an emotional priming stimulus.
Our behaviour data also shows that the largest effect of musi-
cal priming was found for neutral faces with an effect size almost
twice those for happy or sad faces. It was shown earlier [14] that
as compared to emotionally distinct (i.e. happy, angry, sad) facial
stimuli, emotionally ambiguous (i.e. neutral) facial stimuli are more
likely to be influenced by auditory cues in a facial emotion detec-
tion task. The information-content of neutral faces are supposedly
lower than those of happy or sad faces, and since the brain relies on
cues from multiple sensory stimuli to create an optimal represen-
tation of the external environment, emotionally neutral stimuli is
being influenced by emotionally conspicuous stimuli, even though
being generated by different senses. Although in our paradigm,
there is no such explicit requirement of integration of informa-
tion across musical and visual stimuli, our findings suggest that
a generic mechanism of multimodal affective interaction might
exist also in a priming paradigm. Alternatively, this could also be
explained by the affect-as-information hypothesis [19] which relies
on the assumption that affective processes mainly occur outside
of awareness. In contrast to traditional assumption of judgement
and decision making which emphasizes the role of target related
features, this hypothesis states that when making evaluative judge-
ments, participant often ask themselves, “How do I feel about it?”
[18], and therefore, “they may mistake feelings due to a pre-existing
state as a reaction to the target” [17]. Since the emotionally neutral
facial stimuli contain less information than emotionally conspic-
uous (happy or sad) facial stimuli, the transient affect from the
priming musical stimuli has the maximal impact in determining
the evaluation of the neutral facial stimuli.
Our ERP data showed that for neutral facial emotion, happy
music, as compared to sad music, showed a significant effect
during the N1 time period (50–150ms). Earlier Pourtois et al.
[15] have found that simultaneous presentation of emotionally
congruent face–voice pairs produce an enhancement of auditory
N1 component as compared to incongruent pairs, suggesting an
early crossmodal binding. In contrast to this study which reported
enhancement over auditory cortex, our N1 effect was predomi-
nant over fronto-central and midfrontal regions (FC3/4, FCz, Fz, Cz).
Taken together, this suggests that happy or positive auditory emo-
tion is more likely to influence neutral visual emotion by engaging
brain regions responsible for top-down projections.
ERP results also showed priming related enhancement of P2
(190–210ms) component for happy and neutral target faces but
not for sad target faces. Similar modulation of P2 has also recently
been reported [22] for happy picture–voice pairs presented simul-
taneously but not for sad pairs. Enhanced positivity at similar time
window has also been found for processing isolated emotional faces
as compared to neutral faces [7]. However, the functional role of
P2 is not yet clear (but see Ref. [22] for some possible explana-
tions) in mediating interaction between priming musical stimuli
and emotionally selective (happy and neutral, but not sad) target
faces.
Finally, let us offer a few practical remarks. First, the current
paradigm of music-induced emotional priming is quite different
from other mood-induction procedures which are associated with a
longer lasting change of emotional states, whereas our study inves-
tigated the effect of emotional changes on a much shorter time
scale [11]. Secondly, unlike previous ERP studies of facial emotion
Author's personal copy
N. Logeswaran, J. Bhattacharya / Neuroscience Letters 455 (2009) 129–133 133
processing, we alwayscompared the same facial emotional type but
differed only in priming. Therefore, our results indicate a more sub-
tle component in early neural responses which can potentially bias
subsequent emotional evaluation occurring at later stages. This was
also manifested by our robust behavioural findings which called
for an explicit evaluation of facial emotions. Thirdly, our ERP data
primarily reflects an implicit emotional processing since the par-
ticipants were naïve with respect to the specific aims of the study.
Further, as the task of gender detection did not require the par-
ticipants to focus on the presented emotions, the results are less
likely to be attributed to differences in directed attention as a func-
tion of presented emotions. Therefore, the ERP results suggest an
early processing of emotional facial expression primed by musical
stimuli.
In summary, the results of our behavioural and ERP study
revealed some patterns of crossmodal influences by musical emo-
tion. Behavioural data clearly showed that listening to musical
excerpts, albeit short, could significantly influence the subsequent
explicit evaluation of visual emotional stimuli. ERP data showed
that such musical priming could also influence implicit visual emo-
tional processes.
Acknowledgements
The study was supported by JST.ERATO Shimojo project (JB). We
are thankful to Prof. Eckart Altenmüller for the music stimuli, to
Rob Davis for technical support, to Job Lindsen for help in data
pre-processing, and to Prof. Rolf Reber for his helpful comments
as a reviewer. Author contributions: J.B. conceived research; N.L.
collected data; J.B. and N.L. analyzed data and wrote the paper.
References
[1] E. Altenmuller, K. Schurmann, V.K. Lim, D. Parlitz, Hits to the left, flops to the
right: different emotions during listening to music are reflected in cortical
lateralisation patterns, Neuropsychologia 40 (2002) 2242–2256.
[2] T.Canli, J.E. Desmond, Z. Zhao, G. Glover, J.D.E. Gabrieli, Hemispheric asymmetry
for emotional stimuli detected with fMRI, Neuroreport 9 (1998) 3233–3239.
[3] R.J. Davidson, Anterior cerebral asymmetry and the nature of emotion, Brain
and Cognition 20 (1992) 125–151.
[4] R.J. Davidson, G.E. Schwartz, C. Saron, J. Bennett, D.J. Goleman, Frontal versus
parietal EEG asymmetry during positive and negative affect, Psychophysiology
16 (1979) 202–203.
[5] B. de Gelder, J. Vroomen, The perception of emotions by ear and by eye, Cogni-
tion & Emotion 14 (2000) 289–311.
[6] A. Delorme, S. Makeig, EEGLAB: an open source toolbox for analysis of single-
trial EEG dynamics including independent component analysis, Journal of
Neuroscience Methods 134 (2004) 9–21.
[7] M. Eimer, A. Holmes, Event-related brain potential correlates of emotional face
processing, Neuropsychologia 45 (2007) 15–31.
[8] E.O. Flores-Gutierrez, J.L. Diaz, F.A. Barrios, R. Favila-Humara, M.A. Guevara, Y.
del Rio-Portilla, M. Corsi-Cabrera, Metabolic and electric brain patterns during
pleasant and unpleasant emotions induced by music masterpieces, Interna-
tional Journal of Psychophysiology 65 (2007) 69–84.
[9] P.N.Juslin, P. Laukka, Communication of emotions in vocalexpression and music
performance: different channels, same code? Psychological Bulletin 129(2003)
770–814.
[10] P.N. Juslin, J.A. Sloboda (Eds.), Music and Emotion, Oxford University Press,
Oxford, 2001, p. 487.
[11] P.N. Juslin, D. Vastfjall, Emotional responses to music: the need to consider
underlying mechanisms, The Behavioral and Brain Sciences 31 (2008) 559–575
(discussion 575–621).
[12] S. Koelsch, E. Kasper, D. Sammler, K. Schulze, T. Gunter, A.D. Friederici, Music,
language and meaning: brain signatures of semantic processing, Nature Neu-
roscience 7 (2004) 302–307.
[13] D. Lehmann, W. Skrandies, Reference-free identification of components of
checkerboard-evoked multichannel potential fields, Electroencephalography
and Clinical Neurophysiology 48 (1980) 609–621.
[14] D.W. Massaro, P.B. Egan, Perceiving affect from the voice and the face, Psycho-
nomic Bulletin & Review 3 (1996) 215–221.
[15] G. Pourtois, B. de Gelder, J. Vroomen, B. Rossion, M. Crommelinck, The time-
course of intermodal binding between seeing and hearing affectiveinformation,
Neuroreport 11 (2000) 1329–1333.
[16] D.L.Santesso, L.A. Schmidt, L.J. Trainor, Frontal brain electrical activity(EEG) and
heart rate in response to affective infant-directed (ID) speech in 9-month-old
infants, Brain and Cognition 65 (2007) 14–21.
[17] N. Schwarz, Feelings as information: informational and motivational functions
of affective states, in: E.T. Higgins, R. Sorrentino (Eds.), Handbook of Motivation
and Cognition: Foundations of Social Behaviour, vol. 2, Guildford Press, New
York, 1990, pp. 527–561.
[18] N.Schwarz, G.L. Clore, How do I feel about it? The informative function of mood,
in: K. Fiedler,J. Forgas (Eds.), Affect, Cognition and Social Behaviour, C.J. Hogrefe,
Toronto, 1988, pp. 44–62.
[19] N. Schwarz, G.L. Clore, Mood, misattribution, and judgments of well-
being—informative and directive functions of affective states, Journal of
Personality and Social Psychology 45 (1983) 513–523.
[20] M. Serafine, Music as Cognition: The Development of Thought in Sound,
Columbia University Press, New York, 1988.
[21] B. Sollberger, R. Reber, D. Eckstein, Musical chords as affective priming context
in a word-evaluation task, Music Perception 20 (2003) 263–282.
[22] K.N. Spreckelmeyer, M. Kutas, T.P. Urbach,E. Altenmuller, T.F. Munte, Combined
perception of emotion in pictures and musical sounds, Brain Research 1070
(2006) 160–170.
[23] L.J. Trainor, L.A. Schmidt, Processing emotionsinduce d bymusic, in: I. Peretz, R.
Zatorre (Eds.), The Cognitive Neuroscience of Music, OxfordUniversity of Press,
Oxford, 2007, pp. 310–324.
... Growing evidence suggests that musical emotions can bias judgments of the emotion of complex visual stimuli, such as dynamic visual scenes and film (Ansani et al., 2020;Cohen, 2001Cohen, , 2013Herget, 2021;Steffens, 2020), as well as faces (Hanser et al., 2015;Jeong et al., 2011;Jomori et al., 2013;Logeswaran & Bhattacharya, 2009;Marin et al., 2017;Quarto et al., 2014), geometric shapes/figures (Bhattacharya & Lindsen, 2016;Marshall & Cohen, 1988;Weinreich & Gollwitzer, 2016), and pictures (Arriaga et al., 2014;Baumgartner et al., 2006;Campos-Bueno et al., 2015;Marin et al., 2012). It has been shown, for instance, that emotional judgment of neutral facial stimuli can be systematically biased toward the direction of the emotional valence of the music presented a few seconds before the visual stimuli (Logeswaran & Bhattacharya, 2009), and that brightness judgments of geometric shapes can be affected by the emotional valence of the musical primes (Bhattacharya & Lindsen, 2016). ...
... Growing evidence suggests that musical emotions can bias judgments of the emotion of complex visual stimuli, such as dynamic visual scenes and film (Ansani et al., 2020;Cohen, 2001Cohen, , 2013Herget, 2021;Steffens, 2020), as well as faces (Hanser et al., 2015;Jeong et al., 2011;Jomori et al., 2013;Logeswaran & Bhattacharya, 2009;Marin et al., 2017;Quarto et al., 2014), geometric shapes/figures (Bhattacharya & Lindsen, 2016;Marshall & Cohen, 1988;Weinreich & Gollwitzer, 2016), and pictures (Arriaga et al., 2014;Baumgartner et al., 2006;Campos-Bueno et al., 2015;Marin et al., 2012). It has been shown, for instance, that emotional judgment of neutral facial stimuli can be systematically biased toward the direction of the emotional valence of the music presented a few seconds before the visual stimuli (Logeswaran & Bhattacharya, 2009), and that brightness judgments of geometric shapes can be affected by the emotional valence of the musical primes (Bhattacharya & Lindsen, 2016). Effects of musical emotions have also been observed during the concomitant presentation of music and visual stimuli (Arriaga et al., 2014;Baumgartner et al., 2006;Campos-Bueno et al., 2015;Jeong et al., 2011;Marshall & Cohen, 1988;Spreckelmeyer et al., 2006). ...
... Participants were randomly assigned to five groups according to the musical emotion conveyed by the background stimuli presented during the appreciation of the artwork (happy, scary, peaceful, and sad music; or silence). Based on previous research evidence (Arriaga et al., 2014;Baumgartner et al., 2006;Logeswaran & Bhattacharya, 2009;Marin et al., 2012;Spreckelmeyer et al., 2006), we hypothesized that judgments of the artwork emotion would be biased by the valence of the music excerpts presented in the background. In an exploratory analysis, we further assessed whether background music would influence ratings of the artwork's aesthetics (beauty and liking) and examined whether the likability of the background music would interact with the aesthetic experience of the artwork. ...
Article
Aesthetic evaluations can be highly influenced by a myriad of individual and situational factors. Interestingly, little is yet known about the possible effects of background music on the aesthetic experience of visual art. Here, we examined whether musical emotions would influence different dimensions of the aesthetic experience of a visual artwork displayed in a naturalistic environment. A total of 142 visitors of a contemporary art museum appreciated an abstract painting by Wassily Kandinsky while listening to background music conveying different emotions (happy, sad, peaceful, scary) or silence. Our findings suggest that music valence significantly influenced participants’ judgment of the pleasantness of the painting. In addition, music likability had a significant effect on participants’ judgments of the artwork’s valence, beauty, and liking. Specifically, participants who liked the background music rated these dimensions of the artwork aesthetic experience significantly more positively than those who disliked the music. Overall, these results suggest that aspects associated with the aesthetic experience of music may influence the aesthetic experience of visual art, opening new avenues for the investigation of cognitive processes underlying the aesthetic experience induced by objects across different media.
... Even though both sets of music are in minor keys, the new accompaniment with more rhythmic features and a variety of instrumentation may have contributed to the higher rating of facial expression. Despite testing the issue of congruence, the music itself could be a dominant factor in affecting the respondents' emotion; it was found that listening to even a short excerpt of music can affect the evaluation of visual stimuli [26]. However, it is also interesting to note that recognition of music and facial emotion declines as age increases [27]. ...
Conference Paper
While many activities that involve human movement customarily use music or sound as an accompaniment to enhance the overall experience of the performers and viewers, the details of musical aspects have often been neglected. Issues on synchronization between music and movement have often been debated particularly in the field of dance since music can to certain extent be abstract and subjective. Although regulations in many sports routines emphasize that using music merely as background is prohibited, the match between the two has not always been convincing for many reasons such as preference, lack of musical knowledge and musicianship. To address the issue, through composition technique and computer music software, this research constructed simulated models to illustrate the importance of fundamental musical aspects in transforming the visual perceptions of viewers towards the same action or movements. This paper reports a section from a study on how visual perception of expressiveness in a Tai Chi routine is enhanced by increasing the synchronization between music and movement. In the experiment, two videos were produced and then evaluated by thirty viewers. One video features the original music used by the athletes and the other is accompanied by a new composition with increased synchronization between music and movements. The result shows a significant difference between the videos in which the enhanced synchronized video provides a better visual perception of the expression of the performance. This research proposes a simulation approach using theories in multimedia and perception to assist athletes in achieving an optimal combination of music-movement synchronisation.
... The human face is the main factor determining physical attractiveness regarding short-and long-term relationships because it indicates genetic fitness (Currie and Little, 2009), but cognitive factors such as creativity and intelligence may also play a significant role (Boogert et al., 2011). This research approach is backed up by previous findings from the crossmodal priming and multimodal interaction literature, suggesting that sound and music can alter the perception of various types of visual stimuli, including facial expressions (Logeswaran and Bhattacharya, 2009, for a review see Gerdes et al., 2014). Likewise, music performance research has shown that visual information related to the performer can also alter the appreciation of music (Platz and Kopiez, 2012). ...
Article
Full-text available
A number of theories about the origins of musicality have incorporated biological and social perspectives. Darwin argued that musicality evolved by sexual selection, functioning as a courtship display in reproductive partner choice. Darwin did not regard musicality as a sexually dimorphic trait, paralleling evidence that both sexes produce and enjoy music. A novel research strand examines the effect of musicality on sexual attraction by acknowledging the importance of facial attractiveness. We previously demonstrated that music varying in emotional content increases the perceived attractiveness and dating desirability of opposite-sex faces only in females, compared to a silent control condition. Here, we built upon this approach by presenting the person depicted (target) as the performer of the music (prime), thus establishing a direct link. We hypothesized that musical priming would increase sexual attraction, with high-arousing music inducing the largest effect. Musical primes (25 s, piano solo music) varied in arousal and pleasantness, and targets were photos of other-sex faces of average attractiveness and with neutral expressions (2 s). Participants were 35 females and 23 males (heterosexual psychology students, single, and no hormonal contraception use) matched for musical background, mood, and liking for the music used in the experiment. After musical priming, females' ratings of attractiveness and dating desirability increased significantly. In males, only dating desirability was significantly increased by musical priming. No specific effects of music-induced pleasantness and arousal were observed. Our results, together with other recent empirical evidence, corroborate the sexual selection hypothesis for the evolution of human musicality.
... 39 Vice-versa, auditory-induced emotions affect the emotional perception of visual content. 135,144 Hence we hypothesize that visual angle not only affects the emotional responses to visual but also to auditory information. In the current study, we address how the visual angle interacts with auditory content perception to organize emotional responses. ...
Thesis
Full-text available
Human-computer interfaces have the potential to support mental health practitioners in alleviating mental distress. Adaption of this technology in practice is, however, slow. We provide means to extend the design space of human-computer interfaces for mitigating mental distress. To this end, we suggest three complementary approaches: using presentation technology, using virtual environments, and using communication technology to facilitate social interaction. We provide new evidence that elementary aspects of presentation technology affect the emotional processing of virtual stimuli, that perception of our environment affects the way we assess our environment, and that communication technologies affect social bonding between users. By showing how interfaces modify emotional reactions and facilitate social interaction, we provide converging evidence that human-computer interfaces can help alleviate mental distress. These findings may advance the goal of adapting technological means to the requirements of mental health practitioners.
... Con los instrumentos evaluativos sobre el análisis de respuestas fisiológicas y neuronales se trata de evaluar cómo reaccionan las personas ante la música y los parámetros que la conforman, por ejemplo, ante una canción valorada con una valencia positiva y un alto nivel de arousal las personas suelen emitir respuestas comportamentales como microexpresiones faciales o musculares (Logeswaran y Bhattacharya, 2009), respuestas fisiológicas como escalofríos (Harrison y Loui, 2014), o respuestas cognitivas como la mejora del estado anímico (Jallais y Gilet, 2010). ...
Article
Full-text available
The topic of music and its influence on emotions is discussed wide throughout the history of music, especially in an applied way. In recent decades we find scientific studies on music as an emotional stimulus. Some methods, such as neuroimaging or hormones, allow a detailed analysis of the effects of music and sound on the human being. This work
... However, most of the above was the result of an individual's single-channel emotional stimuli processing, that is, judging others' emotions through visual (such as observing facial expression) or auditory (such as analyzing tone) stimuli, but in daily life, people engage in crossmodal emotional processing. A large number of studies have shown that an individual's singlechannel processing results may be affected by other channels (18)(19)(20)(21)(22); therefore, it is inappropriate to generalize the results of research on a single channel directly to the understanding of people's daily lives. The underlying neurocognitive processes that integrate separate streams of information from different sensory channels into the overall experience are often referred to as multisensory integration (MSI) (23). ...
Article
Full-text available
Depression is related to the defect of emotion processing, and people's emotional processing is crossmodal. This article aims to investigate whether there is a difference in audiovisual emotional integration between the depression group and the normal group using a high-resolution event-related potential (ERP) technique. We designed a visual and/or auditory detection task. The behavioral results showed that the responses to bimodal audiovisual stimuli were faster than those to unimodal auditory or visual stimuli, indicating that crossmodal integration of emotional information occurred in both the depression and normal groups. The ERP results showed that the N2 amplitude induced by sadness was significantly higher than that induced by happiness. The participants in the depression group showed larger amplitudes of N1 and P2, and the average amplitude of LPP evoked in the frontocentral lobe in the depression group was significantly lower than that in the normal group. The results indicated that there are different audiovisual emotional processing mechanisms between depressed and non-depressed college students.
... It is also possible that these audio-visual interactions reflect emotional mediation, or the matching of valence and arousal dimensions across modalities (for review, see Spence, 2020). Affective dimensions of music can modify the evaluation of images (Logeswaran & Bhattacharya, 2009) and colors (Palmer, Schloss, Xu, & Prado-León, 2013); the expression of basic emotions (''happy'' music, happy face)-or perhaps their valence, arousal, and potency dimensions (Osgood, Suci, & Tannenbaum, 1957)-thus appears, in certain instances, to mediate crossmodal correspondences (though for conflicting evidence see Marin, Gingras, & Bhattacharya, 2012;Wallmark & Allen, 2020;Whiteford, Schloss, Helwig, & Palmer, 2018). Note that complex musical stimuli are affectively richer than the isolated tones used here, so the applicability of these other findings to the present investigation may be somewhat limited. ...
Article
Full-text available
Musical timbre is often described using terms from non-auditory senses, mainly vision and touch; but it is not clear whether crossmodality in tim-bre semantics reflects multisensory processing or simply linguistic convention. If multisensory processing is involved in timbre perception, the mechanism governing the interaction remains unknown. To investigate whether timbres commonly perceived as ''bright-dark'' facilitate or interfere with visual perception (darkness-brightness), we designed two speeded classification experiments. Participants were presented consecutive images of slightly varying (or the same) brightness along with task-irrelevant auditory primes (''bright'' or ''dark'' tones) and asked to quickly identify whether the second image was brighter/darker than the first. Incongruent prime-stimulus combinations produced significantly more response errors compared to congruent combinations but choice reaction time was unaffected. Furthermore, responses in a deceptive identical-image condition indicated subtle semantically congruent response bias. Additionally, in Experiment 2 (which also incorporated a spatial texture task), measures of reaction time (RT) and accuracy were used to construct speed-accuracy tradeoff functions (SATFs) in order to critically compare two hypothesized mechanisms for timbre-based crossmodal interactions, sensory response change vs. shift in response criterion. Results of the SATF analysis are largely consistent with the response criterion hypothesis, although without conclusively ruling out sensory change.
... [3][4][5][6][7][8][9] For present purposes, it will be necessary to recognize the distinction between experienced emotion and the perception of stimulus emotionality. With regards to the perception of emotionality in stimuli, fewer studies 11,12 have illustrated the possibility of a cross-modal influence. While research on this topic is less plentiful, it does begin to elucidate the phenomenon of interest. ...
Article
Full-text available
The work examines the perception of faces by adolescents and their mothers under the stress associated with the surgery. We hypothesized that stress can facilitate attribution of negative emotions to neutral faces, while feelings of support from the mother and others can play the opposite role. This means that the bias of emotions attributed to neutrals can be used to assess the level of stress. The study involved: adolescents N1 = 46, 12—17 years old, (M = 14.02, SD = 1.57), 59% boys who were treated in the department of pediatric bone pathology and adolescent orthopedics of National Medical Research Center of Traumatology and Orthopedics named after N.N. Priorov, as well as their mothers N2 = 46 (32—51 years old, M = 41.24, SD = 4.47). The following methods were used: Social support questionnaire SOZU-22, Varga-Stolin parental attitude questionnaire, Perceived stress scales for children and adults, Pain scale. Respondents were asked to chose the most suitable adjective to each of 11 images of emotionally neutral faces. The hypotheses put forward were generally not confirmed. For mothers, despite the absence of changes in the level of stress after child’s surgery treatment, the frequency of choosing positive emotions significantly increases and the frequency of attributing negative emotions to neutral faces decreases. In children, there is a significant decrease in stress after surgery, but the change in the assessment of neutral faces is associated not with the stress level, but with the assessment of pain, as well as with the characteristics of the mother’s attitude and characteristics of social support. At the same time, differences were revealed between the results of girls and boys.
Chapter
Full-text available
Can music induce emotions directly and, if so, are these emotions experienced similarly to emotions arising in other contexts? This chapter analyzes these questions from the perspective of neuroscience. Despite the fact that music does not appear to have an obvious survival value for modern adults, research indicates that listening to music does activate autonomic, subcortical, and cortical systems in a manner similar to other emotional stimuli. It is proposed that music may be so intimately connected with emotional systems because caregivers use music to communicate emotionally with their infants before they are able to understand language. In particular, it examines whether music engages the autonomic nervous system, sub-cortical emotion networks, and cortical areas involved in the emotional processing of other types of stimuli. It also investigates whether emotional reactions to music are simply cultural conventions by asking whether and how infants process musical emotions.
Article
Full-text available
This experiment examines how emotion is perceived by using facial and vocal cues of a speaker. Three levels of facial affect were presented using a computer-generated face. Three levels of vocal affect were obtained by recording the voice of a male amateur actor who spoke a semantically neutral word in different simulated emotional states. These two independent variables were presented to subjects in all possible permutations-visual cues alone, vocal cues alone, and visual and vocal cues together-which gave a total set of 15 stimuli. The subjects were asked to judge the emotion of the stimuli in a two-alternative forced choice task (either HAPPY or ANGRY). The results indicate that subjects evaluate and integrate information from both modalities to perceive emotion. The influence of one modality was greater to the extent that the other was ambiguous (neutral). The fuzzy logical model of perception (FLMP) fit the judgments significantly better than an additive model, which weakens theories based on an additive combination of modalities, categorical perception, and influence from only a single modality.
Article
Full-text available
Emotions are expressed in the voice as well as on the face. As a first step to explore the question of their integration, we used a bimodal perception situation modelled after the McGurk paradigm, in which varying degrees of discordance can be created between the affects expressed in a face and in a tone of voice. Experiment 1 showed that subjects can effectively combine information from the two sources, in that identification of the emotion in the face is biased in the direction of the simultaneously presented tone of voice. Experiment 2 showed that this effect occurs also under instructions to base the judgement exclusively on the face. Experiment 3 showed the reverse effect, a bias from the emotion in the face on judgement of the emotion in the voice. These results strongly suggest the existence of mandatory bidirectional links between affect detection structures in vision and audition.
Chapter
Full-text available
reviews research on the impact of affective states on evaluative judgments, presenting evidence that is difficult to reconcile with the assumption that emotional influences on social judgment are mediated by selective recall from memory / rather, the presented research suggests that individuals frequently use their affective state at the time of judgment as a piece of information that may bear on the judgmental task, according to a "how do I feel about it" heuristic extends the informative-functions assumption to research on affective influences on decision making and problem solving, suggesting that affective states may influence the choice of processing strategies / specifically it is argued that negative affective states, which inform the organism that its current situation is problematic, foster the use of effortful, detail oriented, analytical processing strategies, whereas positive affective states foster the use of less effortful heuristic strategies (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Investigated, in 2 experiments, whether judgments of happiness and satisfaction with one's life are influenced by mood at the time of judgment. In Exp I, moods were induced by asking 61 undergraduates for vivid descriptions of a recent happy or sad event in their lives. In Exp II, moods were induced by interviewing 84 participants on sunny or rainy days. In both experiments, Ss reported more happiness and satisfaction with their life as a whole when in a good mood than when in a bad mood. However, the negative impact of bad moods was eliminated when Ss were induced to attribute their present feelings to transient external sources irrelevant to the evaluation of their lives; but Ss who were in a good mood were not affected by misattribution manipulations. The data suggest that (a) people use their momentary affective states in making judgments of how happy and satisfied they are with their lives in general and (b) people in unpleasant affective states are more likely to search for and use information to explain their state than are people in pleasant affective states. (18 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Using an affective priming paradigm, we demonstrated that the affective tone of musical chords influences the evaluation of target words. In Experiment 1, participants heard either consonant chords with three tones or dissonant chords with four tones as primes and then saw a positive or a negative word as target. Even participants who were unaware of the hypothesis of the experiment evaluated target words faster if the words were preceded by a similarly valenced chord (e.g., consonant-holiday) as compared to affectively incongruent chord-word pairs (e.g. dissonant-humor). In Experiment 2, results of Experiment 1 were replicated even when chord density was held constant at three tones per chord. Results suggest that the affective tone of single musical elements is automatically extracted and might therefore be viewed as A basic process contributing to the strong connection between music and affect.