ArticlePDF Available

Abstract and Figures

It remains an open question whether it is possible to assign a single brain operation or psychological function for facial emotion decoding to a certain type of oscillatory activity. Gamma band activity (GBA) offers an adequate tool for studying cortical activation patterns during emotional face information processing. In the present study brain oscillations were analyzed in response to facial expression of emotions. Specifically, GBA modulation was measured when twenty subjects looked at emotional (angry, fearful, happy, and sad faces) or neutral faces in two different conditions: supraliminal (10 ms) vs subliminal (150 ms) stimulation (100 target-mask pairs for each condition). The results showed that both consciousness and significance of the stimulus in terms of arousal can modulate the power synchronization (ERD decrease) during 150-350 time range: an early oscillatory event showed its peak at about 200 ms post-stimulus. GBA was enhanced by supraliminal more than subliminal elaboration, as well as more by high arousal (anger and fear) than low arousal (happiness and sadness) emotions. Finally a left-posterior dominance for conscious elaboration was found, whereas right hemisphere was discriminant in emotional processing of face in comparison with neutral face.
Content may be subject to copyright.
Consciousness and arousal effects on emotional face processing as
revealed by brain oscillations. A gamma band analysis
Michela Balconi
a,
, Claudio Lucchiari
b
a
Laboratory of Cognitive Psychology, Department of Psychology, Catholic University of Milan, Largo Gemelli, 1 20123 Milan, Italy
b
Neurological National Hospital C. Besta, Milan, Italy
Received 4 May 2005; received in revised form 29 August 2007; accepted 8 October 2007
Available online 13 October 2007
Abstract
It remains an open question whether it is possible to assign a single brain operation or psychological function for facial emotion decoding to a
certain type of oscillatory activity. Gamma band activity (GBA) offers an adequate tool for studying cortical activation patterns during emotional
face information processing. In the present study brain oscillations were analyzed in response to facial expression of emotions. Specifically, GBA
modulation was measured when twenty subjects looked at emotional (angry, fearful, happy, and sad faces) or neutral faces in two different
conditions: supraliminal (10 ms) vs subliminal (150 ms) stimulation (100 target-mask pairs for each condition). The results showed that both
consciousness and significance of the stimulus in terms of arousal can modulate the power synchronization (ERD decrease) during 150350 time
range: an early oscillatory event showed its peak at about 200 ms post-stimulus. GBA was enhanced by supraliminal more than subliminal
elaboration, as well as more by high arousal (anger and fear) than low arousal (happiness and sadness) emotions. Finally a left-posterior
dominance for conscious elaboration was found, whereas right hemisphere was discriminant in emotional processing of face in comparison with
neutral face.
© 2007 Elsevier B.V. All rights reserved.
Keywords: Facial expression; EEG; Gamma band; Arousal; Consciousness
1. Introduction
Correlates of affective face processing have been investigat-
ed using a variety of recording techniques. On one side, some
authors studied ERP correlates associated with face compre-
hension. It has been argued that emotional face processing
arises after 200 ms, and that differences between ERPs elicited
by emotional faces and neutral faces were observable
specifically between 250 and 550 ms after stimulus onset
(Krolak-Salmon et al., 2001). An early negative deflection (N2)
of higher amplitude was revealed for arousing facial stimuli
(Balconi and Pozzoli, 2003; Sato et al., 2000; Streit et al., 2000)
in comparison with neutral stimuli. Moreover, there is evidence
that emotion processing is initiated and can proceed without
conscious awareness (Bunce et al., 1999; LeDoux, 1998). An
obvious and well-known example of unconscious perception of
emotion is subliminal stimulation effect. This phenomenon was
studied in a limited number of cases (Wong et al., 1994). Animal
studies suggest that fear-related response are by a direct
subcortical pathway from the thalamus direct to the amygdala,
allowing emotional (and specifically threat) to be processed
automatically and outside awareness. In humans, evidence for
the unconscious perception of masked face has been revealed in
terms of subjective reports (Esteves et al., 1994) autonomic
reaction (Morris et al., 2001), brain imaging measures (Whalen
et al., 1998), as well as ERPs (Kiefer and Spitzer, 2000). In
addition, unconscious stimulation showed to be sensitive to the
emotional content of the stimuli, as revealed by different
behavioural and physiological measures (Lang et al., 1998).
On the other side, brain oscillations were found to be a
powerful tool to analyze the cognitive processes related to
emotion comprehension in general (Başar et al., 1999; Krause,
2003), and, even if less studied, to unconscious perception
A
vailable online at www.sciencedirect.com
International Journal of Psychophysiology 67 (2008) 41 46
www.elsevier.com/locate/ijpsycho
Corresponding author. Tel.: +39 2 72342600; fax: +39 2 72342769.
E-mail address: michela.balconi@unicatt.it (M. Balconi).
0167-8760/$ - see front matter © 2007 Elsevier B.V. All rights reserved.
doi:10.1016/j.ijpsycho.2007.10.002
(Summerfield et al., 2002). Few previous studies on ERD/ERS
responses to emotion-related stimuli have examined the narrow
frequency bands (Aftanas et al., 2001, 2002). Recent researches
showed the event-related theta band power responds specifi-
cally to prolonged visual emotional stimulation (Krause et al.,
2000), and a synchronization was revealed in case of
coordinated response indicating readiness to process informa-
tion (Başar, 1999). Thus, theta EEG power typically increases
with increasing attentional demands and/or task difficulty. Also
the effect of valence in affective picture processing was studied,
showing that valence discrimination is associated with the early
time-locked synchronized theta activity (Aftanas et al., 2001).
Moreover, recent research have demonstrated that the modula-
tion of gamma band activity (GBA) in time windows between
200 and 400 ms following the onset of a stimulus is associated
with perception of coherent visual objects (Müller et al., 1996),
and may be a signature of active memory. In parallel, GBA was
found sensitive to emotional vs nonemotional stimuli and more
specifically it was related to the arousal effect: early GBA was
enhanced in response to aversive or highly arousing stimuli
compared to neutral picture (Balconi and Pozzoli, 2007). This
result was revealed in accordance with previous research that
employed ERP measures for arousing pictures (Schupp et al.,
2000) or emotional face (Balconi and Pozzoli, 2003; Sato et al.,
2000), since these studies found a modulation of the increased
arousal on ERP. Interestingly, previous research has found that
gamma frequency band could also be considered a marker of
degree of consciousness during elaboration of a stimulus:
synchronous oscillations in the gamma frequency range may be
necessary for the entry of information into conscious awareness
(Crick and Koch, 1998). Specifically, Summerfield et al. (2002)
have found that gamma activity increases after subjects had
been made aware of the stimulus, and, therefore, synchronous
gamma oscillations occurred in association with awareness
processes.
Therefore, gamma band is to be considered of main interest
in exploring the effect of arousal as well as the consciousness in
emotional face elaboration. The present study aims at studying
the brain mechanisms underlying human emotional processing
by measuring GBA changes in response to emotional faces
presented visually in both supraliminal and subliminal stimu-
lation. No previous study has widely explored the effect of
consciousness on the processing of emotional faces, in
conjunction with different types of stimulus (low or high
arousing faces). Actually, although brain oscillations have been
investigated in various sensory modalities, their role for brain
functioning for emotion elaboration remains unclear. Secondly,
it remains an open question whether it is possible to assign a
single brain operation or psychological function for emotion
decoding to a certain type of oscillatory activity. Thus, we
intend to explore functional correlates of brain oscillations with
regard to emotional face processing in supraliminal and
subliminal condition and emphasize the importance of distrib-
uted oscillatory networks in gamma frequency band. We
attended that emotional content may be indexed by oscillatory
activity of the brain that was directly related to awareness or
unawareness of the stimulus. Specifically, we hypothesized that
conscious elaboration of emotional stimuli will be indexed by
GBA synchronization, whereas unconscious condition will be
related to a decreased power intensity of this frequency band.
Secondly, we expected that affective significance of a facial
stimulus may result in changes of subjects' EEG responses
(Lang et al., 1993). Emotion evaluated as highly arousing
should be indexed by an enhanced power of gamma band in
conscious condition. Finally, brain lateralization was found
significant for emotional elaboration. As previously shown,
right dominance was revealed for emotional stimuli compared
to neutral ones, and specifically for face. On the contrary, left
hemisphere was found to be more activated by conscious
elaboration than unconscious. The present experiment based on
ERD measure examined whether emotions would be associated
with band modulation as regard as interhemispheric asymme-
tries in the right direction, whereas left hemisphere is expected
to be discriminant for conscious processing if compared to
unconscious.
2. Method
2.1. Subjects
Twenty healthy volunteers took part in the study (eleven
women, age range1925, mean = 23.37, SD = 2.13). They were
all right-handed and with normal or corrected-to-normal visual
acuity. Exclusion criteria were history of psychopathology for
the subjects or immediate family. They gave informed written
consent for participating in the study.
2.2. Stimulus material
Stimulus materials were taken from the set of pictures of
Ekman and Friesen (1976). They were black and white pictures
of male and female actors, presenting respectively a happy, sad,
angry, fearful, or neutral face.
2.3. Supraliminal/subliminal stimulation
A previous study was conducted in which the duration of
target facial stimulus was varied in order to establish threshold
condition (Liddell et al., 2004). In the current study we
employed both an objective threshold, defined as the stimulus
duration where the stimulus is perceived by the subject in 50%
of the cases (Merikle et al., 2001); and a subjective threshold,
defined as the overt lacking of discrimination of the stimulus
and its emotional content. The pre-experimental study and post-
hoc briefing confirmed that subjects were unable to detect target
stimulus in the subliminal condition.
During the experiment we used a masking procedure. Each
facial stimulus (target) was presented for either 10 (subliminal)
or 150 (supraliminal) ms, followed by a neutral face presented
for 150 ms (interstimulus interval 1.5 s) (Bernat et al., 2001;
Brázdil et al., 1998; Liddell et al., 2004). The short stimulus
presentation in subliminal condition prevents the subjects to
have a clear cognition of the stimulus, but it allows for a semantic
elaboration of the emotional faces. No target and mask pair
42 M. Balconi, C. Lucchiari / International Journal of Psychophysiology 67 (2008) 4146
depicted the same individual. In total there were 100 target-mask
pairs in each threshold condition (each expression type was
presented twenty times for condition). The condition was not
counterbalanced across subjects (Bernat et al., 2001).
2.4. Procedure
Subjects were seated comfortably in a moderately lighted
room with the monitor screen positioned approximately 100 cm in
front of their eyes. Pictures were presented in a randomised order
in the center of a computer monitor, with a horizontal angle of 4°
and a vertical angle of 6° (STIM 4.2 software). During the
examination, they were requested to continuously focus their eyes
on the small fixation point and to minimize blinking. Participants
were required to observe the stimulus during ERP recording
(passive task). In the subliminal condition it was emphasized that
sometimes the target face would be difficult to see, but to
concentrate as best they could on this stimulus, and that they
would be asked question about these stimuli after the ERP
recording. An explicit response to the emotional features of the
stimulus was not required. This was done for three main reasons:
to assure a real passive task (implicit elaboration of emotions); to
not cause them to be more attentive to the emotional stimuli than
the neutral ones; to not introduce an unequal condition between
subliminal and supraliminal stimulation. In addition, the absence
of an explicit response avoids confounding motor potentials in
addition to brain potentials. Prior to recording ERPs, the subject
was familiarized with the overall procedure (training session),
where every subject saw in a random order all the emotional
stimuli presented in the successive experimental session (a block
of 10 trials, each expression type repeated twice).
2.5. EEG recording
The EEG was recorded with a 62-channel DC amplifer
(SYNAMPS system) and acquisition software (NEUROSCAN
4.2). An ElectroCap with Ag/AgCl electrodes were used to record
EEG from active scalp sites referred to earlobe (10/20 system of
electrode placement). Additionally two EOG electrodes were
sited on the outer side of the eyes. The data were recorded using
sampling rate of 256 Hz, with a frequency band of 0.1 to 60 Hz.
The impedance of recording electrodes was monitored for each
subject prior to data collection and it was always below 5 kΩ.
After EOG correction and visual inspection only artefact-free
trials were considered. Only fourteen electrodes were used for the
successive statistical analysis (four central, Fz, Cz, Pz, Oz; ten
lateral, F3, F4, C3, C4, T3, T4, P3, P4, O1, O2).
2.6. ERD/ERS data reduction
The digital EEG data were bandpass filtered in the gamma
frequency band (3060 Hz). To obtain a signal proportion to the
power of the EEG frequency band, the filtered signal samples
were squared (Pfurtscheller, 1992). Successively, the data were
epoched, triggered each second, using four different time
windows of 100 ms (50150; 150250; 250350; 350
450 ms). An average absolute power value for each electrode
for each condition (five expression types) was calculated. An
average of the pre-experimental absolute power was used to
determine the individual power during no stimulation. From this
reference power value individual power changes during face
viewing were determined as the relative stimulus-related decrease
(desynchronization). In fact, according to ERD/ERS method,
changes in band power were defined as the percentage of a
decrease (ERD) in band power during a test interval (here 900 ms
post-stimulus) as compared to a reference interval (here 1500 ms
before picture onset). For each subject, after band-pass filtering
ERD was calculated within the four time intervals. The average
ERD values across the respective electrode sites were calculated
for each condition, time interval, and emotional category.
2.7. Data analysis
The data were entered into repeated measures analysis of
variance (ANOVA) with four repeated factors: condition (2,
supraliminal and subliminal), time (four time intervals, 4), stimulus
type (emotion type, 5), and electrode sites (cortical sites, 14).
Secondly, in order to analyze widely the cortical distribution of
band modulation, the data were averaged over anterior (F3, Fz, F4),
central (C3, Cz, C4), and posterior (P3, Pz, P4) electrode location,
and secondly over left (F3, C3, T3, P3, O1) and right (F4, C4, T4,
P4, O2) sides. These new values were entered in two distinct
statistical analyses. For all the ANOVAs, degrees of freedom were
GreenhouseGeisser corrected where appropriate.
3. Results
3.1. Behavioral data
The subjects were asked to analyze the stimuli viewed after the
experimental section. Firstly, they evaluated the emotional
significance of each expression by a categorization task. The
five emotional categories were correctly recognized (for happy
96%, sad 94%, angry 97%, fearful 97% and neutral 95% faces).
Successively, in order to distinguish the effect of arousal for
emotional face the subjects evaluated on a Likert scale (5 points)
their responses as a function of the arousing power of each
stimulus (how do you evaluate the arousing power of this
stimulus for you?). Fear (M=4.78), anger (M=4.51), happiness
(M=3.17), sadness (M= 2.76) and neutral (M=2.20) faces differed
in terms of their arousing power. Specifically, ANOVA showed
significant differences between the emotion (F(4, 19)=12.36,
Pb0.001), and post-hoc comparisons (Tukey) revealed that anger
and fear were considered more emotionally arousing than hap-
piness (respectively F(1, 19)= 10.16, Pb0.001; F(1, 19)= 9.74,
Pb0.001) and sadness (F(1, 19)= 11.69, P=0.001; F(1, 19)=
12.36, Pb0.001), as well as than neutral stimuli (F(1, 19)= 14.03,
Pb0.001; F(1, 19) = 15.41, Pb0.001).
3.2. ERD/ERS data
GBA showed sensitivity to Condition (F(1,19)= 7.07, P=
0.001), Type (F(4,19)= 10.12, Pb0.001), Time (F(3,19)= 8.55,
Pb0.001) and Electrodes (F(13,19)=13.09, Pb0.001), as well as
43M. Balconi, C. Lucchiari / International Journal of Psychophysiology 67 (2008) 4146
for Condition× Type, Condition× Time (F(3,19)= 10.98,
Pb0.001), Time×Electrodes (F(39,19) = 14.89, Pb0.001), and
Typ e × Tim e ( F(12,19)= 11.22, Pb0.001).
As shown in Fig. 1, GBA increases in supraliminal condition
compared with subliminal. In addition post-hoc comparison
(contrast analysis) applied to the main effect of Time and Type
revealed that GBA was maximum in 150250 and 250350 time
interval compared with 50150 ms (respectively F(1,19)= 19,03,
Pb0.001), (F(1,19)= 8.70, Pb0.001) and 350450 ms (F(1,19)=
11.24, Pb0.001), (F(1,19)= 9.93, Pb0.001). Moreover, high
arousal emotions have an increased GBA than low arousal emo-
tions (fear vs happiness F(1,19)= 12.96, Pb0.001; vs sadness F
(1,19)= 14.51, Pb0.001; vs neutral F(1,19)=18.73, Pb0.001;
anger vs happiness F(1,19) = 13.08, Pb0.001; vs sadness F
(1,19)=14.11, Pb0.001; vs neutral F(1,19)= 18.16, Pb0.001).
By analyzing interaction effects, high arousal stimuli showed
increased power GBA in conscious condition than in uncon-
scious condition (for fear F(1,19) = 10.99, Pb0.001; for anger
F(1,19)= 14.04, Pb0.001). Secondly, larger synchronization
was found for anger and fear in 150250 and 250350 post-
stimulus than 50150 (F(1,19) = 12.16, Pb0.001; F(1,19)=
15.05, Pb0.001) and 350450 ms (F(1,19) = 9.07, Pb0.001; F
(1,19)= 10.91, Pb0.001). Finally, enhanced brain gamma
oscillations were observed for conscious condition in second
and third time intervals than unconscious condition (respec-
tively F(1,19)= 8.83, Pb0.001; F(1,19)= 8.03, Pb0.001). No
other post-hoc comparison was statistically significant. We can
summarize these results pointing out that 150350 ms post-
stimulus was significant in distinguishing GBA modulation,
and that it was during this time that types of emotion (high/low
arousing) and condition (conscious/unconscious) differences
emerge.
The second order of analysis took into account Laterality (2)
and Location (3) effects. The analysis revealed differences for
Laterality × Condition (F(1,19)= 7.78, P=0.001), as well as
Type× Laterality (F(4,19)= 10.99, Pb0.001). The second
ANOVA revealed a significant Location ×Condition (F(2,19) =
9.15, Pb0.001) and Location× Type (F(8,19) =11.34,
Pb0.001) interaction effects. Specifically, as revealed by the
contrast analysis, GBA synchronizes mainly in the left
hemisphere than in the right hemisphere in supraliminal
condition (F(1,19)= 6.85, P=0.002). Moreover, emotional
faces (both high and low arousing) elicited a dominance in
power synchronization of right hemisphere than neutral faces (F
(1,19)= 9.06, Pb0.001). Finally, as shown in Fig. 2, post-hoc
comparisons showed that supraliminal stimuli induced an
increased GBA in posterior than in the anterior (F(1,19) =5.09,
P=0.002) or central (F(1,19) =6.73, P= 0.002) sites. In parallel,
all the emotional faces differed from the neutral faces in terms of
local distribution on the scalp of GBA: emotional stimuli were
more posterior (Pz) distributed than neutral stimuli.
4. Discussion
The first main result of the study was that GBA was
increased by presentation of a supraliminal emotional face in
Fig. 1. ERD % for Emotion × Time interaction in gamma band. (a) Supraliminal; (b) Subliminal.
Fig. 2. Supraliminal/subliminal comparison (ERD decreasing) as a function of: (a) right/left hemisphere; (b) anterior/central/posterior site.
44 M. Balconi, C. Lucchiari / International Journal of Psychophysiology 67 (2008) 4146
comparison with a subliminal presentation. Indeed for each type
of emotions we revealed that increasing in gamma band
synchronization was significantly higher for conscious emo-
tional stimuli than for unconscious. Moreover, we observed an
enhanced activity of gamma band during 150350 ms post-
stimulus onset, such as an early oscillatory event that showed its
peak at about 200 ms post-stimulus. This time range was found
to be of main importance in emotional face processing, since it
was found to be discriminant in distinguishing between types of
emotion (low or high arousing power). Therefore, we can
suppose that GABA modulation could represent conscious
processing of the subjects for the emotional face during this
early time (Keil et al., 2001).
Secondly, in both supraliminal and subliminal condition we
found that GBA modulation increased linearly as a function of the
degree of arousal that subjects experienced for each facial
stimulus. Indeed, we noticed a similar increasing of GBA for
anger and fear in comparison with happiness, sadness and neutral
stimuli despite the subthreshold or suprathreshold stimulation,
and this increasing was more pronounced between 250350 ms.
We can hypothesize that GBA could be considered not only a
marker of conscious elaboration of emotional expression but even
an index of an enhanced activation (high level of perceived
arousal) of the subject in elaborating significant stimuli, even if
they prevent to reach the level of awareness. Moreover, this
emotional-effect is observable mainly in the early time of stimulus
elaboration. This suggestion is supported by ERP studies on
emotion, and taken together these results demonstrate that
difference in affective significance of a stimulus influences the
brain activity during 150350 time range (Balconi and Pozzoli,
2003; Sato et al., 2000). A related and interesting point is that
information presented to subjects under subliminal condition may
be processed ona high level even if the subject is not aware of this
information. This is in line with studies that have examined
psychophysiological responses to unconscious emotional stimuli:
they were effective both in capturing attention and in eliciting
autonomic response. Subliminal process appears to have a
preattentive origin, because it can be observed to stimuli that
are prevented from reaching conscious recognition. This fast
processing has adaptive value because it allows an immediate
response to a relevant and potentially threatening stimulus, and
this system can operate even prior to the conscious appraisal of the
stimulus.
With regard of the lateralization effect, we found that the left
hemisphere more than the right can mediate conscious
elaboration, since a clear dominance of left side was found in
the supraliminal condition. This result is in line with previous
research, that underlined a left-conscious/right-unconscious
dichotomy. Moreover, supraliminal faces were mainly elabo-
rated in posterior sites if compared to subliminal faces. Clearly
defined synchronization increase over posterior cortical sites in
response to affective stimuli presented supraliminally could be
attributed to specific function of posterior sites for conscious
stimulation (Summerfield et al., 2002). To summarize, a left-
posterior localization is supposed for consciousness.
A second interesting result on cortical distribution of GBA is
that both types of emotions (high or low arousing) induced greater
synchronization over right-posterior regions of the scalp than
neutral stimuli, and this is the case of both supra- and subliminal
condition. This result may elucidate a different role of the two
hemispheres in comprehending the emotional significance of a
facial stimulus. As it was underlined, whereas anterior brain
regions may be important for the modulation of the valence
(pleasant/unpleasant), posterior regions of the right hemisphere
may be involved in the modulation of arousal dimension. Indeed
emotional (and more arousing stimuli) are preferentially
elaborated in the posterior (and right) side than neutral
(nonemotional) stimuli: for example Aftanas et al. (2002) pointed
out a more posterior emotional vs nonemotional distribution of
gamma oscillations, even it was observed specifically for the
narrow bands (theta).
In sum, gamma band can be represented as a marker of
consciousness as well as of the subject's evaluation of the arousing
power of emotional stimuli. In fact it is not only increased by an
aware processing but it appears to differentiate high from low-
arousing emotional faces. This effect was found in both
supraliminal and subliminal condition. Moreover, GBA resulted
more right-posterior distributed for emotional vs nonemotional
stimuli, and more left-posterior localized in case of conscious
elaboration in comparison with unconscious. More generally
posterior sites appear to be discriminant for emotions, indepen-
dently from the type of facial expression. Finally, methodologi-
cally the present results indicate that gamma frequency band
analysis offers a powerful tool for studying cortical activation
patterns during emotional information processing.
References
Aftanas, L.I., Varlamon, A.A., Pavlov, S.V., Makhnev, V.P., Reva, N.V., 2001.
Affective picture processing: event-related synchronization within individ-
ually defined human theta band is modulated by valence dimension.
Neurosci. Lett. 303, 115118.
Aftanas, L.I., Varlamon, A.A., Pavlov, S.V., Makhnev, V.P., Reva, N.V.,
2002. Time-dependent cortical asymmetries induced by emotional
arousal: EEG analysis of event-related synchronization and desynchro-
nization in individually defined frequency bands. Int. J. Psychophysiol.
44, 6782.
Balconi, M., Pozzoli, U., 2003. Face-selective processing and the effect of
pleasant and unpleasant emotional expressions on ERP correlates. Int. J.
Psychophysiol. 49, 6774.
Balconi, M., Pozzoli, U., 2007. Event-related oscillations (EROs) and event-
related potentials (ERPs) comparison in facial expression recognition.
J. Neuropsychology 1, 283294.
Başar, E., 1999. Brain function and oscillations. II. Integrative brain function.
Neurophysiology and cognitive processes. Springer, Heidelberg.
Başar, E., Başar-Eroğlu, C., Karakaş, S., Schürman, M., 1999. Are cognitive
processes manifested in event-related gamma, alpha, theta and delta
oscillations in the EEG? Neurosci. Lett. 259, 165168.
Bernat, E., Bunce, S., Shevrin, H., 2001. Event-related brain potentials
differentiate positive and negative mood adjectives during both supraliminal
and subliminal visual processing. Int. J. Psychophysiol. 42, 1134.
Brázdil, M., Rektor, I., Dufek, M., Jurák, P., Pavel, D., 1998. Effect of
subthreshold target stimuli on event-related potentials. Electroencephalogr.
Clin. Neurophysiol. 107, 6468.
Bunce, S.C., Bernat, E., Wong, P.S., Shevrin, H., 1999. Further evidence for
unconscious learning: preliminary support for the conditioning of facial
EMG to subliminal stimuli. J. Psychiat. Res. 33, 341347.
Crick, F., Koch, C., 1998. Consciousness and neuroscience. Cereb. Cortex 8,
97107.
45M. Balconi, C. Lucchiari / International Journal of Psychophysiology 67 (2008) 4146
Ekman, P., Friesen, W.V., 1976. Pictures of facial affect. Consulting Psychologist
Press, Palo Alto.
Esteves, F., Parra, C., Dimberg, U., Öhman, A., 1994. Nonconscious associative
learning: pavlovian conditioning of skin conductance responses to masked
fear-relevant stimuli. Psychophysiology 31, 375385.
Keil, A., Müller, M.M., Gruber, T., Wienbruch, C., Stolarova, M., Elbert, T.,
2001. Effects of emotional arousal in the cerebral hemispheres: a study of
oscillatory brain activity and event-related potentials. Clin. Neuropsychol.
112, 20572068.
Kiefer, M., Spitzer, M., 2000. Time course of conscious and unconscious
semantic brain activation. Neuroreport 11, 24012407.
Krause, C.M., 2003. Brain electric oscillations and cognitive processes. In:
Hugdahl, K. (Ed.), Experimental Methods in Neuropsychology. Kluwer,
New York, pp. 11130.
Krause, C.M., Viemerö, V., Rosenqvist, A., Sillanmäki, L., Åström, T., 2000.
Relative electroencephalographic desynchronization and synchronization in
humans to emotional film content: an analysis of the 46, 68, 810 and
1012 Hz frequency bands. Neurosci. Lett. 286, 912.
Krolak-Salmon, P., Fischer, C., Vighetto, A., Mauguière, F., 2001. Processing of
facial emotional expression: spatio-temporal data as assessed by scalp event-
related potentials. Eur. J. Neurosci. 13, 987994.
Lang, P.J., Greenwald, M.K., Bradley, M.M., Hamm, A.O., 1993. Looking at
pictures: affective, facial, visceral and behavioral reactions. Psychophysi-
ology 30, 261273.
Lang, P.J., Bradley, M.M., Fitzsimmons, J.R., Cuthbert, B.N., Scott, J.D.,
Moulder, B., Nangia, B., 1998. Emotional arousal and activation of the
visual cortex: an fMRI analysis. Psychophysiology 35, 199210.
LeDoux, J.E., 1998. The Emotional Brain: The Mysterious Underpinnings of
Emotional Life. Touchstone, New York.
Liddell, B.J., Williams, L.M., Rathjen, J., Shevrin, H., Gordon, E., 2004. A
temporal dissociation of subliminal vs. supraliminal fear perception: an
event-related potential study. J. Cogn. Neurosci. 5, 551565.
Merikle, P.M., Smilek, D., Eastwood, J.D., 2001. Perception without awareness:
Perspectives from cognitive psychology. Cognition 79, 115134.
Morris, J.S., Buchel, C., Dolan, R.J., 2001. Parallel neural responses in
amygdala subregions and sensory cortex during implicit fear conditioning.
Neuroimage 13, 10441052.
Müller, M.M., Bosch, J., Elbert, T., Kreiter, A., Sosa, M.V., Sosa, P.V.,
Rockstroh, B., 1996. Visually induced gamma-band responses in human
electroencephalographic activity a link to animal studies. Exp. Brain Res.
112, 96102.
Pfurtscheller, G., 1992. Event-related synchronization (ERS): an electrophys-
iological correlate of cortical areas at rest. Electroencephalogr. Clin.
Neurophysiol. 83, 6269.
Sato, W., Takanori, K., Sakiko, Y., Michikazu, M., 2000. Emotional expression
boosts earlyvisual processingof the face: ERP recording andits decomposition
by independent component analysis. Neuroreport 12, 709714.
Schupp, H.T., Cuthbert, B.N., Bradley, M.M., Cacioppo, J.T., Ito, T., Lang, P.J.,
2000. Affective picture processing: the late positive potential is modulated
by motivational relevance. Psychophysiology 37, 257261.
Streit, M., Wölwer, W., Brinkmeyer, J., Ihl, R., Gaebel, W., 2000.
Electrophysiological correlates of emotional and structural face processing
in humans. Neurosci. Lett. 278, 1316.
Summerfield, C., Jack, A.I., Burgess, A.P., 2002. Induced gamma activity is
associated with conscious awareness of pattern marked nouns. Int. J.
Psychophysiol. 44, 93100.
Whalen, P.J., Rauch, S.L., Etcoff, N.L., McInerney, S.C., Lee, M.B., Jenike, M.A.,
1998. Masked presentations of emotional facial expressions modulate
amygdala activity without explicit knowledge. J. Neurosci 18, 411418.
Wong, P.S., Shevrin, H., Williams, W.J., 1994. Conscious and nonconscious
processes: An ERP index of an anticipatory response in a conditioning
paradigm using visually masked stimuli. Psychophysiology 31, 87101.
46 M. Balconi, C. Lucchiari / International Journal of Psychophysiology 67 (2008) 4146
... Subjects in the DEAP experiments produced higher gamma and frontal midline theta power while watching emotion provoking music videos [54]. Balconi and Lucchiari [55] reported the Gamma band activity is a marker of the subject's evaluation of the Arousal dimension in emotional faces stimuli. Yang et al. [56] revealed that the network connections in the high Gamma band have signiicant diferences among the positive, neutral, and negative emotional states. ...
Article
Full-text available
Emotion recognition technology through analyzing the EEG signal is currently an essential concept in Artificial Intelligence and holds great potential in emotional health care, human-computer interaction, multimedia content recommendation, etc. Though there have been several works devoted to reviewing EEG-based emotion recognition, the content of these reviews needs to be updated. In addition, those works are either fragmented in content or only focus on specific techniques adopted in this area but neglect the holistic perspective of the entire technical routes. Hence, in this paper, we review from the perspective of researchers who try to take the first step on this topic. We review the recent representative works in the EEG-based emotion recognition research and provide a tutorial to guide the researchers to start from the beginning. The scientific basis of EEG-based emotion recognition in the psychological and physiological levels is introduced. Further, we categorize these reviewed works into different technical routes and illustrate the theoretical basis and the research motivation, which will help the readers better understand why those techniques are studied and employed. At last, existing challenges and future investigations are also discussed in this paper, which guides the researchers to decide potential future research directions.
... Subjects in the DEAP experiments produced higher gamma and frontal midline theta power while watching emotion provoking music videos [54]. Balconi and Lucchiari [55] reported the Gamma band activity is a marker of the subject's evaluation of the Arousal dimension in emotional faces stimuli. Yang et al. [56] revealed that the network connections in the high Gamma band have significant differences among the positive, neutral, and negative emotional states. ...
Preprint
Full-text available
Emotion recognition technology through analyzing the EEG signal is currently an essential concept in Artificial Intelligence and holds great potential in emotional health care, human-computer interaction, multimedia content recommendation, etc. Though there have been several works devoted to reviewing EEG-based emotion recognition, the content of these reviews needs to be updated. In addition, those works are either fragmented in content or only focus on specific techniques adopted in this area but neglect the holistic perspective of the entire technical routes. Hence, in this paper, we review from the perspective of researchers who try to take the first step on this topic. We review the recent representative works in the EEG-based emotion recognition research and provide a tutorial to guide the researchers to start from the beginning. The scientific basis of EEG-based emotion recognition in the psychological and physiological levels is introduced. Further, we categorize these reviewed works into different technical routes and illustrate the theoretical basis and the research motivation, which will help the readers better understand why those techniques are studied and employed. At last, existing challenges and future investigations are also discussed in this paper, which guides the researchers to decide potential future research directions.
... EEG is non-invasive, has high temporal resolution and can detect changes in brain activity over a span of milliseconds. EEG frequency bands are known to correlate with emotions [23], [24]. Handcoded EEG descriptors such as Spectral Power Vectors enable emotion detection while Convolutional Neural Networks (CNNs) can automatically learn cognitive and emotional correlates [25], [21]. ...
Preprint
Full-text available
While Parkinson's disease (PD) is typically characterized by motor disorder, there is evidence of diminished emotion perception in PD patients. This study examines the utility of affective Electroencephalography (EEG) signals to understand emotional differences between PD vs Healthy Controls (HC), and for automated PD detection. Employing traditional machine learning and deep learning methods, we explore (a) dimensional and categorical emotion recognition, and (b) PD vs HC classification from emotional EEG signals. Our results reveal that PD patients comprehend arousal better than valence, and amongst emotion categories, \textit{fear}, \textit{disgust} and \textit{surprise} less accurately, and \textit{sadness} most accurately. Mislabeling analyses confirm confounds among opposite-valence emotions with PD data. Emotional EEG responses also achieve near-perfect PD vs HC recognition. {Cumulatively, our study demonstrates that (a) examining \textit{implicit} responses alone enables (i) discovery of valence-related impairments in PD patients, and (ii) differentiation of PD from HC, and (b) emotional EEG analysis is an ecologically-valid, effective, facile and sustainable tool for PD diagnosis vis-\'a-vis self reports, expert assessments and resting-state analysis.}
... In human relationships, the face is a significant social stimulus [50][51][52][53][54]. Face processing may be separated into a first perceptive phase, in which the person completes the "structural codes" of face and a second phase in which the subject completes the "expression code" implicated in the decoding of emotional facial expressions [55]. The first is thought to be processed separately from complex facial information such as emotional meaning [56][57][58][59][60][61]. Here we argue that the simple presentation of a face receiving painful or non-painful stimulation in the context of pain observation in others activates frontal brain regions connected to emotional regulation of the empathic response, more than somatosensory areas. ...
Article
This research explored how the manipulation of interoceptive attentiveness (IA) can influence the frontal (dorsolateral prefrontal cortex (DLPFC) and somatosensory cortices) activity associated with the emotional regulation and sensory response of observing pain in others. 20 individuals were asked to observe face versus hand, painful/non-painful stimuli in an individual versus social condition while brain hemodynamic response (oxygenated (O2Hb) and deoxygenated hemoglobin (HHb) components) was measured via functional Near-Infrared Spectroscopy (fNIRS). Images represented either a single person (individual condition) or two persons in social interaction (social condition) both for the pain and body part set of stimuli. The participants were split into experimental (EXP) and control (CNT) groups, with the EXP explicitly required to concentrate on its interoceptive correlates while observing the stimuli. Quantitative statistical analyses were applied to both oxy- and deoxy-Hb data. Firstly, significantly higher brain responsiveness was detected for pain in comparison to no-pain stimuli in the individual condition. Secondly, a left/right hemispheric lateralization was found for the individual and social condition, respectively, in both groups. Besides, both groups showed higher DLPFC activation for face stimuli presented in the individual condition compared to hand stimuli in the social condition. However, face stimuli activation prevailed for the EXP group, suggesting the IA phenomenon has certain features, namely it manifests itself in the individual condition and for pain stimuli. We can conclude that IA promoted the recruitment of internal adaptive regulatory strategies by engaging both DLPFC and somatosensory regions towards emotionally relevant stimuli.
... More recently, many reviews have extended the functional role of event-related gamma in the human brain (Başar, 2013;Başar et al., 2001;Başar-Eroglu et al., 1996;Herrmann et al., ,2004Herrmann et al., , , 2010Herrmann & Knight, 2001;Jensen et al., 2007;Singer, 1999;Tallon-Baudry & Bertrand, 1999). More specifically, gamma responses have been linked to episodic memory processes (Başar-Eroglu & Başar, 1991;Debener et al., 2003;Gruber et al., 2002;Herrmann et al., 2004;Kaiser et al., 2003;Karakaş et al., 2000;Miltner et al., 1999;Tallon-Baudry et al., 1998) and the elaboration of emotional valence of stimuli (Balconi & Lucchiari, 2008;Keil et al., 1999Keil et al., , 2001Keil et al., , 2007Luo et al., 2007Luo et al., , 2009Martini et al., 2012;Müller et al., 1999;Sato et al., 2011). In addition, the causal role of brain systems generating gamma was highlighted in a recent review on the beneficial effects of interventions targeting gamma responses in patients with neuropsychiatric diseases (Struber and Herrmann, 2020). ...
Article
Cerebrospinal and structural-molecular neuroimaging in-vivo biomarkers are recommended for diagnostic purposes in Alzheimer’s disease (AD) and other dementias; however, they do not explain the effects of AD neuropathology on neurophysiological mechanisms underpinning cognitive processes. Here, an Expert Panel from the Electrophysiology Professional Interest Area of the Alzheimer’s Association reviewed the field literature and reached consensus on the event-related electroencephalographic oscillations (EROs) that show consistent abnormalities in patients with significant cognitive deficits due to Alzheimer’s, Parkinson’s (PD), Lewy body (LBD), and cerebrovascular diseases. Converging evidence from oddball paradigms showed that, as compared to cognitively unimpaired (CU) older adults, AD patients had lower amplitude in widespread delta (>4 Hz) and theta (4–7 Hz) phase-locked EROs as a function of disease severity. Similar effects were also observed in PD, LBD, and/or cerebrovascular cognitive impairment patients. Non-phase-locked alpha (8–12 Hz) and beta (13–30 Hz) oscillations were abnormally reduced (event-related desynchronization, ERD) in AD patients relative to CU. However, studies on patients with other dementias remain lacking. Delta and theta phase-locked EROs during oddball tasks may be useful neurophysiological biomarkers of cognitive systems at work in heuristic and intervention clinical trials performed in AD patients, but more research is needed regarding their potential role for other dementias.
Article
Developing robust cross-subject or cross-session EEG-based affective models is a key issue in affective brain-computer interfaces, which often suffer from the individual differences and non-stationarity of EEG. Aiming at generalizing the affective model across subjects and sessions, this paper proposes a novel transfer learning strategy with Deep Subdomain Associate Adaptation Network (DSAAN) for EEG emotion recognition. Domain was divided into subdomains according to the sample labels, and the source domain use the true sample labels while the target domain use the predicted pseudo-labels. DSAAN was established as a transfer network by aligning the relevant subdomain distributions based on Subdomain Associate Loop (SAL). The adaptation of networks was achieved by minimizing the summation of source domain classification loss and SAL loss. For the purpose of verifying the generalization of DSAAN, we carried out the cross-session and cross-subject EEG emotion recognition experiments on benchmark SEED and DEAP. Compared with existing domain adaptation methods, the DSAAN achieved outstanding classification results.
Article
Electroencephalogram (EEG)-based affective computing brain-computer interfaces provide the capability for machines to understand human intentions. In practice, people are more concerned with the strength of a certain emotional state over a short period of time, which was called as fine-grained-level emotion in this paper. In this study, we built a fine-grained-level emotion EEG dataset that contains two coarse-grained emotions and four corresponding fine-grained-level emotions. To fully extract the features of the EEG signals, we proposed a corresponding fine-grained emotion EEG network (FG-emotionNet) for spatial-temporal feature extraction. Each feature extraction layer is linked to raw EEG signals to alleviate overfitting and ensure that the spatial features of each scale can be extracted from the raw signals. Moreover, all previous scale features are fused before the current spatial-feature layer to enhance the scale features in the spatial block. Additionally, long short-term memory is adopted as the temporal block to extract the temporal features based on spatial features and classify the category of fine-grained emotions. Subject-dependent and cross-session experiments demonstrated that the performance of the proposed method is superior to that of the representative methods in emotion recognition and similar structure methods with proposed method.
Article
Recently, the combination of neural network and attention mechanism is widely employed for electroencephalogram (EEG) emotion recognition (EER) and has achieved remarkable results. Nevertheless, most of them ignored the individual information in and within different frequency bands, so they just applied a single-layer attention mechanism to the entire EEG signals, with relatively single feature expression. To overcome the shortcoming, a spatial-frequency convolutional self-attention network (SFCSAN) is proposed in this paper to integrate the feature learning from both spatial and frequency domain of EEG signals. In this model, the intra-frequency band self-attention is employed to learn frequency information from each frequency band, and inter-frequency band mapping further maps them into final attention representation to learn their complementary frequency information. Additionally, a parallel convolutional neural network (PCNN) layer is used to excavate the spatial information of EEG signals. By incorporating spatial and frequency band information, the SFCSAN can fully utilize the spatial and frequency domain information of EEG signals for emotion recognition. The experiments conducted on two public EEG emotion datasets achieved the average accuracy of 95.15%/95.76%/95.64%/95.86% on valence/arousal/dominance/liking label for DEAP dataset, and 93.77%/95.80%/96.26% on valence/arousal/dominance label for DREAMER dataset, which all demonstrate that the proposed method is conducive to enhancing the importing of emotion-salient information and generating better recognition performance. The code of our work is available on “https://github.com/qeebeast7/SFCSAN”
Article
The Nataraja is perhaps the most well-recognized anthropomorphic form of the Hindu god, Shiva . This paper articulates a set of principles for a recently developed conceptual lens in systems thinking called Holistic Flexibility for flexible and responsible management practice. The five most important function of the Nataraja, or the panchakritya , have been drawn on to articulate these principles; these principles are – "system as becoming", drawing from srishti or creation, "transformative flexibility", drawing from samhara or transformation, "responsible practice" drawing from tirobhava or (freedom from) ignorance, "spiral of learning" drawing from samhara or liberation, and "pragmatic artistry" drawing from sthithi or assurance. An argument is presented to establish the importance of management consciousness drawing from the Shiva philosophy. Behaviors associated with the principles are enlisted along with the challenges for managers to display these behaviors. The discussions presented argue that Holistic Flexibility and its principles can lend a new character to systems thinking as a state of mind to supersede a rational-analytical approach.
Article
Full-text available
Processing of negative affective pictures typically leads to desynchronization of alpha-to-beta frequencies (ERD) and synchronization of gamma frequencies (ERS). Given that in predictive coding higher frequencies have been associated with prediction errors, while lower frequencies have been linked to expectations, we tested the hypothesis that alpha-to-beta ERD and gamma ERS induced by aversive pictures are associated with expectations and prediction errors, respectively. We recorded EEG while volunteers were involved in a probabilistically cued affective picture task using three different negative valences to produce expectations and prediction errors. Our data show that alpha-to-beta band activity after stimulus presentation was related to the expected valence of the stimulus as predicted by a cue. The absolute mismatch of the expected and actual valence, which denotes an absolute prediction error was related to increases in alpha, beta and gamma band activity. This demonstrates that top-down predictions and bottom-up prediction errors are represented in typical spectral patterns associated with affective picture processing. This study provides direct experimental evidence that negative affective picture processing can be described by neuronal predictive coding computations.
Chapter
Full-text available
Recently, numerous novel methods have flourished in brain research and cognitive sciences, increasing our knowledge of the ways in which the human brain processes information. Experimental studies utilizing modern neurophysiological and neuroimaging techniques (EEG; electroencephalogram, MEG; magnetoencephalogram; fMRI; functional magnetic resonance imaging, PET; positron emission tomography) in association with cognitive processing have provided an opportunity to approach subtle brain-behavior relationships in a more direct and empirical manner than ever before. Although being one of the oldest psychophysiological methods to study brain activity, the electroencephalogram (EEG) can successfully be utilized in modern brain research in order to assess brain activity during cognitive functioning. In this chapter, the role of brain electric oscillations in revealing the neural basis of human cognitive and memory processes will be discussed.
Article
Full-text available
Objective: The present study aimed at examining the time course and topography of oscillatory brain activity and event-related potentials (ERPs) in response to laterally presented affective pictures. Methods: Electroencephalography was recorded from 129 electrodes in 10 healthy university students during presentation of pictures from the international affective picture system. Frequency measures and ERPs were obtained for pleasant, neutral, and unpleasant pictures. Results: In accordance with previous reports, a modulation of the late positive ERP wave at parietal recording sites was found as a function of emotional arousal. Early mid gamma band activity (GBA; 30-45 Hz) at 80 ms post-stimulus was enhanced in response to aversive stimuli only, whereas the higher GBA (46-65 Hz) at 500 ms showed an enhancement of arousing, compared to neutral pictures. ERP and late gamma effects showed a pronounced right-hemisphere preponderance, but differed in terms of topographical distribution. Conclusions: Late gamma activity may represent a correlate of widespread cortical networks processing different aspects of emotionally arousing visual objects. In contrast, differences between affective categories in early gamma activity might reflect fast detection of aversive stimulus features.
Article
Full-text available
Visual presentation of an object produces firing patterns in cell assemblies representing the features of the object. Based on theoretical considerations and animal experiments, it has been suggested that the binding of neuronal representations of the various features is achieved through synchronization of the oscillatory firing patterns. The present study demonstrates that stimulus-induced gamma-band responses can be recorded non-invasively from human subjects attending to a single moving bar. This finding indicates the synchronization of oscillatory activity in a large group of cortical neurons. Gamma-band responses were not as apparent in the presence of two independently moving stimuli, suggesting that the neuronal activity patterns of different objects are not synchronized. These results open a new paradigm for investigating the mechanisms of feature binding and association building in relation to subjective perception.
Article
Recent studies have shown that the late positive component of the event-related-potential (ERP) is enhanced for emotional pictures, presented in an oddball paradigm, evaluated as distant from an established affective context. In other research, with context-free, random presentation, affectively intense pictures (pleasant and unpleasant) prompted similar enhanced ERP late positivity (compared with the neutral picture response). In an effort to reconcile interpretations of the late positive potential (LPP), ERPs to randomly ordered pictures were assessed, but using the faster presentation rate, brief exposure (1.5 s), and distinct sequences of six pictures, as in studies using an oddball based on evaluative distance. Again, results showed larger LPPs to pleasant and unpleasant pictures, compared with neutral pictures. Furthermore, affective pictures of high arousal elicited larger LPPs than less affectively intense pictures. The data support the view that late positivity to affective pictures is modulated both by their intrinsic motivational significance and the evaluative context of picture presentation.
Article
The main purposes of this review are to set out for neuroscientists one possible approach to the problem of consciousness and to describe the relevant ongoing experimental work. We have not attempted an exhaustive review of other approaches.
Article
Functional activity in the visual cortex was assessed using functional magnetic resonance imaging technology while participants viewed a series of pleasant, neutral, or unpleasant pictures. Coronal images at four different locations in the occipital cortex were acquired during each of eight 12-s picture presentation periods (on) and 12-s interpicture interval (off). The extent of functional activation was larger in the right than the left hemisphere and larger in the occipital than in the occipitoparietal regions during processing of all picture contents compared with the interpicture intervals. More importantly, functional activity was significantly greater in all sampled brain regions when processing emotional (pleasant or unpleasant) pictures than when processing neutral stimuli. In Experiment 2, a hypothesis that these differences were an artifact of differential eye movements was ruled out. Whereas both emotional and neutral pictures produced activity centered on the calcarine fissure (Area 17), only emotional pictures also produced sizable clusters bilaterally in the occipital gyrus, in the right fusiform gyrus, and in the right inferior and superior parietal lobules.