Content uploaded by Vincy Chan
Author content
All content in this area was uploaded by Vincy Chan on Oct 17, 2014
Content may be subject to copyright.
Journal of Undergraduate Life Sciences
70
JULS
Review Arcle
The perception and recognition of emotions and
facial expressions
Vincy Chan
Third Year Psychology Specialist with a Health Studies Major, University of Toronto; email: vincy.chan@utoronto.ca.
Abstract
The perception of emotions and the recognition of facial expressions play a critical role in social interaction between humans. Faces
communicate a great deal of information, including dynamic features, such as an individual’s internal emotional state, and static features,
such as a person’s identity. Two major views have evolved from the investigation of how facial expressions are perceived and processed, the
discrete category view and the dimensional theory. According to the discrete category view, basic facial expressions convey discrete and
specic emotions: anger, happiness, surprise, fear, disgust, and sadness. Conversely, the dimensional view suggests that the mental repre-
sentation of emotional space consists of continuous underlying dimensions in which similar emotions are clustered together while dierent
ones are far apart. While both theories postulate that aective information is resistant to contextual inuences, research on this topic has
provided reasons to believe that the relationship between facial expressions and their contexts may play an important role in determining
the perceived emotion. Similarly, studies looking at the right hemisphere and the fusiform face area (FFA) have led researchers to suggest
that factors other than the presence of faces, such as experience and training, can also activate the FFA. This review looks at the role of facial
expressions in everyday life and the two opposing theories on how facial expressions are perceived and processed in the brain. Specically,
the malleability of emotion perception and face recognition and the brain regions involved in emotion are explored.
Introducon
Emotions play a fundamental role in human interactions and
experiences. Emotion (or aect) is the feelings that involve sub-
jective evaluation, physiological processes, and cognitive beliefs
[1]. Regardless of the situation in everyday life, humans endlessly
attempt to decipher social and emotional cues from one another.
Research into the cognitive neuroscience of human social behav-
iour has shown that social behaviour is tightly linked to emotions
[2]. Similarly, facial expressions are a particularly important source
for obtaining social and emotional information [2, 3] and are also
an essential aspect of social cognition, as faces communicate a great
deal of information such as an individual’s internal emotional state
and a person’s identity [4, 5].
e importance of facial perception and recognition is ex-
emplied in patients with prosopagnosia. Individuals with pros-
opagnosia are unable to recognize or dierentiate among faces,
although other objects in their visual modality can be correctly
identied [6]. To compensate for the decit, these patients rely on
visual non-facial information such as a person’s clothing, or on in-
formation in a non-visual modality, such as voice [7]. Clearly, using
the above techniques to distinguish one individual from another
poses a great problem for prosopagnosic patients, as much visual
non-facial information and non-visual modality are not unique
to one particular individual. erefore, the ability to perceive and
recognize faces is a crucial aspect of human interaction.
Subcorcal brain regions involved in emoon
Subcortical regions of the brain, including the amygdala,
hippocampus, hypothalamus, and regions of the cingulate cor-
tex, enable us to react to emotional stimuli quickly. For example,
the amygdala plays a role in fear and emotional learning [8-10].
Lesions of the amygdala interfere with the processing of emotional
information [6], making patients with amygdala damage unable to
detect aversive emotional cues in the visual and auditory stimuli
[11, 12].
e hippocampus is critical in putting emotion into context
[6]. It has been suggested that the main problem with patients
suering from mental disorders is that they are unable to express
their emotions appropriately [13]. Specically, research with post-
traumatic stress disorder (PTSD) patients shows the presence of
hippocampal atrophy, which is thought to be due to the excessive
release of glucocorticoids. is suggests that impaired functioning
of the hippocampus may contribute to the emotional dysfunction
seen in PTSD [14].
e hypothalamus allows for the quick processing of emotional
information [15]. is region of the brain is closely connected with
the amygdala and serves as an important relay station for informa-
tion going into and receiving information out of the amygdala. It
also meditates certain autonomic processes and endocrine reac-
tions [6]. For instance, it helps to coordinate the physical events
that prepare the organism for approach or withdrawal, also known
as the “ght or ight” response.
Finally, the cingulate cortex, viewed as a part of the limbic
system, is an important brain structure in a variety of cognitive
functions, including emotional self-control [6]. Of particular im-
portance in interfacing emotion and cognition are the anterior cin-
gulate gyrus and the retrosplenial cortex. e retrosplenial cortex
mediates interaction between emotional and cognitive processes
[16] and lesions to the anterior cingulate cortex result in apathy,
inattention, and changes in personality [17].
Journal of Undergraduate Life Sciences
Volume 3 • No. 1 • Spring 2009
71
Corcal brain regions involved in emoon
Cortical regions of the brain, such as the prefrontal cortex, the
insula [18], and the parietal lobes also play a role in emotion. ey
use the quick reactions to emotional stimuli made possible through
the subcortical brain regions to inuence more complicated aspects
of our behaviour. One region of the cortical brain that is important
in emotion is the prefrontal cortex, particularly the orbitofrontal
prefrontal cortex (OFC) and the dorsolateral prefrontal cortex
(DLPFC) [6]. e OFC plays a role in emotion regulation, reward,
and punishment. Individuals with damage to the OFC have diculty
anticipating the consequences of their actions and learning from their
mistakes [19, 20] and exhibit perseveration even when contingencies
change [21]. e DLPFC is sensitive to the integration of cognition
and emotion [22] and is involved in the goal directed behaviours that
are inuenced by positive and negative emotional states [13].
e parietal lobe, particularly the right parietotemporal region,
plays an important role in one’s ability to perceive, interpret, and
recall emotionally meaningful information [6] and in appreciating
the emotional signicance of visual information [23, 24]. Patients
with damage to this region compared to those with damage to the
le hemisphere perform poorer when asked to discriminate be-
tween emotional faces or to name emotional scenes [25]. ey also
exhibit more diculties matching emotional expressions [26] and
grouping emotional scenes and faces [27-29].
Malleability of emoon percepon and face
recognion
Emoon percepon, context, and adaptaon.
Research into how facial expressions are perceived and pro-
cessed in the brain has resulted in two major views, the discrete
category view and the dimensional view [3, 30]. According to the
discrete category view [31-33], basic facial expressions convey
discrete and specic emotions: anger, happiness, surprise, fear,
disgust, and sadness. Conversely, the dimensional view [34, 35]
suggests that facial expressions convey values on the dimensions of
valence (pleasant vs. unpleasant) and arousal (high vs. low). ese
values, obtained through the facial expressions, are consequently
used to attribute a specic emotion to the facial expression.
While both theories hypothesize that aective information is
resistant to contextual inuences, extensive research on this topic
has provided reasons to believe that the relationship between facial
expressions and their contexts may play an important role in deter-
mining the perceived emotion. Although early studies on this topic
have resulted in inconsistent results, with some studies demon-
strating insignicant contextual eects [36, 37] and other showing
strong contextual eects [38], more recent studies are demonstrat-
ing that the categorization of emotions from facial expressions can
be strongly inuenced by bodily and scene context.
In 2008, Aviezer and colleagues pointed out that previous stud-
ies have resulted in conicting ndings because they did not take
into account the perceptual similarities among facial expressions
[39], such as the strong similarity between anger and disgust [40].
In this study, they showed that bodily context inuenced the facial
expression that was being perceived. Specically, aective informa-
tion from facial expressions was perceived as noticeably dierent on
dierent bodily contexts, hence diering from the discrete category
view. eir results also contrasted with the dimensional view, as
context also changed the ratings of the valence and arousal of pre-
sented faces. Of particular importance is the nding that the above
eects are dependent on the similarity between the presented facial
expression (e.g., anger) and the facial expression that is typically as-
sociated with the context emotion (e.g., a body holding a knife).
Studies using an adaptation paradigm have also demonstrated
the malleability of facial expressions. Adaptation paradigms allow
for the further investigation of the neural representations of faces
by inducing aereects through prolonged exposure to a stimulus
[41]. In this context, adaptation to a particular facial expression has
been shown to result in a shi in the category boundaries between
two dierent expressions [42]. at is, a previously ambiguous
expression was seen as the opposite expression from the adapt-
ing expression. It is believed that adaptation fatigues the neural
mechanism associated with a particular stimulus and thus, there
is a decrease in the responsiveness of a neural representation to a
constant stimulus [43, 44].
Using an adaptation paradigm, Ellamil, Susskind, and
Anderson examined the identity invariance of facial expressions.
e test stimulus in this experiment consisted of four dierent
expressions – anger, disgust, fear, and surprise, as the pairs of anger
and surprise and of disgust and fear were shown to be most dif-
ferent from one another [39]. ey found that the perception of
facial expressions was reliably and noticeably biased by prolonged
exposure to images of facial expressions, which was consistent with
previous facial expression adaptation studies [41, 42]. is means
that adapting to one facial expression moved the category bound-
ary towards the matching prototype, and thus, required the facial
expression to be more prototypical to be categorized correctly.
Lastly, situations other than environmental and bodily contexts
can also inuence the perception of facial emotions. Kirsh and Mounts
observed a negative processing bias in the recognition of emotional
expressions with violent video games [45]. Consistent with previous
research [46-49], there was a reduction in the “happy face” advantage
(the generally faster ability to identify happy expressions compared
to angry ones) following violent video games, providing further
evidence for the relationship between exposure to violent media and
aggressive biases in social information processing [47, 50-55].
Face recognion, the right hemisphere, and the fusiform face
area.
Studies with neurologically intact individuals and patients
with hemisphere damage have shown that the right hemisphere
of the brain is particularly adept at facial recognition [10, 56, 57].
e inversion eect, which is the greater diculty in remembering
inverted rather than upright stimuli, is oen used in the research
of face recognition.
In 1970, Yin postulated that congural information is espe-
cially important for recognizing faces [58]. In this study, partici-
pants viewed pictures of either faces or houses in the upright posi-
tion and were subsequently asked to identify the items that were
previously presented. When this task was completed for the second
time, the pictures were presented in an inverted orientation. is
manipulation allowed for the investigation of the signicance of
the loss of congural information (e.g., when in faces, the mouth
is no longer below the nose). In this study, Yin discovered that
neurologically intact individuals exhibited an inversion eect for
Journal of Undergraduate Life Sciences
72
faces but not for houses. is suggests that congural information
plays a more important role in recognizing faces and that faces are
processed dierently than other objects.
Further investigation into the malleability of the right hemi-
sphere in face recognition has led researchers to the role of expe-
rience or expertise in face processing. Diamond and Carey have
postulated that the inversion eect may be observed for any objects
with experience [59]. According to this hypothesis, the inversion
eect is observed because faces are a class of objects in which hu-
mans have much experience. Hence, with experience, a congural
strategy would develop for any particular class of objects and the
inversion eect would be observed as well. To test this hypothesis,
they recruited college students and judges of show dogs for an ex-
periment with a similar procedure used by Yin. e results showed
that the judges of show dogs had as large an inversion eect for
show dogs as for faces, allowing the researchers to conclude that
expertise plays an important role in the inversion eect.
Similarly, recent neuroimaging studies have also demonstrated
the role of the right hemisphere in face recognition. In one study [60],
the fusiform face area (FFA) was identied individually for eleven car
experts and eight bird experts by nding the brain region that exhib-
ited a larger response to faces than other objects. e results showed
that there was a greater FFA activation for cars than for other objects,
including birds. Conversely, the bird experts exhibited greater FFA
activation for birds than for other objects, including cars.
Moreover, activation of the FFA increases as individuals be-
come more experienced. In a study by Gauthier and colleagues,
participants were trained to become experts at recognizing novel
objects known as “greebles”. When they were judged by the re-
searchers to be experts at dierentiating “greebles” from one family
to another, they observed activation in the right FFA in the par-
ticipants. When the inversion eect was looked at, they found that
the activation of the right FFA for upright as compared to inverted
“greebles” increased with training [61].
Summary
e ability to perceive and recognize emotions and facial ex-
pressions is critical, with research showing that vital information can
be inferred from facial expressions [2, 3]. In social environments,
emotions contribute in regulating social behaviour and communi-
cate important information and social signals. Conicting theories
regarding how facial expressions are perceived and processed in
the brain and the use of adaptation paradigms have led research-
ers to extensively study the malleability of emotions and facial
expressions. Similarly, studies looking at the brain regions that are
involved in emotions have shown that factors, such as experience
and training, can also activate the FFA, suggesting that this area of
the brain is not only specialized for the processing of faces. Further
research into the malleability, perception, and recognition of emo-
tions and facial expressions will allow us to elucidate the cognitive
neuroscience of human social behaviour.
References
chology. 2nd ed. 2004, Boston: Houghton Miin Co.
xx, 636 p.
7. Damasio, A.R., H. Damasio, and G.W. Van Hoesen,
Prosopagnosia: anatomic basis and behavioral
mechanisms. Neurology, 1982. 32(4): p. 331-41.
8. Breiter, H.C., et al., Response and habituaon of the
human amygdala during visual processing of facial
expression. Neuron, 1996. 17(5): p. 875-87.
9. Morris, J.S., A. Ohman, and R.J. Dolan, Conscious
and unconscious emoonal learning in the human
amygdala. Nature, 1998. 393(6684): p. 467-70.
10. Morris, P.L., et al., Lesion locaon and poststroke
depression. J Neuropsychiatry Clin Neurosci, 1996.
8(4): p. 399-403.
11. Adolphs, R., D. Tranel, and H. Damasio, Emoon
recognion from faces and prosody following
temporal lobectomy. Neuropsychology, 2001. 15(3):
p. 396-404.
12. Adolphs, R., et al., Fear and the human amygdala. J
Neurosci, 1995. 15(9): p. 5879-91.
13. Davidson, R.J., Aecve style, psychopathology,
and resilience: brain mechanisms and plascity. Am
Psychol, 2000. 55(11): p. 1196-214.
14. Durand, V.M., D.H. Barlow, and S.H. Stewart, Essen-
als of abnormal psychology. 1st Canadian ed. 2007,
Toronto: Thomson Nelson. xxiii, 728.
15. Rosenzweig, M.R., S.M. Breedlove, and N.V.
Watson, Biological psychology : an introducon to
behavioral and cognive neuroscience. 4th ed. 2005,
Sunderland, Mass.: Sinauer Associates. 1 v. (various
pagings).
16. Maddock, R.J., The retrosplenial cortex and emoon:
new insights from funconal neuroimaging of the
human brain. Trends Neurosci, 1999. 22(7): p. 310-6.
17. Bush, G., P. Luu, and M.I. Posner, Cognive and
emoonal inuences in anterior cingulate cortex.
Trends Cogn Sci, 2000. 4(6): p. 215-222.
18. Small, D.M., et al., Changes in brain acvity related
to eang chocolate: from pleasure to aversion. Brain,
2001. 124(Pt 9): p. 1720-33.
19. Bechara, A., et al., Insensivity to future conse-
quences following damage to human prefrontal
cortex. Cognion, 1994. 50(1-3): p. 7-15.
20. Rolls, E.T., et al., Emoon-related learning in paents
with social and emoonal changes associated with
frontal lobe damage. J Neurol Neurosurg Psychiatry,
1994. 57(12): p. 1518-24.
21. Thut, G., et al., Acvaon of the human brain by
monetary reward. Neuroreport, 1997. 8(5): p.
1225-8.
22. Gray, J.R., T.S. Braver, and M.E. Raichle, Integraon
of emoon and cognion in the lateral prefrontal
cortex. Proc Natl Acad Sci U S A, 2002. 99(6): p. 4115-
20.
23. Gardner, H., et al., Comprehension and appreciaon
of humorous material following brain damage.
Brain, 1975. 98(3): p. 399-412.
24. Wapner, W., S. Hamby, and H. Gardner, The role of
the right hemisphere in the apprehension of complex
linguisc materials. Brain Lang, 1981. 14(1): p. 15-33.
25. DeKosky, S.T., et al., Recognion and discriminaon
of emoonal faces and pictures. Brain Lang, 1980.
9(2): p. 206-14.
26. Cicone, M., W. Wapner, and H. Gardner, Sensivity
to emoonal expressions and situaons in organic
paents. Cortex, 1980. 16(1): p. 145-58.
27. Etco, N.L., Perceptual and conceptual organizaon
of facial emoons: hemispheric dierences. Brain
Cogn, 1984. 3(4): p. 385-412.
28. Etco, N.L., Selecve aenon to facial identy and
facial emoon. Neuropsychologia, 1984. 22(3): p.
281-95.
29. Kolb, B. and L. Taylor, Aecve behavior in paents
with localized corcal excisions: role of lesion site
and side. Science, 1981. 214(4516): p. 89-91.
30. Posamener, M.T. and H. Abdi, Processing faces and
facial expressions. Neuropsychol Rev, 2003. 13(3): p.
113-43.
31. Ekman, P., An argument for basic emoons. Cogni-
on & Emoon, 1992. 6(3-4): p. 169-200.
32. Izard, C.E., Basic emoons, relaons among emo-
ons, and emoon-cognion relaons. Psychological
Review, 1992. 99(3): p. 561-565.
33. Izard, C.E., Innate and universal facial expressions:
evidence from developmental and cross-cultural
research. Psychological Bullen 1994. 115(2): p. 288-
299.
34. Russell, J.A., A circumplex model of aect. Journal of
Personality and Social Psychology, 1980. 39: p. 1161-
1178.
35. Russell, J.A. and M. Bullock, Fuzzy concepts and the
percepon of emoon in facial expressions. Social
Cognion 1986. 4: p. 309-341.
36. Meeren, H.K., C.C. van Heijnsbergen, and B. de
Gelder, Rapid perceptual integraon of facial expres-
sion and emoonal body language. Proc Natl Acad
Sci U S A, 2005. 102(45): p. 16518-23.
37. Nakamura, M., R. Buck, and D.A. Kenny, Relave
contribuons of expressive behavior and contextual
informaon to the judgment of the emoonal state
of another. J Pers Soc Psychol, 1990. 59(5): p. 1032-9.
38. Carroll, J.M. and J.A. Russell, Do facial expressions
signal specic emoons? Judging emoon from the
face in context. J Pers Soc Psychol, 1996. 70(2): p.
205-18.
39. Susskind, J.M., et al., Human and computer recogni-
on of facial expressions of emoon. Neuropsycholo-
gia, 2007. 45(1): p. 152-62.
40. Dailey, M.N., et al., EMPATH: a neural network that
categorizes facial expressions. J Cogn Neurosci, 2002.
14(8): p. 1158-73.
41. Fox, C.J. and J.J. Barton, What is adapted in face
adaptaon? The neural representaons of expres-
sion in the human visual system. Brain Res, 2007.
1127(1): p. 80-9.
42. Webster, M.A., et al., Adaptaon to natural facial
categories. Nature, 2004. 428(6982): p. 557-61.
43. Grill-Spector, K., R. Henson, and A. Marn, Repe-
on and the brain: neural models of smulus-specic
eects. Trends Cogn Sci, 2006. 10(1): p. 14-23.
44. Kohler, W. and H. Wallach, Figural aereects: an
invesgaon of visual processes Proceedings of the
American Philosophical Society 1944. 88: p. 269-357.
45. Kirsh, S.J. and J.R. Mounts, Violent video game play
impacts facial emoon recognion. Aggress Behav,
2007. 33(4): p. 353-8.
46. Billings, L.S., D.W. Harrison, and J.D. Alden, Age dif-
ferences amogn women in the funconal asymmetry
for bias in facial aect percepon Bullen of the
Psychonomic Society 1993. 31: p. 317-320.
47. Kirsh, S.J., J.R. Mounts, and P.V. Olczak, Violent media
consumpon and the recognion of dynamic facial
expressions. J Interpers Violence, 2006. 21(5): p.
571-84.
48. Leppanen, J. and J. Hietanen, Aect and face percep-
on: odours modulate the recognion advantage of
happy faces. Emoon, 2003. 3: p. 315-326.
49. Leppanen, J., M. Tenhumen, and J. Hietanen, Faster
choice reacon mes to posive than to negave
facial expressions. Journal of Psychophysiology 2003.
17: p. 113-123.
50. Anderson, C.A. and B.J. Bushman, Human aggres-
sion. Annu Rev Psychol, 2002. 53: p. 27-51.
51. Anderson, C.A. and K.E. Dill, Video games and
aggressive thoughts, feelings, and behavior in the
laboratory and in life. J Pers Soc Psychol, 2000. 78(4):
p. 772-90.
52. Bushman, B.J. and R.G. Geen, Role of cognive-
emoonal mediators and individual dierences in
the eects of media violence on aggression. J Pers
Soc Psychol, 1990. 58(1): p. 156-63.
53. Huesmann, L.R., et al., Longitudinal relaons
between children’s exposure to TV violence and their
aggressive and violent behavior in young adulthood:
1977-1992. Dev Psychol, 2003. 39(2): p. 201-21.
54. Kirsh, S.J., Seeing the world through “Mortal Kom-
bat” coloured glasses: violent video games and the
development of a short-term hosle aribuon bias.
Childhood, 1998. 5: p. 177-184.
55. Kirsh, S.J. and P.V. Olczak, Violent comic books and
judgments of relaonal aggression. Violence Vict,
2002. 17(3): p. 373-80.
56. Gaino, G., Emoonal behavior and hemispheric
side of the lesion. Cortex, 1972. 8(1): p. 41-55.
57. Robinson, R.G. and B. Szetela, Mood change follow-
ing le hemispheric brain injury. Ann Neurol, 1981.
9(5): p. 447-53.
58. Yin, R.K., Face recognion by brain-injured paents:
a dissociable ability? Neuropsychologia, 1970. 8(4):
p. 395-402.
59. Diamond, R. and S. Carey, Why faces are and are
not special: an eect of experse. J Exp Psychol Gen,
1986. 115(2): p. 107-17.
60. Gauthier, I., et al., Experse for cars and birds
recruits brain areas involved in face recognion. Nat
Neurosci, 2000. 3(2): p. 191-7.
61. Gauthier, I., et al., Acvaon of the middle fusiform
‘face area’ increases with experse in recognizing
novel objects. Nat Neurosci, 1999. 2(6): p. 568-73.
1. Gazzaniga, M.S., Psychological science. 2nd Canadian
ed. 2007, New York: W. W. Norton & Co. xlvii, 707
2. Adolphs, R., Cognive neuroscience of human social
behaviour. Nat Rev Neurosci, 2003. 4(3): p. 165-78.
3. Aviezer, H., et al., Angry, disgusted, or afraid? Studies
on the malleability of emoon percepon. Psychol
Sci, 2008. 19(7): p. 724-32.
4. Calder, A.J. and A.W. Young, Understanding the
recognion of facial identy and facial expression.
Nat Rev Neurosci, 2005. 6(8): p. 641-51.
5. Ellamil, M., J.M. Susskind, and A.K. Anderson, Ex-
aminaons of identy invariance in facial expression
adaptaon. Cogn Aect Behav Neurosci, 2008. 8(3):
p. 273-81.
6. Banich, M.T., Cognive neuroscience and neuropsy
