Content uploaded by Luiz Pessoa
Author content
All content in this area was uploaded by Luiz Pessoa on Oct 24, 2015
Content may be subject to copyright.
Emotion processing and the amygdala: from a ‘low road’ to
‘many roads’ of evaluating biological significance
Luiz Pessoa and
Department of Psychological and Brain Sciences, Indiana University, Bloomington, Indiana 47405
USA
Ralph Adolphs
Division of Humanities and Social Sciences, and Computation and Neural Systems Program,
California Institute of Technology, Pasadena, California 91125, USA
Abstract
A subcortical pathway through the superior colliculus and pulvinar to the amygdala is commonly
assumed to mediate the non-conscious processing of affective visual stimuli. We review
anatomical and physiological data that argue against the notion that such a pathway plays a
prominent part in processing affective visual stimuli in humans. Instead, we propose that the
primary role of the amygdala in visual processing, like that of the pulvinar, is to coordinate the
function of cortical networks during evaluation of the biological significance of affective visual
stimuli. Under this revised framework, the cortex has a more important role in emotion processing
than is traditionally assumed.
There is tremendous interest in the question of how salient, emotional and socially-charged
visual stimuli are processed by the brain. This topic is important because it addresses a
fundamental question regarding how biological ‘value’ is assigned by an animal to stimuli in
its environment: which stimuli are good and which are bad; which should be approached and
which should be avoided. The topic is also intriguing because it fuels questions about
modularity in the brain (that is, whether there is a specialized way of, or even dedicated
neural substrates for, the processing of affective stimuli). The overarching ‘standard
hypothesis’ runs roughly as follows: ecologically important (emotional and social) stimuli
are processed initially by a dedicated, modular system that operates rapidly, automatically
(without the need to pay attention) and largely independently of conscious awareness1.
Defects in this system are suggested to underlie phobias, mood disorders and post-traumatic
stress syndrome, and variability in its functioning reflects individual differences at the
genotypic and personality level2,3.
© 2010 Macmillan Publishers Limited. All rights reserved
Correspondence to L.P. and R.A. lpessoa@indiana.edu; radolphs@hss.caltech.edu.
Competing interests statement
The authors declare no competing financial interests.
FURTHER INFORMATION
Luiz Pessoa’s homepage: http://emotioncognition.org
Ralph Adolphs’ homepage: http://www.emotion.caltech.edu
SUPPLEMENTARY INFORMATION
See online article: S1 (box)
ALL LINKS ARE ACTIVE IN THE ONLINE PDF
NIH Public Access
Author Manuscript
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
Published in final edited form as:
Nat Rev Neurosci
. 2010 November ; 11(11): 773–783. doi:10.1038/nrn2920.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
The hypothesis has two central and related components. The first component is the
purported role of the amygdala in the rapid, automatic and non-conscious processing of
emotional and social stimuli. The second central component is the proposal of a specific
subcortical route of information processing — the so-called ‘low road’ (REF. 4) — that
bypasses the presumably slower, resource-dependent cortex and that culminates in the
amygdala by way of the superior colliculus and the pulvinar nucleus of the thalamus (FIG.
1a). The fact that this pathway bypasses the cortex is thought to imbue the processing of
emotion-laden visual stimuli with the above list of properties.
Given that the standard hypothesis is shaping both basic and clinical research, this
Perspective article provides a critical re-examination of this hypothesis that we hope will re-
orient thinking about the processing of emotion stimuli and the roles of the amygdala and
pulvinar therein. The main points that we make are as follows: first, there is no evidence for
a functional subcortical route for visual processing in primates; second, the cortex plays a
larger part in the processing of affective visual information than is typically acknowledged;
third, the visual processing of emotion stimuli occurs no faster than visual processing in the
cortex in general; fourth, the amygdala’s contribution to processing of affective visual
information arises from its broad connectivity with the cortex and other subcortical
structures; and finally, the pulvinar plays a part in the processing of emotion stimuli through
its extensive connectivity with cortical sites.
The standard hypothesis
The data and theory that underpin the standard hypothesis are not typically articulated in
detail, and its central concepts are often vague. The main argument is that insofar as
affectively-laden information has survival value, it has driven adaptations in information
processing that are reflected in a functionally and structurally modular system2. The
purported modularity of the system entails automaticity5: owing to the potency of affective
information, this information is processed independently of attention and awareness. For
example, threat-expressing faces have been reported to be processed pre-attentively in
visual search paradigms6, and fearful faces break into consciousness more quickly than
happy expressions during continuous flash suppression (a technique used to render
visual stimuli non-conscious)7. Moreover, haemodynamic responses in the amygdala have
been reported to occur in response to presentation of fearful faces that have (putatively) been
rendered invisible by backward masking8,9 and even during unmasked presentation of
fearful faces in patients with blindsight10,11.
It is also assumed that the anatomical components of the system enable emotion processing
to occur to a substantial extent subcortically12. This suggestion has its roots in rodent
studies that demonstrate the existence of a subcortical pathway, through the auditory
thalamus to the amygdala, that is sufficient for some forms of auditory Pavlovian fear
conditioning4. It is assumed that a similar subcortical route exists for visual information
processing in primates, including humans (see below). The notion of such a subcortical
pathway is appealing because subcortical visual processing is assumed to be faster than
cortical visual processing, and processing of affective stimuli is thought to be adaptive in
part, because it is fast. For example, judgments of threat can be made from facial stimuli that
are presented for as briefly as 39 ms (backward masked)13. Because the pathway is assumed
to be subcortical, processing of visual information along this pathway is assumed to be
coarse. Thus, coarse (that is, low-spatial-frequency) information from affective stimuli is
thought to engage subcortical visual processing, consistent with findings that the amygdala
(a collection of subcortical nuclei) is activated more strongly by emotional faces presented
with low than high spatial frequency14.
Pessoa and Adolphs Page 2
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
In this Perspective we aim to discuss several shortcomings of the standard hypothesis and to
propose an alternative view that we think is better justified by the data, and that we hope
will stimulate new research directions. This Perspective does not focus specifically on the
amygdala, as several comprehensive reviews about the role of this structure in processing
affective information already exist15,16. Instead, we emphasize physiological and anatomical
data concerning the pulvinar, the key ‘link structure’ of the subcortical pathway. Although
most of the data described in this Perspective are well known to parts of the neuroscience
community, the lack of appreciation for these data by ‘affective science’ researchers could
partly explain the widespread acceptance of the standard hypothesis.
We begin by discussing the functional properties of the processing of affective visual
information that are central to the standard hypothesis and focus on the issues of speed and
coarseness. These issues are relevant to the subsequent discussion of the pulvinar and to the
alternative scheme we propose for explaining the properties of visual processing of affective
information. Other important notions that are linked to the standard hypothesis, including
the modularity of the brain and the roles of attention and awareness in visual processing, are
discussed only briefly because they have been reviewed elsewhere17–21. Finally, we propose
an alternative view that assigns a larger role to cortical processing of affective visual
information. We suggest that this scheme, which we call the ‘multiple-waves model’, can
explain the types of findings that have been used to support the standard hypothesis.
General functional issues
Affective visual information is not processed faster than other visual information
Electrophysiological responses evoked by visual stimuli can be modulated by the emotional
content of the stimuli, and this modulation has been reported to occur at short latencies — in
some studies in humans, within ~100 ms of stimulus onset22,23. In addition, the N170
component of the electro encephalography (EEG) signal (or the M170 component in
magnetoencephalography (MEG) studies), which is associated with face identification, is in
some studies modulated by the emotional expression of the presented face24,25. However,
numerous studies only showed effects with longer latencies, ranging from 200–400 ms (for
example, REF. 26). Even in the studies reporting short latency responses to emotional visual
stimuli, localizing the neural sources of those responses using EEG or MEG is problematic
and, therefore, the origin of the signals in these studies might not be in the subcortical
pathway.
In addition, single-unit recordings in monkeys indicate that responses in the cortex (even in
the frontal cortex) occur with latencies that are within the range of the latencies observed in
subcortical areas (BOX 1). This is consistent with the idea of a rapid, feedforward ‘cortical
sweep’ of visual information processing. Moreover, behavioural and electrophysiological
studies of object processing and perception in humans suggest that visual processing in
general (that is, including non-affective processing) can be remarkably fast and that a
substantial amount of information can be gathered from even a single glance at a natural
scene (see Supplementary information S1 (box)). For example, a recent study showed that
only 19–67 ms were required to attain 75% correct performance on several tasks, including
determining a scene’s global property (for example, ‘natural scene’) and basic level
categorization (for example, ‘forest’)27. Thus, there is nothing particularly special about the
processing speed of affective information. Finally, the speed of cortical — as opposed to
subcortical — visual processing could also account for the reported rapid modulation of
evoked brain responses by stimulus valence. Valence, like other affectively relevant
stimulus dimensions, is probably computed in several brain regions, one of which is likely to
be the orbitofrontal cortex (OFC). Short-latency (100–150 ms) electrophysiological
responses in the OFC have been associated with discrimination of the valence of a visual
Pessoa and Adolphs Page 3
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
stimulus28. We develop this final point in greater depth below, in the context of our new
proposal.
Taken together, the findings above indicate that processing of affective visual stimuli is no
faster than cortical processing of visual stimuli in general. Indeed, cortical visual processing
is both efficient and fast. Thus, the argument that a separate subcortical system is required
for fast perception of affective stimuli is problematic (Supplementary information S1 (box)).
Box 1 | Speed of visual processing
One way to assess the speed of visual processing is to measure and contrast response
latencies across brain areas. For example, do responses in the purported subcortical
pathway occur earlier than those in cortical sites? The figure shows that in the macaque
cerebral cortex, the earliest latencies are remarkably short, and even mean response
latencies indicate remarkably fast cortical processing72. Areas that became active at the
given latency after visual stimulation are shown in red, those that were activated earlier
in yellow and those that were not yet activated in white. Areas for which no information
was available are shown in dark grey (see the figure). Visual response latencies in the
pulvinar are between 60–80 ms and overlap with latencies observed in early visual
cortical areas V1 and V2 (REF. 104). In the inferotemporal cortex (that is, ‘late’ visual
cortex) latencies can be as short as 60–85 ms72 and, strikingly, in some frontal sites such
as the frontal eye fields (FEF) as short as 40–70 ms. These latencies again overlap with
those in area V1 (REFS 84,105). Thus, although mean response latencies increase
gradually from posterior to anterior visual cortices, there is considerable overlap (see the
figure). In the context of the standard hypothesis, it therefore seems that pulvinar
responses are not particularly fast. However, it is of interest that visual response latencies
in the superior colliculus are somewhat faster than those observed in the pulvinar,
showing an early, transient response around 40–70 ms that may support rapid eye
movements during orienting106 (note that these response times overlap with FEF
responses).
What are the response latencies of neurons in the amygdala? In the monkey amygdala,
responses to visual stimuli range from 100–200 ms30,107–109, although shorter response
latencies to unspecific stimuli (for example, fixation spots) have been reported30.
Differences in evoked responses between threatening and neutral or appeasing facial
expressions in the monkey amygdala have been found in the range of 120–250 ms30.
Intracranial studies in humans generally find the earliest single-unit responses to visual
stimuli around 200 ms33,110. Moreover, in one study, modulation of amygdala
responses by the affective content of stimuli was observed to start at 200 ms26 (see also
REF. 110).
In summary, subcortical visual processing is not discernably faster than cortical
processing. Furthermore, the crucial variable is not the timing of the initial stimulus
responses but the time at which reliable differences between affective and non-affective
stimuli can be detected. It has been suggested111 that most of the information encoded
by visual neurons may be available in 100-ms-long segments of activity (that is, spiking
data within a 100 ms epoch) and that a fair amount of information is available in
segments of 50 ms, and even some of 20–30 ms (note that these segments consider post-
Pessoa and Adolphs Page 4
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
latency neuronal spikes only). Although these data demonstrate the remarkable speed of
neuronal computation (at least under some conditions), they add milliseconds to the time
that is required to, for example, discriminate between stimuli. A final consideration is
that responses in humans are possibly slower than in monkeys. For example, in one study
in humans, the fastest recording sites had response latencies of just under 60 ms and were
probably located in V1 (or possibly V2)112. In the monkey, the fastest responses in V1
can be observed under 40 ms72. 5, Brodmann area 5; 7a, Brodmann area 7; 7ip,
Brodmann area 7ip (intraparietal); 8a, Brodmann area 8a; EC, entorhinal cortex; FEF,
frontal eye field; FST, fundus of superior temporal cortex; IPa, superior temporal area
IPa; M1, primary motor cortex; MST, medial superior temporal cortex; MT, medial
temporal area (also known as V5); OFC, orbitofrontal cortex; PFC, prefrontal cortex;
PGa, superior temporal area PGa; PreM, premotor cortex; SEF, supplementary eye field;
SMA, supplementary motor area; TAa, anterior subregion of superior temporal area TA;
TE1, inferior temporal area TE1; TE2, inferior temporal area TE2; TE3, inferior temporal
area TE3; TEm/TEa, medial and anterior subregions of inferior temporal area TE; TPO,
superior temporal area TPO; TS, superior temporal sulcus. Figure is reproduced, with
permission, from REF. 72 © (2000) Cell Press.
Processing of affective visual stimuli involves both coarse- and high-spatial-frequency
information
According to the standard hypothesis, the subcortical pathway is particularly effective at
carrying low-spatial-frequency information, mainly because the superior colliculus and
pulvinar are assumed not to convey much high-spatial-frequency information. This notion
was initially based on findings in rodents that simple (‘coarse’) auditory conditioning does
not require the cortex, whereas conditioning that requires more complex stimulus
discriminations does4.
Neuroimaging studies in humans seem to be consistent with this idea. For example, in one
study amygdala responses were stronger when participants viewed low- compared with
high-spatial frequency fearful faces14, and when they viewed fearful faces versus neutral
faces both at low spatial frequency14. Similarly, activation in brain areas consistent with the
location of the superior colliculus and pulvinar was greater in response to fearful faces than
to neutral faces at low spatial frequencies14. Findings of this kind have been interpreted as
suggesting that the amygdala is relatively ‘blind’ to high-spatial-frequency information.
However, the amygdala receives major projections from the anterior inferotemporal
cortex29 that convey highly processed object information — in fact, the amygdala receives
highly processed cortical input from all sensory modalities except olfaction29. Indeed,
electrophysiological studies have shown that the monkey amygdala contains neurons that are
tuned to the identity of specific faces30,31 and that the human amygdala shows category-
specific responses (for example, for animals or natural scenes)32,33. These are properties
that require high-spatial-frequency information.
Moreover, it has been shown that the discrimination of facial expressions relies on both low-
and high-spatial-frequency information34. The perception of fear is particularly reliant on
high-spatial-frequency information35. Indeed, a study in a patient with bilateral amygdala
lesions showed that this patient’s impaired recognition of facial expressions of fear was due
to impaired processing of the eye region of faces, especially of high-spatial-frequency
information about the eyes36. These results demonstrate the importance of high-spatial-
frequency information in fear recognition and indicate that the amygdala is required for this
type of visual processing.
Pessoa and Adolphs Page 5
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
In summary, although some findings are consistent with the notion that subcortical areas
process coarse visual information, the perception of emotional expressions actually involves
both coarse and fine information. Furthermore, the amygdala not only receives inputs that
convey fine spatial information but seems to be crucially involved in using this information
to decode facial expressions (Supplementary information S1 (box)).
The amygdala is not essential for rapid, non-conscious detection of affective information
The standard hypothesis often draws on additional themes, such as the roles of attention and
awareness in visual processing, and the more general issue of the extent to which emotion
and cognition are processed by separate circuits in the brain. Both of these issues are
complex and have been discussed at greater length elsewhere19,20, but we briefly comment
on attention and awareness.
Processing affective information is known to occur under some conditions of inattention and
unawareness. However, as discussed elsewhere, the interpretation of the published literature
in terms of ‘strong automaticity’ is unwarranted20. Briefly, behavioural, functional MRI
(fMRI) and EEG studies indicate that when processing resources are sufficiently consumed
(for example, by engaging attention on challenging tasks), visual processing of emotional
stimuli is greatly reduced or eliminated. This challenges the notion that emotional stimuli
are processed automatically (see REFS 37–39 for further discussion of the role of attention).
A recent study of a patient with complete amygdala lesions40 perhaps provides the most
decisive data on this issue. In this patient, reaction times for detecting fearful faces among
distractor stimuli were within the normal range, and fearful facial expressions broke into
consciousness faster than happy faces during binocular suppression to the same degree as in
control subjects (FIG. 2). These findings demonstrate that the amygdala is not essential for
non-conscious, rapid fear detection, at least in the tasks used in this study40 (see also REF.
41).
Thus, independent lines of evidence challenge the notion that processing of affective visual
information occurs independently of attention and awareness. Moreover, some of the
properties typically connected in the literature with automaticity (for example, detecting
fearful faces among distractors) may not entirely depend on the amygdala.
Physiological and anatomical issues
The pulvinar is a key link element in the purported colliculus–pulvinar–amygdala pathway.
Here, we briefly review physiological and anatomical data regarding the pulvinar — and the
subcortical pathway of which it is thought to be a component — that are relevant in the
context of the standard hypothesis. In particular, we discuss data that are relevant to the
question of whether this structure is better conceptualized as a relatively passive way station
or as a dynamic element of brain circuitry.
Pulvinar input
The pulvinar complex is the largest nuclear mass in the primate thalamus and is thought to
have expanded in size during evolution in parallel with other visual structures42. The
pulvinar does not seem to exist in brains of rodents and other small mammals43. In terms of
connectivity that is relevant for visual processing, it receives direct visual input from the
retina, indirect visual input via the superficial layers of the superior colliculus and massive
input from striate and extrastriate visual cortices (FIG. 3). All of these projections terminate
in the inferior pulvinar. Intriguingly, however, the visual response properties of pulvinar
cells do not reflect those of neurons in the superior colliculus, and the precise contribution
that input from the superior colliculus makes to pulvinar responses remains uncertain43. For
example, superior colliculus lesions have little effect on electrophysiological responses of
Pessoa and Adolphs Page 6
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
pulvinar neurons, in contrast to striate cortex lesions, which abolish responses in the inferior
pulvinar44. In a related fashion, collicular and pulvinar lesions result in different
behavioural impairments45 (see also REF. 46). These observations argue that the pulvinar
may be better thought of as participating in cortical networks, rather than as relaying visual
information from the superior colliculus. This is borne out by the finding that, unlike the
lateral geniculate nucleus (LGN), the pulvinar’s driving inputs in fact originate in the cortex,
whereas subcortical inputs to the pulvinar are typically modulatory47,48; for this reason, the
pulvinar is described as a higher-order thalamic nucleus, as opposed to a first-order nucleus
such as the LGN49.
Pulvinar activity
Studies in monkeys and humans with pulvinar lesions have suggested that this structure is
involved in determining what is salient in a visual scene50,51. Consistent with this notion,
the response of pulvinar neurons to visual stimuli is increased if attention is paid to the
stimulus or if the stimulus has behavioural relevance. For example, pulvinar neurons
respond more vigorously to behaviourally relevant targets than to unattended stimuli45. In
one study, as many as 92% of pulvinar cells exhibited attenuated responses to stimuli that
were task-irrelevant (that is, passively viewed)52 compared to stimuli that were task-
relevant. Furthermore, the impact of attention on evoked responses in the pulvinar is
spatially specific, such that a pulvinar neuron only increases activity when a monkey attends
to a stimulus that falls within the receptive field of the cell53. Finally, the pulvinar seems to
be crucial (as was shown in a study in which the pulvinar was pharmacologically
inactivated) when a distractor stimulus has to be ‘filtered out’54. Thus, it has been proposed
that the pulvinar is involved in attention and/or distractor filtering, and this is consistent with
data from neuroimaging and lesion studies in humans (Supplementary information S1
(box)).
The pulvinar is also important for visual awareness. For example, lesion studies in humans
have revealed that pulvinar damage is associated with visual neglect and with feature-
binding deficits51,55,56. A recent study in monkeys is particularly noteworthy: here, neural
activity was recorded in the pulvinar during a visual illusion that induced the intermittent
perceptual suppression of a bright luminance patch57. Neurons in the pulvinar showed
changes in spiking rate according to trial-by-trial stimulus visibility, suggesting that they
reflected the visual awareness of the stimulus. Similarly, a recent fMRI study in humans
found that the pulvinar responded not to the affective significance of visual stimuli but to
whether or not they were consciously perceived58, again in a trial-by-trial manner. The
fMRI results are consistent with a study in which pulvinar responses were associated with a
subject’s percept of a change59. Notably, responses were observed during ‘false alarm’ trials
(those in which a stimulus change was reported but did not actually occur) but not during
‘miss’ trials (those in which a stimulus change occurred but went unnoticed by the
participant). These results do not support the suggestion that the pulvinar is involved during
non-conscious processing, and are inconsistent with a major role for this structure in the
subcortical pathway proposed by the ‘standard hypothesis’, according to which the pulvinar
behaves as a relatively passive relay.
Pulvinar anatomy
It is also important to consider the anatomical features of the pulvinar that highlight its
extensive bidirectional connectivity with the cortex. For example, all 20–30 known visual
areas connect with the pulvinar, sometimes in a relatively topographic fashion43,60.
Temporal, parietal, cingulate, frontal and insular cortices are all connected with the pulvinar
as well. At a gross level, it is as if the entire convoluted cortex were ‘shrink-wrapped’
around the pulvinar60. Based on the connectivity data, one can discern a ventrolateral to
Pessoa and Adolphs Page 7
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
dorsomedial axis, and this has led to the suggestion that the pulvinar may contain two
connectional ‘domains’ (REFS 42,60) (FIG. 3). The ventral domain (containing the inferior
pulvinar) is densely connected with the visual cortex (including areas V1–V4 and MT)60. It
therefore has a strong visual component (that is, it could be called the ‘visual pulvinar’), and
its projections to the dorsal visual stream may mediate some of the visual abilities in people
with blindsight61,62 (Supplementary information S1 (box)). The dorsal domain (including
the traditional medial subdivision; see below) has connections with the cross-modal
association cortex, including temporal and parietal areas (including area 7A and the lateral
intraparietal (LIP) cortex — areas that are also involved in attention)60. The dorsal domain
receives highly processed visual input from the inferior temporal gyrus area TE (and some
from area TEO) in the inferotemporal cortex60. It is also connected with the cingulate
cortex, frontal cortex (including the OFC), insula and amygdala60 (see below). The dorsal
domain is therefore much more ‘associational’ and, in fact, has remarkable potential to
integrate information from very diverse brain regions (FIG. 3).
In summary, the pulvinar is a complex structure with important visual and integrative
properties. Functional studies have characterized several ways in which the pulvinar is
modulated by attention and awareness. Indeed, the pulvinar is likely to be an important
‘control site’ for attentional mechanisms more broadly63. Taken together, these data are
antithetical to the standard hypothesis, which assumes that automatic processing is mediated
by a subcortical pathway involving the pulvinar.
Does the subcortical pathway exist in primates?
Work on fear conditioning has shown that there are direct, subcortical projections to the
amygdala from the auditory thalamus (that is, from the medial geniculate nucleus (MGN)) in
rats. An analogous projection carrying visual information from the visual thalamic nucleus
(that is, the LGN) to the amygdala has not been documented in rodents or primates.
However, it is frequently assumed that a colliculus—pulvinar—amygdala pathway exists in
the case of vision. Here, we review relevant data from primates that question this
assumption.
Anatomical studies in monkeys have reported connections between the superior colliculus
and the pulvinar42,43, and between the pulvinar and the amygdala64,65. As described
above, the superficial superior colliculus projects to the inferior pulvinar — both of these
structures can be considered to be ‘visual’. Yet, the inferior pulvinar is extensively
interconnected with the visual cortex (consistent with visual functions) but not with the
amygdala. Instead, the projection to the amygdala originates in the medial pulvinar64,65
(although the strength of this connection may be relatively weak66), a nucleus that is
extensively interconnected with much of the cortex, as described above. Furthermore, like
other thalamic nuclei, the primate pulvinar is thought to have neither excitatory nor
inhibitory long-range intrinsic connections67. Considered together, these results do not
support the idea of a colliculus–pulvinar–amygdala visual pathway.
To summarize, there is no evidence for a direct or an indirect subcortical pathway conveying
visual information to the amygdala in monkeys. It is therefore unclear how findings from
auditory fear conditioning studies in rodents can be applied to visual processing of affective
stimuli in primates (see also BOX 2). For further discussion of the pathway from
intermediate and deep layers of the superior colliculus (which are multimodal in nature and
linked to eye movements) to the pulvinar see Supplementary information S1 (box).
Pessoa and Adolphs Page 8
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
The multiple-waves model
The standard hypothesis has influenced both basic and applied research and at first glance
has intuitive appeal. However, we have shown that, in its current form, the hypothesis is
problematic in multiple respects. We therefore suggest that a revision of the standard
hypothesis is in order.
Multiple visual pathways and coarse information processing
One of the primary motivations for the standard hypothesis is the perceived need for rapid
processing: fast but coarse visual processing is just what an organism needs in a dangerous
environment. We argue that visual pathways other than a colliculus–pulvinar–amygdala
pathway carry out this role (see also REFS 68,69).
Visual processing along the ventral processing stream, which is crucial for object
recognition, has historically been described to occur in a relatively hierarchical fashion.
However, important ‘short-cut’ connections link areas V1 to V4 (REF. 70), V2 to TEO70
and V4 to TE71 (FIG. 1b), providing the means for faster information transmission to the
inferotemporal cortex72. Direct connections between the LGN and extrastriate regions,
including V2 (REFS 73,74) and V4 (REF. 73), have also been reported. Indeed, combined
electrophysiology and fMRI studies in monkeys have shown robust visual activation in areas
V2 and V3 in animals with lesions of V1, demonstrating that routes bypassing V1 can be
sufficiently potent to drive extrastriate visual responses75 (see REF. 76 for evidence in
humans). A combined lesion and fMRI study in monkeys77 revealed widespread extrastriate
activation in the absence of V1 and demonstrated a role of LGN-dependent projections for
visual detection. V1-independent responses were observed in areas V2, V3, V4, MT (also
known as V5), the fundus of the superior temporal cortex (FST) and the LIP area. These
findings establish the importance of the LGN for at least some types of blindsight. Longer
range ‘short-cuts’ also exist, such as those that link regions in the ventral visual cortex with
the ventrolateral prefrontal cortex (including the OFC)78. It has been proposed79 that low-
spatial-frequency information may rapidly reach parietal and frontal cortices from the early
visual cortex80, thereby providing coarse information about the gist of a visual scene and
supporting object recognition81. It is thus possible that these initial ‘volleys of activation’
are less susceptible to manipulations of attention and awareness, especially given that they
may primarily convey information from the magnocellular system80,82. Furthermore,
computational models that assume a purely hierarchical structure of the visual system have
failed to provide a good fit to the existing latency data83, which is consistent with the
existence of bypass connections.
Thus, there are multiple parallel routes for visual information processing that lead to
substantial temporal dispersion of evoked responses and that enable ‘high-level’ regions to
respond with surprisingly short latencies84. Each processing stage adds approximately 10 ms
to the latency84. The ‘cost’ of using such bypassing stages may be that, at first, only
relatively coarse information is available about a visual item. This is consistent with a
coarse-to-fine processing strategy in which the more-global contents of a stimulus are
processed earlier than finer details85,86.
Based on the considerations above, we suggest that the initial processing of visual
information proceeds simultaneously along parallel channels, creating ‘multiple waves’ of
activation across the visual cortex and beyond87. In this manner, visual stimuli that have
affective and motivational significance can engage multiple brain sites — including the
amygdala, OFC, anterior insula and anterior cingulate cortex — that can direct processing
towards these behaviourally relevant items. Hence, rapid processing of affective information
Pessoa and Adolphs Page 9
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
is possible even in the absence of a specialized subcortical pathway (FIG. 1b) or a single
specific structure such as the amygdala40.
In light of our proposal, we suggest that affective blindsight involves some of the alternate
pathways described here. A recent EEG study of a patient with complete cortical blindness
used advanced source modelling to investigate the time course of information
processing88. Although all facial expressions, including neutral ones, evoked relatively
short-latency responses (70–120 ms) localized to the superior temporal sulcus, emotion-
specific responses that were localized to the anterior temporal cortex, and possibly the
amygdala, occurred considerably later (120 and 200 ms later, respectively). Although this
study suffers from the localization problems alluded to above, the findings are consistent
with the notion that affective significance is computed in parallel along several circuits (see
also REF.89 for a related proposal).
In light of the change of focus from a single, specialized subcortical pathway to a multiple-
pathway model, it is important to reconsider the roles of both the pulvinar and the amygdala
during processing of emotional visual stimuli.
Box 2 | Subcortical processing: audition in rats versus vision in primates
Historically, the standard hypothesis has derived a considerable portion of its motivation
from the organization of the auditory system in rodents. However, the auditory and visual
systems differ in important ways. The temporal precision of the auditory system is
substantially greater than that of the visual system. In contrast to vision, audition is
omnidirectional, such that information from all directions can be sampled (though at
relatively low spatial resolution). Furthermore, the functional anatomy of the auditory
system is very different from that of the visual system. Properties such as sensitivity to
sound frequency, duration, amplitude, pitch and binaural disparity are already observed at
subcortical levels. In fact, the primary auditory cortex (A1) seems to be involved in high-
level functions and is therefore not equivalent to ‘visual cortex transplanted into the
auditory modality’. Indeed, there are several subcortical stages below the level of the
auditory cortex, and it has been suggested113 that the role of the inferior colliculus in
auditory processing might be equivalent to that of primary visual cortex (V1) in vision
and that the A1 is more analogous to visual areas in the inferotemporal cortex than to V1.
These considerations suggest that a subcortical pathway for auditory input to the
amygdala (in rodents) would not actually be analogous to the purported subcortical visual
pathway (in monkeys). Accordingly, auditory connections from the medial geniculate
nucleus in the thalamus to the amygdala, although bypassing cortex, already convey
highly processed information, in contrast to the suggested monkey (visual) counterpart.
The role of the pulvinar in processing of emotional visual stimuli
Connections between the pulvinar and amygdala have been reported64,65, suggesting that the
pulvinar may have a role in emotion processing. As mentioned above, it has been proposed
that the pulvinar is involved in determining the behavioural relevance of a stimulus,
directing attention to a stimulus and determining awareness of a stimulus. Based on the data
described above, we propose that the pulvinar helps to coordinate and/or regulate the flow of
multimodal information via a series of thalamocortical loops (FIG. 3). This proposal takes
into account that most of the input to the pulvinar comes from the cortex.
In the context of emotion processing, the most relevant nucleus of the pulvinar is probably
the medial nucleus, given that it connects not only with the amygdala but also with a larger
array of other brain regions. We therefore suggest that it may be involved in more general
Pessoa and Adolphs Page 10
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
functions that impact emotion processing, such as determining the behavioural relevance
and/or value of a stimulus. For example, the medial nucleus is connected with parietal
regions that are involved in attention. It is also connected with the OFC and cingulate cortex,
which are important for computing an object’s biological value. Furthermore, it is connected
with the insula, a region that has a role in emotional feelings. These connections are all
bidirectional (except the connection with the amygdala), providing opportunities for
modulating and regulating information flow. According to our proposal, the importance of
the pulvinar in emotion is not due to its status as a subcortical ‘labelled line’ conveying
emotional information to the amygdala, but due instead to its pattern of connectivity with
subcortical and cortical sites that have a role in determining the biological significance of a
stimulus.
Studies by Ward and colleagues have investigated the impact of pulvinar lesions on
processing of affective visual information in humans. A complete unilateral loss of the
pulvinar led to a severe deficit in a patient’s ability to recognize fearful expressions shown
in the contralesional visual field90. Within the framework suggested here, when weak and/
or brief visual stimuli have biological significance, cortico–pulvino–cortical circuits
coordinate and amplify signals in a manner that enhances their behavioural impact. This
framework is consistent with the impairment in recognizing fear in patients with pulvinar
lesions and also with their impairment in recognizing anger (and possibly happiness)90.
Notably, the essential pulvinar damage was found in the medial pulvinar, the region that
projects to the amygdala in monkeys. The proposed framework is also consistent with a
study that reported that viewing complex unpleasant images impaired performance in a
subsequent simple (neutral) visual task in control subjects, but not in a patient with pulvinar
damage91 — according to our framework, the unpleasant stimulus did not garner additional
resources in the patient (which would have interfered with performance, as it did in the
controls).
Pulvinar involvement in the processing of affective information does not seem to reflect
emotion per se, however. In an fMRI study in humans58, a simple contrast between affective
and neutral conditions did not reveal different responses in the pulvinar. Instead, there was a
significant relationship between the magnitude of evoked responses in the pulvinar and the
probability of correctly detecting a target on a trial-by-trial basis during the affective
condition but not during the neutral condition (FIG. 4a). These results reveal an emotion-by-
visibility interaction that may characterize the role of the pulvinar more generally. In other
words, we suggest that the pulvinar amplifies responses to stimuli of potential value to the
animal (such as one that signals the possibility of shock in the experiment) (FIG. 4b).
The role of the amygdala in processing affective visual stimuli
What part is left for the amygdala to play in the processing of affective visual stimuli? Its
connectivity pattern provides some clues. The predominant source of visual input to the
amygdala, specifically the basolateral nucleus, comes from higher-order visual association
cortices in the anterior temporal lobe29. This suggests that the amygdala is a convergence
zone for highly-processed sensory information that is relevant to object processing. In
addition, there are loops between the visual cortex and the lateral and basal nuclei of the
amygdala, and this feedback is thought to modulate visual processing92. Further integrative
functions of the amygdala stem from its extensive connections with much of the cortex. In
addition to its well-recognized connections with medial and orbital territories of the
prefrontal cortex, the amygdala is also connected to the lateral prefrontal cortex, albeit in a
weaker manner93. Importantly, the architecture of the prefrontal cortex (PFC) is such that,
on average, inputs from the amygdala reach approximately 90% of the PFC after a single
connection within the frontal cortex94. Furthermore, the amygdala seems to be part of a
Pessoa and Adolphs Page 11
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
‘core brain circuit’ (REF. 95) that is topologically central in terms of global brain
connectivity and whose functions could include aggregation and distribution of information.
In light of these considerations, we propose that the amygdala’s contribution to the
processing of affective visual information arises not from a subcortical source of visual
input, but from its broad connectivity with the cortex and other subcortical structures. Given
this connectivity, the impact of the amygdala on behaviour can be mediated through many
routes, for example, via both the visual cortex and prefrontal cortex. This suggestion is
consistent with findings of a study that combined the attentional blink task with fear
conditioning96. For emotion-laden stimuli, trial-by-trial fluctuations in evoked responses in
the amygdala predicted whether or not a target was detected. Furthermore, this impact of the
amygdala on behaviour was mediated by both the visual cortex and prefrontal cortex (as
suggested by statistical path analysis), consistent with the idea that during the
processing of affectively significant items, the amygdala enhances sensory processing
through both direct (amygdala–visual cortex) and indirect (amygdala–prefrontal cortex–
visual cortex) paths (FIG. 4c).
For reasons of brevity we have discussed the amygdala as a single entity, but it should be
noted that the amygdala is in fact a complex structure comprised of more than a dozen
nuclei. In particular, the central nucleus has extensive descending connections to the
hypothalamus and other brainstem nuclei that regulate autonomic and endocrine responses
and, in this manner, contributes to several aspects of emotional expression and mobilization
of bodily resources. Among others, this is an important distinction between the roles of the
amygdala and pulvinar during processing of affective visual stimuli.
Conclusions
The evidence we have reviewed here suggests that the idea of a subcortical pathway that is
specialized for the processing of emotional stimuli should be revised. Our reinterpretation
has important implications for the conceptualization of the amygdala’s function in the
processing of emotional visual information. We suggest two revised roles for the amygdala.
First, the amygdala has a mostly modulatory role in a wide array of networks. The precise
functional importance of the amygdala in these networks remains to be investigated, but it is
unlikely that it will map specifically onto emotion. Instead, we think that it corresponds to
broader and more abstract dimensions of information processing, including processing of
salience, significance, ambiguity, unpredictability21,97–99 and other aspects of ‘biological
value’. More broadly, we argue that the amygdala has a key role in solving the following
problem100: how can a limited-capacity information processing system that receives a
constant stream of diverse inputs selectively process those inputs that are the most relevant
to the goals of the animal101?
Thus, the amygdala serves to allocate processing resources to stimuli, at least in part by
modulating (through its connectivity) the anatomical components that are required to
prioritize particular features of information processing in a given situation. Such a role
would come into play not only for affectively significant stimuli but also for other stimuli.
Notably, the amygdala may not be unique in this respect as there are other, largely parallel,
networks with architectures that do not include the amygdala but that also enable diverse
functions — notably the network subserved by the connections between the cortex and the
pulvinar. We think that our proposal is consistent with the majority of findings and can
accommodate several views of amygdala function16,102,103, with the difference that it
provides a broader and more flexible perspective.
Pessoa and Adolphs Page 12
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
The second aspect of our proposed revision is to stress the speed and temporal dispersion of
cortical processing, which render moot the assumed need for a fast subcortical route. Many
visual properties can be processed very rapidly by the initial wave of cortical response,
which suggests that there is ample time for substantial feedback, even within the cortex.
Consequently, understanding the flow of visual information within the cortex should help us
to understand how affective stimuli are processed. Ultimately, the fate of a biologically-
relevant stimulus should not be understood in terms of a ‘low road’ versus a ‘high road’, but
in terms of the ‘multiple roads’ that lead to the expression of observed behaviours.
There is an enormous literature implicating the amygdala in affective dysfunction in nearly
every psychiatric illness, most notably in mood disorders. The revised view in this
Perspective suggests that rather than focusing on neurons within the amygdala, we should
focus on connections within the cortex and between the cortex and subcortical structures
such as the amygdala. This may not come as a big surprise to some readers as, in the main, it
simply reflects the idea that the substrate of brain function is not so much to be found within
neurons as within networks.
Supplementary Material
Refer to Web version on PubMed Central for supplementary material.
Acknowledgments
The authors thank A. Anticevic, L. Oliveira, M. Pereira, R. Todd, and S. Wang for feedback on the manuscript.
They also thank L. Barrett and two anonymous reviewers for comments. The authors’ research is supported by
grants from the National Institute of Mental Health (R01 MH071589 to L.P. and R01 MH080721 to R.A.), the
National Institute of Neurological Disorders and Stroke (P01 NS019632 to R.A.), the Simons Foundation Autism
Research Initiative and the National Science Foundation (NSF 0926,544).
Glossary
Attentional
blink A phenomenon that occurs in experiments in which a rapid stream of
visual items is presented to an observer whose task is to detect two
targets within the stream. When the two targets are separated in time
by a brief interval (for example, 200–500 ms), the successful
detection of the first target impairs detection of the second one (as if
the participant blinked) owing to limited processing capacity.
Backward
masking A phenomenon that occurs in experimental paradigms in which a
target visual stimulus is followed by another salient visual stimulus
that ‘masks’ the perception of the target stimulus, making its
detection or recognition difficult or impossible. Visual masking is
commonly used to manipulate visual awareness.
Blindsight The ability, in humans or monkeys, to respond to visual stimuli
without consciously perceiving them — a situation that may ensue
following a lesion to the primary visual cortex.
Continuous flash
suppression A technique in which a fixed image shown to one eye is suppressed
by a stream of rapidly changing images flashed to the other eye. The
technique is used to manipulate visual awareness.
Labelled line A processing architecture in which a separate pathway conveys
information that is specific to a class of sensory stimuli owing to, for
Pessoa and Adolphs Page 13
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
example, receptor specificity (for example, pain and touch conveyed
by particular somatosensory channels).
Magnocellular
system A visual pathway from the retina to the cortex that conveys relatively
fast, transient and wavelength-insensitive information.
Path analysis A statistical method to investigate the relationship between multiple
variables.
Source
modelling A set of techniques that attempt to estimate the neural ‘sources’ of the
electrical or magnetic signals that are measured at external sensors
(for example, at the scalp in the case of electroencephalography).
Visual search An experimental paradigm in which subjects are asked to indicate the
presence or absence of a ‘target’ item (for example, a fearful face)
among an array of ‘distractor’ items (for example, neutral faces).
References
1. Tamietto M, de Gelder B. Neural bases of the non-conscious perception of emotional signals.
Nature Rev. Neurosci 2010;11:697–709. [PubMed: 20811475]
2. Ohman A, Mineka S. Fears, phobias, and preparedness: toward an evolved module of fear and fear
learning. Psychol. Rev 2001;108:483–522. [PubMed: 11488376]
3. Lonsdorf TB, et al. Genetic gating of human fear learning and extinction: possible implications for
gene-environment interaction in anxiety disorder. Psychol. Sci 2009;20:198–206. [PubMed:
19175757]
4. LeDoux, JE. The Emotional Brain. New York: Simon & Schuster; 1996.
5. Dolan RJ, Vuilleumier P. Amygdala automaticity in emotional processing. Ann. NY Acad. Sci
2003;985:348–355. [PubMed: 12724170]
6. Ohman A, Lundqvist D, Esteves F. The face in the crowd revisited: a threat advantage with
schematic stimuli. J. Pers. Soc. Psychol 2001;80:381–396. [PubMed: 11300573]
7. Yang E, Zald DH, Blake R. Fearful expressions gain preferential access to awareness during
continuous flash suppression. Emotion 2007;7:882–886. [PubMed: 18039058]
8. Morris JS, Ohman A, Dolan RJ. Conscious and unconscious emotional learning in the human
amygdala. Nature 1998;393:467–470. [PubMed: 9624001]
9. Whalen PJ, et al. Masked presentations of emotional facial expressions modulate amygdala activity
without explicit knowledge. J. Neurosci 1998;18:411–418. [PubMed: 9412517]
10. Morris JS, DeGelder B, Weiskrantz L, Dolan RJ. Differential extrageniculostriate and amygdala
responses to presentation of emotional faces in a cortically blind field. Brain 2001;124:1241–1252.
[PubMed: 11353739]
11. Pegna AJ, Khateb A, Lazeyras F, Seghier ML. Discriminating emotional faces without primary
visual cortices involves the right amygdala. Nature Neurosci 2005;8:24–25. [PubMed: 15592466]
12. Morris JS, Ohman A, Dolan RJ. A subcortical pathway to the right amygdala mediating “unseen”
fear. Proc. Natl Acad. Sci. USA 1999;96:1680–1685. [PubMed: 9990084]
13. Bar M, Neta M. Humans prefer curved visual objects. Psychol. Sci 2006;17:645–648. [PubMed:
16913943]
14. Vuilleumier P, Armony JL, Driver J, Dolan RJ. Distinct spatial frequency sensitivities for
processing faces and emotional expressions. Nature Neurosci 2003;6:624–631. [PubMed:
12740580]
15. Davis M, Whalen PJ. The amygdala: vigilance and emotion. Mol. Psychiatry 2001;6:13–34.
[PubMed: 11244481]
16. Whalen, PJ.; Phelps, EA., editors. The Human Amygdala. New York: Guilford Press; 2009.
17. Lewis MD. Bridging emotion theory and neurobiology through dynamic systems modeling. Behav.
Brain Sci 2005;28:169–245. [PubMed: 16201458]
Pessoa and Adolphs Page 14
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
18. Duncan S, Barrett LF. Affect is a form of cognition: a neurobiological analysis. Cogn. Emot
2007;21:1184–1211. [PubMed: 18509504]
19. Pessoa L. On the relationship between emotion and cognition. Nature Rev. Neurosci 2008;9:148–
158. [PubMed: 18209732]
20. Pessoa L. To what extent are emotional visual stimuli processed without attention and awareness?
Curr. Opin. Neurobiol 2005;15:188–196. [PubMed: 15831401]
21. Adolphs R. Fear, faces, and the human amygdala. Curr. Opin. Neurobiol 2008;18:166–172.
[PubMed: 18655833]
22. Pizzagalli D, Regard M, Lehmann D. Rapid emotional face processing in the human right and left
brain hemispheres: an ERP study. Neuroreport 1999;10:2691–2698. [PubMed: 10511425]
23. Halgren E, Raij T, Marinkovic K, Jousmaki V, Hari R. Cognitive response profile of the human
fusiform face area as determined by MEG. Cereb. Cortex 2000;10:69–81. [PubMed: 10639397]
24. Eger E, Jedynak A, Iwaki T, Skrandies W. Rapid extraction of emotional expression: evidence
from evoked potential fields during brief presentation of face stimuli. Neuropsychologia
2003;41:808–817. [PubMed: 12631531]
25. Japee S, Crocker L, Carver F, Pessoa L, Ungerleider LG. Individual differences in valence
modulation of face-selective M170 response. Emotion 2009;9:59–69. [PubMed: 19186917]
26. Krolak-Salmon P, Henaff MA, Vighetto A, Bertrand O, Mauguiere F. Early amygdala reaction to
fear spreading in occipital, temporal, and frontal cortex: a depth electrode ERP study in human.
Neuron 2004;42:665–676. [PubMed: 15157426]
27. Greene MR, Oliva A. The briefest of glances: the time course of natural scene understanding.
Psychol. Sci 2009;20:464–472. [PubMed: 19399976]
28. Kawasaki H, et al. Single-neuron responses to emotional visual stimuli recorded in human ventral
prefrontal cortex. Nature Neurosci 2001;4:15–16. [PubMed: 11135639]
29. Amaral, DG.; Price, JL.; Pitkanen, A.; Carmichael, ST. The Amygdala: Neurobiological Aspects of
Emotion, Memory, and Mental Dysfunction. Aggleton, J., editor. New York: Wiley-Liss; 1992. p.
1-66.
30. Gothard KM, Battaglia FP, Erickson CA, Spitler KM, Amaral DG. Neural responses to facial
expression and face identity in the monkey amygdala. J. Neurophysiol 2007;97:1671–1683.
[PubMed: 17093126]
31. Rolls, ET. Emotion Explained. Oxford: Oxford Univ. Press; 2005.
32. Kreiman G, Koch C, Fried I. Category-specific visual responses of single neurons in the human
medial temporal lobe. Nature Neurosci 2000;3:946–953. [PubMed: 10966627]
33. Mormann F, et al. Latency and selectivity of single neurons indicate hierarchical processing in the
human medial temporal lobe. J. Neurosci 2008;28:8865–8872. [PubMed: 18768680]
34. Smith ML, Cottrell GW, Gosselin F, Schyns PG. Transmitting and decoding facial expressions.
Psychol. Sci 2005;16:184–189. [PubMed: 15733197]
35. Smith FW, Schyns PG. Smile through your fear and sadness: transmitting and identifying facial
expression signals over a range of viewing distances. Psychol. Sci 2009;20:1202–1208. [PubMed:
19694983]
36. Adolphs R, et al. A mechanism for impaired fear recognition after amygdala damage. Nature
2005;433:68–72. [PubMed: 15635411]
37. Vuilleumier P. How brains beware: neural mechanisms of emotional attention. Trends Cogn. Sci
2005;9:585–594. [PubMed: 16289871]
38. Pessoa L, Oliveira L, Pereira MG. Attention and emotion. Scholarpedia 2010;5:6314.
39. Pessoa, L.; Oliveira, L.; Pereira, MG. Handbook of Human Affective Neuroscience. Armony, JL.;
Vuilleumier, P., editors. Cambridge, UK: Cambridge Univ. Press; 2011.
40. Tsuchiya N, Moradi F, Felsen C, Yamazaki M, Adolphs R. Intact rapid detection of fearful faces in
the absence of the amygdala. Nature Neurosci 2009;12:1224–1225. [PubMed: 19718036]
41. Piech RM, et al. Fear-enhanced visual search persists after amygdala lesions. Neuropsychologia
2010;48:3430–3435. [PubMed: 20637217]
42. Grieve KL, Acuna C, Cudeiro J. The primate pulvinar nuclei: vision and action. Trends Neurosci
2000;23:35–39. [PubMed: 10631787]
Pessoa and Adolphs Page 15
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
43. Stepniewska, I. The Primate Visual System. Kaas, J.; Collins, CE., editors. Boca Raton, Florida:
CRC Press; 2004. p. 53-80.
44. Bender DB. Visual activation of neurons in the primate pulvinar depends on cortex but not
colliculus. Brain Res 1983;279:258–261. [PubMed: 6640346]
45. Robinson, DL.; Cowie, RJ. The Thalamus. McCormick, D., editor. New York: Elsevier; 1997. p.
53-92.
46. de Gelder B, et al. Standing up for the body. Recent progress in uncovering the networks involved
in the perception of bodies and bodily expressions. Neurosci. Biobehav. Rev 2010;34:513–527.
[PubMed: 19857515]
47. Guillery RW. Anatomical evidence concerning the role of the thalamus in corticocortical
communication: a brief review. J. Anat 1995;187:583–592. [PubMed: 8586557]
48. Sherman SM, Guillery RW. Functional organization of thalamocortical relays. J. Neurophysiol
1996;76:1367–1395. [PubMed: 8890259]
49. Sherman SM. Thalamus. Scholarpedia 2006;1:1583.
50. Ungerleider LG, Christensen CA. Pulvinar lesions in monkeys produce abnormal scanning of a
complex visual array. Neuropsychologia 1979;17:493–501. [PubMed: 117393]
51. Zihl J, von Cramon D. The contribution of the ‘second’ visual system to directed visual attention in
man. Brain 1979;102:835–856. [PubMed: 509197]
52. Benevento LA, Port JD. Single neurons with both form/color differential responses and saccade-
related responses in the nonretinotopic pulvinar of the behaving macaque monkey. Vis. Neurosci
1995;12:523–544. [PubMed: 7654609]
53. Petersen SE, Robinson DL, Keys W. Pulvinar nuclei of the behaving rhesus monkey: visual
responses and their modulation. J. Neurophysiol 1985;54:867–886. [PubMed: 4067625]
54. Desimone R, Wessinger M, Thomas L, Schneider W. Attentional control of visual perception:
cortical and subcortical mechanisms. Cold Spring Harb. Symp. Quant. Biol 1990;55:963–971.
[PubMed: 2132873]
55. Karnath HO, Himmelbach M, Rorden C. The subcortical anatomy of human spatial neglect:
putamen, caudate nucleus and pulvinar. Brain 2002;125:350–360. [PubMed: 11844735]
56. Ward R, Danziger S, Owen V, Rafal R. Deficits in spatial coding and feature binding following
damage to spatiotopic maps in the human pulvinar. Nature Neurosci 2002;5:99–100. [PubMed:
11780145]
57. Wilke M, Mueller KM, Leopold DA. Neural activity in the visual thalamus reflects perceptual
suppression. Proc. Natl Acad. Sci. USA 2009;106:9465–9470. [PubMed: 19458249]
58. Padmala S, Lim S-L, Pessoa L. Pulvinar and affective significance: responses track moment-to-
moment visibility. Front. Hum. Neurosci 2010;4:1–9. [PubMed: 20204154]
59. Pessoa L, Ungerleider LG. Neural correlates of change detection and change blindness in a
working memory task. Cereb. Cortex 2004;14:511–520. [PubMed: 15054067]
60. Shipp S. The functional logic of cortico-pulvinar connections. Phil. Trans. R. Soc. Lond. B
2003;358:1605–1624. [PubMed: 14561322]
61. Lyon DC, Nassi JJ, Callaway EM. A disynaptic relay from superior colliculus to dorsal stream
visual cortex in macaque monkey. Neuron 2010;65:270–279. [PubMed: 20152132]
62. Berman RA, Wurtz RH. Functional identification of a pulvinar path from superior colliculus to
cortical area MT. J. Neurosci 2010;30:6342–6354. [PubMed: 20445060]
63. Shipp S. The brain circuitry of attention. Trends Cogn. Sci 2004;8:223–230. [PubMed: 15120681]
64. Jones EG, Burton H. A projection from the medial pulvinar to the amygdala in primates. Brain Res
1976;104:142–147. [PubMed: 813820]
65. Romanski LM, Giguere M, Bates JF, Goldman-Rakic PS. Topographic organization of medial
pulvinar connections with the prefrontal cortex in the rhesus monkey. J. Comp. Neurol
1997;379:313–332. [PubMed: 9067827]
66. Aggleton JP, Burton MJ, Passingham RE. Cortical and subcortical afferents to the amygdala of the
rhesus monkey (Macaca mulatta). Brain Res 1980;190:347–368. [PubMed: 6768425]
67. Sherman, SM.; Guillery, RW. Exploring the Thalamus. San Diego: Academic Press; 2001.
Pessoa and Adolphs Page 16
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
68. Cowey A. The 30th Sir Frederick Bartlett lecture. Fact, artefact, and myth about blindsight. Q. J.
Exp. Psychol. A 2004;57:577–609. [PubMed: 15204125]
69. Chen CM, et al. Functional anatomy and interaction of fast and slow visual pathways in macaque
monkeys. Cereb. Cortex 2007;17:1561–1569. [PubMed: 16950866]
70. Nakamura H, Gattass R, Desimone R, Ungerleider LG. The modular organization of projections
from areas V1 and V2 to areas V4 and TEO in macaques. J. Neurosci 1993;13:3681–3691.
[PubMed: 7690064]
71. Felleman DJ, Van Essen DC. Retinotopic organization in human visual cortex and the spatial
precision of functional MRI. Cereb. Cortex 1991;1:1–47. [PubMed: 1822724]
72. Lamme VA, Roelfsema PR. The distinct modes of vision offered by feedforward and recurrent
processing. Trends Neurosci 2000;23:571–579. [PubMed: 11074267]
73. Yukie M, Iwai E. Direct projection from the dorsal lateral geniculate nucleus to the prestriate
cortex in macaque monkeys. J. Comp. Neurol 1981;201:81–97. [PubMed: 7276252]
74. Bullier J, Kennedy H. Projection of the lateral geniculate nucleus onto cortical area V2 in the
macaque monkey. Exp. Brain Res 1983;53:168–172. [PubMed: 6201379]
75. Schmid MC, Panagiotaropoulos T, Augath MA, Logothetis NK, Smirnakis SM. Visually driven
activation in macaque areas V2 and V3 without input from the primary visual cortex. PLoS ONE
2009;4:e5527. [PubMed: 19436733]
76. Boyer JL, Harrison S, Ro T. Unconscious processing of orientation and color without primary
visual cortex. Proc. Natl Acad. Sci. USA 2005;102:16875–16879. [PubMed: 16263934]
77. Schmid MC, et al. Blindsight depends on the lateral geniculate nucleus. Nature 2010;466:373–377.
[PubMed: 20574422]
78. Rempel-Clower NL, Barbas H. The laminar pattern of connections between prefrontal and anterior
temporal cortices in the Rhesus monkey is related to cortical structure and function. Cereb. Cortex
2000;10:851–865. [PubMed: 10982746]
79. Bar M. A cortical mechanism for triggering top-down facilitation in visual object recognition. J.
Cogn. Neurosci 2003;15:600–609. [PubMed: 12803970]
80. Bullier J. Integrated model of visual processing. Brain Res. Brain Res. Rev 2001;36:96–107.
[PubMed: 11690606]
81. Kveraga K, Boshyan J, Bar M. Magnocellular projections as the trigger of top-down facilitation in
recognition. J. Neurosci 2007;27:13232–13240. [PubMed: 18045917]
82. Barrett LF, Bar M. See it with feeling: affective predictions during object perception. Phil. Trans.
R. Soc. Lond. B 2009;364:1325–1334. [PubMed: 19528014]
83. Capalbo M, Postma E, Goebel R. Combining structural connectivity and response latencies to
model the structure of the visual system. PLoS Comput. Biol 2008;4:e1000159. [PubMed:
18769707]
84. Nowak, LG.; Bullier, J. Cerebral Cortex: Extrastriate Cortex in Primate. Rockland, K.; Kass, J.;
Peters, A., editors. New York: Plenum; 1997. p. 205-241.
85. Sugase Y, Yamane S, Ueno S, Kawano K. Global and fine information coded by single neurons in
the temporal visual cortex. Nature 1999;400:142–147.
86. Sripati AP, Olson CR. Representing the forest before the trees: a global advantage effect in
monkey inferotemporal cortex. J. Neurosci 2009;29:7788–7796. [PubMed: 19535590]
87. Rudrauf D, et al. Rapid interactions between the ventral visual stream and emotion-related
structures rely on a two-pathway architecture. J. Neurosci 2008;28:2793–2803. [PubMed:
18337409]
88. Andino SL, Menendez RG, Khateb A, Landis T, Pegna AJ. Electrophysiological correlates of
affective blindsight. Neuroimage 2009;44:581–589. [PubMed: 18835454]
89. Rotshtein P, et al. Amygdala damage affects event-related potentials for fearful faces at specific
time windows. Hum. Brain Mapp 2010;31:1089–1105. [PubMed: 20017134]
90. Ward R, Calder AJ, Parker M, Arend I. Emotion recognition following human pulvinar damage.
Neuropsychologia 2007;45:1973–1978. [PubMed: 17250857]
91. Ward R, Danziger S, Bamford S. Response to visual threat following damage to the pulvinar. Curr.
Biol 2005;15:571–573. [PubMed: 15797028]
Pessoa and Adolphs Page 17
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
92. Vuilleumier P, Richardson MP, Armony JL, Driver J, Dolan RJ. Distant influences of amygdala
lesion on visual cortical activation during emotional face processing. Nature Neurosci
2004;7:1271–1278. [PubMed: 15494727]
93. Ghashghaei HT, Hilgetag CC, Barbas H. Sequence of information processing for emotions based
on the anatomic dialogue between prefrontal cortex and amygdala. Neuroimage 2007;34:905–923.
[PubMed: 17126037]
94. Averbeck BB, Seo M. The statistical neuroanatomy of frontal networks in the macaque. PLoS
Comput. Biol 2008;4:e1000050. [PubMed: 18389057]
95. Modha DS, Singh R. Network architecture of the long-distance pathways in the macaque brain.
Proc. Natl Acad. Sci. USA 2010;107:13485–13490. [PubMed: 20628011]
96. Lim SL, Padmala S, Pessoa L. Segregating the significant from the mundane on a moment-to-
moment basis via direct and indirect amygdala contributions. Proc. Natl Acad. Sci. USA
2009;106:16841–16846. [PubMed: 19805383]
97. Whalen PJ. Fear, vigilance, and ambiguity: initial neuroimaging studies of the human amygdala.
Curr. Dir. Psychol.Sci 1998;7:177–188.
98. Sander D, Grafman J, Zalla T. The human amygdala: an evolved system for relevance detection.
Rev. Neurosci 2003;14:303–316. [PubMed: 14640318]
99. Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF. Neural systems responding to degrees of
uncertainty in human decision-making. Science 2005;310:1680–1683. [PubMed: 16339445]
100. Pessoa L. Emotion and cognition and the amygdala: from “what is it?” to “what’s to be done?”.
Neuropsychologia 2010;48:3416–3429. [PubMed: 20619280]
101. Grossberg S, Levine DS. Neural dynamics of attentionally modulated Pavlovian conditioning:
blocking, interstimulus interval, and secondary reinforcement. Appl. Opt 1987;26:5015–5030.
[PubMed: 20523481]
102. Aggleton, JP., editor. The Amygdala: Neurobiological Aspects of Emotion, Memory, and Mental
Dysfunction. New York: John Wiley & Sons; 1992.
103. Aggleton, J., editor. The Amygdala: A Functional Analysis. Oxford: Oxford Univ. Press; 2000.
104. Ouellette BG, Casanova C. Overlapping visual response latency distributions in visual cortices
and LP-pulvinar complex of the cat. Exp. Brain Res 2006;175:332–341. [PubMed: 16816944]
105. Schmolesky MT, et al. Signal timing across the macaque visual system. J. Neurophysiol
1998;79:3272–3278. [PubMed: 9636126]
106. Boehnke SE, Munoz DP. On the importance of the transient visual response in the superior
colliculus. Curr. Opin. Neurobiol 2008;18:544–551. [PubMed: 19059772]
107. Leonard CM, Rolls ET, Wilson FA, Baylis GC. Neurons in the amygdala of the monkey with
responses selective for faces. Behav. Brain Res 1985;15:159–176. [PubMed: 3994832]
108. Kuraoka K, Nakamura K. Responses of single neurons in monkey amygdala to facial and vocal
emotions. J. Neurophysiol 2007;97:1379–1387. [PubMed: 17182913]
109. Nakamura K, Mikami A, Kubota K. Activity of single neurons in the monkey amygdala during
performance of a visual discrimination task. J. Neurophysiol 1992;67:1447–1463. [PubMed:
1629757]
110. Oya H, Kawasaki H, Howard MA, Adolphs R. Electrophysiological responses in the human
amygdala discriminate emotion categories of complex visual stimuli. J. Neurosci 2002;22:9502–
9512. [PubMed: 12417674]
111. Tovee MJ, Rolls ET. Information encoding in short firing rate epochs by single neurons in the
primate temporal visual cortex. Vis. Cogn 1995;2:35–58.
112. Yoshor D, Bosking WH, Ghose GM, Maunsell JH. Receptive fields in human visual cortex
mapped with surface electrodes. Cereb. Cortex 2007;17:2293–2302. [PubMed: 17172632]
113. King AJ, Nelken I. Unraveling the principles of auditory cortical processing: can we learn from
the visual system? Nature Neurosci 2009;12:698–701. [PubMed: 19471268]
Pessoa and Adolphs Page 18
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Figure 1. Visual pathways
a | A traditional flowchart of visual processing typically emphasizes the LGN–V1–V2–V4–
TEO–TE pathway, although the scheme is not strictly hierarchical. The amygdala, in
particular, is a recipient of visual signals from the anterior visual cortex. According to the
‘standard hypothesis’, a subcortical pathway involving the superior colliculus and the
pulvinar nucleus of the thalamus provides fast and automatic access to the amygdala. b | An
alternative view of the flow of visual signals includes multiple pathways, including both
alternative routes (for example, LGN to MT) and shortcuts (for example, V2 to TEO). Only
some of these are shown. The flow of visual information may be more appropriately viewed
in terms of ‘multiple waves’ of activation that initiate and refine cell responses at a given
processing ‘stage’. For simplicity, feedback pathways, which are known to be quite
extensive, have been omitted. The existence of such feedback pathways dictates, however,
that a complex ebb-and-flow of activation sculpts the neuronal profile of activation
throughout the visual cortex, and likewise the amygdala responses. Some of the connections
between the pulvinar and visual cortex, and between the pulvinar and ‘associational’ areas,
are also indicated. The line in the pulvinar is intended to schematically separate the medial
pulvinar (to the right of the line) from the rest of the structure. FEF, frontal eye field; LGN,
lateral geniculate nucleus; MT, medial temporal area (also known as V5); OFC,
Pessoa and Adolphs Page 19
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
orbitofrontal cortex; SC, superior colliculus; TE, inferior temporal area TE; TEO, inferior
temporal area TEO; V, visual cortex; VLPFC, ventrolateral prefrontal cortex.
Pessoa and Adolphs Page 20
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Figure 2. Intact non-conscious processing of fearful faces in the absence of the amygdala
In a subject with complete amygdala lesions (subject S.M.), fearful faces broke into
consciousness during continuous flash suppression with latencies similar to those of control
subjects. a | Experimental stimulus: fearful or happy faces were shown to the non-dominant
eye while a flashing Mondrian pattern was shown to the dominant eye. This technique is
called continuous flash suppression as it suppresses the visibility of the stimulus presented to
the non-dominant eye. The contrast of the Mondrian pattern was gradually decreased while
that of the face was increased until subjects could detect the face and indicate it with a
button press to establish reaction time. b | Plots of reaction times (RT) in the task. In the case
of S.M., fearful faces broke interocular suppression faster than happy faces (shown by the
red bar) and to the same degree as 7 demographically matched healthy controls (shown by
blue bars; the mean and standard deviation are also shown). Figure is reproduced, with
permission, from REF. 40 © (2009) Macmillan Publishers Ltd. All rights reserved.
Pessoa and Adolphs Page 21
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Figure 3. Schematic layout of the pulvinar
Some traditional characterizations of the pulvinar emphasize the inferior (Inf), lateral (Lat)
and medial (Med) nuclei. Most pulvinar nuclei (including other nuclei and sub-nuclei that
are not shown here) are involved in thalamo–cortical loops that target different cortical
territories (shown in blue)42,60. The inferior nucleus is reciprocally connected to striate and
extrastriate cortices, the lateral nucleus is connected to association cortices in temporal and
parietal lobes (although it is also interconnected with the extrastriate cortex) and the medial
nucleus is connected to the higher-order association cortex in parietal, frontal, orbital (not
shown), cingulate and insular regions (the insula is not shown), in addition to the amygdala.
Thus, the medial nucleus, which is of great interest in the present context, is not only
connected with the amygdala but is also part of multiple thalamo–cortical loops (note,
however, that the connection to the amygdala does not seem to be bidirectional). The
superior colliculus is a layered structure whose superficial layers are visual in nature and
project to the inferior nucleus. Its intermediate and deeper layers are multimodal and
involved in motor preparation, including for eye movements, and project to the medial
nucleus. A ventrolateral to dorsomedial axis that is helpful in understanding the organization
of pulvinar nuclei and potential ‘ventral’ and ‘dorsal’ domains is shown by a dotted line (see
also REF. 42 for a related scheme). Figure is modified, with permission, from REF. 43 ©
(2004) CRC Press. IT, inferior temporal cortex; MT, medial temporal area (also known as
V5).
Pessoa and Adolphs Page 22
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Figure 4. Pulvinar and amygdala during processing of affective stimuli
a | Logistic regression analysis of evoked responses in the left pulvinar as a function of
affective significance for a sample individual during an attentional blink task58. The slope of
the logistic fit indicates the strength of the predictive effect. For clarity, only binned data for
the conditioned stimulus (CS+) condition are included (shown by orange dots). The grey
line shows the fit for these data, and the blue line shows the fit for data from the neutral
stimulus (CS−) condition. The inset shows mean logistic slopes across individuals, revealing
that a relationship was detected for the affective (CS+) but not the neutral (CS−) condition.
b | The medial pulvinar is proposed to amplify evoked responses of behaviourally-relevant
stimuli via circuits involving the cingulate cortex, orbitofrontal cortex (OFC) and amygdala,
all regions important for the valuation of an incoming stimulus. c | Valuation signals in the
amygdala affect behaviour by impacting responses across the brain. During an attentional
blink task using affective stimuli, a response in the amygdala to a stimulus predicted that the
stimulus would be detected96. Statistical path analysis revealed that this effect is mediated
through projections from the amygdala to the visual cortex, as well as through projections
involving the prefrontal cortex (PFC). fMRI, functional MRI. Data in part a from REF. 58.
Pessoa and Adolphs Page 23
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript