ArticlePDF AvailableLiterature Review

Emotion processing and the amygdala: From a 'low road' to 'many roads' of evaluating biological significance

Authors:

Abstract and Figures

A subcortical pathway through the superior colliculus and pulvinar to the amygdala is commonly assumed to mediate the non-conscious processing of affective visual stimuli. We review anatomical and physiological data that argue against the notion that such a pathway plays a prominent part in processing affective visual stimuli in humans. Instead, we propose that the primary role of the amygdala in visual processing, like that of the pulvinar, is to coordinate the function of cortical networks during evaluation of the biological significance of affective visual stimuli. Under this revised framework, the cortex has a more important role in emotion processing than is traditionally assumed.
Visual pathways.a | A traditional flowchart of visual processing typically emphasizes the LGN–V1–V2–V4–TEO–TE pathway, although the scheme is not strictly hierarchical. The amygdala, in particular, is a recipient of visual signals from the anterior visual cortex. According to the 'standard hypothesis', a subcortical pathway involving the superior colliculus and the pulvinar nucleus of the thalamus provides fast and automatic access to the amygdala. b | An alternative view of the flow of visual signals includes multiple pathways, including both alternative routes (for example, LGN to MT) and shortcuts (for example, V2 to TEO). Only some of these are shown. The flow of visual information may be more appropriately viewed in terms of 'multiple waves' of activation that initiate and refine cell responses at a given processing 'stage'. For simplicity, feedback pathways, which are known to be quite extensive, have been omitted. The existence of such feedback pathways dictates, however, that a complex ebb-and-flow of activation sculpts the neuronal profile of activation throughout the visual cortex, and likewise the amygdala responses. Some of the connections between the pulvinar and visual cortex, and between the pulvinar and 'associational' areas, are also indicated. The line in the pulvinar is intended to schematically separate the medial pulvinar (to the right of the line) from the rest of the structure. FEF, frontal eye field; LGN, lateral geniculate nucleus; MT, medial temporal area (also known as V5); OFC, orbitofrontal cortex; SC, superior colliculus; TE, inferior temporal area TE; TEO, inferior temporal area TEO; V, visual cortex; VLPFC, ventrolateral prefrontal cortex.
… 
Content may be subject to copyright.
Emotion processing and the amygdala: from a ‘low road’ to
‘many roads’ of evaluating biological significance
Luiz Pessoa and
Department of Psychological and Brain Sciences, Indiana University, Bloomington, Indiana 47405
USA
Ralph Adolphs
Division of Humanities and Social Sciences, and Computation and Neural Systems Program,
California Institute of Technology, Pasadena, California 91125, USA
Abstract
A subcortical pathway through the superior colliculus and pulvinar to the amygdala is commonly
assumed to mediate the non-conscious processing of affective visual stimuli. We review
anatomical and physiological data that argue against the notion that such a pathway plays a
prominent part in processing affective visual stimuli in humans. Instead, we propose that the
primary role of the amygdala in visual processing, like that of the pulvinar, is to coordinate the
function of cortical networks during evaluation of the biological significance of affective visual
stimuli. Under this revised framework, the cortex has a more important role in emotion processing
than is traditionally assumed.
There is tremendous interest in the question of how salient, emotional and socially-charged
visual stimuli are processed by the brain. This topic is important because it addresses a
fundamental question regarding how biological ‘value’ is assigned by an animal to stimuli in
its environment: which stimuli are good and which are bad; which should be approached and
which should be avoided. The topic is also intriguing because it fuels questions about
modularity in the brain (that is, whether there is a specialized way of, or even dedicated
neural substrates for, the processing of affective stimuli). The overarching ‘standard
hypothesis’ runs roughly as follows: ecologically important (emotional and social) stimuli
are processed initially by a dedicated, modular system that operates rapidly, automatically
(without the need to pay attention) and largely independently of conscious awareness1.
Defects in this system are suggested to underlie phobias, mood disorders and post-traumatic
stress syndrome, and variability in its functioning reflects individual differences at the
genotypic and personality level2,3.
© 2010 Macmillan Publishers Limited. All rights reserved
Correspondence to L.P. and R.A. lpessoa@indiana.edu; radolphs@hss.caltech.edu.
Competing interests statement
The authors declare no competing financial interests.
FURTHER INFORMATION
Luiz Pessoa’s homepage: http://emotioncognition.org
Ralph Adolphs’ homepage: http://www.emotion.caltech.edu
SUPPLEMENTARY INFORMATION
See online article: S1 (box)
ALL LINKS ARE ACTIVE IN THE ONLINE PDF
NIH Public Access
Author Manuscript
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
Published in final edited form as:
Nat Rev Neurosci
. 2010 November ; 11(11): 773–783. doi:10.1038/nrn2920.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
The hypothesis has two central and related components. The first component is the
purported role of the amygdala in the rapid, automatic and non-conscious processing of
emotional and social stimuli. The second central component is the proposal of a specific
subcortical route of information processing — the so-called ‘low road’ (REF. 4) — that
bypasses the presumably slower, resource-dependent cortex and that culminates in the
amygdala by way of the superior colliculus and the pulvinar nucleus of the thalamus (FIG.
1a). The fact that this pathway bypasses the cortex is thought to imbue the processing of
emotion-laden visual stimuli with the above list of properties.
Given that the standard hypothesis is shaping both basic and clinical research, this
Perspective article provides a critical re-examination of this hypothesis that we hope will re-
orient thinking about the processing of emotion stimuli and the roles of the amygdala and
pulvinar therein. The main points that we make are as follows: first, there is no evidence for
a functional subcortical route for visual processing in primates; second, the cortex plays a
larger part in the processing of affective visual information than is typically acknowledged;
third, the visual processing of emotion stimuli occurs no faster than visual processing in the
cortex in general; fourth, the amygdala’s contribution to processing of affective visual
information arises from its broad connectivity with the cortex and other subcortical
structures; and finally, the pulvinar plays a part in the processing of emotion stimuli through
its extensive connectivity with cortical sites.
The standard hypothesis
The data and theory that underpin the standard hypothesis are not typically articulated in
detail, and its central concepts are often vague. The main argument is that insofar as
affectively-laden information has survival value, it has driven adaptations in information
processing that are reflected in a functionally and structurally modular system2. The
purported modularity of the system entails automaticity5: owing to the potency of affective
information, this information is processed independently of attention and awareness. For
example, threat-expressing faces have been reported to be processed pre-attentively in
visual search paradigms6, and fearful faces break into consciousness more quickly than
happy expressions during continuous flash suppression (a technique used to render
visual stimuli non-conscious)7. Moreover, haemodynamic responses in the amygdala have
been reported to occur in response to presentation of fearful faces that have (putatively) been
rendered invisible by backward masking8,9 and even during unmasked presentation of
fearful faces in patients with blindsight10,11.
It is also assumed that the anatomical components of the system enable emotion processing
to occur to a substantial extent subcortically12. This suggestion has its roots in rodent
studies that demonstrate the existence of a subcortical pathway, through the auditory
thalamus to the amygdala, that is sufficient for some forms of auditory Pavlovian fear
conditioning4. It is assumed that a similar subcortical route exists for visual information
processing in primates, including humans (see below). The notion of such a subcortical
pathway is appealing because subcortical visual processing is assumed to be faster than
cortical visual processing, and processing of affective stimuli is thought to be adaptive in
part, because it is fast. For example, judgments of threat can be made from facial stimuli that
are presented for as briefly as 39 ms (backward masked)13. Because the pathway is assumed
to be subcortical, processing of visual information along this pathway is assumed to be
coarse. Thus, coarse (that is, low-spatial-frequency) information from affective stimuli is
thought to engage subcortical visual processing, consistent with findings that the amygdala
(a collection of subcortical nuclei) is activated more strongly by emotional faces presented
with low than high spatial frequency14.
Pessoa and Adolphs Page 2
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
In this Perspective we aim to discuss several shortcomings of the standard hypothesis and to
propose an alternative view that we think is better justified by the data, and that we hope
will stimulate new research directions. This Perspective does not focus specifically on the
amygdala, as several comprehensive reviews about the role of this structure in processing
affective information already exist15,16. Instead, we emphasize physiological and anatomical
data concerning the pulvinar, the key ‘link structure’ of the subcortical pathway. Although
most of the data described in this Perspective are well known to parts of the neuroscience
community, the lack of appreciation for these data by ‘affective science’ researchers could
partly explain the widespread acceptance of the standard hypothesis.
We begin by discussing the functional properties of the processing of affective visual
information that are central to the standard hypothesis and focus on the issues of speed and
coarseness. These issues are relevant to the subsequent discussion of the pulvinar and to the
alternative scheme we propose for explaining the properties of visual processing of affective
information. Other important notions that are linked to the standard hypothesis, including
the modularity of the brain and the roles of attention and awareness in visual processing, are
discussed only briefly because they have been reviewed elsewhere1721. Finally, we propose
an alternative view that assigns a larger role to cortical processing of affective visual
information. We suggest that this scheme, which we call the ‘multiple-waves model’, can
explain the types of findings that have been used to support the standard hypothesis.
General functional issues
Affective visual information is not processed faster than other visual information
Electrophysiological responses evoked by visual stimuli can be modulated by the emotional
content of the stimuli, and this modulation has been reported to occur at short latencies — in
some studies in humans, within ~100 ms of stimulus onset22,23. In addition, the N170
component of the electro encephalography (EEG) signal (or the M170 component in
magnetoencephalography (MEG) studies), which is associated with face identification, is in
some studies modulated by the emotional expression of the presented face24,25. However,
numerous studies only showed effects with longer latencies, ranging from 200–400 ms (for
example, REF. 26). Even in the studies reporting short latency responses to emotional visual
stimuli, localizing the neural sources of those responses using EEG or MEG is problematic
and, therefore, the origin of the signals in these studies might not be in the subcortical
pathway.
In addition, single-unit recordings in monkeys indicate that responses in the cortex (even in
the frontal cortex) occur with latencies that are within the range of the latencies observed in
subcortical areas (BOX 1). This is consistent with the idea of a rapid, feedforward ‘cortical
sweep’ of visual information processing. Moreover, behavioural and electrophysiological
studies of object processing and perception in humans suggest that visual processing in
general (that is, including non-affective processing) can be remarkably fast and that a
substantial amount of information can be gathered from even a single glance at a natural
scene (see Supplementary information S1 (box)). For example, a recent study showed that
only 19–67 ms were required to attain 75% correct performance on several tasks, including
determining a scene’s global property (for example, ‘natural scene’) and basic level
categorization (for example, ‘forest’)27. Thus, there is nothing particularly special about the
processing speed of affective information. Finally, the speed of cortical — as opposed to
subcortical — visual processing could also account for the reported rapid modulation of
evoked brain responses by stimulus valence. Valence, like other affectively relevant
stimulus dimensions, is probably computed in several brain regions, one of which is likely to
be the orbitofrontal cortex (OFC). Short-latency (100–150 ms) electrophysiological
responses in the OFC have been associated with discrimination of the valence of a visual
Pessoa and Adolphs Page 3
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
stimulus28. We develop this final point in greater depth below, in the context of our new
proposal.
Taken together, the findings above indicate that processing of affective visual stimuli is no
faster than cortical processing of visual stimuli in general. Indeed, cortical visual processing
is both efficient and fast. Thus, the argument that a separate subcortical system is required
for fast perception of affective stimuli is problematic (Supplementary information S1 (box)).
Box 1 | Speed of visual processing
One way to assess the speed of visual processing is to measure and contrast response
latencies across brain areas. For example, do responses in the purported subcortical
pathway occur earlier than those in cortical sites? The figure shows that in the macaque
cerebral cortex, the earliest latencies are remarkably short, and even mean response
latencies indicate remarkably fast cortical processing72. Areas that became active at the
given latency after visual stimulation are shown in red, those that were activated earlier
in yellow and those that were not yet activated in white. Areas for which no information
was available are shown in dark grey (see the figure). Visual response latencies in the
pulvinar are between 60–80 ms and overlap with latencies observed in early visual
cortical areas V1 and V2 (REF. 104). In the inferotemporal cortex (that is, ‘late’ visual
cortex) latencies can be as short as 60–85 ms72 and, strikingly, in some frontal sites such
as the frontal eye fields (FEF) as short as 40–70 ms. These latencies again overlap with
those in area V1 (REFS 84,105). Thus, although mean response latencies increase
gradually from posterior to anterior visual cortices, there is considerable overlap (see the
figure). In the context of the standard hypothesis, it therefore seems that pulvinar
responses are not particularly fast. However, it is of interest that visual response latencies
in the superior colliculus are somewhat faster than those observed in the pulvinar,
showing an early, transient response around 40–70 ms that may support rapid eye
movements during orienting106 (note that these response times overlap with FEF
responses).
What are the response latencies of neurons in the amygdala? In the monkey amygdala,
responses to visual stimuli range from 100–200 ms30,107109, although shorter response
latencies to unspecific stimuli (for example, fixation spots) have been reported30.
Differences in evoked responses between threatening and neutral or appeasing facial
expressions in the monkey amygdala have been found in the range of 120–250 ms30.
Intracranial studies in humans generally find the earliest single-unit responses to visual
stimuli around 200 ms33,110. Moreover, in one study, modulation of amygdala
responses by the affective content of stimuli was observed to start at 200 ms26 (see also
REF. 110).
In summary, subcortical visual processing is not discernably faster than cortical
processing. Furthermore, the crucial variable is not the timing of the initial stimulus
responses but the time at which reliable differences between affective and non-affective
stimuli can be detected. It has been suggested111 that most of the information encoded
by visual neurons may be available in 100-ms-long segments of activity (that is, spiking
data within a 100 ms epoch) and that a fair amount of information is available in
segments of 50 ms, and even some of 20–30 ms (note that these segments consider post-
Pessoa and Adolphs Page 4
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
latency neuronal spikes only). Although these data demonstrate the remarkable speed of
neuronal computation (at least under some conditions), they add milliseconds to the time
that is required to, for example, discriminate between stimuli. A final consideration is
that responses in humans are possibly slower than in monkeys. For example, in one study
in humans, the fastest recording sites had response latencies of just under 60 ms and were
probably located in V1 (or possibly V2)112. In the monkey, the fastest responses in V1
can be observed under 40 ms72. 5, Brodmann area 5; 7a, Brodmann area 7; 7ip,
Brodmann area 7ip (intraparietal); 8a, Brodmann area 8a; EC, entorhinal cortex; FEF,
frontal eye field; FST, fundus of superior temporal cortex; IPa, superior temporal area
IPa; M1, primary motor cortex; MST, medial superior temporal cortex; MT, medial
temporal area (also known as V5); OFC, orbitofrontal cortex; PFC, prefrontal cortex;
PGa, superior temporal area PGa; PreM, premotor cortex; SEF, supplementary eye field;
SMA, supplementary motor area; TAa, anterior subregion of superior temporal area TA;
TE1, inferior temporal area TE1; TE2, inferior temporal area TE2; TE3, inferior temporal
area TE3; TEm/TEa, medial and anterior subregions of inferior temporal area TE; TPO,
superior temporal area TPO; TS, superior temporal sulcus. Figure is reproduced, with
permission, from REF. 72 © (2000) Cell Press.
Processing of affective visual stimuli involves both coarse- and high-spatial-frequency
information
According to the standard hypothesis, the subcortical pathway is particularly effective at
carrying low-spatial-frequency information, mainly because the superior colliculus and
pulvinar are assumed not to convey much high-spatial-frequency information. This notion
was initially based on findings in rodents that simple (‘coarse’) auditory conditioning does
not require the cortex, whereas conditioning that requires more complex stimulus
discriminations does4.
Neuroimaging studies in humans seem to be consistent with this idea. For example, in one
study amygdala responses were stronger when participants viewed low- compared with
high-spatial frequency fearful faces14, and when they viewed fearful faces versus neutral
faces both at low spatial frequency14. Similarly, activation in brain areas consistent with the
location of the superior colliculus and pulvinar was greater in response to fearful faces than
to neutral faces at low spatial frequencies14. Findings of this kind have been interpreted as
suggesting that the amygdala is relatively ‘blind’ to high-spatial-frequency information.
However, the amygdala receives major projections from the anterior inferotemporal
cortex29 that convey highly processed object information — in fact, the amygdala receives
highly processed cortical input from all sensory modalities except olfaction29. Indeed,
electrophysiological studies have shown that the monkey amygdala contains neurons that are
tuned to the identity of specific faces30,31 and that the human amygdala shows category-
specific responses (for example, for animals or natural scenes)32,33. These are properties
that require high-spatial-frequency information.
Moreover, it has been shown that the discrimination of facial expressions relies on both low-
and high-spatial-frequency information34. The perception of fear is particularly reliant on
high-spatial-frequency information35. Indeed, a study in a patient with bilateral amygdala
lesions showed that this patient’s impaired recognition of facial expressions of fear was due
to impaired processing of the eye region of faces, especially of high-spatial-frequency
information about the eyes36. These results demonstrate the importance of high-spatial-
frequency information in fear recognition and indicate that the amygdala is required for this
type of visual processing.
Pessoa and Adolphs Page 5
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
In summary, although some findings are consistent with the notion that subcortical areas
process coarse visual information, the perception of emotional expressions actually involves
both coarse and fine information. Furthermore, the amygdala not only receives inputs that
convey fine spatial information but seems to be crucially involved in using this information
to decode facial expressions (Supplementary information S1 (box)).
The amygdala is not essential for rapid, non-conscious detection of affective information
The standard hypothesis often draws on additional themes, such as the roles of attention and
awareness in visual processing, and the more general issue of the extent to which emotion
and cognition are processed by separate circuits in the brain. Both of these issues are
complex and have been discussed at greater length elsewhere19,20, but we briefly comment
on attention and awareness.
Processing affective information is known to occur under some conditions of inattention and
unawareness. However, as discussed elsewhere, the interpretation of the published literature
in terms of ‘strong automaticity’ is unwarranted20. Briefly, behavioural, functional MRI
(fMRI) and EEG studies indicate that when processing resources are sufficiently consumed
(for example, by engaging attention on challenging tasks), visual processing of emotional
stimuli is greatly reduced or eliminated. This challenges the notion that emotional stimuli
are processed automatically (see REFS 3739 for further discussion of the role of attention).
A recent study of a patient with complete amygdala lesions40 perhaps provides the most
decisive data on this issue. In this patient, reaction times for detecting fearful faces among
distractor stimuli were within the normal range, and fearful facial expressions broke into
consciousness faster than happy faces during binocular suppression to the same degree as in
control subjects (FIG. 2). These findings demonstrate that the amygdala is not essential for
non-conscious, rapid fear detection, at least in the tasks used in this study40 (see also REF.
41).
Thus, independent lines of evidence challenge the notion that processing of affective visual
information occurs independently of attention and awareness. Moreover, some of the
properties typically connected in the literature with automaticity (for example, detecting
fearful faces among distractors) may not entirely depend on the amygdala.
Physiological and anatomical issues
The pulvinar is a key link element in the purported colliculus–pulvinar–amygdala pathway.
Here, we briefly review physiological and anatomical data regarding the pulvinar — and the
subcortical pathway of which it is thought to be a component — that are relevant in the
context of the standard hypothesis. In particular, we discuss data that are relevant to the
question of whether this structure is better conceptualized as a relatively passive way station
or as a dynamic element of brain circuitry.
Pulvinar input
The pulvinar complex is the largest nuclear mass in the primate thalamus and is thought to
have expanded in size during evolution in parallel with other visual structures42. The
pulvinar does not seem to exist in brains of rodents and other small mammals43. In terms of
connectivity that is relevant for visual processing, it receives direct visual input from the
retina, indirect visual input via the superficial layers of the superior colliculus and massive
input from striate and extrastriate visual cortices (FIG. 3). All of these projections terminate
in the inferior pulvinar. Intriguingly, however, the visual response properties of pulvinar
cells do not reflect those of neurons in the superior colliculus, and the precise contribution
that input from the superior colliculus makes to pulvinar responses remains uncertain43. For
example, superior colliculus lesions have little effect on electrophysiological responses of
Pessoa and Adolphs Page 6
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
pulvinar neurons, in contrast to striate cortex lesions, which abolish responses in the inferior
pulvinar44. In a related fashion, collicular and pulvinar lesions result in different
behavioural impairments45 (see also REF. 46). These observations argue that the pulvinar
may be better thought of as participating in cortical networks, rather than as relaying visual
information from the superior colliculus. This is borne out by the finding that, unlike the
lateral geniculate nucleus (LGN), the pulvinar’s driving inputs in fact originate in the cortex,
whereas subcortical inputs to the pulvinar are typically modulatory47,48; for this reason, the
pulvinar is described as a higher-order thalamic nucleus, as opposed to a first-order nucleus
such as the LGN49.
Pulvinar activity
Studies in monkeys and humans with pulvinar lesions have suggested that this structure is
involved in determining what is salient in a visual scene50,51. Consistent with this notion,
the response of pulvinar neurons to visual stimuli is increased if attention is paid to the
stimulus or if the stimulus has behavioural relevance. For example, pulvinar neurons
respond more vigorously to behaviourally relevant targets than to unattended stimuli45. In
one study, as many as 92% of pulvinar cells exhibited attenuated responses to stimuli that
were task-irrelevant (that is, passively viewed)52 compared to stimuli that were task-
relevant. Furthermore, the impact of attention on evoked responses in the pulvinar is
spatially specific, such that a pulvinar neuron only increases activity when a monkey attends
to a stimulus that falls within the receptive field of the cell53. Finally, the pulvinar seems to
be crucial (as was shown in a study in which the pulvinar was pharmacologically
inactivated) when a distractor stimulus has to be ‘filtered out’54. Thus, it has been proposed
that the pulvinar is involved in attention and/or distractor filtering, and this is consistent with
data from neuroimaging and lesion studies in humans (Supplementary information S1
(box)).
The pulvinar is also important for visual awareness. For example, lesion studies in humans
have revealed that pulvinar damage is associated with visual neglect and with feature-
binding deficits51,55,56. A recent study in monkeys is particularly noteworthy: here, neural
activity was recorded in the pulvinar during a visual illusion that induced the intermittent
perceptual suppression of a bright luminance patch57. Neurons in the pulvinar showed
changes in spiking rate according to trial-by-trial stimulus visibility, suggesting that they
reflected the visual awareness of the stimulus. Similarly, a recent fMRI study in humans
found that the pulvinar responded not to the affective significance of visual stimuli but to
whether or not they were consciously perceived58, again in a trial-by-trial manner. The
fMRI results are consistent with a study in which pulvinar responses were associated with a
subject’s percept of a change59. Notably, responses were observed during ‘false alarm’ trials
(those in which a stimulus change was reported but did not actually occur) but not during
‘miss’ trials (those in which a stimulus change occurred but went unnoticed by the
participant). These results do not support the suggestion that the pulvinar is involved during
non-conscious processing, and are inconsistent with a major role for this structure in the
subcortical pathway proposed by the ‘standard hypothesis’, according to which the pulvinar
behaves as a relatively passive relay.
Pulvinar anatomy
It is also important to consider the anatomical features of the pulvinar that highlight its
extensive bidirectional connectivity with the cortex. For example, all 20–30 known visual
areas connect with the pulvinar, sometimes in a relatively topographic fashion43,60.
Temporal, parietal, cingulate, frontal and insular cortices are all connected with the pulvinar
as well. At a gross level, it is as if the entire convoluted cortex were ‘shrink-wrapped’
around the pulvinar60. Based on the connectivity data, one can discern a ventrolateral to
Pessoa and Adolphs Page 7
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
dorsomedial axis, and this has led to the suggestion that the pulvinar may contain two
connectional ‘domains’ (REFS 42,60) (FIG. 3). The ventral domain (containing the inferior
pulvinar) is densely connected with the visual cortex (including areas V1–V4 and MT)60. It
therefore has a strong visual component (that is, it could be called the ‘visual pulvinar’), and
its projections to the dorsal visual stream may mediate some of the visual abilities in people
with blindsight61,62 (Supplementary information S1 (box)). The dorsal domain (including
the traditional medial subdivision; see below) has connections with the cross-modal
association cortex, including temporal and parietal areas (including area 7A and the lateral
intraparietal (LIP) cortex — areas that are also involved in attention)60. The dorsal domain
receives highly processed visual input from the inferior temporal gyrus area TE (and some
from area TEO) in the inferotemporal cortex60. It is also connected with the cingulate
cortex, frontal cortex (including the OFC), insula and amygdala60 (see below). The dorsal
domain is therefore much more ‘associational’ and, in fact, has remarkable potential to
integrate information from very diverse brain regions (FIG. 3).
In summary, the pulvinar is a complex structure with important visual and integrative
properties. Functional studies have characterized several ways in which the pulvinar is
modulated by attention and awareness. Indeed, the pulvinar is likely to be an important
‘control site’ for attentional mechanisms more broadly63. Taken together, these data are
antithetical to the standard hypothesis, which assumes that automatic processing is mediated
by a subcortical pathway involving the pulvinar.
Does the subcortical pathway exist in primates?
Work on fear conditioning has shown that there are direct, subcortical projections to the
amygdala from the auditory thalamus (that is, from the medial geniculate nucleus (MGN)) in
rats. An analogous projection carrying visual information from the visual thalamic nucleus
(that is, the LGN) to the amygdala has not been documented in rodents or primates.
However, it is frequently assumed that a colliculus—pulvinar—amygdala pathway exists in
the case of vision. Here, we review relevant data from primates that question this
assumption.
Anatomical studies in monkeys have reported connections between the superior colliculus
and the pulvinar42,43, and between the pulvinar and the amygdala64,65. As described
above, the superficial superior colliculus projects to the inferior pulvinar — both of these
structures can be considered to be ‘visual’. Yet, the inferior pulvinar is extensively
interconnected with the visual cortex (consistent with visual functions) but not with the
amygdala. Instead, the projection to the amygdala originates in the medial pulvinar64,65
(although the strength of this connection may be relatively weak66), a nucleus that is
extensively interconnected with much of the cortex, as described above. Furthermore, like
other thalamic nuclei, the primate pulvinar is thought to have neither excitatory nor
inhibitory long-range intrinsic connections67. Considered together, these results do not
support the idea of a colliculus–pulvinar–amygdala visual pathway.
To summarize, there is no evidence for a direct or an indirect subcortical pathway conveying
visual information to the amygdala in monkeys. It is therefore unclear how findings from
auditory fear conditioning studies in rodents can be applied to visual processing of affective
stimuli in primates (see also BOX 2). For further discussion of the pathway from
intermediate and deep layers of the superior colliculus (which are multimodal in nature and
linked to eye movements) to the pulvinar see Supplementary information S1 (box).
Pessoa and Adolphs Page 8
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
The multiple-waves model
The standard hypothesis has influenced both basic and applied research and at first glance
has intuitive appeal. However, we have shown that, in its current form, the hypothesis is
problematic in multiple respects. We therefore suggest that a revision of the standard
hypothesis is in order.
Multiple visual pathways and coarse information processing
One of the primary motivations for the standard hypothesis is the perceived need for rapid
processing: fast but coarse visual processing is just what an organism needs in a dangerous
environment. We argue that visual pathways other than a colliculus–pulvinar–amygdala
pathway carry out this role (see also REFS 68,69).
Visual processing along the ventral processing stream, which is crucial for object
recognition, has historically been described to occur in a relatively hierarchical fashion.
However, important ‘short-cut’ connections link areas V1 to V4 (REF. 70), V2 to TEO70
and V4 to TE71 (FIG. 1b), providing the means for faster information transmission to the
inferotemporal cortex72. Direct connections between the LGN and extrastriate regions,
including V2 (REFS 73,74) and V4 (REF. 73), have also been reported. Indeed, combined
electrophysiology and fMRI studies in monkeys have shown robust visual activation in areas
V2 and V3 in animals with lesions of V1, demonstrating that routes bypassing V1 can be
sufficiently potent to drive extrastriate visual responses75 (see REF. 76 for evidence in
humans). A combined lesion and fMRI study in monkeys77 revealed widespread extrastriate
activation in the absence of V1 and demonstrated a role of LGN-dependent projections for
visual detection. V1-independent responses were observed in areas V2, V3, V4, MT (also
known as V5), the fundus of the superior temporal cortex (FST) and the LIP area. These
findings establish the importance of the LGN for at least some types of blindsight. Longer
range ‘short-cuts’ also exist, such as those that link regions in the ventral visual cortex with
the ventrolateral prefrontal cortex (including the OFC)78. It has been proposed79 that low-
spatial-frequency information may rapidly reach parietal and frontal cortices from the early
visual cortex80, thereby providing coarse information about the gist of a visual scene and
supporting object recognition81. It is thus possible that these initial ‘volleys of activation’
are less susceptible to manipulations of attention and awareness, especially given that they
may primarily convey information from the magnocellular system80,82. Furthermore,
computational models that assume a purely hierarchical structure of the visual system have
failed to provide a good fit to the existing latency data83, which is consistent with the
existence of bypass connections.
Thus, there are multiple parallel routes for visual information processing that lead to
substantial temporal dispersion of evoked responses and that enable ‘high-level’ regions to
respond with surprisingly short latencies84. Each processing stage adds approximately 10 ms
to the latency84. The ‘cost’ of using such bypassing stages may be that, at first, only
relatively coarse information is available about a visual item. This is consistent with a
coarse-to-fine processing strategy in which the more-global contents of a stimulus are
processed earlier than finer details85,86.
Based on the considerations above, we suggest that the initial processing of visual
information proceeds simultaneously along parallel channels, creating ‘multiple waves’ of
activation across the visual cortex and beyond87. In this manner, visual stimuli that have
affective and motivational significance can engage multiple brain sites — including the
amygdala, OFC, anterior insula and anterior cingulate cortex — that can direct processing
towards these behaviourally relevant items. Hence, rapid processing of affective information
Pessoa and Adolphs Page 9
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
is possible even in the absence of a specialized subcortical pathway (FIG. 1b) or a single
specific structure such as the amygdala40.
In light of our proposal, we suggest that affective blindsight involves some of the alternate
pathways described here. A recent EEG study of a patient with complete cortical blindness
used advanced source modelling to investigate the time course of information
processing88. Although all facial expressions, including neutral ones, evoked relatively
short-latency responses (70–120 ms) localized to the superior temporal sulcus, emotion-
specific responses that were localized to the anterior temporal cortex, and possibly the
amygdala, occurred considerably later (120 and 200 ms later, respectively). Although this
study suffers from the localization problems alluded to above, the findings are consistent
with the notion that affective significance is computed in parallel along several circuits (see
also REF.89 for a related proposal).
In light of the change of focus from a single, specialized subcortical pathway to a multiple-
pathway model, it is important to reconsider the roles of both the pulvinar and the amygdala
during processing of emotional visual stimuli.
Box 2 | Subcortical processing: audition in rats versus vision in primates
Historically, the standard hypothesis has derived a considerable portion of its motivation
from the organization of the auditory system in rodents. However, the auditory and visual
systems differ in important ways. The temporal precision of the auditory system is
substantially greater than that of the visual system. In contrast to vision, audition is
omnidirectional, such that information from all directions can be sampled (though at
relatively low spatial resolution). Furthermore, the functional anatomy of the auditory
system is very different from that of the visual system. Properties such as sensitivity to
sound frequency, duration, amplitude, pitch and binaural disparity are already observed at
subcortical levels. In fact, the primary auditory cortex (A1) seems to be involved in high-
level functions and is therefore not equivalent to ‘visual cortex transplanted into the
auditory modality’. Indeed, there are several subcortical stages below the level of the
auditory cortex, and it has been suggested113 that the role of the inferior colliculus in
auditory processing might be equivalent to that of primary visual cortex (V1) in vision
and that the A1 is more analogous to visual areas in the inferotemporal cortex than to V1.
These considerations suggest that a subcortical pathway for auditory input to the
amygdala (in rodents) would not actually be analogous to the purported subcortical visual
pathway (in monkeys). Accordingly, auditory connections from the medial geniculate
nucleus in the thalamus to the amygdala, although bypassing cortex, already convey
highly processed information, in contrast to the suggested monkey (visual) counterpart.
The role of the pulvinar in processing of emotional visual stimuli
Connections between the pulvinar and amygdala have been reported64,65, suggesting that the
pulvinar may have a role in emotion processing. As mentioned above, it has been proposed
that the pulvinar is involved in determining the behavioural relevance of a stimulus,
directing attention to a stimulus and determining awareness of a stimulus. Based on the data
described above, we propose that the pulvinar helps to coordinate and/or regulate the flow of
multimodal information via a series of thalamocortical loops (FIG. 3). This proposal takes
into account that most of the input to the pulvinar comes from the cortex.
In the context of emotion processing, the most relevant nucleus of the pulvinar is probably
the medial nucleus, given that it connects not only with the amygdala but also with a larger
array of other brain regions. We therefore suggest that it may be involved in more general
Pessoa and Adolphs Page 10
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
functions that impact emotion processing, such as determining the behavioural relevance
and/or value of a stimulus. For example, the medial nucleus is connected with parietal
regions that are involved in attention. It is also connected with the OFC and cingulate cortex,
which are important for computing an object’s biological value. Furthermore, it is connected
with the insula, a region that has a role in emotional feelings. These connections are all
bidirectional (except the connection with the amygdala), providing opportunities for
modulating and regulating information flow. According to our proposal, the importance of
the pulvinar in emotion is not due to its status as a subcortical ‘labelled line’ conveying
emotional information to the amygdala, but due instead to its pattern of connectivity with
subcortical and cortical sites that have a role in determining the biological significance of a
stimulus.
Studies by Ward and colleagues have investigated the impact of pulvinar lesions on
processing of affective visual information in humans. A complete unilateral loss of the
pulvinar led to a severe deficit in a patient’s ability to recognize fearful expressions shown
in the contralesional visual field90. Within the framework suggested here, when weak and/
or brief visual stimuli have biological significance, cortico–pulvino–cortical circuits
coordinate and amplify signals in a manner that enhances their behavioural impact. This
framework is consistent with the impairment in recognizing fear in patients with pulvinar
lesions and also with their impairment in recognizing anger (and possibly happiness)90.
Notably, the essential pulvinar damage was found in the medial pulvinar, the region that
projects to the amygdala in monkeys. The proposed framework is also consistent with a
study that reported that viewing complex unpleasant images impaired performance in a
subsequent simple (neutral) visual task in control subjects, but not in a patient with pulvinar
damage91 — according to our framework, the unpleasant stimulus did not garner additional
resources in the patient (which would have interfered with performance, as it did in the
controls).
Pulvinar involvement in the processing of affective information does not seem to reflect
emotion per se, however. In an fMRI study in humans58, a simple contrast between affective
and neutral conditions did not reveal different responses in the pulvinar. Instead, there was a
significant relationship between the magnitude of evoked responses in the pulvinar and the
probability of correctly detecting a target on a trial-by-trial basis during the affective
condition but not during the neutral condition (FIG. 4a). These results reveal an emotion-by-
visibility interaction that may characterize the role of the pulvinar more generally. In other
words, we suggest that the pulvinar amplifies responses to stimuli of potential value to the
animal (such as one that signals the possibility of shock in the experiment) (FIG. 4b).
The role of the amygdala in processing affective visual stimuli
What part is left for the amygdala to play in the processing of affective visual stimuli? Its
connectivity pattern provides some clues. The predominant source of visual input to the
amygdala, specifically the basolateral nucleus, comes from higher-order visual association
cortices in the anterior temporal lobe29. This suggests that the amygdala is a convergence
zone for highly-processed sensory information that is relevant to object processing. In
addition, there are loops between the visual cortex and the lateral and basal nuclei of the
amygdala, and this feedback is thought to modulate visual processing92. Further integrative
functions of the amygdala stem from its extensive connections with much of the cortex. In
addition to its well-recognized connections with medial and orbital territories of the
prefrontal cortex, the amygdala is also connected to the lateral prefrontal cortex, albeit in a
weaker manner93. Importantly, the architecture of the prefrontal cortex (PFC) is such that,
on average, inputs from the amygdala reach approximately 90% of the PFC after a single
connection within the frontal cortex94. Furthermore, the amygdala seems to be part of a
Pessoa and Adolphs Page 11
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
‘core brain circuit’ (REF. 95) that is topologically central in terms of global brain
connectivity and whose functions could include aggregation and distribution of information.
In light of these considerations, we propose that the amygdala’s contribution to the
processing of affective visual information arises not from a subcortical source of visual
input, but from its broad connectivity with the cortex and other subcortical structures. Given
this connectivity, the impact of the amygdala on behaviour can be mediated through many
routes, for example, via both the visual cortex and prefrontal cortex. This suggestion is
consistent with findings of a study that combined the attentional blink task with fear
conditioning96. For emotion-laden stimuli, trial-by-trial fluctuations in evoked responses in
the amygdala predicted whether or not a target was detected. Furthermore, this impact of the
amygdala on behaviour was mediated by both the visual cortex and prefrontal cortex (as
suggested by statistical path analysis), consistent with the idea that during the
processing of affectively significant items, the amygdala enhances sensory processing
through both direct (amygdala–visual cortex) and indirect (amygdala–prefrontal cortex–
visual cortex) paths (FIG. 4c).
For reasons of brevity we have discussed the amygdala as a single entity, but it should be
noted that the amygdala is in fact a complex structure comprised of more than a dozen
nuclei. In particular, the central nucleus has extensive descending connections to the
hypothalamus and other brainstem nuclei that regulate autonomic and endocrine responses
and, in this manner, contributes to several aspects of emotional expression and mobilization
of bodily resources. Among others, this is an important distinction between the roles of the
amygdala and pulvinar during processing of affective visual stimuli.
Conclusions
The evidence we have reviewed here suggests that the idea of a subcortical pathway that is
specialized for the processing of emotional stimuli should be revised. Our reinterpretation
has important implications for the conceptualization of the amygdala’s function in the
processing of emotional visual information. We suggest two revised roles for the amygdala.
First, the amygdala has a mostly modulatory role in a wide array of networks. The precise
functional importance of the amygdala in these networks remains to be investigated, but it is
unlikely that it will map specifically onto emotion. Instead, we think that it corresponds to
broader and more abstract dimensions of information processing, including processing of
salience, significance, ambiguity, unpredictability21,9799 and other aspects of ‘biological
value’. More broadly, we argue that the amygdala has a key role in solving the following
problem100: how can a limited-capacity information processing system that receives a
constant stream of diverse inputs selectively process those inputs that are the most relevant
to the goals of the animal101?
Thus, the amygdala serves to allocate processing resources to stimuli, at least in part by
modulating (through its connectivity) the anatomical components that are required to
prioritize particular features of information processing in a given situation. Such a role
would come into play not only for affectively significant stimuli but also for other stimuli.
Notably, the amygdala may not be unique in this respect as there are other, largely parallel,
networks with architectures that do not include the amygdala but that also enable diverse
functions — notably the network subserved by the connections between the cortex and the
pulvinar. We think that our proposal is consistent with the majority of findings and can
accommodate several views of amygdala function16,102,103, with the difference that it
provides a broader and more flexible perspective.
Pessoa and Adolphs Page 12
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
The second aspect of our proposed revision is to stress the speed and temporal dispersion of
cortical processing, which render moot the assumed need for a fast subcortical route. Many
visual properties can be processed very rapidly by the initial wave of cortical response,
which suggests that there is ample time for substantial feedback, even within the cortex.
Consequently, understanding the flow of visual information within the cortex should help us
to understand how affective stimuli are processed. Ultimately, the fate of a biologically-
relevant stimulus should not be understood in terms of a ‘low road’ versus a ‘high road’, but
in terms of the ‘multiple roads’ that lead to the expression of observed behaviours.
There is an enormous literature implicating the amygdala in affective dysfunction in nearly
every psychiatric illness, most notably in mood disorders. The revised view in this
Perspective suggests that rather than focusing on neurons within the amygdala, we should
focus on connections within the cortex and between the cortex and subcortical structures
such as the amygdala. This may not come as a big surprise to some readers as, in the main, it
simply reflects the idea that the substrate of brain function is not so much to be found within
neurons as within networks.
Supplementary Material
Refer to Web version on PubMed Central for supplementary material.
Acknowledgments
The authors thank A. Anticevic, L. Oliveira, M. Pereira, R. Todd, and S. Wang for feedback on the manuscript.
They also thank L. Barrett and two anonymous reviewers for comments. The authors’ research is supported by
grants from the National Institute of Mental Health (R01 MH071589 to L.P. and R01 MH080721 to R.A.), the
National Institute of Neurological Disorders and Stroke (P01 NS019632 to R.A.), the Simons Foundation Autism
Research Initiative and the National Science Foundation (NSF 0926,544).
Glossary
Attentional
blink A phenomenon that occurs in experiments in which a rapid stream of
visual items is presented to an observer whose task is to detect two
targets within the stream. When the two targets are separated in time
by a brief interval (for example, 200–500 ms), the successful
detection of the first target impairs detection of the second one (as if
the participant blinked) owing to limited processing capacity.
Backward
masking A phenomenon that occurs in experimental paradigms in which a
target visual stimulus is followed by another salient visual stimulus
that ‘masks’ the perception of the target stimulus, making its
detection or recognition difficult or impossible. Visual masking is
commonly used to manipulate visual awareness.
Blindsight The ability, in humans or monkeys, to respond to visual stimuli
without consciously perceiving them — a situation that may ensue
following a lesion to the primary visual cortex.
Continuous flash
suppression A technique in which a fixed image shown to one eye is suppressed
by a stream of rapidly changing images flashed to the other eye. The
technique is used to manipulate visual awareness.
Labelled line A processing architecture in which a separate pathway conveys
information that is specific to a class of sensory stimuli owing to, for
Pessoa and Adolphs Page 13
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
example, receptor specificity (for example, pain and touch conveyed
by particular somatosensory channels).
Magnocellular
system A visual pathway from the retina to the cortex that conveys relatively
fast, transient and wavelength-insensitive information.
Path analysis A statistical method to investigate the relationship between multiple
variables.
Source
modelling A set of techniques that attempt to estimate the neural ‘sources’ of the
electrical or magnetic signals that are measured at external sensors
(for example, at the scalp in the case of electroencephalography).
Visual search An experimental paradigm in which subjects are asked to indicate the
presence or absence of a ‘target’ item (for example, a fearful face)
among an array of ‘distractor’ items (for example, neutral faces).
References
1. Tamietto M, de Gelder B. Neural bases of the non-conscious perception of emotional signals.
Nature Rev. Neurosci 2010;11:697–709. [PubMed: 20811475]
2. Ohman A, Mineka S. Fears, phobias, and preparedness: toward an evolved module of fear and fear
learning. Psychol. Rev 2001;108:483–522. [PubMed: 11488376]
3. Lonsdorf TB, et al. Genetic gating of human fear learning and extinction: possible implications for
gene-environment interaction in anxiety disorder. Psychol. Sci 2009;20:198–206. [PubMed:
19175757]
4. LeDoux, JE. The Emotional Brain. New York: Simon & Schuster; 1996.
5. Dolan RJ, Vuilleumier P. Amygdala automaticity in emotional processing. Ann. NY Acad. Sci
2003;985:348–355. [PubMed: 12724170]
6. Ohman A, Lundqvist D, Esteves F. The face in the crowd revisited: a threat advantage with
schematic stimuli. J. Pers. Soc. Psychol 2001;80:381–396. [PubMed: 11300573]
7. Yang E, Zald DH, Blake R. Fearful expressions gain preferential access to awareness during
continuous flash suppression. Emotion 2007;7:882–886. [PubMed: 18039058]
8. Morris JS, Ohman A, Dolan RJ. Conscious and unconscious emotional learning in the human
amygdala. Nature 1998;393:467–470. [PubMed: 9624001]
9. Whalen PJ, et al. Masked presentations of emotional facial expressions modulate amygdala activity
without explicit knowledge. J. Neurosci 1998;18:411–418. [PubMed: 9412517]
10. Morris JS, DeGelder B, Weiskrantz L, Dolan RJ. Differential extrageniculostriate and amygdala
responses to presentation of emotional faces in a cortically blind field. Brain 2001;124:1241–1252.
[PubMed: 11353739]
11. Pegna AJ, Khateb A, Lazeyras F, Seghier ML. Discriminating emotional faces without primary
visual cortices involves the right amygdala. Nature Neurosci 2005;8:24–25. [PubMed: 15592466]
12. Morris JS, Ohman A, Dolan RJ. A subcortical pathway to the right amygdala mediating “unseen”
fear. Proc. Natl Acad. Sci. USA 1999;96:1680–1685. [PubMed: 9990084]
13. Bar M, Neta M. Humans prefer curved visual objects. Psychol. Sci 2006;17:645–648. [PubMed:
16913943]
14. Vuilleumier P, Armony JL, Driver J, Dolan RJ. Distinct spatial frequency sensitivities for
processing faces and emotional expressions. Nature Neurosci 2003;6:624–631. [PubMed:
12740580]
15. Davis M, Whalen PJ. The amygdala: vigilance and emotion. Mol. Psychiatry 2001;6:13–34.
[PubMed: 11244481]
16. Whalen, PJ.; Phelps, EA., editors. The Human Amygdala. New York: Guilford Press; 2009.
17. Lewis MD. Bridging emotion theory and neurobiology through dynamic systems modeling. Behav.
Brain Sci 2005;28:169–245. [PubMed: 16201458]
Pessoa and Adolphs Page 14
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
18. Duncan S, Barrett LF. Affect is a form of cognition: a neurobiological analysis. Cogn. Emot
2007;21:1184–1211. [PubMed: 18509504]
19. Pessoa L. On the relationship between emotion and cognition. Nature Rev. Neurosci 2008;9:148–
158. [PubMed: 18209732]
20. Pessoa L. To what extent are emotional visual stimuli processed without attention and awareness?
Curr. Opin. Neurobiol 2005;15:188–196. [PubMed: 15831401]
21. Adolphs R. Fear, faces, and the human amygdala. Curr. Opin. Neurobiol 2008;18:166–172.
[PubMed: 18655833]
22. Pizzagalli D, Regard M, Lehmann D. Rapid emotional face processing in the human right and left
brain hemispheres: an ERP study. Neuroreport 1999;10:2691–2698. [PubMed: 10511425]
23. Halgren E, Raij T, Marinkovic K, Jousmaki V, Hari R. Cognitive response profile of the human
fusiform face area as determined by MEG. Cereb. Cortex 2000;10:69–81. [PubMed: 10639397]
24. Eger E, Jedynak A, Iwaki T, Skrandies W. Rapid extraction of emotional expression: evidence
from evoked potential fields during brief presentation of face stimuli. Neuropsychologia
2003;41:808–817. [PubMed: 12631531]
25. Japee S, Crocker L, Carver F, Pessoa L, Ungerleider LG. Individual differences in valence
modulation of face-selective M170 response. Emotion 2009;9:59–69. [PubMed: 19186917]
26. Krolak-Salmon P, Henaff MA, Vighetto A, Bertrand O, Mauguiere F. Early amygdala reaction to
fear spreading in occipital, temporal, and frontal cortex: a depth electrode ERP study in human.
Neuron 2004;42:665–676. [PubMed: 15157426]
27. Greene MR, Oliva A. The briefest of glances: the time course of natural scene understanding.
Psychol. Sci 2009;20:464–472. [PubMed: 19399976]
28. Kawasaki H, et al. Single-neuron responses to emotional visual stimuli recorded in human ventral
prefrontal cortex. Nature Neurosci 2001;4:15–16. [PubMed: 11135639]
29. Amaral, DG.; Price, JL.; Pitkanen, A.; Carmichael, ST. The Amygdala: Neurobiological Aspects of
Emotion, Memory, and Mental Dysfunction. Aggleton, J., editor. New York: Wiley-Liss; 1992. p.
1-66.
30. Gothard KM, Battaglia FP, Erickson CA, Spitler KM, Amaral DG. Neural responses to facial
expression and face identity in the monkey amygdala. J. Neurophysiol 2007;97:1671–1683.
[PubMed: 17093126]
31. Rolls, ET. Emotion Explained. Oxford: Oxford Univ. Press; 2005.
32. Kreiman G, Koch C, Fried I. Category-specific visual responses of single neurons in the human
medial temporal lobe. Nature Neurosci 2000;3:946–953. [PubMed: 10966627]
33. Mormann F, et al. Latency and selectivity of single neurons indicate hierarchical processing in the
human medial temporal lobe. J. Neurosci 2008;28:8865–8872. [PubMed: 18768680]
34. Smith ML, Cottrell GW, Gosselin F, Schyns PG. Transmitting and decoding facial expressions.
Psychol. Sci 2005;16:184–189. [PubMed: 15733197]
35. Smith FW, Schyns PG. Smile through your fear and sadness: transmitting and identifying facial
expression signals over a range of viewing distances. Psychol. Sci 2009;20:1202–1208. [PubMed:
19694983]
36. Adolphs R, et al. A mechanism for impaired fear recognition after amygdala damage. Nature
2005;433:68–72. [PubMed: 15635411]
37. Vuilleumier P. How brains beware: neural mechanisms of emotional attention. Trends Cogn. Sci
2005;9:585–594. [PubMed: 16289871]
38. Pessoa L, Oliveira L, Pereira MG. Attention and emotion. Scholarpedia 2010;5:6314.
39. Pessoa, L.; Oliveira, L.; Pereira, MG. Handbook of Human Affective Neuroscience. Armony, JL.;
Vuilleumier, P., editors. Cambridge, UK: Cambridge Univ. Press; 2011.
40. Tsuchiya N, Moradi F, Felsen C, Yamazaki M, Adolphs R. Intact rapid detection of fearful faces in
the absence of the amygdala. Nature Neurosci 2009;12:1224–1225. [PubMed: 19718036]
41. Piech RM, et al. Fear-enhanced visual search persists after amygdala lesions. Neuropsychologia
2010;48:3430–3435. [PubMed: 20637217]
42. Grieve KL, Acuna C, Cudeiro J. The primate pulvinar nuclei: vision and action. Trends Neurosci
2000;23:35–39. [PubMed: 10631787]
Pessoa and Adolphs Page 15
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
43. Stepniewska, I. The Primate Visual System. Kaas, J.; Collins, CE., editors. Boca Raton, Florida:
CRC Press; 2004. p. 53-80.
44. Bender DB. Visual activation of neurons in the primate pulvinar depends on cortex but not
colliculus. Brain Res 1983;279:258–261. [PubMed: 6640346]
45. Robinson, DL.; Cowie, RJ. The Thalamus. McCormick, D., editor. New York: Elsevier; 1997. p.
53-92.
46. de Gelder B, et al. Standing up for the body. Recent progress in uncovering the networks involved
in the perception of bodies and bodily expressions. Neurosci. Biobehav. Rev 2010;34:513–527.
[PubMed: 19857515]
47. Guillery RW. Anatomical evidence concerning the role of the thalamus in corticocortical
communication: a brief review. J. Anat 1995;187:583–592. [PubMed: 8586557]
48. Sherman SM, Guillery RW. Functional organization of thalamocortical relays. J. Neurophysiol
1996;76:1367–1395. [PubMed: 8890259]
49. Sherman SM. Thalamus. Scholarpedia 2006;1:1583.
50. Ungerleider LG, Christensen CA. Pulvinar lesions in monkeys produce abnormal scanning of a
complex visual array. Neuropsychologia 1979;17:493–501. [PubMed: 117393]
51. Zihl J, von Cramon D. The contribution of the ‘second’ visual system to directed visual attention in
man. Brain 1979;102:835–856. [PubMed: 509197]
52. Benevento LA, Port JD. Single neurons with both form/color differential responses and saccade-
related responses in the nonretinotopic pulvinar of the behaving macaque monkey. Vis. Neurosci
1995;12:523–544. [PubMed: 7654609]
53. Petersen SE, Robinson DL, Keys W. Pulvinar nuclei of the behaving rhesus monkey: visual
responses and their modulation. J. Neurophysiol 1985;54:867–886. [PubMed: 4067625]
54. Desimone R, Wessinger M, Thomas L, Schneider W. Attentional control of visual perception:
cortical and subcortical mechanisms. Cold Spring Harb. Symp. Quant. Biol 1990;55:963–971.
[PubMed: 2132873]
55. Karnath HO, Himmelbach M, Rorden C. The subcortical anatomy of human spatial neglect:
putamen, caudate nucleus and pulvinar. Brain 2002;125:350–360. [PubMed: 11844735]
56. Ward R, Danziger S, Owen V, Rafal R. Deficits in spatial coding and feature binding following
damage to spatiotopic maps in the human pulvinar. Nature Neurosci 2002;5:99–100. [PubMed:
11780145]
57. Wilke M, Mueller KM, Leopold DA. Neural activity in the visual thalamus reflects perceptual
suppression. Proc. Natl Acad. Sci. USA 2009;106:9465–9470. [PubMed: 19458249]
58. Padmala S, Lim S-L, Pessoa L. Pulvinar and affective significance: responses track moment-to-
moment visibility. Front. Hum. Neurosci 2010;4:1–9. [PubMed: 20204154]
59. Pessoa L, Ungerleider LG. Neural correlates of change detection and change blindness in a
working memory task. Cereb. Cortex 2004;14:511–520. [PubMed: 15054067]
60. Shipp S. The functional logic of cortico-pulvinar connections. Phil. Trans. R. Soc. Lond. B
2003;358:1605–1624. [PubMed: 14561322]
61. Lyon DC, Nassi JJ, Callaway EM. A disynaptic relay from superior colliculus to dorsal stream
visual cortex in macaque monkey. Neuron 2010;65:270–279. [PubMed: 20152132]
62. Berman RA, Wurtz RH. Functional identification of a pulvinar path from superior colliculus to
cortical area MT. J. Neurosci 2010;30:6342–6354. [PubMed: 20445060]
63. Shipp S. The brain circuitry of attention. Trends Cogn. Sci 2004;8:223–230. [PubMed: 15120681]
64. Jones EG, Burton H. A projection from the medial pulvinar to the amygdala in primates. Brain Res
1976;104:142–147. [PubMed: 813820]
65. Romanski LM, Giguere M, Bates JF, Goldman-Rakic PS. Topographic organization of medial
pulvinar connections with the prefrontal cortex in the rhesus monkey. J. Comp. Neurol
1997;379:313–332. [PubMed: 9067827]
66. Aggleton JP, Burton MJ, Passingham RE. Cortical and subcortical afferents to the amygdala of the
rhesus monkey (Macaca mulatta). Brain Res 1980;190:347–368. [PubMed: 6768425]
67. Sherman, SM.; Guillery, RW. Exploring the Thalamus. San Diego: Academic Press; 2001.
Pessoa and Adolphs Page 16
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
68. Cowey A. The 30th Sir Frederick Bartlett lecture. Fact, artefact, and myth about blindsight. Q. J.
Exp. Psychol. A 2004;57:577–609. [PubMed: 15204125]
69. Chen CM, et al. Functional anatomy and interaction of fast and slow visual pathways in macaque
monkeys. Cereb. Cortex 2007;17:1561–1569. [PubMed: 16950866]
70. Nakamura H, Gattass R, Desimone R, Ungerleider LG. The modular organization of projections
from areas V1 and V2 to areas V4 and TEO in macaques. J. Neurosci 1993;13:3681–3691.
[PubMed: 7690064]
71. Felleman DJ, Van Essen DC. Retinotopic organization in human visual cortex and the spatial
precision of functional MRI. Cereb. Cortex 1991;1:1–47. [PubMed: 1822724]
72. Lamme VA, Roelfsema PR. The distinct modes of vision offered by feedforward and recurrent
processing. Trends Neurosci 2000;23:571–579. [PubMed: 11074267]
73. Yukie M, Iwai E. Direct projection from the dorsal lateral geniculate nucleus to the prestriate
cortex in macaque monkeys. J. Comp. Neurol 1981;201:81–97. [PubMed: 7276252]
74. Bullier J, Kennedy H. Projection of the lateral geniculate nucleus onto cortical area V2 in the
macaque monkey. Exp. Brain Res 1983;53:168–172. [PubMed: 6201379]
75. Schmid MC, Panagiotaropoulos T, Augath MA, Logothetis NK, Smirnakis SM. Visually driven
activation in macaque areas V2 and V3 without input from the primary visual cortex. PLoS ONE
2009;4:e5527. [PubMed: 19436733]
76. Boyer JL, Harrison S, Ro T. Unconscious processing of orientation and color without primary
visual cortex. Proc. Natl Acad. Sci. USA 2005;102:16875–16879. [PubMed: 16263934]
77. Schmid MC, et al. Blindsight depends on the lateral geniculate nucleus. Nature 2010;466:373–377.
[PubMed: 20574422]
78. Rempel-Clower NL, Barbas H. The laminar pattern of connections between prefrontal and anterior
temporal cortices in the Rhesus monkey is related to cortical structure and function. Cereb. Cortex
2000;10:851–865. [PubMed: 10982746]
79. Bar M. A cortical mechanism for triggering top-down facilitation in visual object recognition. J.
Cogn. Neurosci 2003;15:600–609. [PubMed: 12803970]
80. Bullier J. Integrated model of visual processing. Brain Res. Brain Res. Rev 2001;36:96–107.
[PubMed: 11690606]
81. Kveraga K, Boshyan J, Bar M. Magnocellular projections as the trigger of top-down facilitation in
recognition. J. Neurosci 2007;27:13232–13240. [PubMed: 18045917]
82. Barrett LF, Bar M. See it with feeling: affective predictions during object perception. Phil. Trans.
R. Soc. Lond. B 2009;364:1325–1334. [PubMed: 19528014]
83. Capalbo M, Postma E, Goebel R. Combining structural connectivity and response latencies to
model the structure of the visual system. PLoS Comput. Biol 2008;4:e1000159. [PubMed:
18769707]
84. Nowak, LG.; Bullier, J. Cerebral Cortex: Extrastriate Cortex in Primate. Rockland, K.; Kass, J.;
Peters, A., editors. New York: Plenum; 1997. p. 205-241.
85. Sugase Y, Yamane S, Ueno S, Kawano K. Global and fine information coded by single neurons in
the temporal visual cortex. Nature 1999;400:142–147.
86. Sripati AP, Olson CR. Representing the forest before the trees: a global advantage effect in
monkey inferotemporal cortex. J. Neurosci 2009;29:7788–7796. [PubMed: 19535590]
87. Rudrauf D, et al. Rapid interactions between the ventral visual stream and emotion-related
structures rely on a two-pathway architecture. J. Neurosci 2008;28:2793–2803. [PubMed:
18337409]
88. Andino SL, Menendez RG, Khateb A, Landis T, Pegna AJ. Electrophysiological correlates of
affective blindsight. Neuroimage 2009;44:581–589. [PubMed: 18835454]
89. Rotshtein P, et al. Amygdala damage affects event-related potentials for fearful faces at specific
time windows. Hum. Brain Mapp 2010;31:1089–1105. [PubMed: 20017134]
90. Ward R, Calder AJ, Parker M, Arend I. Emotion recognition following human pulvinar damage.
Neuropsychologia 2007;45:1973–1978. [PubMed: 17250857]
91. Ward R, Danziger S, Bamford S. Response to visual threat following damage to the pulvinar. Curr.
Biol 2005;15:571–573. [PubMed: 15797028]
Pessoa and Adolphs Page 17
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
92. Vuilleumier P, Richardson MP, Armony JL, Driver J, Dolan RJ. Distant influences of amygdala
lesion on visual cortical activation during emotional face processing. Nature Neurosci
2004;7:1271–1278. [PubMed: 15494727]
93. Ghashghaei HT, Hilgetag CC, Barbas H. Sequence of information processing for emotions based
on the anatomic dialogue between prefrontal cortex and amygdala. Neuroimage 2007;34:905–923.
[PubMed: 17126037]
94. Averbeck BB, Seo M. The statistical neuroanatomy of frontal networks in the macaque. PLoS
Comput. Biol 2008;4:e1000050. [PubMed: 18389057]
95. Modha DS, Singh R. Network architecture of the long-distance pathways in the macaque brain.
Proc. Natl Acad. Sci. USA 2010;107:13485–13490. [PubMed: 20628011]
96. Lim SL, Padmala S, Pessoa L. Segregating the significant from the mundane on a moment-to-
moment basis via direct and indirect amygdala contributions. Proc. Natl Acad. Sci. USA
2009;106:16841–16846. [PubMed: 19805383]
97. Whalen PJ. Fear, vigilance, and ambiguity: initial neuroimaging studies of the human amygdala.
Curr. Dir. Psychol.Sci 1998;7:177–188.
98. Sander D, Grafman J, Zalla T. The human amygdala: an evolved system for relevance detection.
Rev. Neurosci 2003;14:303–316. [PubMed: 14640318]
99. Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF. Neural systems responding to degrees of
uncertainty in human decision-making. Science 2005;310:1680–1683. [PubMed: 16339445]
100. Pessoa L. Emotion and cognition and the amygdala: from “what is it?” to “what’s to be done?”.
Neuropsychologia 2010;48:3416–3429. [PubMed: 20619280]
101. Grossberg S, Levine DS. Neural dynamics of attentionally modulated Pavlovian conditioning:
blocking, interstimulus interval, and secondary reinforcement. Appl. Opt 1987;26:5015–5030.
[PubMed: 20523481]
102. Aggleton, JP., editor. The Amygdala: Neurobiological Aspects of Emotion, Memory, and Mental
Dysfunction. New York: John Wiley & Sons; 1992.
103. Aggleton, J., editor. The Amygdala: A Functional Analysis. Oxford: Oxford Univ. Press; 2000.
104. Ouellette BG, Casanova C. Overlapping visual response latency distributions in visual cortices
and LP-pulvinar complex of the cat. Exp. Brain Res 2006;175:332–341. [PubMed: 16816944]
105. Schmolesky MT, et al. Signal timing across the macaque visual system. J. Neurophysiol
1998;79:3272–3278. [PubMed: 9636126]
106. Boehnke SE, Munoz DP. On the importance of the transient visual response in the superior
colliculus. Curr. Opin. Neurobiol 2008;18:544–551. [PubMed: 19059772]
107. Leonard CM, Rolls ET, Wilson FA, Baylis GC. Neurons in the amygdala of the monkey with
responses selective for faces. Behav. Brain Res 1985;15:159–176. [PubMed: 3994832]
108. Kuraoka K, Nakamura K. Responses of single neurons in monkey amygdala to facial and vocal
emotions. J. Neurophysiol 2007;97:1379–1387. [PubMed: 17182913]
109. Nakamura K, Mikami A, Kubota K. Activity of single neurons in the monkey amygdala during
performance of a visual discrimination task. J. Neurophysiol 1992;67:1447–1463. [PubMed:
1629757]
110. Oya H, Kawasaki H, Howard MA, Adolphs R. Electrophysiological responses in the human
amygdala discriminate emotion categories of complex visual stimuli. J. Neurosci 2002;22:9502–
9512. [PubMed: 12417674]
111. Tovee MJ, Rolls ET. Information encoding in short firing rate epochs by single neurons in the
primate temporal visual cortex. Vis. Cogn 1995;2:35–58.
112. Yoshor D, Bosking WH, Ghose GM, Maunsell JH. Receptive fields in human visual cortex
mapped with surface electrodes. Cereb. Cortex 2007;17:2293–2302. [PubMed: 17172632]
113. King AJ, Nelken I. Unraveling the principles of auditory cortical processing: can we learn from
the visual system? Nature Neurosci 2009;12:698–701. [PubMed: 19471268]
Pessoa and Adolphs Page 18
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Figure 1. Visual pathways
a | A traditional flowchart of visual processing typically emphasizes the LGN–V1–V2–V4–
TEO–TE pathway, although the scheme is not strictly hierarchical. The amygdala, in
particular, is a recipient of visual signals from the anterior visual cortex. According to the
‘standard hypothesis’, a subcortical pathway involving the superior colliculus and the
pulvinar nucleus of the thalamus provides fast and automatic access to the amygdala. b | An
alternative view of the flow of visual signals includes multiple pathways, including both
alternative routes (for example, LGN to MT) and shortcuts (for example, V2 to TEO). Only
some of these are shown. The flow of visual information may be more appropriately viewed
in terms of ‘multiple waves’ of activation that initiate and refine cell responses at a given
processing ‘stage’. For simplicity, feedback pathways, which are known to be quite
extensive, have been omitted. The existence of such feedback pathways dictates, however,
that a complex ebb-and-flow of activation sculpts the neuronal profile of activation
throughout the visual cortex, and likewise the amygdala responses. Some of the connections
between the pulvinar and visual cortex, and between the pulvinar and ‘associational’ areas,
are also indicated. The line in the pulvinar is intended to schematically separate the medial
pulvinar (to the right of the line) from the rest of the structure. FEF, frontal eye field; LGN,
lateral geniculate nucleus; MT, medial temporal area (also known as V5); OFC,
Pessoa and Adolphs Page 19
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
orbitofrontal cortex; SC, superior colliculus; TE, inferior temporal area TE; TEO, inferior
temporal area TEO; V, visual cortex; VLPFC, ventrolateral prefrontal cortex.
Pessoa and Adolphs Page 20
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Figure 2. Intact non-conscious processing of fearful faces in the absence of the amygdala
In a subject with complete amygdala lesions (subject S.M.), fearful faces broke into
consciousness during continuous flash suppression with latencies similar to those of control
subjects. a | Experimental stimulus: fearful or happy faces were shown to the non-dominant
eye while a flashing Mondrian pattern was shown to the dominant eye. This technique is
called continuous flash suppression as it suppresses the visibility of the stimulus presented to
the non-dominant eye. The contrast of the Mondrian pattern was gradually decreased while
that of the face was increased until subjects could detect the face and indicate it with a
button press to establish reaction time. b | Plots of reaction times (RT) in the task. In the case
of S.M., fearful faces broke interocular suppression faster than happy faces (shown by the
red bar) and to the same degree as 7 demographically matched healthy controls (shown by
blue bars; the mean and standard deviation are also shown). Figure is reproduced, with
permission, from REF. 40 © (2009) Macmillan Publishers Ltd. All rights reserved.
Pessoa and Adolphs Page 21
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Figure 3. Schematic layout of the pulvinar
Some traditional characterizations of the pulvinar emphasize the inferior (Inf), lateral (Lat)
and medial (Med) nuclei. Most pulvinar nuclei (including other nuclei and sub-nuclei that
are not shown here) are involved in thalamo–cortical loops that target different cortical
territories (shown in blue)42,60. The inferior nucleus is reciprocally connected to striate and
extrastriate cortices, the lateral nucleus is connected to association cortices in temporal and
parietal lobes (although it is also interconnected with the extrastriate cortex) and the medial
nucleus is connected to the higher-order association cortex in parietal, frontal, orbital (not
shown), cingulate and insular regions (the insula is not shown), in addition to the amygdala.
Thus, the medial nucleus, which is of great interest in the present context, is not only
connected with the amygdala but is also part of multiple thalamo–cortical loops (note,
however, that the connection to the amygdala does not seem to be bidirectional). The
superior colliculus is a layered structure whose superficial layers are visual in nature and
project to the inferior nucleus. Its intermediate and deeper layers are multimodal and
involved in motor preparation, including for eye movements, and project to the medial
nucleus. A ventrolateral to dorsomedial axis that is helpful in understanding the organization
of pulvinar nuclei and potential ‘ventral’ and ‘dorsal’ domains is shown by a dotted line (see
also REF. 42 for a related scheme). Figure is modified, with permission, from REF. 43 ©
(2004) CRC Press. IT, inferior temporal cortex; MT, medial temporal area (also known as
V5).
Pessoa and Adolphs Page 22
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Figure 4. Pulvinar and amygdala during processing of affective stimuli
a | Logistic regression analysis of evoked responses in the left pulvinar as a function of
affective significance for a sample individual during an attentional blink task58. The slope of
the logistic fit indicates the strength of the predictive effect. For clarity, only binned data for
the conditioned stimulus (CS+) condition are included (shown by orange dots). The grey
line shows the fit for these data, and the blue line shows the fit for data from the neutral
stimulus (CS) condition. The inset shows mean logistic slopes across individuals, revealing
that a relationship was detected for the affective (CS+) but not the neutral (CS) condition.
b | The medial pulvinar is proposed to amplify evoked responses of behaviourally-relevant
stimuli via circuits involving the cingulate cortex, orbitofrontal cortex (OFC) and amygdala,
all regions important for the valuation of an incoming stimulus. c | Valuation signals in the
amygdala affect behaviour by impacting responses across the brain. During an attentional
blink task using affective stimuli, a response in the amygdala to a stimulus predicted that the
stimulus would be detected96. Statistical path analysis revealed that this effect is mediated
through projections from the amygdala to the visual cortex, as well as through projections
involving the prefrontal cortex (PFC). fMRI, functional MRI. Data in part a from REF. 58.
Pessoa and Adolphs Page 23
Nat Rev Neurosci. Author manuscript; available in PMC 2011 January 24.
NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
... It is a research area involving intense, unrelenting and unresolved academic debates. Research in the unconscious was contentious since its "first steps" (Ebbinghaus,1908;Field, Aveling & Laird, 1922;Miller, 1942;Kahn, 1943;Fechner, 1948;Goldiamond, 1958; for an overview see Tsikandilakis, Bali, Derrfuss & Chapman, 2019) to itsso to speak -"mid-life crisis" (Burnham, 1967;Dixon, 1971;1981;Goodkin & Phillips, 1980;Merikle & Cheesman, 1987;Frosh, 1989;Bornstein, 1989; for an overview see Tsikandilakis, Bali, Yu, Madan, Derrfuss, Chapman & Groeger, 2021) and has grown methodologically contentious, now, more than ever, among contemporary psychologists (see Bar & Biederman, 1998;Erdelyi, 2005;Pessoa & Adolphs, 2010;Elgendi, Kumar, Barbic, Howard, Abbott & Cichocki, 2018; for an overview see Tsikandilakis et al., 2022d). ...
... The reasons for this topical discontent have been attributed to how different our empirical outcomes for unconscious processing are and how polemically the believers and disbelievers of these outcomes hold on to their theses and antitheses (Pessoa & Adolphs, 2010;Stafford, 2014). Several scholars explored as best they could the scientific causes of the topical discontent (Stanislaw & Todorov, 1999;Erdelyi, 2005;Dienes, 2014;. ...
... The overarching aim of the current work is not merely to review thesearguably (Tsikandilakis et al., 2022d)known and extensively discoursed subjects (see Pessoa, 2004;Japee, Crocker, Carver, Pessoa & Ungerleider, 2009;Pessoa & Adolphs, 2010); it is to show how to overcome them. We present here in full the rationale and a methodology for replication for attaining unbiased individual unconsciousness. ...
Article
Full-text available
Unbiased individual unconsciousness is a methodology that employs non-parametric receiver operating characteristics and Bayesian analyses to provide estimations for thresholds for subjective visual suppression. It can enable a researcher to define among brief durations (e.g., 8.33 or 16.67 or 25 ms), per participant and elicitor type, the threshold of presentation for which each participant is individually unconscious during masking. The outcomes of this method are then used in a subsequent experimental session that involves psychophysiological assessments and participant ratings to explore evidence for unconscious processing and emotional responsivity. Following collegial requests for a dedicated manuscript on the rationale and replication of this method, in this manuscript, we provide a thorough, comprehensive and reader-friendly tutorial-guide. We include empirical illustrations, open-source and ready-to-use methodological, mathematical and statistical coding scripts and step-by-step instructions for replicating this methodology. We discuss the potential contributions and the developing applications of individual unconsciousness in topical research.
... Specifically, subliminally presented stimuli can elicit the fast feedforward activity in the visual cortex, but they fail to initiate the subsequent recurrent activity that is necessary for awareness. While it is possible that stronger activation can be evoked by a subliminally-presented fearful face compared to a neutral face at the initial feedforward stage (e.g., Pessoa & Adolphs, 2010, the differences in the neural activation may not be sufficient for further processing of other aspects of the stimulus such as its spatial location. Our previous ERP results are consistent with this by showing that the spatial location of a target fearful face can be processed only in the aware conditions (Qiu et al., 2022b), yet a fearful face nevertheless enhances early posterior ERPs regardless of its spatial location when awareness is impeded (Qiu et al., 2022c). ...
... Consistent with these claims, our MVPA results showed that, when visual awareness was impeded during subliminal viewings, the presence of a fearful face in the display nevertheless changed the neural responses across brain regions in healthy adults. While it is likely that the presence of a fearful face can be readily encoded during the initial fast feedforward stage of visual processing (Pessoa & Adolphs, 2010, we argue that these changes may also be enabled, at least partly, by the subcortical pathways. Possibly, the amygdala is more strongly activated in response to a fearful face and the signals can be subsequently passed on to the occipital areas via direct connections (Bayle et al., 2009;Catani et al., 2003;Vuilleumier, 2005). ...
Preprint
Full-text available
Mixed findings have been reported for the nonconscious processing of fearful faces. Here, we used multivariate pattern analysis on electroencephalography data from three backward masking experiments to decode the conscious and nonconscious processing of fearful faces. Three groups of participants were shown pairs of faces that were presented either subliminally (16 ms) or supraliminally (266 ms) and were required to complete tasks where the face stimuli were either task-relevant (Experiment 1) or task-irrelevant (Experiments 2 and 3). We decoded the neural activity to examine the temporal dynamics of visual awareness, and to investigate whether the presence and location of a fearful face were processed when levels of awareness varied. The results reveal that the spatial location of fearful faces can be decoded from neural patterns only when they are consciously seen and relevant to participants' task. Nevertheless, the processing of the mere presence of fearful faces can occur in the absence of visual awareness, and the related neural patterns can be generalised to the conscious, non-spatial processing of fearful faces. Additionally, the flexibility of spatial attention seems to modulate the processing of fearful faces.
... Kihlstrom, Mulvaney, Tobias & Tobis, 2000;Tallis, 2002;Winkielman & Berridge, 2004;Kihlstrom, 2008;Ffytche, 2011;Talvitie, 2018;. Instead, our opinion is that they describe very well the origins of why cotemporary research is so divided between believers and disbelievers in unconscious processing, and why contemporary researchers were required to go to great lengths to restore some quantum of faith in conducting research on the unconscious (see Merikle & Cheesman, 1987;Bar & Biederman, 1998;Pessoa & Adolphs, 2010;Elgendi, Kumar, Barbic, Howard, Abbott & Cichocki, 2018 (Fogelson, Kohler, Miller, Granger & Tse, 2014;Knotts, Lau & Peters, 2018;Stein, Utz & Van Opstal, 2020). These arguably stem from similar principles and share common objectives (Faivre, Berthet, & Kouider, 2012). ...
... Applying them with proper scholarly and methodological awareness is the foundation for undertaking and advancing topical research. In simple words, we do, indeed, need to beor becomeconscious of what we are doing, when we are doing unconscious research (Bornstein, 1989;Theus, 1994;Erdelyi, 2004;Pessoa & Adolphs, 2010;Wyer & Srull, 2014;van der Ploeg, Brosschot, Versluis & Verkuil, 2017;Tsikandilakis, Bali, Derrfuss & Chapman, 2019;Vadillo et al., 2020). ...
Article
Full-text available
In this manuscript, we provide a discourse of issues and resolutions that we should be conscious of when doing research in the unconscious using backward masking of faces. First, we revisit subjects that are contributing for understanding the unconscious as a concept. These involve historical episodes and early episodes of controversial experimentation. Subsequently, we revisit and discuss topical concepts, such as the metrics and the statistical analyses applied for assessing perception during backward masking. We proceed to discussing novel and developing issues relating to masked visual processing and contemporary psychophysics. We empirically illustrate how these issues bias experimental research. We present and empirically illustrate resolutions for these issues, such as the application of signal detection theory metrics, Bayesian analysis and advances in the psychophysics of masked presentations using faces. We assess whether and the extent to which we are truly conscious of established and known, and novel and developing issues and resolutions. We make use of backward masking as a rally point to emphasize the importance of conscious scholarly and methodological awareness for undertaking and advancing research in the unconscious.
... Visual input is processed through several areas of the visual cortex (V1, V2, V4) and the temporal cortex before it reaches the amygdala, whereupon autonomic and endocrine mediators in lower midbrain and brainstem structures are engaged. This is considered an evolutionarily later consciously aware and precise, but also slower, visual system which identifies objects (Carr, 2015;LeDoux, 1994;Pessoa & Adolphs, 2010). The amygdala is connected to all areas of the ventral visual stream via feedback projections. ...
Thesis
Emotional-associative learning processes such as fear conditioning and extinction are highly relevant to not only the development and maintenance of anxiety disorders (ADs), but also to their treatment. Extinction, as the laboratory analogue to behavioral exposure, is assumed a core process underlying the treatment of ADs. Although exposure-based treatments are highly effective for the average patient suffering from an AD, there remains a gap in treatment efficacy with over one third of patients failing to achieve clinically significant symptom relief. There is ergo a pressing need for intensified research regarding the underlying neural mechanisms of aberrant emotional-associative learning processes and the neurobiological moderators of treatment (non-)response in ADs. The current thesis focuses on different applications of the fundamental principles of fear conditioning and extinction by using two example cases of ADs from two different multicenter trials. First, we targeted alterations in fear acquisition, extinction, and its recall as a function of psychopathology in panic disorder (PD) patients compared to healthy subjects using fMRI. Second, exposure-based therapy and pre-treatment patient characteristics exerting a moderating influence on this essential learning process later on (i.e. treatment outcome) were examined using multimodal functional and structural neuroimaging in spider phobia. We observed aberrations in emotional-associative learning processes in PD patients compared to healthy subjects indicated by an accelerated fear acquisition and an attenuated extinction recall. Furthermore, pre-treatment differences related to defensive, regulatory, attentional, and perceptual processes may exert a moderating influence on treatment outcome to behavioral exposure in spider phobia. Although the current results need further replication, on an integrative meta level, results point to a hyperactive defensive network system and deficient emotion regulation processes (including extinction processes) and top-down control in ADs. This speaks in favor of transdiagnostic deficits in important functional domains in ADs. Deficits in transdiagnostic domains such as emotion regulation processes could be targeted by enhancing extinction learning or by means of promising tools like neurofeedback. The detection of pre-treatment clinical response moderators, for instance via machine learning frameworks, may help in supporting clinical decision making on individually tailored treatment approaches or, respectively, to avoid ineffective treatment and its related financial costs. In the long run, the identification of neurobiological markers which are capable of detecting non-responders a priori represents an ultimate goal.
... Although the existence of thalamo-amygdala pathways in humans has been questioned (Pessoa & Adolphs, 2010), numerous imaging and electrophysiolgical recording studies support this idea. Emotional stimuli have been shown to elicit an early (under 150 ms) response in the amygdala to emotional stimuli that is unaffected by attentional load or conscious awareness (Luo et al., 2009(Luo et al., , 2010Pourtois et al., 2010;Vuilleumier et al., 2002). ...
Article
Full-text available
The mechanisms underlying the subjective experiences of mental disorders remain poorly understood. This is partly due to long-standing over-emphasis on behavioral and physiological symptoms and a de-emphasis of the patient's subjective experiences when searching for treatments. Here we provide a new perspective on the subjective experience of mental disorders based on findings in neuroscience and artificial intelligence (AI). Specifically, we propose the subjective experience that occurs in visual imagination depends on mechanisms similar to generative adversarial networks that have recently been developed in AI. The basic idea is that a generator network fabricates a prediction of the world, and a discriminator network determines whether it is likely real or not. Given that similar adversarial interactions occur in the two major visual pathways of perception in people, we explored whether we could leverage this AI-inspired approach to better understand the intrusive imagery experiences of patients suffering from mental illnesses such as post-traumatic stress disorder (PTSD) and acute stress disorder. In our model, a nonconscious visual pathway generates predictions of the environment that influence the parallel but interacting conscious pathway. We propose that in some patients, an imbalance in these adversarial interactions leads to an overrepresentation of disturbing content relative to current reality, and results in debilitating flashbacks. By situating the subjective experience of intrusive visual imagery in the adversarial interaction of these visual pathways, we propose testable hypotheses on novel mechanisms and clinical applications for controlling and possibly preventing symptoms resulting from intrusive imagery.
... This indicates that, although the capability of the visual trigger to induce ASMR is weaker than that of the auditory trigger, the visual trigger would be processed as an ASMR-inducing trigger faster than an auditory trigger. This is reasonable considering the property of the modalities: a visual stimulus can induce an emotional response even though it is a still image and requires a few milliseconds to process 24 , whereas an auditory stimulus requires a relatively longer duration 25 . In addition, the change in PANAS scores before and after watching ASMR-inducing stimuli (Table S3) was not signi cant, regardless of the condition (positive: |t| < 1.35, p 0.180; negative: |t| < 1.59, p 0.116). ...
Preprint
Full-text available
Despite the growing research interest in the autonomous sensory meridian response (ASMR), research on the triggers that induce ASMR has been scarce. In particular, the role of visual triggers in ASMR induction remains largely unknown. The current study, using the newly developed stimulus set, showed that the enhancement of auditory-induced ASMR by simultaneously presented visual triggers is due to information about the source of auditory triggers and that ASMR can be induced by visual triggers alone. In Experiment 1, we assessed whether the congruency of auditory and visual triggers affected the experience of ASMR and found that the occurrence of ASMR was significantly more frequent in the condition in which these triggers were congruent than when they were incongruent. In Experiment 2, we assessed whether the visual trigger was capable of inducing ASMR by itself and found that, although the frequency of ASMR occurrence was lower in the visual trigger than the auditory trigger only, the visual trigger was single-handedly able to induce ASMR. The generalization of these findings is provided by the sufficient number of newly prepared stimuli that successfully induced ASMR within a short duration. These results may facilitate psychophysiological research on the properties of ASMR-inducing stimuli.
... In two previous publications, we assessed whether own-culture faces can be processed without awareness, and we implemented a novel methodology for unconsciousness Tsikandilakis et al., 2021a). We employed non-parametric receiver operating characteristics (Stanislaw & Todorov, 1999;Zhang & Mueller, 2005; see also Dienes, 2021), Bayesian analyses for unconscious perception (Dienes, 2014;2016; and hits-versus-miss analyses for assessing participant responses (Pessoa, 2005;Pessoa, Japee, Sturman & Ungerleider, 2005;Pessoa & Adolphs, 2010;Lynn & Barrett, 2014;Sergeev, Vezhleva, Sydikhov, Arapov & Lvova, 2018;Wixted, 2020). ...
Preprint
Full-text available
The aim of the current research was to explore whether we can improve the recognition of cross-cultural freely expressed emotional faces in British participants. We tested several methods for improving the recognition of freely expressed emotional faces, such as different methods for presenting other-culture expressions of emotion from individuals from Chile, New Zealand and Singapore in two experimental stages. In the first experimental stage, in phase one, participants were asked to identify the emotion of cross-cultural freely-expressed faces. In the second phase, different cohorts were presented with interactive side-by-side, back-to-back and dynamic morphing of cross cultural freely-expressed emotional faces, and control conditions. In the final phase, we repeated phase one using novel stimuli. We found that all non-control conditions led to recognition improvements. Morphing was the most effective condition for improving the recognition of cross cultural emotional faces. In the second experimental stage, we presented morphing to different cohorts including own to-other and other-to-own freely-expressed cross-cultural emotional faces and neutral-to-emotional and emotional to-neutral other-culture freely-expressed emotional faces. All conditions led to recognition improvements and the presentation of freely-expressed own-to-other cultural emotional faces provided the most effective learning. These findings suggest that training can improve the recognition of cross-cultural freely-expressed emotional expressions.
Article
During natural viewing, we often recognize multiple objects, detect their motion, and select one object as the target to track. It remains to be determined how such behavior is guided by the integration of visual form and motion perception. To address this, we studied how monkeys made a choice to track moving targets with different forms by smooth pursuit eye movements in a two-target task. We found that pursuit responses were biased toward the motion direction of a target with a hole. By computing the relative weighting, we found that the target with a hole exhibited a larger weight for vector computation. The global hole feature dominated other form properties. This dominance failed to account for changes in pursuit responses to a target with different forms moving singly. These findings suggest that the integration of visual form and motion perception can reshape the competition in sensorimotor networks to guide behavioral selection.
Article
The ability to interrogate specific representations in the brain, determining how, and where, difference sources of information are instantiated can provide invaluable insight into neural functioning. Pattern component modeling (PCM) is a recent analytic technique for human neuroimaging that allows the decomposition of representational patterns in brain into contributing subcomponents. In the current study, we present a novel PCM variant that tracks the contribution of prespecified representational patterns to brain representation across areas, thus allowing hypothesis-guided employment of the technique. We apply this technique to investigate the contributions of hedonic and nonhedonic information to the neural representation of tactile experience. We applied aversive pressure (AP) and appetitive brush (AB) to stimulate distinct peripheral nerve pathways for tactile information (C-/CT-fibers, respectively) while patients underwent functional magnetic resonance imaging (fMRI) scanning. We performed representational similarity analyses (RSAs) with pattern component modeling to dissociate how discriminatory versus hedonic tactile information contributes to population code representations in the human brain. Results demonstrated that information about appetitive and aversive tactile sensation is represented separately from nonhedonic tactile information across cortical structures. This also demonstrates the potential of new hypothesis-guided PCM variants to help delineate how information is instantiated in the brain.
Article
This ultrahigh field 7T fMRI study addressed the question of whether there exists a core network of brain areas at the service of different aspects of body perception. Participants viewed naturalistic videos of monkey and human faces, bodies, and objects along with mosaic-scrambled videos for control of low-level features. Independent component analysis (ICA) based network analysis was conducted to find body and species modulations at both the voxel and the network levels. Among the body areas, the highest species selectivity was found in the middle frontal gyrus and amygdala. Two large-scale networks were highly selective to bodies, dominated by the lateral occipital cortex and right superior temporal sulcus (STS) respectively. The right STS network showed high species selectivity, and its significant human body-induced node connectivity was focused around the extrastriate body area (EBA), STS, temporoparietal junction (TPJ), premotor cortex, and inferior frontal gyrus (IFG). The human body-specific network discovered here may serve as a brain-wide internal model of the human body serving as an entry point for a variety of processes relying on body descriptions as part of their more specific categorization, action, or expression recognition functions.
Article
An evolved module for fear elicitation and fear learning with 4 characteristics is proposed. (a) The fear module is preferentially activated in aversive contexts by stimuli that are fear relevant in an evolutionary perspective. (b) Its activation to such stimuli is automatic. (c) It is relatively impenetrable to cognitive control. (d) It originates in a dedicated neural circuitry, centered on the amygdala. Evidence supporting these propositions is reviewed from conditioning studies, both in humans and in monkeys; illusory correlation studies; studies using unreportable stimuli; and studies from animal neuroscience. The fear module is assumed to mediate an emotional level of fear learning that is relatively independent and dissociable from cognitive learning of stimulus relationships.
Article
The thalamus has long been seen as responsible for relaying information on the way to the cerebral cortex, but it has not been until the last decade or so that the functional nature of this relay has attracted significant attention. Whereas earlier views tended to relegate thalamic function to a simple, machine-like relay process, recent research, reviewed in this article, demonstrates complicated circuitry and a rich array of membrane properties underlying the thalamic relay. It is now clear that the thalamic relay does not have merely a trivial function. Suggestions that the thalamic circuits and cell properties only come into play during certain phases of sleep to effectively disconnect the relay are correct as far as they go, but they are incomplete, because they fail to take into account interesting and variable properties of the relay that, we argue, occur during normal waking behavior. Although the specific function of the circuits and cellular properties of the thalamic relay for waking behavior is far from clear, we offer two related hypotheses based on recent experimental evidence. One is that the thalamus is not used just to relay peripheral information from, for example, visual, auditory, or cerebellar inputs, but that some thalamic nuclei are arranged instead to relay information from one cortical area to another. The second is that the thalamus is not a simple, passive relay of information to cortex but instead is involved in many dynamic processes that significantly alter the nature of the information relayed to cortex.
Article
The book links the analysis of the brain mechanisms of emotion and motivation to the wider context of what emotions are, what their functions are, how emotions evolved, and the larger issue of why emotional and motivational feelings and consciousness might arise in a system organized like the brain. The topics in motivation covered are hunger, thirst, sexual behaviour, brain-stimulation reward, and addiction. The book proposes a theory of what emotions are, and an evolutionary, Darwinian, theory of the adaptive value of emotion, and then describes the brain mechanisms of emotion. The book examines how cognitive states can influence emotions, and in turn, how emotions can influence cognitive states. The book also examines emotion and decision-making, with links to the burgeoning field of neuroeconomics. The book describes the brain mechanisms that underlie both emotion and motivation in a scientific form that can be used by both students and scientists in the fields of neuroscience, psychology, cognitive neuroscience, biology, physiology, psychiatry, and medicine.