How cortical neurons help us see: visual recognition in the human brain.
ABSTRACT Through a series of complex transformations, the pixel-like input to the retina is converted into rich visual perceptions that constitute an integral part of visual recognition. Multiple visual problems arise due to damage or developmental abnormalities in the cortex of the brain. Here, we provide an overview of how visual information is processed along the ventral visual cortex in the human brain. We discuss how neurophysiological recordings in macaque monkeys and in humans can help us understand the computations performed by visual cortex.
- SourceAvailable from: Hao Yan[Show abstract] [Hide abstract]
ABSTRACT: Schizophrenic patients present abnormalities in a variety of eye movement tasks. Exploratory eye movement (EEM) dysfunction appears to be particularly specific to schizophrenia. However, the underlying mechanisms of EEM dysfunction in schizophrenia are not clearly understood. To assess the potential neuroanatomical substrates of EEM, we recorded EEM performance and conducted a voxel-based morphometric analysis of gray matter in 33 schizophrenic patients and 29 well matched healthy controls. In schizophrenic patients, decreased responsive search score (RSS) and widespread gray matter density (GMD) reductions were observed. Moreover, the RSS was positively correlated with GMD in distributed brain regions in schizophrenic patients. Furthermore, in schizophrenic patients, some brain regions with neuroanatomical deficits overlapped with some ones associated with RSS. These brain regions constituted an occipito-tempro-frontal circuitry involved in visual information processing and eye movement control, including the left calcarine cortex [Brodmann area (BA) 17], the left cuneus (BA 18), the left superior occipital cortex (BA 18/19), the left superior frontal gyrus (BA 6), the left cerebellum, the right lingual cortex (BA 17/18), the right middle occipital cortex (BA19), the right inferior temporal cortex (BA 37), the right dorsolateral prefrontal cortex (BA 46) and bilateral precentral gyri (BA 6) extending to the frontal eye fields (FEF, BA 8). To our knowledge, we firstly reported empirical evidence that gray matter loss in the occipito-tempro-frontal neuroanatomical circuitry of visual processing system was associated with EEM performance in schizophrenia, which may be helpful for the future effort to reveal the underlying neural mechanisms for EEM disturbances in schizophrenia.PLoS ONE 10/2011; 6(10):e25805. · 3.53 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Neurophysiological evidence for invariant representations of objects and faces in the primate inferior temporal visual cortex is described. Then a computational approach to how invariant representations are formed in the brain is described that builds on the neurophysiology. A feature hierarchy model in which invariant representations can be built by self-organizing learning based on the temporal and spatial statistics of the visual input produced by objects as they transform in the world is described. VisNet can use temporal continuity in an associative synaptic learning rule with a short-term memory trace, and/or it can use spatial continuity in continuous spatial transformation learning which does not require a temporal trace. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in, for example, spatial and object search tasks. The approach has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene. The approach has also been extended to provide, with an additional layer, for the development of representations of spatial scenes of the type found in the hippocampus.Frontiers in Computational Neuroscience 01/2012; 6:35. · 2.23 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Theoretical foundations of higher order spectral analysis are revisited to examine the use of time-varying bicoherence on non-stationary signals using a classical short-time Fourier approach. A methodology is developed to apply this to evoked EEG responses where a stimulus-locked time reference is available. Short-time windowed ensembles of the response at the same offset from the reference are considered as ergodic cyclostationary processes within a non-stationary random process. Bicoherence can be estimated reliably with known levels at which it is significantly different from zero and can be tracked as a function of offset from the stimulus. When this methodology is applied to multi-channel EEG, it is possible to obtain information about phase synchronization at different regions of the brain as the neural response develops. The methodology is applied to analyze evoked EEG response to flash visual stimulii to the left and right eye separately. The EEG electrode array is segmented based on bicoherence evolution with time using the mean absolute difference as a measure of dissimilarity. Segment maps confirm the importance of the occipital region in visual processing and demonstrate a link between the frontal and occipital regions during the response. Maps are constructed using bicoherence at bifrequencies that include the alpha band frequency of 8Hz as well as 4 and 20Hz. Differences are observed between responses from the left eye and the right eye, and also between subjects. The methodology shows potential as a neurological functional imaging technique that can be further developed for diagnosis and monitoring using scalp EEG which is less invasive and less expensive than magnetic resonance imaging.Journal on Advances in Signal Processing 01/2012; 2012(1). · 0.81 Impact Factor
3054?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
How cortical neurons help us see:
visual recognition in the human brain
Julie Blumberg1,2 and Gabriel Kreiman1,3
1Department of Ophthalmology, Children’s Hospital, Harvard Medical School, Boston, Massachusetts, USA.
2Epilepsy Center, University Hospital Freiburg, Freiburg, Germany. 3Center for Brain Science, Harvard University, Boston, Massachusetts, USA.
More is known about the functions and deficits of the eye than
about the rest of the visual system. However, most visual process-
ing occurs in the cortex of the brain. What we see is a heavily trans-
formed version of the input provided by photons impinging on the
photoreceptors. By and large, that transformation happens in the
vast mass of neocortex devoted to visual processing in the primate
brain, the visual cortex (Figure 1). Combining systematic studies
of the neurophysiology and neuroanatomy of the visual system in
human and non-human animal models is providing new under-
standing of some of the computations performed in the visual
cortex. We hope that this emerging knowledge will play a critical
role in the future to help the large number of subjects with visual
impairment due to cerebral cortex damage.
Even though many visual deficits that are presented clinically
relate to the eyes, the retina mediates only the initial steps in pro-
cessing visual information. There are abundant reports in the liter-
ature of cases that involve visual deficits that are caused by damage
to cerebral cortex. The appearance of visual problems after cortical
damage is not surprising, given that a large part of the cerebral
cortex serves visual processing functions (1–5).
Among the many functions of vision, object recognition is
arguably one of the most crucial. Visual object recognition is
essential for most everyday tasks, including reading, navigation,
and face identification. In a small fraction of a second (approxi-
mately 150 ms), we can recognize complex shapes and categorize
objects and scenes (3, 6–10). Visual object recognition depends on
combining two key properties: visual selectivity and the robust-
ness of recognition to object transformations. Visual selectivity
refers to the ability to discriminate among similar objects. A key
to vision is that we can successfully recognize objects despite
transformations in size, position, rotation, and illumination,
among others (Figure 2). Given that an object can cast an infi-
nite number of projections on the retina, our ability to recognize
objects in a manner that is robust to object transformations is
likely to have played a key role in the evolution of the visual sys-
tem. This robustness to image transformations constitutes one
of the major challenges for computer-based visual recognition,
because most of the image pixels can change dramatically while
the content remains the same.
Here, we provide an overview of the processing steps along the
ventral visual cortex in the human brain, focusing on what is
known (and what is not known) about the mechanisms respon-
sible for visual recognition. Because it is difficult to directly access
human visual cortex through invasive methodologies, much more
is known about the macaque monkey visual cortex than about the
human visual cortex. Throughout this Review, we discuss data
generated from the analysis of both the macaque monkey and the
human visual system. We hope that our article provides an over-
view of a small fraction of the studies of visual cortex. The goal
here is not to be comprehensive, as there is insufficient space for
this; rather, we hope that this article will arouse readers’ curiosity,
and we encourage them to peruse more literature.
Neuroanatomy of the visual system
We start by providing an overview of neural connectivity in the
primate visual system based on anatomical studies in macaque
monkeys (Figure 1; for more details, see refs. 1 and 2). Vision starts
when photons impinge on photoreceptors in the retina (for an
overview of the retinal circuitry, see refs. 2, 11, 12). Information
flows through the retinal circuit from the photoreceptors to retinal
ganglion cells. These are the cells that provide the retinal output.
Retinal ganglion cells project through axons in the optic nerve to
a part of the thalamus called the lateral geniculate nucleus (LGN)
(2). Neurons in the LGN project to the primary visual cortex (V1),
which is located in the occipital neocortex. Like other parts of neo-
cortex, V1 is composed of six layers, many of which can in turn be
subdivided into sublayers (13–16). Neurons in V1 are organized
into columns, approximately perpendicular to the cortical surface,
where neurons share similar functional properties (16, 17). V1 is
the first stage of visual processing, where information from the
right and left eyes is combined (Figure 1).
Perhaps due to its relative accessibility, much more is known
about V1 than about any other part of the visual cortex. Yet there
are a large number of different visual areas in the neocortex out-
side of V1; these are collectively known as the “extrastriate visual
cortex.” From V1, two main pathways can be distinguished:
a dorsal stream and a ventral stream (1, 16, 18, 19). The dorsal
stream includes areas known as area medial temporal (area MT,
also known as area V5) and medial superior temporal (MST) and
is predominantly involved in processing spatial information,
including detecting object motion, position, and stereovision,
i.e., depth perception; the dorsal stream is often referred to as
Conflict?of?interest: The authors have declared that no conflict of interest exists.
Citation?for?this?article: J Clin Invest. 2010;120(9):3054–3063. doi:10.1172/JCI42161.
?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
the “where/how” pathway (20, 21). The ventral stream includes
areas such as V2, V4, and the inferior temporal cortex (ITC) and
is predominantly involved in discriminating colors and shapes;
the ventral stream is often referred to as the “what” pathway
(3, 22–24). To a first approximation, we can think about the visual
cortex as two parallel and hierarchical cascades (Figure 1). How-
ever, this is only a coarse approximation. There are also numerous
interconnections between these two pathways. The V1 to V2, to
V4, to ITC pathway is often thought of as a feed-forward pathway
(whereby information flows “up” the visual hierarchy). In addi-
tion to these forward connections, there are backward connec-
tions at every stage of the visual system; the only exception to this
rule is the retina: there are no known back-projections from the
LGN to the retina. These back-projections are likely to play a key
role in top-down influences in visual perception. Thus, the ITC
projects back to V4; V4 projects back to V2; and so on. There are
also numerous internal recurrent connections within each area.
Furthermore, “bypass connections” link, for example, V1 directly
to V4 and the ITC directly back to V2.
The last purely visual stages in the “what” and “where/how” streams
project to multiple nonvisual areas of the cortex. Among them, two
important targets of these projections are areas of the prefrontal
cortex and the medial temporal lobe, including the hippocampus
and surrounding structures. The prefrontal cortex is likely to play a
key role in the moment-to-moment and task-dependent interpreta-
tion of visual input to orchestrate cognitive decisions (25, 26), while
the hippocampus and surrounding structures play a key role in the
consolidation of information into long-term memories (27, 28).
The description in this section provides a simplified overview
of the neuroanatomy of the visual system. The system is highly
complex (with billions of interconnected neurons), and at this
stage we probably know only a small fraction of the connections
in neocortex. There has been much excitement recently in the field
of connectomics, in which researchers are aiming to develop high-
throughput tools to map connections at high resolution within a
small part of neocortex.
Visual deficits due to cerebral cortex lesions
There are multiple clinical cases where the patient does not have
any apparent deficit in eye function and yet manifests profound
visual deficits. In several cases, these deficits can be ascribed to
damage to specific areas within the neocortex. The study of these
patients has provided key insights about visual recognition by
providing a correlate between brain areas and functions. We pro-
vide an overview of some of these studies, highlighting what we
have learned about the visual cortex. We start the discussion by
describing the functional consequences of damage to early visual
cortex. Lesion and neurophysiological studies in cats, and sub-
sequently in monkeys, led the way to uncover a multiplicity of
visual areas outside V1 (1, 20, 22, 29–31). We summarize below
some of these visual deficits caused by damage to extrastriate
visual areas (e.g., refs. 22, 32–35), dividing the discussion into
damage to dorsal visual cortex and damage to ventral visual cor-
tex. Although the cases involving damage to ventral visual cortex
are particularly relevant for our discussion on visual recognition,
we describe some studies involving dorsal stream damage because
Schematic illustration of the basic architec-
ture in the primate visual system. On the left,
we show a schematic diagram of the human
brain indicating the approximate location of
several brain areas discussed in the text. On
the right, we provide a schematic description
of some of the main pathways involved in pro-
cessing visual information in cortex. VP, ven-
tral stream; DP, dorsal stream; RGC, retinal
ganglion cells; CIT, central ITC; AIT, anterior
ITC; MST, medial superior temporal cortex;
POR, post-rolandic area; PR, pre-rolandic
area. Solid lines indicate forward projections;
dashed lines indicate back-projections; and
the curved lines represent recurrent connec-
tions within a given area. As emphasized in
the text, this diagram is a substantial simplifi-
cation of the actual connectivity in the primate
visual system. Many important visual areas
and connections are not represented in this
diagram; for more details, see ref. 1.
3056?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
they vividly illustrate the relationship between location and dam-
age and also because of the multiple interconnections between the
dorsal and ventral processing streams. Many of these conditions
are described in single case studies, and it is generally difficult to
rigorously quantify their prevalence; some studies suggest that
the combination of visual deficits due to cortical damage is even
more prevalent than the combination of visual deficits that can
be ascribed to the eyes (32).
Deficits associated with lesions in early visual cortex. The fundamental
role of V1 in vision was appreciated early in the last century. Stud-
ies in cats, dogs, and monkeys that involved complete lesion of
the occipital cortex left animals with little if any visual function
(for a historical review, see ref. 31). After World War I, the study of
subjects who suffered limited occipital lobe lesions due to bullet
wounds revealed localized scotomas (i.e., islands of impaired visual
acuity) in the part of the visual field contralateral to the wound
(36, 37). The more severe condition hemianopia refers to loss of
vision in one-half of the field of vision. Effects similar to those
caused by bullet wounds in the occipital lobe are often encoun-
tered through vascular damage, tumors, and trauma.
A few case studies have discussed a phenomenon referred to as
“blindsight” (38, 39). As the name suggests, some subjects with
lesions in the occipital cortex are still capable of certain visual
behavior within the scotoma. For example, subjects may be able
to tell light from dark, or even to detect the direction of a mov-
ing object. Several possibilities were proposed to account for these
observations, including anatomical routes that bypass V1 and the
presence of small intact islands in V1 despite the lesions. Although
the phenomenon has been clearly demonstrated, the range of
visual behaviors preserved within the scotoma is rather limited,
and the subjects’ capacity for fine visual discrimination within it
is clearly lost. V1 is absolutely necessary for most visual functions.
The profound visual deficits caused by V1 lesions in both animals
and humans, combined with the challenges in examining visual
behavior in animals, led several prominent investigators to argue
that V1 is not only necessary but also sufficient for visual percep-
tion. In an interesting historical overview, Gross cites several strik-
ing demonstrations of this narrow-minded perception that we
now know to be completely wrong (31).
Deficits associated with lesions in dorsal visual cortex. One of the condi-
tions in which the extrastriate visual areas of the dorsal stream are
damaged is akinetopsia, which involves a specific inability to dis-
criminate visual motion (40, 41). Zihl et al. described a patient who
suffered a bilateral cerebral vascular lesion, outside V1, and who
had a specific impairment in her ability to see objects in motion
(41). The lesion included the posterior portion of the middle tem-
poral gyrus, the retrorolandic area, and the temporoparietal and
occipital white matter. It is quite tempting to ascribe the deficits in
this subject to area MT, as several investigators have described neu-
rons that are sensitive to the direction of motion in that region of
the macaque monkey visual cortex (21). As in other human lesion
studies, although the lesion in this subject does include the likely
human homolog of area MT (40), making a precise connection is
challenging due to the relatively large region of cortex impaired
in this patient. It is interesting to note that electrical stimulation
through large grid electrodes of area MT of the human brain can
induce direction-specific motion blindness, i.e., blindness to motion
in one direction while perception of motion in other directions is
completely normal (42). Dutton described the opposite defect in a
woman who could not see immobile objects following bioccipital
infarctions (32). While the dorsal pathway was apparently intact,
she was able to navigate independently only when she moved her
head or eyes to see the environment. These studies suggest that the
ability to discriminate specific aspects of the visual image (motion
or lack thereof) can be correlated with the presence of intact cortex
within relatively circumscribed parts of visual cortex.
Damage to the parietal cortex is often associated with differ-
ent variations of neglect syndrome. While this is not necessarily
a strictly visual deficit, it is often manifest as partial blindness.
Individuals with this condition are unable to perceive part of the
environment unless they pay direct attention to the neglected
area. The most common form is left hemispatial neglect (i.e.,
an inability to perceive objects in the left side of the visual field,
caused by posterior parietal lobe lesions in the contralateral right
hemisphere) (35, 43–46). A syndrome that is often associated with
hemineglect is Balint syndrome, which is characterized by the
triad of simultanagnosia, optic ataxia, and ocular apraxia. Simul-
tanagnosia refers to the inability to identify several objects in a
scene at once. Patients perceive the world in fragments and are
Robustness of the human visual system to image transformations. In
spite of large transformations of the original image (top left) in terms
of color, noise, scale, blurring, and inversion, recognition remains rela-
tively straightforward. Careful psychophysics studies have quantified
and documented the robustness of the human visual system to image
transformations. Achieving specificity while maintaining tolerance to
object transformations is one of the key challenges for computer vision
and constitutes one of the hallmarks of object recognition in primates.
Original photo taken by Julie Blumberg.
?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
unable to form the whole picture. They typically show difficulty
in recognizing a toy in a cluttered box or a person in a crowd, but
they do not have a problem identifying the person or toy when it
is isolated. Playing team sports or coping in busy environments
(e.g., a supermarket or a busy playground) can also be challenging.
Optic ataxia and ocular apraxia are conditions in which the visual
input to the eyes cannot be used for visually guided movements
and coordination of eye movements, respectively. Patients with
Balint syndrome are also characterized by an impaired ability to
estimate distances. In six patients with Balint syndrome show-
ing similar visual deficits, bilateral lesions in the splenium and
angular and supramarginal gyri were observed. The etiology of
Balint syndrome is thought to be most frequently bilateral pari-
etal damage, although lesion localization is difficult due to the
small number of cases (32, 47, 48).
Deficits associated with lesions in ventral visual cortex. Several pro-
found visual deficits have also been described as occurring as a
result of damage along the ventral visual stream. These conditions
are particularly relevant to understanding how ventral cortex
contributes to visual recognition. One such condition, known as
achromatopsia, involves a specific inability to perceive colors (30).
This cortical color blindness is distinct and dissociable from reti-
nal color blindness, which can be linked to missing or abnormal
color pigments in the retinal cones. Subjects with achromatopsia
typically have relatively large lesions in the inferior ventromedial
sector of the occipital lobe, including the lingual and fusiform
gyri. Depending on the extent of the lesion, subjects may also dis-
play a scotoma within their visual field and more specific visual
recognition deficits (35). These subjects show poor performance
in hue discrimination tasks but perform well in tasks that involve
verbal instructions such as answering the question “What color are
bananas?” emphasizing the visual nature of the deficit.
An interesting dissociation between the dorsal and ventral streams
was reported in a patient with profound ventral cortex damage,
particularly within the temporal lobe, by Goodale and Milner (34).
This subject, who had suffered from carbon monoxide poisoning,
had severe impairment in object shape recognition. Yet despite her
inability to recognize objects, she showed a rather remarkable abil-
ity to interact with many objects. She showed an appropriate reach
response toward objects whose shapes she could not describe. She
also showed correct behavioral performance in visuomotor tasks.
For example, in one of the experiments, the subject was asked to
indicate the orientation of a slit and performed essentially at chance
levels. Subsequently, she was asked to push an envelope through the
slit. Surprisingly, the subject was almost as good as controls in this
visually guided motor task despite the fact that she was unable to
describe the orientation of the slit. Goodale and Milner proposed
that the dorsal pathway is particularly engaged in “vision for action,”
the immediate use of visual information to carry out specific visual-
ly guided behaviors. In contrast to this action mode, they proposed
that awareness about an object requires activity in the ventral stream
and the temporal lobe in particular.
One of the earliest demonstrations that V1 could not be the
entire story in terms of visual recognition and that there must
be extrastriate visual function was the study of Kluver-Bucy
syndrome (49). After bilateral removal of the temporal lobe in
macaque monkeys, a variety of behavioral effects, including loss
of visual discrimination, but also increased tameness, hypersexu-
ality, and altered eating habits, were initially reported (49). The
work was subsequently refined by inducing smaller lesions that
were able to dissociate different aspects of Kluver-Bucy syndrome.
These circumscribed lesions showed that the ITC plays a critical
role in visual recognition (50–53). Bilateral removal of the ITC in
macaque monkeys leads to impairment in learning visual shapes,
as well as deficits in retaining information about visual discrimi-
nations that were learned before the lesions. The deficits are long
lasting, and the severity of the deficit is typically correlated with
task difficulty. In other words, monkeys can still perform “easy”
visual discrimination tasks after bilateral ITC lesions. The behav-
ioral deficits are restricted to the visual domain and do not affect
discrimination based on tactile, olfactory, or auditory inputs. The
deficits after ITC lesions are restricted to the visual domain. The
other behavioral abnormalities in Kluver-Bucy syndrome were not
present when the lesions were restricted to the ITC. This observa-
tion underscores the importance of studying spatially restricted
lesions (whenever possible) to properly interpret behavioral defi-
cits and highlights the multiple confounding factors that arise
when considering lesions that involve multiple brain locations. In
the same way that Kluver-Bucy syndrome could be fractionated
by more detailed and circumscribed lesions, it is quite likely that
in future studies, more specific lesions within the ITC will further
fractionate the object recognition deficits prevalent after bilateral
ITC ablation. The lesion studies suggest that the ITC plays an
important and necessary role in fine object discrimination.
In humans, to our knowledge, there is no evidence for a global
object recognition deficit as a result of brain damage. Temporal
lobe lesions typically lead to different forms of agnosias, an inability
to recognize certain types of objects (22, 54). The variety of behav-
ioral manifestations in the cases of agnosia described in the litera-
ture can be to a first approximation linked to the variety of brain
regions damaged. Because these are not controlled lesion studies,
as in the animal cases, the exact circuits that are affected vary from
one patient to another. Perhaps the most widely discussed form of
agnosia (although itself a rare condition) is prosopagnosia, a deficit
in recognizing faces even though vision in general and recognition
of other objects is not impaired (55–60). This condition is often
caused by a stroke that affects the right posterior artery, which leads
to obstruction of the fusiform and lingual gyri. Other etiologies for
prosopagnosia have also been described (e.g., refs. 61–63).
The exact interpretation of the nature of the deficit associated
with prosopagnosia has been a matter of debate in the neuroscience
community. While some researchers emphasize that it is a deficit in
recognizing the special category of face stimuli (55, 58, 64), other
investigators consider this to be a specific instance of a more gen-
eral impairment in recognizing specific exemplars that look alike
within a category (65). In addition to acquired prosopagnosia, a few
studies have described cases of apparent congenital prosopagnosia.
The causes of congenital prosopagnosia are not clear. In these cases,
patients sometimes may not even be aware of their deficit, as they eas-
ily recognize people by their clothes, voices, gait, and other clues.
Other forms of agnosia have also been described. For example,
some studies have reported subjects that fail to discriminate
between animate and inanimate objects (66). Other reports have
emphasized specific types of impairment for certain object cat-
egories (35, 67–71). Topographic agnosia refers to the inability
to navigate (32). The extent to which these cases of agnosia are
restricted to the visual modality remains unclear. To show that
a form of agnosia is purely visual, it is necessary to test recogni-
tion using other cues, and this has not been done in all studies.
Thus, it is conceivable that at least some of these cases involve a
3058?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
more generic failure related to information retrieval or semantic
interpretation that is linked to but goes beyond pure visual rec-
ognition. There is also debate about how to define such seman-
tic categories (e.g., consider a definition of “inanimate”). Visual
form agnosias have been reported following disseminated lesions
or medial temporal stroke (34, 35, 72).
Impaired reading ability is also a common deficit, particularly in
children. There is a very large variety of conditions that manifest as
reading difficulties, only a few of which directly relate to cortical
visual impairment (32, 73). Hemianopia, alexia (typically a discon-
nection between the right occipital lobe and the language centers),
simultanagnosia, and temporal lobe dyslexia have all been associ-
ated with poor reading.
Neurophysiological studies of the visual cortex
In order to understand the function of the neuronal circuits
that process visual information, it is necessary to examine the
activity of individual neurons in response to visual stimuli. The
main method of recording the activity of individual neurons in
awake behaving animals or humans is to use microwire electrodes
(microelectrodes) that are implanted extracellularly and monitor
the extracellular voltage at millisecond temporal resolution and
neuronal spatial resolution. Due to technical limitations and the
invasive nature of microwire recordings, little is known about the
activity of individual neurons in the human cortex. A recent review
summarized the properties of neurons in the human temporal lobe
upon presentation of visual stimuli (ref. 5; see also refs. 7, 74).
Primary visual cortex. More work has been done examining the
activity of single neurons in the cat and monkey brains compared
with the human brain (5). Neurophysiological recordings in reti-
nal ganglion cells and LGN neurons have revealed that neuronal
responses are constrained to a relatively small patch of the visual
field known as the receptive field (75) (Figure 3). The receptive
field of a neuron is functionally defined as the region of the visual
field that triggers activity above baseline levels. There is a consid-
erable degree of variability in the size of the receptive field among
different neurons. In particular, neurons with receptive fields in
the fovea (the most sensitive part of the retina) have very small
receptive fields, typically well below one degree of visual angle,
whereas neurons with receptive fields in the periphery can have
receptive field sizes up to one degree of visual angle or more. For
orientation, one degree of visual angle approximately corresponds
to the size of the thumb at arm’s length. In addition to the size,
receptive fields typically have an important substructure that
indicates how light patterns influence the response of the neuron.
The receptive fields of retinal ganglion cells and LGN cells are typ-
ically characterized by a center-surround architecture. Two main
types of cells have been described: “on-center” cells enhance their
activity when stimulated in the center of their receptive fields and
are inhibited by illumination of the surround, while “off-center”
cells show the opposite responses (2).
Hubel and Wiesel pioneered the investigation of V1 through
microwire recordings and opened the way for examining visual
cortex neurophysiology (76). Initially working in cats, and subse-
quently in monkeys, they discovered that neurons in V1 typically
respond to bars shown at a specific orientation within their recep-
tive fields (17, 76, 77). The receptive field size for V1 neurons is typ-
ically below one degree of visual angle. Their responses are tuned
to the orientation of the bar; this type of response is similar to the
type of operations used in computer vision to extract the edges of
an image (78, 79) (Figure 4). This has led to the notion that one
of the initial stages in processing visual information in the cor-
tex is related to the extraction of edges in the image. Neurons in
V1 are retinotopically organized, arranged within columns where
neurons share similar response preferences. Hubel and Wiesel also
proposed a model of how the orientation tuning in V1 could arise
by combining multiple on-center LGN cells in a feed-forward fash-
ion, whereby the centers of the LGN cells follow the orientation
preferred by the V1 neuron. Neurons in V1 are also activated by
motion, and some neurons are sensitive to color (80).
Although V1 remains by far the most studied part of visual cor-
tex, multiple investigators have examined the responses of neurons
in extrastriate visual areas. Overall, less is known about the prop-
erties of extrastriate neurons. Partly, this can be ascribed to the
difficulty in examining the vast and high-dimensional visual space
with limited recording time. Until recently, most neurophysiology
experiments involved inserting and removing a single electrode,
holding a recording for time periods of hours at best. More recent-
ly, several investigators have started to explore the possibility of
using chronic recording technologies that may enable recording
for prolonged periods of time. Given the limits in recording time
and the vast number of possible stimuli, investigators often use
a battery of visual stimuli that are presumed to be of interest to
the neurons under study. A couple of general observations can
be drawn about the responses in extrastriate visual cortex. As we
ascend through the visual hierarchy (along both the dorsal and the
ventral streams), there is a progressive increase in receptive field
size, in response latencies, in the complexity of feature preferences,
and in the degree of tolerance to image transformations. Addition-
Schematic definition of receptive fields. Neurons throughout the visual
system respond only to local patches of the visual field. This schematic
diagram shows an experimental protocol in which the subject’s eyes
are fixated on the center X while investigators show circles at differ-
ent positions (at each moment in time, there is only one circle on the
screen). Above each circle, the ticks indicate the activity of a hypotheti-
cal neuron. This neuron fires vigorously when the circle appears in the
lower left position, defining its receptive field (arrow). Note that there is
spontaneous activity in other locations, but only one location elicits a
vigorous response. 0.25 deg, 0.25 degree of visual angle.
?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
ally, neurons in higher visual areas typically show stronger modu-
lation by attention compared with neurons in earlier visual areas
(81, 82). That is, the activity of a neuron in response to a stimulus
is substantially enhanced if the subject is paying attention to the
receptive field of the neuron compared with the response to the
same stimulus if the subject is paying attention elsewhere.
Neurophysiology along the dorsal stream. In the dorsal stream, V1
projects to area MT. Neurons in area MT show tuning to the direc-
tion of motion within their receptive fields, that is, they respond
preferentially to certain directions of motion (21). MT neurons
are also sensitive to disparity between the two eyes (see ref. 21).
Electrical stimulation of groups of MT neurons has been shown to
bias the perception of monkeys toward the preferred motion direc-
tion of the neuron (83). Lesions in area MT lead to motion percep-
tion deficits in monkeys (21). Combining the neurophysiological,
electrical stimulation, and lesion findings in the macaque, one can
speculate that area MT may play an important role in the condi-
tion known as akinetopsia mentioned in the previous section.
Neurophysiology along the ventral stream. Investigators have described
neurons in the ventral stream that respond to a variety of com-
plex shapes in the ITC (22–24, 31). ITC neurons often respond
vigorously to color, orientation, texture, direction of movement,
and shape. Neurons in the posterior ITC (PIT) show a coarse reti-
notopic organization and an almost complete representation of
the contralateral visual field. The receptive field sizes are typically
larger than the ones found in V4 neurons. The receptive fields in
anterior parts of the ITC are often large, but there is a wide range
of estimates of the sizes of receptive fields in the literature, ranging
from two degrees (84) to several tens of degrees (85, 86). Most ITC
receptive fields include the fovea, i.e., the central pit in the retina,
which shows maximal visual acuity.
Investigators have often found strong responses in ITC neurons
elicited by all sorts of different stimuli. For example, several inves-
tigators have shown that ITC neurons can be stimulated when
faces, hands, and other biological stimuli are presented (87–91).
Figure 5 shows an example of a neurophysiological recording from
a neuron in the human medial temporal lobe, which receives input
from the ITC. This neuron showed a remarkable degree of selec-
tivity and tolerance to transformations (92). Other investigators
have used parametric shape descriptors of abstract shapes (93–95).
Logothetis and Pauls trained monkeys to recognize paperclips
forming different 3D shapes and subsequently found neurons that
were selective for specific 3D configurations of paperclips (96).
While this wide range of responses may appear puzzling at first,
it is perhaps not too surprising given a simple model whereby ITC
neurons are tuned to complex shapes. Our interpretation of the
wide number of stimuli that can drive ITC neurons is that these
units are sensitive to complex shapes that can be found in multi-
ple different types of 2D visual stimuli, including fractal patterns,
faces, and paperclips. This wide range of responses also empha-
sizes that we still do not understand the key principles and tuning
properties of ITC neurons. Tanaka and others have shown that
there is clear topography in the ITC response map. Advancement
of an electrode in a trajectory approximately tangential to cortex
has revealed that neurons within a tangential penetration show
similar visual preferences (86, 97–99). They argue for the presence
of “columns” and higher-order structures such as “hypercolumns”
in the organization of shape preferences in ITC.
While each neuron shows a preference for some shapes over oth-
ers, the amount of information conveyed by individual neurons
about overall shape is limited (85). Additionally, there seems to
be a substantial amount of “noise” in the neuronal responses in
any given image presentation. Hung et al. recorded (sequential-
ly) from hundreds of neurons and used statistical classifiers to
decode the activity of a population of neurons in individual pre-
sentations (100). They found that a relatively small group of ITC
neurons (approximately 200) could support object identification
and categorization quite accurately (up to approximately 90% and
approximately 70% for categorization and identification, respec-
tively), with a very short latency after stimulus onset (approxi-
mately 100 ms after stimulus onset). Furthermore, the population
response could extrapolate across changes in object scale and posi-
tion. Thus, even when each neuron conveys only noisy information
about shape differences, populations of neurons can be quite pow-
erful in discriminating among visual objects in individual trials.
As emphasized above, a key property of visual recognition is the
capacity to recognize objects despite image transformations at
the pixel level. Several studies have shown that ITC neurons show
a substantial degree of tolerance to object transformations. ITC
neurons show similar responses despite large changes in the size
of the stimuli (96, 101, 102). Even if the absolute firing rates are
affected by the stimulus size, the rank order preferences among
different objects can be maintained despite stimulus size changes
(101). ITC neurons also show more tolerance to changes in object
position than neuronal units in earlier parts of the ventral stream
(96, 101, 102) and can tolerate a certain degree of depth rotation
(22). They even show tolerance to the particular cue used to define
the shape (such as luminance, motion, or texture) (103).
There has been increased interest recently in the possibility of exam-
ining the human brain at the neurophysiological level in cases where
electrodes are implanted for clinical reasons. The most common of
these cases involves the study of epileptic patients (5, 7, 74, 104).
These recordings have been performed in subjects that have forms
of epilepsy resistant to pharmacological treatments. Subjects are
implanted with electrodes in order to map the region of the brain
responsible for the seizures and to examine cortical function for
potential surgical treatment of epilepsy. This approach provides
a rare opportunity to examine neurophysiological activity in the
human brain at high spatial and temporal resolution. Recordings
in the human hippocampus, entorhinal cortex, amygdala, and
parahippocampal gyrus have shown strong responses to visual
stimuli and a substantial degree of tolerance to object transforma-
tions (5, 105) (Figure 5). In another study recording from these
areas, investigators found neurons that showed responses to mul-
tiple objects within a semantically defined object category such as
An edge detection algorithm applied to the image on the left produced
the image on the right. Edge detection is thought to mimic one of the
initial steps in processing visual information in primary visual cortex
(2, 79, 136). Original photo taken by Nambi Nallasamy.
3060?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
faces, animals, and cars (105). While these areas receive input from
the highest stages of the visual hierarchy (Figure 1), they are mul-
timodal areas and do not seem to be directly involved in the visual
deficits that are typically described in subjects with neocortical
damage. It is also possible to examine the responses of visual areas
in the human neocortex by recording intracranial field potentials,
also in the context of epilepsy surgery (7, 106). These recordings
have revealed selectivity to complex shapes and tolerance to object
transformations (7, 106).
Recording in patients with epilepsy has provided an extreme
example of tolerance to object transformations. Some neurons
in the human hippocampus, parahippocampal gyrus, entorhinal
cortex, and amygdala show a remarkable degree of selectivity to
individual persons or landmarks. For example, one neuron in
one subject showed a selective response to images in which for-
mer president Bill Clinton was present (Figure 5). Remarkably,
the images that elicited a response in this neuron were quite
distinct in terms of their pixel content, ranging from a black-
and-white drawing to color photographs with different poses
and views (92). As discussed above for the macaque ITC neu-
rons, we still do not have any understanding of the circuits and
mechanisms that give rise to this type of selectivity or tolerance
to object transformations.
How is the selectivity to oriented bars at the level of V1 trans-
formed into the responses to complex shapes and tolerance to
object transformations at the level of ITC? The current line of
thought is that shape preferences and tolerance to object trans-
formations build up gradually as we ascend the visual hierarchy
(24, 85, 107, 108). Consistent with this notion, neurons in areas
V2 and V4 typically show larger receptive fields and responses to
more complex shapes than do neurons in V1 but smaller receptive
fields and less complexity and tolerance to object transformations
than neurons in the ITC (109–112). Much more work is needed to
elucidate the computations that lead to shape recognition along
the ventral visual pathway.
Open questions and future work
Translating visual neuroscience to the clinic. As noted above, there are
a wide variety of visual deficits that arise as a result of damage to
different parts of the visual cortex. It is rather tempting to link the
visual deficits described above (in “Visual deficits due to cerebral
cortex lesions”) and the neurophysiological findings (described in
“Neurophysiological studies of the visual cortex”). However, this
requires solving several challenges. As mentioned earlier, most
human lesions are not well circumscribed and involve multiple
brain areas as well as neurons connecting different areas. It is there-
Example of a neurophysiological recording
from a neuron in the medial temporal lobe.
The figure is based on an experiment per-
formed by our research group, in which the
images shown to participants were similar to
those shown here (see ref. 137 for the actual
images and ref. 138 for more examples). The
raster plots indicate the action potentials fired
by the neuron in response to the image above
the raster. Each line represents a separate
repetition. Below the raster plots, we show a
post-stimulus time histogram (PSTH) aligned
to the stimulus onset, showing the average
activity of the neuron as a function of time. The
activity of this neuron increased in response to
three different images of former president Bill
Clinton and not to other faces or other stimuli.
The neuron showed a remarkable degree of
selectivity and tolerance to transformations
(92, 137). Image credit (tiger photo): www.
?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
fore challenging to establish a one-to-one map between visual defi-
cits and anatomical locations (it is also not clear whether there is a
one-to-one map). The neuroanatomical connectivity in the human
visual cortex is also much less well established than in the macaque
(and even in the macaque, we only know some of the coarse con-
nectivity patterns). Additionally, most of the neurophysiological
data come from primate studies, and the homology of visual areas
between humans and monkeys is not yet clearly established.
Some of these visual challenges may be subtle and hard to diagnose
while others are dramatic and obvious. Dutton has provided several
detailed strategies for the diagnosis of visual impairment due to cor-
tical factors (32, 113). Although it might seem easier to treat visual
deficits originating from the eye than from the brain, physicians can
still help in cases of cerebral visual impairment. A useful and practi-
cally oriented review of therapeutic options can be found in ref. 113.
Orientation, object recognition, and visually guided movements can
be partly trained. The remaining partial function can be supported
by several strategies, including enlarging text, reducing distractions,
usage of good colors and contrasts, usage of specific visual cues, and
other problem-specific solutions. For example, attentional deficits
such as hemineglect can be tackled by clear markers such as colored
stickers in the affected hemifield or approaches such as turning the
plate 180° during every meal. Visual object recognition problems can
be ameliorated by training recognition with nonvisual cues including
labels and text. Even though most often there is no causal treatment
of cerebral visual impairment, there is a lot to be done to improve the
quality of life of the affected individuals. More research and effort is
needed to translate the knowledge gained from the last four decades
of studying visual cortex into methods to help patients who show
visual impairment due to cortical damage or malfunction.
Neuroimaging. There are a large number of studies that use human
neuroimaging to bypass the single-unit recordings and examine
indirect correlates of human brain function in a noninvasive man-
ner. This is a promising research tool given the potential to examine
the human brain in a noninvasive manner. As it stands, the spatial
and temporal resolution is very poor compared with that of neuro-
physiological recordings. The spatial resolution is approximately 1
mm3, which probably encompasses at least tens of thousands of neu-
rons as well as other cells (compare with neurophysiological record-
ings that examine the activity of individual neurons). The temporal
resolution is on the order of seconds, which is about 1,000 times
slower than the speed of the dynamical signals that neurons use to
communicate with each other in cortex. Part of the problem arises
because current imaging techniques only indirectly measure neural
activity. In one of the most common techniques, magnetic reso-
nance imaging signals are used to infer blood oxygenation levels;
blood oxygenation levels are in turn indirectly related to neural activ-
ity (for a detailed review of functional imaging, see refs. 114–117).
Several studies have compared functional imaging measurements
against neurophysiological recordings. Some of these studies have
revealed a rather striking correlation between the two types of mea-
surements (e.g., refs. 118, 119). However, other studies have suggest-
ed that the relationship between these two signals is quite complex
and have described several situations where functional imaging sig-
nals can appear in the absence of concomitant neural activity and
vice versa (e.g., refs. 115, 117, 120–122). Therefore, caution must be
taken in interpreting human neuroimaging studies. For examples
of recent neuroimaging work on human visual cortex, see refs. 18,
58, 123–131. In addition to functional studies, there is increasing
enthusiasm regarding the usage of imaging techniques to examine
other aspects of brain structure in the human brain. In particular,
diffusion tensor imaging holds promise as a key tool to investigate
connectivity in the human brain.
Computational models of visual cortex. Several investigators are devel-
oping computational models of the ventral visual cortex. These
computational models may play an important role in helping us
quantitatively understand the complex and dynamic processes
involved in visual recognition (e.g., refs. 3, 108, 132–134). In the
future, one can imagine that we might be able to induce “lesions”
in the models to examine the type of visual deficits caused by these
lesions and to compare their output against the visual deficits that
patients experience. While we still have a long way to go, combined
with the neurophysiological, neurological, and imaging studies
described above, this is a very promising line of investigation. In
addition to these computational studies, much progress has also
been made through the quantitative study of visual recognition in
psychophysics (e.g., see ref. 135 for an overview).
Visual prosthetics. Combining the knowledge and techniques
described in this article, some investigators are trying to develop
prosthetic devices to help the visually impaired and even blind peo-
ple. The field of prosthetic devices has been particularly active in
the context of paralyzed patients and motor deficits. In the motor
domain, the idea is to “read out” cortical signals, decode them in
real time, and use them to actuate a robotic device. In the visual
domain, one idea is to use digital devices to capture light informa-
tion and then “write in” this information to the subject’s brain.
Investigators are thinking about neural prosthetic approaches at
multiple different levels in the visual hierarchy, from the retina
(retinal prosthesis) to primary visual cortex all the way to higher
visual cortex (cortical prosthesis). Due to space constraints, we
cannot describe these approaches in detail, but we would like to
emphasize that the field of visual prosthetics is possible thanks to a
combination of talent and dedication from researchers in multiple
fields, including those involved in neurophysiological recordings,
psychophysics, cognitive science, neurology, neurosurgery, neu-
roimaging, engineering, applied mathematics, machine learning,
and engineering. This is a highly active and interdisciplinary area
of research that may one day transform the lives of people with
severe visual impairment that cannot be treated by other means.
We acknowledge support from the Massachusetts Lions Foundation,
the NIH (1R21EY019710, 1DP2OD006461 to G. Kreiman), and the
National Science Foundation (0954570 to G. Kreiman). We also
acknowledge support from the Hamburger Foundation (J. Blum-
berg) and the German National Merit Foundation (J. Blumberg).
Address correspondence to: Gabriel Kreiman, Department of
Ophthalmology, Children’s Hospital, Harvard Medical School, 1
Blackfan Circle, Karp 11217, Boston, Massachusetts 02115, USA.
Phone: 617.919.2530; Fax: 617.919.2772; E-mail: gabriel.kreiman@
1. Felleman DJ, Van Essen DC. Distributed hierarchi-
cal processing in the primate cerebral cortex. Cereb
2. Wandell BA. Foundations Of Vision. Sunderland,
Massachusetts, USA: Sinauer Associates Inc; 1995.
3. Serre T, Kreiman G, Kouh M, Cadieu C, Knoblich
U, Poggio T. A quantitative theory of immediate
visual recognition. Prog Brain Res. 2007;165:33–56.
4. Deco G, Rolls ET. Computational Neuroscience of
Vision. New York, New York, USA: Oxford Univer-
sity Press; 2004.
5. Kreiman G. Single neuron approaches to human
3062?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
vision and memories. Curr Opin Neurobiol.
6. Potter M, Levy E. Recognition memory for a
rapid sequence of pictures. J Exp Psychol. 1969;
7. Liu H, Agam Y, Madsen JR, Kreiman G. Timing,
timing, timing: fast decoding of object informa-
tion from intracranial field potentials in human
visual cortex. Neuron. 2009;62(2):281–290.
8. Thorpe S, Fize D, Marlot C. Speed of pro-
cessing in the human visual system. Nature.
9. vanRullen R, Thorpe S. Surfing a spike wave down the
ventral stream. Vision Res. 2002;42(23):2593–2615.
10. Schyns PG, Oliva A. Flexible, diagnosticity-driven,
rather than fixed, perceptually determined scale
selection in scene and face recognition. Perception.
11. Cepko CL. Genomics approaches to photorecep-
tor development and disease. Harvey Lect. 2001–
12. Kandel E, Schwartz J, Jessell T. Principles of Neural
Science. 4th ed. New York, New York, USA: McGraw-
Hill Medical; 2000.
13. Callaway EM. Local circuits in primary visual cor-
tex of the macaque monkey. Annu Rev Neurosci.
14. Callaway EM. Feedforward, feedback and inhibi-
tory connections in primate visual cortex. Neural
15. Douglas RJ, Martin KA. Neuronal circuits of the
neocortex. Annu Rev Neurosci. 2004;27:419–451.
16. Nassi JJ, Callaway EM. Parallel processing strate-
gies of the primate visual system. Nat Rev Neurosci.
17. Hubel DH, Wiesel TN. Receptive fields and func-
tional architecture of monkey striate cortex.
J Physiol. 1968;195(1):215–243.
18. Haxby J, et al. Dissociation of object and spatial visu-
al processing pathways in human extrastriate cortex.
Proc Natl Acad Sci U S A. 1991;88(5):1621–1625.
19. Mishkin M. A memory system in the monkey. Philos
Trans R Soc Lond B Biol Sci. 1982;298(1089):83–95.
20. Maunsell JHR. The brain’s visual world: represen-
tation of visual targets in cerebral cortex. Science.
21. Born RT, Bradley DC. Structure and function of
visual area MT. Annu Rev Neurosci. 2005;28:157–189.
22. Logothetis NK, Sheinberg DL. Visual object recog-
nition. Annu Rev Neurosci. 1996;19:577–621.
23. Tanaka K. Inferotemporal cortex and object vision.
Annu Rev Neurosci. 1996;19:109–139.
24. Connor CE, Brincat SL, Pasupathy A. Transforma-
tion of shape information in the ventral pathway.
Curr Opin Neurobiol. 2007;17(2):140–147.
25. Miller EK, Cohen JD. An integrative theory of
prefrontal cortex function. Annu Rev Neurosci.
26. Miller E. The prefrontal cortex and cognitive control.
Nat Rev Neurosci. 2000;1(1):59–65.
27. Eichenbaum H. A cortical-hippocampal sys-
tem for declarative memory. Nat Rev Neurosci.
28. Squire LR, Stark CE, Clark RE. The medial tempo-
ral lobe. Annu Rev Neurosci. 2004;27:279–306.
29. Allman JM, McGuinness E. Primate visual cortex.
In: Steklis H., Erwin J, eds. Comparative Primate Biol-
ogy. Volume 4. Neurosciences. New York, New York,
USA: John Wiley & Sons, Inc.; 1988:279–326.
30. Zeki S. A century of cerebral achromatopsia. Brain.
31. Gross CG. How inferior temporal cortex became a
visual area. Cereb Cortex. 1994;4(5):455–469.
32. Dutton GN. Cognitive vision, its disorders and
differential diagnosis in adults and children:
knowing where and what things are. Eye (Lond).
33. Gronqvist S, Flodmark O, Tornqvist K, Edlund G,
Hellstrom A. Association between visual impair-
ment and functional and morphological cerebral
abnormalities in full-term children. Acta Ophthal-
mol Scand. 2001;79(2):140–146.
34. Goodale M, Milner A. Separate visual path-
ways for perception and action. Trends Neurosci.
35. Heilman KM, Valenstein E. Clinical Neuropsychol-
ogy. New York, New York, USA: Oxford University
36. Riddoch G. Dissociation of visual perceptions due
to occipital injuries with special reference to appre-
ciation of movement. Brain. 1917;40:15–57.
37. Holmes G. Disturbances of visual orientation. Br J
38. Weiskrantz L. Blindsight revisited. Curr Opin Neuro-
39. Zeki S, Ffytche DH. The Riddoch syndrome:
insights into the neurobiology of conscious vision.
Brain. 1998;121(pt 1):25–45.
40. Zeki S. Cerebral akinetopsia (visual motion blind-
ness). A review. Brain. 1991;114(pt 2):811–824.
41. Zihl J, von Cramon D, Mai N. Selective disturbance
of movement vision after bilateral brain damage.
Brain. 1983;106(pt 2):313–340.
42. Blanke O, Landis T, Safran AB, Seeck M. Direction-
specific motion blindness induced by focal stimu-
lation of human extrastriate cortex. Eur J Neurosci.
43. Farah MJ, Brunn JL, Wong AB, Wallace MA, Car-
penter PA. Frames of reference for allocating atten-
tion to space: evidence from the neglect syndrome.
44. Driver J, Mattingley JB. Parietal neglect and visual
awareness. Nat Neurosci. 1998;1(1):17–22.
45. Berger A, Posner MI. Pathologies of brain attention-
al networks. Neurosci Biobehav Rev. 2000;24(1):3–5.
46. Bisiach E, Luzzatti C. Unilateral neglect of repre-
sentational space. Cortex. 1978;14(1):129–133.
47. Rizzo M, Vecera SP. Psychoanatomical substrates
of Balint’s syndrome. J Neurol Neurosurg Psychiatry.
48. Holmes G. Disturbances of visual orientation. Br J
49. Klüver H, Bucy PC. Preliminary analysis of the
functions of the temporal lobes in monkeys. Arch-
Neurol Psychiatry. 1939;42(6):979–1000.
50. Mishkin M, Pribram KH. Visual discrimination
performance following partial ablations of the
temporal lobe. I. Ventral vs. lateral. J Comp Physiol
51. Mishkin M. Visual discrimination performance
following partial ablations of the temporal lobe.
II. Ventral surface vs. hippocampus. J Comp Physiol
52. Holmes E, Gross C. Stimulus equivalence after infe-
rior temporal lesions in monkeys. Behav Neurosci.
53. Dean P. Effects of inferotemporal lesions on the
behavior of monkeys. Psychol Bull. 1976;83(1):41–71.
54. De Renzi E. Disorders of visual recognition. Semin
55. Damasio A, Tranel D, Damasio H. Face agnosia
and the neural substrates of memory. Annu Rev
56. Sacks O. The Man Who Mistook His Wife For A Hat
And Other Clinical Tales. New York, New York, USA:
Touchstone Books; 1998.
57. Milders M, Perrett D. Recent developments in the
neuropsychology and physiology of face processing.
Baillieres Clin Neurol. 1993;2(2):361–384.
58. Kanwisher N, Moscovitch M. The cognitive neu-
roscience of face processing: an introduction. In:
Kanwisher N, Moscovitch, eds. The Cognitive Neu-
roscience of Face Processing: A Special Issue of Cognitive
Neuropsychology (Special Issues of Cognitive Neuro-
psychology). Florence, Kentucky, USA: Psychology
59. Buxbaum L, Glosser G, Coslett H. Impaired face
and word recognition without object agnosia.
60. Humphreys G. Object recognition: the man who mis-
took his dog for a cat. Curr Biol. 1996;6(7):821–824.
61. Rossion B, Caldara R, Seghier M, Schuller AM,
Lazeyras F, Mayer E. A network of occipito-tem-
poral face-sensitive areas besides the right middle
fusiform gyrus is necessary for normal face pro-
cessing. Brain. 2003;126(pt 11):2381–2395.
62. Steeves JK, et al. The fusiform face area is not suf-
ficient for face recognition: evidence from a patient
with dense prosopagnosia and no occipital face
area. Neuropsychologia. 2006;44(4):594–609.
63. Le S, et al. Seeing, since childhood, without ven-
tral stream: a behavioural study. Brain. 2002;
64. Dixon M, Bub D, Arguin M. Semantic and visual
determinants of face recognition in a prosopag-
nosic patient. J Cogn Neurosci. 1998;10(3):362–376.
65. Gauthier I, Behrmann M, Tarr M. Can face recogni-
tion really be dissociated from object recognition?
J Cogn Neurosci. 1999;11(4):349–370.
66. Gainotti G. Cognitive and anatomical locus of
lesion in a patient with a category-specific semantic
impairment for living beings. Cognitive Neuropsych.
67. Humphreys G, Riddoch M. Object agnosias. Bail-
lieres Clin Neurol. 1993;2(2):339–359.
68. Magnie M, Ferreira C, Giusiano B, Poncet M. Cat-
egory specificity in object agnosia: preservation of
sensorimotor experiences related to objects. Neuro-
69. Warrington E, McCarthy R. Multiple meaning
systems in the brain - A case for visual semantics.
70. Warrington E, McCarthy R. Categories of knowl-
edge - Further fractionations and an attempted
integration. Brain. 1987;110(pt 5):1273–1296.
71. Warrington E, Shallice T. Category specific seman-
tic impairments. Brain. 1984;107(pt 3):829–854.
72. Karnath HO, Ruter J, Mandler A, Himmelbach M.
The anatomy of object recognition--visual form
agnosia caused by medial occipitotemporal stroke.
J Neurosci. 2009;29(18):5854–5862.
73. Rayner K. Eye movements in reading and informa-
tion processing: 20 years of research. Psychol Bull.
74. Agam Y, et al. Robust selectivity to two-object
images in human visual cortex. Curr Biol.
75. Kuffler S. Discharge patterns and functional
organization of mammalian retina. J Neurophysiol.
76. Hubel D, Wiesel T. Receptive fields of single
neurons in the cat’s striate cortex. J Physiol.
77. Hubel DH, Wiesel TN. Receptive fields, binocular
interaction and functional architecture in the cat’s
visual cortex. J Physiol. 1962;160:106–154.
78. Canny J. A computational approach to edge detec-
tion. IEEE Trans Image Process. 1986;8(5):679–714.
79. Horn BK. Robot Vision. Cambridge, Massachusetts,
USA: The MIT Press; 1986.
80. Livingstone M, Hubel D. Segregation of form,
color, movement and depth: anatomy, physiology
and perception. Science. 1988;240(4853):740–749.
81. Reynolds JH, Chelazzi L. Attentional modula-
tion of visual processing. Annu Rev Neurosci.
82. Desimone R, Duncan J. Neural mechanisms
of selective visual attention. Annu Rev Neurosci.
83. Salzman C, Britten K, Newsome W. Cortical micro-
stimulation influences perceptual judgments of
motion direction. Nature. 1990;346(6280):174–177.
84. DiCarlo J, Maunsell J. Anterior inferotemporal
neurons of monkeys engaged in object recognition
?The?Journal?of?Clinical?Investigation http://www.jci.org Volume 120 Number 9 September 2010
can be highly sensitive to object retinal position.
J Neurophysiol. 2004;89(6):3264–3278.
85. Rolls E. Neural organization of higher visual func-
tions. Curr Opin Neurobiol. 1991;1(2):274–278.
86. Tanaka K. Neuronal mechanism of object recognition.
87. Rolls ET. Neurons in the cortex of the tempo-
ral lobe and in the amygdala of the monkey with
responses selective for faces. Hum Neurobiol. 1984;
88. Young MP, Yamane S. Sparse population coding
of faces in the inferior temporal cortex. Science.
89. Perrett D, Rolls E, Caan W. Visual neurones respon-
sive to faces in the monkey temporal cortex. Exp
Brain Res. 1982;47(3):329–342.
90. Desimone R. Face-selective cells in the temporal
cortex of monkeys. J Cogn Neurosci. 1991;3(1):1–8.
91. Gross C, Bender D, Rocha-Miranda C. Visual recep-
tive fields of neurons in inferotemporal cortex of
the monkey. Science. 1969;166(910):1303–1306.
92. Quian Quiroga R, Reddy L, Kreiman G, Koch
C, Fried I. Invariant visual representation by
single neurons in the human brain. Nature. 2005;
93. Schwartz E, Desimone R, Albright T, Gross C.
Shape-recognition and inferior temporal neurons.
Proc Natl Acad Sci U S A. 1983;80(18):5776–5778.
94. Miyashita Y, Chang HS. Neuronal correlate of pic-
torial short-term memory in the primate temporal
cortex. Nature. 1988;331(6151):68–71.
95. Richmond BJ, Optican LM, Spitzer H. Tempo-
ral encoding of two-dimensional patterns by
single units in primate primary visual cortex.
I. Stimulus-response relations. J Neurophysiol.
96. Logothetis NK, Pauls J. Psychophysical and
physiological evidence for viewer-centered object
representations in the primate. Cereb Cortex.
97. Kobatake E, Tanaka K. Neuronal selectivities to
complex object features in the ventral visual path-
way of the macaque cerebral cortex. J Neurophysiol.
98. Gawne TJ, Richmond BJ. How independent are the
messages carried by adjacent inferior temporal cor-
tical neurons? J Neurosci. 1993;13(7):2758–2771.
99. Fujita I, Tanaka K, Ito M, Cheng K. Columns for
visual features of objects in monkey inferotempo-
ral cortex. Nature. 1992;360(6402):343–346.
100. Hung CP, Kreiman GK, Quiroga R, Kraskov A, Pog-
gio T, DiCarlo JJ. Using ‘read-out’ of object identity
to understand object coding in the macaque ante-
rior inferior temporal cortex. Computational and
Systems Neuroscience (COSYNE), Salt Lake City,
Utah, USA, March, 2005.
101. Ito M, Tamura H, Fujita I, Tanaka K. Size and
position invariance of neuronal responses in
monkey inferotemporal cortex. J Neurophysiol.
102. Hung C, Kreiman G, Poggio T, DiCarlo J. Fast read-
out of object identity from macaque inferior tem-
poral cortex. Science. 2005;310(5749):863–866.
103. Sary G, Vogels R, Orban GA. Cue-invariant shape
selectivity of macaque inferior temporal neurons.
104. Engel AK, Moll CK, Fried I, Ojemann GA. Invasive
recordings from the human brain: clinical insights
and beyond. Nat Rev Neurosci. 2005;6(1):35–47.
105. Kreiman G, Koch C, Fried I. Category-specific visu-
al responses of single neurons in the human medial
temporal lobe. Nature Neurosci. 2000;3(9):946–953.
106. Allison T, Puce A, Spencer D, McCarthy G. Elec-
trophysiological studies of human face percep-
tion. I: Potentials generated in occipitotemporal
cortex by face and non-face stimuli. Cereb Cortex.
107. Serre T, Oliva A, Poggio T. A feedforward architec-
ture accounts for rapid categorization. Proc Natl
Acad Sci U S A. 2007;104(15):6424–6429.
108. Deco G, Rolls ET. A neurodynamical cortical model
of visual attention and invariant object recogni-
tion. Vision Res. 2004;44(6):621–642.
109. Hegde J, Van Essen DC. Strategies of shape repre-
sentation in macaque visual area V2. Vis Neurosci.
110. Hegde J, Van Essen DC. A comparative study of
shape representation in macaque visual areas v2
and v4. Cereb Cortex. 2007;17(5):1100–1116.
111. Pasupathy A, Connor CE. Population coding of
shape in area V4. Nat Neurosci. 2002;5(12):1332–8.
112. Brincat S, Connor C. Dynamic shape synthesis
in posterior inferior temporal cortex. Neuron.
113. McKillop E, Dutton GN. Impairment of vision in
children due to damage to the brain: a practical
approach. Br Ir Orthop J. 2008;5:8–14.
114. Logothetis N. Can current fMRI techniques reveal
the microarchitecture of cortex? Nature Neurosci.
115. Logothetis NK. The neural basis of the blood-oxy-
gen-level-dependent functional magnetic reso-
nance imaging signal. Philos Trans R Soc Lond B Biol
116. Logothetis NK. The underpinnings of the BOLD
functional magnetic resonance imaging signal.
J Neurosci. 2003;23(10):3963–3971.
117. Logothetis NK. What we can do and what we cannot
do with fMRI. Nature. 2008;453(7197):869–878.
118. Mukamel R, Gelbard H, Arieli A, Hasson U, Fried I,
Malach R. Coupling between neuronal firing, field
potentials, and FMRI in human auditory cortex.
119. Tsao DY, Freiwald WA, Tootell RB, Livingstone MS.
A cortical region consisting entirely of face-selec-
tive cells. Science. 2006;311(5761):670–674.
120. Maier A, Wilke M, Aura C, Zhu C, Ye FQ, Leopold
DA. Divergence of fMRI and neural signals in V1
during perceptual suppression in the awake mon-
key. Nat Neurosci. 2008;11(10):1193–1200.
121. Leopold DA. Neuroscience: pre-emptive blood
flow. Nature. 2009;457(7228):387–388.
122. Sirotin YB, Das A. Anticipatory haemodynamic
signals in sensory cortex not predicted by local
neuronal activity. Nature. 2009;457(7228):475–479.
123. Hasson U, Yang E, Vallines I, Heeger DJ, Rubin N. A
hierarchy of temporal receptive windows in human
cortex. J Neurosci. 2008;28(10):2539–2550.
124. Logothetis NK, Wandell BA. Interpreting the
BOLD signal. Annu Rev Physiol. 2004;66:735–769.
125. Haxby J, Hoffman E, Gobbini M. The distributed
human neural system for face perception. Trends
Cogn Sci. 2000;4(6):223–233.
126. Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten
JL, Pietrini P. Distributed and overlapping repre-
sentations of faces and objects in ventral temporal
cortex. Science. 2001;293(5539):2425–2430.
127. Ishai A, Ungerleider LG, Martin A, Schouten JL,
Haxby JV. Distributed representation of objects in
the human ventral visual pathway. Proc Natl Acad
Sci U S A. 1999;96(16):9379–9384.
128. Grill-Spector K, Malach R. The human visual cortex.
Annu Rev Neurosci. 2004;27:649–677.
129. Epstein R, Kanwisher N. A cortical representa-
tion of the local visual environment. Nature.
130. Kanwisher N, McDermott J, Chun MM. The fusi-
form face area: a module in human extrastriate
cortex specialized for face perception. J Neurosci.
131. Tong F, Nakayama K, Moscovitch M, Weinrib
O, Kanwisher N. Response properties of the
human fusiform face area. Cognitive Neuropsych.
132. Foldiak P. Learning invariance from transforma-
tion sequences. Neural Comput. 1991;3(2):194–200.
133. Mel B. SEEMORE: combining color, shape and tex-
ture histogramming in a neurally inspired approach
to visual object recognition. Neural Comput.
134. Riesenhuber M, Poggio T. Hierarchical mod-
els of object recognition in cortex. Nat Neurosci.
135. Reagan D. Human Perception of Objects: Early Visual
Processing of Spatial Form Defined by Luminance, Color,
Texture, Motion, and Binocular Disparity. Sunderland,
Massachusetts, USA: Sinauer Associates Inc; 2000.
136. Dayan P, Abbott L. Theoretical Neuroscience: Computa-
tional and Mathematical Modeling of Neural Systems. Cam-
bridge, Massachusetts, USA: The MIT Press; 2001.
137. Kreiman G. On the Neuronal Activity in the Human
Brain during Visual Recognition, Imagery and Binocular
Rivalry [dissertation]. Pasadena, California, USA:
California Institute of Technology; 2002.
138. Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I.
Invariant visual representation by single neurons in
the human brain. Nature. 2005;435(7045):1102–1107.