ArticlePDF Available

Segregation of object and background motion in the retina


Abstract and Figures

An important task in vision is to detect objects moving within a stationary scene. During normal viewing this is complicated by the presence of eye movements that continually scan the image across the retina, even during fixation. To detect moving objects, the brain must distinguish local motion within the scene from the global retinal image drift due to fixational eye movements. We have found that this process begins in the retina: a subset of retinal ganglion cells responds to motion in the receptive field centre, but only if the wider surround moves with a different trajectory. This selectivity for differential motion is independent of direction, and can be explained by a model of retinal circuitry that invokes pooling over nonlinear interneurons. The suppression by global image motion is probably mediated by polyaxonal, wide-field amacrine cells with transient responses. We show how a population of ganglion cells selective for differential motion can rapidly flag moving objects, and even segregate multiple moving objects.
Transient excitation and inhibition are synchronous during coherent motion, causing suppression of firing.a, Schematic proposal for the inputs to an OMS ganglion cell: excitation from motion in the receptive field centre, and inhibition from motion in the periphery. Both consist of transient events and are triggered by the same motion features. Under coherent motion they coincide in time, but under incoherent motion they are uncorrelated. b, Average firing rate of Fast OFF ganglion cells (GCs) as the jitter trajectories of the object and background regions are shifted in time relative to each other (thick red line). Firing rates were averaged over 5-min stimulus presentations, normalized to the cell's average firing rate under the Object Only condition, then averaged over five neurons. Average membrane potential of polyaxonal amacrine cells (AC Vm) during global (Eye Only) jitter, as a function of time before or after a ganglion cell spike in the Object Only condition using the same trajectory (thin blue line). Each amacrine cell's membrane potential was normalized by subtracting its mean and dividing by its standard deviation, which was 4 1 mV (mean s.e.m.; n = 3); note inverted axis, depolarization is downward. c, Membrane potential of a polyaxonal amacrine cell in response to coherent motion (top; Eye Only condition). Spiking response of a salamander Fast OFF cell to motion in the Object Only condition, using the same trajectory as for the amacrine cell (bottom). The amacrine and ganglion cells were recorded in different retinas. d, Vertical projection of the confocal image of the amacrine cell in c, superimposed on its receptive field (left). Receptive field of the ganglion cell in c, on the same scale (right).
Content may be subject to copyright.
Segregation of object and background
motion in the retina
Bence P. O
lveczky*†, Stephen A. Baccus & Markus Meister
* Division of Health Sciences and Technology, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, Massachusetts 02138, USA
Program in Neuroscience, Harvard University, 220 Longwood Avenue, Boston, Massachusetts 02115, USA
Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, Massachusetts 02138, USA
An important task in vision is to detect objects moving within a stationary scene. During normal viewing this is complicated
by the presence of eye movements that continually scan the image across the retina, even during fixation. To detect moving
objects, the brain must distinguish local motion within the scene from the global retinal image drift due to fixational eye
movements. We have found that this process begins in the retina: a subset of retinal ganglion cells responds to motion in the
receptive field centre, but only if the wider surround moves with a different trajectory. This selectivity for differential motion is
independent of direction, and can be explained by a model of retinal circuitry that invokes pooling over nonlinear interneurons. The
suppression by global image motion is probably mediated by polyaxonal, wide-field amacrine cells with transient responses. We
show how a population of ganglion cells selective for differential motion can rapidly flag moving objects, and even segregate
multiple moving objects.
Movements of the eye are a fundamental component of vision, as
they directly influence the stimulus falling on the retina. There are
two main types of eye movement: the large and rapid saccades
or pursuit movements by which we redirect our gaze, and the
smaller fixational eye movements that occur between saccades
Whereas ballistic gaze-shifting eye movements suppress vision
small fixational eye movements are essential for seeing: if the retinal
image is stabilized, visual perception fades within a tenth of a
During fixation, the retinal image drifts over about 0.58 of visual
angle, or about 60 cone receptive fields, at an average speed of
approximately 0.5 degrees s
, and any processing of visual infor-
mation must occur on the background of this drifting motion
Similar eye movements occur in other animals, including salaman-
and rabbit
. Despite their importance to vision, the effect of
these eye movements on retinal function has received rather limited
Given the presence of continuous eye movements, the funda-
mental task of detecting object motion within a scene becomes a
significant computational problem. The task is not simply to detect
motion on the retina; rather, the visual system must discriminate
between local motion patterns specific to an object and global
motion induced by fixational eye movements
. Humans perceive
this task as effortless: movements anywhere within a scene immedi-
ately ‘pop-out’ and attract our attention
, even if their velocity and
amplitude is only a fraction of the image motion caused by
Figure 1 Simulating local object motion on the retina in the presence of fixational eye movements. a, Receptive field profile of a rabbit ON Brisk Transient retinal ganglion cell (left; see
Methods). A stripe grating representing an object was projected in and around the cell’s receptive field centre, while the remainder of the retina was presented with a background grating
(right). b, Trajectory of vertical fixational eye movements in an unrestrained rabbit, acquired using a scleral search coil
(left). The right panel shows a sample of the random walk
trajectory used to jitter the gratings.
NATURE | VOL 423 | 22 MAY 2003 | 401
© 2003
fixational eye movements
. A neuron that subserves this function
should respond to local motion on the retina, but only if the motion
trajectory differs from that in a large surrounding region. Neurons
capable of such computations have been described in the visual
and superior colliculus
of mammals, and also in the
optic tectum of birds
. However, given that the fixational eye
movements in the two eyes differ
, extraction of object motion
probably happens before the visual pathways from the two eyes
merge. Here we show that the segregation of object motion and
image motion induced by eye movements happens in the retina.
The retina senses differential motion
We recorded the spike trains of ganglion cells in the isolated retina of
salamander and rabbit. The stimulus display was divided into a
small ‘object’ region overlying the receptive field centre of the
ganglion cell and a surrounding large ‘background’ region covering
the rest of the retina (Fig. 1a). Both object and background were
given a visual texture by a simple stripe grating. The background
grating jittered laterally with a random walk trajectory, similar to
that of fixational eye movements (Fig. 1b). The object grating also
jittered in a random walk with the same statistics, either coherently
with the background, or incoherently with a different trajectory.
The coherent condition simulated the global image motion on the
retina that results when viewing a stationary scene in the presence of
eye movements only (‘Eye Only’ condition). The incoherent con-
dition simulated, in addition, local motion of an object within that
scene (‘Eye þ Object’ condition).
In both the salamander and rabbit retina we found ganglion cells
that were highly selective for motion within the scene (Fig. 2): these
neurons responded vigorously to the Eye þ Object condition
(Fig. 2a), but were almost completely suppressed under the Eye
Only condition (Fig. 2b), even though their receptive field centres
experienced the same stimulus under both conditions. When the
background region was uniformly grey (‘Object Only’, Fig. 2c), the
responses were similar to the Eye þ Object condition (Fig. 2a),
indicating that an incoherently moving background has little effect
on the centre response. Whereas the stimulus condition in Fig. 2a
simulated an object jittering within the scene, a steady drift of the
object relative to the background also elicited reliable responses
(‘Eye þ Object Drift’, Fig. 2d).
As these ganglion cells are selective for local object motion over
global motion, we will refer to them as OMS (object motion
sensitive) cells, noting that this class comprises several recognized
cell types (for example, ‘ON Brisk Transient’ cells and ‘ON–OFF
Direction Selective’ cells in rabbit, and ‘Fast OFF’ cells in salaman-
der; Fig. 2e). Both retinas contain other cell types that show much
Figure 2 Certain retinal ganglion cells are selective for object motion. ad, Responses to
15 s of jitter from a rabbit ON Brisk Transient cell and a salamander Fast OFF cell. Each
panel shows a raster plot with spikes on eight identical stimulus trials (top) and a peri-
stimulus time histogram of the firing rate averaged over all trials (bottom). The stimulus
conditions are: Eye þ Object (a), object and background gratings jittered incoherently;
Eye Only (b), object and background jittered coherently with same trajectory as the
object in a; Object Only (c), object jittered as in a, background grey; Eye þ Object Drift (d),
object and background jittered as in b with a steady drift (450
) added to the
object region. e, Ratio of firing rates in the coherent (b) and incoherent (a) motion
condition. Data from 6 ON Brisk Transient (ON BT) cells, 11 ON–OFF Direction Selective
(ON–OFF DS) cells, 5 Local Edge Detector (LED) cells, and 7 OFF Brisk Transient (OFF BT)
cells in two rabbit retinas; 41 Fast OFF cells, 8 Weak OFF cells and 8 ON cells in nine
salamander retinas.
NATURE | VOL 423 | 22 MAY 2003 |
© 2003
smaller, if any, difference between the coherent and incoherent
motion conditions (for example, ‘OFF Brisk Transient’ cells in
rabbit and ‘ON’ cells in salamander, Fig. 2e). Thus, the selectivity
for object motion may be a special feature of a few of the parallel
pathways that convey retinal output to the brain.
Mechanism of suppression from coherent motion
OMS ganglion cells are excited by motion in or near the receptive
field centre, but are suppressed if the same image motion extends
over the wider surround. We measured the extent of these antago-
nistic regions by gradually increasing the size of the object, while
keeping the trajectories in the object and background regions
different (Fig. 3a). For salamander Fast OFF cells, the firing rate
increased up to an object radius of approximately 250
m: this is the
extent of the region excited by grating motion and corresponds very
well to the classic receptive field centre as measured by flashing spots
of increasing size (Fig. 3b). As the object grew in size, it began to
invade the suppressive surround region, and the response gradually
decreased out to radii of about 1,000
m. Thus, the suppressive
effect of coherent motion extends over at least 1 mm on the retina.
Applying strychnine largely blocked the antagonistic effect of
coherent surround motion, suggesting that it is caused by long-
range glycine-mediated inhibition, presumably from wide-field
amacrine cells
Ganglion cells can be suppressed by peripheral motion
However, this alone does not explain our findings, as the amount
of motion in the background region was identical for both the
Eye þ Object and the Eye Only conditions, and for all the stimuli in
Fig. 3a. As seen in Fig. 2c, the OMS cells respond to a jittering object
over the receptive field centre with brief bursts of spikes that are
precisely timed to the motion trajectory. We hypothesized that
inhibition from peripheral motion arrives in similar brief pulses
that have the same dependence on the motion trajectory as the
excitatory events from the centre (Fig. 4a). Under stimulation with
coherent motion, the peripheral inhibition would coincide with the
excitation from the centre and suppress the cells response. In
response to object motion, when the object and background regions
jitter incoherently, inhibition would arrive with random timing
Figure 3 Spatial interactions that produce the sensitivity to object motion. a, Relative
firing rate of salamander Fast OFF cells (n ¼ 5) as a function of object size in the
Eye þ Object condition. b, Relative firing rate of the same cells to a 1-Hz flashing spot of
increasing size. The black trace shows the effect of 10
M strychnine (STR). Firing rates
were averaged over 2 min of stimulation and normalized to the maximum firing rate of
each cell.
Figure 4 Transient excitation and inhibition are synchronous during coherent motion,
causing suppression of firing. a, Schematic proposal for the inputs to an OMS ganglion
cell: excitation from motion in the receptive field centre, and inhibition from motion in the
periphery. Both consist of transient events and are triggered by the same motion features.
Under coherent motion they coincide in time, but under incoherent motion they are
uncorrelated. b, Average firing rate of Fast OFF ganglion cells (GCs) as the jitter
trajectories of the object and background regions are shifted in time relative to each other
(thick red line). Firing rates were averaged over 5-min stimulus presentations, normalized
to the cell’s average firing rate under the Object Only condition, then averaged over five
neurons. Average membrane potential of polyaxonal amacrine cells (AC V
) during global
(Eye Only) jitter, as a function of time before or after a ganglion cell spike in the Object Only
condition using the same trajectory (thin blue line). Each amacrine cell’s membrane
potential was normalized by subtracting its mean and dividing by its standard deviation,
which was 4 ^ 1 mV (mean ^ s.e.m.; n ¼ 3); note inverted axis, depolarization is
downward. c, Membrane potential of a polyaxonal amacrine cell in response to coherent
motion (top; Eye Only condition). Spiking response of a salamander Fast OFF cell to motion
in the Object Only condition, using the same trajectory as for the amacrine cell (bottom).
The amacrine and ganglion cells were recorded in different retinas. d, Vertical projection
of the confocal image of the amacrine cell in c, superimposed on its receptive field (left).
Receptive field of the ganglion cell in c, on the same scale (right).
NATURE | VOL 423 | 22 MAY 2003 | 403
© 2003
relative to the excitation and thus be ineffective. To test this idea, we
used the same jitter trajectory for the object and background
regions, but shifted them with respect to each other in time. As
predicted, the suppressive effect was limited to a very brief time
window around zero delay (Fig. 4b, thick red trace). This suggests
that peripheral inhibition indeed arrives in brief pulses approxi-
mately 100 ms wide, triggered by the same motion features as the
excitatory events from the centre.
We searched, using intracellular recordings in the salamander
retina, for interneurons that might mediate this long-range inhi-
bition. We encountered a type of amacrine cell that responded to
coherent jitter (Eye Only) with sharp depolarizations, about 100 ms
wide, often carrying action potentials (Fig. 4c). These depolarizing
events in amacrine cells aligned perfectly with the excitatory inputs
to OMS cells, as marked by the bursts of spikes produced in the
Object Only condition (Fig. 4c). If such amacrine cells inhibit the
OMS ganglion cells, this could explain the suppression of firing
under coherent motion. By calculating the average amacrine cell
membrane potential relative to the time of a ganglion cell spike, we
predicted how this inhibition should depend on the time delay
between object and background motion. Figure 4b shows that the
time course of the amacrine cell membrane potential nicely predicts
the measured time course of ganglion cell suppression.
These amacrine cells had visual receptive fields of about 150
radius (Fig. 4d), probably mediated by inputs on a small field of
dendrites near the soma
(see Supplementary Fig. S1). Several long
output processes extended .1 mm across the retina (Fig. 4d). A
ganglion cell collecting inhibitory input from these amacrine cell
processes will be suppressed whenever the motion in the distant
periphery matches the motion over the ganglion cell’s receptive field
centre. Thus both the anatomy and the physiology of these amacrine
cells are consistent with their being the source of inhibition onto
OMS cells. Amacrine cells with a similar polyaxonal morphology are
found in other species including rabbit
and macaque
Object motion selectivity is independent of spatial pattern
For a neuron to be selective for object motion under natural viewing
conditions, the timing of the excitation and inhibition should
ideally depend only on the motion trajectory, and be largely
independent of the spatial pattern of the stimulus. To test this we
changed both the spatial phase and frequency of the object grating,
while keeping the background uniformly grey. Indeed, the time
Figure 5 The response of OMS cells is largely independent of the spatial pattern.
a, Responses of a salamander polyaxonal amacrine cell (top) and a Fast OFF ganglion cell
(bottom) to a jittering grating (08) and its contrast-reversed version (1808). b, Responses of
a different Fast OFF ganglion cell to a grating of varying spatial period. c, The average time
course of the retinal speed of the jittered grating before a spike for a salamander
polyaxonal amacrine cell (Eye Only condition) and three different types of OMS cells
(Object Only condition): the Fast OFF cell in salamander, and the ON Brisk Transient and
ON–OFF Direction Selective cells in rabbit. The time-averaged speed for the entire
stimulus is marked by the horizontal line.
NATURE | VOL 423 | 22 MAY 2003 |
© 2003
course of firing of OMS cells did not depend on the spatial phase of
the grating: even a 1808 phase shift, corresponding to a complete
reversal of black and white bars, did not significantly alter the cell’s
response (Fig. 5a). Nor did such a phase reversal alter the response
of polyaxonal amacrine cells (Fig. 5a). Varying the spatial fre-
quency of the grating also had little effect on the firing pattern of
the ganglion cells (Fig. 5b), except for very fine gratings with
period ,40
m. In that limit, the firing rate declined and the spike
bursts were triggered at different time points in the motion
trajectory. Polyaxonal amacrine cell responses were similarly
robust to changes in spatial frequency (data not shown). These
properties differed from those of other amacrine cell types. For
example, the responses of sustained OFF-type amacrine cells to
coherent motion (Eye Only) were uncorrelated with the excitatory
inputs to OMS ganglion cells, and strongly depended on the phase
and spatial frequency of the jittered grating (Supplementary
Fig. S2).
Whereas the response of OMS cells is largely independent of the
spatial pattern, it is almost completely determined by the motion
trajectory (Fig. 5a, b). We calculated the average image speed
before a spike for three types of OMS cells, and for salamander
polyaxonal amacrine cells, in response to a jittered object. The
average motion feature that triggered spikes was an acceleration of
the grating after a period of slower than average speed (Fig. 5c).
For most OMS cells this stimulus was effective regardless of the
direction of motion; however, the rabbit ON–OFF Direction
Selective cell responded only when the grating accelerated in its
preferred direction.
Increasing or decreasing the speed of the jitter by a factor of two
did not significantly alter the shape of the preferred speed profile;
neither did it alter the OMS cells’ sensitivity to differential motion
(not shown). This suggests that the function of OMS cells is robust
to changes in the statistics of eye movements, which accompany
changes in the behavioural state of the animal
. Note also that the
preferred motion feature for the polyaxonal amacrine cell is
remarkably similar to that which excites the receptive field centres
of OMS ganglion cells. This suggests that an inhibitory network
involving a single type of amacrine cell could serve to suppress
different types of OMS ganglion cells in the same retina.
A model explaining object motion selectivity
The fact that these cells respond to gratings much finer than the
receptive field centre, and independently of the phase of the grating,
is reminiscent of the Y-type ganglion cells found in cat
and many
other mammals. It is thought that the Y-cell pools excitation from
many small subunits in its receptive field, each of which contributes
a rectified response
. We implemented a concrete version of this
idea (Fig. 6a) and simulated how it would respond to the jittered
grating stimulus in the Object Only condition. With just two free
parameters, this simple model reproduced with good accuracy the
timing of ganglion cell firing in both salamander and rabbit retina
(Fig. 6b).
The model’s response is independent of the direction of motion
and the phase of the grating, because the receptive field centre
contains subunits arranged symmetrically in all directions and at all
possible phases relative to the grating. Similarly, the response is
largely independent of spatial frequency, as long as the bars of the
grating are larger than the subunit width. When the stripes become
smaller, the models output declines, and a comparison to the
observed responses (Fig. 5b) suggests a minimum subunit width
of 20–40
m. It has been proposed that bipolar cells form the
nonlinear subunits of ganglion cell receptive fields
. We recorded
intracellularly from bipolar cells in the salamander retina. Their
receptive field centres measured 35–120
m(n ¼ 7) in width (data
not shown), in agreement with the dendritic field size of salamander
bipolar cells
, and consistent with their role as nonlinear subunits.
The model in Fig. 6a includes an inhibitory network that pools
over many nonlinear subunits in the periphery. The fact that the
excitatory and inhibitory networks receive input from the same type
of subunits results in a selectivity for differential motion between
Figure 6 A model of retinal processing that accounts for differential motion sensitivity.
a, The OMS ganglion cell (G) receives additive excitatory input from many nonlinear
subunits underlying the object region. It is also inhibited by amacrine cells (A) that pool
over similar nonlinear subunits underlying the background. Each subunit (right) pools light
over a small receptive field, passes the result through a temporal filter, and rectifies the
result above a threshold (arrow). b, Simulated responses to a jittered grating using the
model in a, compared to the responses of real OMS cells to the same jitter (see Methods).
The stimulus trajectory was the same for both salamander and rabbit cells.
NATURE | VOL 423 | 22 MAY 2003 | 405
© 2003
the receptive field centre and the surround. This is accomplished
without explicitly computing and comparing the motion vectors in
the two regions, contrary to what was found for motion contrast
sensors in higher visual centres
. In fact, the behaviour of this
model is not in any way dependent on the direction of motion, only
on the speed. To test this prediction, we performed an experiment in
which the object and background regions jittered with exactly
opposite trajectories, and found that the recorded responses from
OMS cells were indeed suppressed, just as under the coherent
motion condition (Supplementary Fig. S3). Although this situation
represents a departure from the ideal of differential motion detec-
tion, it rarely arises in nature, because the relative motion of an
object within the scene would have to mimic the observer’s eye
Among the rabbit Brisk Transient cells, only the ON-type is
suppressed by global motion (Fig. 2e); presumably the OFF-type
receives different inhibitory input. For the purpose of detecting
object motion it is not essential to have OMS properties in both
ON- and OFF-type Brisk Transient cells. Owing to the nonlinear
spatial summation, the model of Fig. 6a predicts that an OMS cell
would have the same response to a moving pattern whether its
subunits are ON-type or OFF-type. This prediction was confirmed
experimentally, as the rabbit ON-type OMS cell and the salamander
OFF-type OMS cell red at similar times during the motion
trajectory (see Figs 2a–d, 5c and 6b).
Motion pop-out and binding
Finally, we consider how a population of OMS cells represents a
visual scene composed of several objects. For this, the stimulus
included two objects moving with different trajectories on an
incoherently jittering background (Fig. 7a). We recorded the
response of many Fast OFF ganglion cells in the salamander retina
at more than 200 positions relative to the stimulus display. Figure 7b
shows a map of the firing rate in this population. In the region
covered by the two moving objects, ganglion cells fired vigorously.
In the background region, the cells were suppressed, because they
experienced coherent motion between their receptive field centres
and the wider periphery. Thus, a population of OMS cells could
support the perceptual motion pop-out’ effect, which flags local
motion within the scene and attracts our visual attention. This pop-
out would involve only a short delay: in salamander, the median
spike latency of these cells from the onset of object motion was
about 230 ms (data not shown).
Sudden movement of an image on the retina is known to
synchronize firing in multiple ganglion cells
. Inspection of the
spike trains in the two-object experiment showed that OMS cells
covering the same object indeed fired in synchrony (Fig. 7c, d),
whereas OMS cells seeing differently moving objects were uncorre-
lated (Fig. 7d). This is expected from the model of Fig. 6a, as the
receptive field centres of cells covering the same object experience
the same trajectory, and consequently the same speed time course.
Because the firing responses are sparse, two neurons belonging to
differently moving objects will only rarely fire together by chance.
Thus, a group of OMS cells with persistent coincident activity can
define a moving object, regardless of its visual pattern or its exact
motion trajectory. Segregation of objects based on motion cues is a
well-described perceptual phenomenon
. It has been suggested that
synchronous firing is the tag that ‘binds’ neurons together in an
assembly to represent a visual object
. The circuitry of Fig. 6a is a
candidate for the underlying neural mechanism.
Our experiments involved the retinas of rabbit and salamander, but
the essential building blocks required for OMS cells are present in
many other species, including primates. Approximately 20% of the
magnocellular ganglion cells in the primate retina show nonlinear
spatial summation similar to that of our model (Fig. 6a)
Transient, polyaxonal amacrine cells as in Fig. 4d with narrow
dendritic and receptive fields and large axonal aborizations have
also been found in the macaque retina
. Thus, it is probable that
ganglion cells with OMS properties exist in many species, including
humans. These cells serve as an information channel for object
motion and may support diverse functions such as segregating
object from background, and directing the gaze towards moving
One would predict that inadvertent stimulation of OMS neurons
Figure 7 Pop-out of moving objects in a population of OMS ganglion cells. a, Schematic of
the stimulus: two object regions (red and green, each 800
m in diameter) are moving
with different trajectories on an incoherently jittering background. Encircled numbers
denote the receptive field positions of the cells in c. b, Map of the firing rate in a population
of salamander Fast OFF ganglion cells responding to the stimulus in a (see Methods).
c, Segment of the spike trains from two pairs of cells covering two differently moving
objects: 1 and 2 respond to the object on the left (red), whereas 3 and 4 respond to the
object on the right (green). d, Cross-correlation function between the spike trains of two
cells responding to either the same object (1,2; thick brown line), or differently moving
objects (1,3; thin line), normalized by the product of the two cells’ mean firing rates.
NATURE | VOL 423 | 22 MAY 2003 |
© 2003
should evoke an erroneous perception of object motion. We
propose that this happens when viewing certain geometrical pat-
terns, such as the Ouchi illusion
(Fig. 8). Owing to the unusual
geometry of the display, retinal motion of the pattern is mainly
governed by vertical eye movements in the periphery, and by
horizontal eye movements in the centre. Because the eye executes
horizontal and vertical eye movements independently
, centre and
periphery experience different image motion
analogous to the
Eye þ Object stimulus (Fig. 2a). This triggers the OMS cells to
signal an apparent movement of the circle. Although this is an
example of a spatial pattern influencing the response of OMS cells, it
should be noted that the geometric peculiarities required for this
effect are rare in the natural environment, where retinal function
We have shown that certain retinal ganglion cells with nonlinear
spatial summation have a special role in the presence of fixational
eye movements: such a neuron signals when an object in its
receptive field centre moves relative to the background, but is
almost completely suppressed when the object moves together
with the background. This selectivity for object motion in any
direction is accomplished by a rather simple mechanism with three
crucial ingredients: excitation from many rectified subunits in the
receptive field centre; inhibition from the same type of subunits in a
wide surround; and the random nature of fixational eye movements
that produces transient and sparse activation of both the excitatory
and inhibitory networks. A
Retinas of larval tiger salamanders, and Dutch belted and New Zealand white rabbits were
isolated in darkness and superfused with oxygenated Ringer’s medium at room
temperature (salamander) or Ames’ solution at 36 8C (rabbit). A piece of retina, 6–8 mm
on a side, was placed with the ganglion cell layer facing down on a multi-electrode array,
which recorded spike trains simultaneously from many ganglion cells, as described
. For intracellular recordings from salamander
, sharp microelectrodes were
filled with 2 M potassium acetate and 4% neurobiotin, having a final impedance of
150–250 MQ. Polyaxonal amacrine cell resting membrane potentials ranged from 250 to
275 mV, and their peak responses to jittered gratings were 17 ^ 4 mV (mean ^ s.e.m.;
n ¼ 7) in amplitude. To analyse amacrine cell spiking (Fig. 5c), the action potentials were
detected by setting a threshold for the derivative of the membrane potential. After
recording, cells were filled iontophoretically (1–5 nA pulses, about 10–15 min), stained
with 5
streptavidin Alexa-488 (Molecular Probes), and imaged using a confocal
microscope with a £40 oil immersion objective.
Visual stimuli were projected from a computer monitor onto the photoreceptor layer, as
. All experiments used a mean photopic intensity of 8 mWm
. Unless
otherwise stated, the jittered grating consisted of black and white bars with a periodicity of
m. The jitter trajectory was generated by stepping the grating randomly in one
dimension every 15 ms with a step size of 6.7
m. The seeds for generating the random
walk in the object and background regions were the same or different for the Eye Only and
Eye þ Object conditions, respectively. The object region, 800
m in diameter, was
separated from the background region, measuring 4,300 £ 3,200
m, by a 67-
m grey
annulus, except for the experiment in Fig. 3a, where no annulus was present. For Fig. 3b
the stimulus was a spot of varying size flashing from black to white at 1 Hz on a grey
background. In experiments with different stimulus conditions, the trials were interleaved.
Receptive field mapping
The spatio-temporal receptive fields of all neurons were measured by reverse correlation to
a flickering black-and-white checkerboard stimulus
. The spatio-temporal receptive field
was approximated as the product of a spatial profile and a temporal filter. The receptive
field centre of a ganglion cell was estimated as the region where the spatial profile was
larger than one-third of its maximum value. The diameter of the receptive field centre of
amacrine and bipolar cells was approximated by the full-width at half-maximum of the
two-dimensional gaussian function that best fit the spatial profile.
Cell types
Retinal ganglion cells appear in distinct functional types. For salamander, we classified
them on the basis of their spatio-temporal receptive fields
. We report on the responses of
the Fast OFF (,60% of recorded cells), Weak OFF (,12%) and ON cells (,12%). Rabbit
ganglion cells were classified on the basis of the spatio-temporal receptive field and the
spike train’s autocorrelation function, following the criteria of ref. 42. For rabbit, we report
on data from ON–OFF Direction Selective cells (,30% of recorded cells), OFF Brisk
Transient cells (,20%), ON Brisk Transient cells (,15%) and Local Edge Detectors
(,15%). Other cell types in rabbit were encountered rarely, and are not reported.
Only ganglion cells with receptive field centres enclosed by the object region were included
in the analyses, except for Fig. 3, where only cells with receptive fields concentric with the
object region were included. Error bars in figures denote standard error, derived from
variation among cells. For calculating the average speed profiles in Fig. 5c, the jitter
trajectory was smoothed using a 30-ms box filter. During the experiments for Fig. 7b, the
object regions were shifted in increments of 530
m along both the horizontal and vertical
dimensions. Responses from 11 salamander Fast OFF cells were analysed, each sampled at
20 independent positions of the stimulus. Each cell’s firing rate was normalized with
respect to its firing rate under the Eye Only condition. In Fig. 7b, the image value at a given
point is the average normalized firing rate of all cells whose receptive field centre contained
that point. The results were mirrored on the axis of symmetry in the stimulus, and
smoothed using a two-dimensional gaussian filter (standard deviation of 70
For the simulations in Fig. 6, the subunit width (full width at half maximum of a parabolic
profile) was chosen as 42
m for both salamander and rabbit, within the measured range of
bipolar cell receptive fields; simulations were robust to changes in this parameter. The
waveform of the temporal filter was measured by reverse correlation of ganglion cell spikes
to a flicker stimulus
. The threshold was set so that the subunit outputs were non-zero
2.5% (salamander) or 3.5% (rabbit) of the time. The subunits were centred 6.7
effectively sampling all possible phases relative to the stimulus grating. The region covered
by the excitatory subunits was chosen larger than one grating period.
Received 20 December 2002; accepted 18 March 2003; doi:10.1038/nature01652.
Published online 11 May 2003.
1. Yarbus, A. L. Eye Movements and Vision (Plenum, New York, 1967).
2. Kowler, E. Eye Movements and their Role in Visual and Cognitive Processes (Elsevier, New York, 1990).
3. Ross, J., Morrone, M. C., Goldberg, M. E. & Burr, D. C. Changes in visual perception at the time of
saccades. Trends Neurosci. 24, 113–121 (2001).
4. Coppola, D. & Purves, D. The extraordinarily rapid disappearance of entopic images. Proc. Natl Acad.
Sci. USA 93, 8001–8004 (1996).
5. Skavenski, A. A., Hansen, R. M., Steinman, R. M. & Winterson, B. J. Quality of retinal image
stabilization during small natural and artificial body rotations in man. Vision Res. 19, 675–683 (1979).
6. Manteuffel, G., Plasa, L., Sommer, T. J. & Wess, O. Involuntary eye movements in salamanders.
Naturwissenschaften 64, 533–534 (1977).
7. Van der Steen, J. & Collewijn, H. Ocular stability in the horizontal, frontal and sagittal planes in the
rabbit. Exp. Brain Res. 56, 263–274 (1984).
8. Gibson, J. J. The Perception of the Visual World (Houghton Mifflin, Boston, 1950).
9. Vernon, M. D. The Psychology of Perception (Penguin, Baltimore, Maryland, 1962).
10. Graham, C. H. in Vision and Visual Perception (ed. Graham, C. H.) 575–588 (Wiley, New York, 1965).
11. Hammond, P. & Smith, A. T. On the sensitivity of complex cells in feline striate cortex to relative
motion. Exp. Brain Res. 47, 457–460 (1982).
12. Born, R. T. & Tootell, R. B. Segregation of global and local motion processing in primate middle
temporal visual area. Nature 357, 497–499 (1992).
13. Bender, D. B. & Davidson, R. M. Global visual processing in the monkey superior colliculus. Brain Res.
381, 372–375 (1986).
14. Sterling, P. & Wickelgren, B. G. Visual receptive fields in the superior colliculus of the cat.
J. Neurophysiol. 32, 1–15 (1969).
15. Frost, B. J. & Nakayama, K. Single visual neurons code opposing motion independent of direction.
Science 220, 744–745 (1983).
Figure 8 A motion illusion revealed by the Japanese artist Ouchi
. The circle appears to
float and jitter relative to the background.
NATURE | VOL 423 | 22 MAY 2003 | 407
© 2003
16. Steinman, R. M. & Collewijn, H. Binocular retinal image motion during active head rotation. Vision
Res. 20, 415–429 (1980).
17. Cook, P. B., Lukasiewicz, P. D. & McReynolds, J. S. Action potentials are required for the lateral
transmission of glycinergic transient inhibition in the amphibian retina. J. Neurosci. 18, 2301–2308
18. Werblin, F. S. Lateral interactions at inner plexiform layer of vertebrate retina: antagonistic responses
to change. Science 175, 1008–1010 (1972).
19. Enroth-Cugell, C. & Jakiela, H. G. Suppression of cat retinal ganglion cell responses by moving
patterns. J. Physiol. 302, 49–72 (1980).
20. Passaglia, C. L., Enroth-Cugell, C. & Troy, J. B. Effects of remote stimulation on the mean firing rate of
cat retinal ganglion cells. J. Neurosci. 21, 5794–5803 (2001).
21. Werblin, F., Maguire, G., Lukasiewicz, P., Eliasof, S. & Wu, S. M. Neural interactions mediating the
detection of motion in the retina of the tiger salamander. Visual Neurosci. 1, 317–329 (1988).
22. Famiglietti, E. V. Polyaxonal amacrine cells of rabbit retina: size and distribution of PA1 cells. J. Comp.
Neurol. 316, 406–421 (1992).
23. Volgyi, B., Xin, D., Amarillo, Y. & Bloomfield, S. A. Morphology and physiology of the polyaxonal
amacrine cells in the rabbit retina. J. Comp. Neurol. 440, 109–125 (2001).
24. Dacey, D. M. Axon-bearing amacrine cells of the macaque monkey retina. J. Comp. Neurol. 284,
275–293 (1989).
25. Stafford, D. K. & Dacey, D. M. Physiology of the A1 amacrine: a spiking, axon-bearing interneuron of
the macaque monkey retina. Vis. Neurosci. 14, 507–522 (1997).
26. Grossman, G. E., Leigh, R. J., Bruce, E. N., Huebner, W. P. & Lanska, D. J. Performance of the human
vestibuloocular reflex during locomotion. J. Neurophysiol. 62, 264–272 (1989).
27. Hochstein, S. & Shapley, R. M. Linear and nonlinear spatial subunits in Y cat retinal ganglion cells.
J. Physiol. 262, 265–284 (1976).
28. Shapley, R. M. & Victor, J. D. Nonlinear spatial summation and the contrast gain control of cat retinal
ganglion cells. J. Physiol. 290, 141–161 (1979).
29. Victor, J. D. & Shapley, R. M. The nonlinear pathway of Y ganglion cells in the cat retina. J. Gen.
Physiol. 74, 671–689 (1979).
30. Demb, J. B., Zaghloul, K., Haarsma, L. & Sterling, P. Bipolar cells contribute to nonlinear spatial
summation in the brisk-transient (Y) ganglion cell in mammalian retina. J. Neurosci. 21, 7447–7454
31. Wu, S. M., Gao, F. & Maple, B. R. Functional architecture of synapses in the inner retina: segregation of
visual signals by stratification of bipolar cell axon terminals. J. Neurosci. 20, 4462–4470 (2000).
32. Greschner, M., Bongard, M., Rujan, P. & Ammermuller, J. Retinal ganglion cell synchronization by
fixational eye movements improves feature estimation. Nature Neurosci. 5, 341–347 (2002).
33. Regan, D. Human Perception of Objects: Early Visual Processing of Spatial Form Defined by Luminance,
Color, Texture, Motion, and Binocular Disparity (Sinauer, Sunderland, Massachusetts, 2000).
34. Singer, W. Neuronal synchrony: a versatile code for the definition of relations? Neuron 24, 49–65
35. Blakemore, C. & Vital-Durand, F. Organization and post-natal development of the monkey’s lateral
geniculate nucleus. J. Physiol. 380, 453–491 (1986).
36. Kaplan, E. & Shapley, R. M. The primate retina contains two types of ganglion cells, with high and low
contrast sensitivity. Proc. Natl Acad. Sci. USA 83, 2755–2757 (1986).
37. Shapley, R., Kaplan, E. & Soodak, R. Spatial summation and contrast sensitivity of X and Y cells in the
lateral geniculate nucleus of the macaque. Nature 292, 543–545 (1981).
38. Ouchi, H. Japanese Optical and Geometrical Art (Dover, New York, 1977).
39. Meister, M., Pine, J. & Baylor, D. A. Multi-neuronal signals from the retina: acquisition and analysis.
J. Neurosci. Methods 51, 95–106 (1994).
40. Baccus, S. A. & Meister, M. Fast and slow contrast adaptation in retinal circuitry. Neuron 36, 909–919
41. Warland, D. K., Reinagel, P. & Meister, M. Decoding visual information from a population of retinal
ganglion cells. J. Neurophysiol. 78, 2336–2350 (1997).
42. DeVries, S. H. Correlated firing in rabbit retinal ganglion cells. J. Neurophysiol. 81, 908–920 (1999).
43. Chichilnisky, E. J. A simple white noise analysis of neuronal light responses. Network 12, 199–213
Supplementary Information accompanies the paper on
Acknowledgements We thank members of the Meister laboratory for advice; P. Cavanagh,
F. Engert, V. Murthy and K. Nakayama for comments on the manuscript; and H. van der Steen for
providing the eye movement data in Fig. 1b. This work was supported by a grant from NEI
(M.M.) and NRSA (S.A.B.).
Competing interests statement The authors declare that they have no competing financial
Correspondence and requests for materials should be addressed to M.M.
NATURE | VOL 423 | 22 MAY 2003 |
© 2003
... Even at the earliest stages of visual processing, the retina performs nonlinear computations to encode essential aspects of the visual scene. Retinal networks are flexible enough to encode a wide variety of complex stimulus features, such as object motion [31,32], motion reversals [33][34][35], and omitted stimuli [36]. These early computations support efficient downstream readout by throwing away redundant information and preserving features that facilitate perception. ...
... These models explain a wide array of complex retinal computations (e.g. motion onset [58], omitted stimulus response [36] , background vs object motion [32], reversal response [33][34][35]). As a complement to that, BCs have diverging projections onto multiple RGCs on the retina [63]. ...
Full-text available
Everything that the brain sees must first be encoded by the retina, which maintains a reliable representation of the visual world in many different, complex natural scenes while also adapting to stimulus changes. Decomposing the population code into independent and cell-cell interactions reveals how broad scene structure is encoded in the adapted retinal output. By recording from the same retina while presenting many different natural movies, we see that the population structure, characterized by strong interactions, is consistent across both natural and synthetic stimuli. We show that these interactions contribute to encoding scene identity. We also demonstrate that this structure likely arises in part from shared bipolar cell input as well as from gap junctions between retinal ganglion cells and amacrine cells.
... 20,46 One successful approach directly tests computational models of hypothetical neurons proposed to perform a function against interneuron recordings. 18 Our approach generalizes this strategy across a wide range of stimuli and interneurons, yielding an automatic approach to hypothesizing specific roles for interneurons in the response to any stimulus. Guided by an attribution analysis that reveals the internal model states (Figure 8), a future goal will be defining a minimal sufficient stimulus set, including natural images, movies, and potentially artificial stimuli that engage all circuit properties, thus efficiently generating a model that captures all nonlinear properties and phenomena of the retinal circuit. ...
... Pixel regions of 50 × 50 size were then selected from each image at a random location without spatial averaging for presentation. Images drifted in two dimensions in a random walk, 18 moving with a standard deviation of 0.5 pixels per video frame horizontally and vertically. The image also abruptly changed in a single frame to a different location every one second, representing saccades, although such transitions did not contain a sweeping shift in the image. ...
Full-text available
Understanding the circuit mechanisms of the visual code for natural scenes is a central goal of sensory neuroscience. We show that a three-layer network model predicts retinal natural scene responses with an accuracy nearing experimental limits. The model's internal structure is interpretable, as interneurons recorded separately and not modeled directly are highly correlated with model interneurons. Models fitted only to natural scenes reproduce a diverse set of phenomena related to motion encoding, adaptation, and predictive coding, establishing their ethological relevance to natural visual computation. A new approach decomposes the computations of model ganglion cells into the contributions of model interneurons, allowing automatic generation of new hypotheses for how interneurons with different spatiotemporal responses are combined to generate retinal computations, including predictive phenomena currently lacking an explanation. Our results demonstrate a unified and general approach to study the circuit mechanisms of ethological retinal computations under natural visual scenes.
... This spatial nonlinearity is mediated via functional subunits in the receptive fields of retinal ganglion cells. These enable various specific computations that would be impossible without them, from sensitivity to fine spatial structures to various types of motion and pattern sensitivity (Ölveczky et al., 2003;Münch et al., 2009;Zhang et al., 2012;Krishnamoorthy et al., 2017;Zapp et al., 2022;Krüppel et al., 2023). Moreover, nonlinear spatial integration also plays a major role in shaping ganglion cell responses to natural stimuli (Cao et al., 2011;Turner and Rieke, 2016;Karamanlis and Gollisch, 2021). ...
Full-text available
Spatially nonlinear stimulus integration by retinal ganglion cells lies at the heart of various computations performed by the retina. It arises from the nonlinear transmission of signals that ganglion cells receive from bipolar cells, which thereby constitute functional subunits within a ganglion cell's receptive field. Inferring these subunits from recorded ganglion cell activity promises a new avenue for studying the functional architecture of the retina. This calls for efficient methods, which leave sufficient experimental time to leverage the acquired knowledge. Here, we combine concepts from super-resolution microscopy and computed tomography and introduce super-resolved tomographic reconstruction (STR) as a technique to efficiently stimulate and locate receptive field subunits. Simulations demonstrate that this approach can reliably identify subunits across a wide range of model variations, and application in recordings of primate parasol ganglion cells validates the experimental feasibility. STR can potentially reveal comprehensive subunit layouts within less than an hour of recording time, making it ideal for online analysis and closed-loop investigations of receptive field substructure in retina recordings.
... Object motion relative to the observer drives oculomotor tracking [26,27] and is an essential part of many crucial behaviors, like prey capture. Specialized circuitry as early as the retina distinguishes between object and background motion [28,29], while entire brain regions in the visual cortex of primates specialize in processing motion [30] with increasing complexity along the dorsal stream [31]. ...
Some of the most important tasks of visual and motor systems involve estimating the motion of objects and tracking them over time. Such systems evolved to meet the behavioral needs of the organism in its natural environment, and may therefore be adapted to the statistics of motion it is likely to encounter. By tracking the movement of individual points in videos of natural scenes, we begin to identify common properties of natural motion across scenes. As expected, objects in natural scenes move in a persistent fashion, with velocity correlations lasting hundreds of milliseconds. More subtly, we find that the observed velocity distributions are heavy-tailed and can be modeled as a Gaussian scale-mixture. Extending this model to the time domain leads to a dynamic scale-mixture model, consisting of a Gaussian process multiplied by a positive scalar quantity with its own independent dynamics. Dynamic scaling of velocity arises naturally as a consequence of changes in object distance from the observer, and may approximate the effects of changes in other parameters governing the motion in a given scene. This modeling and estimation framework has implications for the neurobiology of sensory and motor systems, which need to cope with these fluctuations in scale in order to represent motion efficiently and drive fast and accurate tracking behavior.
... Building precise computational models of neural response to natural visual stimuli is a fundamental scientific problem in sensory neuroscience. These models can offer insights into neural circuit computations, reveal new mechanisms, and validate theoretical predictions [1,2,3,4,5,6]. However, constructing such models is challenging due to the complex nonlinear processes involved in neural coding, such as synaptic transmission and spiking dynamics. ...
Conference Paper
Full-text available
Developing computational models of neural response is crucial for understanding sensory processing and neural computations. Current state-of-the-art neural network methods use temporal filters to handle temporal dependencies, resulting in an unrealistic and inflexible processing paradigm. Meanwhile, these methods target trial-averaged firing rates and fail to capture important features in spike trains. This work presents the temporal conditioning spiking latent variable models (TeCoS-LVM) to simulate the neural response to natural visual stimuli. We use spiking neurons to produce spike outputs that directly match the recorded trains. This approach helps to avoid losing information embedded in the original spike trains. We exclude the temporal dimension from the model parameter space and introduce a temporal conditioning operation to allow the model to adaptively explore and exploit temporal dependencies in stimuli sequences in a natural paradigm. We show that TeCoS-LVM models can produce more realistic spike activities and accurately fit spike statistics than powerful alternatives. Additionally, learned TeCoS-LVM models can generalize well to longer time scales. Overall, while remaining computationally tractable, our model effectively captures key features of neural coding systems. It thus provides a useful tool for building accurate predictive computational accounts for various sensory perception circuits.
... [31,32] In biology, the detection of moving target is achieved by the collaboration of bipolar cells and retinal neurons. [33] To replicate this function, we reassemble the m × n pixels (m and n represent the length and width of target image, respectively) using the long-term potentiation (LTP) and long-term depression (LTD) nonlinear index [34][35][36][37] of the TPPS and then perform frame difference calculations. Among them, the nonlinear LTD index is fitted by the inhibition of post-synaptic potential under the action of post light pulse voltage (1 V with 10 Hz, Figures S14 and S15, Supporting Information). ...
Full-text available
Neuromorphic vision based on photonic synapses has the ability to mimic sensitivity, adaptivity, and sophistication of bio‐visual systems. Significant advances in artificial photosynapses are achieved recently. However, conventional photosyanptic devices normally employ opaque metal conductors and vertical device configuration, performing a limited hemispherical field of view. Here, a transparent planar photonic synapse (TPPS) is presented that offers dual‐side photosensitive capability for nearly panoramic neuromorphic vision. The TPPS consisting of all two dimensional (2D) carbon‐based derivatives exhibits ultra‐broadband photodetecting (365–970 nm) and ≈360° omnidirectional viewing angle. With its intrinsic persistent photoconductivity effect, the detector possesses bio‐synaptic behaviors such as short/long‐term memory, experience learning, light adaptation, and a 171% pair‐pulse‐facilitation index, enabling the synapse array to achieve image recognition enhancement (92%) and moving object detection.
... Defocused images have blurred edges, different focused planes, and varied light intensities compared with focused images. Amacrine cells may provide feedback inhibition, surround inhibition, adaptation, signal averaging, and noise reduction (Olveczky et al., 2003) to the signaling from RGCs, suggesting that they may take part in encoding focused/defocused images. As a control, Cx36 KO mice lose the filters in the outer and inner retina. ...
Full-text available
The etiology of myopia remains unclear. This study investigated whether retinal ganglion cells (RGCs) in the myopic retina encode visual information differently from the normal retina and to determine the role of Connexin (Cx) 36 in this process. Generalized linear models (GLMs), which can capture stimulus-dependent changes in real neurons with spike timing precision and reliability, were used to predict RGCs responses to focused and defocused images in the retinas of wild-type (normal) and Lens-Induced Myopia (LIM) mice. As the predominant subunit of gap junctions in the mouse retina and a plausible modulator in myopia development, Cx36 knockout (KO) mice were used as a control for an intact retinal circuit. The kinetics of excitatory postsynaptic currents (EPSCs) of a single αRGC could reflect projection of both focused and defocused images in the retinas of normal and LIM, but not in the Cx36 knockout mice. Poisson GLMs revealed that RGC encoding of visual stimuli in the LIM retina was similar to that of the normal retina. In the LIM retinas, the linear-Gaussian GLM model with offset was a better fit for predicting the spike count under a focused image than the defocused image. Akaike information criterion (AIC) indicated that nonparametric GLM (np-GLM) model predicted focused/defocused images better in both LIM and normal retinas. However, the spike counts in 33% of αRGCs in LIM retinas were better fitted by exponential GLM (exp-GLM) under defocus, compared to only 13% αRGCs in normal retinas. The difference in encoding performance between LIM and normal retinas indicated the possible amendment and plasticity of the retinal circuit in myopic retinas. The absence of a similar response between Cx36 KO mice and normal/LIM mice might suggest that Cx36, which is associated with myopia development, play a role in encoding focused and defocused images.
Full-text available
Neural computations arise from highly precise connections between specific types of neurons. Retinal ganglion cells (RGCs) with similar stratification patterns are positioned to receive similar inputs but often display different response properties. In this study, we used intersectional mouse genetics to achieve single-cell type labeling and identified an object motion sensitive (OMS) AC type, COMS-AC(counter-OMS AC). Optogenetic stimulation revealed that COMS-AC makes glycinergic synapses with the OMS-insensitive HD2p-RGC, while chemogenetic inactivation showed that COMS-AC provides inhibitory control to HD2p-RGC during local motion. This local inhibition, combined with the inhibitory drive from TH2-AC during global motion, explains the OMS-insensitive feature of HD2p-RGC. In contrast, COMS-AC fails to make synapses with W3(UHD)-RGC, allowing it to exhibit OMS under the control of VGlut3-AC and TH2-AC. These findings reveal modular interneuron circuits that endow structurally similar RGC types with different responses and present a mechanism for redundancy-reduction in the retina to expand coding capacity.
Full-text available
Neuro-inspired vision systems hold great promise to address the growing demands of mass data processing for edge computing, a distributed framework that brings computation and data storage closer to the sources of data. In addition to the capability of static image sensing and processing, the hardware implementation of a neuro-inspired vision system also requires the fulfilment of detecting and recognizing moving targets. Here, we demonstrated a neuro-inspired optical sensor based on two-dimensional NbS2/MoS2 hybrid films, which featured remarkable photo-induced conductance plasticity and low electrical energy consumption. A neuro-inspired optical sensor array with 10 × 10 NbS2/MoS2 phototransistors enabled highly integrated functions of sensing, memory, and contrast enhancement capabilities for static images, which benefits convolutional neural network (CNN) with a high image recognition accuracy. More importantly, in-sensor trajectory registration of moving light spots was experimentally implemented such that the post-processing could yield a high restoration accuracy. Our neuro-inspired optical sensor array could provide a fascinating platform for the implementation of high-performance artificial vision systems.
Human vision relies on a tiny region of the retina, the 1-deg foveola, to achieve high spatial resolution. Foveal vision is of paramount importance in daily activities, yet its study is challenging, as eye movements incessantly displace stimuli across this region. Here I will review work that, building on recent advances in eye-tracking and gaze-contingent display, examines how attention and eye movements operate at the foveal level. This research highlights how exploration of fine spatial detail unfolds following visuomotor strategies reminiscent of those occurring at larger scales. It shows that, together with highly precise control of attention, this motor activity is linked to non-homogenous processing within the foveola and selectively modulates sensitivity both in space and time. Overall, the picture emerges of a highly dynamic foveal perception in which fine spatial vision, rather than simply being the result of placing a stimulus at the center of gaze, is the result of a finely tuned and orchestrated synergy of motor, cognitive, and attentional processes.
Full-text available
We examined the morphology and physiological response properties of the axon-bearing, long-range amacrine cells in the rabbit retina. These so-called polyaxonal amacrine cells all displayed two distinct systems of processes: (1) a dendritic field composed of highly branched and relatively thick processes and (2) a more extended, often sparsely branched axonal arbor derived from multiple thin axons emitted from the soma or dendritic branches. However, we distinguished six morphological types of polyaxonal cells based on differences in the fine details of their soma/dendritic/axonal architecture, level of stratification within the inner plexiform layer (IPL), and tracer coupling patterns. These morphological types also showed clear differences in their light-evoked response activity. Three of the polyaxonal amacrine cell types showed on-off responses, whereas the remaining cells showed on-center responses; we did not encounter polyaxonal cells with off-center physiology. Polyaxonal cells respected the on/off sublamination scheme in that on-off cells maintained dendritic/axonal processes in both sublamina a and b of the IPL, whereas processes of on-center cells were restricted to sublamina b. All polyaxonal amacrine cell types displayed large somatic action potentials, but we found no evidence for low-amplitude dendritic spikes that have been reported for other classes of amacrine cell. The center-receptive fields of the polyaxonal cells were comparable to the diameter of their respective dendritic arbors and, thus, were significantly smaller than their extensive axonal fields. This correspondence between receptive and dendritic field size was seen even for cells showing extensive homotypic and/or heterotypic tracer coupling to neighboring neurons. These data suggest that all polyaxonal amacrine cells are polarized functionally into receptive dendritic and transmitting axonal zones. J. Comp. Neurol. 440:109–125, 2001. © 2001 Wiley-Liss, Inc.
A white noise technique is presented for estimating the response properties of spiking visual system neurons. The technique is simple, robust, efficient and well suited to simultaneous recordings from multiple neurons. It provides a complete and easily interpretable model of light responses even for neurons that display a common form of response nonlinearity that precludes classical linear systems analysis. A theoretical justification of the technique is presented that relies only on elementary linear algebra and statistics. Implementation is described with examples. The technique and the underlying model of neural responses are validated using recordings from retinal ganglion cells, and in principle are applicable to other neurons. Advantages and disadvantages of the technique relative to classical approaches are discussed.
A new and remarkable type of amacrine cell has been identified in the primate retina. Application of the vital dye acridine orange to macaque retinas maintained in vitro produced a stable fluorescence in the somata of apparently all retinal neurons in both the inner nuclear and ganglion cell layers. Large somata (∼15‐20 μm diam) were also consistently observed in the approximate center of the inner plexiform layer (IPL). Intracellular injections of horseradish peroxidase (HRP) made under direct microscopic control showed that the cells in the middle of the IPL constitute a single, morphologically distinct amacrine cell subpopulation. An unusual and characteristic feature of this cell type is the presence of multiple axons that arise from the dendritic tree and project beyond it to form a second, morphologically distinct arborization within the IPL; these cells have thus been referred to as axon‐bearing amacrine cells. The dendritic tree of the axon‐bearing amacrine cell is highly branched (∼40‐50 terminal dendrites) and broadly stratified, spanning the central 50% of the IPL so that the soma is situated between the outermost and innermost branches. Dendritic field size increases from ∼200 μm in diameter within 2 mm of the fovea to ∼500 μm in the retinal periphery. HRP injections of groups of neighboring cells revealed a regular intercell spacing (∼200‐300 μm in the retinal periphery), suggesting that dendritic territories uniformly cover the retina. One to four axons originate from the proximal dendrites as thin (<0.5 μm). smooth processes. The axons increase in diameter (∼1‐2 μm) as they course beyond the dendritic field and bifurcate once or twice into secondary branches. These branches give rise to a number of thin, bouton‐bearing collaterals that extend radially from the dendritic tree for1‐3 mm without much further branching. The result is a sparsely branched and widely spreading axonal tree that concentrically surrounds the smaller, more highly branched dendritic tree. The axonal tree is narrowly stratified over the central 0‐20% of the IPL; it is approximately ten times the diameter of the dendritic tree, resulting in a 100 times greater coverage factor. The clear division of an amacrine cell's processes into distinct dendritic and axonal components has recently been observed in other, morphologically distinct amacrine cell types of the cat and monkey retina and therefore represents a property common to a number of functionally distinct cell types. It is hypothesized that the axon‐bearing amacrine cells, like classical neurons, use action potentials to transmit signals over long distances in the IPL and, on the basis of previous immunohistochemical results, contain the inhibitory neurotransmitter GABA.
We frequently reposition our gaze by making rapid ballistic eye movements that are called saccades. Saccades pose problems for the visual system, because they generate rapid, large-field motion on the retina and change the relationship between the object position in external space and the image position on the retina. The brain must ignore the one and compensate for the other. Much progress has been made in recent years in understanding the effects of saccades on visual function and elucidating the mechanisms responsible for them. Evidence suggests that saccades trigger two distinct neural processes: (1) a suppression of visual sensitivity, specific to the magnocellular pathway, that dampens the sensation of motion and (2) a gross perceptual distortion of visual space in anticipation of the repositioning of gaze. Neurophysiological findings from several laboratories are beginning to identify the neural substrates involved in these effects.