Retinotopic memory is more precise than
Julie D. Golomb1and Nancy Kanwisher
McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139
Edited by Tony Movshon, New York University, New York, NY, and approved December 14, 2011 (received for review August 10, 2011)
Successful visually guided behavior requires information about spa-
tiotopic (i.e., world-centered) locations, but how accurately is this
visual input? We conducted a spatial working memory task in which
subjects remembered a cued location in spatiotopic or retinotopic
delay. Surprisingly, after a saccade, subjects were significantly more
locations. This difference grew with each eye movement, such that
spatiotopic memory continued to deteriorate, whereas retinotopic
memory did not accumulate error. The loss in spatiotopic fidelity
is therefore not a generic consequence of eye movements, but a
direct result of converting visual information from native retinotopic
coordinates. Thus, despite our conscious experience of an effortlessly
stable spatiotopic world and our lifetime of practice with spatiotopic
tasks, memory is actually more reliable in raw retinotopic coordinates
than in ecologically relevant spatiotopic coordinates.
egocentric representation|gaze-centered representation|remapping|
transsaccadic memory|reference frame
However, a fundamental challenge complicates this seemingly ef-
fortless task: visual input arrives at the eyes in retinotopic (i.e., eye-
centered) coordinates, but visually guided behavior requires in-
formation about spatiotopic (i.e., world-centered) locations. How
do we adapt retinotopic input to support spatiotopic behavior?
One possibility is that we simply “act in the moment” and rec-
reate the visual world anew with each fixation (1, 2). However, this
option is feasible only in cases in which visual information is con-
stantly present; it would clearly fail in cases in which something
must be remembered, attended, or compared across an eye
movement. When we do need to maintain spatial information
across an eye movement, it is an object’s location in the world, not
its location on our retinae, that is generally relevant for behavior.
Tokeeptrack ofreal-worldobjectlocationsacross eye movements,
addition to early retinotopic maps, there also exists somewhere in
the brain a “hard-wired spatiotopic” map.* This idea is appealing
because the initial retinotopic location of an object could be im-
mediately translated into spatiotopic coordinates (using eye posi-
tion information) and stored as a spatiotopic position. Thus,
spatiotopic position would need to be computed only once,†after
which it would remain stable regardless of subsequent changes in
eye position. Indeed, spatiotopic effects have been reported be-
haviorally (3–7) and physiologically (8). However, these effects do
not necessarily require an explicit hard-wired spatiotopic map, and
evidence for such large-scale, explicit spatiotopic neural organiza-
plus updating” solution is to only maintain information in reti-
each eye movement based on information about eye position (15),
corollary discharge from the eye movement (16, 17), and/or stable
retinotopic-plus-updating processes in some brain areas might
coexist with hard-wired spatiotopic maps in others, but it is also
o catch a ball, reach for a cup of coffee, or find a friend in
a crowd, we need to first determine the object’s location.
possible that stability could be achieved on the basis of retinotopic-
Most current theories of visual stability favor some form of
retinotopic-plus-updating, but they vary widely in how quickly,
automatically, or successfully this updating might occur. For
example, several groups have demonstrated that locations can be
rapidly “remapped,” sometimes even in anticipation of an eye
movement (20–22), but a recent set of studies has argued that
updating requires not only remapping to the new location, but
also extinguishing the representation at the previous location,
and this latter process may occur on a slower time scale (23, 24).
Moreover, it is an open question just how accurate we are at
spatiotopic perception. If we do use retinotopic-plus-updating,
does this process actually solve the stability problem and produce
behavior as optimal as we would expect from a hard-wired spa-
tiotopic system? We are not consciously aware of the world
shifting with each eye movement, and intuitively we feel like our
percept is based on a world-centered representation. However,
might our percept of spatiotopic stability actually be somewhat of
an illusion and the process of updating spatiotopic information
not as efficient as it feels?
In the present experiment, we directly tested this counterin-
tuitive prediction: that, despite our conscious experience of an
effortlessly stable spatiotopic world and our lifetime of practice
with spatiotopic tasks, people might actually be better at re-
membering raw retinotopic locations than more ecologically
relevant spatiotopic locations. Subjects performed two sessions
of a transsaccadic spatial memory task (Fig. 1): one in which they
were asked to remember the spatiotopic (i.e., absolute) location
of a cue, and another in which they were asked to remember the
retinotopic location of the cue (i.e., relative to the eyes). Cru-
cially, we asked not simply whether subjects could remember
spatiotopic and retinotopic locations, but how precise this
memory is. By measuring the accuracy of location memory after
zero, one, or two eye movements, we tested whether spatial
memory in either coordinate system was impaired by eye
movements and, if so, whether memory performance continued
to degrade with each subsequent shift in eye position. To tease
apart saccade-related memory decrements from generic memory
decay over time, we included two key comparisons that were
matched for retention interval and number of saccades. First, we
compared retinotopic and spatiotopic performance for each
Author contributions: J.D.G. and N.K. designed research; J.D.G. performed research; J.D.G.
analyzed data; and J.D.G. and N.K. wrote the paper.
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
1To whom correspondence should be addressed. E-mail: email@example.com.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.
*Note that by “hard-wired” we mean “explicit”; we are not presuming anything
†Tobemoreexact, spatiotopicpositionwouldonlyneed to becomputed onceforeach change
in environment. Our experiment does not differentiate between different types of nonretino-
topic frames (e.g., room-centered, monitor-centered, body-centered, head-centered) because
none of these references ever change in our task. Thus, a spatiotopic representation here is
| January 31, 2012
| vol. 109
| no. 5 www.pnas.org/cgi/doi/10.1073/pnas.1113168109
(always the same number of runs for the retinotopic and spatiotopic tasks
for a given subject). The eye tracker was calibrated at the beginning of the
experiment and recalibrated as necessary between runs. Subjects were given
20 practice trials on the relevant task before each session.
Experiment 2. Experiment 2 included the presence of a constantly visible
spatiotopic landmark on the screen, to control for the possibility that the
fixation dot could have served as a retinotopic landmark. The white fixation
dot was now framed with a black outline. When the fixation dot jumped to
a new location on saccade trials, the entire fixation dot reappeared at the
new fixation location, but the black outline remained visible at the original
fixation location (Fig. 1B).
Ideally, we would have removed all possible landmark cues in both ref-
in the appropriate locations throughout the task, we could not eliminate the
fixation dot. Because transsaccadic memory is thought to improve with the
number and proximity of available landmarks (45), we included a single
equidistant spatiotopic reference point to match the single retinotopic ref-
erence point. We used the original fixation location because it was equi-
distant from the cue (on average across trials), and because it was an object
of interest in the task and more likely to be attended.
Experiment 2 tested only the no-saccade and one-saccade conditions. Each
run consisted of 24 trials (n = 8 no-saccade, n = 8 horizontal saccade, n = 8
vertical saccade), with all possible saccade directions presented in random
order. Subjects completed six runs of each task. Otherwise, the design was
identical to that of experiment 1.
Experiment 3. Experiment 3 included a visual mask after the memory cue
to eliminate any afterimages. The mask was composed of 2,000 squares (of
the same size and color as the cue) randomly positioned across the extent
of the screen. After the memory cue was presented for 200 ms, there was
a 200-ms blank fixation period, followed by a 500-ms masking period. The
fixation dot was still visible over the mask. The rest of the trial proceeded
The outline of the initial fixation location remained visible after saccades,
as in experiment 2, and we tested all four conditions (no-saccade, one-sac-
cade, two-saccade, and return-saccade), as in experiment 1. Additionally, cue
and mask stimuli were colored white and presented on a black background,
with minimal external room illumination.
Analyses. For each trial, error magnitude was calculated as the distance
between the subject’s report and the correct location. Values were averaged
separately for each subject, task, and saccade condition and submitted to
random-effects analyses. Trials in which the subject responded in the wrong
region of the screen (error >5.5°) were considered incorrect and excluded.
These occurred on less than 0.5% of trials, with the exception of one subject
in experiment 3, who had an error rate of more than 10% in the reti-
ACKNOWLEDGMENTS. We thank Aude Oliva for use of the eye tracker and
Andrew Leber, Daniel Dilks, and Sam Norman-Haignere for helpful discus-
sion. This work was supported by National Institutes of Health Grants R01-
EY13455 (to N.K.) and F32-EY020157 (to J.D.G.).
1. O’Regan JK, Lévy-Schoen A (1983) Integrating visual information from successive
fixations: does trans-saccadic fusion exist? Vision Res 23:765–768.
2. Irwin DE (1992) Perceiving an integrated visual world. Attention and Performance
XIV: Synergies in Experimental Psychology, Artificial Intelligence, and Cognitive
Neuroscience, eds Meyer DE, Kornblum S (MIT Press, Cambridge, MA), pp 121–142.
3. Mays LE, Sparks DL (1980) Saccades are spatially, not retinocentrically, coded. Science
4. Melcher D, Morrone MC (2003) Spatiotopic temporal integration of visual motion
across saccadic eye movements. Nat Neurosci 6:877–881.
5. Hayhoe M, Lachter J, Feldman J (1991) Integration of form across saccadic eye
movements. Perception 20:393–402.
6. Ong WS, Hooshvar N, Zhang M, Bisley JW (2009) Psychophysical evidence for spa-
tiotopic processing in area MT in a short-term memory for motion task. J Neuro-
7. Pertzov Y, Zohary E, Avidan G (2010) Rapid formation of spatiotopic representations
as revealed by inhibition of return. J Neurosci 30:8882–8887.
8. Duhamel JR, Bremmer F, BenHamed S, Graf W (1997) Spatial invariance of visual re-
ceptive fields in parietal cortex neurons. Nature 389:845–848.
9. d’Avossa G, et al. (2007) Spatiotopic selectivity of BOLD responses to visual motion in
human area MT. Nat Neurosci 10:249–255.
10. Gardner JL, Merriam EP, Movshon JA, Heeger DJ (2008) Maps of visual space in human
occipital cortex are retinotopic, not spatiotopic. J Neurosci 28:3988–3999.
11. Ong WS, Bisley JW (2011) A lack of anticipatory remapping of retinotopic receptive
fields in the middle temporal area. J Neurosci 31:10432–10436.
12. Krekelberg B, Kubischik M, Hoffmann KP, Bremmer F (2003) Neural correlates of vi-
sual localization and perisaccadic mislocalization. Neuron 37:537–545.
13. Golomb JD, Kanwisher N (2012) Higher-level visual cortex represents retinotopic, not
spatiotopic, object location. Cereb Cortex, 10.1093/cercor/bhr357.
14. Cavanagh P, Hunt AR, Afraz A, Rolfs M (2010) Visual stability based on remapping of
attention pointers. Trends Cogn Sci 14:147–153.
15. Andersen RA, Bracewell RM, Barash S, Gnadt JW, Fogassi L (1990) Eye position effects
on visual, memory, and saccade-related activity in areas LIP and 7a of macaque. J
16. Wurtz RH (2008) Neuronal mechanisms of visual stability. Vision Res 48:2070–2089.
17. Henriques DY, Klier EM, Smith MA, Lowy D, Crawford JD (1998) Gaze-centered re-
mapping of remembered visual space in an open-loop pointing task. J Neurosci 18:
18. Deubel H, Bridgeman B, Schneider WX (1998) Immediate post-saccadic information
mediates space constancy. Vision Res 38:3147–3159.
19. McConkie GW, Currie CB (1996) Visual stability across saccades while viewing complex
pictures. J Exp Psychol Hum Percept Perform 22:563–581.
20. Duhamel JR, Colby CL, Goldberg ME (1992) The updating of the representation of
visual space in parietal cortex by intended eye movements. Science 255:90–92.
21. Melcher D (2007) Predictive remapping of visual features precedes saccadic eye
movements. Nat Neurosci 10:903–907.
22. Rolfs M, Jonikaitis D, Deubel H, Cavanagh P (2011) Predictive remapping of attention
across eye movements. Nat Neurosci 14:252–256.
23. Golomb JD, Chun MM, Mazer JA (2008) The native coordinate system of spatial at-
tention is retinotopic. J Neurosci 28:10654–10662.
24. Golomb JD, Marino AC, Chun MM, Mazer JA (2011) Attention doesn’t slide: spatio-
topic updating after eye movements instantiates a new, discrete attentional locus.
Atten Percept Psychophys 73:7–14.
25. Sheth BR, Shimojo S (2001) Compression of space in visual memory. Vision Res 41:
26. Prime SL, Vesia M, Crawford JD (2011) Cortical mechanisms for trans-saccadic memory
and integration of multiple object features. Philos Trans R Soc Lond B Biol Sci 366:
27. Henderson JM, Hollingworth A (2003) Eye movements and visual memory: detecting
changes to saccade targets in scenes. Percept Psychophys 65:58–71.
28. Irwin DE, Gordon RD (1998) Eye movements, attention and trans-saccadic memory. Vis
29. Irwin DE (1991) Information integration across saccadic eye movements. Cognit Psy-
30. Lawrence BM, Myerson J, Oonk HM, Abrams RA (2001) The effects of eye and limb
movements on working memory. Memory 9:433–444.
31. Baker JT, Harper TM, Snyder LH (2003) Spatial memory following shifts of gaze. I.
Saccades to memorized world-fixed and gaze-fixed targets. J Neurophysiol 89:
32. Fiehler K, Rösler F, Henriques DY (2010) Interaction between gaze and visual and
proprioceptive position judgements. Exp Brain Res 203:485–498.
33. Golomb JD, Nguyen-Phuc AY, Mazer JA, McCarthy G, Chun MM (2010) Attentional
facilitation throughout human visual cortex lingers in retinotopic coordinates after
eye movements. J Neurosci 30:10493–10506.
34. Medendorp WP, Goltz HC, Vilis T, Crawford JD (2003) Gaze-centered updating of
visual space in human parietal cortex. J Neurosci 23:6209–6214.
35. Merriam EP, Genovese CR, Colby CL (2003) Spatial updating in human parietal cortex.
36. Wang RF, Spelke ES (2000) Updating egocentric representations in human navigation.
37. Byrne PA, Cappadocia DC, Crawford JD (2010) Interactions between gaze-centered
and allocentric representations of reach target location in the presence of spatial
updating. Vision Res 50:2661–2670.
38. Prime SL, Vesia M, Crawford JD (2008) Transcranial magnetic stimulation over pos-
terior parietal cortex disrupts transsaccadic memory of multiple objects. J Neurosci 28:
39. Ross J, Morrone MC, Goldberg ME, Burr DC (2001) Changes in visual perception at the
time of saccades. Trends Neurosci 24:113–121.
40. Hoffman JE, Subramaniam B (1995) The role of visual attention in saccadic eye
movements. Percept Psychophys 57:787–795.
41. Awh E, Armstrong KM, Moore T (2006) Visual and oculomotor selection: links, causes
and implications for spatial attention. Trends Cogn Sci 10:124–130.
42. Rizzolatti G, Riggio L, Dascola I, Umiltá C (1987) Reorienting attention across the
horizontal and vertical meridians: Evidence in favor of a premotor theory of atten-
tion. Neuropsychologia 25(1A):31–40.
43. Karn KS, Møller P, Hayhoe MM (1997) Reference frames in saccadic targeting. Exp
Brain Res 115:267–282.
44. Deubel H (2004) Localization of targets across saccades: Role of landmark objects. Vis
45. Verfaillie K (1997) Transsaccadic memory for the egocentric and allocentric position
of a biological-motion walker. J Exp Psychol Learn Mem Cogn 23:739–760.
46. Lin IF, Gorea A (2011) Location and identity memory of saccade targets. Vision Res 51:
Golomb and KanwisherPNAS
| January 31, 2012
| vol. 109
| no. 5