Retinal prosthetic strategy with the capacity to restore
Sheila Nirenberg1and Chethan Pandarinath
Department of Physiology and Biophysics, Weill Medical College of Cornell University, New York, NY 10065
Edited* by Norma Graham, Columbia University, New York, NY, and approved July 9, 2012 (received for review May 7, 2012)
Retinal prosthetics offer hope for patients with retinal degenera-
tive diseases. There are 20–25 million people worldwide who are
blind or facing blindness due to these diseases, and they have few
treatment options. Drug therapies are able to help a small fraction
of the population, but for the vast majority, their best hope is
through prosthetic devices [reviewed in Chader et al. (2009) Prog
Brain Res 175:317–332]. Current prosthetics, however, are still very
limited in the vision that they provide: for example, they allow
for perception of spots of light and high-contrast edges, but not
natural images. Efforts to improve prosthetic capabilities have fo-
cused largely on increasing the resolution of the device’s stimula-
tors (either electrodes or optogenetic transducers). Here, we show
that a second factor is also critical: driving the stimulators with
the retina’s neural code. Using the mouse as a model system, we
generated a prosthetic system that incorporates the code. This
dramatically increased the system’s capabilities—well beyond
what can be achieved just by increasing resolution. Furthermore,
the results show, using 9,800 optogenetically stimulated ganglion
cell responses, that the combined effect of using the code and
high-resolution stimulation is able to bring prosthetic capabilities
into the realm of normal image representation.
neural prosthetic|retinal degeneration|vision restoration|
neural coding|macular degeneration
nitis pigmentosa, which affect 20–25 million people worldwide
(1–3). These diseases cause a progressive loss of the retina’s
input cells, the photoreceptors, which leads to severe visual
impairment. While the photoreceptors and neighboring tissue
degenerate, the retina’s output cells remain largely intact.
Prosthetic devices make use of this. The approach is to bypass
the damaged tissue and provide direct stimulation to the sur-
viving cells, driving them to send visual information to the brain.
While the approach generated a great deal of excitement at its
initiation, the devices have not yet achieved the success that was
hoped for. Current devices still provide only very limited vision.
For example, they allow patients to see spots of light and high-
contrast edges, which provide some ability for navigation and
gross feature detection, but they are far from providing patients
with normal representations of faces, landscapes, etc. (4–6).
[With respect to navigation, the devices enable the detection of
light sources, such as doorways and lamps, and, with respect to
feature detection, they allow discrimination of objects or letters
if they span ∼7 ° of visual angle (5); this corresponds to about 20/
1,400 vision; for comparison, 20/200 is the acuity-based legal
definition of blindness in the United States (7)].
In the past several years, there has been a major push toward
improving prosthetic performance. The effort has focused largely
on increasing the resolution of the devices’ stimulators. The
working hypothesis is that the main problem is resolution, and
if it were to be increased, prosthetic performance would be
substantially boosted. Several groups have been addressing
this, using both electrode approaches [high-density, fine-tipped
electrode arrays (8, 9)] and optogenetic methods [primarily
Channelrhodopsin-2 (ChR2) and its derivatives (10–15)].
However, there is another significant problem, too, and that
is how to drive the stimulators to produce normal retinal output.
rosthetic devices offer hope for patients with retinal de-
generative diseases, such as macular degeneration and reti-
Briefly, when images enter the retina, they are transformed via
retinal processing into patterns of action potentials. The patterns
are in a code the brain can read, a code the brain is expecting.
Prosthetic devices have not yet incorporated this, raising the
possibility that the reason they have not reached their goal is not
just because of a resolution problem, but also because of a cod-
ing problem (for discussion, see refs. 13, 14, 16–18).
Here we present a prosthetic system that captures this trans-
formation and produces the retina’s code; that is, it converts
visual input into the same patterns of action potentials that the
retina normally produces—and it does this reliably for a broad
range of stimuli, including faces, landscapes, animals, people
walking, etc. Our results show that incorporation of the code
dramatically increases prosthetic performance, well beyond what
can be achieved just by increasing resolution. Moreover, they
show that the combination of the code and high-resolution
stimulation is able to bring prosthetic capabilities up to the level
of normal or near-normal image representation.
The prosthetic consists of two parts: an encoder and a transducer
(Fig. 1). The encoder mimics the transformations performed by
the retina; that is, it converts visual input into the code used by
the retina’s output cells (the ganglion cells), and the transducer
then drives the ganglion cells to fire as the code specifies. Briefly,
the encoder is an input/output model of the retina, and the
transducer is the light-sensitive protein ChR2 (19, 20); a complete
description of the encoder and transducer is given in SI Materials
As shown in Fig. 1, the steps from visual input to retinal output
proceed as follows: Images enter a device that contains the en-
coder and a stimulator [a modified minidigital light projector
(mini-DLP)]. The encoder converts the images into streams of
electrical pulses, analogous to the streams of action potentials
that would be produced by the normal retina in response to the
same images. The electrical pulses are then converted into light
pulses (via the mini-DLP) to drive the ChR2, which is expressed
in the ganglion cells.
This approach confers on blind retinas the ability to produce
normal output, that is, patterns of action potentials that closely
match those produced by the normal retina. To show this, we
used three sets of recordings. The first set shows normal ganglion
cell firing patterns (Fig. 2A, Top). We presented movies of nat-
ural scenes (spatiotemporally varying scenes that include land-
scapes, faces, people walking, cars, etc.) to the normal mouse
retina and recorded ganglion cell responses using a multielec-
trode array. The responses of several cells, each viewing a dif-
ferent region of the movie, are shown.
Author contributions: S.N. designed research; S.N. and C.P. performed research; S.N. and
C.P. contributed new reagents/analytic tools; S.N. and C.P. analyzed data; and S.N. wrote
Conflict of interest statement: S.N. and C.P. have a patent application filed through
*This Direct Submission article had a prearranged editor.
1To whom correspondence should be addressed. E-mail: email@example.com.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.
| September 11, 2012
| vol. 109
| no. 37www.pnas.org/cgi/doi/10.1073/pnas.1207035109
The second set shows ganglion cell firing patterns from a blind
retina when it was presented with the same movies, but this time
through the encoder-ChR2 prosthetic (Fig. 2A, Middle). To ob-
tain blind retinas, we used a standard mouse model of retinal
degeneration, the rd1/rd1 mouse line (11, 13, 21, 22). In this line,
the degeneration of the outer retina is severe, such that the
photoreceptor outer segments are completely lost by 8 wk of age
(Fig. 2C, Upper Right), and, at the physiological level, no visual
responses are observed (Fig. 2E). To obtain blind retinas that
also express ChR2, we bred the rd1/rd1 animals into a mouse line
that expresses ChR2 under the regulation of the Thy1 promoter,
which is active in retinal ganglion cells (Fig. 2C, Lower Left); this
produced Thy1-ChR2 rd1/rd1 animals. As shown in Fig. 2C,
Lower Right, retinas from these animals show both the severe
degeneration of the rd1/rd1 mice (the complete absence of
a photoreceptor layer) and the targeted expression of ChR2 in
the ganglion cells. Fig. 2D shows a whole-mount view of a retina
from one of these mice, indicating the dense expression of ChR2
in the ganglion cell population [∼33% of ganglion cells express
the ChR2 gene (13)].
To present the movies to these retinas through the encoder-
ChR2 prosthetic, we proceeded as in Fig. 1: We presented the
movies to the encoder/stimulator device, which converted them
into streams of electrical pulses and then into streams of light
pulses to drive the ChR2. Fig. 2A, Middle, shows the results: the
spike patterns produced by the blind retinas closely match those
produced by the normal retinas.
The reason it works is twofold: the encoder (the retinal
input/output model) is able to mimic retinal transformations
for a broad range of stimuli, including natural scenes, and thus
produce normal patterns of pulses (for extensive quantification,
see ref. 23), and the transducer (ChR2) has the necessary ki-
netics/sensitization to follow the encoder’s signals. (For longer
stretches of data, higher resolution rasters, animated figures, and
model performance using primate retina, see Figs. S1, S2, S3,
and S4 and Movie S1).
Finally, the last set of recordings shows ganglion cell responses
from a blind retina viewing the movies through the standard
optogenetic method, where the visual input is presented as is,
with no encoding, as in refs. 10–13, 15 (see Discussion for a
summary of other optogenetic approaches). To set up a fair
comparison, we presented the movies using the same stimulator
(same mini-DLP, same wavelength, same peak intensity), so that
the only difference between the standard approach and the ap-
proach shown in Fig. 2A, Middle, is the use of the encoder. The
results are shown in Fig. 2A, Bottom: the firing patterns produced
by the standard method are clearly different from those pro-
duced by the normal retina (compare Fig. 2A, Bottom and Top).
Although the standard approach is very effective in producing
ganglion cell firing, the firing patterns are not the normal pat-
terns. Fig. 2B shows the same series of results as in Fig. 2A, but
for a second movie, one with different image statistics, to em-
phasize the reliability of the results.
Fig. 2 A and B shows that this prosthetic approach allows
blind retinas to produce normally coded output. How important
is this: what is the potential impact on prosthetic capability? We
assessed this using three methods: confusion matrices (Fig. 3),
image reconstructions (Fig. 4), and behavioral performance on
an optomotor task (Fig. 5).
We first show the results for the confusion matrices (Fig. 3).
Briefly, a confusion matrix gives the probability that a neural
response to a presented stimulus will be decoded as that stimu-
lus. The vertical axis of the matrix indicates the presented
stimulus (i), and the horizontal axis indicates the decoded
stimulus (j). The matrix element at position (i,j) gives the
probability that stimulus i is decoded as stimulus j. If j = i, the
stimulus is decoded correctly; otherwise, the stimulus is decoded
incorrectly. Put briefly, elements on the diagonal indicate correct
decoding; elements off the diagonal indicate confusion.
The top rows of Fig. 3 show the results for ganglion cells from
normal retinas, using natural scene stimuli (movies containing
faces, landscapes, people walking, etc.). On the left is the per-
formance for individual cells; on the right, for a population of
cells. As shown, the individual cells each carry a fair amount of
information, and, as a population, they perform very well (nearly
all stimuli are identified correctly) (Fig. 3A, Top Right). The next
row shows the results for ganglion cells from blind retinas pre-
sented with the same movies, but through the encoder-ChR2
method. The individual cells do not carry quite as much in-
formation (there is some scatter across the matrix), but together
as a population, they also perform very well (Fig. 3A, Middle).
The last row shows the performance with the standard opto-
genetic method. The individual cells here carry little information,
and even as a population, they are still quite limited (Fig. 3A,
Bottom). Thus, the incorporation of the retina’s code, even for
a small population of 20 cells, has a very large effect and dra-
matically boosts prosthetic capabilities. Fig. 3B shows the same
analysis for a second set of movies (those shown in Fig. 2B).
We quantified the performance by calculating the fraction of
times that the responses were correctly decoded, averaged over all
stimuli. For each matrix, the “fraction correct” is the mean of the
values on the diagonal. For the data in Fig. 3 A and B, the fraction
correct for the blind retinas treated with the encoder-ChR2
prosthetic, normalized to the fraction correct for the normal
retina, was 96% (Fig. 3A) and 81% (Fig. 3B). The fraction correct
for the blind retinas treated with the standard method, normalized
to the fraction correct for the normal retina, was 25% (Fig. 3A)
and 8% (Fig. 3B). On average, the fraction correct for the en-
coder-ChR2 method was 88% and, for the standard method, 17%.
Next, we performed stimulus reconstructions (Fig. 4). For
these experiments we presented an image (of a baby’s face) to
blind retinas using the two prosthetic systems—the encoder-
ChR2 method and the standard optogenetic method—and recor-
ded ganglion cell responses. We then reconstructed the face from
the responses using maximum likelihood (25–27), an essentially
assumption-free method (SI Materials and Methods). To obtain a
large enough dataset for the complete reconstruction, we moved
the image systematically across the region of the retina from
which we were recording, so that responses to all parts of the
image could be obtained with a single or small number of retinas.
Approximately 9,800 ganglion cell responses were recorded for
.. .. ... .
. . ... .
. . ..
... . .
.. . ....
. ... . .
.. ... ..
.. .. . .
. . ... .
channelrhodopsin-2 (ChR2), etc.
and a transducer. The encoder converts the image into the code used by the
retinal output cells, the ganglion cells; the transducer drives the ganglion
cells to fire spike patterns as the code specifies. As indicated, the transducers
can be electrodes, optogenetic stimulators such as ChR2, etc. (B) The steps
from visual input to retinal output for a blind retina. The input enters a
device that contains the encoder and a stimulator (a mini-DLP). The encoder
converts the input into patterns of electrical pulses, analogous to the patterns
of spikes that would be produced by the normal retina to the same visual
input. The patterns of electrical pulses are then converted into patterns of
light pulses (via the DLP) to drive the ChR2 in the ganglion cells. For a sche-
matic of the setup as used in the in vitro and in vivo experiments, see Fig. S5.
Schematic of the prosthetic. (A) The two components: an encoder
Nirenberg and PandarinathPNAS
| September 11, 2012
| vol. 109
| no. 37
Fig. 4A shows the original image and Fig. 4B shows the image
reconstructed from the output of the encoder. Not only is it
possible to discern that the image is a baby’s face, but also it is
possible to tell that it is this particular baby (the baby in Fig. 4A),
an extremely challenging task. Fig. 4C shows the image recon-
structed from the responses of the blind retina driven by the en-
coder-ChR2 method; while there is some loss of information due
to passing the code through ChR2, the responses still also pro-
duce a close match. Finally, Fig. 4D shows the image recon-
structed from the responses of a blind retina driven by the
standard method. This image is very much degraded with respect
to the original. These results, again, indicate that incorporation
of the retina’s code into a prosthetic has substantial impact.
(Pearson’s correlation coefficient between Fig. 4B and the orig-
inal is 0.889; between Fig. 4C and the original, 0.760; and be-
tween Fig. 4D and the original, 0.221.)
Note that the number and density of cells used in Fig. 4 C
and D (∼9,800 ganglion cell responses each) correspond to the
number and density of cells used in the normal mouse retina (see
SI Materials and Methods for number of cells per degree of visual
angle). Fig. 4D, therefore, essentially gives an upper bound on the
performance of the standard method (same number and density
of cells as the animal uses, with their responses decoded using
optimal, i.e., Bayesian, decoding). However, the image shows little
Ganglion cell firing patterns Ganglion cell firing patterns
Movies of natural
Movies of natural
presented with movies of natural scenes (landscapes, faces, people walking, etc). Several cells are shown, including both ON transient and ON sustained cells.
(For longer stretches of data, see Figs. S1 and S4.) (Middle) Ganglion cell firing patterns from a blind retina (Thy1-ChR2 rd1/rd1 retina) when it was presented
with the same movies, but through the encoder-ChR2 prosthetic. As shown, the prosthetic confers on the blind retina the ability to produce firing patterns
that closely match those of the normal retina. (Bottom) Ganglion cell firing patterns from a blind retina (Thy1-ChR2 rd1/rd1 retina) when it was presented
with the same movies, but through the standard optogenetic prosthetic (just ChR2, no encoder). To allow comparison with the middle set of recordings, the
movies were presented with the same stimulator (same mini-DLP, same wavelength, same peak intensity), so that the only difference between the Middle and
Bottom recordings was the use of the encoder. As shown, although the standard approach is very effective in driving the ganglion cells, the firing patterns are
not the normal patterns. The same receptive field locations were used for all panels to allow comparison of firing patterns (white circles on the movie images).
(B) Same as in A, but using a movie with different image statistics and different ganglion cell types (OFF and ON-OFF types), indicating the robustness of the
results. (C) Retinal cross sections from wild-type, rd1/rd1, Thy1-ChR2, and Thy1-ChR2 rd1/rd1 mice, respectively. The retina from the Thy1-ChR2 rd1/rd1 animal
shows both the severe degeneration characteristic of rd1/rd1 animals (21, 22) (the absence of the photoreceptor layer) and the targeted expression of ChR2 to
the ganglion cells. (Scale bar, 20 μm.) (D) Whole-mount view of the retina showing the ChR2-expressing cells. (Scale bar, 50 μm.) (E) Recordings from rd1/rd1
retinas that do not express ChR2. These retinas were stimulated using the same stimulator as in A (same mini-DLP, same wavelength, same peak intensity).
(Top Left) Ganglion cell responses when the retina was stimulated with the encoded movies. (Top Right) Ganglion cell responses when the retina was
stimulated with a periodic flash. As shown, and as expected, because these retinas have no photoreceptor outer segments and no ChR2, there is no response
to either stimulus, just baseline firing. Bottom, Left and Right, are same as Top, Left and Right, but with the addition of the neurotransmitter blockers APB,
CPP, and NBQX, as in ref. 11; again, no stimulus-dependent responses were observed. For all recordings from degenerated retinas in this paper, i.e., for Figs.
2–4, the retinas were rendered blind using both methods: the use of rd1/rd1 animals and the blockers, the latter included for compulsivity.
Blind retinas produce the same output as the normal retina. (A) (Top) Ganglion cell-firing patterns from a normal, wild-type retina when it was
| www.pnas.org/cgi/doi/10.1073/pnas.1207035109Nirenberg and Pandarinath
detail. In other words, the results show that the standard approach,
which focuses on increasing resolution, has an inherent ceiling
on the quality of the image it can produce; even at maximal
resolution, image quality is very short of normal. Incorporation
of the code breaks through the ceiling.
Finally, we performed a set of behavioral experiments using
optomotor tracking. Optomotor behavior was chosen for two
reasons: First, it is a reflex and, therefore, does not require be-
havioral training, which is difficult in animals undergoing retinal
degeneration. Second, it requires only a single group of ganglion
cells to be targeted: the set of nuclei that controls the optomotor
response in the mouse, the accessory optic system, receives input
only from ON cells (24, 28), so we can drive the optomotor re-
sponse with just ON ganglion cell encoders and test the effec-
tiveness of the approach. For these experiments, the animal was
head-fixed and placed in front of a liquid crystal display (LCD),
which we used to deliver the coded pulses. The LCD was used,
rather than the mini-DLP, so that an eye-tracking system could
be placed next to the animal’s eye. Eye position was tracked
using an ISCAN tracking system, which measures infrared reflec-
tions off the cornea (SI Materials and Methods).
Fig. 5 shows the results. Fig. 5A shows the baseline condition
for the animal: blind mice show a drift in eye position when no
stimulus is present, similar to that observed with blind humans.
Fig. 5 B and C show the results with the standard optogenetic
method and the encoder-ChR2 method, respectively. For the
standard optogenetic method, the stimulus, a drifting sine wave
grating, was presented as is; for the encoder-ChR2 method, the
grating was presented in the encoded form (encoders for tran-
sient ON cells were used). As shown in Fig. 5, the encoder-ChR2
method produces tracking, whereas the standard method does
not. These results further reveal the importance of incorporating
the code: when the image was converted into the code used by
the ON ganglion cells, the animal became able to track it.
We have developed a unique prosthetic system that consists of
two parts: an encoder and a transducer. The encoder converts
visual input into the retina’s code, that is, the code that the retina
normally uses to communicate with the brain. The transducer
then drives the retina’s output cells, the ganglion cells, to fire as
the code specifies.
The system is effective for two reasons. First, the encoder is able
to capture the input/output relations of the retina for a broad
range of stimuli, including landscapes, faces, animals, people
walking, playing, etc.; that is, it produces normal ganglion cell-
firing patterns to natural stimuli with great fidelity (as shown here
and, additionally, in ref. 23). Second, the transducer (ChR2) has
the necessary kinetics to follow the signals the encoder produces.
The effectiveness was shown four ways, starting with the
raw data. Fig. 2 shows the firing patterns of the normal retina in
response to movies of natural scenes and, below that, the firing
patterns produced by the blind retina stimulated via the pros-
thetic. As shown, the firing patterns produced by the prosthetic
. . .
. . .
. . .
. . .
. . .
. . .
measured using confusion matrices. (A) (Top) Confusion matrices generated from the responses of a normal wild-type retina when presented with movies of
natural scenes, which include faces, people walking, landscapes, etc. (Middle) Confusion matrices generated from the responses of a blind retina (Thy1-ChR2
rd1/rd1 retina) when it was presented with the same movies, but through the encoder-ChR2 prosthetic. (Bottom) Confusion matrices generated from
the responses of a blind retina (Thy1-ChR2 rd1/rd1 retina) when it was presented with the same movies, but through the standard optogenetic prosthetic.
(B) Same as A, but for a different set of movies. As shown in both A and B, the confusion matrices generated from the responses of the blind retinas treated
with the encoder-ChR2 prosthetic closely match those of the normal wild-type retinas: the data in the right-most matrices (the population performances)
lie along the diagonal line, indicating correct identification of the stimuli. See Fig. S7 for the same analysis performed with an array of bin sizes.
The output of blind retinas carries the same amount of information, and the same quality of information, as the output of normal retinas, as
trains of the blind retinas. Although the brain does
not necessarily reconstruct images, reconstructions
serve as a convenient way to compare methods and
give an approximation of the level of visual resto-
ration possible with each approach. (A) Original
image (a baby’s face). (B) Image reconstructed from
the responses of the encoder. (C) Image recon-
structed from the responses of the blind retina
when it was presented with the face through the
encoder-ChR2 prosthetic. (D) Image reconstructed from the responses of the blind retina when it was presented with the face through the standard
optogenetic prosthetic. The reconstructions were carried out on a processing cluster in blocks of 10 × 10 or 7 × 7 checks. The decoding was performed using
maximum likelihood; that is, for each block, we found the array of gray values that maximized the probability of the observed responses (following ref. 27 for
high dimensional searches). The number and density of cells used match that of the normal mouse retina—∼9,800 ganglion cell responses per image (see
Materials and Methods for numbers).
Nirenberg and PandarinathPNAS
| September 11, 2012
| vol. 109
| no. 37
closely match those of the normal retina. (For longer stretches of
data, higher resolution rasters, and animated figures, see Figs.
S1, S2, S3, S4, and Movie S1).
We then quantified the results in Figs. 3–5; that is, we mea-
sured the extent to which the firing patterns produced by the
prosthetic match those of the normal retina using three in-
dependent assays. We first assessed the match using probabilistic
(Bayesian) decoding, an essentially assumption-free method. We
started by decoding the firing patterns from the normal retina.
Briefly, we presented an array of images to the normal retina and
recorded the resulting spike trains. We then presented the same
images to the blind retina through the prosthetic and again
recorded the resulting spike trains. We then asked: would the
spike trains from the blind retina be interpreted correctly? If the
brain were to receive the spike trains produced by the prosthetic,
would it interpret them correctly? This is what the confusion
matrices show. Fig. 3 A and B, Middle, shows the fraction of
times the spike trains produced by the prosthetic mapped to the
same images as the spike trains produced by the normal retina.
We emphasize that we carried this out in a particularly rig-
orous way: we decoded the spike trains from the prosthetic into
the response distributions produced by the normal retina (the
likelihoods from the normal retina). This allowed us to explicitly
ask: if the brain were expecting spike trains from the normal
retina, but instead received spike trains from the prosthetic,
would it recognize them and map them to the correct image, or
would it be confused? What the results showed is that, with high
reliability (almost 90% of the time), the brain would map the spike
trains from a completely blind retina driven by the prosthetic to
the same images as the spike trains from the normal retina.
Fig. 4 also showed probabilistic (Bayesian) decoding, but in
the context of a reconstruction task: we reconstructed full gray-
scale images (faces) on a grid of 35 × 32 pixels, which is a very
high dimensional problem, especially with uniform (i.e., un-
informative) priors. For these experiments, we used several
thousand ganglion cell responses, the same number of cells
that the animal would use (see Materials and Methods for cell
numbers). Similar to the results shown in Fig. 3, these results
showed that the spike trains produced by the prosthetic were
highly reliable: the baby’s face was faithfully reconstructed—to
the extent that it could be recognized as the same face as
In addition to showing the reconstruction from the blind ret-
ina, we also showed a reconstruction from the pulses produced
by the encoder. This demonstrated the effectiveness of the two
parts of the prosthetic separately. The reconstruction from the
pulses of the encoder shows how well we captured the code, and
the reconstruction from the blind retina shows how well ChR2
follows the encoder. As shown, although there is some information
lost as the code passes through ChR2, it is relatively small
(compare Fig. 4 B and C).
Finally, we tested the efficacy of the prosthetic at the level of
behavior (Fig. 5). We chose an optomotor tracking task because
it does not require training (which is difficult in animals with
retinal degeneration), and, because it can be driven by a single
group of ganglion cells, the ON cells, we were able to drive the
response with just ON cell encoders and assess the effectiveness
of the approach. The results showed that the prosthetic makes
tracking possible: when the stimulus, a drifting grating, was pre-
sented through the encoder, it became detectable to the animal,
and tracking ensued. This emphasizes, at the level of an in vivo
experiment, that the encoder component is critical; without it,
tracking responses are random.
In sum, the raw data (Fig. 2) and the three functional assays
(Figs. 3–5) provide very strong evidence that we have the essential
building blocks for building a highly effective retinal prosthetic.
Comparison with Other Optogenetic Approaches. As mentioned in
the Introduction, other groups have used optogenetic approaches
to drive the degenerated retina. The main strategy has been to
drive the treated retina directly with the visual stimulus, intensified
to a level that could drive ChR2. The target cells were initially
ganglion cells (10, 12, 13), and, later, bipolar cells (11, 15). These
were major pioneering studies, showing that blind retinas could
become responsive to light using ChR2. However, there was still
the issue of processing, of driving the retina to produce normal
responses. This was one of the main reasons for shifting from
ganglion cells to bipolar cells (11): because bipolar cells are closer
aspects of normal processing to be preserved. However, the retinal
output was not normal, even for simple stimuli, such as flashes
(11, 15). Presumably, this is because optogenetic stimulation at the
level of the bipolar cell still bypasses too much significant circuitry,
such as circuitry that contributes to temporal processing, center/
surround antagonism, and possibly other operations.
Recently, Greenberg et al. (14) focused on center/surround
antagonism using a clever optogenetic strategy. They created an
excitatory center and an inhibitory surround at the level of the
ganglion cell using antagonistic opsins. Specifically, they directed
ChR2 to the soma and halorhodopsin to the dendrites using
specific gene regulatory elements. This approach by itself did not
create center/surround with normal spatial characteristics, but they
were able to remedy this with a preprocessing step of Gaussian
blur. Still, as the authors themselves state, this constituted only
a first step toward building retinal processing; that is, it creates
center/surround, but other aspects of the processing have not yet
Our approach is different: instead of trying to recreate com-
putations in the retina, i.e., in the actual tissue, we carry them out
in an external device. This has two key advantages. First, one can
develop a more complete model of retinal computations without
the additional constraint of having to find molecular components
to implement them. Second, because we are not using ChR2 as a
computing mechanism, we are making much smaller demands
on what it needs to achieve. We use ChR2 only as a transducer,
a means to transfer the output of the computational model to the
nervous system, not as a computing device itself. These advan-
tages also allow the approach to generalize to other prosthetics
encoder-ChR2 method. (A) Baseline condition (no stimulus present). Blind
animals show a drift in eye position when no stimulus is present, similar to
blind humans; the direction of the drift is downward. (B) Response to
drifting gratings presented using the standard optogenetic method, that is,
where the stimulus was presented unencoded. As shown, no tracking occurs,
just the downward drift. (C) Response to drifting gratings presented using
the encoder-ChR2 method, where the stimulus was presented in its encoded
form (i.e., using ON cell encoders; see main text). As shown, when the
stimulus was converted into the code used by the ganglion cells, the animal
became able to track it. (A–C, Top) Representative example of a raw eye-
position trace. (A–C, Middle) Smooth component (saccades and movement
artifacts removed). (A–C, Bottom) Average trajectory across all trials (n = 15,
14, and 15 trials, respectively).
Behavioral performance: optomotor tracking occurs with the
| www.pnas.org/cgi/doi/10.1073/pnas.1207035109Nirenberg and Pandarinath
problems (or other brain/machine interface applications). The
strategy can be applied to any system in which the computations
can be mathematically characterized. This allows problems to be
solved on a much faster timescale.
Potential for Translation. For delivery of channelrhodopsin, several
groups, including ourselves, are using adeno-associated viral
(AAV) vectors, as clinical and preclinical trials have shown both
safe and efficient gene delivery with AAV in ocular settings
(29–33). In terms of driving the channelrhodopsin, the mini-
DLP, because it is small, can be readily incorporated into an
eyeglasses-based device (Figs. S5 and S6).
One potential bottleneck for the development of this pros-
thetic system for patients is the issue of targeting codes to spe-
cific ganglion cell classes. The ganglion cell population contains
several cell classes. This might raise the concern that, to produce
quality vision, one would need to stimulate each class with its
appropriate code. However, strong evidence suggests that this
is not the case. Patients with Duchenne’s muscular dystrophy
lack ON channel transmission, and, thus, see exclusively through
OFF cells (34, 35). These patients do not report vision problems
(34, 35). They are actually unaware of the deficiency; it becomes
apparent only through electrophysiological measures (35). Thus,
driving just OFF cells with their code has the potential to pro-
duce substantial vision restoration [see SI Materials and Methods
for encoder performance for specific ganglion cell types: for
mouse (Fig. S1) and for primate (Fig. S4), including ON and
OFF midget and parasol classes (Fig. S4)].
Thus, although it is likely that there will be many hurdles to
overcome to bring this technology to patients, the major ones—
a vector (AAV) for delivering ChR2 to ganglion cells, an en-
coder/stimulator device to drive them, and the fact that targeting
a single ganglion cell class by itself can bring substantial vision
restoration—have already been addressed, substantially increasing
the probability of success.
Importance of Retinal Coding in Light of Limited Plasticity. Recent
studies have shown that front-end processing in the visual
system is much less plastic than that in auditory and somato-
sensory systems (summarized in ref. 36). This has been shown in
animals and also in adult patients with macular degeneration (36,
37). The lower plasticity may explain why standard retinal pros-
thetics have been less successful than cochlear implants. The
limited plasticity puts a much heavier burden on retinal pros-
thetics to produce signals close to those of the normal retina.
In sum, our results show that incorporating the code dramati-
cally increases prosthetic capabilities. Although increasing resolu-
tion also improves performance, there is an inherent ceiling on the
quality of image this can produce; adding the code breaks through
this barrier. The coded output combined with high-resolution
stimulation makes natural vision restoration possible.
Materials and Methods
A complete description of materials and methods, including a description of
the model, the device, the animals, and the analysis methods, is provided in
SI Materials and Methods.
ACKNOWLEDGMENTS. We thank Hugh Cahill and Jeremy Nathans for
providing equipment and advice for the optomotor experiments, Illya
Bomash for help with the primate recording, Zack Nichols for sharing his
reconstruction software, and Keith Purpura and Jonathan Victor for com-
ments on the manuscript. This work was supported by National Institutes of
Health (NIH) Grant EY012978, Cornell’s Institute for Computational Biomed-
icine, and NIH Grant EY07977.
1. Chopdar A, Chakravarthy U, Verma D (2003) Age related macular degeneration. BMJ
2. Resnikoff S, Pascolini D, Mariotti SP, Pokharel GP (2008) in Bull World Health Organ
3. Gordois A, Pezzullo L, Cutler H (2010) The Global Economic Cost of Visual Impairment
(Access Economics Pty Limited, Canberra, Australia).
4. Chader GJ, Weiland J, Humayun MS (2009) Artificial vision: Needs, functioning, and
testing of a retinal electronic prosthesis. Prog Brain Res 175:317–332.
5. Zrenner E, et al. (2009) Subretinal microelectrode arrays allow blind retinitis pig-
mentosa patients to recognize letters and combine them to words. Proc 2nd Int Conf
IEEE Biomed Eng Inf, pp 1–4.
6. Winter JO, Cogan SF, Rizzo, JF, III (2007) Retinal prostheses: Current challenges and
future outlook. J Biomater Sci Polym Ed 18:1031–1055.
7. Congdon N, et al.; Eye Diseases Prevalence Research Group (2004) Causes and prev-
alence of visual impairment among adults in the United States. Arch Ophthalmol 122:
8. Loudin JD, et al. (2007) Optoelectronic retinal prosthesis: System design and perfor-
mance. J Neural Eng 4:S72–S84.
9. Sekirnjak C, et al. (2008) High-resolution electrical stimulation of primate retina for
epiretinal implant design. J Neurosci 28:4446–4456.
10. Bi A, et al. (2006) Ectopic expression of a microbial-type rhodopsin restores visual
responses in mice with photoreceptor degeneration. Neuron 50(1):23–33.
11. Lagali PS, et al. (2008) Light-activated channels targeted to ON bipolar cells restore
visual function in retinal degeneration. Nat Neurosci 11:667–675.
12. Tomita H, et al. (2009) Visual properties of transgenic rats harboring the channelr-
hodopsin-2 gene regulated by the thy-1.2 promoter. PLoS ONE 4:e7679.
13. Thyagarajan S, et al. (2010) Visual function in mice with photoreceptor degeneration
and transgenic expression of channelrhodopsin 2 in ganglion cells. J Neurosci 30:
14. Greenberg KP, Pham A, Werblin FS (2011) Differential targeting of optical neuro-
modulators to ganglion cell soma and dendrites allows dynamic control of center-
surround antagonism. Neuron 69:713–720.
15. Doroudchi MM, et al. (2011) Virally delivered channelrhodopsin-2 safely and effec-
tively restores visual function in multiple mouse models of blindness. Mol Ther 19:
16. Eckmiller R, Neumann D, Baruth O (2005) Tunable retina encoders for retina implants:
why and how. J Neural Eng 2:S91–S104.
17. Nirenberg S (2011) Strategies for finding neural codes. Visual Population Codes, eds
Kriegeskorte N, Kreiman G (MIT Press, Cambridge, MA).
18. Jacobs AL, et al. (2009) Ruling out and ruling in neural codes. Proc Natl Acad Sci USA
19. Nagel G, et al. (2003) Channelrhodopsin-2, a directly light-gated cation-selective
membrane channel. Proc Natl Acad Sci USA 100:13940–13945.
20. Boyden ES, Zhang F, Bamberg E, Nagel G, Deisseroth K (2005) Millisecond-timescale,
genetically targeted optical control of neural activity. Nat Neurosci 8:1263–1268.
21. Grimm C, et al. (2004) Constitutive overexpression of human erythropoietin protects
the mouse retina against induced but not inherited retinal degeneration. J Neurosci
22. Hackam AS, et al. (2004) Identification of gene expression changes associated with
the progression of retinal degeneration in the rd1 mouse. Invest Ophthalmol Vis Sci
23. Nirenberg S, Pandarinath C, Ohiohenuan I (2011) Retina prosthesis. International
24. Dann JF, Buhl EH (1987) Retinal ganglion cells projecting to the accessory optic
system in the rat. J Comp Neurol 262(1):141–158.
25. Quian Quiroga R, Panzeri S (2009) Extracting information from neuronal populations:
Information theory and decoding approaches. Nat Rev Neurosci 10(3):173–185.
26. Kass RE, Ventura V, Brown EN (2005) Statistical issues in the analysis of neuronal data.
J Neurophysiol 94:8–25.
27. Paninski L, Pillow J, Lewi J (2007) Statistical models for neural encoding, decoding,
and optimal stimulus design. Prog Brain Res 165:493–507.
28. Giolli RA, Blanks RHI, Lui F (2006) The accessory optic system: Basic organization with
an update on connectivity, neurochemistry, and function. Prog Brain Res 151:407–440.
29. Maguire AM, et al. (2008) Safety and efficacy of gene transfer for Leber’s congenital
amaurosis. N Engl J Med 358:2240–2248.
30. Hauswirth WW, et al. (2008) Treatment of leber congenital amaurosis due to RPE65
mutations by ocular subretinal injection of adeno-associated virus gene vector: short-
term results of a phase I trial. Hum Gene Ther 19:979–990.
31. Cideciyan AV, et al. (2009) Vision 1 year after gene therapy for Leber’s congenital
amaurosis. N Engl J Med 361:725–727.
32. Simonelli F, et al. (2010) Gene therapy for Leber’s congenital amaurosis is safe and
effective through 1.5 years after vector administration. Mol Ther 18:643–650.
33. Mancuso K, et al. (2009) Gene therapy for red-green colour blindness in adult
primates. Nature 461:784–787.
34. Fitzgerald KM, Cibis GW, Giambrone SA, Harris DJ (1994) Retinal signal transmission
in Duchenne muscular dystrophy: Evidence for dysfunction in the photoreceptor/
depolarizing bipolar cell pathway. J Clin Invest 93:2425–2430.
35. Cibis GW, Fitzgerald KM (2001) The negative ERG is not synonymous with night-
blindness. Trans Am Ophthalmol Soc 99:171–175, discussion 175–176.
36. Fine I (2008) The effects of visual deprivation. Artificial Sight, eds Humayun MS,
Weiland JD, Chader G, Greenbaum E (Springer, New York), pp 47–70.
37. Baseler HA, et al. (2011) Large-scale remapping of visual cortex is absent in adult
humans with macular degeneration. Nat Neurosci 14:649–655.
Nirenberg and PandarinathPNAS
| September 11, 2012
| vol. 109
| no. 37