ArticlePDF Available

Novel stimuli evoke excess activity in the mouse primary visual cortex


Abstract and Figures

Significance Rapid detection and processing of stimulus novelty are key elements of adaptive behavior. Predictive coding theories postulate that novel stimuli should be encoded differently from familiar stimuli. Here, we show that the majority of neurons in layer 2/3 of the mouse primary visual cortex exhibit a significant excess response to novel visual stimuli. The distinction between novel and familiar images developed rapidly, requiring only a few repeated presentations. We show that this phenomenon can be described by a model of cascading adaptation. This ubiquitous mechanism makes it likely that similar computations could be carried out in many brain areas.
Content may be subject to copyright.
Novel stimuli evoke excess activity in the mouse
primary visual cortex
Jan Homann
, Sue Ann Koay
, Kevin S. Chen
, David W. Tank
, and Michael J. Berry II
Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544
Edited by Yang Dan, Department of Molecular and Cell Biology, Helen Wills Neuroscience Institute, University of California, Berkeley, CA; received May 21,
2021; accepted December 14, 2021
To explore how neural circuits represent novel versus familiar
inputs, we presented mice with repeated sets of images with
novel images sparsely substituted. Using two-photon calcium
imaging to record from layer 2/3 neurons in the mouse primary
visual cortex, we found that novel images evoked excess activity
in the majority of neurons. This novelty response rapidly emerged,
arising with a time constant of 2.6 ±0.9 s. When a new image set
was repeatedly presented, a majority of neurons had similarly ele-
vated activity for the rst few presentations, which decayed to
steady state with a time constant of 1.4 ±0.4 s. When we
increased the number of images in the set, the novelty responses
amplitude decreased, dening a capacity to store 15 familiar
images under our conditions. These results could be explained
quantitatively using an adaptive subunit model in which presyn-
aptic neurons have individual tuning and gain control. This result
shows that local neural circuits can create different representa-
tions for novel versus familiar inputs using generic, widely avail-
able mechanisms.
visual system jprimary visual cortex jnovelty response jadaptation j
predictive coding
Because the behavioral consequences of a sensory stimulus
can depend on whether that stimulus is novel or familiar,
sensory systems can benefit from employing different represen-
tations of novel versus familiar stimuli. At the level of human
psychophysics, stimulus novelty can enhance salience and cap-
ture attention (1–3), while familiarity can speed visual search
(4). Novelty also affects aversive conditioning (5–7) and fear
conditioning (8, 9). In human brain imaging, novel stimuli have
been shown to generate the mismatch negativity (MMN) (10,
11) while repeated stimuli lead to repetition suppression (12).
Explicit representation of novelty has been shown at higher
stages of the sensory hierarchy, such as in the hippocampus
(13) and inferotemporal cortex (14–16), and has been inter-
preted as a possible substrate of recognition memory (17).
Lower in sensory hierarchies, the representation of novelty can
be enhanced by stimulus-specific adaptation (SSA) (18–21) as
well as by gain control (22, 23). Novelty signals are also promi-
nently present in midbrain dopamine neurons (24).
Explicit representation of stimulus novelty is also related to
theories of predictive coding, in which neural circuits carry out
computations that emphasize novel or surprising information.
Theories of predictive coding have had a long history, starting
with ideas about how the receptive field structure of retinal gan-
glion cells more efficiently encodes natural visual scenes by remov-
ing redundant data (25–28) and including the idea that active
adaptation may aid in this process (18). Theories of predictive
coding in the neocortex have typically focused on the idea that
feedback from higher cortical areas encodes a prediction about
lower-level sensory data (29) that is subtracted from the lower-
level representation, so that the signals traveling up the cortical
hierarchy represent surprise or novelty (30, 31). However, a recent
study failed to find these signatures of predictive coding (32).
Here, we investigate novelty processing in the mouse primary
visual cortex. We repeatedly presented a set of images, each
composed of a random superposition of Gabor functions, and
then occasionally presented novel images drawn from the same
ensemble. Using two-photon imaging of the Ca
GCaMP6f to measure neural activity in layer 2/3 of awake,
head-fixed mice (33), we found that the majority of neurons
exhibited excess activity in response to a novel image. This dis-
tinction between novel versus familiar images was quickly
reached, emerging with a time constant of 2.6 ±0.9 s. Similarly,
when we began presenting a new set of images, a majority of
the neurons exhibited elevated firing that relaxed to a steady
state with a time constant of 1.4 ±0.4 s. When we presented
novel images within larger image sets, the amplitude of novelty
response decreased, defining a capacity of the system to encode
15 familiar images. All of these findings could be explained
qualitatively using an adaptive subunit model in which neurons
presynaptic to a recorded neuron have both individual tuning
to visual stimuli and adaptive gain control.
In order to explore how the primary visual cortex encodes nov-
elty, we used two-photon Ca
fluorescence imaging in mice
that were awake and head-fixed but free to move on a styro-
foam ball placed below them (Fig. 1A) (34, 35). Animals were
transgenics from a Thy1 line (GP5.3; Janelia) that expressed
the protein GCaMP6f (36) in excitatory neurons. In order to
locate the primary visual cortex, we first carried out large-scale
brain mapping with a one-photon macroscope using drifting
bars (Fig. 1Band SI Appendix,Supplementary Methods). We
then selected a field of view in V1 and imaged at cellular
Rapid detection and processing of stimulus novelty are key
elements of adaptive behavior. Predictive coding theories
postulate that novel stimuli should be encoded differently
from familiar stimuli. Here, we show that the majority of
neurons in layer 2/3 of the mouse primary visual cortex
exhibit a signicant excess response to novel visual stimuli.
The distinction between novel and familiar images devel-
oped rapidly, requiring only a few repeated presentations.
We show that this phenomenon can be described by a
model of cascading adaptation. This ubiquitous mechanism
makes it likely that similar computations could be carried
out in many brain areas.
Author contributions: J.H., D.W.T., and M.J.B. designed research; J.H., S.A.K., K.S.C.,
and M.J.B. performed research; J.H. and M.J.B. analyzed data; and J.H., D.W.T., and
M.J.B. wrote the paper.
The authors declare no competing interest.
This article is a PNAS Direct Submission.
This article is distributed under Creative Commons Attribution-NonCommercial-
NoDerivatives License 4.0 (CC BY-NC-ND).
To whom correspondence may be addressed. Email:
This article contains supporting information online at
Published January 31, 2022.
PNAS 2022 Vol. 119 No. 5 e2108882119 j
resolution in layer 2/3. We manually identified regions of inter-
est (ROIs) corresponding to cell bodies having a “halo” pattern
of fluorescence indicating expression mostly in the cytoplasm
(Fig. 1C). After averaging across pixels in an ROI to obtain
the average fluorescence F, we constructed the time course of the
fractional change in fluorescence (ΔF/F). The activity in single
ROIs typically exhibited sparse events on a background (Fig. 1E).
In this two-photon setup a toroidal projection screen was
placed around the mouse, which allowed us to display visual
stimuli. We wanted to create an ensemble of diverse images
that shared the same low-level statistics but were different in
detail. We also wanted those images to drive V1 strongly. To
this end, we chose images that consisted of a random superpo-
sition of 100 Gabor functions on a gray background. Those
Gabor functions were drawn from a distribution that matched
the range of receptive field parameters found in the literature
for mice (SI Appendix,Supplementary Methods) (Fig. 1D) (37,
38). Images constructed in this way were statistically similar in
global light level, contrast, and spatial scale. We then formed sets
of three or four of those images, each displayed for 250 or 300 ms,
depending on the specific experiment. In order to create a familiar
stimulus, we repeated this image set many times without blanks in
image drawn from the same distribution (Fig. 1D).
The Novelty Response. In the novelty experiment we repeated a
set of four images, each with a duration of 250 ms. A single
image set was repeated for 10 min while novel images were
randomly substituted every 6s(SI Appendix,Supplementary
Methods). All novel images were unique images drawn from the
same ensemble of random Gabor images that the repeated
image set was drawn from. Neurons generally showed a noisy,
weakly modulated response to the repeated image set. How-
ever, when a novel image was substituted, neurons showed a
large, brief response (Fig. 1 Eand F). Some neurons responded
to a majority of the novel images (example cell in Fig. 1F), indi-
cating that their novelty response was somewhat unspecific to
the particular locations and orientations of the Gabor patches
overlapping the cell’s receptive field.
To quantify this response across the neural population, we
subtracted the activity triggered by the repeated image set from
that triggered by the presentation of a novel image (Fig. 2A,
black line). We found that a large majority of the 1,134 neural
responses in our sample showed, on average, excess activity to
a novel image (Fig. 2B)(P<0.05 for 878/1,134 =77%; SI
Appendix,Supplementary Methods). The amplitude of the excess
activity varied across the population ranging from slightly nega-
tive for some neurons up to values greater than ΔF/F 0.2
(Fig. 2C). We formed averages over groups of neurons sorted
by their rank (Fig. 2D); this analysis showed that the temporal
dynamics of the novelty response were the same, regardless of
response amplitude.
In order to better quantify the response amplitude, we first
averaged the raw responses across all the neurons (Fig. 2F,
black line). Then, we fit a simple curve to capture the dynamics
of the novelty response (Fig. 2F, dotted line). Finally, we quan-
tified the calcium dynamics by analyzing published data (36)
(Fig. 2E) and deconvolved to estimate the spiking rate of the
population as a function of time (Fig. 2F, blue line). Even
though the averaged excess signal was relatively small in peak
amplitude (0.04 ΔF/F), deconvolution revealed a significant
increase in spike rate from a baseline of 1.2 Hz per cell up to
a peak of 4 Hz per cell. The integrated area of this excess
response corresponded to about 0.5 spikes per cell per trial.
cell 1
cell 2
2 sec
Fig. 1. Measuring a novelty response. (A) Awake mice were head-xed and placed on an air-suspended styrofoam ball. Visual stimuli were projected on
a toroidal screen surrounding the animal. Neural activity was recorded with a two-photon microscope. (B) Wide-eld image of visual areas, as determined
by one-photon uorescence measurements (see Materials and Methods). The black square within area V1 shows the size of the eld of view for the two-
photon microscope. (C) Portion of a eld of view taken by the two-photon microscope with ROIs shown in red. (Scale bar, 50 μm.) (D) Stimulus design.
Step 1: Images were constructed from a superposition of randomly chosen Gabor functions. Step 2: A set of different images was formed and presented
repeatedly in the same order; image sets are represented by plotting the image index versus time. Step 3: Occasionally, an image was substituted by
unique novel images drawn from the same image ensemble. (E) Example activity traces with the times of novel image presentations shown in red; all
novel images were unique. (F,Top) Matrix of trial-by-trial responses of an example cell to novel images. (F,Middle) Activity averaged across trials. (F,Bot-
tom) Repeated sequence with the time of novel images shown in red.
2of12 jPNAS Homann et al. Novel stimuli evoke excess activity in the mouse primary visual cortex
Multiplying by the number of neurons, we estimate that a novel
image elicits 150,000 excess spikes in all of V1 (SI Appendix,
Supplementary Methods). Therefore, the fact that the novelty
response was present in such a large fraction of neurons led to
a substantial increase in population activity.
Neuropil Subtraction. Because the novelty response was so
widely distributed among neurons, we were concerned that sig-
nals from the neuropil might be making a significant contribu-
tion. To this end, we estimated the local neuropil activity
around each neuron by constructing an annulus around the
soma and averaging over all of the pixels in this annulus. We
first identified fast events in the neuropil signal, which are likely
to be action potentials from other neurons. If these signals
were to “bleed over” into the ROI we were observing, then we
would see a corresponding event, albeit at a small amplitude, in
the ROI for every such fast signal. We found many examples of
fast signals in the neuropil that were not reflected in the ROI
(SI Appendix, Fig. S1A), implying that contamination of the
ROI signal by the neuropil was not significant. However, it is
still possible that this contamination was present but noisy
enough that it was hard to measure on a single trial.
To this end, we constructed the average activity in the ROI
triggered on the peak of fast events in the neuropil, with both
signals normalized so that the peak of the neuropil signal had
an amplitude of unity (SI Appendix, Fig. S1B). The peak signal
in the ROIs was 0.27, which implied that neuropil may have
contaminated the ROI signal by as much as 27%. However, it
is also possible that fast events in the neuropil were distinct
from fast events in the ROI but were correlated in their occur-
rence. To further investigate, we examined the trial-by-trial
amplitude of fast neuropil events versus the corresponding
amplitude of ROI signals both as a histogram of the response
ratio (SI Appendix, Fig. S1C) and as a scatter plot (SI Appendix,
Fig. S1D). These data have considerable scatter, but they do
not rule out the possibility of direct contamination.
Fig. 2. Population summary for the novelty response. (A,Top) Repeated image set containing novel images shown in orange. (A,Middle) Activity of one
example neuron, averaged across trials with and without novel images (red vs. blue). (A,Bottom) Excess activity due to the occurrence of novel images
(black). (B) Excess activity for 1,134 trial-averaged neural responses (rows) plotted versus time relative to the occurrence of novel images and sorted by
response amplitude (color scale =z-score). (C) Histogram of amplitudes of the excess activity for all cells. (D) Excess neural activity (normalized) for differ-
ent groups of neurons sorted by response rank (colors). (E) GCaMP6f response to a single spike, taken from ref. 36. (F) Population-averaged excess activity
(black line; ΔF/F) with a curve t capturing the response dynamics (dotted black line) and a model of the spiking rate (blue line).
Homann et al.
Novel stimuli evoke excess activity in the mouse primary visual cortex
PNAS j3of12
Therefore, we examined the novelty response after subtracting
a maximal amount of contamination from the raw fluorescence
trace, F
. This neuropil subtraction had a
negligible effect on the novelty response (SI Appendix,Fig.S1E
and F). Exploring the neuropil subtraction process in more detail,
we found that the population-averaged fluorescence in the neuro-
pil and ROIs were qualitatively the same, so that subtraction only
produced a small baseline offset (SI Appendix,Fig.S1G). At the
single-cell level, subtraction reduced the fraction of neurons with
a significant novelty response from 0.89 to 0.79 in this examined
dataset (SI Appendix,Fig.S1Hand I). Overall, we concluded that
the novelty response is clearly present in individual neurons and
not merely an artifact of neuropil contamination. Because our
data are also consistent with the neuropil containing signals that
are distinct but correlated with those in the ROI, we subsequently
report uncorrected fluorescence data.
At the same time, we found that novelty responses were
strongly reflected in the average neuropil signal. When we plotted
the mean neuropil signal following a novel image, we found clear
novelty responses in all of the annuli studied (SI Appendix,Fig.S2
Aand D). This signal was similar to that found in ROIs for both
its amplitude distribution (SI Appendix,Fig.S2B) and population
average (SI Appendix,Fig.S2C). Overall, the novelty response in
the neuropil was similar to that found in ROIs, but with some-
what greater extent. These results are all consistent with the find-
ing that most neurons had a novelty response, as processes from
nearby neurons comprise much of the neuropil.
Is the Novelty Response Caused by Surprise? Sudden, unexpected
visual stimuli, like a dark looming disk from above (39) or the
onset of a bright light (40), can cause strong behavioral
responses in mice, like flight or freezing. Thus, the novel image
might trigger changes in locomotion, which could then influ-
ence neural activity. To address this possibility, we tested for
changes in running speed at the onset of a novel image. Even
though our mice showed strong alternations between restful-
ness and bouts of running (Fig. 3A), we found that there was
no change in running speed that was triggered by a novel image
(Fig. 3B). From these control analyses, we conclude that the
novelty response is not the result of the animal’s change
in locomotion.
Could the novelty response be the result of other forms of
behavioral surprise? Surprise can also lead to activation of the
sympathetic nervous system, which in turn can increase the
evoked firing rates of neurons in V1 through release of norepi-
nephrine and acetylcholine in V1 (41–44). Changes in pupil
diameter can be used as a proxy for changes in norepinephrine
activity (45). Thus, we tracked the animal’s eye position and
measured their pupil diameter. This allowed us to test whether
the presentation of a novel image changed the pupil diameter
of the mouse. While pupil diameter did fluctuate strongly dur-
ing a recording session (Fig. 3A), this fluctuation was not corre-
lated with the onset of a novel image (Fig. 3B).
Finally, behavioral surprise could cause eye movements,
which could then lead to increased neural activity. Thus, we
tested whether the novel image triggered saccadic eye move-
ments. We found no correlation between the onset of a novel
image and eye velocity (Fig. 3B). Together, these analyses
strongly suggest that the novelty response is not caused by a
generalized startle or behavioral surprise response. While this
result might seem counterintuitive, it is consistent with the fact
that the presentation of a novel image under these conditions has
very low salience for human observers. Readers are encouraged
to view video clips of the visual stimulus (Movies S1 and S2).
Influence of Locomotion and Eye Movements. It is known that
signals in V1 can be modulated by the animal’s locomo-
tion (46–48). Perhaps the novelty response is greatly enhanced
by locomotion? To test this possibility, we averaged the Ca
response to a novel image over periods when the animal was
either running or still. We found an enhancement of the novelty
response during running (SI Appendix, Fig. S3A), similar to pre-
vious reports. Combined across our entire recording, we found
a mild positive correlation between running speed and neural
activity (SI Appendix, Fig. S3B).
Pupil dilation can also modulate neural activity in the pri-
mary visual cortex. To test whether pupil diameter enhanced
the novelty response, we similarly averaged neural activity on
trials with dilated versus constricted pupils. Pupil dilation led to
a substantial increase in baseline activity but little change in the
amplitude of the novelty response relative to baseline (SI
Appendix, Fig. S3C). At the same time, the correlation between
pupil diameter and neural activity was stronger than for run-
ning (SI Appendix, Fig. S3D).
Finally, we also tested whether saccadic eye movements
enhanced the novelty response. Similar to the influence of pupil
diameter, we found that on trials with a saccadic eye movement
neural activity had a higher baseline than for trials with no eye
Pupil diameter
Eye velocity
Running speed
Novel image
1 sec
Eye velocity
a. u.
a. u.
Pupil diameter
0 5 10 15 20 25
cm / sec
Running speed
Fig. 3. The novelty response is not caused by running, pupil size, or eye movements. (A) Traces of behavioral variables during the course of a 1-h experi-
ment. (Top) Pupil displacement from resting position. (Middle) Pupil diameter. (Bottom) Running speed. (B) Behavioral variables triggered on the occur-
rence of novel images (red dotted line). Shaded areas are uncertainty estimates. (Top) Pupil displacement (blue). (Upper Middle) Pupil diameter (green).
(Lower Middle) Running speed (red). (Bottom) Trial-averaged response of the neural population (black).
4of12 jPNAS Homann et al. Novel stimuli evoke excess activity in the mouse primary visual cortex
movements (SI Appendix, Fig. S3E). However, the amplitude of
the novelty response relative to baseline was unchanged. We
found fairly little correlation between saccades and neural
activity (SI Appendix, Fig. S3F), suggesting that the increase
baseline activity may have resulted from correlation between
eye movements and the animal’s modulatory state.
Influence of Image Order. Is the novelty response generated by a
violation in the expected temporal order of images or by the
spatial novelty of the image alone? To this end, we designed an
order violation experiment by occasionally switching the order
of familiar images rather than inserting novel images (SI
Appendix,Supplementary Methods). We found very little excess
activity in this experiment (Fig. 4), suggesting that the novelty
response does not signal a violation of the temporal order per
se, but rather the spatial novelty of an image. We nevertheless
retained a stimulus design that presents familiar images in a
consistent order. This temporally regular presentation of famil-
iar images within a set avoids the trial-to-trial variability that
would be otherwise be caused by a randomized presenta-
tion order.
Emergence of the Novelty Response. Conceptually, a neuron can-
not exhibit a differential response to a novel image until the
original set of images is repeated at least once. We wanted to
see how many repetitions of a sequence were necessary so that
neurons would show an elevated response to a novel image. To
this end, we designed a variable repetition experiment in which
we displayed sets with three images of 300-ms duration. We
presented a given image set in a block with either L=1, 3, 8,
18, or 38 repetitions of the image set before showing a novel
image (Fig. 5Aand SI Appendix,Supplementary Methods). Each
block contained a unique image set and a unique choice of rep-
etitions, L. Blocks followed each other seamlessly with no inter-
trial period or blank frame.
We found that the novelty response emerged rapidly. Signifi-
cant excess activity was observed in the population after as few
as L=1repeats of a new image set (Fig. 5A,Bottom). The
effect increased with more repetitions and saturated at L20
(Fig. 5 A,To p ). Fitting an exponential curve to the effect ampli-
tude versus number of repetitions revealed a time constant of
=2.6 ±0.9 s, or alternatively 2.9 ±1.0 repeats, for the
emergence of the novelty response.
The Transient Response. We also noticed that neurons exhibited
elevated activity when we began presenting a new image set
(Fig. 5A, elevated activity near t =0). In this experiment, each
adaptation block was always preceded seamlessly by a block
that used a different image set with no intertrial period or
blank frame in between. Therefore, this initial elevated activity
is conceptually very similar to the novelty response. This tran-
sient response adapted strongly as a given image set was
repeated, quickly reaching a steady state. This adaptation pro-
cess had a time constant of τ
=1.4 ±0.4 s in the variable
repetition experiment.
A transient response is also evident after the occurrence of
a novel image, because then a new image set was presented
(Fig. 5A, elevated activity near t =40 s). Similar to the case of
the novelty response, a given image set must be repeated before
a response can emerge from the transition to a novel sequence.
We measured the amplitude of the transient response elicited
by the transition to a novel sequence as a function of the num-
ber of repeats of the preceding image set, L, and found that the
emergence of the transient response closely paralleled that of
the novelty response (Fig. 5 A,To p ).
How long does adaptation to familiar images persist? All
sequences used in the variable repetition experiment were dif-
ferent. To answer this question, we designed a repeated image
set experiment that adapted neurons to a given image set and
then, after various lags, presented the same sequence again
(Fig. 5Band SI Appendix,Supplementary Methods). After a suf-
ficiently long recovery time, we expect the transient response to
recover to its maximum amplitude, similar to what we found
for the emergence of the novelty response. To make sure we
did not create transition effects due to the lack of neural stimu-
lation during the recovery period, we filled this period with
other image sets.
We measured the amplitude of the transient response as
the difference between the response to the last few repeats of
the previous sequence and the peak calcium fluorescence of the
response elicited by the new sequence. The previous sequence
was repeated sufficiently often so neurons had reached steady-
state activity. When we plotted the amplitude of the transient
response as a function of the time since the same image set was
presented, we found an exponential rise with a time constant of
=33 ±11 s (Fig. 5B). The amplitude at a time interval
of zero was set to zero, because in this case there was no pause
between the adaptor stimulus and the probe, and thus no tran-
sition that could elicit an excess response.
The Novelty Response as a Probe of Capacity. In order for a neu-
ral circuit to distinguish novel from familiar images, it must
maintain some representation of familiar images. To explore
the limitations of this representation, we designed a variable
image number experiment, in which we varied the number of
images in the set. Similar to the variable repetition experiment,
we formed adaptation blocks. In each block, i, we randomly
chose the number of images to be S
=3, 6, 9, or 12 and then
randomly generated a new image set of this size (Fig. 6Aand
SI Appendix,Supplementary Methods). In all blocks, the given
image set was presented for 17 times before a novel image was
introduced in the 18th trial. This allowed us to measure the
amplitude of the novelty response after adapting to image sets
of different size.
Fig. 4. The novelty response is not caused by changes of image order. (A)
Population-averaged excess activity triggered by novel images. (B)
Population-averaged excess activity triggered by an image order violation.
In Aand B, error bands were computed by rst taking the average of all
cell responses for each mouse and then computing the SEM across those
ve traces (gray). Error bands are therefore indicative of mouse-to-mouse
Homann et al.
Novel stimuli evoke excess activity in the mouse primary visual cortex
PNAS j5of12
We found that the novelty response systematically decreased
as the number of images, S, increased (Fig. 6 Band C). This
result is consistent with the intuition that as the neural circuit
encodes more images, then yet another image will seem less
novel. Although the novelty response decreased with larger image
sets, it was still present for sets of S=12 images. This indicates
that under our experimental conditions the capacity was at least
as large as 12 images. Another way of characterizing the capacity
is to examine the trend of novelty response amplitude versus size
of the image set, S. Here, we found an exponential decrease with
a decay constant of τ
=15 images (Fig. 6C). This decay cons-
tant can be seen as a measure of the capacity to store familiar
One way that the neural circuit could generate a smaller nov-
elty response could be by having a higher steady-state response
for larger image sets but always the same response level to
0 10203040
T (sec)
Recovery from adaptation
Emergence of novelty response
T (sec)
transient response F/F
recovery = 33 ± 11 sec
0 20 40 60 80 100 120 140
. . .
sequence repeats
novelty response F/F
transient response T
novelty response N
= 2.8 ± 1 repeats
0 20304010
Fig. 5. Novelty responses emerge quickly within a repeated sequence. (A) In the variable repetition experiment, we formed blocks containing a new,
randomly chosen image set (shown in blue; previous block shown in green). This unique image set was presented once and then immediately followed by
Lrepetitions. Next, an image set was presented with a unique novel image substituted (shown by red bars). Finally, two more image sets were presented
before a new block began (shown in purple). (Bottom) Neural activity averaged over the entire population for different choices of L(curves offset for
clarity). (Top) Amplitude of the novelty response versus the number of repetitions, L. Dotted line: Exponential curve t; error bars are SEM over n=5
mice. (B,Top) Design of the repeated image set experiment. A given image set was repeated in a block until adaptation reached steady-state (blue). The
same image set was presented again after a variable delay during which other image sets were presented (green). (B,Bottom) Transient response ampli-
tude versus delay with an exponential curve t (red); error bars are SEM over n=5 mice.
Fig. 6. Novelty response from within larger image sets. (A) Image sets with either S=3, 6, 9, or 12 images were presented 17 times before one of the
images was substituted by a unique novel image. All images were presented for 300 ms each. (B) Trial-averaged response of the neural population to
novel images during image sets of different size, S(shown in color). (C) Amplitude of the novelty response versus image set size, S. Dotted line is an expo-
nential curve t; error bars are SEM for n=5 mice. (D) Population-averaged steady-state neural activity versus image set size, S; error bars are SEM for
n=5 mice.
6of12 jPNAS Homann et al. Novel stimuli evoke excess activity in the mouse primary visual cortex
novel images. In this case, there would be less dynamic range to
generate a large novelty response. However, we found that this
was not the case (Fig. 6D). Because the adaptation process
reached the same steady-state activity as a function of the
image set size, S, we interpret the decrease in the novelty
response amplitude as arising from limitations in the represen-
tation of familiar images. The constancy of steady-state neuron
activity across image sets of different sizes is consistent with
previous observations that cortical adaptation helps to achieve
a form of population homeostasis (49).
Adaptation versus Steady-State Response. In order to study the
connection between this novelty response and known adapta-
tion phenomena in V1, we examined how the amplitude of the
transient response depended on the steady-state activity. In
classical contrast adaptation in V1, neurons show a transiently
elevated firing in response to higher contrast stimuli that adapts
down to a lower steady-state level. Then, when contrast is
reduced, firing rates start lower and recover back to a higher
steady-state level (Fig. 7 A,Left) (50, 51). Our results were
qualitatively different, in that a transition to any new set of
images always caused a transiently elevated activity that
decayed downward to a new baseline that could be higher or
lower than the previous steady-state activity (Fig. 7 A,Right).
Although our stimuli do not differ globally in contrast, the con-
trast in a local region the size of a cell’s receptive field does dif-
fer across images. In addition, the local orientation can change
across images, which can drive orientation-selective neurons at
different strengths.
In order to explore those different types of stimulus transi-
tions further, we separated out individual stimuli according to
how much steady-state activity they evoked in individual neu-
rons. For this, we used the repeated image set experiment and
calculated the event-triggered activity (ETA) of each neuron
during each unique image set. Because of the great heterogene-
ity among neurons, we divided the ETAs from all of our mea-
sured conditions into six groups according to the rank of their
average steady-state activity. We further divided each of these
into six more groups according to the average steady-state activ-
ity during the preceding sequence. This resulted in a 6-by-6
matrix of activity traces, where rows show neural responses with
increasing steady-state activity to the current image set and
columns show increasing steady-state activity to the preceding
image set (SI Appendix,Fig.S4). In this manner, we could
/ ( F/F)
0.00 0.05 0.10
ΔF/F sustained
ΔF/F transient
y=1.36x + 0.016
Fig. 7. Classical contrast adaptation versus the novelty response. (A,Left) Schematic of classical contrast adaptation. The transition from high to low con-
trast typically causes a reduction in ring followed by a slow recovery to higher steady-state ring. (A,Right) Schematic of the novelty response. All tran-
sitions to a new image set cause a transient response, regardless of the steady-state activity. (B) Example trace of population neural activity for transitions
to image sets that drive stronger (Top) versus weaker (Bottom) steady-state activity. (C) Amplitude of the transient response versus the ratio of steady-
state response to the current versus preceding image set; dots are for each of 36 rank-ordered groups of neurons (see main text). (D) Transient activity
versus steady-state activity; dots are averages over rank-ordered groups of 60 neurons. Black dotted line: unity; red line: linear curve t. (Upper Inset) Nor-
malized activity of neurons with large steady-state responses (from blue oval). (Lower Inset) Normalized activity of neurons with weak steady-state
responses (from magenta oval).
Homann et al.
Novel stimuli evoke excess activity in the mouse primary visual cortex
PNAS j7of12
systematically investigate how the transient response depended
on the steady-state response of the current and preceding image
We found a transient response with excess activity following
nearly all transitions between different levels of steady-state
activity. Of note, we show an example group with a transition
to much larger steady-state activity along with the opposite
transition (Fig. 7B). In both cases, a transient response was
clearly seen. We summarized our results by plotting the ampli-
tude of the excess transient activity versus the ratio of steady-
state activity in current versus preceding image sets (Fig. 7C).
We found robust excess transient activity across these
A more detailed analysis of the transient response showed a
mix of additive and multiplicative component. To demonstrate
this, we plotted the activity during the first presentation of a
new image set (transient activity) versus the activity during the
last presentation (steady-state activity; Fig. 7D). If the transient
activity was a multiple of the steady-state activity, then these
data would lie along a line with slope greater than 1. If instead
the transient response was independent of the steady-state
activity, then the transient activity would have an additive
We found a combination of both effects. Neurons with larger
steady-state activity had transient activity scaled by a factor
1.36, while neurons with smaller steady-state activity had an
additive offset of 0.016 ΔF/F. Thus, in weakly responding
cells, the additive component contributed the most, whereas in
strongly responding cells the multiplicative component domi-
nated. Because the population-averaged novelty response has
an amplitude of 0.04 ΔF/F (Fig. 4A), the additive component
contributes 0.016/0.04 =40% and the multiplicative compo-
nent contributes 60%.
In addition, we found that the dynamics of adaptation
depended on the steady-state activity. Neurons with larger
steady-state activity tended to have a slower decay of activity
=4.0 ±0.3 s), reminiscent of contrast adaptation (Fig. 7
D,Upper Inset). Conversely, neurons with smaller steady-state
activity had a faster decay of activity (τ
=0.93 ±0.06 s) (Fig.
7D,Lower Inset).
A principal component analysis of the temporal profiles of
activity following the transition to a new image set resulted
in two principal components capturing 87% of the variance
(Fig. 8 Aand B). While the second principal component
showed fast, transient activity, the first component showed a
faster initial decay on top of a slower decay of activity. How-
ever, linear combinations of the two first principal components
closely matched both the fast, transient response observed in
weakly active cells and the slowly decaying response observed
in strongly active cells (Fig. 8 Cand D).
We can understand these results by assuming that neurons
exhibit a combination of novelty response and classical contrast
adaptation. Because the novelty response tended to have a
modest amplitude, this activity would barely be visible in a neu-
ron with large steady-state activity. Since these neurons would
typically have lower steady-state activity in the preceding
sequence, we can think of the neuron as experiencing higher
effective contrast, which would trigger contrast adaptation. On
the other hand, neurons with weak steady-state activity would
be experiencing a lower effective contrast or would simply not
have enough steady-state activity to exhibit classic contrast
adaption. Thus, for these neurons, most of their activity would
arise from the novelty response. Together, these results argue
that the novelty response may be distinct from contrast adapta-
tion in V1 neurons.
Adaptive Subunit Model. In order to explain and unify our obser-
vations, we formulated an adaptive subunit model. The idea is
that a given neuron that we recorded from in layer 2/3 received
inputs from many other cortical neurons (subunits) that need
not have been observed and that each of these presynaptic neu-
rons had its own individual tuning to visual stimuli along with a
cell-intrinsic mechanism of adaptation (Fig. 9A). Because each
presynaptic neuron had different tuning to stimuli, there was
always a subset of those subunits that had high sensitivity to
any novel image. As that image was repeated, those subunits
adapted, giving rise to lower activity in the recorded neuron. At
the same time, subunits that were poorly stimulated by a given
set of images will recover their full sensitivity, so that excess
activity was observed in response to a different image.
In this model (SI Appendix,Supplementary Methods), subu-
nits have a cellular time constant, τ
, chosen to be 20 ms,
along with a time scale of adaptation, τ
, chosen to be 30 s to
match the measured recovery time constant (Fig. 5B). The
most likely mechanism underlying this gain control is an adap-
tive ion channel (52), such as the Na
-dependent K
found in V1 neurons, which has a time constant of 10 to 30 s
(51, 53). Other mechanisms of spike frequency adaptation can
have similar time scales and effects (54).
This process of adaptation causes the subunit’s gain and
hence its response to repeated presentations of the same image
to decrease (Fig. 9B). Notice that the induction of adaptation
depends on τ
as well as both strength parameters, p
and q
However, because of the sparseness of subunit activation,
recovery is determined primarily by τ
. Recorded cells, k,
integrate over all of their subunits with randomly chosen synap-
tic weights, q
, and with the same cellular time constant, τ
The model contains many recorded cells and averages over
their activity to compare with population-averaged experimen-
tal data.
0 5 10 15 20
principal component nr.
% variance explained
0 5 10 15 20
time (sec)
PCA components
0 5 10 15 20
time (sec)
a. u.
slow component
PC1- 0.3 PC2
0 5 10 15 20
time (sec)
a. u.
fast component
0.3 PC1+PC2
Fig. 8. Two principal component analysis (PCA) components capture most
of the response variance when transitioning from one set of images to the
next. (A)Therst ve PCA components, scaled by their component weights,
from responses transitioning from one set of images to the next set of
images. The duration of one repeat of an image sequence was 900 ms. (B)
Variance explained by each of the rst 20 principal components. The two rst
principal components captured 87% of the variance. (C)Thedecayin
response amplitude of the strongly active cells was well captured by a linear
combination of the rst two principal components. (D) Similarly, the decay in
response amplitude of the weakly active cells was also well captured by
another linear combination of the rst two principal components.
8of12 jPNAS Homann et al. Novel stimuli evoke excess activity in the mouse primary visual cortex
Population activity in the model starts out high during the
presentation of a new image set and then adapts down to a
lower steady state (Fig. 9C). When a novel image is presented,
there is excess activity (Fig. 9C). This excess activity decays
to a steady-state baseline with a time constant, τ
=2.3 s
(Fig. 9D), that agrees with experimental data (Fig. 5). All new
image sets and novel images evoke similar levels of increased
When we increase the number of image-set repetitions, L,
before the presentation of a novel image, the amplitude of the
novelty response, N, increases (Fig. 9D) with a roughly expo-
nential dependence on L(Fig. 9E). This trend is dominated by
the fact that the transient response had not fully adapted away
for small L. In addition, there is a modest recovery of sensitivity
of the subunits most strongly driven by the novel stimulus that
leads to a small increase in the absolute amplitude of the nov-
elty response. This recovery occurs because most of the subu-
nits driven by the novel image were not activated by the image
set, allowing their gain to recover.
When we increase the size of the image set, S, the amplitude
of the novelty response, N, decreases (Fig. 9F) with a roughly
exponential dependence (Fig. 9G). This phenomenon results
from cross-adaptation. Subunits tended to have a strong response
to a small fraction of all the images and weak responses to the
other images. With a larger image set, the weak responses to
these other images induces increasingly more adaptation. Because
Time (sec)
Transient Response, T
Time Interval (sec)
= 30.2 sec
= 30 sec
Population Activity
Time (sec)
L = 1
L = 3
L = 5
L = 8
L = 18
= 2.3 sec
Novelty Response, N
Repeats, L
= 3.6 sec
Novelty Response, N
Image Set Size, S
= 9.6
Novelty Response, N
Time (sec)
S = 3
S = 6
S = 9
S = 12
Time (sec)
Image i
Subunit j
Neuron ksynapses
Adaptive Subunit Model Subunit Dynamics
Fig. 9. Adaptive subunit model. (A) Schematic of the model: Images, i, provide input to Msubunits, j, via synapses, p
, and subunits provide input to N
neurons, k, via synapses, q
.(B) Illustration of subunit dynamics: A sequence of images (Bottom) drives sparse responses in a subunit (Top) that decrease
over time due to a reduction in gain (Middle) during each image presentation. (C) Population-averaged ring rate (Bottom) in neurons during presenta-
tion of three different image sets (Top). (D) Population-averaged ring rate during adaptation blocks with different numbers of repetitions of the same
image set before the presentation of a novel image, L; dotted line is an exponential curve t. (E) Amplitude of the novelty response, N, versus number of
repetitions, L; dotted line is an exponential curve t. (F) Population-averaged ring rate following presentation of a novel image during adaptation
blocks with different sizes of image set, S.(G) Amplitude of the novelty response, N, versus size of the image set, S; dotted line is an exponential curve
t. (H) Amplitude of the transient response, T, plotted as a function of the time interval between image set presentations; dotted line is an exponential
curve t that denes the recovery time, τ
. All error bars are SEM calculated over n=4 instantiations.
Homann et al.
Novel stimuli evoke excess activity in the mouse primary visual cortex
PNAS j9of12
of this accumulated weak adaptation, subunits had a smaller gain
when the novel image is presented.
Finally, the transient response for a given image set, T, recovers
with a time constant of τ
=30.2 s, consistent with our measure-
ments (Fig. 5B). Taken together, the adaptive subunit model pro-
vides a qualitative match for all of our experimental observations
with 6 free parameters (SI Appendix,Supplementary Methods).
Only three of these parameters were fully adjusted to match
experimental data (p
); the others were constrained
to reasonable values.
In this study we found that the excitatory neurons in layer 2/3
of the mouse primary visual cortex exhibited a distinctive pat-
tern of excess activity when a novel image was presented among
a set of familiar images. This excess activity consisted of moder-
ate spiking, estimated to be 0.5 spike per neuron, which was
present in a large fraction of all neurons (Fig. 2B). Because of
this widely distributed activity, the population spiking rate
increased by a factor of approximately fourfold. This excess
activity did not result from changes in running speed, pupil
diameter, or eye movements (Fig. 3 and SI Appendix, Fig. S3).
When we switched from one familiar set of images to a new set,
there was a similar distributed pattern of transient excess neu-
ral activity.
The process that determined the novelty of an image was
rapid, as the differential response between novel and familiar
images could be observed after only one repetition of an image
set (Fig. 5A). The characteristic time scale over which the nov-
elty response emerged was approximately three repeats (Fig. 5 A,
Inset). Similarly, the excess activity evoked by the switching to a
new image set decayed to steady state after approximately two
repeats of that image set. The familiarity of images must be
maintained by some form of memory. By presenting novel
images in the context of image sets of different durations, S,we
found that the novelty response decayed for larger S(Fig. 6).
This defined a capacity for the system to encode 15 familiar
images, under our experimental conditions. This representation
of familiarity decayed relatively slowly, with a recovery from
adaptation on a time scale of 30 s (Fig. 5B).
Mechanisms of the Novelty Response. In order to gain insight into
the circuit mechanisms responsible for novelty responses, we
formulated an adaptive subunit model. In this model, each
observed neuron received rectified input from many subunits
that each had different tuning for stimuli—a form similar to
Hubel and Wiesel’s model of the complex cell (55) as well as
models of cascaded adaptation (56–58). In addition, each subu-
nit had its own gain control. This model could account for all of
the main qualitative features of our data, including: 1) excess
activity evoked by all novel images, 2) rapid emergence of the
novelty response among a new set of familiar images, 3) rapid
decay of excess activity as novel images were repeated, 4) decay
of the novelty response among larger sets of familiar images,
and 5) recovery of full activity with a longer time scale.
While mechanisms of short-term synaptic plasticity, like
depression, appear to be nearly ubiquitous in early sensory
pathways (59), we did not explicitly include such terms. This
was for two reasons: 1) The dominant time scale of synaptic
depression in layer 2/3 of V1 is 400 ms (60), which is too fast
to account for our observations, and 2) the gain control in each
subunit adjusts the strength of presynaptic inputs in a manner
similar to depression, obviating the need for another term. In
our model, the gain control variable in subunits accounted for
all adaptation in subcortical pathways as well as adaptation in
layer 4 of V1. As such, we used a stronger gain control parame-
ter (p
). Breaking up all of these processes into multiple
biophysical mechanisms would presumably have allowed for a
closer quantitative match to our observations but would have
been achieved at the cost of a more complicated model. One
strength of our resulting model is that it captures such a wide
range of phenomena with a relatively simple structure.
Contrast adaptation also leads to excess activity when the
stimulus changes to an image that strongly activates a given
neuron (61). However, in classical contrast adaptation, the tran-
sition back to an image that more weakly activates that neuron
evokes reduced activity that rises to its steady state (50, 51).
Our data instead showed excess activity for all transitions
between image sets, including those that led to lower steady-
state activity (Fig. 6). Contrast adaptation in V1 is pattern-
specific (62–64). This property could lead to excess activity for
all stimulus transitions, as for each image there could be unad-
apted receptive field subunits that had high sensitivity (65). In
fact, this structure is captured by our adaptive subunit model.
However, some properties of the novelty response differ from
pattern-specific contrast adaptation: 1) Compared to its steady
state, the excess activity had not just a multiplicative component
characteristic of contrast adaptation, but also an additive compo-
nent (Fig. 7D) and 2) the dynamics of excess activity exhibited
two principle components, with a slow component matching con-
trast adaptation and an additional fast component (Fig. 8). Fur-
thermore, most studies of cortical adaptation have focused on
steady-state activity and not on transient dynamics.
Another closely related phenomenon is SSA, first reported
in the primary auditory cortex (19). Construing this phenome-
non broadly, our findings are certainly an example of adapta-
tion that is stimulus-specific. While our specific experimental
design differs from previous experiments, our results share
many features of SSA reported in A1 (66–68). Construed more
narrowly, SSA, defined as a reduced response to familiar stim-
uli, has been distinguished from deviance detection per se, a
phenomenon linked to the MMN (69). One paradigm to define
deviance detection is to compare the response of a neuron to
an equally rare stimulus when the background is a single com-
mon stimulus versus many different stimuli (the many stand-
ards control) (70).
Neural deviance responses have been observed in V1 (21) in
awake mice and A1 first in membrane potentials of pyramidal
cells and interneurons as a delayed signal (71) and also as
spikes in awake mice (72). Deviance detection in the auditory
system builds along the hierarchy from inferior colliculus to the
auditory belt (72) and is found prominently in the prefrontal
cortex (73). It is also noteworthy that auditory deviance detec-
tion was statistically significant in awake mice but not anesthe-
tized rats. Deviance detection in the visual system was
decreased when prefrontal cortex input to V1 was inhibited
(74). This suggests an involvement of the prefrontal cortex spe-
cifically in the generation of a deviance response. This could
potentially explain the absence of a statistically significant devi-
ance response in A1 of anesthetized rats.
In the context of these previous results, we believe that the nov-
elty response most likely has components due both to stimulus-
specific adaptation and deviance detection. However, we cannot
make exactly the same decomposition, because we used different
experimental design. It is also worth noting that the distinction
between deviance detection and stimulus-specific adaptation does
not cleanly map onto different circuit mechanisms. In particular,
the adaptive subunit model, which might appear to be a model of
stimulus specific adaptation alone, actually can exhibit deviance
detection. This occurs when the subunits in the model respond to
more than one stimulus, resulting in a greater level of “cross-
adaptation” in the many standards control than in the oddball
experiment. Given the diversity of neural tuning in the cortex—for
even as simple a property as orientation tuning (75, 76)—the
assumption of cross-talk in the adaptive subunit model seems
10 of 12 jPNAS Homann et al. Novel stimuli evoke excess activity in the mouse primary visual cortex
reasonable. Because inhibition of the prefrontal cortex reduces
the amplitude of deviance detection in V1 (74), deviance detec-
tion may have properties that are not captured by the adaptive
subunit model. Further investigation will be needed to resolve the
full set of mechanisms that give rise to the novelty response.
Interpretation of the Novelty Response. The existence of a novelty
response implies that the system has some representation or
“memory” of familiar images. We want to be clear, however, that
this form of memory may not have any relationship to the memo-
ries encoded by the hippocampus. In fact, the structure of the
adaptive subunit model suggests that this memory could be pri-
marily encoded by synaptic and cellular variables of neurons within
V1. At the same time, recent results have shown that responses in
V1 can be influenced by activity in the hippocampus (77, 78), so it
would be premature to rule out any such involvement.
This form of memory for familiar images might not allow
explicit retrieval of the remembered information, but it can
help serve a “passive” role in determining whether an image
has been seen before or not—a phenomenon known as recogni-
tion memory (79). While the perirhinal cortex is thought to
play a central role in recognition memory (17), the novelty
response in V1 may contribute to this ability as well.
The novelty response can also be thought of as a form of pre-
dictive coding, which emphasizes the representation of novel or
surprising information. While many theories of predictive coding
in the neocortex focus on the role of descending feedback (80,
81), mechanisms of adaptation can also contribute (82).
The fact that a simple and biophysically common mechanism
can give rise to the novelty response suggests that similar proc-
essing may be present at many stages of the sensory processing
hierarchy. If this operation were repeated at multiple stages,
then adaptation would cascade and potentially get stronger,
faster, or longer-lasting. In fact, more than half of the neurons
in the human hippocampus had responses to a novel image
that disappeared after a single presentation (13). In perirhinal
and inferotemporal cortex, more than half of recorded
neurons show an image-specific response reduction after a
single presentation (14, 16, 83, 84). Furthermore, these novelty
responses lasted for 24 h, as compared with the 60-s dura-
tion we measured in V1 (Fig. 4B). If a mechanism like the
adaptive subunit model existed at multiple cortical stages, then
novelty responses could be present for increasingly complex
stimulus features at higher stages of the hierarchy, without
needing to invoke further changes, like adaptive synapses (56,
85). Thus, this simple mechanism could lead to powerful adap-
tive processing across the cortical hierarchy.
Materials and Methods
A summary of our methods is provided here, with additional details contained
in SI Appendix,Supplementary Methods. All experiments were performed
according to the Guide for the Care and Use of Laboratory Animals (86), and
procedures were approved by Princeton Universitys Animal Care and Use
Committee. In short, a cranial window was implanted over V1 in ve trans-
genic mice expressing GCaMP6f under the thy1 promoter (36, 87). After a
recovery period, activity was recorded from layer 2/3 neurons, while mice
passively viewed repeated images containing random Gabor functions. For
surgical details, details about the imaging procedure and our analysis pipeline
SI Appendix,Supplementary Methods.Repeated and unique images were pre-
sented in various sequences, either for 250 ms or 300 ms per image. Experi-
ments lasted between 10 min and 1 h. See SI Appendix,Supplementary
Methods for a detailed description of the stimuli. We built a computer model
that mimics our experimental results. This model contained two layers of
units, representing layer 2/3 neurons and their upstream inputs. Those input
neurons were differentially driven by the stimulus. Both sets of neurons
dynamically adapted to their input. The model contained eight parameters
that were either picked based on plausible assumptions or estimated form a
t to the data. Parameters and equations are described in SI Appendix,
Supplementary Methods.
Data Availability. Raw data used for analysis are available upon request. All
other data are included in the main text and/or SI Appendix.
ACKNOWLEDGMENTS. M.J.B. acknowledges support from the National Eye
Institute (EY014196), the NSF (PHY 1504977 and PHY 1806932), and the Bezos
Center (Princeton University). J.H. acknowledges support from the Princeton
Innovation Fund.
1. C. Escera, K. Alho, I. Winkler, R. N
anen, Neural mechanisms of involuntary atten-
tion to acoustic novelty and change. J. Cogn. Neurosci. 10, 590604 (1998).
2. K. A. Diliberto, J. Altarriba, W. T. Neill, Novel popout and familiar popout in a bright-
ness discrimination task. Percept. Psychophys. 62, 14941500 (2000).
3. J. Schomaker, M. Meeter, Novelty enhances visual perception. PLoS One 7, e50599
4. Q. Wang, P. Cavanagh, M. Green, Familiarity and pop-out in visual search. Percept.
Psychophys. 56, 495500 (1994).
5. J. W. Kalat, Taste salience depends on novelty, not concentration, in taste-aversion
learning in the rat. J. Comp. Physiol. Psychol. 86,4750 (1974).
6. S. Fenwick, P. J. Mikulka, S. B. Klein, The effect of different levels of pre-exposure to
sucrose on the acquisition and extinction of a conditioned aversion. Behav. Biol. 14,
231235 (1975).
7. E. Quintero et al., Effects of context novelty vs. familiarity on latent inhibition
with a conditioned taste aversion procedure. Behav. Processes 86, 242249
8. J. Radulovic, J. Kammermeier, J. Spiess, Relationship between FOS production and
classical fear conditioning: Effects of novelty, latent inhibition, and unconditioned
stimulus preexposure. J. Neurosci. 18, 74527461 (1998).
9. S. O. King II, C. L. Williams, Novelty-induced arousal enhances memory for cued classi-
cal fear conditioning: Interactions between peripheral adrenergic and brainstem
glutamatergic systems. Learn. Mem. 16, 625634 (2009).
10. R. N
anen, P. Paavilainen, T. Rinne, K. Alho, The mismatch negativity (MMN) in
basic research of central auditory processing: A review. Clin. Neurophysiol. 118,
25442590 (2007).
11. R. N
anen, M. Simpson, N. E. Loveless, Stimulus deviance and evoked potentials.
Biol. Psychol. 14,5398 (1982).
12. K. Grill-Spector, R. Henson, A. Martin, Repetition and the brain: Neural models of
stimulus-specic effects. Trends Cogn. Sci. 10,1423 (2006).
13. U. Rutishauser, A. N. Mamelak, E. M. Schuman, Single-trial learning of novel stimuli
by individual neurons of the human hippocampus-amygdala complex. Neuron 49,
805813 (2006).
14. L. Li, E. K. Miller, R. Desimone, The representation of stimulus familiarity in anterior
inferior temporal cortex. J. Neurophysiol. 69, 19181929 (1993).
15. L. Woloszyn, D. L. Sheinberg, Effects of long-term visual experience on responses of
distinct classes of single units in inferior temporal cortex. Neuron 74, 193205 (2012).
16. T. Meyer, N. C. Rust, Single-exposure visual memory judgments are reected in infer-
otemporal cortex. eLife 7, e32259 (2018).
17. M. W. Brown, J. P. Aggleton, Recognition memory: What are the roles of the perirhi-
nal cortex and hippocampus? Nat. Rev. Neurosci. 2,5161 (2001).
18. T. Hosoya, S. A. Baccus, M. Meister, Dynamic predictive coding by the retina. Nature
19. N. Ulanovsky, L. Las, I. Nelken, Processing of low-probability sounds by cortical neu-
rons. Nat. Neurosci. 6, 391398 (2003).
20. A. Yaron, I. Hershenhoren, I. Nelken, Sensitivity to complex statistical regularities in
rat auditory cortex. Neuron 76, 603615 (2012).
21. J. P. Hamm, R. Yuste, Somatostatin interneurons control a key component of mis-
match negativity in mouse visual cortex. Cell Rep. 16, 597604 (2016).
22. E. Y. Chen, J. Chou, J. Park, G. Schwartz, M. J. Berry II, The neural circuit mechanisms
underlying the retinal response to moti onre versal. J.Neurosci. 34, 1555715575 (2014).
23. E. Y. Chen et al., Alert response to motion onset in the retina. J. Neurosci. 33, 120132
(2013). Correction in: J. Neurosci. 33, 2728 (2013).
24. W. Schultz, P. Dayan, P. R. Montague, A neural substrate of prediction and reward.
Science 275, 15931599 (1997).
25. F. Attneave, Some informational aspects of visual perception. Psychol. Rev. 61,
183193 (1954).
26. M. V. Srinivasan, S. B. Laughlin, A. Dubs, Predictive coding: A fresh view of inhibition
in the retina. Proc. R. Soc. Lond. B Biol. Sci. 216, 427459 (1982).
27. J. J. Atick, A. N. Redlich, Towards a theory of early visual processing. Neural Comput.
2, 308320 (1990).
28. J. J. Atick, A. N. Redlich, What does the retina know about natural scenes? Neural
Comput. 4, 196210 (1992).
29. D. Mumford, On the computational architecture of the neocortex. II. The role of
cortico-cortical loops. Biol. Cybern. 66, 241251 (1992).
30. R. P. N. Rao, D. H. Ballard, Predictive coding in the visual cortex: A functional interpre-
tation of some extra-classical receptive-eld effects. Nat. Neurosci. 2,7987 (1999).
31. A. M. Bastos et al., Canonical microcircuits for predictive coding. Neuron 76, 695711
Homann et al.
Novel stimuli evoke excess activity in the mouse primary visual cortex
PNAS j11 of 12
32. S. S. Solomon, H. Tang, E. Sussman, A. Kohn, Limited evidence for sensory prediction
error responses in visual cortex of macaques and humans. Cereb. Cortex 31,
31363152 (2021).
33. D. A. Dombeck, C. D. Harvey, L. Tian, L. L. Looger, D. W. Tank, Functional imaging of
hippocampal place cells at cellular resolution during virtual navigation. Nat. Neuro-
sci. 13, 14331440 (2010).
34. D.A.Dombeck,A.N.Khabbaz,F.Collman,T.L.Adelman,D.W.Tank,Imaginglarge-scale
neural activity with cellular resolution in awake, mobi le mice. Neuron 56,4357 (2007).
35. C. D. Harvey, P. Coen, D. W. Tank, Choice-specic sequences in parietal cortex during
a virtual-navigation decision task. Nature 484,6268 (2012).
36. T. W. Chen et al., Ultrasensitive uorescent proteins for imaging neuronal activity.
Nature 499, 295300 (2013).
37. C. M. Niell, M. P. Stryker, Highly selective receptive elds in mouse visual cortex.
J. Neurosci. 28, 75207536 (2008).
38. G. Van den Bergh, B. Zhang, L. Arckens, Y. M. Chino, Receptive-eldpropertiesofV1
and V2 neuronsin mice and macaquemonkeys. J.Comp. Neurol.518,20512070 (2010).
39. M. Yilmaz, M. Meister, Rapid innate defensive responses of mice to looming visual
stimuli. Curr. Biol. 23, 20112015 (2013).
40. B. P. Godsil, M. S. Fanselow, Light stimulus change evokes an activity response in the
rat. Learn. Behav. 32, 299310 (2004).
41. M. Vinck, R. Batista-Brito, U. Knoblich, J. A. Cardin, Arousal and locomotion make dis-
tinct contributions to cortical activity patterns and visual encoding. Neuron 86,
740754 (2015).
42. P. O. Polack, J. Friedman, P. Golshani, Cellular mechanisms of brain state-dependent
gain modulation in visual cortex. Nat. Neurosci. 16, 13311339 (2013).
43. H. Sato, Y. Hata, K. Hagihara, T. Tsumoto, Effects of cholinergic depletion on neuron
activities in the cat visual cortex. J. Neurophysiol. 58, 781794 (1987).
44. R. Rodriguez, U. Kallenbach, W. Singer, M. H. Munk, Stabilization of visual responses
through cholinergic activation. Neuroscience 165, 944954 (2010).
45. J. Reimer et al., Pupil uctuations track rapid changes in adrenergic and cholinergic
activity in cortex. Nat. Commun. 7, 13289 (2016).
46. G. B. Keller, T. Bonhoeffer, M. H
ubener, Sensorimotor mismatch signals in primary
visual cortex of the behaving mouse. Neuron 74, 809815 (2012).
47. C. M. Niell, M. P. Stryker, Modulation of visual responses by behavioral state in mouse
visual cortex. Neuron 65, 472479 (2010).
48. A. B. Saleem, A. Ayaz, K. J. Jeffery, K. D. Harris, M. Carandini, Integration of visual
motion and locomotion in mouse visual cortex. Nat. Neurosci. 16, 18641869 (2013).
49. A. Benucci, A. B. Saleem, M. Carandini, Adaptation maintains population homeosta-
sis in primary visual cortex. Nat. Neurosci. 16, 724729 (2013).
50. I. Ohzawa, G. Sclar, R. D. Freeman, Contrast gain control in the cats visual system.
J. Neurophysiol. 54, 651667 (1985).
51. M. V. Sanchez-Vives, L. G. Nowak, D. A. McCormick, Cellular mechanisms of long-
lastingadaptation in visualcortical neuronsin vitro. J. Neurosci.20,42864299 (2000).
52. M. Carandini, D. Ferster, A tonic hyperpolarization underlying contrast adaptation in
cat visual cortex. Science 276, 949952 (1997).
53. M. V. Sanchez-Vives, L. G. Nowak, D. A. McCormick, Membrane mechanisms underly-
ing contrast adaptation in cat area 17 in vivo. J. Neurosci. 20, 42674285 (2000).
54. C. J. Whitmire, G. B. Stanley, Rapid sensory adaptation redux: A circuit perspective.
Neuron 92, 298315 (2016).
55. D. H. Hubel, T. N. Wiesel, Receptive elds, binocular interaction and functional archi-
tecture in the cats visual cortex. J. Physiol. 160, 106154 (1962).
56. N. T. Dhruv, M. Carandini, Cascaded effects of spatial adaptation in the early visual
system. Neuron 81, 529535 (2014).
57. L. Khouri, I. Nelken, Detecting the unexpected. Curr. Opin. Neurobiol. 35, 142147
58. R. Mill, M. Coath, T. Wennekers, S. L. Denham, A neurocomputational model of
stimulus-specic adaptation to oddball and Markov sequences. PLOS Comput. Biol.
7, e1002117 (2011).
59. M. A. Castro-Alamancos, B. W. Connors, Short-term synaptic enhancement and
long-term potentiation in neocortex. Proc. Natl. Acad. Sci. U.S.A. 93, 13351339
60. J. A. Varela et al., A quantitative description of short-term plasticity at excitatory syn-
apses in layer 2/3 of rat primary visual cortex. J. Neurosci. 17, 79267940 (1997).
61. A. Kohn, Visual adaptation: Physiology, mechanisms, and functional benets. J. Neu-
rophysiol. 97, 31553164 (2007).
62. J. A. Movshon, P. Lennie, Pattern-selective adaptation in visual cortical neurones.
Nature 278, 850852 (1979).
63. J. R. M
uller, A. B. Metha, J. Krauskopf, P. Lennie, Rapid adaptation in visual cortex to
the structure of images. Science 285, 14051408 (1999).
64. M. Carandini, H. B. Barlow, L. P. OKeefe, A. B. Poirson, J. A. Movshon, Adaptation to
contingencies in macaque primary visual cortex. Philos. Trans. R. Soc. Lond. B Biol. Sci.
352, 11491154 (1997).
65. S. M. Smirnakis, M. J. Berry, D. K. Warland, W. Bialek, M. Meister, Adaptation of reti-
nal processing to image contrast and spatial scale. Nature 386,6973 (1997).
66. N. Ulanovsky, L. Las, D. Farkas, I. Nelken, Multiple time scales of adaptation in audi-
tory cortex neurons. J. Neurosci. 24, 1044010453 (2004).
67. N. Taaseh, A. Yaron, I. Nelken, Stimulus-specic adaptation and deviance detection
in the rat auditory cortex. PLoS One 6, e23369 (2011).
68. R. G. Natan et al., Complementary control of sensory adaptation by two types of cor-
tical interneurons. eLife 4, e09868 (2015).
69. M. I. Garrido, J. M. Kilner, K. E. Stephan, K. J. Friston, The mismatch negativity: A
review of underlying mechanisms. Clin. Neurophysiol. 120, 453463 (2009).
70. L. Harms, P. T. Michie, R. N
anen, Criteria for determining whether mismatch
responses exist in animal models: Focus on rodents. Biol. Psychol. 116,2835 (2016).
71. I. W. Chen, F. Helmchen, H. L
utcke, Specic early and late oddball-evoked responses
in excitatory and inhibitory neurons of mouse auditory cortex. J. Neurosci. 35,
1256012573 (2015).
72. G. G. Parras et al., Neurons along the auditory pathway exhibit a hierarchical organi-
zation of prediction error. Nat. Commun. 8, 2148 (2017).
73. L. Casado-Rom
an, G. V. Carbajal, D. P
alez, M. S. Malmierca, Prediction
error signaling explains neuronal mismatch responses in the medial prefrontal cor-
tex. PLoS Biol. 18, e3001019 (2020).
74. J. P. Hamm, Y. Shymkiv, S. Han, W. Yang, R. Yuste, Cortical ensembles selective for
context. Proc. Natl. Acad. Sci. U.S.A. 118, e2026179118 (2021).
75. H. Jia, N. L. Rochefort, X. Chen, A. Konnerth, Dendritic organization of sensory input
to cortical neurons in vivo. Nature 464, 13071312 (2010).
76. M. F. Iacaruso, I. T. Gasler, S. B. Hofer, Synaptic organization of visual space in primary
visual cortex. Nature 547, 449452 (2017).
77. A. Fiser et al., Experience-dependent spatial expectations in mouse visual cortex. Nat.
Neurosci. 19, 16581664 (2016).
78. P. S. B. Finnie, R. W. Komorowski, M. F. Bear, The spatiotemporal organization of
experience dictates hippocampal involvement in primary visual cortical plasticity.
Curr. Biol. 31, 39964008.e6 (2021).
79. G. Mandler, Recognizing: The judgment of previous occurrence. Psychol. Rev. 87,
252271 (1980).
80. M. W. Spratling, A review of predictive coding algorithms. Brain Cogn. 112,9297
81. D. J. Heeger, Theory of cortical function. Proc. Natl. Acad. Sci. U.S.A. 114, 17731782
82. A. I. Weber, K. Krishnamurthy, A. L. Fairhall, Coding principles in adaptation. Annu.
Rev. Vis. Sci. 5, 427449 (2019).
83. J. L. Ringo, Stimulus specic adaptation in inferior temporal and medial temporal
cortex of the monkey. Behav. Brain Res. 76, 191197 (1996).
84. J. Z. Xiang, M. W. Brown, Differential neuronal encoding of novelty, familiarity and
recency in regions of the anterior temporal lobe. Neuropharmacology 37, 657676
85. S. G. Solomon, A. Kohn, Moving sensory adaptation beyond suppressive effects in sin-
gle neurons. Curr. Biol. 24, R1012R1022 (2014).
86. National Research Council, Guide for the Care and Use of Laboratory Animals
(National Academies Press, Washington, DC, ed. 8, 2011).
87. H. Dana et al., Thy1-GCaMP6 transgenic mice for neuronal population imaging
in vivo. PLoS One 9, e108697 (2014).
12 of 12 jPNAS Homann et al. Novel stimuli evoke excess activity in the mouse primary visual cortex
... In the context of PP, neurons in L2/3 are often thought to signal deviations between predicted and actual inputs (cf. [113]), while activity in L5/6 corresponds to internal representations [14,15]. The laminar distribution of non-visual inputs is therefore highly informative of the nature of their interaction with processing information of retinal origin. ...
Full-text available
The definition of the visual cortex is primarily based on the evidence that lesions of this area impair visual perception. However, this does not exclude that the visual cortex may process more information than of retinal origin alone, or that other brain structures contribute to vision. Indeed, research across the past decades has shown that non-visual information, such as neural activity related to reward expectation and value, locomotion, working memory and other sensory modalities, can modulate primary visual cortical responses to retinal inputs. Nevertheless, the function of this non-visual information is poorly understood. Here we review recent evidence, coming primarily from studies in rodents, arguing that non-visual and motor effects in visual cortex play a role in visual processing itself, for instance disentangling direct auditory effects on visual cortex from effects of sound-evoked orofacial movement. These findings are placed in a broader framework casting vision in terms of predictive processing under control of frontal, reward- and motor-related systems. In contrast to the prevalent notion that vision is exclusively constructed by the visual cortical system, we propose that visual percepts are generated by a larger network—the extended visual system—spanning other sensory cortices, supramodal areas and frontal systems. This article is part of the theme issue ‘Decision and control processes in multisensory perception’.
... On the whole-brain level, electroencephalogram measurements reveal that presenting the deviant stimulus leads to a strong negative deflection in the EEG signal compared to the signal following from a standard stimulus presentation, termed "mismatch negativity." 167,168 Similarly, measurements of either single neurons or neuronal populations in sensory cortices reveal elevated neuronal responses for deviant compared to the standard stimuli [169][170][171] [ Fig. 4(a)]. Computational models have proven to be useful for understanding the mechanisms underlying short-term adaptation in the brain (see also Secs. ...
Adaptivity is a dynamical feature that is omnipresent in nature, socio-economics, and technology. For example, adaptive couplings appear in various real-world systems, such as the power grid, social, and neural networks, and they form the backbone of closed-loop control strategies and machine learning algorithms. In this article, we provide an interdisciplinary perspective on adaptive systems. We reflect on the notion and terminology of adaptivity in different disciplines and discuss which role adaptivity plays for various fields. We highlight common open challenges and give perspectives on future research directions, looking to inspire interdisciplinary approaches.
... The visual system, including primary cortical areas, has emerged over the last decade as an unexpectedly rich system to study time, sequence learning, and temporal predictions. Starting with the demonstration that cells in rat V1 can learn evoked dynamics that predict when a reward will be delivered following visual stimulation (Shuler and Bear, 2006), visual cortex has also been shown capable of learning to produce trajectories (Xu et al., 2012;Ekman et al., 2017), recognizing sequences (Gavornik and Bear, 2014;Sidorov et al., 2020;Ekman et al., 2023) and making predictions about expected visual inputs (Fiser et al., 2016;Leinweber et al., 2017;Pakan et al., 2018;Homann et al., 2022) in various animals including humans. The Cone and Shouval model was developed to explain how a recurrent timing mechanism developed to explain interval timing in V1 (Gavornik et al., 2009;Gavornik and Shouval, 2010) could be extended to explain spatiotemporal sequence learning. ...
Full-text available
The brain uses temporal information to link discrete events into memory structures supporting recognition, prediction, and a wide variety of complex behaviors. It is still an open question how experience-dependent synaptic plasticity creates memories including temporal and ordinal information. Various models have been proposed to explain how this could work, but these are often difficult to validate in a living brain. A recent model developed to explain sequence learning in the visual cortex encodes intervals in recurrent excitatory synapses and uses a learned offset between excitation and inhibition to generate precisely timed "messenger" cells that signal the end of an instance of time. This mechanism suggests that the recall of stored temporal intervals should be particularly sensitive to the activity of inhibitory interneurons that can be easily targeted in vivo with standard optogenetic tools. In this work we examined how simulated optogenetic manipulations of inhibitory cells modifies temporal learning and recall based on these mechanisms. We show that disinhibition and excess inhibition during learning or testing cause characteristic errors in recalled timing that could be used to validate the model in vivo using either physiological or behavioral measurements.
... Trial and reward history have also been shown to influence visual responses [McMahon and Olson, 2007, Meyer et al., 2014, Nikolić et al., 2009, Shuler and Bear, 2006, Ramadan et al., 2022, Gillon et al., 2021, indicating that sensory coding is influenced not only by current state but also prior experience and expectations. Neurons in primary visual cortex (V1) learn short spatiotemporal sequences of stimuli upon repeated presentation [Gavornik and Bear, 2014], and enhance their activity for unexpected oddball images, as well as at the start of a novel sequence [Homann et al., 2022, Kim et al., 2019b. Predictive coding has been proposed as a theory that can account for contextual modulation of sensory responses [Khan et al., 2018, Keller and, framing sensory perception as a process of active inference. ...
Full-text available
The classic view that neural populations in sensory cortices preferentially encode responses to incoming stimuli has been strongly challenged by recent experimental studies. Despite the fact that a large fraction of variance of visual responses in rodents can be attributed to behavioral state and movements, trial-history, and salience, the effects of contextual modulations and expectations on sensory-evoked responses in visual and association areas remain elusive. Here, we present a comprehensive experimental and theoretical study showing that hierarchically connected visual and association areas differentially encode the temporal context and expectation of naturalistic visual stimuli, consistent with the theory of hierarchical predictive coding. We measured neural responses to expected and unexpected sequences of natural scenes in the primary visual cortex (V1), the posterior medial higher order visual area (PM), and retrosplenial cortex (RSP) using 2-photon imaging in behaving mice collected through the Allen Institute Mindscope's OpenScope program. We found that information about image identity in neural population activity depended on the temporal context of transitions preceding each scene, and decreased along the hierarchy. Furthermore, our analyses revealed that the conjunctive encoding of temporal context and image identity was modulated by expectations of sequential events. In V1 and PM, we found enhanced and specific responses to unexpected oddball images, signaling stimulus-specific expectation violation. In contrast, in RSP the population response to oddball presentation recapitulated the missing expected image rather than the oddball image. These differential responses along the hierarchy are consistent with classic theories of hierarchical predictive coding whereby higher areas encode predictions and lower areas encode deviations from expectation. We further found evidence for drift in visual responses on the timescale of minutes. Although activity drift was present in all areas, population responses in V1 and PM, but not in RSP, maintained stable encoding of visual information and representational geometry. Instead we found that RSP drift was independent of stimulus information, suggesting a role in generating an internal model of the environment in the temporal domain. Overall, our results establish temporal context and expectation as substantial encoding dimensions in the visual cortex subject to fast representational drift and suggest that hierarchically connected areas instantiate a predictive coding mechanism.
... The Gabor sequence stimulus was adapted from a previously published study 20 . Specifically, it consisted of repeating 1.5-second sequences, each comprising five consecutive images (A-B-C-D-G) presented for 300 ms each. ...
Full-text available
The apical dendrites of pyramidal neurons in sensory cortex receive primarily top-down signals from associative and motor regions, while cell bodies and nearby dendrites are heavily targeted by locally recurrent or bottom-up inputs from the sensory periphery. Based on these differences, a number of theories in computational neuroscience postulate a unique role for apical dendrites in learning. However, due to technical challenges in data collection, little data is available for comparing the responses of apical dendrites to cell bodies over multiple days. Here we present a dataset collected through the Allen Institute Mindscope’s OpenScope program that addresses this need. This dataset comprises high-quality two-photon calcium imaging from the apical dendrites and the cell bodies of visual cortical pyramidal neurons, acquired over multiple days in awake, behaving mice that were presented with visual stimuli. Many of the cell bodies and dendrite segments were tracked over days, enabling analyses of how their responses change over time. This dataset allows neuroscientists to explore the differences between apical and somatic processing and plasticity.
... Regions as widely separated as the cerebellum (Wagner & Luo, 2020;De Zeeuw, Lisberger, & Raymond, 2021), striatum (e.g., van der Meer & Redish, 2011), PFC (e.g., Rainer, Rao, & Miller, 1999;Ning, Bladon, & Hasselmo, 2022), OFC (e.g., Namboodiri et al., 2019;Schoenbaum, Chiba, & Gallagher, 1998;Young & Shapiro, 2011), hippocampus (Ferbinteanu & Shapiro, 2003;Duvelle, Grieves, & van der Meer, 2022) and thalamus (Komura et al., 2001) contain active representations that code for the future. One can find evidence of predictive signals extending over long periods of time that modulate firing in primary visual cortex (Gavornik & Bear, 2014;Kim, Homann, Tank, & Berry, 2019;Homann, Koay, Chen, Tank, & Berry, 2022;Yu et al., 2022). Prediction apparently involves a substantial proportion of the brain. ...
Full-text available
Recent advances in neuroscience and psychology show that the brain has access to timelines of both the past and the future. Spiking across populations of neurons in many regions of the mammalian brain maintains a robust temporal memory, a neural timeline of the recent past. Behavioral results demonstrate that people can estimate an extended temporal model of the future, suggesting that the neural timeline of the past could extend through the present into the future. This paper presents a mathematical framework for learning and expressing relationships between events in continuous time. We assume that the brain has access to a temporal memory in the form of the real Laplace transform of the recent past. Hebbian associations with a diversity of synaptic time scales are formed between the past and the present that record the temporal relationships between events. Knowing the temporal relationships between the past and the present allows one to predict relationships between the present and the future, thus constructing an extended temporal prediction for the future. Both memory for the past and the predicted future are represented as the real Laplace transform, expressed as the firing rate over populations of neurons indexed by different rate constants $s$. The diversity of synaptic timescales allows for a temporal record over the much larger time scale of trial history. In this framework, temporal credit assignment can be assessed via a Laplace temporal difference. The Laplace temporal difference compares the future that actually follows a stimulus to the future predicted just before the stimulus was observed. This computational framework makes a number of specific neurophysiological predictions and, taken together, could provide the basis for a future iteration of RL that incorporates temporal memory as a fundamental building block.
... On the whole-brain level, electroencephalogram measurements reveal that presenting the deviant stimulus leads to a strong negative deflection in the EEG signal compared to the signal following from standard stimulus presentation, termed 'mismatch negativity' 149,150 . Similarly, measurements of either single neurons or neuronal populations in sensory cortices reveal elevated neuronal responses for deviant compared to the standard stimuli [151][152][153] (Fig. 3A). Computational models have proven to be useful for understanding the mechanisms underlying short-term adaptation in the brain (see also Secs. ...
Full-text available
Adaptivity is a dynamical feature that is omnipresent in nature, socio-economics, and technology. For example, adaptive couplings appear in various real-world systems like the power grid, social, and neural networks, and they form the backbone of closed-loop control strategies and machine learning algorithms. In this article, we provide an interdisciplinary perspective on adaptive systems. We reflect on the notion and terminology of adaptivity in different disciplines and discuss which role adaptivity plays for various fields. We highlight common open challenges, and give perspectives on future research directions, looking to inspire interdisciplinary approaches.
... Consistently, short-term visual adaptation to an oriented grating or Gabor stimulus shifts the tuning curve toward or away from the adapted orientation depending on various stimulus features and enhances novelty detection [20][21][22][23][24][25][26][27][28][29] . Similarly, the multi-day repeated passive experience of orientation grating stimulus has been shown to increase orientation selectivity 10 , whereas other . ...
Full-text available
Familiarity creates subjective memory of repeated passive innocuous experiences, reduces neural and behavioral responsiveness to those experiences, and enhances novelty detection. The neural correlates of the internal model of familiarity and the cellular mechanisms of enhanced novelty detection following multi-day repeated passive experience remain to be better understood. Using the mouse visual cortex as a model system, we test how the repeated passive experience of an orientation-grating stimulus for multiple days alters spontaneous, and non-familiar stimuli evoked neural activity in neurons tuned to familiar or non-familiar stimuli. We found that familiarity elicits stimulus competition such that stimulus selectivity reduces in neurons tuned to the familiar stimulus, whereas it increases in those tuned to non-familiar stimuli. Consistently, neurons tuned to non-familiar stimuli dominate local functional connectivity. Furthermore, responsiveness to natural images, which consists of familiar and non-familiar orientations, increases subtly in neurons that exhibit stimulus competition. We also show the similarity between familiar grating stimulus-evoked and spontaneous activity increases, indicative of an internal model of altered experience.
Full-text available
How do neural populations adapt to the time-varying statistics of sensory input? To investigate, we measured the activity of neurons in primary visual cortex adapted to different environments, each associated with a distinct probability distribution over a stimulus set. Within each environment, a stimulus sequence was generated by independently sampling from its distribution. We find that two properties of adaptation capture how the population responses to a given stimulus, viewed as vectors, are linked across environments. First, the ratio between the response magnitudes is a power law of the ratio between the stimulus probabilities. Second, the response directions are largely invariant. These rules can be used to predict how cortical populations adapt to novel, sensory environments. Finally, we show how the power law enables the cortex to preferentially signal unexpected stimuli and to adjust the metabolic cost of its sensory representation to the entropy of the environment.
Repeated exposure to visual sequences changes the form of evoked activity in the primary visual cortex (V1). Predictive coding theory provides a potential explanation for this, namely that plasticity shapes cortical circuits to encode spatiotemporal predictions and that subsequent responses are modulated by the degree to which actual inputs match these expectations. Here we use a recently developed statistical modeling technique called Model-Based Targeted Dimensionality Reduction (MbTDR) to study visually evoked dynamics in mouse V1 in the context of an experimental paradigm called “sequence learning.” We report that evoked spiking activity changed significantly with training, in a manner generally consistent with the predictive coding framework. Neural responses to expected stimuli were suppressed in a late window (100–150 ms) after stimulus onset following training, whereas responses to novel stimuli were not. Substituting a novel stimulus for a familiar one led to increases in firing that persisted for at least 300 ms. Omitting predictable stimuli in trained animals also led to increased firing at the expected time of stimulus onset. Finally, we show that spiking data can be used to accurately decode time within the sequence. Our findings are consistent with the idea that plasticity in early visual circuits is involved in coding spatiotemporal information.
Full-text available
The mismatch negativity (MMN) is a key biomarker of automatic deviance detection thought to emerge from 2 cortical sources. First, the auditory cortex (AC) encodes spectral regularities and reports frequency-specific deviances. Then, more abstract representations in the prefrontal cortex (PFC) allow to detect contextual changes of potential behavioral relevance. However, the precise location and time asynchronies between neuronal correlates underlying this frontotemporal network remain unclear and elusive. Our study presented auditory oddball paradigms along with “no-repetition” controls to record mismatch responses in neuronal spiking activity and local field potentials at the rat medial PFC. Whereas mismatch responses in the auditory system are mainly induced by stimulus-dependent effects, we found that auditory responsiveness in the PFC was driven by unpredictability, yielding context-dependent, comparatively delayed, more robust and longer-lasting mismatch responses mostly comprised of prediction error signaling activity. This characteristically different composition discarded that mismatch responses in the PFC could be simply inherited or amplified downstream from the auditory system. Conversely, it is more plausible for the PFC to exert top-down influences on the AC, since the PFC exhibited flexible and potent predictive processing, capable of suppressing redundant input more efficiently than the AC. Remarkably, the time course of the mismatch responses we observed in the spiking activity and local field potentials of the AC and the PFC combined coincided with the time course of the large-scale MMN-like signals reported in the rat brain, thereby linking the microscopic, mesoscopic, and macroscopic levels of automatic deviance detection.
Full-text available
Our visual memory percepts of whether we have encountered specific objects or scenes before are hypothesized to manifest as decrements in neural responses in inferotemporal cortex (IT) with stimulus repetition. To evaluate this proposal, we recorded IT neural responses as two monkeys performed a single-exposure visual memory task designed to measure the rates of forgetting with time. We found that a weighted linear read-out of IT was a better predictor of the monkeys’ forgetting rates and reaction time patterns than a strict instantiation of the repetition suppression hypothesis, expressed as a total spike count scheme. Behavioral predictions could be attributed to visual memory signals that were reflected as repetition suppression and were intermingled with visual selectivity, but only when combined across the most sensitive neurons.
Full-text available
Perception is characterized by a reciprocal exchange of predictions and prediction error signals between neural regions. However, the relationship between such sensory mismatch responses and hierarchical predictive processing has not yet been demonstrated at the neuronal level in the auditory pathway. We recorded single-neuron activity from different auditory centers in anaesthetized rats and awake mice while animals were played a sequence of sounds, designed to separate the responses due to prediction error from those due to adaptation effects. Here we report that prediction error is organized hierarchically along the central auditory pathway. These prediction error signals are detectable in subcortical regions and increase as the signals move towards auditory cortex, which in turn demonstrates a large-scale mismatch potential. Finally, the predictive activity of single auditory neurons underlies automatic deviance detection at subcortical levels of processing. These results demonstrate that prediction error is a fundamental component of singly auditory neuron responses.
The hippocampus and neocortex are theorized to be crucial partners in the formation of long-term memories. Here, we assess hippocampal involvement in two related forms of experience-dependent plasticity in the primary visual cortex (V1) of mice. Like control animals, those with hippocampal lesions exhibit potentiation of visually evoked potentials after passive daily exposure to a phase-reversing oriented grating stimulus, which is accompanied by long-term habituation of a reflexive behavioral response. Thus, low-level recognition memory is formed independently of the hippocampus. However, response potentiation resulting from daily exposure to a fixed sequence of four oriented gratings is severely impaired in mice with hippocampal damage. A feature of sequence plasticity in V1 of controls, which is absent in lesioned mice, is the generation of predictive responses to an anticipated stimulus element when it is withheld or delayed. Thus, the hippocampus is involved in encoding temporally structured experience, even within the primary sensory cortex.
Neural processing of sensory information is strongly influenced by context. For instance, cortical responses are reduced to predictable stimuli, while responses are increased to novel stimuli that deviate from contextual regularities. Such bidirectional modulation based on preceding sensory context is likely a critical component or manifestation of attention, learning, and behavior, yet how it arises in cortical circuits remains unclear. Using volumetric two-photon calcium imaging and local field potentials in primary visual cortex (V1) from awake mice presented with visual “oddball” paradigms, we identify both reductions and augmentations of stimulus-evoked responses depending, on whether the stimulus was redundant or deviant, respectively. Interestingly, deviance-augmented responses were limited to a specific subset of neurons mostly in supragranular layers. These deviance-detecting cells were spatially intermixed with other visually responsive neurons and were functionally correlated, forming a neuronal ensemble. Optogenetic suppression of prefrontal inputs to V1 reduced the contextual selectivity of deviance-detecting ensembles, demonstrating a causal role for top-down inputs. The presence of specialized context-selective ensembles in primary sensory cortex, modulated by higher cortical areas, provides a circuit substrate for the brain’s construction and selection of prediction errors, computations which are key for survival and deficient in many psychiatric disorders.
A recent formulation of predictive coding theory proposes that a subset of neurons in each cortical area encodes sensory prediction errors, the difference between predictions relayed from higher cortex and the sensory input. Here, we test for evidence of prediction error responses in spiking responses and local field potentials (LFP) recorded in primary visual cortex and area V4 of macaque monkeys, and in complementary electroencephalographic (EEG) scalp recordings in human participants. We presented a fixed sequence of visual stimuli on most trials, and violated the expected ordering on a small subset of trials. Under predictive coding theory, pattern-violating stimuli should trigger robust prediction errors, but we found that spiking, LFP and EEG responses to expected and pattern-violating stimuli were nearly identical. Our results challenge the assertion that a fundamental computational motif in sensory cortex is to signal prediction errors, at least those based on predictions derived from temporal patterns of visual stimulation.
Adaptation is a common principle that recurs throughout the nervous system at all stages of processing. This principle manifests in a variety of phenomena, from spike frequency adaptation, to apparent changes in receptive fields with changes in stimulus statistics, to enhanced responses to unexpected stimuli. The ubiquity of adaptation leads naturally to the question: What purpose do these different types of adaptation serve? A diverse set of theories, often highly overlapping, has been proposed to explain the functional role of adaptive phenomena. In this review, we discuss several of these theoretical frameworks, highlighting relationships among them and clarifying distinctions. We summarize observations of the varied manifestations of adaptation, particularly as they relate to these theoretical frameworks, focusing throughout on the visual system and making connections to other sensory systems. Expected final online publication date for the Annual Review of Vision Science Volume 5 is September 16, 2019. Please see for revised estimates.