Access to this full-text is provided by PLOS.
Content available from PLOS Computational Biology
This content is subject to copyright.
RESEARCH ARTICLE
Contrasting the effects of adaptation and
synaptic filtering on the timescales of
dynamics in recurrent networks
Manuel BeiranID*, Srdjan Ostojic ID*
Group for Neural Theory, Laboratoire de Neurosciences Cognitives Computationnelles, De
´partement
d’E
´tudes Cognitives, E
´cole Normale Supe
´rieure, INSERM U960, PSL University, Paris, France
*manuel.beiran@ens.fr (MB); srdjan.ostojic@ens.fr (SO)
Abstract
Neural activity in awake behaving animals exhibits a vast range of timescales that can be
several fold larger than the membrane time constant of individual neurons. Two types of
mechanisms have been proposed to explain this conundrum. One possibility is that large
timescales are generated by a network mechanism based on positive feedback, but this
hypothesis requires fine-tuning of the strength or structure of the synaptic connections. A
second possibility is that large timescales in the neural dynamics are inherited from large
timescales of underlying biophysical processes, two prominent candidates being intrinsic
adaptive ionic currents and synaptic transmission. How the timescales of adaptation or syn-
aptic transmission influence the timescale of the network dynamics has however not been
fully explored. To address this question, here we analyze large networks of randomly con-
nected excitatory and inhibitory units with additional degrees of freedom that correspond to
adaptation or synaptic filtering. We determine the fixed points of the systems, their stability
to perturbations and the corresponding dynamical timescales. Furthermore, we apply
dynamical mean field theory to study the temporal statistics of the activity in the fluctuating
regime, and examine how the adaptation and synaptic timescales transfer from individual
units to the whole population. Our overarching finding is that synaptic filtering and adaptation
in single neurons have very different effects at the network level. Unexpectedly, the macro-
scopic network dynamics do not inherit the large timescale present in adaptive currents. In
contrast, the timescales of network activity increase proportionally to the time constant of
the synaptic filter. Altogether, our study demonstrates that the timescales of different bio-
physical processes have different effects on the network level, so that the slow processes
within individual neurons do not necessarily induce slow activity in large recurrent neural
networks.
Author summary
Brain activity spans a wide range of timescales, as it is required to interact in complex
time-varying environments. However, individual neurons are primarily fast devices: their
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 1 / 33
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
OPEN ACCESS
Citation: Beiran M, Ostojic S (2019) Contrasting
the effects of adaptation and synaptic filtering on
the timescales of dynamics in recurrent networks.
PLoS Comput Biol 15(3): e1006893. https://doi.
org/10.1371/journal.pcbi.1006893
Editor: Peter E. Latham, UCL, UNITED KINGDOM
Received: December 19, 2018
Accepted: February 19, 2019
Published: March 21, 2019
Copyright: ©2019 Beiran, Ostojic. This is an open
access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
Data Availability Statement: All relevant data are
within the manuscript and its Supporting
Information files. The codes that generate the
figures are available at https://github.com/
emebeiran/adapt_synapt.
Funding: This work was funded by the Ecole des
Neurosciences de Paris Ile-de-France, the
Programme Emergences of City of Paris, Agence
Nationale de la Recherche grants ANR-16-CE37-
0016-01 and ANR-17-ERC2-0005-01, and the
program “Investissements d’Avenir” launched by
the French Government and implemented by the
ANR, with the references ANR-10-LABX-0087 IEC
membrane time constant is of the order of a few tens of milliseconds. Yet, neurons are
also subject to additional biophysical processes, such as adaptive currents or synaptic fil-
tering, that introduce slower dynamics in the activity of individual neurons. In this study,
we explore the possibility that slow network dynamics arise from such slow biophysical
processes. To do so, we determine the different dynamical properties of large networks of
randomly connected excitatory and inhibitory units which include an internal degree of
freedom that corresponds to either adaptation or synaptic filtering. We show that the net-
work dynamics do not inherit the slow timescale present in adaptive currents, while syn-
aptic filtering is an efficient mechanism to scale down the timescale of the network
activity.
Introduction
Adaptive behavior requires processing information over a vast span of timescales [1], ranging
from micro-seconds for acoustic localisation [2], milliseconds for detecting changes in the
visual field [3], seconds for evidence integration [4] and working memory [5], to hours, days
or years in the case of long-term memory. Neural activity in the brain is matched to the
computational requirements imposed by behavior, and consequently displays dynamics over a
similarly vast range of timescales [6–8]. Since the membrane time constant of an isolated neu-
ron is of the order of tens of milliseconds, the origin of the long timescales observed in the neu-
ral activity has been an outstanding puzzle.
Two broad classes of mechanisms have been proposed to account for the existence of long
timescales in the neural activity. The first class relies on non-linear collective dynamics that
emerge from synaptic interactions between neurons in the local network. Such mechanisms
have been proposed to model a variety of phenomena that include working memory [9], deci-
sion-making [10] and slow variability in the cortex [11]. In those models, long timescales
emerge close to bifurcations between different types of dynamical states, and therefore typi-
cally rely on the fine tuning of some parameter [12]. An alternative class of mechanisms posits
that long timescales are directly inherited from long time constants that exist within individual
neurons, at the level of hidden internal states [13]. Indeed biophysical processes at the cellular
and synaptic level display a rich repertoire of timescales. These include short-term plasticity
that functions at the range of hundreds of milliseconds [14,15], a variety of synaptic channels
with timescales from tens to hundreds of milliseconds [16–19], ion channel kinetics imple-
menting adaptive phenomena [20], calcium dynamics [21] or shifts in ionic reversal potentials
[22]. How the timescales of these internal processes affect the timescales of activity at the net-
work level has however not been fully explored.
In this study, we focus on adaptative ion-channel currents, which are known to exhibit
timescales over several orders of magnitude [23–25]. We contrast their effects on recurrent
network dynamics with the effect of the temporal filtering of inputs through synaptic cur-
rents, which also expands over a large range of timescales [26]. To this end, we extend classi-
cal rate models [27–30] of randomly connected recurrent networks by including for each
individual unit a hidden variable that corresponds to either the adapting of the synaptic cur-
rent. We systematically determine the types of collective activity that emerge in such net-
works. We then compare the timescales on the level of individual units with the activity
within the network.
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 2 / 33
and ANR11-IDEX-0001-02 PSLResearch
University. The funders had no role in study
design, data collection and analysis, decision to
publish, or preparation of the manuscript.
Competing interests: The authors have declared
that no competing interests exist.
Results
We consider Ncoupled inhibitory and excitatory units whose dynamics are given by two vari-
ables: the input current x
i
and a slow variable s
i
or w
i
that accounts for the synaptic filtering or
adaptation current respectively. The instantaneous firing rate of each neuron is obtained by
applying a static non-linearity ϕ(x) to the input current at every point in time. For simplicity,
we use a positive and bounded threshold-linear transfer function
xð Þ ¼ ½xgþif xg< max
max otherwise;
8
<
:ð1Þ
where []
+
indicates the positive part, γis the activation threshold and ϕ
max
the maximum fir-
ing rate.
Single neuron adaptation is described by the variable w(t) that low-pass filters the linearized
firing rate with a timescale τ
w
, slower than the membrane time constant τ
m
, and feeds it back
with opposite sign into the input current dynamics (see Methods). The dynamics of the i-th
adaptive neuron are given by
tm_
xiðtÞ ¼ xiðtÞ þ PN
j¼1JijðxjðtÞÞ gwwiðtÞ þ IiðtÞ
tw_
wiðtÞ ¼ wiðtÞ þ xiðtÞ g;
(ð2Þ
where I
i
(t) is the external input current to neuron i.
Synaptic filtering consists in low-pass filtering the synaptic input received by a cell with
time constant τ
s
, before it contributes to the input current. The dynamics of the i-th neuron in
a network with synaptic filtering are
tm_
xiðtÞ ¼ xiðtÞ þ siðtÞ
ts_
siðtÞ ¼ siðtÞ þ PN
j¼1JijðxjðtÞÞ þ IiðtÞ:
(ð3Þ
The matrix element J
ij
corresponds to the synaptic coupling strength from neuron jonto
neuron i. In this study we focus on neuronal populations of inhibitory and excitatory units,
whose connectivity is sparse, random, with constant in-degree: all neurons receive exactly the
same number of excitatory and inhibitory connections, C
E
and C
I
, as in [31–33]. All excitatory
synapses have equal strength Jand all inhibitory neurons −gJ. Furthermore, we consider the
large network limit where the number of synaptic neurons Nis large while keeping the excit-
atory and inhibitory inputs C
E
and C
I
fixed.
Single unit: Timescales of dynamics
In the models studied here the input current of individual neurons is described by a linear sys-
tem. Thus, their activity is fully characterized by the response h(t) to a brief impulse signal, i.e.
the linear filter. When such neurons are stimulated with a time-varying input I(t), the response
is the convolution of the filter with the input, x(t) = (hI)(t). These filters can be determined
analytically for both neurons with adaptation or synaptic filtering and directly depend on the
parameters of these processes. Analyzing the differences that these two slow processes produce
in the linear filters is useful for studying the differences in the response of adaptive and synap-
tic filtering neurons to temporal stimuli (Fig 1A), and will serve as a reference for comparison
to the effects that emerge at the network level.
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 3 / 33
In particular, the filter of a neuron with synaptic filtering, h
s
(t), is the sum of two exponen-
tially decaying filters of opposite signs and equal amplitude, with time constants τ
s
and τ
m
:
hstð Þ ¼ 1
tstm
et
tset
tm
Ytð Þ;ð4Þ
where Θ(t) is the Heaviside function (see Methods). Thus, the current response of a neuron to
an input pulse received from an excitatory presynaptic neuron is positive and determined by
two different timescales. The response first grows with timescale τ
m
, so that the neuron cannot
respond to any abrupt changes in the synaptic input faster than this timescale, and then
decreases back to zero with timescale τ
s
(grey curves, Fig 1B).
The adaptation filter is given as well by the linear combination of two exponential func-
tions. In contrast to the synaptic filter, since the input in the adaptive neuron model affects
directly the current variable x
i
(t), there is an instantaneous change in the firing rate to an input
delta-function (red curves, Fig 1B). The timescales of the two exponentials can be calculated as
t¼2tmtw
twþtm
1ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
14tmtwð1þgwÞ
ðtmþtwÞ2
s
!1
:ð5Þ
When the argument of the square root in Eq (5) is negative, the two timescales correspond to a
pair of complex conjugate numbers, so that the filter is an oscillatory function whose ampli-
tude decreases monotonically to zero at a single timescale. If the argument of the square root is
positive, for slow enough adaptation, the two timescales are real numbers and correspond to
exponential functions of opposing signs of decaying amplitude. However, the amplitudes of
these two exponentials are different (see Methods). To illustrate this, we focus on the limit of
large adaptation time constants with respect to the membrane time constant, where the two
exponential functions evolve with timescales that decouple the contribution of the membrane
Fig 1. Activity of individual neurons with adaptation or synaptic filtering. A: Firing rate response of two different neurons with adaptation (red curves) and two
different neurons with synaptic filtering (grey curves) to the same time-varying input (black curve). B: Normalized linear filters for the neurons shownin A. C:
Timescales of the linear filter for neurons with adaptation (red lines) and for neurons with synaptic filtering (grey lines) as a function of the timescale τ
w
or τ
s
,
respectively. The dashed lines indicate the effective timescale of the evoked activity obtained by weighing each individual timescale with its amplitude in the linear
filter. The effective timescale for neurons with adaptation saturates for large adaptation time constants, while it grows proportionally to the synaptic time constant for
neurons with synaptic filtering. Note that for the adaptive neuron, if the two eigenvalues are complex conjugate, there is only one decay timescale. The triangles on the
temporal axis indicate the time constants used in A and B. Adaptation coupling g
w
= 5. D: Variance of the input current as a function of the slow time constant when
the adaptive and synaptic neurons are stimulated with Gaussian white noise of unit variance. In the case of neurons with adaptation, twodifferent values of the
adaptation coupling g
w
are shown. Time in units of the membrane time constant τ
m
.
https://doi.org/10.1371/journal.pcbi.1006893.g001
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 4 / 33
time constant and the adaptation current. In that limit, the adaptive filter reads
hwtð Þ ¼ gw
tw
e1þgw
ð Þ t
twþ1
tm
et
tm
Ytð Þ:ð6Þ
The amplitude of the slow exponential is inversely related to its timescale so that the integral of
this mode is fixed, and independent of the adaptation time constant. This implies that a sever-
alfold increase of the adaptation time constant does not lead to strong changes in the single
neuron activity for time-varying signals (Fig 1A).
Furthermore, we can characterize the timescale of the single neuron response as the sum of
the exponential decay timescales weighed by their relative amplitude, and study how this char-
acteristic timescale evolves as a function of the time constants of either the synaptic or the
adaptive current (Fig 1C). For adaptive neurons, the activity timescale is bounded as a conse-
quence of the decreasing amplitude of the slow mode, i.e. increasing the adaptation time con-
stant beyond a certain value will not lead to a slower response. In contrast, the activity of an
individual neuron with synaptic filtering scales proportionally to the synaptic filter time,
since the relative amplitudes of the two decaying exponentials are independent of the time
constants.
When any of the two neuron types are stimulated with white Gaussian noise, the variance
in the response is always smaller than the input variance, due to the low pass filtering proper-
ties of the neurons. However, this gain in the variance of the input currents is modulated by
the different neuron parameters (Fig 1D). For a neuron with synaptic filtering, the gain is
inversely proportional to the time constant τ
s
. In contrast, for a neuron with adaptation,
increasing the adaptation time constant has the opposite effect of increasing the variance of
the current response. This is because when the adaptation time constant increases, the ampli-
tude of the slow exponential decreases accordingly, and the low-pass filtering produced by this
slow component is weaker. Following the same reasoning, increasing the adaptation coupling
corresponds to strengthening the low-pass filtering performed by adaptation, so that the vari-
ance decreases (Fig 1D, dashed vs full red curves).
Population-averaged dynamics
In the absence of any external input, a non-trivial equilibrium for the population averaged
activity emerges due to the recurrent connectivity of the network. The equilibrium firing rate
is identical across network units, since all units are statistically equivalent. We can write the
input current x
0
at the fixed point as the solution to the transcendental equation
ð1þgwÞx0¼JðCEgCIÞðx0Þ þ gwg;ð7Þ
for the network with adaptation, and to
x0¼JðCEgCIÞðx0Þ;ð8Þ
for synaptic filtering (see Methods). Based on Eq (7), we find that the adaptation coupling g
w
reduces the mean firing rate of the network, independently of whether the network is domi-
nated by inhibition or excitation (Fig 2A). Synaptic filtering instead does not play any role in
determining the equilibrium activity of the neurons, since Eq (8) is independent of the synap-
tic filtering parameter τ
s
.
We next study the stability and dynamics of the equilibrium firing rate in response to a
small perturbation uniform across the network, x
i
(t) = x
0
+δx(t). Because of the fixed in-
degree of the connectivity matrix, the linearized dynamics of each neuron are identical, so that
the analysis of the homogeneous perturbation on the network reduces to the study of a two-
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 5 / 33
dimensional deterministic system of differential equations which corresponds to the dynamics
of the population-averaged response (see Methods). The stability and timescales around equi-
librium depend on the two eigenvalues of this linear 2D-system. More specifically, the fixed
point is stable to a homogeneous perturbation if the two eigenvalues of the dynamic system
have negative real part, in which case the inverse of the unsigned real part of the eigenvalues
determines the timescales of the response. For both the network with synaptic filtering and the
network with adaptive neurons, the order parameter of the connectivity that determines the
stability of the fixed point is the effective recurrent coupling J(C
E
−gC
I
) each neuron receives,
resulting from the sum of all input synaptic connections. A positive (negative) effective cou-
pling corresponds to a network where recurrent excitation (inhibition) dominates and the
recurrent input provides positive (negative) feedback [32,33].
For networks with synaptic filtering, we find that the synaptic time constant does not alter
the stability of the equilibrium state, so that the effective coupling alone determines the stabil-
ity of the population-averaged activity. As the effective input coupling strength is increased,
the system undergoes a saddle-node bifurcation when the effective input is J(C
E
−gC
I
) = 1 (Fig
2C). In other words, the strong positive feedback loop generated by the excitatory recurrent
connections destabilizes the system.
To analyze the timescales elicited by homogeneous perturbations, we calculate the eigenval-
ues and eigenvectors of the linearized dynamic system (see Methods). We find that for inhibi-
tion-dominated networks (J(C
E
−gC
I
)<0), the network shows population-averaged activity
at timescales that interpolate between the membrane time constant and the synaptic time con-
stant. As the effective coupling is increased, the slow timescale at the network level can be
made arbitrarily slow by tuning the effective synaptic coupling close to the bifurcation value, a
well-known network mechanism to achieve slow neural activity [12].
In the limit of very slow synaptic timescale, the two timescales of the population-averaged
activity are
tþ¼ts
1JðCEgCIÞ;ð9Þ
t¼tm1J CEgCI
ð Þ ts
tm
;ð10Þ
Fig 2. Equilibrium firing rate and phase diagrams of the population-averageddynamics. A: Firing rate of the network with adaptation at the
equilibrium ϕ(x
0
) for increasing adaptive couplings and three different values of the effective recurrent coupling J
eff
=J(C
E
−gC
I
). Stronger adaptation
leads to lower firing rates at equilibrium. B: Phase diagram of the population-averaged activity for the network with adaptation. C: Phase diagram for
the network with synaptic filtering.
https://doi.org/10.1371/journal.pcbi.1006893.g002
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 6 / 33
so that the timescale τ
−
is proportional to the membrane time constant and τ
+
is proportional
to the slow synaptic time constant, effectively decoupling the two timescales. The relative con-
tribution of these two timescales is the same, independently of the time constant τ
s
, as we
found in the single neuron analysis.
The network with adaptation shows different effects on the population-averaged activity.
First, the presence of adaptation modifies the region of stability: the system is stable when the
effective recurrent input J(C
E
−gC
I
) is less than the minimum of 1 + g
w
and 1þtm
tw(see Meth-
ods). Therefore, the stability region is larger than for the network with synaptic filtering (Fig
2B vs Fig 2C). In other words, the effective excitatory feedback required to destabilize the net-
work is larger due to the counterbalance provided by adaptation. Moreover, adaptation allows
the network to undergo two different types of bifurcations as the effective input strength
increases, depending on the adaptation parameters. One possibility is a saddle-node bifurca-
tion, as in the synaptic case, which takes place when J(C
E
−gC
I
) = 1 + g
w
. Beyond that instabil-
ity all neurons in the network saturate. The other possible bifurcation, which happens if
tm
tw<gw, at an effective coupling strength J CEgCI
ð Þ ¼ 1þtm
tw, is a Hopf bifurcation: the fixed
point of network becomes unstable, leading in general to oscillating dynamics of the popula-
tion-averaged response. Note that in the limit of very slow adaptation, the system can only
undergo a Hopf bifurcation (Fig 2B).
The two timescales of the population-averaged activity in the stable regime for the adaptive
network decouple the two single neuron time constants when adaptation is much slower than
the membrane time constant. In this limit, up to first order of the adaptive time ratio tm
tw, the
two activity timescales are
tþ¼tm
1JðCEgCIÞ;ð11Þ
t¼twð1JðCEþgCIÞÞ
1þgwJðCEgCIÞ:ð12Þ
Similar to the single neuron dynamics, the amplitude of the slow mode, corresponding to τ
−
,
decreases as τ
w
is increased, so that the contribution of the slow timescale is effectively reduced
when τ
w
is very large. On the contrary, the mode corresponding to τ
+
, proportional to the
membrane time constant can be tuned to reach arbitrarily large values. This network mecha-
nism to obtain slow dynamics does not depend on the adaptation properties.
Heterogeneous activity
Linear stability analysis. Previous studies have shown that random connectivity can lead
to heterogeneous dynamics where the activity of each unit fluctuates strongly in time [29,33–
35]. To assess the effects of additional hidden degrees of freedom on the emergence and time-
scales of such fluctuating activity, we examine the dynamics when each unit is perturbed inde-
pendently away from the equilibrium, x
i
(t) = x
0
+δx
i
(t). By linearizing the full 2N-dimensional
dynamics around the fixed point, we can study the stability and timescales of the activity char-
acterized by the set of eigenvalues of the linearized system, λ
s
and λ
w
for the network with syn-
aptic filtering neurons and adaptation, respectively. These sets of eigenvalues are determined
by a direct mapping to the eigenvalues of the connectivity matrix, λ
J
(see Methods). The eigen-
values λ
J
of the connectivity matrices considered are known in the limit of large networks [33,
36]: they are enclosed in a circle of radius Jffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
CEþg2CI
p, except for an outlier that corresponds
to the population-averaged dynamics, studied in the previous section. Therefore, we can map
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 7 / 33
the circle that encloses the eigenspectrum λ
J
into a different shape in the space of eigenvalues
λ
s/w
(insets Fig 3). In order to determine the stability of the response to the perturbation, we
assess whether the real part of the eigenspectrum λ
s/w
is negative at all possible points. Further-
more, the type of bifurcation is determined by whether the curve enclosing the eigenvalues λ
s,w
crosses the imaginary axis at zero frequency or at a finite frequency when the synaptic coupling
strength is increased, leading respectively to a zero-frequency or to a Hopf bifurcation [37].
The order parameter of the connectivity that affects the stability and dynamics of the net-
work is now the radius of the circle of eigenvalues λ
J
, i.e. Jffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ðCEþg2CIÞ
p. This parameter is the
standard deviation of the synaptic input weights of a neuron (see Methods), which contrasts
with the order parameter of the population-averaged response, that depends on the mean of
the synaptic input weights. The mean and standard deviation of the synaptic connectivity can
be chosen independently, so that while the population-averaged activity remains stable, the
individual neurons might not display stable dynamics. To analyze solely the heterogeneous
response of the network to the perturbation, we focus in the following on network connectivi-
ties whose population-averaged activity is stable, i.e. the effective synaptic coupling is inhibi-
tory or weakly excitatory.
Fig 3. Dynamical regimes as the coupling strength is increased. Numerical integration of the dynamics for the network with adaptive neurons (row A)
and the network with synaptic filtering (row B), as the coupling standard deviation Jcs ¼Jffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
CEþg2CI
pis increased. Colored lines correspond to the firing
rates of individual neurons, the black line indicates the population average activity. Insets: complex eigenspectrum λ
w/s
of the linearized dynamical matrix
around the fixed point. Dots: eigenvalues of the connectivity matrix used in the network simulation. Solid line: theoretical prediction for the envelope of
the eigenspectrum. The imaginary axis, Re(λ) = 0, is the stability boundary. i. Both the network with adaptation and synaptic transmission are stable. ii.
The network with synaptic filtering crosses the stability boundary and shows fluctuations in time and across neurons, while the network with adaptation
remains stable. iii. The network with synaptic filtering displays stronger fluctuations. The network with adaptive neurons undergoes a Hopf bifurcation
leading to strong oscillations at a single frequency with uncorrelated phases across units. Note in the inset that for this connectivity matrix there is only one
pair of complex conjugate unstable eigenvalues in the finite network. iv. The network with synaptic filtering shows strong fluctuations. The network with
adaptation displays fluctuating activity with an oscillatory component. Parameters: in A, g
w
= 0.5, and τ
w
= 5, in B, τ
s
= 5.
https://doi.org/10.1371/journal.pcbi.1006893.g003
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 8 / 33
We find that in the network with synaptic filtering, the eigenspectrum λ
s
always crosses the
stability bound through the real axis, which takes place when the spectral radius of the connec-
tivity is one, Jffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
CEþg2CI
p¼1. Thus the system undergoes a zero-frequency bifurcation simi-
lar to randomly connected networks without hidden variables [29,33,35,38], leading to
strong fluctuations at the single neuron level that are self-sustained by the network connectiv-
ity (Fig 3Bii–3Biv). The critical coupling at which the equilibrium firing rate loses stability is
independent of the synaptic time constant, i.e. synaptic filtering does not affect the stability of
heterogeneous responses (Fig 4A). However, the synaptic time constant τ
s
affects the time-
scales at which the system returns to equilibrium after a perturbation, because the eigenvalues
λ
s
(see Eq (69) in Methods) depend explicitly on τ
s
.
For a network with adaptive neurons, we calculate the eigenspectrum λ
w
and find that the
transition to instability Re(λ
w
) = 0 can happen either at zero frequency or at a finite frequency
(see Methods), leading to a Hopf bifurcation (as in inset Fig 3Aiii). In particular, the network
dynamics undergo a Hopf bifurcation when
tw>tm
gwþffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
2gwðgwþ1Þ
p;ð13Þ
so that strong adaptation coupling and slow adaptation time constants lead to a finite fre-
quency bifurcation. In particular, if the coupling g
w
is larger than ffiffiffi
5
p20:236, only the
Hopf bifurcation is possible, since by construction tm
tw<1. We can also calculate the frequency
of oscillations at the Hopf bifurcation. We find that, for slow adaptive currents, the Hopf fre-
quency is inversely related to the adaptation time constant (Fig 4B), so that slower adaptation
currents produce slower oscillations at the bifurcation.
Adaptation also increases the stability of the equilibrium firing rate to a heterogeneous per-
turbation, in comparison to a network with synaptic filtering (Fig 4C). This can be intuitively
explained in geometrical terms by analyzing how adaptation modifies the shape of the eigen-
spectrum λ
w
with respect to the circular eigenspectrum of the connectivity matrix λ
J
.
The Hopf bifurcation leads to the emergence of a new dynamical regime in the network
(Fig 3Aiv), which is studied in the following section. Right at the Hopf bifurcation, the system
shows marginal oscillations at a single frequency that can be reproduced in finite-size simula-
tions whenever only one pair of complex conjugate eigenvalues is unstable (Fig 3Aiii).
Fig 4. Phase diagram and frequency of the bifurcation for the heterogeneous activity. A: Phase diagram for the network with synaptic transmission. The only
relevant parameter to assess the dynamical regime is the connectivity strength. The circles indicate the parameters used in Figs 3and 6. Triangles correspond to the
parameter combinations used in Fig 5. B: Frequency at which the eigenspectrum loses stability for the network with adaptive neurons as a function of the ratio between
membrane and adaptation time constant, τ
m
/τ
w
, for three different adaptive couplings. The dots indicate the fastest adaptive time constant for which the system
undergoes a Hopf bifurcation (Eq 84). C: Phase diagrams for the two adaptation parameters, (i) the coupling g
w
and (ii) the adaptive time constant τ
w
vs the coupling
standard deviation.
https://doi.org/10.1371/journal.pcbi.1006893.g004
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 9 / 33
Fluctuating activity: Dynamical mean field theory. The classical tools of linear stability
theory applied so far can only describe the dynamics of the system up to the bifurcation. To
study the fluctuating regime, we take a different approach and focus on the temporal statistics
of the activity, averaged over different connectivity matrices: we determine the mean and auto-
correlation function of the single neuron firing rate, and characterize the timescale of the fluc-
tuating dynamics [29,33–35,38–40]. For large networks, the dynamics can be statistically
described by applying dynamical mean field theory (DMFT), which approximates the deter-
ministic input to each unit by an independent Gaussian noise process. The full network is then
reduced to a two-dimensional stochastic differential equation, where the first and second
moments of the noise must be calculated self-consistently. We solve the self-consistent equa-
tions using a numerical iterative procedure, similar to the schemes followed in [34,41–44] (see
Methods for an explanation of the iterative algorithm and its practical limitations).
For the network with synaptic filtering, we find that the autocorrelation function of the fir-
ing rates in the fluctuating regime corresponds to a monotonically decreasing function (Fig
5A), qualitatively similar to the correlation obtained in absence of synaptic filtering [33]. This
fluctuating state has often been referred to as rate chaos and shows non-periodical heteroge-
neous activity which is intrinsically generated by the network connectivity. The main effect of
synaptic filtering is on the timescale of these fluctuations. When the synaptic time constant is
much larger than the membrane time constant, the timescale of the network activity is propor-
tional to the synaptic time constant τ
s
, as indicated by the linear dependence between the half-
width of the autocorrelation function and the synaptic timescale τ
s
, when all other network
parameters are fixed (Fig 5B).
For the network with adaptation, we focus on large adaptation time constant τ
w
, where the
network dynamics always undergo a Hopf bifurcation. The autocorrelation function in such a
case displays damped oscillations (Fig 5C). The decay in the envelope of the autocorrelation
function is due to the chaotic-like fluctuations of the firing rate activity.
We define the time lag at which the envelope of the autocorrelation function decreases as
the timescale of the network dynamics (see Methods). The timescale of the activity increases as
the adaptation timescale is increased, when all the other parameters are fixed (Fig 5D). How-
ever, this activity timescale saturates for large values of the adaptation timescale: the presence
of very slow adaptive currents, beyond a certain value, will not slow down strongly the network
activity. This saturation value depends on the connectivity strength.
Fig 5. Autocorrelation function and timescale of the network activity in the fluctuating regime. A: Autocorrelation function of the firing rates in the network
with synaptic filtering; dynamical mean field results (solid lines) with their corresponding envelopes (dashed lines), and results from simulations (empty dots).
Connectivity strength Jcs ¼Jffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
CEþg2CI
p¼1:2. B: Effective timescale of the network activity as a function of the synaptic time constant for the network with
synaptic filtering. The network coupling does not have a strong effect on the effective timescale. C: Autocorrelation function of the firing rates, as in A, for the system
with adaptive neurons. J
cs
= 1.3. D: Effective timescale of the firing rates, as in B, for the system with adaptive currents.
https://doi.org/10.1371/journal.pcbi.1006893.g005
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 10 / 33
Effects of noise. The networks studied so far, for a fixed connectivity matrix, are
completely deterministic. We next study the effects of additional white noise inputs to each
neuron, as a proxy towards understanding recurrent networks of spiking neurons with adapta-
tion and synaptic filtering. On the mean-field level, such noise is equivalent to studying a
recurrent network whose neurons fire action potentials as a Poisson process with instanta-
neous firing rate ϕ(x
i
(t)) [35,45].
Numerical simulations show that in the stable regime the additive external noise generates
weak, fast stationary dynamics around the fixed point (Fig 6Ai and 6Bi). The timescale of these
fluctuations and their amplitude depend on the distance of the eigenspectrum to the stability
line, so that the stable fluctuations for weak synaptic coupling standard deviation (Fig 6Ai) are
smaller in amplitude than those for larger coupling standard deviation (Fig 6Aii), whose eigen-
spectrum is closer to the stability boundary. For adaptation, in the fluctuating regime beyond
the Hopf bifurcation, the network activity shows again a combination of fluctuating activity
and oscillations.
We further extend the DMFT analysis to account for the additional variance of the external
white noise sources (see Methods). The autocorrelation function of the firing rates, as pre-
dicted by DMFT, does not vary drastically when weak noise is added to the network, except
Fig 6. Dynamical regimes for the network with adaptation or synaptic filtering with additive external noise. Numerical integration of the
dynamics with units receiving additive external white noise, as a proxy for spiking noise. A: Network with adaptive neurons. B: Network with synaptic
filtering. Colored lines correspond to the firing rate of individual neurons, the black line indicates the population average activity. Insets: complex
eigenspectrum λ
w/s
of the dynamic matrix at the fixed point. Dots: eigenvalues of the connectivity matrix used in the network simulation. Solid line:
theoretical prediction for the envelope of the eigenspectrum. i. Both the network with adaptation and synaptic transmission are stable, the external
noise generates stationary fluctuations around the fixed point. ii. The network with synaptic filtering undergoes a zero-frequency bifurcation. Noise
adds fast temporal variability in the firing rates. The network with adaptation remains stable, and the fluctuations are larger in amplitude. iii. The
network with adaptation undergoes a Hopf bifurcation. The firing rate activity combines the fast fluctuations produced by white noise and the chaotic
activity with an oscillatory component. iv. The network with adaptation shows highly irregular activity, and strong effects due to the activation and
saturation bounds of the transfer function. Parameters as in Fig 4, external noise σ
η
= 0.06.
https://doi.org/10.1371/journal.pcbi.1006893.g006
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 11 / 33
for very short time lags, at which white noise introduces fast fluctuations (see Fig 7). For the
network with adaptation, the autocorrelation function of the firing rates still shows damped
oscillations (Fig 7A), while for the network with synaptic filtering, similarly, weak noise does
not affect much the decay of the autocorrelation function (Fig 7D). Very strong external noise
on the other hand will reduce the effect of the underlying recurrent dynamics of the rate net-
work, since the signal to noise ratio in the synaptic input of all neurons is low.
For a fixed external noise intensity, reducing the adaptation coupling or increasing the
adaptation time constant increases the variance of the firing rate (Fig 7B), which resembles the
dependence of the variance gain for individual neurons (Fig 1D). Conversely, slower synaptic
filtering reduces the variance of the neuron’s firing rates. This is because in the network with
synaptic filtering the noise is also filtered at the synapses –in the limit of very large τ
s
, the
whole white noise is filtered out– whereas in the network with adaptation the noise affects
directly the input current, without being first processed by the adaptation variable.
However, the timescale of the activity is nonetheless drastically affected by strong noise.
External noise adds fast fluctuations on top of the intrinsically generated dynamics of the het-
erogeneous network with adaptation or synaptic filtering. If the noise is too strong, the effec-
tive timescale of the activity takes into account mostly this fast component. In that limit, the
Fig 7. Autocorrelation function, variance of the firing rates and timescale of the network activity with external noise predicted by
dynamical mean field theory. A: Autocorrelation function of the firing rates for the network with adaptive neurons for three different noise
intensities. Adaptation time constant τ
w
= 1.25. B: Variance of the firing rate as a function of the adaptation time constant for two different
adaption couplings g
w
. Increasing the adaptation time constant or decreasing the adaptation coupling increasesthe variance. σ
η
= 0.15. C:
Timescale of the firing rate as a function of the adaptation time constant, and three different noise levels. Parameters: g
w
= 0.5, and
Jffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
CEþg2CI
p¼1:2. D: Autocorrelation function of the firing rate for the network with synaptic transmission for three different noise levels.
Synaptic time constant τ
s
= 1.25. E: Variance of the firing rate as a function of the synaptic time constant, for three different external noise levels.
Synaptic filtering reduces the variance. F: Timescale of the activity for the network with synaptic filtering and external noise.
https://doi.org/10.1371/journal.pcbi.1006893.g007
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 12 / 33
timescale of the activity is almost independent of the synaptic or adaptive time constants (Fig
7C and 7F, largest noise intensity).
Discussion
We examined dynamics of excitatory-inhibitory networks in which each unit had a hidden
degree of freedom that represented either firing-rate adaptation or synaptic filtering. The core
difference between adaptation and synaptic filtering was how external inputs reached the sin-
gle-unit activation variable that represents the membrane potential. In the case of adaptation,
the inputs directly entered the activation variable, which was then filtered by the hidden, adap-
tive variable through a negative feedback loop. In the case of synaptic filtering, the external
inputs instead reached first the hidden, synaptic variable and were therefore low-pass filtered
before being propagated in a feed-forward fashion to the activation variable. While both mech-
anisms introduce a second timescale in addition to the membrane time constant, our main
finding is that the interplay between those two timescales is very different in the two situations.
Surprisingly, in presence of adaptation, the membrane timescale remains the dominant one in
the dynamics, while the contribution of the adaptation timescale appears to be weak. In con-
trast, in a network with synaptic filtering, the dominant timescale of the dynamics is directly
set by the synaptic variable, and the overall dynamics are essentially equivalent to a network in
which the membrane time-constant is replaced with the synaptic one.
We used a highly abstracted model, in which each neuron is represented by membrane cur-
rent that is directly transformed into a firing-rate through a non-linear transfer function. This
class of models has been popular for dissecting dynamics in excitatory-inhibitory [27,28,46–
48] or randomly-connected networks [29,30,33], and for implementing computations [49,
50]. Effects of adaptation in this framework have to our knowledge not been examined so far,
but see [51] for a simultaneously and independently developed study of adaptation in networks
of multidimensional rate units with random Gaussian connectivity. We therefore extended the
standard rate networks by introducing adaptation in an equally abstract fashion [24], as a hid-
den variable specified solely by a time constant and a coupling strength. Different values of
those parameters can be interpreted as corresponding to different specific membrane conduc-
tances that implement adaptation, e.g. the calcium dependent potassium I
ahp
current or the
slow voltage-dependent potassium current I
m
, which are known to exhibit timescales over sev-
eral orders of magnitude [52,53]. To cover the large range of adaptation timescales observed
in experiments [23], it would be straightforward to superpose several hidden variables with dif-
ferent time constants. Our approach could also be easily extended to include simultaneously
adaptation and synaptic filtering.
A number of previous works have studied the effects of adaptation within more biologically
constrained, integrate-and-fire models. These works have in particular examined the effects of
adaptation on the spiking statistics [54–56], firing-rate response [57,58], synchronisation [25,
56,59–61], perceptual bistability [62] or single-neuron coding [63,64]. In contrast, we have
focused here on the relation between the timescales of adaptation and those of network
dynamics. While our results rely on a simplified firing-rate model, we expect that they can be
directly related to networks of spiking neurons by exploiting quantitative techniques for map-
ping adaptive integrate-and-fire models to effective firing rate descriptions [65].
A side result of our analysis is the finding that strong coupling in random recurrent net-
works with adaptation generically leads to a novel dynamical state, in which individual units
exhibit a mixture of oscillatory and strong temporal fluctuations. The characteristic signature
of this dynamical state is a damped oscillation found in the auto-correlation function of single-
unit activity. In contrast, classical randomly connected networks lead to a fluctuating, chaotic
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 13 / 33
state in which the auto-correlation function decays monotonically [29,33–35]. Note that the
oscillatory activity of different units is totally out of phase, so that no oscillation is seen at the
level of population activity. This dynamical phenomenon is analogous to heterogeneous oscil-
lations in anti-symmetrically connected networks with delays [37]. In both cases, the oscil-
latory dynamics emerge through a bifurcation in which a continuum of eigenvalues crosses
the instability line at a finite-frequency. Similar dynamics can be also found in networks in
which the connectivity is a superposition of a random and a rank two structured part [33]. In
that situation, the heterogeneous oscillations however originate from a Hopf bifurcation due
to an isolated pair of eigenvalues that correspond to the structured part of the connectivity.
Our main aim here was to determine how hidden variables could induce long timescales in
randomly-connected networks. Long timescales could alternatively emerge from non-random
connectivity structure. As extensively investigated in earlier works, one general class of mecha-
nism relies on setting the connectivity parameters close to a bifurcation that induces arbitrarily
long timescales [12,29]. Another possibility is that non-random features of the connectivity,
such as the over-representation of reciprocal connections [66,67] slow down the dynamics
away from any bifurcation. A recent study [68] has indeed found such a slowing-down. Weak
connectivity structures of low-rank type provide yet another mechanism for the emergence of
long timescales. Indeed, rank-two networks can generate slow manifolds corresponding to
ring attractors provided a weak amount of symmetry is present [69].
Ultimately, the main reason for looking for long timescales in the dynamics is their poten-
tial role in computations performed by recurrent networks [70,71]. Recent works have pro-
posed that adaptive currents may help implement computations in spiking networks by either
introducing slow timescales or reducing the amount of noise due to spiking [72,73]. Our
results suggest that synaptic filtering is a much more efficient mechanism to this end than
adaptation. Identifying a clear computational role for adaptation in recurrent networks there-
fore remains an open and puzzling question.
Methods
Network model
We compare the dynamics of two different models: a recurrent network with adaptive neu-
rons, and a recurrent network with synaptic filtering. Each model is defined as a set of 2Ncou-
pled differential equations. The state of the i-th neuron is determined by two different
variables, the input current x
i
(t) and the adaptation (synaptic) variable w
i
(t) (s
i
(t)).
Adaptation. The dynamics of the recurrent network with adaptive neurons are given by
tm_
xiðtÞ ¼ xiðtÞ gwwiðtÞ þ IiðtÞ
tw_
wiðtÞ ¼ wiðtÞ þ ðxiðtÞÞ;
(ð14Þ
where ϕ(x) is a monotonically increasing non-linear function that transforms the input current
into firing rate. In this study, we use a threshold-linear transfer function with saturation:
xð Þ ¼ ½xgþif xg< max
max otherwise:
8
<
:ð15Þ
In Eq (14) adaptation in single neuron rate models is defined as a low-pass filtered version
with timescale τ
w
of the neuron’s firing rate ϕ(x
i
(t)), and is fed back negatively into the input
current, with a strength that we call the adaptation coupling g
w
. For the sake of mathematical
tractability, we linearize the dynamics of the adaptation variable by linearizing the transfer
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 14 / 33
function (Eq 15), ϕ(x
i
(t)) x
i
(t)−γ. Therefore, the dynamics of the network model with
adaptation studied here read
tm_
xiðtÞ ¼ xiðtÞ gwwiðtÞ þ IiðtÞ
tw_
wiðtÞ ¼ wiðtÞ þ xiðtÞ g;
(ð16Þ
Note that this approximation allows for adaptation to increase the input current of a neu-
ron, when the neuron’s current is below the activation threshold γ.
Synaptic filtering. For the recurrent network with synaptic filtering, the dynamics are
tm_
xiðtÞ ¼ xiðtÞ þ siðtÞ þ IiðtÞ
ts_
siðtÞ ¼ siðtÞ þ IiðtÞ:
(ð17Þ
In Eqs (14), (16) and (17), I(t) represents the total external input received by the neuron. In
general, we are interested in the internally generated dynamical regimes of the network, so
that the input is given by the synaptic inputs
IiðtÞ ¼ Isyn;i¼X
j
JijðxjðtÞÞ:ð18Þ
The matrix element J
ij
indicates the coupling strength of the j-th neuron onto the i-th neuron.
The connectivity matrix is sparse and random, with constant in-degree [32,33,74]: all neurons
receive the same number of input connections C, from which C
E
are excitatory and C
I
inhibi-
tory. All excitatory synapses have coupling strength Jwhile the strength of all inhibitory synap-
ses is −gJ. Moreover, each neuron can only either excite or inhibit the rest of the units in the
network, following Dale’s principle. Therefore, the total effective input coupling strength,
which is the same for all neurons, is
Jeff ≔X
j
Jij ¼JðCEgCIÞ:ð19Þ
We used the parameters in Table 1 for all figures unless otherwise specified.
Single neuron dynamics
The dynamics of each individual neuron are described by a two-dimensional linear system,
which implies that the input current response x(t) to a time-dependent input I(t) is the convo-
lution of the input with a linear filter h(τ) that depends on the parameters of the linear system:
xðtÞ ¼ ðhIÞðtÞ ¼ Zþ1
1
dt0hðt0ÞIðtt0Þ:ð20Þ
Table 1. Parameter values used in the simulations.
Parameter Value
Number of units N3000
In-degree C100
Excitatory inputs C
E
80
Inhibitory inputs C
I
20
Ratio I-E coupling strength g4.1
Threshold γ-0.5
Maximum firing rate ϕ
max
2
https://doi.org/10.1371/journal.pcbi.1006893.t001
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 15 / 33
In general, for any linear dynamic system _
zðtÞ ¼ Az þbðtÞ, where Ais a square matrix in
RNNand b(t) is a N-dimensional vector, the dynamics are given by
zðtÞ ¼ Z1
1
dt0eAt0Yðt0Þbðtt0Þ;ð21Þ
where Θ(t) is the Heaviside function. Thus, comparing Eqs (21) and (20), the linear filter is
determined by the elements of the so-called propagator matrix P(t) = e
At
Θ(t).
Synaptic filtering. For a single neuron wit synaptic filtering, the dynamics are given by
Eq (17), where the input I
i
(t) represents the external current. We write the response in its vec-
tor form (x(t), s(t))
T
and the input as (0, I(t))
T
. The dynamic matrix is
As¼t1
mt1
m
0t1
s
!:ð22Þ
The linear filter, h
s
(t0), is given by the entries of the propagator matrix that links the input I
(t) to the output element x(t), which are in this case only the entry in row one and column two:
h
s
(t0) = [P(t0)]
12
. To compute the required entry of the propagator, we diagonalize the dynamic
matrix A=VDV
−1
. The matrix Dis a diagonal matrix with the eigenvalues of matrix Ain the
diagonal entries, and Vis a matrix whose columns are the corresponding eigenvectors. Apply-
ing the identity etVDV1¼VetD V1and the definition of propagator we obtain that
hstð Þ ¼ Ytð Þ 1
tmts
et
tmet
ts
:ð23Þ
The two timescales of the activity are defined by the inverse of the eigenvalues of the system,
which coincide with τ
m
and τ
s
. Every time a pulse is given to the neuron, both modes get acti-
vated with equal amplitude and opposing signs, as indicated by Eq 23. This means that there is
a fast ascending phase after a pulse, at a temporal scale τ
m
, and a decay towards zero with time-
scale τ
s
.
Adaptation. The dynamics of a single adaptive neuron are determined by Eq (16), where
I
i
(t) is the external input to the neuron. We apply the same procedure to determine the time-
scales of the response of an adaptive neuron to time-dependent perturbations. The dynamic
matrix for an adaptive neuron reads
Aw¼t1
mgwt1
m
t1
wt1
w
!:ð24Þ
Its eigenvalues are
l
w¼1
2t1
mt1
wffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ðt1
mþt1
wÞ24ð1þgwÞt1
mt1
w
q
:ð25Þ
and the eigenvectors
x¼gw
tm
;1
21
tmþ1
twffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
1
tm1
tw
2
4gw
tmtw
s
0
@1
A
0
@1
A
T
:ð26Þ
The eigenvalues are complex if and only if g
w
>(4τ
m
τ
w
)
−1
(τ
w
−τ
m
)
2
, and in that case their real
part is 1
2tmtwtmþtw
ð Þ. As the adaptive time constant becomes slower, at a certain critical adap-
tation time constant both eigenvalues become real. We are interested in the behavior when the
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 16 / 33
adaptation time constant is large. The absolute value of the inverse of the eigenvalues deter-
mines the time constants of the dynamics. Therefore, for large τ
w
we can calculate the two real
eigenvalues to first order of t1
w
lþ
w¼ 1þgw
twþOt2
w
ð27Þ
l
w¼ t1
mþgwt1
wþOðt2
wÞ:ð28Þ
In this limit of slow adaptation, the time constant of one eigenmode is proportional to τ
w
,
whereas the second mode scales with τ
m
. We are interested in the amplitude of each mode
with respect to the other.
By explicitly calculating the first entry of the propagator matrix we obtain the adaptive filter
in terms of the eigenvectors and eigenvalues,
hwtð Þ ¼ 1
tm
1
xþ
1x
2x
1xþ
2
xþ
1x
2elþtx
1xþ
2elt
;ð29Þ
where we use the notation xþ
1to indicate the first component of the eigenvector associated to
the eigenvalue λ
+
. Approximating to leading order of t1
wthe eigenvectors in Eq (26), we obtain
the eigenvectors
x¼1
tmðgw;0ÞT1
twð0;gwÞT¼gw
1
tm
;1
tw
T
ð30Þ
xþ¼1
tmðgw;1ÞTþ1
twð0;1þgwÞT¼gw
tm
;1
tmþ1þgw
tw
T
:ð31Þ
Then, using Eqs (29), (30) and (31), we determine the linear filter:
hwtð Þ ¼ gw
tmð2gwþ1Þ tw
e1þgw
twtþ1
tm
11þgw
ð Þtm
tw
11þ2gw
ð Þtm
tw
e1
tmgw
tw
ð Þt:ð32Þ
Interestingly, in contrast with synaptic filtering, the amplitude of the two modes are not
equal. The amplitude of the slow mode (first term in Eq 32), whose timescale is proportional
to τ
w
, decays proportionally to t1
wwith respect to the fast mode, when τ
w
τ
m
(2g
w
+ 1).
Therefore, the area under the linear filter corresponding to this mode is independent of τ
w
for
very large adaptation time constants:
lim
tw!1 Z1
0
hþ
wtð Þdt ¼lim
tw!1
gwtw
tmðgwþ1Þð2gwþ1Þ ðgwþ1Þtw¼ gw
gwþ1:ð33Þ
It follows that, if the adaptation timescale is increased, its relative contribution to the activity
will decrease by the same factor, so that very slow adaptive currents will effectively be masked
by the fast mode.
Equilibrium activity
The two systems possess a non-trivial equilibrium state at which the input current of all units
stays constant. Since all units are statistically equivalent, the equilibrium activity is the same
for all units. For synaptic filtering, the input current at equilibrium is given by a transcendental
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 17 / 33
equation, that is obtained by setting to zero the left hand side of Eq (17):
x0¼JðCEgCIÞðx0Þ:ð34Þ
This equilibrium coincides with the fixed point of the system without synaptic filtering.
For adaption, instead, from Eq (16) we obtain that the equilibrium is determined by
x0¼1
1þgw
J CEgCI
ð Þx0
ð Þ þ gwgð Þ:ð35Þ
We further assume unless otherwise specified that the fixed point of the system is in the linear
regime of the transfer function, so that ϕ(x) = x−γ. In that case x
0
= (J(C
E
−gC
I
)−g
w
) (x
0
−
γ), so that larger adaptation coupling corresponds to weaker input currents, i.e. decreasing sta-
tionary firing rate. The adaptation time constant does not affect the fixed point.
Dynamics of homogeneous perturbations
We study the neuronal dynamics in response to a small perturbation uniform across the net-
work
xiðtÞ ¼ x0þdxðtÞ:ð36Þ
Synaptic filtering. Linearizing Eq 17 we obtain
tmd_
xiðtÞ ¼ dxðtÞ þ dsiðtÞ
tsd_
siðtÞ ¼ dsiðtÞ þ 0
0PjJijdxðtÞ;
(ð37Þ
where we use the notation 0
0≔dðxÞ
dx jx0. Because the perturbation δxin Eq (37) is independent
of j, using Eq (19) the dynamics for all units are equivalent to the population-averaged dynam-
ics and are given by
tmd_
xðtÞ ¼ dxðtÞ þ dsðtÞ
tsd_
sðtÞ ¼ dsðtÞ þ 0
0JðCEgCIÞdx:
(ð38Þ
From Eq (38) we can define the dynamic matrix
As¼1
tm
1 1
0
0J CEgCI
ð Þtm
tstm
ts
0
@1
A:ð39Þ
The only difference in the linearized dynamics of the population-averaged current with respect
to the single neuron dynamics (Eq 22) is the non-diagonal entry 0
0JðCEgCIÞ. When either
the derivative at the fixed point cancels, or when the total effective input is zero, the population
dynamics equals the dynamics of a single neuron. The eigenvalues of the population-averaged
dynamics are
l
s¼ tmþts
2tstmffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
tmts
2tstm
2
þJðCEgCIÞ
tmts
s:ð40Þ
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 18 / 33
and the eigenvectors
x
s¼ 1;tmts
2tstmffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
tmts
2tstm
2
þJðCEgCIÞ
tmts
s
0
@1
A
T
:ð41Þ
For very large synaptic time constants, the eigenvalues are approximated to leading order as
lþ
s¼JðCEgCIÞ 1
tsþOt2
s
ð42Þ
l
s¼ 1
tmJðCEgCIÞ
tsð43Þ
Approximating as well the eigenvectors to leading order, we obtain
xþ¼1
tm
;1
tm1JðCEgCIÞ
ts
T
ð44Þ
x¼1
tm
;JðCEgCIÞ
ts
T
ð45Þ
the filter of the linear response to weak homogeneous perturbations reads:
hsðtÞ ¼ 1
ts
x
1xþ
1
xþ
1x
2x
1xþ
2
eltelþt
ð46Þ
¼1
ts
tstmð1JðCEgCIÞÞ
tstmð12JðCEgCIÞÞ eltelþt
ð47Þ
Note that the amplitude of the two exponential terms is the same, independently of the effec-
tive coupling and time constants.
Adaptation. For the system with adaptive neurons, the linearized system reads
tmd_
xiðtÞ ¼ dxiðtÞ gwdwiðtÞðtÞ þ 0
0PjJijdxðtÞ
twd_
wiðtÞ ¼ dwiðtÞ þ dxðtÞ:
(ð48Þ
As for the network with synaptic filtering, the dynamics of the perturbation are equivalent for
each unit, so that we can write down the dynamic matrix for the population-averaged response
to homogeneous perturbations
Aw¼1
tm
1þ0
0JðCEgCIÞ gw
tm
twtm
tw
0
@1
A:ð49Þ
The difference with respect to the linear single neuron dynamics (Eq 48) is that the effective
recurrent coupling appears now in the first diagonal entry of the dynamic matrix.
When the fixed point is located within the linear range of the transfer function, the deriva-
tive is one, so that we do not further specify the factor 0
0in the following equations. Conse-
quently, the dynamics of the system to small perturbations do not depend on the exact value of
the fixed point, which does not hold for more general transfer functions.
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 19 / 33
The eigenvalues of the system read
l
w¼ 1Jeff
2tm1
2tw
1ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
1þ4tmðJeff 1gwÞ
twJeff 1tm
tw
2
v
u
u
t
0
B
@1
C
A;ð50Þ
with eigenvectors
x
w¼2gw;tm
twþJeff 1ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
tm
twJeff þ1
2
4tm
tw
gwJeff þ1ð Þ
s
0
@1
A
T
ð51Þ
In the limit of very slow adaptation, given that the two eigenvalues are real, they can be
approximated to leading order as
lþ
w¼1þtm
twðJðCEgCIÞ 1ÞþOt2
w
ð52Þ
l
w¼ 1
tw
1gw
JðCEgCIÞ 1
þOt2
w
ð53Þ
and the corresponding eigenvectors read
xþ
w¼1;1
Jeff 1
tm
tw
T
ð54Þ
x
w¼gw;Jeff 1þtm
tw
1gw
Jeff 1
T
:ð55Þ
Therefore, if the perturbation is stable (see next section) we can write down the corresponding
linear filter as
hwtð Þ ¼ 1
tm
Jeff 1þtm
tw1gw
Jeff
Jeff 1þtm
tw12gw
Jeff
elþ
wtgw
twðJeff 1Þ2þtmðJeff 12gwÞel
wt:ð56Þ
The area under the slow mode is again independent of the adaptation time constant in this
limit,
lim
tw!1 Z1
0
h
wtð Þdt ¼ gw
ðJeff 1ÞðJeff 1gwÞ:ð57Þ
Stability of homogeneous perturbations
The equilibrium point is stable when the real part of all eigenvalues is negative. Equivalently,
in a two dimensional system –as it is the case for the population-averaged dynamics–, the
dynamics are stable when the trace of the dynamic matrix is negative and the determinant
positive.
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 20 / 33
Synaptic filtering. In the system with synaptic filtering, the trace and determinant are
Trs¼ 1
tm1
tsð58Þ
Dets¼1JðCEgCIÞ
tmts
:ð59Þ
The trace is therefore always negative. The determinant is positive, and therefore the popula-
tion-averaged dynamics are stable, when the effective coupling J(C
E
−gC
I
) is smaller than
unity. In contrast, if the effective coupling is larger than unity, i.e. if positive feedback is too
strong, the equilibrium firing rate is unstable, so that any small perturbation to the equilibrium
firing rate will lead the system to a different state. Right at the critical effective coupling, one
eigenvalues is zero and the other one equals Tr
s
, implying that the population-averaged
dynamics undergo a saddle-node bifurcation. Beyond the bifurcation, the network reaches a
state where the firing rates of all neurons saturate.
Adaptation. In the adaptive population dynamics, the recurrent connectivity has a differ-
ent effect on the stability of the adaptive population dynamics. The trace and determinant of
the dynamic matrix are
Tr w¼ 1
tm1
twþt1
mJ CEgCI
ð Þ;ð60Þ
Det w¼ ðtmtwÞ1ð1JðCEgCIÞ þ gwÞ:ð61Þ
Both the timescale τ
w
and the strength g
w
of adaptation affect the trace and determinant of the
dynamic matrix, and therefore the stability. The system is unstable if the determinant is nega-
tive (one positive and one negative real eigenvalue) or if the determinant is positive and the
trace is positive. The determinant is negative, and therefore the system becomes unstable
through a saddle-node bifurcation, when J(C
E
−gC
I
)>1 + g
w
. Note that the adaptation
strength increases the stability of the system: a stronger positive feedback loop is required to
destabilize the fixed point, in comparison to the network with synaptic filtering. The determi-
nant and trace are positive if J(C
E
−gC
I
)<1 + g
w
but J CEgCI
ð Þ >1þtm
tw, respectively,
leading to a Hopf bifurcation: the system produces sustained marginal oscillations at the bifur-
cation in response to small perturbations around the fixed point. Beyond the Hopf bifurcation,
the oscillations are maintained in time, unless the system shows a fixed point when all neurons
saturate (x0¼1
1gwJ CEgCI
ð Þmax þgwgð Þ). This fixed point exists if x
0
>ϕ
max
+γ.
Heterogeneous activity
We next study the network dynamics beyond the population-averaged activity, along modes
where different units have different amplitudes. We study perturbations of the type
xiðtÞ ¼ x0þdxiðtÞ:ð62Þ
We define the 2N-dimensional vector x¼ ðdx1; :::; dx1
N;dw1
1; :::; dw1
NÞT. Since the dynamics of
each unit is now different, the dynamic matrix of the linearized system, A, is described by a
squared matrix of dimensionality 2N. Therefore, the perturbations generate dynamics along
2Ndifferent modes whose timescales are determined by the eigenvalues of the matrix A. The
eigenvalues are determined by the characteristic equation |A−λI| = 0. In order to calculate
these eigenvalues, we make use of the following identity which holds for any block matrix
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 21 / 33
Z=A−λI, that is composed by the four square matrices P,Q,R, and Sand the block Sis
invertible:
jZj≔P Q
R S
¼ jSjjPQS1Rj:ð63Þ
Consequently, if we set Eq (63) to zero, since we assumed that |S|6¼ 0, we obtain
jZj ¼ 0) jPQS1Rj ¼ 0:ð64Þ
The identity in Eq (63) can be shown by using the decomposition
Z¼I0
0S
! I Q
0I
! PQS1R0
S1R I
!;ð65Þ
together with the fact that when a non-diagonal block is zero. The determinant of such a
matrix is the product of determinants of the diagonal blocks.
Synaptic filtering. The dynamical matrix for the network with synaptic filtering, obtained
by linearizing Eq (17), is
ð66Þ
The matrix Jis the connectivity matrix. Again, we assume in the following that the fixed point
is located in the linear range of the transfer function, so that 0
0¼1.
The characteristic equation, obtained by combining Eqs (64) and (66), reads
1þtmls
ð ÞIþtm
tsþtmls
1tm
ts
J
¼ 1þtmls
ð Þ þ lJ
1þtsls¼0;ð67Þ
where λ
J
are the eigenvalues of the connectivity matrix. Solving for λ
J
we obtain the equation
which maps the eigenvalues of the synaptic filtering network dynamics λ
s
onto the eigenvalues
of the connectivity matrix λ
J
,
lJ¼ ð1þtmlsÞð1þtslsÞ:ð68Þ
In contrast, solving for the eigenvalues of the dynamic matrix λ
s
we obtain the inverse map-
ping
l2
sþtsþtm
tstm
lsþ1lJ
tstm¼0:ð69Þ
In other words, Eqs 69 and 68 constitute two different approaches to assessing the stability of
the system [37]. One approach is to examine whether the domain of eigenvalues λ
s
resulting
from Eq (69) intersect the line Re (λ
s
) = 0 (Fig 3, insets in B). The eigenvalues λ
J
of the connec-
tivity matrix are distributed within a circle in the complex plane, whose radius is proportional
to the synaptic strength, lJ<Jffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
CEþg2CI
pplus an outlier real eigenvalue at J(C
E
−gC
I
) that
corresponds to the homogeneous perturbations studied above (see [36]). We focus in this sec-
tion on the bulk of eigenvalues that corresponds to modes of activity with different amplitudes
for different units. We can therefore parametrize the eigenvalues λ
J
as
lJðyÞ ¼ Jffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
CEþg2CI
peiyð70Þ
and introduce the parametrization into Eq (69) to obtain an explicit expression for the curve
Effects of adaptation and synaptic filtering on the timescales of recurrent networks
PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 22 / 33
that encloses the eigenspectrum λ
s
. Note that in an abuse of notation, we denote the limits of
the eigenspectrum as λand not the eigenvalues themselves that constitute the eigenspectrum.
The alternative approach is to use the inverse mapping from the eigenvalues λ
s
to the eigen-
values of the connectivity λ
J
, by mapping the line Re (λ
s
) = 0 into the space of eigenvalues λ
J
(see S1 Fig). More specifically, the line Re (λ
s
) = 0 can be parametrized as
ls¼ io;ð71Þ
and introduced into Eq (68). In this case, the stability is assessed by whether the eigenspectrum
of the connectivity matrix Jcrosses the stability boundary or not (insets in Fig 3). This alterna-
tive approach is useful for some calculations due to the simple geometry of the connectivity
eigenspectrum λ
J
.
Taking the alternative approach, introducing Eq (71) into Eq (68), we obtain the stability
bound in the complex plane of eigenvalues λ
J
:
lsb
J¼ ð1þitmoÞð1þitsoÞ:ð72Þ
The first point of the stability curve lsb
JðoÞintersecting with a circle of increasing radius
centered at the origin is the closest point of the curve to the origin, i.e. the minimum of jlsb
Jj2
with respect to ω. The squared distance to the origin