Access to this full-text is provided by PLOS.

Content available from

**PLOS Computational Biology**This content is subject to copyright.

RESEARCH ARTICLE

Contrasting the effects of adaptation and

synaptic filtering on the timescales of

dynamics in recurrent networks

Manuel BeiranID*, Srdjan Ostojic ID*

Group for Neural Theory, Laboratoire de Neurosciences Cognitives Computationnelles, De

´partement

d’E

´tudes Cognitives, E

´cole Normale Supe

´rieure, INSERM U960, PSL University, Paris, France

*manuel.beiran@ens.fr (MB); srdjan.ostojic@ens.fr (SO)

Abstract

Neural activity in awake behaving animals exhibits a vast range of timescales that can be

several fold larger than the membrane time constant of individual neurons. Two types of

mechanisms have been proposed to explain this conundrum. One possibility is that large

timescales are generated by a network mechanism based on positive feedback, but this

hypothesis requires fine-tuning of the strength or structure of the synaptic connections. A

second possibility is that large timescales in the neural dynamics are inherited from large

timescales of underlying biophysical processes, two prominent candidates being intrinsic

adaptive ionic currents and synaptic transmission. How the timescales of adaptation or syn-

aptic transmission influence the timescale of the network dynamics has however not been

fully explored. To address this question, here we analyze large networks of randomly con-

nected excitatory and inhibitory units with additional degrees of freedom that correspond to

adaptation or synaptic filtering. We determine the fixed points of the systems, their stability

to perturbations and the corresponding dynamical timescales. Furthermore, we apply

dynamical mean field theory to study the temporal statistics of the activity in the fluctuating

regime, and examine how the adaptation and synaptic timescales transfer from individual

units to the whole population. Our overarching finding is that synaptic filtering and adaptation

in single neurons have very different effects at the network level. Unexpectedly, the macro-

scopic network dynamics do not inherit the large timescale present in adaptive currents. In

contrast, the timescales of network activity increase proportionally to the time constant of

the synaptic filter. Altogether, our study demonstrates that the timescales of different bio-

physical processes have different effects on the network level, so that the slow processes

within individual neurons do not necessarily induce slow activity in large recurrent neural

networks.

Author summary

Brain activity spans a wide range of timescales, as it is required to interact in complex

time-varying environments. However, individual neurons are primarily fast devices: their

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 1 / 33

a1111111111

a1111111111

a1111111111

a1111111111

a1111111111

OPEN ACCESS

Citation: Beiran M, Ostojic S (2019) Contrasting

the effects of adaptation and synaptic filtering on

the timescales of dynamics in recurrent networks.

PLoS Comput Biol 15(3): e1006893. https://doi.

org/10.1371/journal.pcbi.1006893

Editor: Peter E. Latham, UCL, UNITED KINGDOM

Received: December 19, 2018

Accepted: February 19, 2019

Published: March 21, 2019

Copyright: ©2019 Beiran, Ostojic. This is an open

access article distributed under the terms of the

Creative Commons Attribution License, which

permits unrestricted use, distribution, and

reproduction in any medium, provided the original

author and source are credited.

Data Availability Statement: All relevant data are

within the manuscript and its Supporting

Information files. The codes that generate the

figures are available at https://github.com/

emebeiran/adapt_synapt.

Funding: This work was funded by the Ecole des

Neurosciences de Paris Ile-de-France, the

Programme Emergences of City of Paris, Agence

Nationale de la Recherche grants ANR-16-CE37-

0016-01 and ANR-17-ERC2-0005-01, and the

program “Investissements d’Avenir” launched by

the French Government and implemented by the

ANR, with the references ANR-10-LABX-0087 IEC

membrane time constant is of the order of a few tens of milliseconds. Yet, neurons are

also subject to additional biophysical processes, such as adaptive currents or synaptic fil-

tering, that introduce slower dynamics in the activity of individual neurons. In this study,

we explore the possibility that slow network dynamics arise from such slow biophysical

processes. To do so, we determine the different dynamical properties of large networks of

randomly connected excitatory and inhibitory units which include an internal degree of

freedom that corresponds to either adaptation or synaptic filtering. We show that the net-

work dynamics do not inherit the slow timescale present in adaptive currents, while syn-

aptic filtering is an efficient mechanism to scale down the timescale of the network

activity.

Introduction

Adaptive behavior requires processing information over a vast span of timescales [1], ranging

from micro-seconds for acoustic localisation [2], milliseconds for detecting changes in the

visual field [3], seconds for evidence integration [4] and working memory [5], to hours, days

or years in the case of long-term memory. Neural activity in the brain is matched to the

computational requirements imposed by behavior, and consequently displays dynamics over a

similarly vast range of timescales [6–8]. Since the membrane time constant of an isolated neu-

ron is of the order of tens of milliseconds, the origin of the long timescales observed in the neu-

ral activity has been an outstanding puzzle.

Two broad classes of mechanisms have been proposed to account for the existence of long

timescales in the neural activity. The first class relies on non-linear collective dynamics that

emerge from synaptic interactions between neurons in the local network. Such mechanisms

have been proposed to model a variety of phenomena that include working memory [9], deci-

sion-making [10] and slow variability in the cortex [11]. In those models, long timescales

emerge close to bifurcations between different types of dynamical states, and therefore typi-

cally rely on the fine tuning of some parameter [12]. An alternative class of mechanisms posits

that long timescales are directly inherited from long time constants that exist within individual

neurons, at the level of hidden internal states [13]. Indeed biophysical processes at the cellular

and synaptic level display a rich repertoire of timescales. These include short-term plasticity

that functions at the range of hundreds of milliseconds [14,15], a variety of synaptic channels

with timescales from tens to hundreds of milliseconds [16–19], ion channel kinetics imple-

menting adaptive phenomena [20], calcium dynamics [21] or shifts in ionic reversal potentials

[22]. How the timescales of these internal processes affect the timescales of activity at the net-

work level has however not been fully explored.

In this study, we focus on adaptative ion-channel currents, which are known to exhibit

timescales over several orders of magnitude [23–25]. We contrast their effects on recurrent

network dynamics with the effect of the temporal filtering of inputs through synaptic cur-

rents, which also expands over a large range of timescales [26]. To this end, we extend classi-

cal rate models [27–30] of randomly connected recurrent networks by including for each

individual unit a hidden variable that corresponds to either the adapting of the synaptic cur-

rent. We systematically determine the types of collective activity that emerge in such net-

works. We then compare the timescales on the level of individual units with the activity

within the network.

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 2 / 33

and ANR11-IDEX-0001-02 PSLResearch

University. The funders had no role in study

design, data collection and analysis, decision to

publish, or preparation of the manuscript.

Competing interests: The authors have declared

that no competing interests exist.

Results

We consider Ncoupled inhibitory and excitatory units whose dynamics are given by two vari-

ables: the input current x

i

and a slow variable s

i

or w

i

that accounts for the synaptic filtering or

adaptation current respectively. The instantaneous firing rate of each neuron is obtained by

applying a static non-linearity ϕ(x) to the input current at every point in time. For simplicity,

we use a positive and bounded threshold-linear transfer function

xð Þ ¼ ½xgþif xg< max

max otherwise;

8

<

:ð1Þ

where []

+

indicates the positive part, γis the activation threshold and ϕ

max

the maximum fir-

ing rate.

Single neuron adaptation is described by the variable w(t) that low-pass filters the linearized

firing rate with a timescale τ

w

, slower than the membrane time constant τ

m

, and feeds it back

with opposite sign into the input current dynamics (see Methods). The dynamics of the i-th

adaptive neuron are given by

tm_

xiðtÞ ¼ xiðtÞ þ PN

j¼1JijðxjðtÞÞ gwwiðtÞ þ IiðtÞ

tw_

wiðtÞ ¼ wiðtÞ þ xiðtÞ g;

(ð2Þ

where I

i

(t) is the external input current to neuron i.

Synaptic filtering consists in low-pass filtering the synaptic input received by a cell with

time constant τ

s

, before it contributes to the input current. The dynamics of the i-th neuron in

a network with synaptic filtering are

tm_

xiðtÞ ¼ xiðtÞ þ siðtÞ

ts_

siðtÞ ¼ siðtÞ þ PN

j¼1JijðxjðtÞÞ þ IiðtÞ:

(ð3Þ

The matrix element J

ij

corresponds to the synaptic coupling strength from neuron jonto

neuron i. In this study we focus on neuronal populations of inhibitory and excitatory units,

whose connectivity is sparse, random, with constant in-degree: all neurons receive exactly the

same number of excitatory and inhibitory connections, C

E

and C

I

, as in [31–33]. All excitatory

synapses have equal strength Jand all inhibitory neurons −gJ. Furthermore, we consider the

large network limit where the number of synaptic neurons Nis large while keeping the excit-

atory and inhibitory inputs C

E

and C

I

fixed.

Single unit: Timescales of dynamics

In the models studied here the input current of individual neurons is described by a linear sys-

tem. Thus, their activity is fully characterized by the response h(t) to a brief impulse signal, i.e.

the linear filter. When such neurons are stimulated with a time-varying input I(t), the response

is the convolution of the filter with the input, x(t) = (hI)(t). These filters can be determined

analytically for both neurons with adaptation or synaptic filtering and directly depend on the

parameters of these processes. Analyzing the differences that these two slow processes produce

in the linear filters is useful for studying the differences in the response of adaptive and synap-

tic filtering neurons to temporal stimuli (Fig 1A), and will serve as a reference for comparison

to the effects that emerge at the network level.

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 3 / 33

In particular, the filter of a neuron with synaptic filtering, h

s

(t), is the sum of two exponen-

tially decaying filters of opposite signs and equal amplitude, with time constants τ

s

and τ

m

:

hstð Þ ¼ 1

tstm

et

tset

tm

Ytð Þ;ð4Þ

where Θ(t) is the Heaviside function (see Methods). Thus, the current response of a neuron to

an input pulse received from an excitatory presynaptic neuron is positive and determined by

two different timescales. The response first grows with timescale τ

m

, so that the neuron cannot

respond to any abrupt changes in the synaptic input faster than this timescale, and then

decreases back to zero with timescale τ

s

(grey curves, Fig 1B).

The adaptation filter is given as well by the linear combination of two exponential func-

tions. In contrast to the synaptic filter, since the input in the adaptive neuron model affects

directly the current variable x

i

(t), there is an instantaneous change in the firing rate to an input

delta-function (red curves, Fig 1B). The timescales of the two exponentials can be calculated as

t¼2tmtw

twþtm

1ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

14tmtwð1þgwÞ

ðtmþtwÞ2

s

!1

:ð5Þ

When the argument of the square root in Eq (5) is negative, the two timescales correspond to a

pair of complex conjugate numbers, so that the filter is an oscillatory function whose ampli-

tude decreases monotonically to zero at a single timescale. If the argument of the square root is

positive, for slow enough adaptation, the two timescales are real numbers and correspond to

exponential functions of opposing signs of decaying amplitude. However, the amplitudes of

these two exponentials are different (see Methods). To illustrate this, we focus on the limit of

large adaptation time constants with respect to the membrane time constant, where the two

exponential functions evolve with timescales that decouple the contribution of the membrane

Fig 1. Activity of individual neurons with adaptation or synaptic filtering. A: Firing rate response of two different neurons with adaptation (red curves) and two

different neurons with synaptic filtering (grey curves) to the same time-varying input (black curve). B: Normalized linear filters for the neurons shownin A. C:

Timescales of the linear filter for neurons with adaptation (red lines) and for neurons with synaptic filtering (grey lines) as a function of the timescale τ

w

or τ

s

,

respectively. The dashed lines indicate the effective timescale of the evoked activity obtained by weighing each individual timescale with its amplitude in the linear

filter. The effective timescale for neurons with adaptation saturates for large adaptation time constants, while it grows proportionally to the synaptic time constant for

neurons with synaptic filtering. Note that for the adaptive neuron, if the two eigenvalues are complex conjugate, there is only one decay timescale. The triangles on the

temporal axis indicate the time constants used in A and B. Adaptation coupling g

w

= 5. D: Variance of the input current as a function of the slow time constant when

the adaptive and synaptic neurons are stimulated with Gaussian white noise of unit variance. In the case of neurons with adaptation, twodifferent values of the

adaptation coupling g

w

are shown. Time in units of the membrane time constant τ

m

.

https://doi.org/10.1371/journal.pcbi.1006893.g001

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 4 / 33

time constant and the adaptation current. In that limit, the adaptive filter reads

hwtð Þ ¼ gw

tw

e1þgw

ð Þ t

twþ1

tm

et

tm

Ytð Þ:ð6Þ

The amplitude of the slow exponential is inversely related to its timescale so that the integral of

this mode is fixed, and independent of the adaptation time constant. This implies that a sever-

alfold increase of the adaptation time constant does not lead to strong changes in the single

neuron activity for time-varying signals (Fig 1A).

Furthermore, we can characterize the timescale of the single neuron response as the sum of

the exponential decay timescales weighed by their relative amplitude, and study how this char-

acteristic timescale evolves as a function of the time constants of either the synaptic or the

adaptive current (Fig 1C). For adaptive neurons, the activity timescale is bounded as a conse-

quence of the decreasing amplitude of the slow mode, i.e. increasing the adaptation time con-

stant beyond a certain value will not lead to a slower response. In contrast, the activity of an

individual neuron with synaptic filtering scales proportionally to the synaptic filter time,

since the relative amplitudes of the two decaying exponentials are independent of the time

constants.

When any of the two neuron types are stimulated with white Gaussian noise, the variance

in the response is always smaller than the input variance, due to the low pass filtering proper-

ties of the neurons. However, this gain in the variance of the input currents is modulated by

the different neuron parameters (Fig 1D). For a neuron with synaptic filtering, the gain is

inversely proportional to the time constant τ

s

. In contrast, for a neuron with adaptation,

increasing the adaptation time constant has the opposite effect of increasing the variance of

the current response. This is because when the adaptation time constant increases, the ampli-

tude of the slow exponential decreases accordingly, and the low-pass filtering produced by this

slow component is weaker. Following the same reasoning, increasing the adaptation coupling

corresponds to strengthening the low-pass filtering performed by adaptation, so that the vari-

ance decreases (Fig 1D, dashed vs full red curves).

Population-averaged dynamics

In the absence of any external input, a non-trivial equilibrium for the population averaged

activity emerges due to the recurrent connectivity of the network. The equilibrium firing rate

is identical across network units, since all units are statistically equivalent. We can write the

input current x

0

at the fixed point as the solution to the transcendental equation

ð1þgwÞx0¼JðCEgCIÞðx0Þ þ gwg;ð7Þ

for the network with adaptation, and to

x0¼JðCEgCIÞðx0Þ;ð8Þ

for synaptic filtering (see Methods). Based on Eq (7), we find that the adaptation coupling g

w

reduces the mean firing rate of the network, independently of whether the network is domi-

nated by inhibition or excitation (Fig 2A). Synaptic filtering instead does not play any role in

determining the equilibrium activity of the neurons, since Eq (8) is independent of the synap-

tic filtering parameter τ

s

.

We next study the stability and dynamics of the equilibrium firing rate in response to a

small perturbation uniform across the network, x

i

(t) = x

0

+δx(t). Because of the fixed in-

degree of the connectivity matrix, the linearized dynamics of each neuron are identical, so that

the analysis of the homogeneous perturbation on the network reduces to the study of a two-

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 5 / 33

dimensional deterministic system of differential equations which corresponds to the dynamics

of the population-averaged response (see Methods). The stability and timescales around equi-

librium depend on the two eigenvalues of this linear 2D-system. More specifically, the fixed

point is stable to a homogeneous perturbation if the two eigenvalues of the dynamic system

have negative real part, in which case the inverse of the unsigned real part of the eigenvalues

determines the timescales of the response. For both the network with synaptic filtering and the

network with adaptive neurons, the order parameter of the connectivity that determines the

stability of the fixed point is the effective recurrent coupling J(C

E

−gC

I

) each neuron receives,

resulting from the sum of all input synaptic connections. A positive (negative) effective cou-

pling corresponds to a network where recurrent excitation (inhibition) dominates and the

recurrent input provides positive (negative) feedback [32,33].

For networks with synaptic filtering, we find that the synaptic time constant does not alter

the stability of the equilibrium state, so that the effective coupling alone determines the stabil-

ity of the population-averaged activity. As the effective input coupling strength is increased,

the system undergoes a saddle-node bifurcation when the effective input is J(C

E

−gC

I

) = 1 (Fig

2C). In other words, the strong positive feedback loop generated by the excitatory recurrent

connections destabilizes the system.

To analyze the timescales elicited by homogeneous perturbations, we calculate the eigenval-

ues and eigenvectors of the linearized dynamic system (see Methods). We find that for inhibi-

tion-dominated networks (J(C

E

−gC

I

)<0), the network shows population-averaged activity

at timescales that interpolate between the membrane time constant and the synaptic time con-

stant. As the effective coupling is increased, the slow timescale at the network level can be

made arbitrarily slow by tuning the effective synaptic coupling close to the bifurcation value, a

well-known network mechanism to achieve slow neural activity [12].

In the limit of very slow synaptic timescale, the two timescales of the population-averaged

activity are

tþ¼ts

1JðCEgCIÞ;ð9Þ

t¼tm1J CEgCI

ð Þ ts

tm

;ð10Þ

Fig 2. Equilibrium firing rate and phase diagrams of the population-averageddynamics. A: Firing rate of the network with adaptation at the

equilibrium ϕ(x

0

) for increasing adaptive couplings and three different values of the effective recurrent coupling J

eff

=J(C

E

−gC

I

). Stronger adaptation

leads to lower firing rates at equilibrium. B: Phase diagram of the population-averaged activity for the network with adaptation. C: Phase diagram for

the network with synaptic filtering.

https://doi.org/10.1371/journal.pcbi.1006893.g002

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 6 / 33

so that the timescale τ

−

is proportional to the membrane time constant and τ

+

is proportional

to the slow synaptic time constant, effectively decoupling the two timescales. The relative con-

tribution of these two timescales is the same, independently of the time constant τ

s

, as we

found in the single neuron analysis.

The network with adaptation shows different effects on the population-averaged activity.

First, the presence of adaptation modifies the region of stability: the system is stable when the

effective recurrent input J(C

E

−gC

I

) is less than the minimum of 1 + g

w

and 1þtm

tw(see Meth-

ods). Therefore, the stability region is larger than for the network with synaptic filtering (Fig

2B vs Fig 2C). In other words, the effective excitatory feedback required to destabilize the net-

work is larger due to the counterbalance provided by adaptation. Moreover, adaptation allows

the network to undergo two different types of bifurcations as the effective input strength

increases, depending on the adaptation parameters. One possibility is a saddle-node bifurca-

tion, as in the synaptic case, which takes place when J(C

E

−gC

I

) = 1 + g

w

. Beyond that instabil-

ity all neurons in the network saturate. The other possible bifurcation, which happens if

tm

tw<gw, at an effective coupling strength J CEgCI

ð Þ ¼ 1þtm

tw, is a Hopf bifurcation: the fixed

point of network becomes unstable, leading in general to oscillating dynamics of the popula-

tion-averaged response. Note that in the limit of very slow adaptation, the system can only

undergo a Hopf bifurcation (Fig 2B).

The two timescales of the population-averaged activity in the stable regime for the adaptive

network decouple the two single neuron time constants when adaptation is much slower than

the membrane time constant. In this limit, up to first order of the adaptive time ratio tm

tw, the

two activity timescales are

tþ¼tm

1JðCEgCIÞ;ð11Þ

t¼twð1JðCEþgCIÞÞ

1þgwJðCEgCIÞ:ð12Þ

Similar to the single neuron dynamics, the amplitude of the slow mode, corresponding to τ

−

,

decreases as τ

w

is increased, so that the contribution of the slow timescale is effectively reduced

when τ

w

is very large. On the contrary, the mode corresponding to τ

+

, proportional to the

membrane time constant can be tuned to reach arbitrarily large values. This network mecha-

nism to obtain slow dynamics does not depend on the adaptation properties.

Heterogeneous activity

Linear stability analysis. Previous studies have shown that random connectivity can lead

to heterogeneous dynamics where the activity of each unit fluctuates strongly in time [29,33–

35]. To assess the effects of additional hidden degrees of freedom on the emergence and time-

scales of such fluctuating activity, we examine the dynamics when each unit is perturbed inde-

pendently away from the equilibrium, x

i

(t) = x

0

+δx

i

(t). By linearizing the full 2N-dimensional

dynamics around the fixed point, we can study the stability and timescales of the activity char-

acterized by the set of eigenvalues of the linearized system, λ

s

and λ

w

for the network with syn-

aptic filtering neurons and adaptation, respectively. These sets of eigenvalues are determined

by a direct mapping to the eigenvalues of the connectivity matrix, λ

J

(see Methods). The eigen-

values λ

J

of the connectivity matrices considered are known in the limit of large networks [33,

36]: they are enclosed in a circle of radius Jﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

CEþg2CI

p, except for an outlier that corresponds

to the population-averaged dynamics, studied in the previous section. Therefore, we can map

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 7 / 33

the circle that encloses the eigenspectrum λ

J

into a different shape in the space of eigenvalues

λ

s/w

(insets Fig 3). In order to determine the stability of the response to the perturbation, we

assess whether the real part of the eigenspectrum λ

s/w

is negative at all possible points. Further-

more, the type of bifurcation is determined by whether the curve enclosing the eigenvalues λ

s,w

crosses the imaginary axis at zero frequency or at a finite frequency when the synaptic coupling

strength is increased, leading respectively to a zero-frequency or to a Hopf bifurcation [37].

The order parameter of the connectivity that affects the stability and dynamics of the net-

work is now the radius of the circle of eigenvalues λ

J

, i.e. Jﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

ðCEþg2CIÞ

p. This parameter is the

standard deviation of the synaptic input weights of a neuron (see Methods), which contrasts

with the order parameter of the population-averaged response, that depends on the mean of

the synaptic input weights. The mean and standard deviation of the synaptic connectivity can

be chosen independently, so that while the population-averaged activity remains stable, the

individual neurons might not display stable dynamics. To analyze solely the heterogeneous

response of the network to the perturbation, we focus in the following on network connectivi-

ties whose population-averaged activity is stable, i.e. the effective synaptic coupling is inhibi-

tory or weakly excitatory.

Fig 3. Dynamical regimes as the coupling strength is increased. Numerical integration of the dynamics for the network with adaptive neurons (row A)

and the network with synaptic filtering (row B), as the coupling standard deviation Jcs ¼Jﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

CEþg2CI

pis increased. Colored lines correspond to the firing

rates of individual neurons, the black line indicates the population average activity. Insets: complex eigenspectrum λ

w/s

of the linearized dynamical matrix

around the fixed point. Dots: eigenvalues of the connectivity matrix used in the network simulation. Solid line: theoretical prediction for the envelope of

the eigenspectrum. The imaginary axis, Re(λ) = 0, is the stability boundary. i. Both the network with adaptation and synaptic transmission are stable. ii.

The network with synaptic filtering crosses the stability boundary and shows fluctuations in time and across neurons, while the network with adaptation

remains stable. iii. The network with synaptic filtering displays stronger fluctuations. The network with adaptive neurons undergoes a Hopf bifurcation

leading to strong oscillations at a single frequency with uncorrelated phases across units. Note in the inset that for this connectivity matrix there is only one

pair of complex conjugate unstable eigenvalues in the finite network. iv. The network with synaptic filtering shows strong fluctuations. The network with

adaptation displays fluctuating activity with an oscillatory component. Parameters: in A, g

w

= 0.5, and τ

w

= 5, in B, τ

s

= 5.

https://doi.org/10.1371/journal.pcbi.1006893.g003

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 8 / 33

We find that in the network with synaptic filtering, the eigenspectrum λ

s

always crosses the

stability bound through the real axis, which takes place when the spectral radius of the connec-

tivity is one, Jﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

CEþg2CI

p¼1. Thus the system undergoes a zero-frequency bifurcation simi-

lar to randomly connected networks without hidden variables [29,33,35,38], leading to

strong fluctuations at the single neuron level that are self-sustained by the network connectiv-

ity (Fig 3Bii–3Biv). The critical coupling at which the equilibrium firing rate loses stability is

independent of the synaptic time constant, i.e. synaptic filtering does not affect the stability of

heterogeneous responses (Fig 4A). However, the synaptic time constant τ

s

affects the time-

scales at which the system returns to equilibrium after a perturbation, because the eigenvalues

λ

s

(see Eq (69) in Methods) depend explicitly on τ

s

.

For a network with adaptive neurons, we calculate the eigenspectrum λ

w

and find that the

transition to instability Re(λ

w

) = 0 can happen either at zero frequency or at a finite frequency

(see Methods), leading to a Hopf bifurcation (as in inset Fig 3Aiii). In particular, the network

dynamics undergo a Hopf bifurcation when

tw>tm

gwþﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

2gwðgwþ1Þ

p;ð13Þ

so that strong adaptation coupling and slow adaptation time constants lead to a finite fre-

quency bifurcation. In particular, if the coupling g

w

is larger than ﬃﬃﬃ

5

p20:236, only the

Hopf bifurcation is possible, since by construction tm

tw<1. We can also calculate the frequency

of oscillations at the Hopf bifurcation. We find that, for slow adaptive currents, the Hopf fre-

quency is inversely related to the adaptation time constant (Fig 4B), so that slower adaptation

currents produce slower oscillations at the bifurcation.

Adaptation also increases the stability of the equilibrium firing rate to a heterogeneous per-

turbation, in comparison to a network with synaptic filtering (Fig 4C). This can be intuitively

explained in geometrical terms by analyzing how adaptation modifies the shape of the eigen-

spectrum λ

w

with respect to the circular eigenspectrum of the connectivity matrix λ

J

.

The Hopf bifurcation leads to the emergence of a new dynamical regime in the network

(Fig 3Aiv), which is studied in the following section. Right at the Hopf bifurcation, the system

shows marginal oscillations at a single frequency that can be reproduced in finite-size simula-

tions whenever only one pair of complex conjugate eigenvalues is unstable (Fig 3Aiii).

Fig 4. Phase diagram and frequency of the bifurcation for the heterogeneous activity. A: Phase diagram for the network with synaptic transmission. The only

relevant parameter to assess the dynamical regime is the connectivity strength. The circles indicate the parameters used in Figs 3and 6. Triangles correspond to the

parameter combinations used in Fig 5. B: Frequency at which the eigenspectrum loses stability for the network with adaptive neurons as a function of the ratio between

membrane and adaptation time constant, τ

m

/τ

w

, for three different adaptive couplings. The dots indicate the fastest adaptive time constant for which the system

undergoes a Hopf bifurcation (Eq 84). C: Phase diagrams for the two adaptation parameters, (i) the coupling g

w

and (ii) the adaptive time constant τ

w

vs the coupling

standard deviation.

https://doi.org/10.1371/journal.pcbi.1006893.g004

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 9 / 33

Fluctuating activity: Dynamical mean field theory. The classical tools of linear stability

theory applied so far can only describe the dynamics of the system up to the bifurcation. To

study the fluctuating regime, we take a different approach and focus on the temporal statistics

of the activity, averaged over different connectivity matrices: we determine the mean and auto-

correlation function of the single neuron firing rate, and characterize the timescale of the fluc-

tuating dynamics [29,33–35,38–40]. For large networks, the dynamics can be statistically

described by applying dynamical mean field theory (DMFT), which approximates the deter-

ministic input to each unit by an independent Gaussian noise process. The full network is then

reduced to a two-dimensional stochastic differential equation, where the first and second

moments of the noise must be calculated self-consistently. We solve the self-consistent equa-

tions using a numerical iterative procedure, similar to the schemes followed in [34,41–44] (see

Methods for an explanation of the iterative algorithm and its practical limitations).

For the network with synaptic filtering, we find that the autocorrelation function of the fir-

ing rates in the fluctuating regime corresponds to a monotonically decreasing function (Fig

5A), qualitatively similar to the correlation obtained in absence of synaptic filtering [33]. This

fluctuating state has often been referred to as rate chaos and shows non-periodical heteroge-

neous activity which is intrinsically generated by the network connectivity. The main effect of

synaptic filtering is on the timescale of these fluctuations. When the synaptic time constant is

much larger than the membrane time constant, the timescale of the network activity is propor-

tional to the synaptic time constant τ

s

, as indicated by the linear dependence between the half-

width of the autocorrelation function and the synaptic timescale τ

s

, when all other network

parameters are fixed (Fig 5B).

For the network with adaptation, we focus on large adaptation time constant τ

w

, where the

network dynamics always undergo a Hopf bifurcation. The autocorrelation function in such a

case displays damped oscillations (Fig 5C). The decay in the envelope of the autocorrelation

function is due to the chaotic-like fluctuations of the firing rate activity.

We define the time lag at which the envelope of the autocorrelation function decreases as

the timescale of the network dynamics (see Methods). The timescale of the activity increases as

the adaptation timescale is increased, when all the other parameters are fixed (Fig 5D). How-

ever, this activity timescale saturates for large values of the adaptation timescale: the presence

of very slow adaptive currents, beyond a certain value, will not slow down strongly the network

activity. This saturation value depends on the connectivity strength.

Fig 5. Autocorrelation function and timescale of the network activity in the fluctuating regime. A: Autocorrelation function of the firing rates in the network

with synaptic filtering; dynamical mean field results (solid lines) with their corresponding envelopes (dashed lines), and results from simulations (empty dots).

Connectivity strength Jcs ¼Jﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

CEþg2CI

p¼1:2. B: Effective timescale of the network activity as a function of the synaptic time constant for the network with

synaptic filtering. The network coupling does not have a strong effect on the effective timescale. C: Autocorrelation function of the firing rates, as in A, for the system

with adaptive neurons. J

cs

= 1.3. D: Effective timescale of the firing rates, as in B, for the system with adaptive currents.

https://doi.org/10.1371/journal.pcbi.1006893.g005

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 10 / 33

Effects of noise. The networks studied so far, for a fixed connectivity matrix, are

completely deterministic. We next study the effects of additional white noise inputs to each

neuron, as a proxy towards understanding recurrent networks of spiking neurons with adapta-

tion and synaptic filtering. On the mean-field level, such noise is equivalent to studying a

recurrent network whose neurons fire action potentials as a Poisson process with instanta-

neous firing rate ϕ(x

i

(t)) [35,45].

Numerical simulations show that in the stable regime the additive external noise generates

weak, fast stationary dynamics around the fixed point (Fig 6Ai and 6Bi). The timescale of these

fluctuations and their amplitude depend on the distance of the eigenspectrum to the stability

line, so that the stable fluctuations for weak synaptic coupling standard deviation (Fig 6Ai) are

smaller in amplitude than those for larger coupling standard deviation (Fig 6Aii), whose eigen-

spectrum is closer to the stability boundary. For adaptation, in the fluctuating regime beyond

the Hopf bifurcation, the network activity shows again a combination of fluctuating activity

and oscillations.

We further extend the DMFT analysis to account for the additional variance of the external

white noise sources (see Methods). The autocorrelation function of the firing rates, as pre-

dicted by DMFT, does not vary drastically when weak noise is added to the network, except

Fig 6. Dynamical regimes for the network with adaptation or synaptic filtering with additive external noise. Numerical integration of the

dynamics with units receiving additive external white noise, as a proxy for spiking noise. A: Network with adaptive neurons. B: Network with synaptic

filtering. Colored lines correspond to the firing rate of individual neurons, the black line indicates the population average activity. Insets: complex

eigenspectrum λ

w/s

of the dynamic matrix at the fixed point. Dots: eigenvalues of the connectivity matrix used in the network simulation. Solid line:

theoretical prediction for the envelope of the eigenspectrum. i. Both the network with adaptation and synaptic transmission are stable, the external

noise generates stationary fluctuations around the fixed point. ii. The network with synaptic filtering undergoes a zero-frequency bifurcation. Noise

adds fast temporal variability in the firing rates. The network with adaptation remains stable, and the fluctuations are larger in amplitude. iii. The

network with adaptation undergoes a Hopf bifurcation. The firing rate activity combines the fast fluctuations produced by white noise and the chaotic

activity with an oscillatory component. iv. The network with adaptation shows highly irregular activity, and strong effects due to the activation and

saturation bounds of the transfer function. Parameters as in Fig 4, external noise σ

η

= 0.06.

https://doi.org/10.1371/journal.pcbi.1006893.g006

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 11 / 33

for very short time lags, at which white noise introduces fast fluctuations (see Fig 7). For the

network with adaptation, the autocorrelation function of the firing rates still shows damped

oscillations (Fig 7A), while for the network with synaptic filtering, similarly, weak noise does

not affect much the decay of the autocorrelation function (Fig 7D). Very strong external noise

on the other hand will reduce the effect of the underlying recurrent dynamics of the rate net-

work, since the signal to noise ratio in the synaptic input of all neurons is low.

For a fixed external noise intensity, reducing the adaptation coupling or increasing the

adaptation time constant increases the variance of the firing rate (Fig 7B), which resembles the

dependence of the variance gain for individual neurons (Fig 1D). Conversely, slower synaptic

filtering reduces the variance of the neuron’s firing rates. This is because in the network with

synaptic filtering the noise is also filtered at the synapses –in the limit of very large τ

s

, the

whole white noise is filtered out– whereas in the network with adaptation the noise affects

directly the input current, without being first processed by the adaptation variable.

However, the timescale of the activity is nonetheless drastically affected by strong noise.

External noise adds fast fluctuations on top of the intrinsically generated dynamics of the het-

erogeneous network with adaptation or synaptic filtering. If the noise is too strong, the effec-

tive timescale of the activity takes into account mostly this fast component. In that limit, the

Fig 7. Autocorrelation function, variance of the firing rates and timescale of the network activity with external noise predicted by

dynamical mean field theory. A: Autocorrelation function of the firing rates for the network with adaptive neurons for three different noise

intensities. Adaptation time constant τ

w

= 1.25. B: Variance of the firing rate as a function of the adaptation time constant for two different

adaption couplings g

w

. Increasing the adaptation time constant or decreasing the adaptation coupling increasesthe variance. σ

η

= 0.15. C:

Timescale of the firing rate as a function of the adaptation time constant, and three different noise levels. Parameters: g

w

= 0.5, and

Jﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

CEþg2CI

p¼1:2. D: Autocorrelation function of the firing rate for the network with synaptic transmission for three different noise levels.

Synaptic time constant τ

s

= 1.25. E: Variance of the firing rate as a function of the synaptic time constant, for three different external noise levels.

Synaptic filtering reduces the variance. F: Timescale of the activity for the network with synaptic filtering and external noise.

https://doi.org/10.1371/journal.pcbi.1006893.g007

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 12 / 33

timescale of the activity is almost independent of the synaptic or adaptive time constants (Fig

7C and 7F, largest noise intensity).

Discussion

We examined dynamics of excitatory-inhibitory networks in which each unit had a hidden

degree of freedom that represented either firing-rate adaptation or synaptic filtering. The core

difference between adaptation and synaptic filtering was how external inputs reached the sin-

gle-unit activation variable that represents the membrane potential. In the case of adaptation,

the inputs directly entered the activation variable, which was then filtered by the hidden, adap-

tive variable through a negative feedback loop. In the case of synaptic filtering, the external

inputs instead reached first the hidden, synaptic variable and were therefore low-pass filtered

before being propagated in a feed-forward fashion to the activation variable. While both mech-

anisms introduce a second timescale in addition to the membrane time constant, our main

finding is that the interplay between those two timescales is very different in the two situations.

Surprisingly, in presence of adaptation, the membrane timescale remains the dominant one in

the dynamics, while the contribution of the adaptation timescale appears to be weak. In con-

trast, in a network with synaptic filtering, the dominant timescale of the dynamics is directly

set by the synaptic variable, and the overall dynamics are essentially equivalent to a network in

which the membrane time-constant is replaced with the synaptic one.

We used a highly abstracted model, in which each neuron is represented by membrane cur-

rent that is directly transformed into a firing-rate through a non-linear transfer function. This

class of models has been popular for dissecting dynamics in excitatory-inhibitory [27,28,46–

48] or randomly-connected networks [29,30,33], and for implementing computations [49,

50]. Effects of adaptation in this framework have to our knowledge not been examined so far,

but see [51] for a simultaneously and independently developed study of adaptation in networks

of multidimensional rate units with random Gaussian connectivity. We therefore extended the

standard rate networks by introducing adaptation in an equally abstract fashion [24], as a hid-

den variable specified solely by a time constant and a coupling strength. Different values of

those parameters can be interpreted as corresponding to different specific membrane conduc-

tances that implement adaptation, e.g. the calcium dependent potassium I

ahp

current or the

slow voltage-dependent potassium current I

m

, which are known to exhibit timescales over sev-

eral orders of magnitude [52,53]. To cover the large range of adaptation timescales observed

in experiments [23], it would be straightforward to superpose several hidden variables with dif-

ferent time constants. Our approach could also be easily extended to include simultaneously

adaptation and synaptic filtering.

A number of previous works have studied the effects of adaptation within more biologically

constrained, integrate-and-fire models. These works have in particular examined the effects of

adaptation on the spiking statistics [54–56], firing-rate response [57,58], synchronisation [25,

56,59–61], perceptual bistability [62] or single-neuron coding [63,64]. In contrast, we have

focused here on the relation between the timescales of adaptation and those of network

dynamics. While our results rely on a simplified firing-rate model, we expect that they can be

directly related to networks of spiking neurons by exploiting quantitative techniques for map-

ping adaptive integrate-and-fire models to effective firing rate descriptions [65].

A side result of our analysis is the finding that strong coupling in random recurrent net-

works with adaptation generically leads to a novel dynamical state, in which individual units

exhibit a mixture of oscillatory and strong temporal fluctuations. The characteristic signature

of this dynamical state is a damped oscillation found in the auto-correlation function of single-

unit activity. In contrast, classical randomly connected networks lead to a fluctuating, chaotic

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 13 / 33

state in which the auto-correlation function decays monotonically [29,33–35]. Note that the

oscillatory activity of different units is totally out of phase, so that no oscillation is seen at the

level of population activity. This dynamical phenomenon is analogous to heterogeneous oscil-

lations in anti-symmetrically connected networks with delays [37]. In both cases, the oscil-

latory dynamics emerge through a bifurcation in which a continuum of eigenvalues crosses

the instability line at a finite-frequency. Similar dynamics can be also found in networks in

which the connectivity is a superposition of a random and a rank two structured part [33]. In

that situation, the heterogeneous oscillations however originate from a Hopf bifurcation due

to an isolated pair of eigenvalues that correspond to the structured part of the connectivity.

Our main aim here was to determine how hidden variables could induce long timescales in

randomly-connected networks. Long timescales could alternatively emerge from non-random

connectivity structure. As extensively investigated in earlier works, one general class of mecha-

nism relies on setting the connectivity parameters close to a bifurcation that induces arbitrarily

long timescales [12,29]. Another possibility is that non-random features of the connectivity,

such as the over-representation of reciprocal connections [66,67] slow down the dynamics

away from any bifurcation. A recent study [68] has indeed found such a slowing-down. Weak

connectivity structures of low-rank type provide yet another mechanism for the emergence of

long timescales. Indeed, rank-two networks can generate slow manifolds corresponding to

ring attractors provided a weak amount of symmetry is present [69].

Ultimately, the main reason for looking for long timescales in the dynamics is their poten-

tial role in computations performed by recurrent networks [70,71]. Recent works have pro-

posed that adaptive currents may help implement computations in spiking networks by either

introducing slow timescales or reducing the amount of noise due to spiking [72,73]. Our

results suggest that synaptic filtering is a much more efficient mechanism to this end than

adaptation. Identifying a clear computational role for adaptation in recurrent networks there-

fore remains an open and puzzling question.

Methods

Network model

We compare the dynamics of two different models: a recurrent network with adaptive neu-

rons, and a recurrent network with synaptic filtering. Each model is defined as a set of 2Ncou-

pled differential equations. The state of the i-th neuron is determined by two different

variables, the input current x

i

(t) and the adaptation (synaptic) variable w

i

(t) (s

i

(t)).

Adaptation. The dynamics of the recurrent network with adaptive neurons are given by

tm_

xiðtÞ ¼ xiðtÞ gwwiðtÞ þ IiðtÞ

tw_

wiðtÞ ¼ wiðtÞ þ ðxiðtÞÞ;

(ð14Þ

where ϕ(x) is a monotonically increasing non-linear function that transforms the input current

into firing rate. In this study, we use a threshold-linear transfer function with saturation:

xð Þ ¼ ½xgþif xg< max

max otherwise:

8

<

:ð15Þ

In Eq (14) adaptation in single neuron rate models is defined as a low-pass filtered version

with timescale τ

w

of the neuron’s firing rate ϕ(x

i

(t)), and is fed back negatively into the input

current, with a strength that we call the adaptation coupling g

w

. For the sake of mathematical

tractability, we linearize the dynamics of the adaptation variable by linearizing the transfer

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 14 / 33

function (Eq 15), ϕ(x

i

(t)) x

i

(t)−γ. Therefore, the dynamics of the network model with

adaptation studied here read

tm_

xiðtÞ ¼ xiðtÞ gwwiðtÞ þ IiðtÞ

tw_

wiðtÞ ¼ wiðtÞ þ xiðtÞ g;

(ð16Þ

Note that this approximation allows for adaptation to increase the input current of a neu-

ron, when the neuron’s current is below the activation threshold γ.

Synaptic filtering. For the recurrent network with synaptic filtering, the dynamics are

tm_

xiðtÞ ¼ xiðtÞ þ siðtÞ þ IiðtÞ

ts_

siðtÞ ¼ siðtÞ þ IiðtÞ:

(ð17Þ

In Eqs (14), (16) and (17), I(t) represents the total external input received by the neuron. In

general, we are interested in the internally generated dynamical regimes of the network, so

that the input is given by the synaptic inputs

IiðtÞ ¼ Isyn;i¼X

j

JijðxjðtÞÞ:ð18Þ

The matrix element J

ij

indicates the coupling strength of the j-th neuron onto the i-th neuron.

The connectivity matrix is sparse and random, with constant in-degree [32,33,74]: all neurons

receive the same number of input connections C, from which C

E

are excitatory and C

I

inhibi-

tory. All excitatory synapses have coupling strength Jwhile the strength of all inhibitory synap-

ses is −gJ. Moreover, each neuron can only either excite or inhibit the rest of the units in the

network, following Dale’s principle. Therefore, the total effective input coupling strength,

which is the same for all neurons, is

Jeff ≔X

j

Jij ¼JðCEgCIÞ:ð19Þ

We used the parameters in Table 1 for all figures unless otherwise specified.

Single neuron dynamics

The dynamics of each individual neuron are described by a two-dimensional linear system,

which implies that the input current response x(t) to a time-dependent input I(t) is the convo-

lution of the input with a linear filter h(τ) that depends on the parameters of the linear system:

xðtÞ ¼ ðhIÞðtÞ ¼ Zþ1

1

dt0hðt0ÞIðtt0Þ:ð20Þ

Table 1. Parameter values used in the simulations.

Parameter Value

Number of units N3000

In-degree C100

Excitatory inputs C

E

80

Inhibitory inputs C

I

20

Ratio I-E coupling strength g4.1

Threshold γ-0.5

Maximum firing rate ϕ

max

2

https://doi.org/10.1371/journal.pcbi.1006893.t001

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 15 / 33

In general, for any linear dynamic system _

zðtÞ ¼ Az þbðtÞ, where Ais a square matrix in

RNNand b(t) is a N-dimensional vector, the dynamics are given by

zðtÞ ¼ Z1

1

dt0eAt0Yðt0Þbðtt0Þ;ð21Þ

where Θ(t) is the Heaviside function. Thus, comparing Eqs (21) and (20), the linear filter is

determined by the elements of the so-called propagator matrix P(t) = e

At

Θ(t).

Synaptic filtering. For a single neuron wit synaptic filtering, the dynamics are given by

Eq (17), where the input I

i

(t) represents the external current. We write the response in its vec-

tor form (x(t), s(t))

T

and the input as (0, I(t))

T

. The dynamic matrix is

As¼t1

mt1

m

0t1

s

!:ð22Þ

The linear filter, h

s

(t0), is given by the entries of the propagator matrix that links the input I

(t) to the output element x(t), which are in this case only the entry in row one and column two:

h

s

(t0) = [P(t0)]

12

. To compute the required entry of the propagator, we diagonalize the dynamic

matrix A=VDV

−1

. The matrix Dis a diagonal matrix with the eigenvalues of matrix Ain the

diagonal entries, and Vis a matrix whose columns are the corresponding eigenvectors. Apply-

ing the identity etVDV1¼VetD V1and the definition of propagator we obtain that

hstð Þ ¼ Ytð Þ 1

tmts

et

tmet

ts

:ð23Þ

The two timescales of the activity are defined by the inverse of the eigenvalues of the system,

which coincide with τ

m

and τ

s

. Every time a pulse is given to the neuron, both modes get acti-

vated with equal amplitude and opposing signs, as indicated by Eq 23. This means that there is

a fast ascending phase after a pulse, at a temporal scale τ

m

, and a decay towards zero with time-

scale τ

s

.

Adaptation. The dynamics of a single adaptive neuron are determined by Eq (16), where

I

i

(t) is the external input to the neuron. We apply the same procedure to determine the time-

scales of the response of an adaptive neuron to time-dependent perturbations. The dynamic

matrix for an adaptive neuron reads

Aw¼t1

mgwt1

m

t1

wt1

w

!:ð24Þ

Its eigenvalues are

l

w¼1

2t1

mt1

wﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

ðt1

mþt1

wÞ24ð1þgwÞt1

mt1

w

q

:ð25Þ

and the eigenvectors

x¼gw

tm

;1

21

tmþ1

twﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

1

tm1

tw

2

4gw

tmtw

s

0

@1

A

0

@1

A

T

:ð26Þ

The eigenvalues are complex if and only if g

w

>(4τ

m

τ

w

)

−1

(τ

w

−τ

m

)

2

, and in that case their real

part is 1

2tmtwtmþtw

ð Þ. As the adaptive time constant becomes slower, at a certain critical adap-

tation time constant both eigenvalues become real. We are interested in the behavior when the

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 16 / 33

adaptation time constant is large. The absolute value of the inverse of the eigenvalues deter-

mines the time constants of the dynamics. Therefore, for large τ

w

we can calculate the two real

eigenvalues to first order of t1

w

lþ

w¼ 1þgw

twþOt2

w

ð27Þ

l

w¼ t1

mþgwt1

wþOðt2

wÞ:ð28Þ

In this limit of slow adaptation, the time constant of one eigenmode is proportional to τ

w

,

whereas the second mode scales with τ

m

. We are interested in the amplitude of each mode

with respect to the other.

By explicitly calculating the first entry of the propagator matrix we obtain the adaptive filter

in terms of the eigenvectors and eigenvalues,

hwtð Þ ¼ 1

tm

1

xþ

1x

2x

1xþ

2

xþ

1x

2elþtx

1xþ

2elt

;ð29Þ

where we use the notation xþ

1to indicate the first component of the eigenvector associated to

the eigenvalue λ

+

. Approximating to leading order of t1

wthe eigenvectors in Eq (26), we obtain

the eigenvectors

x¼1

tmðgw;0ÞT1

twð0;gwÞT¼gw

1

tm

;1

tw

T

ð30Þ

xþ¼1

tmðgw;1ÞTþ1

twð0;1þgwÞT¼gw

tm

;1

tmþ1þgw

tw

T

:ð31Þ

Then, using Eqs (29), (30) and (31), we determine the linear filter:

hwtð Þ ¼ gw

tmð2gwþ1Þ tw

e1þgw

twtþ1

tm

11þgw

ð Þtm

tw

11þ2gw

ð Þtm

tw

e1

tmgw

tw

ð Þt:ð32Þ

Interestingly, in contrast with synaptic filtering, the amplitude of the two modes are not

equal. The amplitude of the slow mode (first term in Eq 32), whose timescale is proportional

to τ

w

, decays proportionally to t1

wwith respect to the fast mode, when τ

w

τ

m

(2g

w

+ 1).

Therefore, the area under the linear filter corresponding to this mode is independent of τ

w

for

very large adaptation time constants:

lim

tw!1 Z1

0

hþ

wtð Þdt ¼lim

tw!1

gwtw

tmðgwþ1Þð2gwþ1Þ ðgwþ1Þtw¼ gw

gwþ1:ð33Þ

It follows that, if the adaptation timescale is increased, its relative contribution to the activity

will decrease by the same factor, so that very slow adaptive currents will effectively be masked

by the fast mode.

Equilibrium activity

The two systems possess a non-trivial equilibrium state at which the input current of all units

stays constant. Since all units are statistically equivalent, the equilibrium activity is the same

for all units. For synaptic filtering, the input current at equilibrium is given by a transcendental

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 17 / 33

equation, that is obtained by setting to zero the left hand side of Eq (17):

x0¼JðCEgCIÞðx0Þ:ð34Þ

This equilibrium coincides with the fixed point of the system without synaptic filtering.

For adaption, instead, from Eq (16) we obtain that the equilibrium is determined by

x0¼1

1þgw

J CEgCI

ð Þx0

ð Þ þ gwgð Þ:ð35Þ

We further assume unless otherwise specified that the fixed point of the system is in the linear

regime of the transfer function, so that ϕ(x) = x−γ. In that case x

0

= (J(C

E

−gC

I

)−g

w

) (x

0

−

γ), so that larger adaptation coupling corresponds to weaker input currents, i.e. decreasing sta-

tionary firing rate. The adaptation time constant does not affect the fixed point.

Dynamics of homogeneous perturbations

We study the neuronal dynamics in response to a small perturbation uniform across the net-

work

xiðtÞ ¼ x0þdxðtÞ:ð36Þ

Synaptic filtering. Linearizing Eq 17 we obtain

tmd_

xiðtÞ ¼ dxðtÞ þ dsiðtÞ

tsd_

siðtÞ ¼ dsiðtÞ þ 0

0PjJijdxðtÞ;

(ð37Þ

where we use the notation 0

0≔dðxÞ

dx jx0. Because the perturbation δxin Eq (37) is independent

of j, using Eq (19) the dynamics for all units are equivalent to the population-averaged dynam-

ics and are given by

tmd_

xðtÞ ¼ dxðtÞ þ dsðtÞ

tsd_

sðtÞ ¼ dsðtÞ þ 0

0JðCEgCIÞdx:

(ð38Þ

From Eq (38) we can define the dynamic matrix

As¼1

tm

1 1

0

0J CEgCI

ð Þtm

tstm

ts

0

@1

A:ð39Þ

The only difference in the linearized dynamics of the population-averaged current with respect

to the single neuron dynamics (Eq 22) is the non-diagonal entry 0

0JðCEgCIÞ. When either

the derivative at the fixed point cancels, or when the total effective input is zero, the population

dynamics equals the dynamics of a single neuron. The eigenvalues of the population-averaged

dynamics are

l

s¼ tmþts

2tstmﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

tmts

2tstm

2

þJðCEgCIÞ

tmts

s:ð40Þ

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 18 / 33

and the eigenvectors

x

s¼ 1;tmts

2tstmﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

tmts

2tstm

2

þJðCEgCIÞ

tmts

s

0

@1

A

T

:ð41Þ

For very large synaptic time constants, the eigenvalues are approximated to leading order as

lþ

s¼JðCEgCIÞ 1

tsþOt2

s

ð42Þ

l

s¼ 1

tmJðCEgCIÞ

tsð43Þ

Approximating as well the eigenvectors to leading order, we obtain

xþ¼1

tm

;1

tm1JðCEgCIÞ

ts

T

ð44Þ

x¼1

tm

;JðCEgCIÞ

ts

T

ð45Þ

the filter of the linear response to weak homogeneous perturbations reads:

hsðtÞ ¼ 1

ts

x

1xþ

1

xþ

1x

2x

1xþ

2

eltelþt

ð46Þ

¼1

ts

tstmð1JðCEgCIÞÞ

tstmð12JðCEgCIÞÞ eltelþt

ð47Þ

Note that the amplitude of the two exponential terms is the same, independently of the effec-

tive coupling and time constants.

Adaptation. For the system with adaptive neurons, the linearized system reads

tmd_

xiðtÞ ¼ dxiðtÞ gwdwiðtÞðtÞ þ 0

0PjJijdxðtÞ

twd_

wiðtÞ ¼ dwiðtÞ þ dxðtÞ:

(ð48Þ

As for the network with synaptic filtering, the dynamics of the perturbation are equivalent for

each unit, so that we can write down the dynamic matrix for the population-averaged response

to homogeneous perturbations

Aw¼1

tm

1þ0

0JðCEgCIÞ gw

tm

twtm

tw

0

@1

A:ð49Þ

The difference with respect to the linear single neuron dynamics (Eq 48) is that the effective

recurrent coupling appears now in the first diagonal entry of the dynamic matrix.

When the fixed point is located within the linear range of the transfer function, the deriva-

tive is one, so that we do not further specify the factor 0

0in the following equations. Conse-

quently, the dynamics of the system to small perturbations do not depend on the exact value of

the fixed point, which does not hold for more general transfer functions.

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 19 / 33

The eigenvalues of the system read

l

w¼ 1Jeff

2tm1

2tw

1ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

1þ4tmðJeff 1gwÞ

twJeff 1tm

tw

2

v

u

u

t

0

B

@1

C

A;ð50Þ

with eigenvectors

x

w¼2gw;tm

twþJeff 1ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

tm

twJeff þ1

2

4tm

tw

gwJeff þ1ð Þ

s

0

@1

A

T

ð51Þ

In the limit of very slow adaptation, given that the two eigenvalues are real, they can be

approximated to leading order as

lþ

w¼1þtm

twðJðCEgCIÞ 1ÞþOt2

w

ð52Þ

l

w¼ 1

tw

1gw

JðCEgCIÞ 1

þOt2

w

ð53Þ

and the corresponding eigenvectors read

xþ

w¼1;1

Jeff 1

tm

tw

T

ð54Þ

x

w¼gw;Jeff 1þtm

tw

1gw

Jeff 1

T

:ð55Þ

Therefore, if the perturbation is stable (see next section) we can write down the corresponding

linear filter as

hwtð Þ ¼ 1

tm

Jeff 1þtm

tw1gw

Jeff

Jeff 1þtm

tw12gw

Jeff

elþ

wtgw

twðJeff 1Þ2þtmðJeff 12gwÞel

wt:ð56Þ

The area under the slow mode is again independent of the adaptation time constant in this

limit,

lim

tw!1 Z1

0

h

wtð Þdt ¼ gw

ðJeff 1ÞðJeff 1gwÞ:ð57Þ

Stability of homogeneous perturbations

The equilibrium point is stable when the real part of all eigenvalues is negative. Equivalently,

in a two dimensional system –as it is the case for the population-averaged dynamics–, the

dynamics are stable when the trace of the dynamic matrix is negative and the determinant

positive.

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 20 / 33

Synaptic filtering. In the system with synaptic filtering, the trace and determinant are

Trs¼ 1

tm1

tsð58Þ

Dets¼1JðCEgCIÞ

tmts

:ð59Þ

The trace is therefore always negative. The determinant is positive, and therefore the popula-

tion-averaged dynamics are stable, when the effective coupling J(C

E

−gC

I

) is smaller than

unity. In contrast, if the effective coupling is larger than unity, i.e. if positive feedback is too

strong, the equilibrium firing rate is unstable, so that any small perturbation to the equilibrium

firing rate will lead the system to a different state. Right at the critical effective coupling, one

eigenvalues is zero and the other one equals Tr

s

, implying that the population-averaged

dynamics undergo a saddle-node bifurcation. Beyond the bifurcation, the network reaches a

state where the firing rates of all neurons saturate.

Adaptation. In the adaptive population dynamics, the recurrent connectivity has a differ-

ent effect on the stability of the adaptive population dynamics. The trace and determinant of

the dynamic matrix are

Tr w¼ 1

tm1

twþt1

mJ CEgCI

ð Þ;ð60Þ

Det w¼ ðtmtwÞ1ð1JðCEgCIÞ þ gwÞ:ð61Þ

Both the timescale τ

w

and the strength g

w

of adaptation affect the trace and determinant of the

dynamic matrix, and therefore the stability. The system is unstable if the determinant is nega-

tive (one positive and one negative real eigenvalue) or if the determinant is positive and the

trace is positive. The determinant is negative, and therefore the system becomes unstable

through a saddle-node bifurcation, when J(C

E

−gC

I

)>1 + g

w

. Note that the adaptation

strength increases the stability of the system: a stronger positive feedback loop is required to

destabilize the fixed point, in comparison to the network with synaptic filtering. The determi-

nant and trace are positive if J(C

E

−gC

I

)<1 + g

w

but J CEgCI

ð Þ >1þtm

tw, respectively,

leading to a Hopf bifurcation: the system produces sustained marginal oscillations at the bifur-

cation in response to small perturbations around the fixed point. Beyond the Hopf bifurcation,

the oscillations are maintained in time, unless the system shows a fixed point when all neurons

saturate (x0¼1

1gwJ CEgCI

ð Þmax þgwgð Þ). This fixed point exists if x

0

>ϕ

max

+γ.

Heterogeneous activity

We next study the network dynamics beyond the population-averaged activity, along modes

where different units have different amplitudes. We study perturbations of the type

xiðtÞ ¼ x0þdxiðtÞ:ð62Þ

We define the 2N-dimensional vector x¼ ðdx1; :::; dx1

N;dw1

1; :::; dw1

NÞT. Since the dynamics of

each unit is now different, the dynamic matrix of the linearized system, A, is described by a

squared matrix of dimensionality 2N. Therefore, the perturbations generate dynamics along

2Ndifferent modes whose timescales are determined by the eigenvalues of the matrix A. The

eigenvalues are determined by the characteristic equation |A−λI| = 0. In order to calculate

these eigenvalues, we make use of the following identity which holds for any block matrix

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 21 / 33

Z=A−λI, that is composed by the four square matrices P,Q,R, and Sand the block Sis

invertible:

jZj≔P Q

R S

¼ jSjjPQS1Rj:ð63Þ

Consequently, if we set Eq (63) to zero, since we assumed that |S|6¼ 0, we obtain

jZj ¼ 0) jPQS1Rj ¼ 0:ð64Þ

The identity in Eq (63) can be shown by using the decomposition

Z¼I0

0S

! I Q

0I

! PQS1R0

S1R I

!;ð65Þ

together with the fact that when a non-diagonal block is zero. The determinant of such a

matrix is the product of determinants of the diagonal blocks.

Synaptic filtering. The dynamical matrix for the network with synaptic filtering, obtained

by linearizing Eq (17), is

ð66Þ

The matrix Jis the connectivity matrix. Again, we assume in the following that the fixed point

is located in the linear range of the transfer function, so that 0

0¼1.

The characteristic equation, obtained by combining Eqs (64) and (66), reads

1þtmls

ð ÞIþtm

tsþtmls

1tm

ts

J

¼ 1þtmls

ð Þ þ lJ

1þtsls¼0;ð67Þ

where λ

J

are the eigenvalues of the connectivity matrix. Solving for λ

J

we obtain the equation

which maps the eigenvalues of the synaptic filtering network dynamics λ

s

onto the eigenvalues

of the connectivity matrix λ

J

,

lJ¼ ð1þtmlsÞð1þtslsÞ:ð68Þ

In contrast, solving for the eigenvalues of the dynamic matrix λ

s

we obtain the inverse map-

ping

l2

sþtsþtm

tstm

lsþ1lJ

tstm¼0:ð69Þ

In other words, Eqs 69 and 68 constitute two different approaches to assessing the stability of

the system [37]. One approach is to examine whether the domain of eigenvalues λ

s

resulting

from Eq (69) intersect the line Re (λ

s

) = 0 (Fig 3, insets in B). The eigenvalues λ

J

of the connec-

tivity matrix are distributed within a circle in the complex plane, whose radius is proportional

to the synaptic strength, lJ<Jﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

CEþg2CI

pplus an outlier real eigenvalue at J(C

E

−gC

I

) that

corresponds to the homogeneous perturbations studied above (see [36]). We focus in this sec-

tion on the bulk of eigenvalues that corresponds to modes of activity with different amplitudes

for different units. We can therefore parametrize the eigenvalues λ

J

as

lJðyÞ ¼ Jﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

CEþg2CI

peiyð70Þ

and introduce the parametrization into Eq (69) to obtain an explicit expression for the curve

Effects of adaptation and synaptic filtering on the timescales of recurrent networks

PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006893 March 21, 2019 22 / 33

that encloses the eigenspectrum λ

s

. Note that in an abuse of notation, we denote the limits of

the eigenspectrum as λand not the eigenvalues themselves that constitute the eigenspectrum.

The alternative approach is to use the inverse mapping from the eigenvalues λ

s

to the eigen-

values of the connectivity λ

J

, by mapping the line Re (λ

s

) = 0 into the space of eigenvalues λ

J

(see S1 Fig). More specifically, the line Re (λ

s

) = 0 can be parametrized as

ls¼ io;ð71Þ

and introduced into Eq (68). In this case, the stability is assessed by whether the eigenspectrum

of the connectivity matrix Jcrosses the stability boundary or not (insets in Fig 3). This alterna-

tive approach is useful for some calculations due to the simple geometry of the connectivity

eigenspectrum λ

J

.

Taking the alternative approach, introducing Eq (71) into Eq (68), we obtain the stability

bound in the complex plane of eigenvalues λ

J

:

lsb

J¼ ð1þitmoÞð1þitsoÞ:ð72Þ

The first point of the stability curve lsb

JðoÞintersecting with a circle of increasing radius

centered at the origin is the closest point of the curve to the origin, i.e. the minimum of jlsb

Jj2

with respect to ω. The squared distance to the origin