Estimation of thalamocortical and intracortical network models from joint thalamic single-electrode and cortical laminar-electrode recordings in the rat barrel system.

Abstract

A new method is presented for extraction of population firing-rate models for both thalamocortical and intracortical signal transfer based on stimulus-evoked data from simultaneous thalamic single-electrode and cortical recordings using linear (laminar) multielectrodes in the rat barrel system. Time-dependent population firing rates for granular (layer 4), supragranular (layer 2/3), and infragranular (layer 5) populations in a barrel column and the thalamic population in the homologous barreloid are extracted from the high-frequency portion (multi-unit activity; MUA) of the recorded extracellular signals. These extracted firing rates are in turn used to identify population firing-rate models formulated as integral equations with exponentially decaying coupling kernels, allowing for straightforward transformation to the more common firing-rate formulation in terms of differential equations. Optimal model structures and model parameters are identified by minimizing the deviation between model firing rates and the experimentally extracted population firing rates. For the thalamocortical transfer, the experimental data favor a model with fast feedforward excitation from thalamus to the layer-4 laminar population combined with a slower inhibitory process due to feedforward and/or recurrent connections and mixed linear-parabolic activation functions. The extracted firing rates of the various cortical laminar populations are found to exhibit strong temporal correlations for the present experimental paradigm, and simple feedforward population firing-rate models combined with linear or mixed linear-parabolic activation function are found to provide excellent fits to the data. The identified thalamocortical and intracortical network models are thus found to be qualitatively very different. While the thalamocortical circuit is optimally stimulated by rapid changes in the thalamic firing rate, the intracortical circuits are low-pass and respond most strongly to slowly varying inputs from the cortical layer-4 population.

Full text

Page 1

Estimation of Thalamocortical and Intracortical Network
Models from Joint Thalamic Single-Electrode and Cortical
Laminar-Electrode Recordings in the Rat Barrel System
Patrick Blomquist1, Anna Devor2,3, Ulf G. Indahl1, Istvan Ulbert4,5, Gaute T. Einevoll1*, Anders M. Dale3
1Department of Mathematical Sciences and Technology and Center for Integrative Genetics, Norwegian University of Life Sciences, A˚s, Norway, 2Athinoula A. Martinos
Center, Massachusetts General Hospital, Charlestown, Massachusetts, United States of America, 3Departments of Radiology and Neurosciences, University of California
San Diego, La Jolla, California, United States of America, 4Institute for Psychology of the Hungarian Academy of Sciences, Budapest, Hungary, 5Peter Pazmany Catholic
University, Department of Information Technology, Budapest, Hungary
Abstract
A new method is presented for extraction of population firing-rate models for both thalamocortical and intracortical signal
transfer based on stimulus-evoked data from simultaneous thalamic single-electrode and cortical recordings using linear
(laminar) multielectrodes in the rat barrel system. Time-dependent population firing rates for granular (layer 4),
supragranular (layer 2/3), and infragranular (layer 5) populations in a barrel column and the thalamic population in the
homologous barreloid are extracted from the high-frequency portion (multi-unit activity; MUA) of the recorded extracellular
signals. These extracted firing rates are in turn used to identify population firing-rate models formulated as integral
equations with exponentially decaying coupling kernels, allowing for straightforward transformation to the more common
firing-rate formulation in terms of differential equations. Optimal model structures and model parameters are identified by
minimizing the deviation between model firing rates and the experimentally extracted population firing rates. For the
thalamocortical transfer, the experimental data favor a model with fast feedforward excitation from thalamus to the layer-4
laminar population combined with a slower inhibitory process due to feedforward and/or recurrent connections and mixed
linear-parabolic activation functions. The extracted firing rates of the various cortical laminar populations are found to
exhibit strong temporal correlations for the present experimental paradigm, and simple feedforward population firing-rate
models combined with linear or mixed linear-parabolic activation function are found to provide excellent fits to the data.
The identified thalamocortical and intracortical network models are thus found to be qualitatively very different. While the
thalamocortical circuit is optimally stimulated by rapid changes in the thalamic firing rate, the intracortical circuits are low-
pass and respond most strongly to slowly varying inputs from the cortical layer-4 population.
Citation: Blomquist P, Devor A, Indahl UG, Ulbert I, Einevoll GT, et al. (2009) Estimation of Thalamocortical and Intracortical Network Models from Joint Thalamic
Single-Electrode and Cortical Laminar-Electrode Recordings in the Rat Barrel System. PLoS Comput Biol 5(3): e1000328. doi:10.1371/journal.pcbi.1000328
Editor: Karl J. Friston, University College London, United Kingdom
Received October 10, 2008; Accepted February 10, 2009; Published March 27, 2009
Copyright: ? 2009 Blomquist et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: Work was financially supported by the Research Council of Norway under the BeMatA and eVITA programmes, and by the National Institute of Health
[R01 EB00790, NS051188]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: Gaute.Einevoll@umb.no
Introduction
Following pioneering work in the 1970s by, e.g., Wilson and
Cowan [1] and Amari [2] a substantial effort has been put into the
investigation of neural network models, particularly in the form of
firing-rate or neural field models [3]. Some firing-rate network
models, in particular for the early visual system ([4], Ch.2), have
been developed to account for particular physiological data.
However, for strongly interconnected cortical networks, few
mechanistic network models directly accounting for specific
neurobiological data have been identified. Instead most work has
been done on generic network models and has focused on the
investigation of generic features, such as the generation and
stability of localized bumps, oscillatory patterns, traveling waves
and pulses and other coherent structures, for reviews see
Ermentrout [5] or Coombes [6].
We here (1) present a new method for identification of specific
population firing-rate network models from extracellular record-
ings, (2) apply the method to extract network models for
thalamocortical and intracortical signal processing based on
stimulus-evoked data from simultaneous single-electrode and
multielectrode extracellular recordings in the rat somatosensory
(barrel) system, and (3) analyze and interpret the identified firing-
rate models using techniques from dynamical systems analysis.
Our study reveals large differences in the transfer function
between thalamus (VPM) and layer 4 of the barrel column,
compared to that between cortical layers, and thus sheds direct
light on how whisker stimuli is encoded in population firing-
activity in the somatosensory system.
The derivation of biologically realistic, cortical neural-network
models has generally been hampered by the lack of relevant
experimental data to constrain and test the models. Single
electrodes can generally only measure the firing activity of
individual neurons, not the joint activity of populations of cells
typically predicted by population firing-rate models. Kyriazi and
Simons [7] and Pinto et al. [8,9] thus developed models for the
somatosensory thalamocortical signal transformation based on
pooled data from single-unit recordings from numerous animals.
PLoS Computational Biology | www.ploscompbiol.org1March 2009 | Volume 5 | Issue 3 | e1000328

Page 2

By contrast, multielectrode arrays provide a convenient and
powerful technology for obtaining simultaneous recordings from
all layers of the cerebral cortex, at one or more cortical locations
[10]. The signal at each low-impedance electrode contact
represents a weighted sum of the potential generated by synaptic
currents and action potentials of neurons within a radius of a few
hundred micrometers of the contact, where the weighting factors
depend on the shape and position of the neurons, as well as the
electrical properties of the conductive medium [11–13].
In the present paper we describe a new method for extraction of
population firing-rate models for both thalamocortical and
intracortical transfer on the basis of data from simultaneous
thalamic single-electrode and cortical recordings using linear
(laminar) multielectrodes in the rat barrel system. With so called
laminar population analysis (LPA) Einevoll et al. [11] jointly modeled
the low-frequency (local field potentials; LFP) and high-frequency
(multi-unit activity; MUA) parts of such stimulus-evoked laminar
electrode data to estimate (1) the laminar organization of cortical
populations in a barrel column, (2) time-dependent population
firing rates, and (3) the LFP signatures following firing in a
particular population. These ‘postfiring’ population LFP signa-
tures were further used to estimate the synaptic connection
patterns between the various populations using both current source
density (CSD) estimation techniques and a new LFP template-fitting
technique [11].
Here we use the stimulus-evoked time-dependent firing rates for
the cortical populations estimated using LPA, in combination with
single-electrode recordings of the firing activity in the homologous
barreloid in VPM, to identify population firing-rate models. The
models are formulated as nonlinear Volterra integral equations
with exponentially decaying coupling kernels allowing for a
mapping of the systems to sets of differential equations, the more
common mathematical representation of firing-rate models [5,14].
The population responses were found to increase monotonically
both with increasing amplitude and velocity of the whisker flick
[11,15,16]. A stimulus set varying both the whisker-flicking
amplitude and the rise time was found to provide a rich variety
of thalamic and cortical responses and thus to be well suited for
distinguishing between candidate models. The optimal model
structure and corresponding model parameters are estimated by
minimizing the mean-square deviation between the population
firing rates predicted by the models and the experimentally
extracted population firing rates.
A first focus is on the estimation of mathematical models for the
signal transfer between thalamus (VPM) and the layer-4
population, the population receiving the dominant thalamic input.
For this thalamocortical transfer our experimental data favors a
modelwith(1)fastfeedforwardexcitation,(2)aslowerpredominantly
inhibitory process mediated by a combination of recurrent (within
layer 4) and feedforward interactions (from thalamus), and (3) a
mixed linear-parabolic activation function. The identified thalamo-
cortical circuits are seen to have a band-pass property, and in the
frequency domain the largest responses for the layer-4 population is
obtainedforthalamicfiringrateswithfrequenciesaroundtwentyHz.
Very different population firing-rate models are identified for
the intracortical circuits, i.e., the spread of population activity from
layer 4 to supragranular (layer 2/3) and infragranular (layer 5)
layers. For the present experimental paradigm the extracted firing
rates of the various cortical laminar populations are found to
exhibit strong temporal correlations and simple feedforward
models with linear or mixed linear-parabolic activation function
are found to account excellently for the data. The functional
properties of the identified thalamocortical and intracortical
network models are thus qualitatively very different: while the
thalamocortical circuit is optimally stimulated by rapid changes in
the thalamic firing rate, the intracortical circuits are low-pass and
respond strongest to slowly varying inputs.
Preliminary results from this project were presented earlier in
poster format [17].
Results
Here we illustrate our approach by first showing results from
one of the six experimental data sets considered, and next show the
thalamic and cortical laminar population responses extracted from
these experimental data. Further, we outline the general form of
the neural population models we explore to account for these
population firing. We then go on to test specific candidate
thalamocortical models against the experimentally observed popula-
tion responses in layer 4 using the experimental thalamic response
as driving input to the model. The identified thalamocortical
models found by minimizing the deviation between model
predictions and experimental population firing rates are further
analyzed and explored using tools from dynamical system analysis.
Finally, we correspondingly examine how the experimentally
measured laminar population responses in layer 2/3 and layer 5,
respectively, can be explained by intracortical network models with
the experimental layer-4 population responses as input.
Experimental data and extracted population firing rates
In Figure 1 we show trial-averaged multi-unit activity (MUA) data
for one of six experiments, labeled ‘experiment 1’, from stimulus
onset, i.e., onset of whisker-flick, until 100 ms after. This is the
time window of data used in the further analysis. Color plots depict
the laminar-electrode recordings while the line plots below show
the corresponding trial-averaged thalamic recordings. Results for 9
of 27 stimulus conditions are shown, corresponding to three
different stimulus rise times t9,t5,t1
ð
amplitudes a1,a2,a3
ð
to both stimulus amplitude and stimulus velocity (and thus
stimulus rise time) [11,15,16], and we found that a set of stimuli
Þ for three different stimulus
Þ. Neurons in the barrel cortex are sensitive
Author summary
Many of the salient features of individual cortical neurons
appear well understood, and several mathematical models
describing their physiological properties have been
developed. The present understanding of the highly
interconnected cortical neural networks is much more
limited. Lack of relevant experimental data has in general
prevented the construction and rigorous testing of
biologically realistic cortical network models. Here we
present a new method for extracting such models. In
particular we estimate specific mathematical models
describing the sensory activation of thalamic and cortical
neuronal populations in the rat whisker system from joint
recordings of extracellular potentials in thalamus and
cortex. The mathematical models are formulated in terms
of average firing rates of a thalamic population and a set of
laminarly organized cortical populations, the latter extract-
ed from data from linear (laminar) multielectrodes inserted
perpendicularly through cortex. The identified models
describing the signal processing from thalamus to cortex
are found to be qualitatively very different from the
models describing the processing between cortical pop-
ulations; while the thalamocortical circuit is optimally
stimulated by rapid changes in the thalamic firing rate, the
intracortical circuits respond most strongly to slowly
varying inputs.
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org2 March 2009 | Volume 5 | Issue 3 | e1000328

Page 3

varying both the amplitude and rise time provide a rich variety of
responses suitable for the present study.
Fig. 2 shows the estimated thalamic
rx
nt ð Þ,n~2=3,4,5
in Fig. 1 (procedure described in Materials and Methods). The
estimated thalamic population firing rate is essentially just a low-
pass filtered version of the thalamic MUA. The cortical population
firing rates have been estimated by a method from the recently
developed laminar population analysis (LPA) [11] where the MUA
from the full set of electrode contacts on the laminar electrode is
modeled as a sum over contributions from a set of laminar
populations, cf. Eqs. (16–17). In the present case four cortical
populations are assumed in this firing-rate extraction scheme [11],
but the figure only shows results for the three cortical populations
considered in this study, namely the layer 2/3 n~2=3
n~4
ð
population was also identified, but it was left out of the present
analysis since predictions related to the postsynaptic LFP-profiles
from this particular population were foundtovarybetweendifferent
experiments, thus questioning its proper identification [11].
rx
Tt ð Þ
??
and cortical
??population firing rates extracted from the data
ðÞ, layer 4
Þ, and layer 5
n~5
ðÞ populations. A deeper laminar
Form of thalamocortical model
We first focus on the signal transfer between thalamus and the
dominant input layer of the barrel column, namely layer 4.
Specifically we seek a mathematical model predicting the layer-4
population firing rate rx
4t ð Þ for all 27 stimulus conditions (of which
nine are depicted in Fig. 2) given the corresponding thalamic
population firing rx
Tt ð Þ as model input.
The stimulus-evoked dynamics of the excitatory and inhibitory
neurons in layer 4 of the barrel column is complex. The excitatory
neurons receive both feedforward excitation from thalamic
neurons and recurrent excitation from other layer 4 excitatory
cells [9,18–20]. In addition, the excitatory cells are inhibited by
layer-4 interneurons [21]. These interneurons in turn receive
feedforward excitation from thalamus [22–24] and recurrent
inputs from other layer-4 neurons [9,21].
For mathematical convenience and conceptual simplicity we
choose to formulate the population firing-rate model as Volterra
integral equations [5,14]. In our so called full thalamocortical model
where all the abovementioned feedforward and recurrent
connections are included, we then have
r4t ð Þ~F4
bEfhEf{bIfhIf
ð Þ ? rx
? t ð ÞÞ
T
??t ð Þz
?
bErhEr{bIrhIr
ðÞ ? r4
½
ð1Þ
Here the h t ð Þ are temporal coupling kernels, and h ? r
convolution between the temporal kernel h t ð Þ and the population
½? t ð Þ is a temporal
Figure 1. Example experimental data. Trial-averaged experimental data for experiment 1 for 9 out of 27 applied stimulus conditions for three
different amplitudes (rows) and three different rise times (columns). Contour plots show depth profile of MUA as recorded by the laminar electrode,
normalized to its largest value over the 22 electrode traces and 27 stimulus condition. Line plots below show the corresponding thalamic MUA
recorded by a single electrode, normalized to the largest value over the 27 stimulus conditions.
doi:10.1371/journal.pcbi.1000328.g001
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org3March 2009 | Volume 5 | Issue 3 | e1000328

Page 4

firing rate r t ð Þ (Eq. 19). We here assume exponentially decaying
temporal coupling kernels (Eq. 20) which allows for a mapping of
the integral equation to a set of differential equations [5,14].
The four terms within the large parentheses on the right hand
side of Eq. (1) can be interpreted as four contributions to the input
current entering the somas of neurons in the layer-4 population.
The first and second terms correspond to feedforward excitation
bEfhEf? rx
thalamus, respectively. The coupling kernel hEf t ð Þ can be
interpreted as the weight of the excitatory contribution to the
present current input to the layer-4 population due to firing that
has occurred in the thalamic population a time t in the past.
Likewise, the third and fourth terms correspond to recurrent
excitation bErhEr? r4
cortical layer-4 neurons. The model structure is illustrated in
Fig. 3B. Note that corticothalamic feedback [25] is not explicitly
modeled as this effect will only change the thalamic firing rate.
Since we do not model the thalamic firing rates and instead use the
experimentally extracted rx
Tas input to the model, any effect of
cortical feedback affecting the thalamic firing rate will be included
automatically.
The function F4:ð Þ in Eq. (1) is an activation function that converts
the net input current to population firing rate r4t ð Þ. This activation
function is often modeled as a sigmoidal function [4,8], but here
we found that a simpler four-parameter threshold-type function
T
??t ð Þ
??
and inhibition
bIfhIf? rx
T
??t ð Þ
??
from the
½? t ð ÞðÞ and inhibition bIrhIr? r4
½? t ð ÞðÞ from
with a linear and/or parabolic activation above threshold
provided a better fit to the experimental data, cf. Eq. (21). The
form of this activation function is illustrated in Fig. 3A. Such a
threshold-type activation function has also been seen experimen-
tally when studying firing characteristics of excitatory neurons in
the barrel column [26].
The temporal coupling kernel h t ð Þ is described by two
parameters, a time constant t and a time-delay parameter D, cf.
Eq. (20). To reduce the number of parameters in the model in
Eq. (1), the time delays in the recurrent temporal kernels are set to
zero, i.e., DEr~DIr~0. Further, one of the weight parameters b
can be fixed without loss of generality. We therefore set bEf:1.
Nevertheless, this full thalamocortical model encompasses 13
parameters (4 for the activation function, 3 weights, 6 specifying
temporal kernels). This is a sizable number of adjustable model
parameters given the variability of the experimental data, and
when comparing with our experimental data it was observed that
in particular the feedforward and recurrent inhibition terms had
strongly overlapping effects on the model predictions for the layer-
4 firing rate so that their individual contributions were difficult to
assess. We thus chose to mainly investigate reduced versions of
the full thalamocortical model in Eq. (1) where either (1) only the
recurrent inhibition term (‘recurrent’ model) or (2) only the
feedforward inhibition term (‘feedforward’ model) was kept,
respectively.
Figure 2. Example extracted population firing rates. Extracted population firing rates for experiment 1 for 9 out of 27 applied stimulus
conditions. Green: thalamus. Black: layer 2/3. Blue: layer 4. Red: layer 5. Note that the extracted firing rates are normalized so that the maximum
response over all stimulus conditions is unity. For layer 2/3, layer 4 and layer 5 this occurs for the stimulus a3,t1
thalamic rate it occurs for stimulus a3,t2
ð
doi:10.1371/journal.pcbi.1000328.g002
ðÞ shown in the figure. For the
Þ, not shown in the figure.
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org4 March 2009 | Volume 5 | Issue 3 | e1000328

Page 5

Recurrent thalamocortical model
We first consider the so called recurrent thalamocortical model
where the feedforward inhibitory term in the full model in Eq. (1)
has been omitted, i.e.,
r4t ð Þ~F4I4t ð Þ
½
ðÞ, I4t ð Þ: hEf? rx
bErhEr{bIrhIr
T
??t ð Þz
ð Þ ? r4
? t ð Þ
ð2Þ
This model has a special significance in that it maps (by use of
the linear chain trick [14]) to a set of differential equations that are
structurally similar to the firing-rate model suggested by Pinto et
al. [8,9], cf. Eqs. (28–29) in Materials and Methods.
The model was fitted to the experimentally extracted population
firing rate rx
4t ð Þ (for all 27 stimulus conditions simultaneously) by
minimizing the mean square deviation between model and
experimental results (Eq. 22) using the optimization method
described in Materials and Methods.
Fits to recurrent model.
recurrent model successfully accounts for the experimental
observations in all six experiments considered. In the panels
each dot corresponds to an experimentally measured layer-4 firing
rate rx
4ti
ð Þ at a particular time ti plotted against the value I4
predicted by the fitted model at the same time ti. If the model was
perfect and the data noiseless, these points should all lie on the
solid line corresponding to the fitted value of the activation
function F4I4
ð Þ. A small spread is generally observed in Fig. 4, and
this is reflected by the low error value, i.e., e in Eq. (22), of the best
fits. As seen is Table 1 the errors are all less than 6%. In Table 1
we also list the fitted parameter values for the six experiments.
Fitted model parameters for recurrent model.
ments 1–3 correspond to the stimulus paradigm with 3 different
amplitudes and 9 different rise times, while experiments 4–6
correspond to 27 different amplitudes but a single fixed rise time.
As seen, for example, in Fig. 2 the variation in the rise time affects
the thalamic and cortical responses more than variation in the
stimulus amplitude. Thus experiments 1–3 exhibit a greater
variation in the responses and thus a richer data set for the models
to be tested against. In fact, for experiments 4–6 only a parabolic
part of the activation function F4I4
and the linear part of the activation function in Eq. (21) was
omitted from the model.
The choice of stimuli set also appears to affect the fitted
parameter values somewhat as indicated by comparing the fitted
parameter values for experiments 1, 2, 4 and 5, all carried out
using the same preparation in the same rat. As seen in Table 1 the
fitted parameter values of experiment 1 and 2 are very similar,
likewise for experiment 4 and 5. The parameter values for
experiments 1 and 2 also compare well with the values for
experiments 4 and 5, but less so. In the following we will focus
mostly on experiments 1–3 where the 369 stimulus paradigm is
used, and a more varied set of responses are obtained.
For experiments 1 and 2 the fitted feedforward time constants
tEf are between 3 and 6 milliseconds and the feedforward delay
DEf between 2 and 3 milliseconds (see Table 1). The recurrent
time constants are significantly longer: between 8 and 10 milli-
seconds for the recurrent excitation tEr and between 13 and
15 milliseconds for the recurrent inhibition tIr. For all six
experiments we observe the recurrent inhibition to have the
longest time constant. In experiments 3 and 6 the fitted time
constant of the feedforward excitation is seen to be longer than for
the recurrent excitation.
In all experiments the weight of the recurrent inhibition bIrwas
found to be larger than the weight of the recurrent excitation bEr.
However, the values of bErand bIrare seen to vary significantly
between the experiments. This partially reflects that when the
recurrent terms dominate the feedforward term, large values of the
As illustrated in Fig. 4 the
Experi-
ð Þ could be seen in the data,
Figure 3. Model structure. (A) Illustration of form of activation
function FnIn
ð Þ in Eq. (21). (B) Illustration of structure of population
firing-rate models with feedforward and recurrent terms in Eqs. (1) and
(10). Filled and open dots represent excitatory and inhibitory
connections, respectively.
doi:10.1371/journal.pcbi.1000328.g003
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org5 March 2009 | Volume 5 | Issue 3 | e1000328

Page 6

recurrent weights can be compensated by making the activation
function less steep, and vice versa. However, the ratio bEr=bIris
found to be rather constant, typically between 0.75 and 0.9 (with
experiment 6 as the only exception). Thus while excitation may
dominate at short time scales due to the shorter excitatory time
constants, inhibition dominates at longer time scales.
The positive correlation between bErand bIrwas confirmed by
direct inspection of the correlation matrices for the ensembles of
fitted model parameters obtained during the numerical optimiza-
tion procedure (Materials and Methods). It was also seen by
inspection of the eigenvectors of the most ‘sloppy’ direction of the
Levenberg-Marquardt Hessian L (Eq. 23), i.e., the component
with the smallest eigenvalue. A ‘sloppy’ direction of L means that
the fitting error e will change little when moving along this
direction in parameter space [27]. A positive correlation between
bErand bIrwould imply that the fitting error will change little
when both parameters are increased (or decreased) simultaneously.
Indeed, the eigenvector of the most ‘sloppy’ eigenvalue revealed
exactly this property.
Further inspection revealed that bEris negatively correlated
with the slope of the activation function (parameters a and b in
Eq. (21)). This is as expected since a decrease in the recurrent
excitatory weight bErcan be compensated for by an increase in the
activation-function slope. Moreover, it was observed that bEr(and
bIr) is positively correlated to tErand negatively correlated to tIr.
Phase-plane analysis of recurrent model.
recurrent thalamocortical models on integral form can be mapped
The identified
to a corresponding set of two differential firing-rate equations
using the ‘linear-chain trick’ [5,14] as described in Materials and
Methods, cf. Eqs. (25–29). This is very useful since it allows for the
use of standard techniques from dynamical systems analysis to
investigate the identified models.
With slightly redefined auxiliary variables compared to Eqs.
(25–27) in Materials and Methods, that is,
XE4: hEr? r4
½? t ð Þ, XI4: hIr? r4
½? t ð Þ, XTt ð Þ: hEf? rT
½? t ð Þ,
ð3Þ
the recurrent thalamocortical model can be reformulated as (cf.
Eqs. 28–29)
tErdXE4
dt
~{XE4zF4bErXE4{bIrXI4zbEfXTt ð ÞðÞ,
ð4Þ
tIrdXI4
dt
~{XI4zF4bErXE4{bIrXI4zbEfXTt ð ÞðÞð5Þ
This equation set resembles the model derived using ‘semirigor-
ous’ techniques by Pinto et al. in [8], and further explored in [9], for
populations of interconnected layer-4 excitatory and inhibitory
barrel neurons receiving stimulus-evoked thalamic input.
The formulation in terms of two dynamical variables XE4and
XI4, with XT only providing an external input, allows for
Figure 4. Fits of recurrent thalamocortical model. Illustration of fits of recurrent thalamocortical model in Eq. (2) to data from experiments 1–6.
Each black dot corresponds to the experimentally measured layer-4 firing rate rx
I4ti
ð Þ. The red dots are corresponding experimental data points taken from the first 5 ms after stimulus onset (for all 27 stimuli). These data points
show the activity prior to any stimulus-evoked thalamic or cortical firing and correspond to background activity. The solid green curve corresponds to
the fitted model activation function F4I4
ð Þ.
doi:10.1371/journal.pcbi.1000328.g004
4ti
ð Þ at a specific time point tiplotted against the model value of
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org6March 2009 | Volume 5 | Issue 3 | e1000328

Page 7

visualization of the circuit behavior using phase-plane analysis. In
analogy with Pinto et al. [9] we show in Fig. 5 the layer-4 model
response for the weakest a1,t9
ð
experiment 1. In the five phase plots for the two stimuli considered
the instantaneous excitatory and inhibitory nullclines at five different
times (twi,i~1,...,5, for weak stimulus; tsi,i~1,...,5, for strong
stimulus) are shown. These nullclines are obtained assuming a
constant thalamic synaptic drive equal to the instantaneous values
XTtwi
ð
nullcline dXE4=dt dXI4=dt
ð
illustrates the magnitude of dXE4=dt.
As seen in Fig. 5 the network model for the strong stimulus
a3,t1
ð
XE4,XI4
ð
be readily understood by investigation of the phase-plane plots for
the times shortly after onset. For the strong stimulus the trajectory
is seen to be in a region with a large dXE4=dt at, for example, the
times ts1and ts2resulting in a rapid growth and large maximum
Þ and strongest a3,t1
ðÞ stimuli for
Þ or XTtsi
ð Þ. Below the instantaneous excitatory (inhibitory)
Þ is negative, and the color code
Þ makes a significantly more extended excursion in the
Þ phase plane than the weaker stimulus a1,t9
ðÞ. This can
value of XE4. For these short times the fast feedforward and
recurrent excitation dominates the slower inhibition. For the weak
stimulus, however, the trajectory remains in regions with small
values of dXE4=dt (i.e., without dominant excitation). As a
consequence we observe that while the difference in maximum
amplitude of the thalamic synaptic drives between the strong and
weak stimuli is seen in the upper panels of Fig. 5 to be less than a
factor two, the difference in the maximum layer-4 firing rate is
more than a factor three. Thus the amplitude of the thalamic
population response does not explain the layer-4 response alone;
the rise time of the thalamic response is also important [9].
The qualitative conclusions from Fig. 5 is in agreement with the
corresponding analysis by Pinto et al. [9]. This is an interesting
result in itself since our model and the model of Pinto et al. [9] are
derived in very different ways. Here the parameters are extracted
from a single experiment measuring population activity directly.
By contrast, in the study by Pinto et al. the model and its
parameters were chosen to qualitatively reproduce pooled barrel
population responses, i.e., single-unit poststimulus time histograms
(PSTHs), recorded from numerous animals [8,9]. Further, the
thalamic responses used to generate their phase-plots analogous to
Fig. 5 were modeled with a simplified triangular temporal rate
profile with varying rise times [9], not extracted from experimental
data like here.
Due to differences between our model (Eqs. 4–5) and the model
of Pinto et al. [9] a direct comparison between the applied model
parameters is generally not possible. However, the time constants
can be compared directly: their choice of tEr=5 ms and
tIr=15 ms is qualitatively similar to what was extracted from
our experiments 1 and 2, i.e., tEr=8–10 ms and tIr=13–15 ms.
Their choice of weight parameters appears to differ more from our
results: for example, they set the ratio bEr=bIrto be between 1.7 and
2.3, i.e., stronger recurrent excitation than recurrent inhibition. For
experiments 1–3 we see in Table 1 that the ratio is found to be
between0.75and0.9,i.e.,therecurrentinhibitionhasalargerweight
than recurrent excitation. For the tension, i.e., the ratio of recurrent
excitationover the excitation fromthalamus bEr=bEf[8],theychose
a value of about 0.9. For our experiments this ratio showed a
significant variation, varying from 1.3 to 4.3 for experiments 1–3.
Stability of background states for recurrent model.
differential form our population-firing rate model is readily
available for stability analysis. Due to the recurrent excitatory
term the recurrent thalamocortical model can in principle have
unstable equilibrium points, but the identified models must clearly
exhibit stable background states when only the stationary thalamic
input (corresponding to the absence of whisking) is present. For
experiments 1–3 the background state (cf. red dots in Fig. 4) is
found to be on the linear flank of the activation function, and the
stability of these background states can be demonstrated on the
basis of the fitted model parameters a4,tEr,tIr,bEr,bIrlisted in
Table 1. A full stability analysis for the recurrent thalamocortical
model is given in Materials and Methods (Eqs. 30–35), and the
criteria for stability of the background state is found to be
On
1ztEr
tIrza4 bIr
tEr
tIr{bEr
??
w0
ð6Þ
1za4bIr{bEr
ðÞw0
ð7Þ
Insertion of the fitted numerical parameter values into the
inequalities Eqs. (6–7) demonstrates that both conditions are
Table 1. Fitted model parameters for recurrent
thalamocortical model.
Exp. 1 Exp. 2Exp. 3
tEf (ms)3.75.7 3.3
DEf (ms) 2.52.04.5
tEr (ms) 9.38.01.3
tIr (ms) 13.7 14.830.660.7
bEr
4.2760.03 2.9360.161.2660.03
bIr
4.8160.033.8560.161.4860.02
bEr=bIr
0.890.7660.01 0.8560.02
a4
0.550.560.30
b4
1.48 1.8860.01 0.2660.02
I{
4
I?
4
20.06
20.05
20.11
0.41 0.35 0.21
error e ð Þ
0.05220.0430 0.0586
Exp. 4 Exp. 5Exp. 6
tEf (ms) 2.81.9 8.2
DEf (ms)3.5 4.03.5
tEr (ms) 14.715.73.3
tIr (ms)20.322.919.6
bEr
5.9860.076.1560.010.61
bIr
6.6560.077.5360.01 2.06
bEr=bIr
0.900.820.29
a4
---
b4
0.420.432.94
I{
4
I?
4
I?
4
I?
4
I?
4
20.37
20.39
20.13
error e ð Þ
0.04140.03710.0397
Resulting optimized parameters for the recurrent thalamocortical network
model (Eq. 2) incorporating feedforward excitation and recurrent excitation and
inhibition. The listed parameter values correspond to the mean of fitted
parameter values from 25 selected models giving essentially the same error (see
Materials and Methods). The standard deviations are only listed if they exceed
the last digit of the mean. Note that the value of bEr=bIris included to illustrate
the relative constancy of the ratio of the fitted values for bErand bIr.
doi:10.1371/journal.pcbi.1000328.t001
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org7 March 2009 | Volume 5 | Issue 3 | e1000328

Page 8

fulfilled and that the background state is indeed stable. Inspection
of Eqs. (6–7) further shows that instability is promoted by an
increased weight of the recurrent excitation bEr
slope of the activation function a4
ð
of the recurrent inhibition tIr
ð
1 the background state becomes unstable if bEris increased by a
factor 1.5, a4by a factor 3.2, or tIrby a factor 1.9.
ðÞ, an increased
Þ, or an increased time constant
Þ [28]. For example, for experiment
Inclusion of feedforward inhibition in thalamocortical
model fits
We further investigated whether the full thalamocortical model
in Eq. (1), i.e., the recurrent model amended with the feedforward
inhibition term bIfhIf? rx
the recurrent model (Eq. 2) alone. Numerical investigations
showed that the improvement, i.e., reduction of the error e, was
marginal: for experiments 1–3 the reduction in error was in all
cases less than 0.001. Thus the recurrent inhibitory term alone
seems to be sufficient to account well for the present experimental
data.
T
??, gives an improved fit compared to
We then investigated whether a pure feedforward model, i.e., a
model with feedforward excitation and inhibition only, could
account for the thalamocortical data. This model reads
r4t ð Þ~F4
bEfhEf{bIfhIf
ðÞ ? rx
T
??
t ð Þ
??
ð8Þ
and in Table 2 and Fig. 6 we show the results of the optimization.
Comparison of fits to recurrent and feedforward
thalamocortical models
For experiments 1–3, which exhibit the largest variation of
responses, the fits of the experimental data for the feedforward
model are poorer than for the corresponding fits of the recurrent
model. While the errors e for the recurrent model are seen in
Table 1 to be 0.052, 0.043 and 0.059 for experiments 1, 2 and 3,
respectively, the corresponding errors for the purely feedforward
model are seen in Table 2 to be 0.064, 0.069 and 0.071, i.e.,
relative increases in errors of 23%, 60% and 20%, respectively.
However, for experiments 4–6 where only the stimulus amplitude
Figure 5. Phase-plane analysis of recurrent thalamocortical model for experiment 1. Phase-plane analysis of recurrent thalamocortical
model in Eqs. (4–5) for fitted model parameters for experiment 1, cf. Table 1. Upper panels show the synaptic drive from thalamus XTt ð Þ and the
firing rate r4t ð Þ~F4bErXE4t ð Þ{bIrXI4t ð ÞzXTt ð Þ
panels show the points in time, twi i~1,...,5
ð
panels. In each of the phase plots (five for each of the stimuli) the instantaneous excitatory (black) and inhibitory (blue) nullclines are shown, i.e., the
nullclines obtained with a constant thalamic synaptic drive equal to the instantaneous value XTtwi
activity is indicated by the black response curve, where the head of the curve marks the network’s current state, and the tail represents prior states.
doi:10.1371/journal.pcbi.1000328.g005
ðÞ for the weakest (a1,t9; left) and strongest (a3,t1; right) stimuli. The filled dots in the two upper
Þ for a1,t9
ðÞ and tsi i~1,...,5
ðÞ for a3,t1
ðÞ at which the network states are shown in the two lower
ðÞ for a1,t9
ðÞ and XTtsi
ð Þ for a3,t1
ðÞ. The network’s
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org8March 2009 | Volume 5 | Issue 3 | e1000328

Page 9

and not the rise time is varied, there is no such clear preference,
i.e., lower fitting errors, for the recurrent model.
In Fig. 7 we compare the best fits of the recurrent (Eq. 2) and
feedforward (Eq. 8) thalamocortical models with the experimen-
tally extracted layer-4 firing rate for experiment 1 for the 9
stimulus conditions depicted in Fig. 2. Both models underestimate
the peak values for the layer-4 firing rate for the shortest rise time
t1
ð Þ giving the largest responses. For the stimulus a3,t1 giving the
overall largest layer-4 response, the fitted recurrent model both
follows the rise and the ensuing fall of the response peak better
than the feedforward model, but this difference is less pronounced
for the other stimuli providing the larger responses. A detailed
investigation of the fits for individual stimulus conditions in fact
shows that the fitted recurrent model typically is closer to the
experimental curves than the feedforward model for the entire set
of stimulus conditions (data not shown).
The fitted activation functions for the recurrent and feedfor-
ward models for experiments 1 to 3 are quite different (cf. Figs. 4
and 6). In particular, for experiments 1 and 2 the activation
functions for recurrent models have pronounced linear flanks
which cover about half the dynamic range of I4. In contrast, the
activation functions of the feedforward models for these two
experiments are essentially parabolic in the entire dynamic range.
In the recurrent model the ‘boosting’ of the salient stimuli thus
appears to a large extent to be provided by recurrent excitation
allowing for a partially linear activation function. In contrast, in
the feedforward model the observed amplification of strong stimuli
requires a purely parabolic activation function.
Another way to compare the candidate thalamocortical models
is to investigate their so called ‘sloppiness’ [27]. Sloppiness is a
measure of the sensitivity of the model fit to changes in model
parameters: large sloppiness means that some parameter combi-
nations can be varied significantly without changing the quality of
the model fit. Sloppiness can be quantified by an eigenvalue
analysis of the Hessian matrix composed of the partial 2nd
derivatives of the model error function [27,29]. Here the so called
Levenberg-Marquardt Hessian matrix L (Eq. 23) is used, cf.
Materials and Methods. Fig. 8 shows the eigenvalue spectra of L
(23) for experiments 1–3 for the three investigated thalamocortical
models: (1) the recurrent model (Eq. 2), (2) the feedforward model
(Eq. 8), and (3) the full model including both feedforward
inhibition and the recurrent connections (Eq. 1). The spectra are
Figure 6. Fits of feedforward thalamocortical model. Illustration of fits of feedforward thalamocortical model (Eq. 8) to data from experiments
1–3. Each dot corresponds to the experimentally measured layer-4 firing rate rx
The red dots are corresponding experimental data points taken from the first 5 ms after stimulus onset (for all 27 stimuli). These data points show the
activity prior to any stimulus-evoked thalamic or cortical firing and represent background activity. The solid green curve corresponds to the fitted
model activation function F4I4
ð Þ.
doi:10.1371/journal.pcbi.1000328.g006
4ti
ð Þ at a specific time point tiplotted against the model value of I4ti
ð Þ.
Table 2. Fitted model parameters for feedforward
thalamocortical model.
Exp. 1Exp. 2 Exp. 3
tEf (ms)8.4 9.8 9.3
DEf (ms) 2.53.55.0
tIf (ms)20.560.3 18.260.1 100*
DIf (ms) 2.5 3.55.0
bIf
0.941.063.61
a4
0.2860.02 0.5460.010.14
b4
8.960.229.561.216.260.3
I{
4
I?
4
20.1560.01
20.11
20.52
20.03
20.000.02
error e ð Þ
0.06360.0691 0.0706
Exp. 4 Exp. 5Exp. 6
tEf (ms) 8.55.25.1
DEf (ms)2.53.0 5.0
tIf (ms)93.560.287.360.6 17.760.4
DIf (ms) 2.5 1.55.0
bIf
1.8 2.0 1.3
a4
0.090.050.01
b4
3.21.98.760.4
I{
4
I?
4
20.54
20.7660.02
25.8061.1
20.06
20.13
20.07
error e ð Þ
0.0394 0.03510.0629
Resulting optimized parameters for the feedforward thalamocortical network
model (Eq. 8) incorporating feedforward excitation and inhibition. The listed
parameter values correspond to the mean of fitted parameter values from 25
selected models giving essentially the same error (see Materials and Methods).
The standard deviations are only listed if they exceed the last digit of the mean.
Note that for experiments 4–6 the parameter a4was fixed to zero in the
optimization procedure. Also, for experiment 3 the listed value 100 ms for tIf
corresponds to the maximum allowed value for this parameter.
doi:10.1371/journal.pcbi.1000328.t002
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org9 March 2009 | Volume 5 | Issue 3 | e1000328

Page 10

normalized so that the largest eigenvalue is unity. Small
eigenvalues correspond to large sloppiness, and variations in
model parameters along the direction of the corresponding
eigenvectors in parameter space will have small effects on the
overall model error [27]. The number of eigenvalues corresponds
to the dimension of L. The feedforward delay parameters Dmn,
which are discrete parameters in the present modeling scheme, are
absent in L (cf. Materials and Methods), and the number of
eigenvalues in Fig. 8 is thus nine for the recurrent model, seven for
the feedforward model, and eleven for the full model.
In Fig. 8 we see more ‘sloppy’ eigenvalues for the feedforward
model than the recurrent model. For example, only one of nine
eigenvalues is found to be less than 2?1025(measured relative to
the largest eigenvalues) for the recurrent model. In contrast, two of
seven eigenvalues are less than 2?1025for the feedforward model.
This larger ‘sloppiness’ is in accordance with the poorer fits
observed for the feedforward model; the functional flexibility
inherent in the feedforward model appears not to be well suited
to account for the present data with 3 different amplitudes and 9
different stimulus rise times. For the full model we find three of
eleven eigenvalues to be smaller than 1025. Thus compared to
the recurrent model, the added feedforward inhibition with two
new parameters seems only to add two ‘sloppy’ eigenvalues, in
line with the observation that the overall fit is not improved
much.
To summarize, our results so far suggest that the model with
recurrent inhibition accounts slightly better for the present
experimental data than the model with feedforward inhibition,
i.e., that the layer-4 interneurons providing the inhibition of the
layer-4 excitatory neurons are in turn mainly driven by excitation
from layer-4 neurons. Since the relative strength of recurrent
effects compared to feedforward effects appears to increase with
increasing thalamic stimuli [19], this might indicate that the
present fits to the experimental data might be dominated by
‘strong’ inputs. To explore this further we investigated the
properties of the thalamocortical transfer using linear-systems
measures such as transfer functions in frequency space and
impulse-response functions.
Thalamocortical transfer
Thalamocortical
thalamocortical models are in general nonlinear due to the
nonlinear activation functions F4I4
ð Þ. For small deviations around
a working point, however, the above models can be linearized, so
that linear-systems measures such as transfer functions in frequency
space and impulse-response functions can be explored [30].
For the thalamocortical models the derivation of the transfer
function amounts to deriving the response to a small sinusoidal
modulation ~ r rTv
ð Þ in the thalamic input superimposed on a
constant background activity rT0. As shown in Materials and
frequencyresponse.
Ouridentified
Figure 7. Comparison of thalamocortical model fits. Comparison of fits of the recurrent and feedforward thalamocortical models with
experimental data set 1 for the same 9 (of 27) stimulus conditions considered in Fig. 2. The green line corresponds to the thalamic input firing rate
rx
4t ð Þ, while the red and blue lines correspond to the best fits of the recurrent
(Eq. 2) and feedforward (Eq. 8) models for the layer-4 firing rate r4t ð Þ, respectively. Time zero corresponds to stimulus onset.
doi:10.1371/journal.pcbi.1000328.g007
Tt ð Þ, the black line to the experimentally extracted layer-4 firing rate rx
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org10 March 2009 | Volume 5 | Issue 3 | e1000328

Page 11

Methods the modulated response ~ r r4v
layer 4 is then given by ~ r r4v
the linear transfer function [30–32]. The transfer function for the full
thalamocortical model Eq. (1) is derived in Materials and
Methods, cf. Eq. (42), and for the recurrent model in Eq. (2) it
reduces to
ð Þ (complex notation) in
ð Þ~T4=Tv
ð Þ~ r rTv
ð Þ where T4=Tv
ð Þ is
T4=Tv
ð Þ~~ r r4v
ð Þ
ð Þ~
~ r rTv
F4’I40
ð
ÞbEr~h hErv
Þ~h hEf v
ð ÞzF4’I40
ð Þ
1{F4’I40
ððÞbIr~h hIrv
ð Þ
ð9Þ
Here bEfand bIfin Eq. (42) has been set to one and zero,
respectively. F4’I40
ð
the working point I40, and the Fourier-transformed coupling
kernels~h hEf v
ð Þ,~h hIf v
exp ivD
ð
This transfer function describes how a small modulation of the
thalamic population firing rate transfers to the layer-4 population
firing rate. The transfer function T4=Tv
where the transfer amplitude is given by the absolute value
T4=Tv
ð Þ
and layer-4 output is given by the complex phase angle.
Fig. 9 shows an example of how the Fourier amplitude (lower left)
and phase (lower right) of the recurrent thalamocortical transfer-
function model in Eq. (9) depends on frequency. The parameter
Þ is the derivative of the activation function at
ð Þ,~h hErv
Þ= 1{itv
ð Þ,~h hIrv
ð Þ are all given by~h h v
ð Þ~
ðÞ, cf. Eq. (39).
ð Þ is a complex quantity
????, and the change in phase between the thalamic input
values correspond to the fitted values of bEr,bIr,tEf,tEr,tIrand DEf
for experiment 1 taken from Table 1. Since the fitted activation
function F4I4
ð Þ is nonlinear, there is no unique derivative F4’I4
the fitted model. In Fig. 9 we thus show results for two different
values of the derivative: (i) F4’I4
ð Þ~a4~0:55 corresponding to the
derivative on the linear flank, and (ii) F4’I4~0:5
sponding a larger derivative more characteristic for the parabolic
part of the activation function. In both cases the amplitude of the
model transfer functionhas its maximum at finite frequencies,16 Hz
and 18 Hz, respectively. This peak stems from a resonance-like
phenomenon for frequencies where the denominator of Eq. (9)
becomessmall.Even larger valuesof F4’I4
value further.
In Fig. 9 (lower right) we further observe that the phase of the
transfer function is negative for the smallest frequencies. The
negative transfer-function phase implies that the maximum in the
layer-4 responses will precede the maximum of the thalamic input
for a small sinusoidal input for these frequencies.
Fig. 9 further shows the amplitude of the Fourier components of
the thalamic input rx
Tv
ð Þ (upper left), the layer-4 firing rate rx
(upper right), as well as the amplitude (lower left) and phase (lower
right) of the experimental transfer ratio rx
stimulus conditions providing the strongest response in experiment
1 (a3,t1; a3,t2; a3,t3; a2,t1; a2,t2; a2,t3). Also the average of
the amplitudes and phases of the experimental transfer ratios
across these six stimulus conditions are shown. The same
ð Þ for
ðÞ~0:81 corre-
ð Þ would increase the peak
4v
ð Þ
4v
ð Þ?rx
Tv
ð Þ for six of the
Figure 8. Sloppiness analysis of thalamocortical model. Illustration of parameter sensitivities, i.e., ‘sloppiness’, of model fits for recurrent (rec.),
feedforward (feedf.) and full (full) thalamocortical models corresponding to Eqs. (2), (8), and (1), respectively. Eigenvalue spectra of the Levenberg-
Marquardt Hessian L (Eq. 23) for experiments 1–3 are shown. The eigenvalues are normalized to the largest eigenvalue. Delay parameters Dmnare left
out of the analysis, since in our models these parameters are set to be discrete (with a time step of 0.5 ms).
doi:10.1371/journal.pcbi.1000328.g008
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org11 March 2009 | Volume 5 | Issue 3 | e1000328

Page 12

characteristic resonance peak is seen in the experimental transfer
ratios as for the linear-model transfer function in Eq. (9). The
theoretical function for the largest shown value for F4’I4
F4’I4~0:5
ð
curves. This is not surprising since the depicted experimental
curves correspond to six of the stimuli giving the largest responses
for which also the parabolic part of the activation function is
encountered. We see, however, that the maxima of the
experimental transfer ratios are systematically shifted to somewhat
higher frequencies compared to the predictions of the linear
transfer-function model in Eq. (9). The same small deviation is also
observed for the transfer ratio found by replacing rx
Fourier transform of the best fit r4v
Eq. (2) (red curve in lower left panel of Fig. 9). Thus the deviation
is presumably due to the non-linearities inherent in the system
under the present stimulus condition, represented by the nonlinear
activation function in the recurrent model. In Fig. 9 (lower right)
we also observe a good qualitative agreement between the
predicted phase difference between the thalamic input and layer-
4 output from the linear transfer-function model and the
experimentally measured phase differences.
The experimental stimulus set with 3 different amplitudes and 9
different stimulus rise times was found to provide the largest
ð Þ, i.e.,
Þ~0:81, is seen to be closest to the experimental
4v
ð Þ with the
ð Þ to the recurrent model in
variation in neuronal responses and thus provide the best test data
for distinguishing the two candidate models. As seen in Tables 1
and 2 the recurrent thalamocortical model give better fits for these
experiments, i.e., experiments 1–3, than the feedforward model.
The increases in the error measure e (Eq. 22) for the feedforward
model compared to the recurrent model are between 20% and
60%. The results for the thalamocortical transfer in experiment 1
shown in Fig. 9 indicate that the apparent superiority of the
recurrent models mainly lies in its ability to account for the rapidly
varying parts of the measured layer-4 population response. As seen
in the lower left panel of this figure, the ratio of the frequency
contents of the experimentally extracted layer-4 and thalamic
signals (solid line) is better accounted for by the recurrent model
(red line) than the feedforward model (blue line) for frequencies
above 20 Hz. For frequencies below 20 Hz the difference is less.
For experiment 2, where the increase in error from the recurrent
to the feedforward model is even larger, this tendency is even more
pronounced (data not shown).
Thalamocortical impulse response.
dynamical systems measure is the impulse-response function, i.e., the
response to a very brief input signal [30]. Fig. 10 shows the
impulse response of the recurrent thalamocortical model (Eq. 2)
with model parameters fitted to experiment 1, cf. Table 1. In this
Another traditional
Figure 9. Thalamocortical transfer for experiment 1. Frequency content, i.e., Fourier amplitudes, of the thalamic input rx
the layer-4 firing rate rx
42pf
ð
a2,t2; a2,t3). (Note that f~v= 2p
ð
rx
42pf
ð
model predictions for amplitude (lower left) and phase (lower right) of the recurrent thalamocortical transfer-function model (Eq. 9) are shown for the
fitted parameter values bEr,bIr,tEf,tEr,tIrand DEffor experiment 1 taken from Table 1; green lines correspond to choosing F4’I4
dashed green lines to F4’I4~0:5
ð
phase (lower right) of the transfer ratios found for the six stimuli listed above when driving the fitted recurrent (red) and feedforward (blue) models in
Eqs. (2) and (8), respectively, with the experimentally extracted thalamic input rx
doi:10.1371/journal.pcbi.1000328.g009
T2pf
ðÞ (upper left) and
Þ (upper right) for six stimulus conditions providing the strongest response in experiment 1 (a3,t1; a3,t2; a3,t3; a2,t1;
Þ.) The corresponding amplitude (lower left) and phase (lower right) of the experimental transfer ratio
Þ are shown as thin grey lines while the thick black lines represent the corresponding averages for these six stimuli. Examples of
Þ?rx
T2pf
ð
ð Þ~a4~0:55, and
Þ~0:81. The red and blue lines in the lower panels correspond to the average of the amplitude (lower left) and
Tt ð Þ.
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org12 March 2009 | Volume 5 | Issue 3 | e1000328

Page 13

context the impulse response is the additional evoked firing in the
layer-4 population when adding a sharp impulse, ‘d-pulse’, to the
background thalamic input. The solid line represents the impulse
response for a background activity corresponding to the linear
flank of the activation function F4I4
pulse is so small that the circuit remains in the linear regime. The
observed impulse response is biphasic with an excitatory phase
lasting about 15 ms followed by an inhibitory phase lasting up to
50 ms. The dashed line shows the response to a very strong d-pulse
so that the nonlinear regions of the activation function, both the
parabolic and half-wave rectifying parts, are encountered. The
response to this strong excitation is seen to be qualitatively similar
to the linear impulse response. However, the excitatory phase is
somewhat sharper, and the long inhibitory phase is modified by
the half-wave rectification imposed by the requirement of
nonnegative firing rates.
The biphasic thalamocortical impulse-response observed in
Fig. 10 qualitatively resembles the temporal shape (i.e., excitation
followed by suppression) of the ‘impulse-response’ to single whisker
flicks observed in about two thirds of the barrel-cortex neurons in a
single-unit study of Webber and Stanley [33]. The remaining third
of the neurons in this study was found to have a second excitatory
phase following the suppression [33]. It should be noted, however,
that this ‘stimulus-cortical’ impulse-response is different from the
thalamocortical impulse-response extracted here, since the ‘stim-
ulus-cortical’ response also incorporates the processing between
the whisker and the thalamus. A second excitatory phase in the
‘stimulus-cortical’ impulse response could thus be compatible with
ð Þ where the strength of the d-
the observed biphasic thalamocortical impulse-response if the
thalamic response to a single whisker flick has two temporally
distinct positive phases (f.ex., due to cortical feedback onto the
thalamic neurons).
Intracortical model
Layer 4 is generally thought of as the dominant input layer for
sensory activation of cortex. The supragranular population of
layer-2/3 pyramidal neurons appears to receive a dominant input
from the layer-4 population, and the initial stimulus-evoked
response in layer 2/3 appears to largely stem from layer 4 neurons
which in turn are activated by the thalamic input [11,34–41]. We
thus investigated models that could account for the experimentally
extracted population firing-rate of the layer-2/3 population with
the extracted layer-4 population rate as input. We initially
described this activity using a combined feedforward and recurrent
model analog to Eq. (1), i.e.,
r2=3t ð Þ~F2=3
bEfhEf{bIfhIf
ð Þ ? r4
?t ð Þ?
½? t ð Þz
ð
bErhEr{bIrhIr
ðÞ ? r2=3
?
ð10Þ
This form assumes that the interlaminar connections from layer
4 to layer 2/3 are predominantly provided by the excitatory
neurons [38]. Fitting of this form to the experimental data revealed
that the extracted population firing rates can be explained with a
strongly reduced version incorporating only the excitatory
feedforward term in Eq. (10), i.e.,
r2=3t ð Þ~F2=3
bEfhEf? rx
4
??t ð Þ
??
ð11Þ
As before the weight bEfwas set to unity without loss of
generality. Further, the feedforward delay DEf was found to be
very small and consequently set to zero. Table 3 and Fig. 11
(upper row) show the results of fitting this model to the
experimentally extracted layer-2/3 firing rates rx
layer-4 firing rate rx
4as input for experiments 1–3. A first
observation is that the simple model in Eq. (11) accounts
excellently for the experimental data; the error is less than 0.03
for all experiments. The fitted time constants tEfare all very short,
less than 2 ms. The fitted activation functions for experiments 1
and 2 have significant non-linear, i.e., parabolic, contributions
with shorter linear flanks than the corresponding activation
functions for the recurrent thalamocortical model.
A strong feedforward connection between the layer-4 and
layer-2/3 populations is in accordance with previous experi-
mental studies [34–36], and also with the results from the joint
modeling of the present MUA data and the corresponding LFP
data using laminar population analysis (LPA) in Einevoll et al. [11].
However, this LPA analysis also indicated a recurrent interaction
within layer 2/3, in accordance with experimental observations
by Feldmeyer et al. [42] of a substantial connectivity between
layer-2/3 pyramidal neurons. Such a recurrent connection
between excitatory neurons could amplify synchronous feedfor-
ward excitation from layer 4 [42]. In the present modeling study
we find that no such excitatory recurrent terms are needed to
account for the present stimulus-evoked experimental data.
Instead the stronger inputs are amplified in our model by means
of the boosting nonlinearity inherent in the activation function
F2=3:ð Þ, cf. Fig. 11. We therefore investigated to what extent
recurrent connections, i.e., a model of the recurrent form in Eq.
2=3using the
Figure 10. Thalamocortical impulse response for experiment 1.
Impulse response for thalamocortical recurrent model (Eq. 2), i.e.,
additional response due to sharp ‘d-function’-like impulse, ‘d-pulse’, of
extra thalamic firing superimposed on the steady-state thalamic
background activity. The parameter values from fitting to experiment
1 are used (cf. Table 1). The steady-state thalamic firing is set to
corresponds to a steady-state layer-4 input ‘current’ I4~0:02, i.e., on the
linear flank of the layer-4 activation function (cf. Fig. 4). Solid line
corresponds to a linear impulse response, i.e., response to a sufficiently
weak d-pulse so that only the linear part of the activation F4I4
encountered. Dashed line corresponds to a strong d-pulse of extra
thalamic firing resulting in a maximum value of I4~0:89, i.e., far into
the non-linear region of the activation function.
doi:10.1371/journal.pcbi.1000328.g010
ð Þ is
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org13 March 2009 | Volume 5 | Issue 3 | e1000328

Page 14

(10) with bIf~0, improved the fit for the situation with a linear
activation function. In our test runs, it did not. For experiment 1
with linear activation function the error was found to only be
reduced from e~0:051 to e~0:046 when adding recurrent
excitation and inhibition. In comparison, the error for the simple
feedforward model with a non-linear activation function was
found to be only 0.021, cf. Table 3.
We also investigated to what the extent the extracted layer-5
population firing rate can be accounted for by the same type of
firing-rate model with the layer-4 population providing the
stimulus-evoked input [43,44]. Table 3 and Fig. 11 (lower row)
show the results of fitting a model of the type in Eq. (11) to the
experimentally extracted layer-5 firing rates rx
firing rate rx
4as input for experiments 1–3. Again we observed that
the simple feedforward model accounts excellently for the
experimental data with an error of less than 0.02 for all
experiments. The fitted time constants tEf are again short, less
than 3 ms. The most striking difference from the fits of the layer-5
firing rate is the almost linear activation functions. Thus the layer-
5 firing rate appears to be related to the layer-4 firing rate in a very
simple manner for the present experimental situation.
The presently extracted intracortical models have very simple
mathematical forms. For example, for the layer-4 to layer-2/3
transfer the model on integral form can be translated to a simple
5using the layer-4
differential equation using the ‘linear-chain trick’ as described in
Materials and Methods, i.e.,
tEf
dx2=3
dt
~{x2=3zr4t ð Þð12Þ
where the layer-2/3 firing rate is given by r2=3t ð Þ~F2=3x2=3t ð Þ
The same type of differential equation is found for the connection
between layer 4 and layer 5 (with x2=3t ð Þ substituted by x5t ð Þ in
Eq. (12)). In this case the signal transfer is essentially described by a
simple linear differential equations for r5t ð Þ since r5t ð Þ~
F5x5t ð Þ
??.
ðÞ^a5x5t ð Þ.
Intracortical transfer
We next investigated the intracortical transfer functions. The
linear transfer function from the layer-4 population to the layer-2/
3 and layer-5 populations is found from Eq. (11) to be given by
Tn=4v
ð Þ~~ r rnv
ð Þ=~ r rTv
ð Þ~Fn’In0
ðÞ~h hEf v
ð Þ~Fn’In0
1{itEfv
ðÞ
ð13Þ
where n represents 2/3 or 5, and~h hEf v
corresponds to a low-pass filter, i.e., no resonance behavior as for
ð Þ is found in Eq. (39). This
Figure 11. Fits of intracortical model. Illustration of fits of the intracortical model (Eq. 11) to data from experiments 1–3. Each dot corresponds to
the experimentally measured layer-2/3 firing rate rx
2=3ti
ð Þ (upper row) or layer-5 firing rate rx
the fitted model values of I2=3ti
ð Þ or I5ti
stimulus onset (for all 27 stimuli). These data points show the activity prior to any stimulus-evoked thalamic or cortical firing and represent
background activity. The solid green curves correspond to the fitted model activation functions F2=3I2=3
respectively.
doi:10.1371/journal.pcbi.1000328.g011
5ti
ð Þ (lower row) at specific time points tiplotted against
ð Þ, respectively. The red dots are the corresponding experimental data points taken from the first 5 ms after
??
(upper row) and F5I5
ð Þ (lower row),
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org 14March 2009 | Volume 5 | Issue 3 | e1000328

Page 15

the recurrent thalamocortical model in Eq. (9). Further, the
impulse-response function is the simple (monophasic) exponen-
tially decaying function, contrasting the biphasic thalamocortical
impulse-response functions seen in Fig. 10.
In Fig. 12 we compare predictions for the transfer ratio from
this simple linear feedforward model using the fitted parameters
from Table 3 with the corresponding experimental result
rx
nv
ð Þ (n~2=3 or 5) for six of the stimulus conditions
providing the strongest cortical responses in experiment 1
(a3,t1;a3,t2;a3,t3;a2,t1;a2,t2;a2,t3). For the layer-5/layer-4
transfer ratio (lower row in Fig. 12) we see an excellent agreement
between experiments and linear theory. This is as expected since
the fitted layer-5 activity functions F5:ð Þ are essentially linear (cf.
Fig. 11). For the transfer ratio between layer 2/3 and layer 4 a
more substantial deviation between the experimental and linear-
model transfer rations is observed, in particular a slightly growing
transfer-ratio amplitude for large frequencies. This deviation
presumably reflects the non-linearities of the layer-2/3 activity
functions F2=3:ð Þ, because the same effect is also observed for the
transfer ratio between the best model fit r2=3and rx
upper left panel of Fig. 12).
ð Þ?rx
4v
4(cf. red line in
Discussion
A main objective of the present paper is the description and
application of a new method for extraction of population firing-
rate models for thalamocortical and intracortical signal transfer on
the basis of trial-averaged data from simultaneous cortical
recordings with laminar multielectrodes in a rat barrel column
and single-electrode recordings in the homologous thalamic
barreloid. Below we first discuss the results from the model
estimation for both the thalamocortical and the intracortical signal
transfer, followed by a discussion of the model estimation method
itself.
Thalamocortical models
For the thalamocortical signal transfer our investigations clearly
identifies a model with (1) fast feedforward excitation, (2) a slower
predominantly inhibitory process mediated by recurrent (within
layer 4) and/or feedforward interactions (from thalamus), and (3) a
mixed linear-parabolic activation function. The relative impor-
tance of the feedforward compared to the recurrent connections is
more difficult to determine and has been a subject of significant
interest [7–9,16,45]. Here we have compared two alternative
models: (i) a purely feedforward thalamocortical model with
feedforward excitation and inhibition and (ii) a recurrent model
with feedforward excitation, recurrent excitation and recurrent
inhibition (but no feedforward inhibition). In reality the inhibitory
layer-4 neurons receive excitatory inputs both from thalamic
neurons and other layer-4 neurons, and the inhibition felt by the
excitatory layer-4 neurons will thus be both ‘feedforward’ and
‘recurrent’; the different models thus explore what part appears to
dominate in the present experimental situation.
Our results suggest that the model with recurrent inhibition
better accounts for the present experimental data than the model
with feedforward inhibition, i.e., that the layer-4 interneurons
providing the inhibition of the layer-4 excitatory neurons are in
turn mainly driven by excitation from layer-4 neurons. Since the
relative strength of recurrent effects compared to feedforward
effects appears to increase with increasing thalamic stimuli [19],
this might indicate that the present fits to the experimental data is
dominated by ‘strong’ inputs. Alternatively, it may be that a strong
feedforward inhibition is also present, but that it is strongly
overlapping in time with the feedforward excitation. Then the
present modeling scheme and experimental data are unable to
separate them and instead lump the effect of this inhibition
together with the feedforward excitation.
An interesting result from the present investigation is that we
find a similar thalamocortical population model as Pinto et al.
[8,9]: the recurrent model on integral form in Eq. (2) can be mapped
to a set of two differential equations (Eqs. 4–5) structurally similar
to the model suggested and investigated by Pinto and coworkers.
The correspondence between the behavior of these models is
demonstrated by the phase-plane plots in Fig. 5, which exhibits the
same qualitative features as the analogous phase-plane plots in
Ref. [9]. This close correspondence is striking since the methods
used to derive the models are very different. Pinto et al. chose the
model to account for a large collection of single-unit PSTHs
recorded with sharp electrodes from numerous animals. By
contrast, our model parameters were extracted based on
simultaneous thalamic and cortical recordings from individual
animals. As multielectrode arrays allow for convenient simulta-
neous recordings from all layers of the cerebral cortex, even at
more than one cortical location [10], the proposed modeling
approach scales more readily to complex networks involving
multiple, spatially separated, neuronal populations.
Intracortical model
The presently identified intracortical model for the connection
between the layer-4 and layer-2/3 populations has a very simple
mathematical form including only a single feedforward term
(Eqs. 11–12), but was nevertheless found to account excellently for
the experimental data, cf. Fig. 11. Such a strong feedforward
connection between these populations is in accordance with
previous experimental studies [34–36]. An experimental study by
Feldmeyer et al. [42] found substantial recurrent connectivity
between layer-2/3 pyramidal neurons, but no such excitatory
recurrent terms were needed to account for the present stimulus-
evoked experimental data. Compared to the thalamocortical
Table 3. Fitted model parameters for intracortical models.
L4RL2/3 Exp. 1Exp. 2 Exp. 3
tEf (ms)1.21.21.8
a2=3
0.5160.010.450.40
b2=3
0.4960.010.450.89
I{
2=3
20.03
20.03
20.02
I?
2=3
0.1360.020.130.27
error (e)
0.02140.01410.0279
L4RL5 Exp. 1Exp. 2 Exp. 3
tEf (ms) 3.02.11.4
a5
0.960.8660.01 0.73
b5
0.240.100.27
I{
5
I?
5
20.009
20.003
20.004
0.0030.02460.043 0.26860.004
error (e)
0.0195 0.0193 0.0167
Resulting optimized parameters for the intracortical network model (Eq. 11) for
experiments 1–3. Upper row: Layer-2/3 firing rate predicted from layer-4 input.
Lower row: Layer-5 firing rate predicted from layer-4 input (where the
subscripts ‘2/3’ are replaced by ‘5’ in Eq. (11)). The listed parameter values
correspond to the mean of estimated parameters from 25 selected models
giving essentially the same error (see Materials and Methods). The standard
deviations are only listed if they exceed the last digit of the mean.
doi:10.1371/journal.pcbi.1000328.t003
Neural Network Model Estimation
PLoS Computational Biology | www.ploscompbiol.org15 March 2009 | Volume 5 | Issue 3 | e1000328