Connection strategies in neocortical networks.

Full text

Page 1

Connection strategies in neocortical networks
A. Herzog1, K. Kube1, B. Michaelis1, A. D. de Lima2, T. Voigt2 ∗
1- Otto-von-Guericke-University Magdeburg
Institute of Electronics, Signal Processing and Communications
P.O. Box 4120, D-39016 Magdeburg - Germany
2- Otto-von-Guericke-University Magdeburg
Institute of Physiology
Leipziger Str. 44, D-39120 Magdeburg - Germany
Abstract.
This study considers the impact of different connection strategies in de-
veloping neocortical networks. An adequate connectivity is a requisite for
synaptogenesis and the development of synchronous oscillatory network
activity during maturation of cortical networks. In a defined time win-
dow early in development neurites have to grow out and connect to other
neurons. Based on morphological observations we postulate that the un-
derlying mechanism differs from common strategies of unspecific global or
small world strategies. We show here that a displaced local connection
mode is a very effective approach to connect neurons with minimal costs.
1Introduction
Spontaneous large-scale wave-like activity during early development of the neo-
cortex can be observed in cell cultures [1], where networks show similar develop-
ment as in the brain, using imaging techniques and electrophysiological patch-
clamp-measurements. The synchronous activity appear in culture at the begin-
ning of the second week and eventually includes the entire neuronal population
about 1 wk later [2]. In standard culture conditions neurons become electrically
active spontaneously and independently. Next neurons connect to each other,
form synapses, and begin to burst simultaneously, discharging collectively about
once per minute. Time histograms show that the portion of synchronously firing
neurons increases with time. Because non-active neurons die before the end of
the second week in vitro [3], the participation in synchronous oscillatory activity
seems to play an important role in the early development of the mammalian
cortex.
In [4] we have investigated conditions and parameters for the emergence of
oscillatory network activity by accumulated activity of distinct single cells in
intrinsic driven networks. Tabak et al. [5]described a global model to transform
spontaneous activity in random connected networks in episodic events of syn-
chronized activity. Like Tabak’s model, many population models consist only
of unspecific random connections (overview in [6]). Maeda et al. [7] showed in
physiological experiments that the source of synchronized bursts change on every
stimulation episode and there is a time delay from source of burst to distance
∗Supported by the federal state of Saxony-Anhalt FKZ Xn3590C/0305M.
ESANN'2006 proceedings - European Symposium on Artificial Neural Networks
Bruges (Belgium), 26-28 April 2006, d-side publi., ISBN 2-930307-06-4.
215

Page 2

neurons. This behaviour is coherent with a locally connected network. On the
other hand, ’small world’ networks [8] contain a majority of local connections
between cells, but a few of the connections are long distance connections. This
content of long-distance connections can impact the synchronization and waves
in epilepsy [9].
The correlation between neuritic growth dynamics and the development of
effective network connectivity was recently highlighted [10]. To build a function-
ally interconnected network, single neurons have to become interconnected in a
defined time window, that is limited by the emergence of synchronous network
activity. The biologically most plausible mechanism is not a global random or a
small world one, and simple local connections do not have the capacity to con-
nect a network in a way that it can easily perform synchronized events. Latham
[11] introduced a local connection type which build a cluster of connections at
a distance of the source neuron. This connection mode resembles the connec-
tion mode of neurons in vivo [1]. The mechanism to build these connections are
very similar to local connections by a probability map with falling probability
from center, but here the probability map is displaced. The center is not the
source cell itself but a branching point in a specified distance from neuron. Here
we analyse the displaced connection strategy and compare it with small world
topologies.
2Methods
A network section of n = 400 neurons was assembled, arranged on the planar
area of 1 x 1 mm2, a simple representation reproducing culture dish plating.
Assuming the network structure to be homogenous (and prevent overlying of
neuronal positions), these cells positions were de-clustered, scattered uniformly
over the section’s area using the neural-gas-algorithm according to [12], which
minimizes the spatial entropy of the position’s distribution.
We compare two connection methods. The small world connections starts by
pure local connections (Fig. 1). Cells connect only in the immediate neighbour-
hood. The probability to connect a cell a on position xawith a cell b on position
xbdirectly depends on the Euclidean distance dab= |xa− xb| and is modulated
by Gaussian function pl
(ρ) of connections breaks and reconnects as global unspecific connections. The
probability of these connections is independent from relative position of source
and destination cell and constant to all pairs of cells pg
Based on work of Latham [11] and morphological observations [[2], [1], [10]]
we defined an advanced type of local connections with a displaced local probabil-
ity map (displaced local connections) . In the early development of network the
neurites (axons) first grow and then at a certain distance from the cell body build
connections in a local area (see Fig. 1 D). In that way neurons may protect them-
selves against formation of small feedback loops and self-excitation. Each axon
grows in a random direction y. In a distance of ldispit builds a local probability
map in same way as on pure local connections: pl
ab= pme−dab/σ2. In small world mode a defined part
a,b= pm= const.
a,b= pme−dc
a,b/σ2but distances
ESANN'2006 proceedings - European Symposium on Artificial Neural Networks
Bruges (Belgium), 26-28 April 2006, d-side publi., ISBN 2-930307-06-4.
216

Page 3

are defined from the displaced cluster center: dc
cluster center approximates the branching of axons. The small world network by
Watts [8] allows a floating transition from local to unspecific global connections
by defining the fraction of global connections. The displacement of probability
map allows a floating transition from pure local connections to a more natural
connections mode by defining the branching distance of axon.
a,b= |(xa+ ldisp∗ y) − xb|. The
AD
B
C
ldisp
Fig. 1: Connection methods: (A) Local (ρ = 0), (B) Global (ρ = 1.0) (C) Small
world (ρ = 0.2) (D) Displaced local (ldisp)
Network generation and analysis are done by c++ programs on a linux pc-
cluster (42 nodes). Additional simulations are done by neuron [13] program.
3 Experiments and Results
3.1Network properties
Networks were generated using two connection modes. The parameters (see Fig.
2) of probability maps for different connection types are set in a way that the
degree of neurons, i.e. the number of incoming and outgoing connections per
neuron, is nearly constant. The total connection length per neuron is estimated
by the summed length of its outgoing connections. In pure local (ρ = 0) and
in global (ρ = 1.0) connections the total connection length is simply the sum of
Euclidian distances to destination cells. On displaced local connection a branch-
ing point of axon is defined. The total connection length result here as the sum
of the Euclidian distance to the branching point and the Euclidian distances
from branching point to destination cells. To check the overall connectivity we
measured the minimal network distance among neurons. We define the mini-
mal network distance as the minimal number of synaptic connections between
two neurons. The minimal network distance between neurons is important for
the capacity of the network to synchronize activity. We show in [14] that the
shortcuts in small world networks can stabilize synchrony.
The connection length per neuron is small on local connection method com-
pared to the random unspecific method (see Fig. 2 A-B). A moderate proportion
of global (long range) connections (ρ = 0.034) minimally raises the connection
length of local networks (Fig. 2 C). If we compare minimal network distance, the
opposite effect is shown. In locally connected networks neurons are separated by
a relatively large number of nodes (Fig. 3 ). Information transmission through
ESANN'2006 proceedings - European Symposium on Artificial Neural Networks
Bruges (Belgium), 26-28 April 2006, d-side publi., ISBN 2-930307-06-4.
217

Page 4

relative number of neurons
total connection length in µm
01000
2000
0
0.05
0.1
0.15
0.2
0
4000 10000
16000
0.05
0.1
0.15
0.2
0.25
0.3
0 500 1000 1500 2000
0.05
0.1
0.15
0.2
500 10002000
0.05
0.1
0.15
0.2
B
C
A
D
Fig. 2: Histograms of total connection length per neuron. Degree mean = 12.5,
standarddeviation = 4. Connection methods: (A) Local (ρ = 0, σ = 100µm,
pm = 1.0), (B) Global (ρ = 1.0, pm = 0.06) (C) Moderate small world (ρ =
0.034) (D) Displaced local (ldisp= 300µm, σ = 100µm, pm= 1.0)
the network is slow and it is hard to synchronize neuronal activity. On other
side neurons in networks with unspecific random connections are separated by
few nodes (max.= 4), what allows a very fast synchronization of activity with
small phase lags. Even a moderate number of long range connections decreases
the minimal network distance dramatically (Fig. 3 A-C). Similarly, in a network
with displaced local connections the total connection length increases minimally
with the additional length from source cell to the axonal branching point (Fig.
2 D). Again, the minimal network distance is considerably smaller than in lo-
cally connected networks (Fig. 3 D). The additional cost is relative small and
disappear by a larger number of postsynaptic neurons.
0510
0
0.02
0.06
0.1
0.14
0.18 A
5
0
0.1
0.2
0.3
0.4
0.5
0.6 B
0
0.1
0.2
0.3
0.4
0.5
1
3
5
C
1
35
0
0.1
0.2
0.3
0.4
0.5 D
13
relative number of neurons
minimal network distance in nodes
Fig. 3: Histograms of minimal network distance.
Connection methods: (A) Local (B) Global (C) Small world (D) Displaced local
Parameters as in Fig.2.
3.2 Optimal displacement
In summary this experiment shows that local connection with displaced proba-
bility map is an effective way to reach a good connectivity with minimal costs.
Fig 4 (left) shows the minimal network distance and the connection length in
relation to the distance between source neuron and axonal branching point in
displaced local connections. In biology connection length is limited by energy
ESANN'2006 proceedings - European Symposium on Artificial Neural Networks
Bruges (Belgium), 26-28 April 2006, d-side publi., ISBN 2-930307-06-4.
218

Page 5

costs. If we assumed the comsumption of network as average of network distance
times average of connection length per neuron, we found a minimum which may
be considered as the optimal displacement for this placement area and neuron
degree (Fig. 4 (right)).
0100 200
displacement in µm
300400500
2.6
2.8
3
3.2
3.4
3.6
3.8
4
4.2
4.4
network distance in nodes
90
100
110
120
130
140
150
Avarage output length per neuron in µm
network distance
output lenght
0100200
displacement in µm
300400500
0.8
0.9
1
relative cost
Fig. 4: Variation of Displacement (ldisp= 0..600µm, other parameters as in Fig.
2). left: network properties; right: minimum cost at 330µm.
4Discussion and Conclusion
This paper shows that local network connection mode with displaced center of
probability map is an effective way to build recurrent networks. The obtained
network properties are similar to the very effective small world networks. We
proved the rhythmic activity capacity in all network by simulations with the
same parameter as in [14]. The dynamics of networks with displaced local con-
nections are not different from those networks with small world connections. But
it is a local mechanism with limited connection length. During development the
networks must be connected as fast as possible to start rhythmic bursting. Neu-
ritic growth in young networks is thus limited to a permissive time window, what
may bias the network to a local connection mode.
Comparing local with global and small world methods we also have to con-
sider the different effects of scaling the network. On local methods the total
connection length per neuron is constant, but the minimal network distance
grows with the placement area. On global and small world methods the min-
imal network distance between neurons do not depend on placement area but
only on number of neurons. But on other hand the average and the maximal
connection length is dependent on placement area. So local connections with
displaced probability map is very interesting to interconnect neurons in small
areas. Larger areas can be overlayed by a global network. To combine it, some
local connections can be reconnected in the same way as in small world net-
works. In young network there are hints that a special type of neurons (large
GABA, note that GABA is excitatory at this period of maturation) builds an
ESANN'2006 proceedings - European Symposium on Artificial Neural Networks
Bruges (Belgium), 26-28 April 2006, d-side publi., ISBN 2-930307-06-4.
219

Page 6

overlay network and synchronizes large arrays of neurons [1]. This result may
also be of interest for hardware realizations of artificial neural networks. Here
local connections can be wired directly on chips, but global connection must be
designed addressable by a router [15]. Displaced maps can be established by
first delivering one output signal by a router to a cluster center anywere on the
chip and spread it from there by wiring. This may reduces the number of routed
global connections enormously.
References
[1] Thomas Voigt, Thoralf Opitz, and Ana D. de Lima. Synchronous oscillatory activity
in immature cortical network is driven by gabaergic preplate neurons. The Journal of
Neuroscience, 21(22):895–905, Nov. 15 2001.
[2] Thoralf Opitz, Ans D. De Lima, and Voigt Thomas. Spontaneous development of syn-
chronous oscillatory activity during maturation of cortical networks in vitro. Journal of
Neurophysiology, 88:2196–2206, 2002.
[3] Thomas Voigt, H Baier, and Ana Dolabela de Lima. Synchronization of calcium activity
promotes survival of individual rat neocortical neurons in early development. European
Journal of Neuroscience, 9(18):990–999, 1997.
[4] K Kube, A. Herzog, V. Spravedlyvyy, B. Michaelis, T. Opitz, A. de Lima, and T. Voigt.
Modelling of biologically plausible excitatory networks: Emergence and modulation of
neural synchrony. In A. Verleysen, editor, Proceedings of the ESANN 2004 - European
Symposium on Artificial Neural Networks, pages 379–384. d-side publishings, 4 2004.
[5] Joel Tabak, Walter Senn, Michael J. O´Donovan, and John Rinzel. Modeling of sponta-
neous activity in developing spinal cord using activity-dependent depression in an exci-
tatory network. The Journal of Neuroscience, pages 3041–3056, 2000.
[6] Wulfram Gerstner and Werner Kistler. Spiking Neuron Models. Cambridge University
Press, 2002.
[7] Eisaku Maeda, Hugh P. C. Robinson, and Akio Kawana. The mechanisms of generation
and propagation of synchronized bursting in developing networks of cortical neurons.
Journal of Neuroscience, 10(15):6834–6845, October 1995.
[8] DJ Watts and SH Strogatz.
393:440–442, 1998.
Collective dynamics of ’small-world’ networks.Nature,
[9] Theoden I. Netoff, Robert Clewley, Scott Arno, Tara Keck, and John A. White. Epilepsy
in small-world networks.The Journal of Neuroscience, 37(24):8075–8083, September
2004.
[10] Thomas Voigt, Thoralf Opitz, and Ana Dolabela de Lima. Activation of early silent
synapses by spontaneous synchronous network activity limits the range of neocortical
connections. Journal of Neuroscience, 25(18):4605–4615, May 2005.
[11] P.E. Latham, B. J. Richmond, P. G. Nelson, and S. Nirenberg. Intrinsic dynamics in
neuronal networks. i. theory. Journal of Neurophysiology, pages 808 – 827, Feb 2000.
[12] Fritzke B. Vektorbasierte Neuronale Netze. Habilitation Thesis, Universitaet Erlangen-
Nuernberg. Shaker Verlag Aachen, 1998.
[13] M.L. Hines and N.T. Carnevale. Neuron: a tool for neuroscientists. The Neuroscientist,
pages pp. 123–135, 2001.
[14] K. Kube, A. Herzog, B. Michaelis, A. Al-Hamadi, A. de Lima, and T. Voigt. Spike-
timing-dependent-plasticity in small world networks. In Proceedings of the ESANN 2005
- European Symposium on Artificial Neural Networks, pages 601–606, 2005.
[15] D Matolin, J Schreiter, and Sch¨ uffny R. Simulation and implementation of an analog vlsi
pulse-coupled neural network for image segmentation. In Proc. of the 47th International
Midwest Symposium on Circuits and Systems, Hiroshima, pages 397–400, 2004.
ESANN'2006 proceedings - European Symposium on Artificial Neural Networks
Bruges (Belgium), 26-28 April 2006, d-side publi., ISBN 2-930307-06-4.
220