[Show abstract][Hide abstract] ABSTRACT: Experimental evidence suggests that spike timing might be used by neurons to process and store information. Unfortunately, the mathematical analysis of recurrent networks with spiking neurons is highly non trivial. Most analytical studies have therefore focused on rate-based models, whereas spiking models tend to be studied numerically. In order to bridge this gap, we propose an effective spiking neuron model which still allows for the application of non-equilibrium statistical mechanical techniques. The model is flexible and its parameters can be adjusted in order to match real data. We analyze the population dynamics in the simple case of constant excitatory synapses.
[Show abstract][Hide abstract] ABSTRACT: At very short timescales neuronal spike trains may be compared to binary streams where each neuron gives at most one spike per bin and therefore its state can be described by a binary variable. Time-averaged activity like the mean firing rate can be generally used on longer timescales to describe the dynamics; nevertheless, enlarging the space of the possible states up to the continuum may seriously bias the true statistics if the sampling is not accurate. We propose a simple transformation on binary variables which allows us to fix the dimensionality of the space to sample and to vary the temporal resolution of the analysis. For each time length interactions among simultaneously recorded neurons are evaluated using log-linear models. We illustrate how to use this method by analysing two different sets of data, recorded respectively in the temporal cortex of freely moving rats and in the inferotemporal cortex of behaving monkeys engaged in a visual fixation task. A detailed study of the interactions is provided for both samples. In both datasets we find that some assemblies share robust interactions, invariant at different time lengths, while others cooperate only at delimited time resolutions, yet the size of the samples is too small to allow an unbiased estimate of all possible interactions. We conclude that an extensive application of our method to larger samples of data, together with the development of techniques to correct the bias in the estimate of the coefficients, would provide significant information about the structure of the interactions in populations of neurons.
Network Computation in Neural Systems 03/2004; 15(1):13-28. DOI:10.1088/0954-898X/15/1/002 · 0.87 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Recent studies have explored theoretically the ability of populations of neurons to carry information about a set of stimuli, both in the case of purely discrete or purely continuous stimuli, and in the case of multidimensional continuous angular and discrete correlates, in the presence of additional quenched disorder in the distribution. An analytical expression for the mutual information has been obtained in the limit of large noise by means of the replica trick. Here, we show that the same results can actually be obtained in most cases without the use of replicas, by means of a much simpler expansion of the logarithm. Fitting the theoretical model to real neuronal data, we show that the introduction of correlations in the quenched disorder improves the fit, suggesting a possible role of signal correlations-actually detected in real data-in a redundant code. We show that even in the more difficult analysis of the asymptotic regime, an explicit expression for the mutual information can be obtained without resorting to the replica trick despite the presence of quenched disorder, both with a Gaussian and with a more realistic thresholded-Gaussian model. When the stimuli are mixed continuous and discrete, we find that with both models the information seem to grow logarithmically to infinity with the number of neurons and with the inverse of the noise, even though the exact general dependence cannot be derived explicitly for the thresholded-Gaussian model. In the large noise limit, lower values of information were obtained with the thresholded-Gaussian model, for a fixed value of the noise and of the population size. On the contrary, in the asymptotic regime, with very low values of the noise, a lower information value is obtained with the Gaussian model.
[Show abstract][Hide abstract] ABSTRACT: We present four `case study' examples of solvable problems in the theory of recurrent neural networks, which are relevant to our understanding of information processing in the brain, but which are also interesting from a purely statistical mechanical point of view, even at the level of simple models (which helps in stimulating interdisciplinary work). The examples concern issues in network dynamics, network connectivity, spike timing and synaptic plasticity.
[Show abstract][Hide abstract] ABSTRACT: We present four 'case study' examples of solvable problems in the theory of recurrent neural networks, which are relevant to our understanding of information processing in the brain, but which are also interesting from a purely statistical mechanical point of view, even at the level of simple models (which helps in stimulating interdisciplinary work). The examples concern issues in network dynamics, network connectivity, spike timing and synaptic plasticity.
Reviews in the neurosciences 02/2003; 14(1-2):181-93. DOI:10.1515/REVNEURO.2003.14.1-2.181 · 3.33 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: In a recent work we have introduced a novel approach to study the effect of weak nonlinearity in the transfer function on the information transmitted by an analogue channel, by means of a perturbative diagrammatic expansion. We extend here the analysis to all orders in perturbation theory, which allows us to release any constraint concerning the magnitude of the expansion parameter and to establish the rules to calculate easily the contribution at any order. As an example we explicitly compute the information up to the second order in nonlinearity, in presence of random Gaussian connectivity and in the limit when the output noise is not small. We analyze the first and second order contributions to the mutual information as a function of the nonlinearity and as a function of the number of output units. We believe that an extensive application of our method via the analysis of the different contributions at distinct orders might be able to fill a gap between well-known analytical results obtained for linear channels and nontrivial treatments which are required to study highly nonlinear channels.
International Journal of Modern Physics B 09/2002; 16:3527-3543. DOI:10.1142/S0217979202012049 · 0.94 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: In a previous paper we have evaluated analytically the mutual information between the firing rates of N independent units and a set of multidimensional continuous and discrete stimuli, for a finite population size and in the limit of large noise. Here, we extend the analysis to the case of two interconnected populations, where input units activate output ones via Gaussian weights and a threshold linear transfer function. We evaluate the information carried by a population of M output units, again about continuous and discrete correlates. The mutual information is evaluated solving saddle-point equations under the assumption of replica symmetry, a method that, by taking into account only the term linear in N of the input information, is equivalent to assuming the noise to be large. Within this limitation, we analyze the dependence of the information on the ratio M/N, on the selectivity of the input units and on the level of the output noise. We show analytically, and confirm numerically, that in the limit of a linear transfer function and of a small ratio between output and input noise, the output information approaches asymptotically the information carried in input. Finally, we show that the information loss in output does not depend much on the structure of the stimulus, whether purely continuous, purely discrete or mixed, but only on the position of the threshold nonlinearity, and on the ratio between input and output noise.
[Show abstract][Hide abstract] ABSTRACT: In a recent study, the initial rise of the mutual information between the firing rates of N neurons and a set of p discrete stimuli has been analytically evaluated, under the assumption that neurons fire independently of one another to each stimulus and that each conditional distribution of firing rates is Gaussian. Yet real stimuli or behavioral correlates are high dimensional, with both discrete and continuously varying features. Moreover, the Gaussian approximation implies negative firing rates, which is biologically implausible. Here, we generalize the analysis to the case where the stimulus or behavioral correlate has both a discrete and a continuous dimension, like orientation and shape could be in a visual stimulus, or type and direction in a motor action. The functional relationship between the firing patterns and the continuous correlate is expressed through the tuning curve of the neuron, using two different parameters to modulate its width and its flatness. In the case of large noise, we evaluate the mutual information up to the quadratic approximation as a function of population size. We also show that in the limit of large N and assuming that neurons can discriminate between continuous values with a resolution Delta(theta), the mutual information grows to infinity like ln(1/Delta(theta)) when Delta(theta) goes to zero. Then we consider a more realistic distribution of firing rates, truncated at zero, and we prove that the resulting correction, with respect to the Gaussian firing rates, can be expressed simply as a renormalization of the noise parameter. Finally, we demonstrate the effect of averaging the distribution across the discrete dimension, evaluating the mutual information only with respect to the continuously varying correlate.
[Show abstract][Hide abstract] ABSTRACT: A previous study revealed that neuronal activity in primary motor cortex (MI) and supplementary motor area (SMA) of the monkey depends both on which arm(s) moved and on the direction of movement. At the level of single cells, no differences were found between the areas in the information conveyed about each correlate. We constructed pseudosimultaneous response vectors and applied a decoding algorithm to quantify differences at a population level. We found that, on average, samples of 20 MI units carried less information about both movement type and direction than SMA units in a time window of 500 ms across the movement onset; a more detailed temporal analysis has revealed that SMA precedes M1 in motor planning and execution and that along the trial M1 cells carry as much information about direction as SMA cells.
[Show abstract][Hide abstract] ABSTRACT: We evaluate the mutual information between the input and the output of a two layer network in the case of a noisy and nonlinear analogue channel. In the case where the nonlinearity is small with respect to the variability in the noise, we derive an exact expression for the contribution to the mutual information given by the nonlinear term in first order of perturbation theory. Finally we show how the calculation can be simplified by means of a diagrammatic expansion. Our results suggest that the use of perturbation theories applied to neural systems might give an insight on the contribution of nonlinearities to the information transmission and in general to the neuronal dynamics.
International Journal of Modern Physics B 01/2001; 15(3):281-295. DOI:10.1142/S0217979201004496 · 0.94 Impact Factor