An introductory review of information theory in the context of computational neuroscience.

Institute for Telecommunications Research, University of South Australia.
Biological Cybernetics (Impact Factor: 1.93). 07/2011; 105(1):55-70. DOI: 10.1007/s00422-011-0451-9
Source: DBLP

ABSTRACT This article introduces several fundamental concepts in information theory from the perspective of their origins in engineering. Understanding such concepts is important in neuroscience for two reasons. Simply applying formulae from information theory without understanding the assumptions behind their definitions can lead to erroneous results and conclusions. Furthermore, this century will see a convergence of information theory and neuroscience; information theory will expand its foundations to incorporate more comprehensively biological processes thereby helping reveal how neuronal networks achieve their remarkable information processing abilities.

1 Bookmark
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Although several measurements and analyses are done to support the idea that the brain is energy-optimized, there is one disturbing, contradictory observation: In theory, computation limited by thermal noise can occur as cheaply as ~$2.9\cdot 10^{-21}$ joules per bit (kTln2). Unfortunately, for a neuron, the ostensible discrepancy from this minimum is startling - ignoring inhibition the discrepancy is $10^6$ times this amount and taking inhibition into account $1.4\cdot 10^8$. Here we point out that what has been defined as neural computation is actually a combination of computation and neural communication: the communication costs, transmission from each excitatory postsynaptic activation to the S4-gating-charges of the fast Na+ channels of the initial segment (fNa's), dominate the joule-costs. Making this distinction between communication to the initial segment and computation at the initial segment (i.e., adding up of the activated fNa's) implies that the size of the average synaptic event reaching the fNa's is the size of the standard deviation of the thermal noise, $(kT)^{1/2}$. Moreover, when computation is defined as the addition of activated fNa's, a biophysically plausible mechanism produces the appropriate number of bits for the cost to hit the minimum joules. This mechanism, requiring something like the electrical engineer's equalizer (not much more than the action potential generating conductances), only operates just at or just below threshold. This active filter modifies the last few synaptic excitations, providing barely enough energy to transport the last set of mostly sub-threshold gating charges. That is, the last, threshold-achieving S4-subunit activation requires an energy that matches the information being provided by the last few synaptic events, a ratio that is kTln2 joules per bit.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Oscillatory activity is ubiquitous in nervous systems, with solid evidence that synchronisation mechanisms underpin cognitive processes. Nevertheless, its informational content and relationship with behaviour are still to be fully understood. In addition, cognitive systems cannot be properly appreciated without taking into account brain-body- environment interactions. In this paper, we developed a model based on the Kuramoto Model of coupled phase oscillators to explore the role of neural synchronisation in the performance of a simulated robotic agent in two different minimally cognitive tasks. We show that there is a statistically significant difference in performance and evolvability depending on the synchronisation regime of the network. In both tasks, a combination of information flow and dynamical analyses show that networks with a definite, but not too strong, propensity for synchronisation are more able to reconfigure, to organise themselves functionally and to adapt to different behavioural conditions. The results highlight the asymmetry of information flow and its behavioural correspondence. Importantly, it also shows that neural synchronisation dynamics, when suitably flexible and reconfigurable, can generate minimally cognitive embodied behaviour.
    Biological Cybernetics 07/2012; 106(6-7):407-27. · 1.93 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: We calculate and analyze the information capacity-achieving conditions and their approximations in a simple neuronal system. The input-output properties of individual neurons are described by an empirical stimulus-response relationship and the metabolic cost of neuronal activity is taken into account. The exact (numerical) results are compared with a popular "low-noise" approximation method which employs the concepts of parameter estimation theory. We show, that the approximate method gives reliable results only in the case of significantly low response variability. By employing specialized numerical procedures we demonstrate, that optimal information transfer can be near-achieved by a number of different input distributions. It implies that the precise structure of the capacity-achieving input is of lesser importance than the value of capacity. Finally, we illustrate on an example that an innocuously looking stimulus-response relationship may lead to a problematic interpretation of the obtained Fisher information values.
    Bio Systems 04/2013; · 1.27 Impact Factor

Full-text (2 Sources)

Available from
Jun 4, 2014