An introductory review of information theory in the context of computational neuroscience

Institute for Telecommunications Research, University of South Australia.
Biological Cybernetics (Impact Factor: 1.71). 07/2011; 105(1):55-70. DOI: 10.1007/s00422-011-0451-9
Source: DBLP


This article introduces several fundamental concepts in information theory from the perspective of their origins in engineering. Understanding such concepts is important in neuroscience for two reasons. Simply applying formulae from information theory without understanding the assumptions behind their definitions can lead to erroneous results and conclusions. Furthermore, this century will see a convergence of information theory and neuroscience; information theory will expand its foundations to incorporate more comprehensively biological processes thereby helping reveal how neuronal networks achieve their remarkable information processing abilities.

Download full-text


Available from: Mark D Mcdonnell
  • Source
    • "The problem of information processing and transmission in the brain is historically one of the most intensively studied topics in neurosciences [2, 10, 39]. Frequently, the theoretical approach to this problem relies on the methods of information theory [14] with emphasis on channel capacity as the ultimate fidelity criterion [32, 35, 41, 48]. The operational interpretation of channel capacity relies on an essentially digital transmission protocol (not necessarily binary) and a special encoding-decoding setup known as the separation assumption [14, 42] . "
    [Show abstract] [Hide abstract]
    ABSTRACT: Recently, it has been suggested that certain neurons with Poisso-nian spiking statistics may communicate by discontinuously switching between two levels of firing intensity. Such a situation resembles in many ways the optimal information transmission protocol for the continuous-time Poisson channel known from information theory. In this contribution we employ the classical information-theoretic results to analyze the efficiency of such a transmission from different perspectives, emphasising the neurobiological viewpoint. We address both the ultimate limits, in terms of the information capacity under metabolic cost constraints, and the achievable bounds on performance at rates below capacity with fixed decoding error probability. In doing so we discuss optimal values of experimentally measurable quantities that can be compared with the actual neuronal recordings in a future effort.
    Full-text · Article · Jun 2016 · Mathematical Biosciences and Engineering
  • Source
    • "sensory and motor patterns may vary over time). Transfer Entropy, in this scenario, is suggested as a suitable and robust information theoretic tool (Lungarella et al., 2007b,a), and has also been applied to investigate real neural assemblies and other neuroscience problems (Borst & Theunissen, 1999; Gourévitch & Eggermont, 2007; Buehlmann & Deco, 2010; McDonnell et al., 2011; Vicente et al., 2011); it will, thus, be used in our analysis. The paper is organised as follows: the next section presents some theoretical background to the main concepts explored in this work: oscillators and synchronisation, focusing on the Kuramoto Model, evolutionary robotics (ER) and the two minimally cognitive tasks studied, and Information Theory in an agent-environment context, describing Transfer Entropy. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Oscillatory activity is ubiquitous in nervous systems, with solid evidence that synchronisation mechanisms underpin cognitive processes. Nevertheless, its informational content and relationship with behaviour are still to be fully understood. In addition, cognitive systems cannot be properly appreciated without taking into account brain-body- environment interactions. In this paper, we developed a model based on the Kuramoto Model of coupled phase oscillators to explore the role of neural synchronisation in the performance of a simulated robotic agent in two different minimally cognitive tasks. We show that there is a statistically significant difference in performance and evolvability depending on the synchronisation regime of the network. In both tasks, a combination of information flow and dynamical analyses show that networks with a definite, but not too strong, propensity for synchronisation are more able to reconfigure, to organise themselves functionally and to adapt to different behavioural conditions. The results highlight the asymmetry of information flow and its behavioural correspondence. Importantly, it also shows that neural synchronisation dynamics, when suitably flexible and reconfigurable, can generate minimally cognitive embodied behaviour.
    Full-text · Article · Jul 2012 · Biological Cybernetics
  • Source
    • "These schemes will be evaluated by measuring the amount of information that the neuronal response convey about the stimulus. For this purpose it will be used concepts of the information theory (McDonnell et al. 2011). "
    [Show abstract] [Hide abstract]
    ABSTRACT: When rats acquire sensory information by actively moving their vibrissae, a neural code is manifested at different levels of the sensory system. Behavioral studies in tactile discrimination agree that rats can distinguish different roughness surfaces by whisking their vibrissae. The present study explores the existence of neural encoding in the afferent activity of one vibrissal nerve. Two neural encoding schemes based on "events" were proposed (cumulative event count and median inter-event time). The events were detected by using an event detection algorithm based on multiscale decomposition of the signal (Continuous Wavelet Transform). The encoding schemes were quantitatively evaluated through the maximum amount of information which was obtained by the Shannon's mutual information formula. Moreover, the effect of difference distances between rat snout and swept surfaces on the information values was also studied. We found that roughness information was encoded by events of 0.8 ms duration in the cumulative event count and event of 1.0 to 1.6 ms duration in the median inter-event count. It was also observed that an extreme decrease of the distance between rat snout and swept surfaces significantly reduces the information values and the capacity to discriminate among the sweep situations.
    Full-text · Article · Jun 2012 · Journal of Computational Neuroscience
Show more