An introductory review of information theory in the context of computational neuroscience

Institute for Telecommunications Research, University of South Australia.
Biological Cybernetics (Impact Factor: 1.71). 07/2011; 105(1):55-70. DOI: 10.1007/s00422-011-0451-9
Source: DBLP


This article introduces several fundamental concepts in information theory from the perspective of their origins in engineering. Understanding such concepts is important in neuroscience for two reasons. Simply applying formulae from information theory without understanding the assumptions behind their definitions can lead to erroneous results and conclusions. Furthermore, this century will see a convergence of information theory and neuroscience; information theory will expand its foundations to incorporate more comprehensively biological processes thereby helping reveal how neuronal networks achieve their remarkable information processing abilities.

Download full-text


Available from: Mark D Mcdonnell,
  • Source
    • "sensory and motor patterns may vary over time). Transfer Entropy, in this scenario, is suggested as a suitable and robust information theoretic tool (Lungarella et al., 2007b,a), and has also been applied to investigate real neural assemblies and other neuroscience problems (Borst & Theunissen, 1999; Gourévitch & Eggermont, 2007; Buehlmann & Deco, 2010; McDonnell et al., 2011; Vicente et al., 2011); it will, thus, be used in our analysis. The paper is organised as follows: the next section presents some theoretical background to the main concepts explored in this work: oscillators and synchronisation, focusing on the Kuramoto Model, evolutionary robotics (ER) and the two minimally cognitive tasks studied, and Information Theory in an agent-environment context, describing Transfer Entropy. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Oscillatory activity is ubiquitous in nervous systems, with solid evidence that synchronisation mechanisms underpin cognitive processes. Nevertheless, its informational content and relationship with behaviour are still to be fully understood. In addition, cognitive systems cannot be properly appreciated without taking into account brain-body- environment interactions. In this paper, we developed a model based on the Kuramoto Model of coupled phase oscillators to explore the role of neural synchronisation in the performance of a simulated robotic agent in two different minimally cognitive tasks. We show that there is a statistically significant difference in performance and evolvability depending on the synchronisation regime of the network. In both tasks, a combination of information flow and dynamical analyses show that networks with a definite, but not too strong, propensity for synchronisation are more able to reconfigure, to organise themselves functionally and to adapt to different behavioural conditions. The results highlight the asymmetry of information flow and its behavioural correspondence. Importantly, it also shows that neural synchronisation dynamics, when suitably flexible and reconfigurable, can generate minimally cognitive embodied behaviour.
    Biological Cybernetics 07/2012; 106(6-7):407-27. DOI:10.1007/s00422-012-0507-5 · 1.71 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The paper puts the emphasis on surveying information-theoretic network measures for analyzing the structure of networks. In order to apply the quantities interdisciplinarily, we also discuss some of their properties such as their structural interpretation and uniqueness.
    Symmetry 12/2011; 3(4):767-779. DOI:10.3390/sym3040767 · 0.83 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: When rats acquire sensory information by actively moving their vibrissae, a neural code is manifested at different levels of the sensory system. Behavioral studies in tactile discrimination agree that rats can distinguish different roughness surfaces by whisking their vibrissae. The present study explores the existence of neural encoding in the afferent activity of one vibrissal nerve. Two neural encoding schemes based on "events" were proposed (cumulative event count and median inter-event time). The events were detected by using an event detection algorithm based on multiscale decomposition of the signal (Continuous Wavelet Transform). The encoding schemes were quantitatively evaluated through the maximum amount of information which was obtained by the Shannon's mutual information formula. Moreover, the effect of difference distances between rat snout and swept surfaces on the information values was also studied. We found that roughness information was encoded by events of 0.8 ms duration in the cumulative event count and event of 1.0 to 1.6 ms duration in the median inter-event count. It was also observed that an extreme decrease of the distance between rat snout and swept surfaces significantly reduces the information values and the capacity to discriminate among the sweep situations.
    Journal of Computational Neuroscience 06/2012; 34(1). DOI:10.1007/s10827-012-0408-6 · 1.74 Impact Factor
Show more