-
[show abstract]
[hide abstract]
ABSTRACT: Temporal spike codes play a crucial role in neural information processing. In particular, there is strong experimental evidence that interspike intervals (ISIs) are used for stimulus representation in neural systems. However, very few algorithmic principles exploit the benefits of such temporal codes for probabilistic inference of stimuli or decisions. Here, we describe and rigorously prove the functional properties of a spike-based processor that uses ISI distributions to perform probabilistic inference. The abstract processor architecture serves as a building block for more concrete, neural implementations of the belief-propagation (BP) algorithm in arbitrary graphical models (e.g., Bayesian networks and factor graphs). The distributed nature of graphical models matches well with the architectural and functional constraints imposed by biology. In our model, ISI distributions represent the BP messages exchanged between factor nodes, leading to the interpretation of a single spike as a random sample that follows such a distribution. We verify the abstract processor model by numerical simulation in full graphs, and demonstrate that it can be applied even in the presence of analog variables. As a particular example, we also show results of a concrete, neural implementation of the processor, although in principle our approach is more flexible and allows different neurobiological interpretations. Furthermore, electrophysiological data from area LIP during behavioral experiments are assessed in light of ISI coding, leading to concrete testable, quantitative predictions and a more accurate description of these data compared to hitherto existing models.
Neural Computation 05/2013; · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The brain combines and integrates multiple cues to take coherent, context-dependent action using distributed, event-based computational primitives. Computational models that use these principles in software simulations of recurrently coupled spiking neural networks have been demonstrated in the past, but their implementation in hybrid analog/digital Very Large Scale Integration (VLSI) spiking neural networks remains challenging. Here, we demonstrate a distributed spiking neural network architecture comprising multiple neuromorphic VLSI chips able to reproduce these types of cue combination and integration operations. This is achieved by encoding cues as population activities of input nodes in a network of recurrently coupled VLSI Integrate-and-Fire (I&F) neurons. The value of the cue is place-encoded, while its uncertainty is represented by the width of the population activity profile. Relationships among different cues are specified through bidirectional connectivity matrices, shared between the individual input node populations and an intermediate node population. The resulting network dynamics bidirectionally relate not only the values of three variables ac-cording to a specified relation, but also their uncertainties. When cues on two populations are specified, the standard deviation of the activity in the unspecified population varies approximately linearly with the widths of the two input cues, and has less than 6% error in position compared to the value specified by the inputs. The results suggest a mechanism for recurrently relating cues such that missing information can both be recovered and assigned a level of certainty.
IJCNN 2012, Brisbane, Australia; 06/2012
-
[show abstract]
[hide abstract]
ABSTRACT: An increasing number of research groups are developing custom hybrid analog/digital very large scale integration (VLSI) chips and systems that implement hundreds to thousands of spiking neurons with biophysically realistic dynamics, with the intention of emulating brainlike real-world behavior in hardware and robotic systems rather than simply simulating their performance on general-purpose digital computers. Although the electronic engineering aspects of these emulation systems is proceeding well, progress toward the actual emulation of brainlike tasks is restricted by the lack of suitable high-level configuration methods of the kind that have already been developed over many decades for simulations on general-purpose computers. The key difficulty is that the dynamics of the CMOS electronic analogs are determined by transistor biases that do not map simply to the parameter types and values used in typical abstract mathematical models of neurons and their networks. Here we provide a general method for resolving this difficulty. We describe a parameter mapping technique that permits an automatic configuration of VLSI neural networks so that their electronic emulation conforms to a higher-level neuronal simulation. We show that the neurons configured by our method exhibit spike timing statistics and temporal dynamics that are the same as those observed in the software simulated neurons and, in particular, that the key parameters of recurrent VLSI neural networks (e.g., implementing soft winner-take-all) can be precisely tuned. The proposed method permits a seamless integration between software simulations with hardware emulations and intertranslatability between the parameters of abstract neuronal models and their emulation counterparts. Most important, our method offers a route toward a high-level task configuration language for neuromorphic VLSI systems.
Neural Computation 07/2011; 23(10):2457-97. · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Biological systems are based on an entirely different concept of construction than human artifacts. They construct themselves by a process of self-organization that is a systematic spatio-temporal generation of, and interaction between, various specialized cell types. We propose a framework for designing gene-like codes for guiding the self-construction of neural networks. The description of neural development is formalized by defining a set of primitive actions taken locally by neural precursors during corticogenesis. These primitives can be combined into networks of instructions similar to biochemical pathways, capable of reproducing complex developmental sequences in a biologically plausible way. Moreover, the conditional activation and deactivation of these instruction networks can also be controlled by these primitives, allowing for the design of a "genetic code" containing both coding and regulating elements. We demonstrate in a simulation of physical cell development how this code can be incorporated into a single progenitor, which then by replication and differentiation, reproduces important aspects of corticogenesis.
Frontiers in Computational Neuroscience 01/2011; 5:57. · 2.15 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The winner-take-all (WTA) computation in networks of recurrently connected neurons is an important decision element of many models of cortical processing. However, analytical studies of the WTA performance in recurrent networks have generally addressed rate-based models. Very few have addressed networks of spiking neurons, which are relevant for understanding the biological networks themselves and also for the development of neuromorphic electronic neurons that commmunicate by action potential like address-events. Here, we make steps in that direction by using a simplified Markov model of the spiking network to examine analytically the ability of a spike-based WTA network to discriminate the statistics of inputs ranging from stationary regular to nonstationary Poisson events. Our work extends previous theoretical results showing that a WTA recurrent network receiving regular spike inputs can select the correct winner within one interspike interval. We show first for the case of spike rate inputs that input discrimination and the effects of self-excitation and inhibition on this discrimination are consistent with results obtained from the standard rate-based WTA models. We also extend this discrimination analysis of spiking WTAs to nonstationary inputs with time-varying spike rates resembling statistics of real-world sensory stimuli. We conclude that spiking WTAs are consistent with their continuous counterparts for steady-state inputs, but they also exhibit high discrimination performance with nonstationary inputs.
Neural Computation 07/2009; 21(9):2437-65. · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: From a theoretical point of view, statistical inference is an attractive model of brain operation. However, it is unclear how to implement these inferential processes in neuronal networks. We offer a solution to this problem by showing in detailed simulations how the belief propagation algorithm on a factor graph can be embedded in a network of spiking neurons. We use pools of spiking neurons as the function nodes of the factor graph. Each pool gathers "messages" in the form of population activities from its input nodes and combines them through its network dynamics. Each of the various output messages to be transmitted over the edges of the graph is computed by a group of readout neurons that feed in their respective destination pools. We use this approach to implement two examples of factor graphs. The first example, drawn from coding theory, models the transmission of signals through an unreliable channel and demonstrates the principles and generality of our network approach. The second, more applied example is of a psychophysical mechanism in which visual cues are used to resolve hypotheses about the interpretation of an object's shape and illumination. These two examples, and also a statistical analysis, demonstrate good agreement between the performance of our networks and the direct numerical evaluation of belief propagation.
Neural Computation 07/2009; 21(9):2502-23. · 1.88 Impact Factor
-
Rafael Serrano-Gotarredona,
Matthias Oster,
Patrick Lichtsteiner,
Alejandro Linares-Barranco,
Rafael Paz-Vicente,
Francisco Gómez-Rodríguez,
Luis Camuñas-Mesa,
Raphael Berner,
Manuel Rivas-Pérez,
Tobi Delbrück, [......],
A Linares-Barranco,
R Paz-Vicente,
F Gómez-Rodríguez,
M Rivas-Pérez,
G Jiménez-Moreno,
A Civit,
T R Berner,
S.-C Delbrück,
R Liu,
Douglas
[show abstract]
[hide abstract]
ABSTRACT: This paper describes CAVIAR, a massively par-allel hardware implementation of a spike-based sensing–pro-cessing–learning–actuating system inspired by the physiology of the nervous system. CAVIAR uses the asychronous address–event representation (AER) communication framework and was de-veloped in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45k neurons (spiking cells), up to 5M synapses, performs 12G synaptic operations per second, and achieves millisecond object recognition and tracking latencies. Index Terms—Address–event representation (AER), neuromor-phic chips, neuromorphic systems, vision.
IEEE Transactions on Neural Networks 01/2009; 20. · 2.95 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The development of neural tissue is a complex organizing process, in which it is difficult to grasp how the various localized interactions between dividing cells leads relentlessly to global network organization. Simulation is a useful tool for exploring such complex processes because it permits rigorous analysis of observed global behavior in terms of the mechanistic axioms declared in the simulated model. We describe a novel simulation tool, CX3D, for modeling the development of large realistic neural networks such as the neocortex, in a physical 3D space. In CX3D, as in biology, neurons arise by the replication and migration of precursors, which mature into cells able to extend axons and dendrites. Individual neurons are discretized into spherical (for the soma) and cylindrical (for neurites) elements that have appropriate mechanical properties. The growth functions of each neuron are encapsulated in set of pre-defined modules that are automatically distributed across its segments during growth. The extracellular space is also discretized, and allows for the diffusion of extracellular signaling molecules, as well as the physical interactions of the many developing neurons. We demonstrate the utility of CX3D by simulating three interesting developmental processes: neocortical lamination based on mechanical properties of tissues; a growth model of a neocortical pyramidal cell based on layer-specific guidance cues; and the formation of a neural network in vitro by employing neurite fasciculation. We also provide some examples in which previous models from the literature are re-implemented in CX3D. Our results suggest that CX3D is a powerful tool for understanding neural development.
Frontiers in Computational Neuroscience 01/2009; 3:25. · 2.15 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The primate neocortex is characterized by a highly expanded supragranular layer (SGL). The interareal connectivity of the neurons in the SLG largely determines the cortical hierarchy that constrains information flow through the cortex. Interareal connectivity is made by precise numbers of connections, raising the possibility that the physiology of a target area is dictated by the numbers of connections and hierarchical distance in each of the pathways that it receives. The developmental mechanisms ensuring the precision of these interareal networks is in part determined by (i) the numbers of SGL neurons generated by the OSVZ, a primate-specific germinal zone. Neuron generation rate in the OSVZ is determined by regulation of the G1 phase of the cell-cycle. This regulation is area-specific and is linked to thalamic projections to the OSVZ; (ii) Prolonged pre- and postnatal pruning of connections originating from the SGL when the infant monkey visually explores its environment. Remodelling serves to sharpen initial patterns of connections and establishes the adult hierarchy. These results suggest that primate cortical networks underlying high-level function undergo prolonged self-organization via regressive phenomena in the cortical plate (axon elimination) and progressive phenomena (directed growth of cortical axons).
Novartis Foundation symposium 02/2007; 288:178-94 discussion 195-8, 276-81.
-
[show abstract]
[hide abstract]
ABSTRACT: Circuits composed of threshold gates (McCulloch-Pitts neurons, or perceptrons) are simplified models of neural circuits with the advantage that they are theoretically more tractable than their biological counterparts. However, when such threshold circuits are designed to perform a specific computational task, they usually differ in one important respect from computations in the brain: they require very high activity. On average every second threshold gate fires (sets a 1 as output) during a computation. By contrast, the activity of neurons in the brain is much sparser, with only about 1% of neurons firing. This mismatch between threshold and neuronal circuits is due to the particular complexity measures (circuit size and circuit depth) that have been minimized in previous threshold circuit constructions. In this letter, we investigate a new complexity measure for threshold circuits, energy complexity, whose minimization yields computations with sparse activity. We prove that all computations by threshold circuits of polynomial size with entropy O(log n) can be restructured so that their energy complexity is reduced to a level near the entropy of circuit states. This entropy of circuit states is a novel circuit complexity measure, which is of interest not only in the context of threshold circuits but for circuit complexity in general. As an example of how this measure can be applied, we show that any polynomial size threshold circuit with entropy O(log n) can be simulated by a polynomial size threshold circuit of depth 3. Our results demonstrate that the structure of circuits that result from a minimization of their energy complexity is quite different from the structure that results from a minimization of previously considered complexity measures, and potentially closer to the structure of neural circuits in the nervous system. In particular, different pathways are activated in these circuits for different classes of inputs. This letter shows that such circuits with sparse activity have a surprisingly large computational power.
Neural Computation 01/2007; 18(12):2994-3008. · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Spatio-temporal processing of spike trains by neuronal networks depends on a variety of mechanisms distributed across synapses, dendrites, and somata. In natural systems, the spike trains and the processing mechanisms cohere though their common physical instantiation. This coherence is lost when the natural system is encoded for simulation on a general purpose computer. By contrast, analog VLSI circuits are, like neurons, inherently related by their real-time physics, and so, could provide a useful substrate for exploring neuronlike event-based processing. Here, we describe a hybrid analog-digital VLSI chip comprising a set of integrate-and-fire neurons and short-term dynamical synapses that can be configured into simple network architectures with some properties of neocortical neuronal circuits. We show that, despite considerable fabrication variance in the properties of individual neurons, the chip offers a viable substrate for exploring real-time spike-based processing in networks of neurons.
IEEE Transactions on Neural Networks 10/2004; 15(5):1305-14. · 2.95 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We present an analog circuit for implementing models of synapses with short--term adaptation, derive analytical solutions for spiking input signals, and present experimental results measured from a chip fabricated using a standard 1.5m CMOS technology. The circuit is suitable for integration in large arrays of integrate--and--fire neurons and consequently for evaluating computational roles of short-- term adaptation at the network level.
05/2004;
-
[show abstract]
[hide abstract]
ABSTRACT: We present a VLSI device comprising an array of leaky integrate--and-- fire (I&F) neurons and adaptive synapses with spike--timing dependent plasticity (STDP). The neurons transmit spikes off chip and the synapses receive spikes from external devices using a communication protocol based on the "Address-- Event Representation" (AER). We studied the response properties of the neurons in the array to uniform input currents, and measured their AER outputs. We characterized the properties of the STDP synapses using AER input spike trains. Our results indicate that these circuits can be reliably used in massively parallel VLSI networks of I&F neurons to simulate real--time complex spike--based learning algorithms.
04/2004;
-
[show abstract]
[hide abstract]
ABSTRACT: We describe a programmable multi-chip VLSI neuronal system that can be used for exploring spike-based information processing models.
04/2003;
-
[show abstract]
[hide abstract]
ABSTRACT: We describe a programmable multi-chip VLSI neuronal system that can be used for exploring spike-based information processing models.
04/2002;
-
[show abstract]
[hide abstract]
ABSTRACT: Vision is one of the most useful sensory functions, but the real-time processing of the continuous, high-dimensional input
signals provided by vision sensors is a major challenge in robot design. Conventional digital vision sensors tend to have
excessive power consumption, size, and cost for useful applications. In this TechView, Indiveri and Douglas discuss an alternative
strategy, namely neuromorphic vision sensors that are based on biological vision systems. In these sensors, specialized sensory
processing functions inspired by biological systems such as fly eyes are integrated in parallel, asynchronous circuits that
respond in real time. These sensors offer significant advantages over conventional vision sensors.
Science 05/2000; 288(5469):1189-1190. · 31.20 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Circuits composed of threshold gates (McCulloch-Pitts neurons, or perceptrons) are simplified models of neural circuits with
the advantage that they are theoretically more tractable than their biological counterparts. However, when such threshold
circuits are designed to perform a specific computational task they usually differ in one important respect from computations
in the brain: they require very high activity. On average every second threshold gate fires (sets a “1” as output) during
a computation. By contrast, the activity of neurons in the brain is much more sparse, with only about 1% of neurons firing.
This mismatch between threshold and neuronal circuits is due to the particular complexity measures (circuit size and circuit
depth) that have been minimized in previous threshold circuit constructions. In this article we investigate a new complexity
measure for threshold circuits, energy complexity, whose minimization yields computations with sparse activity. We prove that all computations by threshold circuits of polynomial
size with entropy O(logn) can be restructured so that their energy complexity is reduced to a level near the entropy of circuit states. This entropy of circuit states is a novel circuit complexity measure, which is of interest not only in the context of threshold
circuits, but for circuit complexity in general. As an example of how this measure can be applied we show that any polynomial
size threshold circuit with entropy O(logn) can be simulated by a polynomial size threshold circuit of depth 3.
01/1970: pages 631-642;
-
[show abstract]
[hide abstract]
ABSTRACT: Strong regularities of early visual areas interconnections led to the suggestion that rostral directed connections are feedforward (FF) pathways channelling information from lower to higher order areas, while caudal directed connections constitute feedback (FB) pathways (Rockland and Pandya, 1979). Analysis of these pathways in primate enabled the identification of a hierarchical organization (Felleman and Van Essen, 1991), providing a major conceptual framework for understanding structure-function relationships of the cortex. Because previous description of cortical topology have been restricted to binary connectivity leading to strong indeterminacy (Hilgetag et al., 1996), we re-examined network description of cortex structure by making retrograde tracer injections in areas spanning all cortical lobes. We used quantitative tools to estimate hierarchical distance and relative weights of connections (Barone et al., 2000, Vezoli et al., 2004) and used computational modeling analysis to analyse the underlying hierarchical structure of cortical networks. Comparing weighted and unweighted analyses, we demonstrated a significant hierarchical tendency in the pattern of laminar relations between cortical areas. Further, we evidenced a highly parallel system with high degree of reciprocity and found that rare pairs of areas are reciprocally connected by FF connections, constituting unexpected descending paths (Crick and Koch, 1998) in an otherwise surprisingly hierarchical system of cortical areas.
Cinquième conférence plénière française de Neurosciences Computationnelles, "Neurocomp'10".
-
[show abstract]
[hide abstract]
ABSTRACT: Cortical neurogenesis is a complex process in which progenitor cells have the ability to acquire specific fates and eventually differentiate toward a final cell type. The adoption of a particular cell state is the result of intrinsic genetic programs, which regulate cell behaviour, cell-cell interactions, and the influence of environmental cues. However there is still no comprehensive understanding of the control of cell fate specification and more in general on the self-constructing principles involved in the process. Insight into the mechanisms underlying corticogenesis is provided by the genealogical history of every precursor cell (cell lineage). We use a method based on spectral clustering to identify recurrent developmental patterns on reconstructed cortical lineages. The obtained state diagram represent compact state machine description of the developmental process. We evaluate the degree of similarities between precursors types and compare it to known single-cell gene expression profiles, with good agreement between the data and the model. Additionally we recast the model into a formal gene-like language and discuss the implications for genetic regulation.
Cinquième conférence plénière française de Neurosciences Computationnelles, "Neurocomp'10".
