No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Biorealistic Spiking Neural Network on FPGA
Abstract
In this paper, we present a digital hardware implementation of a biorealistic spiking neural network composed of 117 Izhikevich neurons. This digital system works in hard real-time, which means that it keeps the same biological time of simulation at the millisecond scale. The Izhikevich neuron implementation requires few resources. The neurons behavior is validated by comparing their firing rate to biological data. The interneuron connections are composed of biorealistic synapses. The architecture of the network implementation allows working on a single computation core. It is freely configurable from an independent-neuron configuration to all-to-all configuration or a mix with several independent small networks. This spiking neural network will be used for the development of a new proof-of-concept Brain Machine Interface, i.e. a neuromorphic chip for neuroprosthesis, which has to replace the functionality of a damaged part of the central nervous system.
... This article focuses on the area-and power-efficient design and implementation of a generalized hardware architecture for moderately-sized SNNs, in the range of tens of thousands of neurons, based on the Izhikevich neuron model. While most of the previously-published work have elaborated on fixed dedicated hardware architectures, supporting a predefined number of neurons and synaptic interconnect topologies [18][19][20][21], the proposed architecture aims to exploit the reconfigurable nature of the field-programmable gate arrays (FPGAs) to support an arbitrary number of spiking neurons and diverse network interconnect structures, while offering an adjustable time resolution. In addition to FPGA [18][19][20][21][22][23] and programmable processor [24] realizations of SNNs, application-specific integrated circuit (ASIC) implementations of SNNs have also been reported [25][26][27][28][29][30]. ...
... While most of the previously-published work have elaborated on fixed dedicated hardware architectures, supporting a predefined number of neurons and synaptic interconnect topologies [18][19][20][21], the proposed architecture aims to exploit the reconfigurable nature of the field-programmable gate arrays (FPGAs) to support an arbitrary number of spiking neurons and diverse network interconnect structures, while offering an adjustable time resolution. In addition to FPGA [18][19][20][21][22][23] and programmable processor [24] realizations of SNNs, application-specific integrated circuit (ASIC) implementations of SNNs have also been reported [25][26][27][28][29][30]. While FPGA and processor realizations offer greater flexibility in realizing diverse SNNs, ASIC implementations can consume significantly lower power and can be implemented in a smaller form factor and hence, are practical for the in-vivo brain implantations. ...
... Dedicated reconfigurable hardware realization of SNNs has also received interest [18][19][20][21][22][23], many of which have focused on accelerating the emulation of large-scale SNNs. The work in [18] presents an FPGA-based implementation of an Izhikevich-based SNN, which supports a fully-connected network of 1024 neurons with over one million synapses. ...
This article presents an area- and power-efficient hardware architecture for the brain-implantable spiking neural networks (SNNs). The proposed generalized hardware architecture is parameterizable and reconfigurable such that the maximum supported number of neurons, the interconnection structure among neurons, and the resolution of the time step can be readily adjusted for realizing various SNN topologies. The designed SNN hardware architecture is capable of emulating moderately-sized SNNs with tens of thousands of neurons in real-time with varying degrees of parallelism, while reducing the resource utilization by 34% for similarly sized SNNs implemented on a single field-programmable gate array (FPGA). We evaluate the model using the MNIST digit recognition benchmark and show that the network can accurately classify handwritten digits with 89.8% accuracy. Compared to the other recently implemented SNN emulators based on FPGAs, the designed and implemented single-FPGA system is able to emulate moderately-sized SNNs instead of using a cluster of FPGAs or CPUs. The application-specific integrated circuit (ASIC) implementation of a moderately-sized SNN is estimated to occupy 3.6 mm² of silicon area. Post-layout synthesis and simulation results show that the ASIC will dissipate 3.6 mW of power from a 1.16 V supply while operating at 34.7 MHz in a standard 32-nm CMOS process.
... In Ambroise et al. [26], a folded low-resources architecture capable of emulating 117 Izhikevich neurons in real-time with a time resolution of 1 ms is presented. The system is implemented on a Xilinx Virtex-4 chip, and the interconnections of the neuron are configurable, ranging from zero to a fully connected network. ...
... Table 1 summarizes the main characteristics of the abovementioned FPGA works. It is possible to notice how the presented architecture, being fully connected, owns a different balance between its number of neurons and synapses than most of the works in Table 1 [9], [10], [17], [20], [22], [24], a significantly higher number of synapses and neurons than the other works addressing fully connected neural networks [21], [26], and a higher number of synapses in general. In fact, the presented work has been conceived as a tool to study the behaviors of biological neural networks, rather than a machine learning accelerator. ...
... We selected 10 bits for the integer part. Using less than 10 bits causes data overflows, as pointed out in [26]. On the other hand, using more than 10 bits for the integer part does not provide any accuracy benefits. ...
Closed-loop experiments involving biological and artificial neural networks would improve the understanding of neural cells functioning principles and lead to the development of new generation neuroprosthesis. Several technological challenges require to be faced, as the development of real-time spiking neural network emulators which could bear the increasing amount of data provided by new generation High-Density Multielectrode Arrays. This work focuses on the development of a real-time spiking neural network emulator addressing fully-connected neural networks. This work presents a new way to increase the number of synapses supported by real-time neural network accelerators. The proposed solution has been implemented on the Xilinx Zynq 7020 All-Programmable SoC and can emulate fully connected spiking neural networks counting up to 3,098 Izhikevich neurons and 9.6e6 synapses in real-time, with a resolution of 0.1 ms.
... In paper proposed by Ambroise, Levi, Bornat and Saighi [1], 117 neurons are implemented in an FPGA board. Their breakthrough was their suggestion of dividing the current I IZH into three other currents I exc , I inh and I stat . ...
... Based on [3], [2] and [1], we have previously proposed [7] an implementation of IZH model on FPGA boards that combines the suggestions being made in [3], [2] and [1]. This fact implies that some modification should be made on (1). ...
... Based on [3], [2] and [1], we have previously proposed [7] an implementation of IZH model on FPGA boards that combines the suggestions being made in [3], [2] and [1]. This fact implies that some modification should be made on (1). ...
... To avoid the inefficiencies, some other frameworks accelerate the internal state updates by implementing a few neuron models on field-programmable gate arrays (FPGAs) [16], [17] or application-specific integrated circuits (ASICs) [5], [11], [18], [19]. Although they excel at efficiently simulating their target neuron models, their model-driven designs prevent them from supporting diverse neuron behaviors. ...
... Although they excel at efficiently simulating their target neuron models, their model-driven designs prevent them from supporting diverse neuron behaviors. For instance, Ambroise et al. [16] and IBM TrueNorth [5] only support a greatly simplified neuron model (i.e., linear leak integrate-and-fire) whose biological accuracy is too low for accurate biological simulations. As another example, Neurogrid [19] supports a much more complex neuron model; however, it cannot support simplified neuron models (e.g., IBM TrueNorth) as its design is bound to the complex neuron model. ...
... In order to minimize the high overheads of neuron computation, prior work has proposed to implement neuron models on FPGAs [16], [17] and ASICs [11], [18], [19], [39], [40]. Employing specialized accelerators can significantly improve SNN simulation efficiency in terms of latency and energy efficiency; however, their model-driven digital neurons prevent the accelerators from supporting diverse neuron behaviors. ...
... The most related work [99] utilizes many types of resources, including counters, comparators, adders, and even subtractors. The number of DFFs in [87] and [92], which use the IZK model, is more than twenty times our architecture. That means the proposed design of the ADRAF neuron saves energy and chip area by 70% to 90%, according to Table 4. ...
Integrate-and-fire (IAF) and leaky integrate-and-fire (LIF) models are the popular models for spiking neurons and spiking neuron networks (SNN). They lack the dynamic properties of the ordinary differential equations (ODEs) model but are simpler. More than half of all neuron implementations were digital circuits. The majority of them were IAF and LIF models. The resonate-and-fire (RAF) model proposed by Izhikevich has both simplicity and dynamic properties. However, the RAF model does not have general structures for hardware implementation. Most realizations of the RAF neuron models were analog designs. In this study, we developed and simulated the model’s equations with digital pulses. Based on this analysis, we proposed block models of digital and all-digital RAF neurons to turn the models into digital hardware. We verified the properties of the proposed models by implementing an all-digital RAF neuron on a Intel 28 nm Cyclone V Field-programmable gate array (FPGA) device. The register-transfer level (RTL) structure of the implementation consumed fewer resources, with just 23 D flip-flops, without floating-point operations or multipliers.
... The first method uses analog integrated circuits to produce different neurological patterns [26], [27]. This rapid implementation method, is inflexible and has a long development time [28], [29]. The second method utilizes digital circuits or digital electronic packages. ...
Digital realization of neuron models, especially implementation on a field programmable gate array (FPGA), is one of the key objectives of neuromorphic research, because the effective hardware realization of the biological neural networks plays a crucial role in implementing the behaviors of the brain for future applications. In this paper, a hybrid FitzHugh Nagumo-Morris Lecar (FNML) neuron model with electromagnetic flux coupling is considered, and two multiplierless piecewise linear (PWL) models, which have similar behaviors to the biological neuron, are presented. A comparison between digital implementation results of the original FNML and PWL models illustrates that, the PWL1 model provides a 65% speed-up with an overall saving (in FPGA resources) of 66.2%, and the PWL2 model yields a 71% speed-up with an overall saving of 78.2%.
... (i) It is a major challenge to incorporate them in a large network with dissipative interconnections [35], where the resistive loss becomes significant and results in inefficiency. (ii) Bio-realism is a crucial requirement for a network to be brain-like [36]- [38]. Inherent characteristics of a device determine the degree of bio-realism that can be achieved through the design approach [39]. ...
The revolution in artificial intelligence (AI) brings up an enormous storage and data processing requirement. Large power consumption and hardware overhead have become the main challenges for building next-generation AI hardware. Therefore, it is imperative to look for a new architecture capable of circumventing these bottlenecks of conventional von Neumann architecture. Since the human brain is the most compact and energy-efficient intelligent device known, it was intuitive to attempt to build an architecture that could mimic our brain, and so the chase for neuromorphic computing began. While relentless research has been underway for years to minimize the power consumption in neuromorphic hardware, we are still a long way off from reaching the energy efficiency of the human brain. Besides, design complexity, process variation, etc. hinder the large-scale implementation of current neuromorphic platforms. Recently, the concept of implementing neuromorphic computing systems in cryogenic temperature has garnered immense attention. Several cryogenic devices can be engineered to work as neuromorphic primitives with ultra-low demand for power. Cryogenic electronics has therefore become a promising exploratory platform for an energy-efficient and bio-realistic neuromorphic system. Here we provide a comprehensive overview of the reported cryogenic neuromorphic hardware. We carefully classify the existing cryogenic neuromorphic hardware into different categories and draw a comparative analysis based on several performance metrics. Finally, we explore the future research prospects to circumvent the challenges associated with the current technologies.
... In this paper, the neuron model that is going to be used is a modified version of the Izhikevich model [10], proposed by Ambroise et al. [11] including some aspects of the work presented by A.Cassidy and and A. G. Andreou [12]. It is a biologically-inspired model able to reproduce the behavior of biological neurons, including both spiking and bursting and mixed mode firing patterns. ...
... The downside of this method is that they cannot make use of dedicated hardware such as GPUs at the moment, being restricted to highly parallel CPUs in most cases. There are exemptions of Field-Programmable Gate Array (FPGA) implementations to speed up the low-latency processing [88,89,90,91]. Exemplary applications of event-by-event methods include event stream classification [85,92], optical flow [93,94,95,96], corner detection [97,98,99], pose estimation [100] and tracking [101]. ...
The demand for computing power steadily increases to enable new and more intelligent functionalities in our current technology. The combined computing power of mobile systems such as phones, drones, autonomous vehicles and embedded systems increases rapidly, but each system has a limited power budget. Efficient computation is thus of utmost importance. For the past decades we have relied on the growing amount of transistors per unit area to keep up with computing demand while keeping power consumption in check, but this trend is declining as transistor sizes are reaching physical limits. While architecture improvements stagnate, we find ourselves in the early stages of creating intelligent systems, which raises the question how current system can scale and which makes the exploration of alternative computing principles worth wile. This thesis examines the role of new bio-inspired computation paradigms for low-power computation, to drive a future generation of intelligent systems. Neuromorphic computing is an emerging interdisciplinary field that looks at biological systems such as the retina or the brain for inspiration on how to compute efficiently. From that it is possible to create sensors, algorithms and hardware that process information much closer to how the biological model works than current conventional computer architecture.We examine how neuromorphic cameras, algorithms and hardware can gradually replace conventional components to make the system overall use less power. We approach the issue through the lens of efficiency, and propose an event-based face detection algorithm, a framework that brings event-based computer vision to mobile devices with optimised hardware and methods based on precise timing for spiking neural networks on neuromorphic hardware. In this attempt we bring technology into being that starts to resemble the organic counterpart, to show the capabilities of brain-inspired computing.
... The pioneer of the neuron model, famously known as the Hodgkin and Huxley neuron model was developed by Hodgkin and Huxley in 1952 [15]. A mathematical model developed by them can reproduce all kinds of neurons with better precisions in terms of shape and complex firing activities [16] and is the most biologically plausible model till date. ...
The immense computation and huge memory requirement are challenging the computation efficiency of today's systems. Consequently, neuromorphic systems have become a topical subject in research to mimic the brain's power efficiency and computational speed. There have always been certain major bottlenecks in the conventional architectures. In this paper, we develop a Network-on-Chip based Spiking Neural Network (NoCSNN), having a highly parallel architecture for the neuromorphic computing systems. It also benefits from the use of NoC in terms of scalability, latency and speed. The neurons in our proposed SNN model communicates through NoC architecture. Our proposed model consisting of 64 neurons is synthesized in 28nm technology node achieving a power dissipation of 29.22 mW and a die area of 1.61 mm 2. The NoC model is also explored in terms of latency, throughput and energy.
... Soleimani et al. (2012) implement a classic Izhikevich model using PWL method to prove that the method can simplify the hardware implementation with showing similar dynamic behaviors. Ambroise et al. (2013) also implement an Izhikevich model on FPGA, but it is mainly to propose an architecture to reproduce a neural network with only one computation core (one neuron) based on one multiplier. Bonabi et al. (2012) implement a Hodgkin-Huxley (H-H) single neuron with the CORDIC algorithm and some LUTs that show high precision with more compact used logic. ...
Purkinje cell is an important neuron for the cerebellar information processing. In this work, we present an efficient implementation of a cerebellar Purkinje model using the Coordinate Rotation Digital Computer (CORDIC) algorithm and implement it on a Large-Scale Conductance-Based Spiking Neural Networks (LaCSNN) system with cost-efficient multiplier-less methods, which are more suitable for large-scale neural networks. The CORDIC-based Purkinje model has been compared with the original model in terms of the voltage activities, dynamic mechanisms, precision, and hardware resource utilization. The results show that the CORDIC-based Purkinje model can reproduce the same biological activities and dynamical mechanisms as the original model with slight deviation. In the aspect of the hardware implementation, it can use only logic resources, so it provides an efficient way for maximizing the FPGA resource utilization, thereby expanding the scale of neural networks that can be implemented on FPGAs.
... In [12], the authors used the Maxeller HLS infrastructure to achieve 3.8X faster performance than a GPU using six FPGAs. Prior to the HLS development flow, most studies were focused on the HDL implementations of Izhikevich models [13,14,15] because the previous generations of FPGAs lacked a large number of DSPs and memory blocks for efficiently implementing the great number of parameters and equations in HH models. ...
Field-programmable gate arrays (FPGAs) are becoming a promising choice as a heterogeneous computing component when floating-point optimized architectures are added to the current FPGAs. The maturing high-level synthesis tools offer a streamlined design flow for researchers to develop a parallel application using a high-level language on FPGAs. In this paper, we choose a random network of Hodgkin-Huxley (HH) neurons with exponential synaptic conductance to evaluate the performance of the simulation of networks of spiking neurons on an FPGA. Focused on the conductance-based HH benchmark, we execute the benchmark on a general-purpose simulator for spiking neural networks, identify a computationally intensive kernel in the generated C++ code, convert the kernel to a portable OpenCL kernel, and describe the optimizations which can reduce the resource utilizations and improve the kernel performance. We evaluate the kernel on an Intel Arria 10 based FPGA platform, an Intel Xeon 16-core CPU, an Intel Xeon 4-core low-power processor with a CPU and a GPU integrated on the same chip, and an NVIDIA Tesla P100 discrete GPU. For the kernel execution time, the Arria 10 GX1150 FPGA is 2X and 3X faster than the two CPUs, but it is 2.5X and 4.8X slower than the two GPUs, respectively. The FPGA consumes the least power, but its performance per watt is 1.56X and 1.96X lower than the two GPUs, respectively.
... The neural network implementation architecture operates on a single computation core. This real-time digital system requires few resources and low power consumption [16][17][18][19]. Table 1 summarizes the stated resources and Figure 2 describes two CPGs with different periods. ...
The design of biomimetic robot is one popular research. To achieve this goal, the reproduction of animal locomotion is mandatory. Animal locomotion is created by the activities of Central Pattern Generator (CPG). CPGs are neural networks capable of producing rhythmic patterned outputs without rhythmic sensory or central input. We propose a network of several biomimetic CPGs using biomimetic neuron model and synaptic plasticity. This network is implemented on a field programmable gate array. We designed one unsupervised snake robot using this network of CPG. It is composed of one head wagon followed by seven slave wagons. Infrared sensors are also embedded in the head wagon. This robot can reproduce the locomotion of one snake.
... Fidjeland and Shanahan [47] presented the GPU-based SNN with the Izhikevich model. Ambroise et al. [48] presented an approach to implementing 167 neurons with a smaller FPGA device. Pani et al. [49] successfully implemented the fully connected SNN with 1440 Izhikevich neurons, which is meaningful for digital neuromorphic engineering. ...
Multicompartment emulation is an essential step to enhance the biological realism of neuromorphic systems and to further understand the computational power of neurons. In this paper, we present a hardware efficient, scalable, and real-time computing strategy for the implementation of large-scale biologically meaningful neural networks with one million multi-compartment neurons (CMNs). The hardware platform uses four Altera Stratix III field-programmable gate arrays, and both the cellular and the network levels are considered, which provides an efficient implementation of a large-scale spiking neural network with biophysically plausible dynamics. At the cellular level, a cost-efficient multi-CMN model is presented, which can reproduce the detailed neuronal dynamics with representative neuronal morphology. A set of efficient neuromorphic techniques for single-CMN implementation are presented with all the hardware cost of memory and multiplier resources removed and with hardware performance of computational speed enhanced by 56.59% in comparison with the classical digital implementation method. At the network level, a scalable network-on-chip (NoC) architecture is proposed with a novel routing algorithm to enhance the NoC performance including throughput and computational latency, leading to higher computational efficiency and capability in comparison with state-of-the-art projects. The experimental results demonstrate that the proposed work can provide an efficient model and architecture for large-scale biologically meaningful networks, while the hardware synthesis results demonstrate low area utilization and high computational speed that supports the scalability of the approach.
... In analog implementation, the rapid prototyping of neural algorithms is enabled to test the neural computation theories and network architectures [13], [22]. Although this method can be fast and efficient, it suffers from inflexibility and long development time [9], [23]. • Digital implementation: digital implementations of neurobiological systems are flexible with shorter development times but they are often large, computationally slow, and they consume more silicon area and power in comparison to analog implementations [3], [7], [11], [15]. ...
Fast speed and high accuracy implementation of biological plausible neural networks are vital key objectives to achieve new solutions to model, simulate and cure the brain diseases. Efficient hardware implementation of Spiking Neural Networks (SNN) is a significant approach in biological neural networks. This paper presents a Multiplierless Noisy Izhikevich Neuron (MNIN) model, which is used for digital implementation of biological neural networks in large scale. Simulation results show that the MNIN model reproduces the same operations of the original noisy Izhikevich neuron. The proposed model has a low-cost hardware implementation property compared with the original neuron model. The FPGA realization results demonstrated that the MNIN model follows the different spiking patterns, appropriately.
... This device will be integrated in a bigger system composed of biomimetic Spiking Neural Network (SNN) [17][18][19][20]. The spike timing of SNN will trigger the microfluidic neuron for biomimetic stimulation on neuron culture or on brain organoid [21]. ...
Millions of people worldwide have incurable and debilitating conditions called neurodegenerative diseases that influence one’s cognitive and/or motor functions. There is currently an increasing number of neuroprosthesis, but they have power consumption and bio-compatibility issues. To bring neuroprosthesis into realization and for future long-term replacement of damaged brain areas with artificial devices, understanding of neurophysiological behaviors and investigations on the interaction of neuronal cell assemblies is essential. To circumvent the limitations, in this article we propose a biomimetic artificial neuron to mimic and/or to replace the biological neurons. This biomimetic neuron is based on microfluidic technique with ionic exchange capable of performing bio-hybrid experiments.
... Methods like principal component analysis and multilayer perceptron are implemented in FPGA. Neural network based work are carried out [12,13] with hardware prototype keeping similar biological simulation time. But the convergence behavior of PCA is hard to formulate and predict. ...
The paper presents a brain computer interface system for patient monitoring to detect and correct seizure. About 4–7% of people are suffering from seizure and there are only less medical attention available. The objective of the work described in this paper is to detect and cure seizure automatically without physician intervention. The therapeutic device is a modeled chip using Field Programmable Gate Array Logic and SoC in which the size of the instrument is small. The EEG signals of the brain are recorded using scalp electrodes and converted to digital data. The EEG signal is preprocessed using quadrature spline wavelet transform. FPGA implementation of quadrature spline wavelet transform filter was done with Baugh Wooley and array multiplier. From the results it is found that the power and logic elements are improved when Baugh Wooley multiplier is used. But when delay is compared array multiplier has better performance. The preprocessed data is feature extracted using double stage pattern search method where the seizure event is detected. Once the seizure is detected, the seizure control block is activated. It controls the seizure in few seconds. For seizure detection and correction, adaptive methods and hardware simulators are used. The performance and efficiency is compared for various devices using Quartus in FPGA cyclone II and III.
... Spikes accumulation is performed at 32 bits to preserve as much as possible the precision. In Ambroise et al. (2013), the authors present a similar approach, that is capable of emulating up to 167 neurons (it is worth to notice that these results have been achieved on a smaller device: no data is provided on large FPGAs). Compared to the proposed one, such a work uses a higher number of resources due to higher data precision and to a fairly more complex processing of the synaptic current. ...
In the last years, the idea to dynamically interface biological neurons with artificial ones has become more and more urgent. The reason is essentially due to the design of innovative neuroprostheses where biological cell assemblies of the brain can be substituted by artificial ones. For closed-loop experiments with biological neuronal networks interfaced with in silico modeled networks, several technological challenges need to be faced, from the low-level interfacing between the living tissue and the computational model to the implementation of the latter in a suitable form for real-time processing. Field programmable gate arrays (FPGAs) can improve flexibility when simple neuronal models are required, obtaining good accuracy, real-time performance, and the possibility to create a hybrid system without any custom hardware, just programming the hardware to achieve the required functionality. In this paper, this possibility is explored presenting a modular and efficient FPGA design of an in silico spiking neural network exploiting the Izhikevich model. The proposed system, prototypically implemented on a Xilinx Virtex 6 device, is able to simulate a fully connected network counting up to 1,440 neurons, in real-time, at a sampling rate of 10 kHz, which is reasonable for small to medium scale extra-cellular closed-loop experiments.
The mimicry of the biological brain's structure in information processing enables spiking neural networks (SNNs) to exhibit significantly reduced power consumption compared to conventional systems. Consequently, these networks have garnered heightened attention and spurred extensive research endeavors in recent years, proposing various structures to achieve low power consumption, high speed, and improved recognition ability. However, researchers are still in the early stages of developing more efficient neural networks that more closely resemble the biological brain. This development and research require suitable hardware for execution with appropriate capabilities, and field-programmable gate array (FPGA) serves as a highly qualified candidate compared to existing hardware such as central processing unit (CPU) and graphics processing unit (GPU). FPGA, with parallel processing capabilities similar to the brain, lower latency and power consumption, and higher throughput, is highly eligible hardware for assisting in the development of spiking neural networks. In this review, an attempt has been made to facilitate researchers' path to further develop this field by collecting and examining recent works and the challenges that hinder the implementation of these networks on FPGA.
The combination of neuromorphic computing (NC) and 3-D integrated circuits -the 3-D stacking neuromorphic system can be the most advanced architecture that inherits the benefits of both computing and interconnect paradigms. However, simply shifting to the third dimension cannot exploit the 3-D structure and also end up with a low yield rate issue. Therefore, in this article, we propose a methodology to design 3-D stacking synaptic memory for power-efficient operations and yield rate improvement of neuromorphic systems. In this proposed methodology, the synaptic weights are stacked on top of the processing elements (PEs), and these weights are split into multiple subsets placed in different layers. Furthermore, with the support of 3-D technology, the supply voltage of each layer can be controlled independently which leads to power reduction by scaling down or turning off the supply voltage of the memory layer(s) containing the least significant bits (LSBs) while maintaining acceptable accuracy. On top of that, this work also proposes a methodology to deal with the low yield rate issue by treating the defective memory cells as noises. In our evaluation with the CMOS 45 nm technology, the energy per synaptic operation (SOP) for MNIST classification, when undervolting two upper memory layers (from 1.1 to 0.8 V), reduces by 21.62% while the accuracy only reduces sightly by 0.51%. This energy reduction increases to 66.77% with 6.58% accuracy loss when our system uses both power-gating and undervolting for all memory layers. Furthermore, the system can also improve the yield rate by 0.18% or 12.4% while suffering 0.38% or 1.7% of accuracy loss, respectively.
Modeling and hardware implementation of artificial neurons is an important goal in the field of neuromorphic and brain-inspired computing. This paper presents a hardware implementation method of the two-dimensional piecewise linear spiking neuron (PLSN) model that can express various dynamical behaviors under different bifurcation mechanisms. The neuronal digital circuit of the PLSN model is designed and implemented on a field programmable gate array (FPGA), where all computation processes in the arithmetic tree are pipelined to maintain fast computing. The hardware is used to demonstrate 20 neurocomputational properties that are the most prominent dynamical behaviors of biological neurons. The 6 cortical neurons are also simulated using the PLSN circuit on FPGA. The results show that the proposed neuron circuit can express complex spiking and bursting behaviors of cortical neurons.
Neuromorphic computing is a promising technology that realizes computation based on event-based spiking neural networks (SNNs). However, fault-tolerant on-chip learning remains a challenge in neuromorphic systems. This study presents the first scalable neuromorphic fault-tolerant context-dependent learning (FCL) hardware framework. We show how this system can learn associations between stimulation and response in two context-dependent learning tasks from experimental neuroscience, despite possible faults in the hardware nodes. Furthermore, we demonstrate how our novel fault-tolerant neuromorphic spike routing scheme can avoid multiple fault nodes successfully and can enhance the maximum throughput of the neuromorphic network by 0.9%-16.1% in comparison with previous studies. By utilizing the real-time computational capabilities and multiple-fault-tolerant property of the proposed system, the neuronal mechanisms underlying the spiking activities of neuromorphic networks can be readily explored. In addition, the proposed system can be applied in real-time learning and decision-making applications, brain-machine integration, and the investigation of brain cognition during learning.
L’hybridation est une technique qui consiste à interconnecter un réseau de neurones biologiqueet un réseau de neurones artificiel, utilisée dans la recherche en neuroscience età des fins thérapeutiques. Durant ces trois années de doctorat, ce travail de thèse s’estfocalisé sur l’hybridation dans un plan rapproché (communication directe bi-diretionnelle entrel’artificiel et le vivant) et dans un plan plus élargies (interopérabilité des systèmes neuromorphiques).Au début des années 2000, cette technique a permis de connecter un système neuromorphiqueanalogique avec le vivant. Ce travail est dans un premier temps, centré autour de la conceptiond’un réseau de neurones numérique, en vue d’hybridation, dans deux projets multi-disciplinairesen cours dans l’équipe AS2N de l’IMS, présentés dans ce document : HYRENE (ANR 2010-Blan-031601), ayant pour but le développement d’un systèmehybride de restauration de l’activité motrice dans le cas d’une lésion de la moelle épinière, BRAINBOW (European project FP7-ICT-2011-C), ayant pour objectif l’élaboration deneuro-prothèses innovantes capables de restaurer la communication autour de lésionscérébrales.Possédant une architecture configurable, un réseau de neurones numérique a été réalisé pources deux projets. Pour le premier projet, le réseau de neurones artificiel permet d’émuler l’activitéde CPGs (Central Pattern Generator), à l’origine de la locomotion dans le règne animale. Cetteactivité permet de déclencher une série de stimulations dans la moelle épinière lésée in vitro et derecréer ainsi la locomotion précédemment perdue. Dans le second projet, la topologie du réseaude neurones sera issue de l’analyse et le décryptage des signaux biologiques issues de groupesde neurones cultivés sur des électrodes, ainsi que de modélisations et simulations réalisées parnos partenaires. Le réseau de neurones sera alors capable de réparer le réseau de neurones lésé.Ces travaux de thèse présentent la démarche de conception des deux différents réseaux et desrésultats préliminaires obtenus au sein des deux projets.Dans un second temps, ces travaux élargissent l’hybridation à l’interopérabilité des systèmesneuromorphiques. Au travers d’un protocole de communication utilisant Ethernet, il est possibled’interconnecter des réseaux de neurones électroniques, informatiques et biologiques.
Recent advances in bioelectronics and neural engineering allowed the development of brain machine interfaces and neuroprostheses, capable of facilitating or recovering functionality in people with neurological disability. To realize energy-efficient and real-time capable devices, neuromorphic computing systems are envisaged as the core of next-generation systems for brain repair. We demonstrate here a real-time hardware neuromorphic prosthesis to restore bidirectional interactions between two neuronal populations, even when one is damaged or missing. We used in vitro modular cell cultures to mimic the mutual interaction between neuronal assemblies and created a focal lesion to functionally disconnect the two populations. Then, we employed our neuromorphic prosthesis for bidirectional bridging to artificially reconnect two disconnected neuronal modules and for hybrid bidirectional bridging to replace the activity of one module with a real-time hardware neuromorphic Spiking Neural Network. Our neuroprosthetic system opens avenues for the exploitation of neuromorphic-based devices in bioelectrical therapeutics for health care.
This paper generalizes the Izhikevich neuron model in the fractional-order domain for better modeling of neuron dynamics. Accurate and computationally efficient numerical techniques such as non-standard finite difference (NSFD) scheme is used to solve the neuron system in the fractional-order domain for different cases. Neuron synchronization plays an important role in the process of information exchange among coupled neurons. The general formula for the synchronization of different Izhikevich neurons is proposed. Also, the synchronization of two and three neurons are studied at different fractional orders. Furthermore, the fractional-order regular spiking neuron of Izhikevich model is implemented on Xilinx (XC5VLX30T) Virtex 5 FPGA kit using only combinational logic. FPGAs are known for their reconfigurability and parallelism which makes them suitable for large-scale neural network simulations.
Low-cost accurate digital realization of the spiking neural networks is crucial to investigate the behaviors of human brain performance. This paper presents a multiplierless burst-mode Fitz-Hugh Nagumo (MBM-FHN) model, which is used for generating the burst-mode of the FHN neuron model. Using extra parameters (time-varying function) in neuron equations and estimating it by linear function, efficient low-cost and high-speed implementation is achieved. The simulation results show that the MBM-FHN model generates the similar bursting patterns of the original FHN neuron. The comparison shows that the MBM-FHN model has a better performance and best cost reduction related to the original neuron model especially in overall saving and speed-up parameters.
The Izhikevich’s simple model (ISM) for neural activity presents a good compromise between waveform quality and computational cost. FPGAs (Field Programmable Gate Array) are powerful, flexible, and inexpensive digital hardware that can implement such model. In this paper, we present a highly combinational, low latency implementation of ISM for FPGA. In the absence of official benchmark to compare different implementations, we propose two different metrics to compare the technical literature with our implementation. In this benchmark, we can implement a system that, when compared to the literature, has almost 1.5 times the number of digital neurons (DN), and latency more than 56 times smaller. This shows that our implementation is best suited for hybrid network systems and presents a fair performance for only-artificial networks.
The analysis of the environment for crime prediction is based on the premise that criminal behavior is influenced by the nature of the environment in which occurs. Street-level images are the closest digital depiction available of the urban environment, in which most street crimes take place. This work proposes a crime rate prediction model that uses street-level images to classify street crimes into low or high crime rate levels. For that, we use a 4-Cardinal Siamese Convolution Neural Network (4-CSCNN) and train and test our analytic model in two regions of Rio de Janeiro, Brazil, that showed high street crime concentrations between the years of 2007 and 2016. With this preliminary experiment, we investigate the use of convolutional neural networks (CNN) for the task of crime rating through visual scene analysis and found possibilities towards automatic crime rate predictions using CNN models.
The objective of this work is to obtain a complete synchronization of Hopfield Neural Networks (HNN) with a delay using a Field Programmable Gate Array (FPGA) simulating in real-time a Natural Neural Networks (NNN). This work is motivated by research in Neurosciences involving the implantation of chips between the skull and the brain to prevent or ameliorate diseases such as Parkinson’s, Epilepsy and Depression. Our contribution is the introduction of new synchronization techniques based on the Qualitative Theory of Differential Equations, Chaos Theory and Algebraic Topology substituting calculations using the Lyapunov Stability Criterion (LSC). The presented technique does not depend on the Neural Networks to be synchronized but also presents a lower computational cost in comparison with previous works. The results show that FPGAs are good platforms for such experiments.
Since their introduction, FPGAs can be seen in more and more different fields of applications. The key advantage is the combination of software-like flexibility with the performance otherwise common to hardware. Nevertheless, every application field introduces special requirements to the used computational architecture. This paper provides an overview of the different topics FPGAs have been used for in the last 15 years of research and why they have been chosen over other processing units like e.g. CPUs.
Understanding and modeling the brain is one of the key scientific challenges in the twenty-first century, and a grown effort is rising on a global scale. Due to its high parallelism, the hardware implementation of large-scale spiking neural networks (SNNs) promises superior execution speed compared to sequential software approaches. Such systems can significantly benefit from the use of networks-on-chip(NoC), as they scale very well concerning area, performance, power/energy consumption, and overall design effort. We developed a hierarchical network-on-chip for a hardware SNN architecture to improve the communication and scalability of the system. The architecture was implemented in an Altera Stratix IV FPGA, and a logic synthesis was performed to evaluate the system, achieving an area of 0.23mm 2 and a power dissipation of 147mW for a 256 neurons implementation.
The basal ganglia (BG) comprise multiple subcortical nuclei, which are responsible for cognition and other functions. Developing a brain-machine interface (BMI) demands a suitable solution for the real-time implementation of a portable BG. In this study, we used a digital hardware implementation of a BG network containing 256 modified Izhikevich neurons and 2048 synapses to reliably reproduce the biological characteristics of BG on a single field programmable gate array (FPGA) core. We also highlighted the role of Parkinsonian analysis by considering neural dynamics in the design of the hardware-based architecture. Thus, we developed a multi-precision architecture based on a precise analysis using the FPGA-based platform with fixed-point arithmetic. The proposed embedding BG network can be applied to intelligent agents and neurorobotics, as well as in BMI projects with clinical applications. Although we only characterized the BG network with Izhikevich models, the proposed approach can also be extended to more complex neuron models and other types of functional networks.
Copyright © 2015 Elsevier Ltd. All rights reserved.
Research on brain-machine interfaces has been ongoing for at least a decade. During this period, simultaneous recordings of the extracellular electrical activity of hundreds of individual neurons have been used for direct, real-time control of various artificial devices. Brain-machine interfaces have also added greatly to our knowledge of the fundamental physiological principles governing the operation of large neural ensembles. Further understanding of these principles is likely to have a key role in the future development of neuroprosthetics for restoring mobility in severely paralysed patients.
Paralysis following spinal cord injury, brainstem stroke, amyotrophic lateral sclerosis and other disorders can disconnect the brain from the body, eliminating the ability to perform volitional movements. A neural interface system could restore mobility and independence for people with paralysis by translating neuronal activity directly into control signals for assistive devices. We have previously shown that people with long-standing tetraplegia can use a neural interface system to move and click a computer cursor and to control physical devices. Able-bodied monkeys have used a neural interface system to control a robotic arm, but it is unknown whether people with profound upper extremity paralysis or limb loss could use cortical neuronal ensemble signals to direct useful arm actions. Here we demonstrate the ability of two people with long-standing tetraplegia to use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. Participants controlled the arm and hand over a broad space without explicit training, using signals decoded from a small, local population of motor cortex (MI) neurons recorded from a 96-channel microelectrode array. One of the study participants, implanted with the sensor 5 years earlier, also used a robotic arm to drink coffee from a bottle. Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, our results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.
Past models of somatosensory cortex have successfully
demonstrated map formation and subsequent map reorganization
following localized repetitive stimuli or deafferentation. They
provide an impressive demonstration that fairly simple assumptions
about ...
Neuromorphic circuits are being used to develop a new generation of computing technologies based on the organizing principles of the biological nervous system. Within this context, we present neuromorphic circuits for implementing massively parallel VLSI net- works of integrate-and-fire neurons with adaptation and spike-based plasticity mechanisms. We describe both analog continuous time and digital asynchronous event-based circuits for constructing spiking neural network devices, and present a VLSI implementation of a spike- based learning mechanisms for carrying out robust classification of spatio-temporal patterns, and real-time sensory signal processing. We argue that these types of devices have great po- tential for exploiting future scaled VLSI processes and are ideal for implementing sensory- motor processing units on autonomous and humanoid robots.
We review here the development of Hodgkin-Huxley (HH) type models of cerebral cortex and thalamic neurons for network simulations. The intrinsic electrophysiological properties of cortical neurons were analyzed from several preparations, and we selected the four most prominent electrophysiological classes of neurons. These four classes are "fast spiking", "regular spiking", "intrinsically bursting" and "low-threshold spike" cells. For each class, we fit "minimal" HH type models to experimental data. The models contain the minimal set of voltage-dependent currents to account for the data. To obtain models as generic as possible, we used data from different preparations in vivo and in vitro, such as rat somatosensory cortex and thalamus, guinea-pig visual and frontal cortex, ferret visual cortex, cat visual cortex and cat association cortex. For two cell classes, we used automatic fitting procedures applied to several cells, which revealed substantial cell-to-cell variability within each class. The selection of such cellular models constitutes a necessary step towards building network simulations of the thalamocortical system with realistic cellular dynamical properties.
Neuromotor prostheses (NMPs) aim to replace or restore lost motor functions in paralysed humans by routeing movement-related signals from the brain, around damaged parts of the nervous system, to external effectors. To translate preclinical results from intact animals to a clinically useful NMP, movement signals must persist in cortex after spinal cord injury and be engaged by movement intent when sensory inputs and limb movement are long absent. Furthermore, NMPs would require that intention-driven neuronal activity be converted into a control signal that enables useful tasks. Here we show initial results for a tetraplegic human (MN) using a pilot NMP. Neuronal ensemble activity recorded through a 96-microelectrode array implanted in primary motor cortex demonstrated that intended hand motion modulates cortical spiking patterns three years after spinal cord injury. Decoders were created, providing a 'neural cursor' with which MN opened simulated e-mail and operated devices such as a television, even while conversing. Furthermore, MN used neural control to open and close a prosthetic hand, and perform rudimentary actions with a multi-jointed robotic arm. These early results suggest that NMPs based upon intracortical neuronal ensemble spiking activity could provide a valuable new neurotechnology to restore independence for humans with paralysis.
This article concludes a series of papers concerned with the flow of electric current through the surface membrane of a giant nerve fibre (Hodgkinet al., 1952,J. Physiol.116, 424–448; Hodgkin and Huxley, 1952,J. Physiol.116, 449–566). Its general object is to discuss the results of the preceding papers (Section 1), to put them into mathematical form (Section 2) and to show that they will account for conduction and excitation in quantitative terms (Sections 3–6).
This article concludes a series of papers concerned with the flow of electric current through the surface membrane of a giant
nerve fibre (Hodgkinet al., 1952,J. Physiol.
116, 424–448; Hodgkin and Huxley, 1952,J. Physiol.
116, 449–566). Its general object is to discuss the results of the preceding papers (Section 1), to put them into mathematical
form (Section 2) and to show that they will account for conduction and excitation in quantitative terms (Sections 3–6).
We present an array of dynamical digital silicon neurons implementing the Izhikevich neuron model. The FPGA based array consists of 32 physical neurons, each time multiplexing the state of 8 virtual neurons, for a total of 256 independent neurons. The neural array operates at 5,000 times faster than real time, performing over 20.48 GOPS (giga operations per second). It is intended for neural simulation acceleration, neural prostheses, and neuromorphic systems.
I. ABSTRACT In this paper, we present an architecture and corresponding analysis for large-scale neuromorphic systems using a digital approach where neurons are abstracted as arithmetic logic units and communication processors. After presenting the architecture, we establish a few basic architectural principles, particularly the scaling of the system to large arrays. We demonstrate the reality of a single chip million neuron system built from these principles in commercial off the shelf FPGA.
This article concludes a series of papers concerned with the flow of electric current through the surface membrane of a giant nerve fibre (Hodgkinet al., 1952,J. Physiol.116, 424–448; Hodgkin and Huxley, 1952,J. Physiol.116, 449–566). Its general object is to discuss the results of the preceding papers (Section 1), to put them into mathematical form (Section 2) and to whow that they will account for conduction and excitation in quantitative terms (Sections 3–6).
A model is presented that reproduces spiking and bursting behavior of known types of cortical neurons. The model combines the biologically plausibility of Hodgkin-Huxley-type dynamics and the computational efficiency of integrate-and-fire neurons. Using this model, one can simulate tens of thousands of spiking cortical neurons in real time (1 ms resolution) using a desktop PC.
Rapid design time, low cost, flexibility, digital precision, and stability are characteristics that favor FPGAs as a promising alternative to analog VLSI based approaches for designing neuromorphic systems. High computational power as well as low size, weight, and power (SWAP) are advantages that FPGAs demonstrate over software based neuromorphic systems. We present an FPGA based array of Leaky-Integrate and Fire (LIF) artificial neurons. Using this array, we demonstrate three neural computational experiments: auditory Spatio-Temporal Receptive Fields (STRFs), a neural parameter optimizing algorithm, and an implementation of the Spike Time Dependant Plasticity (STDP) learning rule.
We discuss the biological plausibility and computational efficiency of some of the most useful models of spiking and bursting neurons. We compare their applicability to large-scale simulations of cortical neural networks.
Minimal Hodgkin-Huxley type models for different classes of cortical and thalamic neurons
- M Pospichil
- M Toledo-Rodriguez
- C Monier
- Z Piwkowska
- T Bal
- Y Frégnac
- H Markram
- A Destexhe
Pospichil, M., Toledo-Rodriguez, M., Monier, C., Piwkowska, Z.,
Bal, T., Frégnac, Y., Markram, H., Destexhe A., "Minimal Hodgkin-Huxley type models for different classes of cortical and thalamic
neurons", Biological Cybernetics, 99:427-441, 2008.