PreprintPDF Available
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

SpiNNaker is an ARM-based processor platform optimized for the simulation of spiking neural networks. This brief describes the roadmap in going from the current SPINNaker1 system, a 1 Million core machine in 130nm CMOS, to SpiNNaker2, a 10 Million core machine in 22nm FDSOI. Apart from pure scaling, we will take advantage of specific technology features, such as runtime adaptive body biasing, to deliver cutting-edge power consumption. Power management of the cores allows a wide range of workload adaptivity, i.e. processor power scales with the complexity and activity of the spiking network. Additional numerical accelerators will enhance the utility of SpiNNaker2 for simulation of spiking neural networks as well as for executing conventional deep neural networks. These measures should increase the simulation capacity of the machine by a factor >>50. The interplay between the two domains, i.e. spiking and rate based, will provide an interesting field for algorithm exploration on SpiNNaker2. Apart from the platforms' traditional usage as a neuroscience exploration tool, the extended functionality opens up new application areas such as automotive AI, tactile internet, industry 4.0 and biomedical processing.
Content may be subject to copyright.
SpiNNaker 2: A 10 Million Core Processor
System for Brain Simulation and Machine
Learning
Christian Mayr a,1, Sebastian Hppner aand Steve Furber b
aChair of Highly-Parallel VLSI-Systems and Neuromorphic Circuits, Institute of Circuits
and Systems, Technische Universitt Dresden, Dresden, Germany
bAdvanced Processor Technologies Group, School of Computer Science, University of
Manchester, Manchester, United Kingdom
Abstract. SpiNNaker is an ARM-based processor platform optimized for the simula-
tion of spiking neural networks. This brief describes the roadmap in going from the
current SPINNaker1 system, a 1 Million core machine in 130nm CMOS, to SpiN-
Naker2, a 10 Million core machine in 22nm FDSOI. Apart from pure scaling, we will
take advantage of specific technology features, such as runtime adaptive body biasing,
to deliver cutting-edge power consumption. Power management of the cores allows
a wide range of workload adaptivity, i.e. processor power scales with the complexity
and activity of the spiking network. Additional numerical accelerators will enhance
the utility of SpiNNaker2 for simulation of spiking neural networks as well as for ex-
ecuting conventional deep neural networks. These measures should increase the sim-
ulation capacity of the machine by a factor >50. The interplay between the two do-
mains, i.e. spiking and rate based, will provide an interesting field for algorithm explo-
ration on SpiNNaker2. Apart from the platforms’ traditional usage as a neuroscience
exploration tool, the extended functionality opens up new application areas such as
automotive AI, tactile internet, industry 4.0 and biomedical processing.
Keywords. MPSoC, neuromorphic computing, SpiNNaker2, power management,
22nm FDSOI, numerical accelerators
Introduction
The ”Spiking Neural Network Architecture” SpiNNaker is a processor platform optimized
for the simulation of neural networks. A large number of ARM cores is integrated in a system
architecture optimized for communication and memory access. Specifically, to take advan-
tage of the asynchronous, naturally parallel and independent subcomputations of biological
neurons, each core simulates neurons independently and communicates via a lightweight,
spike-optimized asynchronous communication protocol [1]. Neurons are simulated for a cer-
tain timestep (typically 1ms), and then activity patterns exchanged between cores, on the as-
sumption that time models itself, i.e. an exchange of activity every millisecond is assumed to
represent biological real time. This allows the energy efficient simulation of neural network
models in real time, with SpiNNaker significantly outperforming conventional high perfor-
mance computing wrt both these aspects. The first generation of SpiNNaker has been de-
signed by the University of Manchester and is currently operational at its intended maximum
system size, i.e. 1 Million ARM processors, as well as in the form of smaller boards in mo-
1Corresponding Author: Christian Mayr, TU Dresden, Mommsenstr. 12, 01062 Dresden, Germany. Tel.: +49
351 463 42392; Fax: +49 351 463 37794; E-mail: christian.mayr@tu-dresden.de.
arXiv:1911.02385v1 [cs.ET] 6 Nov 2019
bile applications (see lower row of images in Fig. 1). One Million cores allow the simulation
of spiking neural networks on the order of 1% of the human brain. Since 2013, Technische
Universitt Dresden and the University of Manchester have been jointly developing the next
generation SpiNNaker2 system in the framework of the EU flagship Human Brain Project
(upper row of images in Fig1). Specific target is a scaling of number of cores by a factor of 10,
while staying in the same power budget (i.e. 10x better power efficiency). Overall increase in
system capacity (simulated numbers of neurons) is expected to be a factor of approx. 50.
1. Spinnaker2 capabilities and main new building blocks
SpiNNaker1
SpiNNaker1 board
(864 ARM cores)
capocaccia.ethz.ch
„Large Scale“ SpiNNaker1
Machine (1 Million Processors) Mobile Systems
SpiNNaker2
Large Scale SpiNNaker2 Machine
SpiNNaker2 Chip
Ca. 144 ARM M4F Processors
18MByte SRAM
8GByte DRAM in PoP Package
Approx. 50x
Size Scaling
„Brain-Size“
Network
Simulations
Mobile Systems
56 Chips x 25 Boards x 5Racks x 10 Cabinets
10 Million Processors
SpiNNaker1 Chip (18 cores)
Figure 1. First and second generation of the SpiNNaker system. Please note: The above numbers for SpiN-
Naker2 represent the current state of design and are subject to change.
The 50x capacity scaling is exemplified in the middle of Fig1, i.e. the simulation capac-
ity of a current 48 processor board in SpiNNaker1 is expected to be carried inside a single
SpiNNaker2 chip. The additional scaling of 5 above the factor 10 (i.e. increase in core num-
ber) is primarily expected from a higher clock rate and numerical accelerators for common
synaptic operations. The general approach for SpiNNaker2 can be broken down in four main
aspects: (1) Keep the processor-based flexibility of SpiNNaker1, which has been a major
advantage compared to more streamlined neuromorphic approaches such as Truenorth [2] or
analog neuromorphic circuits [3,4]. (2) Dont do everything in software in the processors,
i.e. incorporate numerical accelerators for the most common operations. Current prototypes
contain accelerators for exponential functions [5] and derive random numbers from the ther-
mal noise of the clock generators inherent in each core, i.e. at virtually zero circuit overhead
[6]. For an example of the overall benefit of the accelerators in terms of clock cycles, see [7]
and other upcoming papers. Other accelerators, i.e. a log function, are in discussion. (3) Use
the latest technologies and features for energy efficiency. We are targeting a 22nm FDSOI
technology for SpiNNaker2, with on-chip adaptive body biasing (ABB) enabling reliable op-
eration down to 0.4V under virtually all operating conditions. ABB can narrow down the
transistor threshold voltage spread at runtime (i.e. over aging, temperature, manufacturing
corner) to enable robust near-threshold logic operation. (4) Allow workload adaptivity on
all levels. Similar to the brain, power consumption should be proportional to the task be-
ing carried out. Specifically, all processors in SpiNNaker2 operate under dynamic voltage
and frequency scaling [8]. Their operating voltage and frequency are individually adjusted to
the load of incoming spikes per every millisecond and the expected clock cycles required to
compute this load. Thus, at low computational load times, a task is stretched out over time
and can run at less supply voltage, reducing both dynamic and leakage power. Besides this
computational adaptivity, communication has also been streamlined. For example, the chip-
to-chip links feature very fast power up and down functionality, allowing energy proportion-
ality with regard to the bits transmitted. In addition to the above four main development lines,
multiply accumulate arrays (MAC) have been incorporated in the latest prototype chip to en-
hance the usefulness of SpiNNaker2 beyond simulation of spiking neural networks. With the
MACs, the synaptic weighing and accumulation of entire layers of conventional deep neural
networks can be offloaded from the processors, freeing them to carry out spike based sim-
ulations in parallel. For a discussion on how to parallelize these new network types across
processors, see [7].
2. Brain simulation and other applications
Sheer scaling of SpiNNaker2 compared to SpiNNaker1 should allow the simulation of sig-
nificantly larger and more complex spiking neural networks. In an upcoming publication,
we are showing an implementation of synaptic sampling [9], i.e. the most complex synaptic
plasticity ever implemented on SpiNNaker. This implementation is significantly aided by the
addition of the new numerical accelerators. In addition, there is scope for extending these
spiking simulations to multiscale networks, by using the MACs to simulate entire networks
as mesoscopic, black box modules. SpiNNaker2 should be able to run models like BioSpaun
[10] on levels of abstraction from multicompartment neurons via spiking point neurons, rate-
based neurons up to mesoscopic models. Beyond these extensions of traditional use cases,
the upper left corner of Fig. 1 shows other future uses. Specifically, SpiNNaker2 combines
high throughput machine learning, sensor-actuator processing with inherent millisecond la-
tency and IoT-device-level energy efficiency, which represents a breakthrough in the field of
real-time, mobile human-machine interaction. Targeted applications for SpiNNaker2 in this
area include the tactile internet (e.g. tele-learning, robotics interaction), autonomous driving,
industry 4.0 (e.g. real time predictive maintenance) or biomedical (e.g. closed-loop neural
implants with spiking and machine learning functionality). For an overview of the current
SpiNNaker2 development and technical details (MACs, FDSOI approach, accelerators, etc),
see [11]. A last word about the timetable: After the current phase of prototyping, we expect
the final SpiNNaker2 chip to become available sometime late 2020/early 2021, so look out
for the 2021 Capo Caccia and Telluride neuromorphic workshops.
Acknowledgements
The authors thank ARM for IP contributions. Discussions at the yearly Capo Caccia work-
shop have contributed to the prototype designs and benchmarks. The research leading to
these results has received funding from the European Union Seventh Framework Programme
(FP7) under grant agreement No 604102 and the EU’s Horizon 2020 research and innovation
programme under grant agreements No 720270 and 785907 (Human Brain Project, HBP).
References
[1] Steve B Furber, David R Lester, Luis A Plana, Jim D Garside, Eustace Painkras, Steve Temple, and
Andrew D Brown. Overview of the spinnaker system architecture. IEEE Transactions on Computers,
62(12):2454–2467, 2013.
[2] Paul A Merolla, John V Arthur, Rodrigo Alvarez-Icaza, Andrew S Cassidy, Jun Sawada, Filipp Akopyan,
Bryan L Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, et al. A million spiking-neuron integrated
circuit with a scalable communication network and interface. Science, 345(6197):668–673, 2014.
[3] Mihai A Petrovici, Sebastian Schmitt, Johann Kl¨
ahn, David St¨
ockel, Anna Schroeder, Guillaume Bel-
lec, Johannes Bill, Oliver Breitwieser, Ilja Bytschok, Andreas Gr ¨
ubl, et al. Pattern representation and
recognition with accelerated analog neuromorphic systems. In Circuits and Systems (ISCAS), 2017 IEEE
International Symposium on, pages 1–4. IEEE, 2017.
[4] A K¨
onig, C Mayr, T Bormann, and C Klug. Dedicated implementation of embedded vision systems
employing low-power massively parallel feature computation. In Proceedings of the 3rd VIVA-Workshop
on Low-Power Information Processing, pages 1–8, 2002.
[5] Johannes Partzsch, Sebastian H¨
oppner, Matthias Eberlein, Rene Sch¨
uffny, Christian Mayr, David R Lester,
and Steve Furber. A fixed point exponential function accelerator for a neuromorphic many-core system.
In Circuits and Systems (ISCAS), 2017 IEEE International Symposium on, pages 1–4. IEEE, 2017.
[6] Felix Neumarker, Sebastian H¨
oppner, Andreas Dixius, and Christian Mayr. True random number gen-
eration from bang-bang adpll jitter. In Nordic Circuits and Systems Conference (NORCAS), 2016 IEEE,
pages 1–5. IEEE, 2016.
[7] Chen Liu, Guillaume Bellec, Bernhard Vogginger, David Kappel, Johannes Partzsch, Sebastian H¨
oppner,
Wolfgang Maass, Steve B Furber, Robert Legenstein, Christian G Mayr, et al. Memory-efficient deep
learning on a spinnaker 2 prototype. Frontiers in neuroscience, 12:840, 2018.
[8] Sebastian H¨
oppner, Yexin Yan, Bernhard Vogginger, Andreas Dixius, Johannes Partzsch, Felix Neum¨
arker,
Stephan Hartmann, Stefan Schiefer, Stefan Scholze, Georg Ellguth, et al. Dynamic voltage and frequency
scaling for neuromorphic many-core systems. In Circuits and Systems (ISCAS), 2017 IEEE International
Symposium on, pages 1–4. IEEE, 2017.
[9] David Kappel, Robert Legenstein, Stefan Habenschuss, Michael Hsieh, and Wolfgang Maass. A dynamic
connectome supports the emergence of stable computational function of neural circuits through reward-
based learning. eNeuro, 5(2):ENEURO–0301, 2018.
[10] Chris Eliasmith, Jan Gosmann, and Xuan Choo. Biospaun: A large-scale behaving brain model with
complex neurons. arXiv preprint arXiv:1602.05220, 2016.
[11] Sebastian H¨
oppner and Christian Mayr. Spinnaker2-towards extremely efficient digital neuromorphics
and multi-scale brain emulation. Neuro Inspired Computational Elements (NICE 2018), 2018.
... In this context, as we will see, the objective of the present work is to develop a software solution that mimics the behavior of human biochemical intelligence, specifically neurotransmitters. Advances in spiking neural networks [28][29][30] and neuromorphic hardware [31][32][33][34][35][36] illustrate the potential for ...
Article
Full-text available
The aim of this work is to introduce a computer science solution to manage emotions and affections and connect them to the causes as in humans. The scientific foundation of this work lies in the ability to model the affective and emotional states of an individual or artificial intelligence (AI). Then, in this study, we go a step further by exploring how to extend this capability by linking it to the underlying causes—specifically, by establishing a connection between emotions, affective states, and neurotransmitter activities. The methods used in this study pertain to decision support systems based on complexity theory. Specifically, for the training of the platform to study the link between emotions/affections and neurotransmitters, an electroencephalogram (EEG) acquisition module is integrated into the platform. As a result, this solution provides the bedrock for next-generation AI, i.e., artificial rational–emotive decision-makers. In addition, this research studies the connection of EEG data with neurotransmitters’ activity, opening pathways to applications such as emotional monitoring, mental health, and brain–computer interfaces, adding to cognitively and emotionally enriched AI. The main result of this study is a platform able to manage artificial neurotransmitters such as adrenaline, GABA, dopamine, serotonin, oxytocin, endorphins, and the hormone cortisol for emulating and motivating emotive and affective states. In conclusion, this study highlights the following: (i) the possibility of conducting indirect measurements of emotional states based on EEG data, (ii) the development of a framework capable of generating a wide spectrum of emotional states by modulating neurotransmitter levels within a defined discrete range, and (iii) the ability to establish a connection between neurotransmitters (causes) and emotional states (effects).
... However, neuromorphic systems currently explore a broad range of varied implementation approaches, board-level integration, and development maturity 67,68 , and such platform diversity creates difficult challenges for completely consistent methodology. For instance, among mature large-scale neuromorphic systems, implementation strategies range from digital 58,59,69 to mixed-signal approaches 70,71 , and system-level integration extends past single-chip boards into servers including hundreds of chips 59,72,73 . Thus, to enable an initial step towards consistency in the system track while ensuring openness, we focus on the development of guidelines for transparent documentation, as they provide the foundation for shared methodology among highly diverse solutions. ...
Article
Full-text available
Neuromorphic computing shows promise for advancing computing efficiency and capabilities of AI applications using brain-inspired principles. However, the neuromorphic research field currently lacks standardized benchmarks, making it difficult to accurately measure technological advancements, compare performance with conventional methods, and identify promising future research directions. This article presents NeuroBench, a benchmark framework for neuromorphic algorithms and systems, which is collaboratively designed from an open community of researchers across industry and academia. NeuroBench introduces a common set of tools and systematic methodology for inclusive benchmark measurement, delivering an objective reference framework for quantifying neuromorphic approaches in both hardware-independent and hardware-dependent settings. For latest project updates, visit the project website ( neurobench.ai ).
... This study targets a wholly parallel calculation based on the more efficient matrix parallelism. We emulate the SNN inference on SpiNNaker 2 (Mayr et al., 2019), which integrates the MAC array in each processing element (PE). The parallelism of this integrated hardware component has the potential of speeding up SNN inference in a sufficiently parallel manner. ...
Article
Full-text available
The potential low-energy feature of the spiking neural network (SNN) engages the attention of the AI community. Only CPU-involved SNN processing inevitably results in an inherently long temporal span in the cases of large models and massive datasets. This study introduces the MAC array, a parallel architecture on each processing element (PE) of SpiNNaker 2, into the computational process of SNN inference. Based on the work of single-core optimization algorithms, we investigate the parallel acceleration algorithms for collaborating with multi-core MAC arrays. The proposed Echelon Reorder model information densification algorithm, along with the adapted multi-core two-stage splitting and authorization deployment strategies, achieves efficient spatio-temporal load balancing and optimization performance. We evaluate the performance by benchmarking a wide range of constructed SNN models to research on the influence degree of different factors. We also benchmark with two actual SNN models (the gesture recognition model of the real-world application and balanced random cortex-like network from neuroscience) on the neuromorphic multi-core hardware SpiNNaker 2. The echelon optimization algorithm with mixed processors realizes 74.28% and 85.78% memory footprint of the original MAC calculation on these two models, respectively. The execution time of echelon algorithms using only MAC or mixed processors accounts for ≤ 24.56% of the serial ARM baseline. Accelerating SNN inference with algorithms in this study is essentially the general sparse matrix-matrix multiplication (SpGEMM) problem. This article explicitly expands the application field of the SpGEMM issue to SNN, developing novel SpGEMM optimization algorithms fitting the SNN feature and MAC array.
... and NeuronGPU (Golosio et al., 2021) running on GPU-based systems. Recent iterations of these systems target a broader range of tasks for example in the area of Machine Learning, and feature higher degrees of flexibility and efficiency (Mayr et al., 2019;Billaudelle et al., 2020). In addition, recent dedicated systems exist that target the simulation at higher degrees of abstraction (Wang et al., 2018) or aim at solving Machine Learning tasks (Panchapakesan et al., 2022). ...
Article
Full-text available
Introduction Research in the field of computational neuroscience relies on highly capable simulation platforms. With real-time capabilities surpassed for established models like the cortical microcircuit, it is time to conceive next-generation systems: neuroscience simulators providing significant acceleration, even for larger networks with natural density, biologically plausible multi-compartment models and the modeling of long-term and structural plasticity. Methods Stressing the need for agility to adapt to new concepts or findings in the domain of neuroscience, we have developed the neuroAIx-Framework consisting of an empirical modeling tool, a virtual prototype, and a cluster of FPGA boards. This framework is designed to support and accelerate the continuous development of such platforms driven by new insights in neuroscience. Results Based on design space explorations using this framework, we devised and realized an FPGA cluster consisting of 35 NetFPGA SUME boards. Discussion This system functions as an evaluation platform for our framework. At the same time, it resulted in a fully deterministic neuroscience simulation system surpassing the state of the art in both performance and energy efficiency. It is capable of simulating the microcircuit with 20× acceleration compared to biological real-time and achieves an energy efficiency of 48nJ per synaptic event.
... The SpiNNaker system is currently an active research platform, with a 1M core machine operating and maintained by the University of Manchester, UK. In parallel to exploring SNN applications on this system, research and development into next-generation hardware is also on-going in the form of the SpiNNaker2 system (Mayr et al., 2019). The two platforms are explored in this work, implementing models on both the SpiNNaker system and a SpiNNaker2 prototype chip (Jib2), to enable comparison and evaluation of performance. ...
Article
Full-text available
Neuromorphic systems aim to provide accelerated low-power simulation of Spiking Neural Networks (SNNs), typically featuring simple and efficient neuron models such as the Leaky Integrate-and-Fire (LIF) model. Biologically plausible neuron models developed by neuroscientists are largely ignored in neuromorphic computing due to their increased computational costs. This work bridges this gap through implementation and evaluation of a single compartment Hodgkin-Huxley (HH) neuron and a multi-compartment neuron incorporating dendritic computation on the SpiNNaker, and SpiNNaker2 prototype neuromorphic systems. Numerical accuracy of the model implementations is benchmarked against reference models in the NEURON simulation environment, with excellent agreement achieved by both the fixed- and floating-point SpiNNaker implementations. The computational cost is evaluated in terms of timing measurements profiling neural state updates. While the additional model complexity understandably increases computation times relative to LIF models, it was found a wallclock time increase of only 8× was observed for the HH neuron (11× for the mutlicompartment model), demonstrating the potential of hardware accelerators in the next-generation neuromorphic system to optimize implementation of complex neuron models. The benefits of models directly corresponding to biophysiological data are demonstrated: HH neurons are able to express a range of output behaviors not captured by LIF neurons; and the dendritic compartment provides the first implementation of a spiking multi-compartment neuron model with XOR-solving capabilities on neuromorphic hardware. The work paves the way for inclusion of more biologically representative neuron models in neuromorphic systems, and showcases the benefits of hardware accelerators included in the next-generation SpiNNaker2 architecture.
... Next, in order to provide concrete examples, we implement and evaluate SNNs for two processing steps in software. Furthermore, as we plan a future implementation on digital neuromorphic hardware, such as Loihi (Davies et al., 2018) or SpiNNaker2 (Mayr et al., 2019), we derive the specific requirements and challenges of neuromorphic radar processing. ...
Article
Full-text available
Frequency-modulated continuous wave radar sensors play an essential role for assisted and autonomous driving as they are robust under all weather and light conditions. However, the rising number of transmitters and receivers for obtaining a higher angular resolution increases the cost for digital signal processing. One promising approach for energy-efficient signal processing is the usage of brain-inspired spiking neural networks (SNNs) implemented on neuromorphic hardware. In this article we perform a step-by-step analysis of automotive radar processing and argue how spiking neural networks could replace or complement the conventional processing. We provide SNN examples for two processing steps and evaluate their accuracy and computational efficiency. For radar target detection, an SNN with temporal coding is competitive to the conventional approach at a low compute overhead. Instead, our SNN for target classification achieves an accuracy close to a reference artificial neural network while requiring 200 times less operations. Finally, we discuss the specific requirements and challenges for SNN-based radar processing on neuromorphic hardware. This study proves the general applicability of SNNs for automotive radar processing and sustains the prospect of energy-efficient realizations in automated vehicles.
Article
Full-text available
Spiking neural networks and neuromorphic hardware platforms that simulate neuronal dynamics are getting wide attention and are being applied to many relevant problems using Machine Learning. Despite a well-established mathematical foundation for neural dynamics, there exists numerous software and hardware solutions and stacks whose variability makes it difficult to reproduce findings. Here, we establish a common reference frame for computations in digital neuromorphic systems, titled Neuromorphic Intermediate Representation (NIR). NIR defines a set of computational and composable model primitives as hybrid systems combining continuous-time dynamics and discrete events. By abstracting away assumptions around discretization and hardware constraints, NIR faithfully captures the computational model, while bridging differences between the evaluated implementation and the underlying mathematical formalism. NIR supports an unprecedented number of neuromorphic systems, which we demonstrate by reproducing three spiking neural network models of different complexity across 7 neuromorphic simulators and 4 digital hardware platforms. NIR decouples the development of neuromorphic hardware and software, enabling interoperability between platforms and improving accessibility to multiple neuromorphic technologies. We believe that NIR is a key next step in brain-inspired hardware-software co-evolution, enabling research towards the implementation of energy efficient computational principles of nervous systems. NIR is available at neuroir.org
Chapter
Spiking Neural Networks (SNNs) offer bioinspired computation based on local adaptation and plasticity as well as close biological compatibility. In this work, after reviewing the Hardware Emulator of Evolving Neural Systems (HEENS) architecture and its Computer-Aided Engineering (CAE) design flow, a spiking implementation of an adaptive physical sensor input scheme based on time-rate Band-Pass Filter (BPF) is proposed for real-time execution of large dynamic range sensory edge processing nodes. Simulation and experimental results of the SNN operating in real-time with an adaptive-range accelerometer input example are shown. This work opens the path to compute with SNNs multiple physical sensor information for perception applications.
Article
Full-text available
The memory requirement of deep learning algorithms is considered incompatible with the memory restriction of energy-efficient hardware. A low memory footprint can be achieved by pruning obsolete connections or reducing the precision of connection strengths after the network has been trained. Yet, these techniques are not applicable to the case when neural networks have to be trained directly on hardware due to the hard memory constraints. Deep Rewiring (DEEP R) is a training algorithm which continuously rewires the network while preserving very sparse connectivity all along the training procedure. We apply DEEP R to a deep neural network implementation on a prototype chip of the 2nd generation SpiNNaker system. The local memory of a single core on this chip is limited to 64 KB and a deep network architecture is trained entirely within this constraint without the use of external memory. Throughout training, the proportion of active connections is limited to 1.3%. On the handwritten digits dataset MNIST, this extremely sparse network achieves 96.6% classification accuracy at convergence. Utilizing the multi-processor feature of the SpiNNaker system, we found very good scaling in terms of computation time, per-core memory consumption, and energy constraints. When compared to a X86 CPU implementation, neural network training on the SpiNNaker 2 prototype improves power and energy consumption by two orders of magnitude.
Article
Full-text available
Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapse-autonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.
Article
Full-text available
SpiNNaker (a contraction of Spiking Neural Network Architecture) is a million-core computing engine whose flagship goal is to be able to simulate the behavior of aggregates of up to a billion neurons in real time. It consists of an array of ARM9 cores, communicating via packets carried by a custom interconnect fabric. The packets are small (40 or 72 bits), and their transmission is brokered entirely by hardware, giving the overall engine an extremely high bisection bandwidth of over 5 billion packets/s. Three of the principal axioms of parallel machine design (memory coherence, synchronicity, and determinism) have been discarded in the design without, surprisingly, compromising the ability to perform meaningful computations. A further attribute of the system is the acknowledgment, from the initial design stages, that the sheer size of the implementation will make component failures an inevitable aspect of day-to-day operation, and fault detection and recovery mechanisms have been built into the system at many levels of abstraction. This paper describes the architecture of the machine and outlines the underlying design philosophy; software and applications are to be described in detail elsewhere, and only introduced in passing here as necessary to illuminate the description.
Article
Inspired by the brain’s structure, we have developed an efficient, scalable, and flexible non–von Neumann architecture that leverages contemporary silicon technology. To demonstrate, we built a 5.4-billion-transistor chip with 4096 neurosynaptic cores interconnected via an intrachip network that integrates 1 million programmable spiking neurons and 256 million configurable synapses. Chips can be tiled in two dimensions via an interchip communication interface, seamlessly scaling the architecture to a cortexlike sheet of arbitrary size. The architecture is well suited to many applications that use complex neural networks in real time, for example, multiobject detection and classification. With 400-pixel-by-240-pixel video input at 30 frames per second, the chip consumes 63 milliwatts.
Dedicated implementation of embedded vision systems employing low-power massively parallel feature computation
  • A König
  • T Mayr
  • C Bormann
  • Klug
A König, C Mayr, T Bormann, and C Klug. Dedicated implementation of embedded vision systems employing low-power massively parallel feature computation. In Proceedings of the 3rd VIVA-Workshop on Low-Power Information Processing, pages 1-8, 2002.
True random number generation from bang-bang adpll jitter
  • Felix Neumarker
  • Sebastian Höppner
  • Andreas Dixius
  • Christian Mayr
Felix Neumarker, Sebastian Höppner, Andreas Dixius, and Christian Mayr. True random number generation from bang-bang adpll jitter. In Nordic Circuits and Systems Conference (NORCAS), 2016 IEEE, pages 1-5. IEEE, 2016.
Spinnaker2-towards extremely efficient digital neuromorphics and multi-scale brain emulation
  • Sebastian Höppner
  • Christian Mayr
Sebastian Höppner and Christian Mayr. Spinnaker2-towards extremely efficient digital neuromorphics and multi-scale brain emulation. Neuro Inspired Computational Elements (NICE 2018), 2018.