Abstract and Figures

Neural signals for locomotion are influenced both by the neural network architecture and sensory inputs coordinating and adapting the gait to the environment. Adaptation relies on the ability to change amplitude, frequency, and phase of the signals within the sensorimotor loop in response to external stimuli. However, in order to experiment with closed-loop control, we first need a better understanding of the dynamics of the system and how adaptation works. Based on insights from biology, we developed a spiking neural network capable of continuously changing amplitude, frequency, and phase online. The resulting network is deployed on a hexapod robot in order to observe the walking behavior. The morphology and parameters of the network results in a tripod gait, demonstrating that a design without afferent feedback is sufficient to maintain a stable gait. This is comparable to results from biology showing that deafferented samples exhibit a tripod-like gait and adds to the evidence for a meaningful role of network topology in locomotion. Further, this work enables research into the role of sensory feedback and high-level control signals in the adaptation of gait types. A better understanding of the neural control of locomotion relates back to biology where it can provide evidence for theories that are currently not testable on live insects.
| (A) Representation of the spiking CPG implementation, 1 and 2 represent the two neuron populations within the sCPG. The motor neuron population is excited by the "first" population within each sCPG. The motor neuron population post-processes the output before relaying it to the joint. (B) Representation of the coupling between two legs as well as intra-leg coupling. TC represents the thoraxcoxa joint and CF represents the coxa-femur joint. Each of these consists of a mutually inhibitory pair of neuron populations but is represented by a single circle for figure simplicity. The motor neuron populations are driven by excitatory connections from one of the neuron populations, as seen in (A). The FT, femur-tibia, joint is held fixed. (C) Representation of the coupling between the six legs both intra-and inter-segment. Each leg representation consists of two coupled sCPGs as shown in (B). FL, front left; ML, middle left; BL, back left; FR, front right; MR, middle right; BR, back right. The segments are named for clarity. (D) Illustration of a single leg of the MORF (Thor, 2019), identifying the location of the movable joints. (E) Picture of the hexapod robot used in this study. (F) Spike events vs. rate-coding output of a single population consisting of 5 neurons. Spike events are added based on a sliding time window of 5 ms. (G) Zoomed in selection of plot F to visualize individual spike events. The number of spikes inside the time window are counted, the resulting value is indicated using a black dot. This shows how spike events translate into an analog output. The simulation data allows the rate-coding algorithm to look forward in time but live experiments would use past data.
… 
Content may be subject to copyright.
ORIGINAL RESEARCH
published: 26 June 2020
doi: 10.3389/fnbot.2020.00041
Frontiers in Neurorobotics | www.frontiersin.org 1June 2020 | Volume 14 | Article 41
Edited by:
Jorg Conradt,
Royal Institute of Technology, Sweden
Reviewed by:
Fernando Perez-Peña,
University of Cádiz, Spain
Horacio Rostro Gonzalez,
University of Guanajuato, Mexico
*Correspondence:
Beck Strohmer
stroh@mmmi.sdu.dk
Received: 16 April 2020
Accepted: 26 May 2020
Published: 26 June 2020
Citation:
Strohmer B, Manoonpong P and
Larsen LB (2020) Flexible Spiking
CPGs for Online Manipulation During
Hexapod Walking.
Front. Neurorobot. 14:41.
doi: 10.3389/fnbot.2020.00041
Flexible Spiking CPGs for Online
Manipulation During Hexapod
Walking
Beck Strohmer*, Poramate Manoonpong and Leon Bonde Larsen
SDU Biorobotics, Maersk McKinney Moller Institute, University of Southern Denmark, Odense, Denmark
Neural signals for locomotion are influenced both by the neural network architecture
and sensory inputs coordinating and adapting the gait to the environment. Adaptation
relies on the ability to change amplitude, frequency, and phase of the signals within the
sensorimotor loop in response to external stimuli. However, in order to experiment with
closed-loop control, we first need a better understanding of the dynamics of the system
and how adaptation works. Based on insights from biology, we developed a spiking
neural network capable of continuously changing amplitude, frequency, and phase
online. The resulting network is deployed on a hexapod robot in order to observe the
walking behavior. The morphology and parameters of the network results in a tripod gait,
demonstrating that a design without afferent feedback is sufficient to maintain a stable
gait. This is comparable to results from biology showing that deafferented samples exhibit
a tripod-like gait and adds to the evidence for a meaningful role of network topology in
locomotion. Further, this work enables research into the role of sensory feedback and
high-level control signals in the adaptation of gait types. A better understanding of the
neural control of locomotion relates back to biology where it can provide evidence for
theories that are currently not testable on live insects.
Keywords: CPG (central pattern generator), spiking neural network (SNN), hexapod robot, biological neuron,
neuromorphic engineering
1. INTRODUCTION
In nature, oscillating networks called central pattern generators (CPGs) are known to drive
repetitive behaviors like locomotion and breathing in the absence of sensory feedback (Marder
and Bucher, 2001). However, in order to adapt locomotive behaviors such as walking patterns,
environmental interaction is required to inform the frequency, amplitude, and phase of leg
movements (Bidaye et al., 2017). Tripod-like gaits have been observed in deafferented samples but
it is still unknown how large a role network topology plays as compared to sensory feedback when
it comes to these adaptations (Mantziaris et al., 2017). Recent research on the locust, indicates a
balance between neurological network topology and sensory feedback (Reches et al., 2019). This
balance differs depending on the insect, with stick insects on the feedback-driven end of the
spectrum as compared to the more network-driven cockroach and the locust somewhere in the
middle (Reches et al., 2019). Yeldesbay and Daun (2020) study deafferented samples from the stick
insect to develop a theory of which neural architectures are most likely to be found in these insects,
specifically focusing on the levator-depressor muscle pair. Similar to the deafferented experiments,
this study investigates biologically-plausible oscillators without sensory feedback to examine the
effects of network topology on walking behaviors.
Strohmer et al. Flexible Spiking CPGs
Neural networks can replicate natural locomotion patterns
and adaptations on robots. Most of the current research explores
artificial neural networks (ANNs) based on non-spiking neurons
but there is a growing body of research into spiking neural
networks (SNNs) (Bing et al., 2018). Non-spiking neural
networks represent activation as a number to mimic the spike
rate of a neuron population whereas spiking neurons represent
activation as temporal events, communicating through action
potentials called spikes. Importantly, spiking neuron models are
able to incorporate time (Walter et al., 2016) because biological
studies indicate that neurons communicate through frequency
of spikes as well as precise spike timing (Bohte, 2004). Spiking
neuron models have also been shown to use fewer neurons
for modeling some computations (Maass, 1997). Furthermore,
spiking neurons are able to produce bursting behavior which
may be important for biological fidelity as a single neural
network can initiate different muscle behaviors by changing
output bursting patterns (Diehl et al., 2013). Additionally,
implementing spiking neurons for deep learning has been
shown to decrease power consumption when performing
classification tasks, though it comes at the cost of additional
delays (Han et al., 2016). Overall, biological plausibility, energy
efficiency, increased speed and computational power, and
spatio-temporal processing capabilities make spiking neurons
desirable for researching biologically-inspired locomotion
networks (Bing et al., 2018).
A review of adaptive inter-limb control networks using
ANNs was completed by Aoi et al. (2017) focusing on
the importance of environmental feedback for coordination.
Other works such as Owaki et al. (2013) and Owaki et al.
(2017) look into decentralized inter-leg coordination driven by
environmental interaction without neural connectivity between
legs. These studies confirm adaptability during walking on a
quadruped and a hexapod robot, respectively. Similarly, Sun
et al. (2018) looks into adaptive self-organized locomotion
showing that decoupled non-spiking CPGs (nCPGs) each
controlling a single leg can use sensory feedback to learn
biologically-plausible quadruped gaits. Self-organizing behavior
and adaptability are also shown by Barikhan et al. (2014)
using decoupled nCPGs and sensory feedback where robots
were able to roll a ball and mimic front leg stepping
behavior of insects. These works explore the importance of
environmental interaction on coordination using non-spiking
neurons. As Aoi et al. (2017) point out, “Legged robots are
becoming a valuable tool for understanding the locomotion
mechanism including interlimb coordination. In the future,[...]
it will be important to enhance biological plausibility and
feasibility by the integration with sophisticated models of neural
and musculoskeletal systems[...].” This study addresses the
need for more biological neural dynamics by using spiking
neurons. We explore the effects of decentralized spiking CPGs
(sCPGs) without feedback as a primitive control network to
gain understanding in a simplistic manner before introducing
sensory inputs.
There is a significant body of research into learning using non-
spiking neurons which investigates closed-loop feedback control
and online adaptation of legged robot locomotion. Thor and
Manoonpong (2019) and Pitchai et al. (2019) use learning to
find efficient parameters within nCPGs. The controllers are able
to adapt amplitude, frequency, and phase of nCPGs within the
network. The manipulation of these characteristics can be used
to change walking speed (Thor and Manoonpong, 2019), traverse
obstacles (Goldschmidt et al., 2012), or compensate for loss of
leg functionality (Ren et al., 2015). The mentioned studies use
non-spiking neurons, working with traditional ANNs to execute
adaptive walking patterns. Building upon the ideas and results
from these studies, this work manipulates amplitude, frequency,
and phase of sCPGs.
A thorough treatment of sCPG controllers can be found
in Gutierrez-Galan et al. (2020) though more focused on
engineering solutions than biological. Most notably, Gutierrez-
Galan et al. (2020) and Rostro-Gonzalez et al. (2015) have
implemented sCPGs capable of online switching between three
discrete gaits. These correspond to the three main gaits observed
during insect walking—wave gait (Hughes, 1952), tetrapod
coordination, and tripod coordination (Graham, 1972) reflecting
slow, medium, and fast walking speeds, respectively.
Although some studies using SNNs for robot locomotion have
been inspired by CPGs, they mainly focus on the engineering
task of getting the robot to walk and less on studying the CPG.
As such, Gutierrez-Galan et al. (2020) determine the gait by
the network topology, using the spiking neurons to produce
a regular frequency which drives the selection of the gait.
Rostro-Gonzalez et al. (2015) implement an architecture where a
single neuron controls each motor and connections are changed
discretely between these neurons to produce the desired behavior.
In this way, these studies are able to successfully produce a
signal with a steady frequency to produce distinct walking
gaits but stray from the known biology. ANNs using non-
spiking neurons such as Walknet (Schilling et al., 2013) adopt
the switching principle as well. Walknet uses a decentralized
structure to control 18 degrees of freedom of a hexapod robot.
The controller selects the best fit for the current conditions
in order to adapt walking patterns using synaptic weights
previously tuned offline. These works switch between gaits
using pre-defined network parameters to achieve each pattern.
However, biological studies indicate that insects change gaits
continuously (Dürr et al., 2018). Therefore, this study expands
upon the previous work by implementing a controller capable
of online changes as a first step toward a controller that can
continuously switch between gaits once feedback is added.
Instead of producing a spike train with a regular frequency, this
study uses the intrinsic properties of spiking neurons to produce
regular bursting which is filtered to create an analog output for
the motors.
The aim of this paper is to explore how the network
structure can generate coordinated walking behavior. The
primary contribution of this research is the implementation
of a distributed network of online-adjustable coupled sCPGs
which results in a coordinated walking behavior. This research
takes the first step toward a real-time adaptable spiking motor
control network. Inspired by the biological approach of studying
deafferented neural networks, we explore the limitations of an
sCPG network when lacking sensory feedback.
Frontiers in Neurorobotics | www.frontiersin.org 2June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
2. METHODS
Biological central pattern generators (bCPGs) are used in
nature to generate repetitive behaviors including locomotion.
There are two main methods which have been observed
for creating rhythmic output. Some networks are driven by
“pacemaker neurons” while others create oscillatory output
through recurrent connections between non-intrinsically
rhythmic neurons (commonly called half-center oscillators).
The pacemaker driven networks can be composed of one or
several neurons driving oscillation. They are typically pre-motor
interneurons which drive the motor neurons. The motor neurons
are able to shape the behavior of the bCPG even if they do not
contribute directly to the rhythmic output. Neuromodulators
also contribute to the updating of bCPGs, changing synaptic
strength as well as intrinsic cell properties to affect frequency and
phase (Marder and Bucher, 2001).
Figure 1A shows the generic structure of the sCPGs used
in this study with mutually inhibitory neuron populations
sending oscillatory output to a motor neuron population. The
sCPGs implemented in this study are designed as pacemaker
networks, taking advantage of neuron parameters that allow
regular bursting. As the neurons are intrinsically firing, excitation
is created through a static current bias together with gaussian
white noise.
Each pacemaker network consists of two neuron populations
mutually inhibiting each other. This configuration mimics
the biological setup of two coupled neuron populations for
controlling an antagonistic muscle pair (Bidaye et al., 2017).
Initial testing of the network began with just one neuron per
population, inspired by the sea slug Dendronotus iris. It has
one of the smallest networks found in nature consisting of two
half-center oscillators, a total of four neurons, generating anti-
phasic motor outputs for swimming (Sakurai and Katz, 2016).
Once the output is confirmed, the population sizes are increased
until spiking stabilized from all motor neuron populations.
Based on this testing, each neuron population consists of five
neurons. Results per population size can be found in the
Supplementary Material.
2.1. Neuron Model
The adaptive exponential integrate and fire (AdEx) neuron
model is selected for the network because it is able to model
different neuron firing patterns, including bursting behaviors,
while reducing computational complexity (Brette and Gerstner,
2005). The Izhikevich neuron model is also able to produce
bursting behaviors (Izhikevich, 2003) and we expect the network
behavior could be reproduced using Izhikevich neurons.
Equations (1) and (2) define the dynamics of the adaptive
exponential integrate and fire neuron model.
CdV
dt = −gL(VEL)+gL(1T)eVVT
1Tw+Ie(1)
when V>VTthen VVr
τw
dw
dt =a(VEL)w(2)
when V>VTthen ww+b
where C is membrane capacitance, Vis membrane voltage, EL
is resting potential, gLis leakage conductance, Ieis bias current,
ais sub-threshold adaptation conductance, bis spike-triggered
adaptation, 1Tis slope/sharpness factor, τwis adaptation time
constant, VTis spike threshold, and wis spike adaptation current.
In Equation (1), the first term is the leakage while the second
term is the exponential non-linearity defining sharpness. At the
reset threshold, ’V’ is reset to a fixed variable (Vr) whereas ‘w’ is
reset by a fixed amount (b) allowing it to accumulate during the
spike train and enabling bursting behaviors (Naud et al., 2008).
The values to be used are extracted from the “regular bursting”
parameters as defined by Naud et al. (2008). However, three
parameters are manually tuned through trial and error: VT,
Ie, and tref . The paper does not define a refractory period so
a biologically-plausible refractory period of 2 ms is selected
(Roeder, 1998). Once this is established, Iecan be determined by
using the original (Naud et al., 2008) parameters and increasing
the bias current incrementally. Finally, the range of VTis tested
by stepping through a range and finding the minimum and
maximum values. During these trials, the resulting parameters
are considered usable when they result in stable oscillatory
behavior within a desirable frequency range.
Table 1 outlines the values used for the adaptive exponential
integrate and fire neuron model vs. the values from the Naud et al.
(2008) paper. The shape of the sCPG is constructed using the
regular bursting behavior of the neurons. However, the current
is increased to promote repetitive spiking (Naud et al., 2008)
and gaussian white noise is added to mimic noise in the nervous
system (Faisal et al., 2008). The mean and standard deviation of
the noise is manually found through testing as investigation into
these values is outside the scope of this work.
The second ordinary differential equation allows for the
possibility of natural periodic behavior stemming from the
complex dynamics of a two-dimensional system. For this
research, the voltage dependent conductance-based model of
neuron dynamics (Equation 1) is used together with an alpha
synaptic function (Equation 3). A current-based model was also
tested but the same oscillations could not be achieved so it was
not implemented.
gsyn(t)=gsyn α2teαt(3)
This function describes the change in conductance after a
spike, allowing current to flow along the synapse. It includes
a term for synaptic rise time (α) and the strength of
the synapse (g), both terms influence how often a neuron
spikes (Van Vreeswijk et al., 1994).
The neuron and synapse models have been chosen to allow
for intrinsically bursting, online-adaptable periodic output. This
creates an output reminiscent of a sinusoidal once a low pass filter
is applied. The AdEx neuron model consists of two differential
equations creating a two dimensional system. Two-dimensional
systems have closed orbits allowing for periodic behavior to
emerge, making them well-suited for driving oscillations.
Frontiers in Neurorobotics | www.frontiersin.org 3June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
FIGURE 1 | (A) Representation of the spiking CPG implementation, 1 and 2 represent the two neuron populations within the sCPG. The motor neuron population is
excited by the “first” population within each sCPG. The motor neuron population post-processes the output before relaying it to the joint. (B) Representation of the
coupling between two legs as well as intra-leg coupling. TC represents the thoraxcoxa joint and CF represents the coxa-femur joint. Each of these consists of a
mutually inhibitory pair of neuron populations but is represented by a single circle for figure simplicity. The motor neuron populations are driven by excitatory
connections from one of the neuron populations, as seen in (A). The FT, femur-tibia, joint is held fixed. (C) Representation of the coupling between the six legs both
intra- and inter-segment. Each leg representation consists of two coupled sCPGs as shown in (B). FL, front left; ML, middle left; BL, back left; FR, front right; MR,
middle right; BR, back right. The segments are named for clarity. (D) Illustration of a single leg of the MORF (Thor, 2019), identifying the location of the movable joints.
(E) Picture of the hexapod robot used in this study. (F) Spike events vs. rate-coding output of a single population consisting of 5 neurons. Spike events are added
based on a sliding time window of 5 ms. (G) Zoomed in selection of plot F to visualize individual spike events. The number of spikes inside the time window are
counted, the resulting value is indicated using a black dot. This shows how spike events translate into an analog output. The simulation data allows the rate-coding
algorithm to look forward in time but live experiments would use past data.
2.2. Network Topology
The implemented sCPG network architecture is biologically-
inspired by the stick insect, using one sCPG for each of the joints.
The three main leg joints identified in stick insects are the thorax-
coxa (TC), coxa-trochanter (coxa-femur) (CF), and femur-tibia
(FT) (see Figure 1D for location of these joints on the robot
leg). These are responsible for horizontal, vertical, and extension
movements respectively (Bidaye et al., 2017). The individual
sCPGs are coupled to the other joints within the leg and between
the legs in order to create a distributed network of oscillators.
Figure 1B is a high-level overview of the coupling between
two legs as well as within a single leg. Two joints per leg
are implemented and coupled to their contralateral counterpart
to form the inter-leg connection. In order to meaningfully
Frontiers in Neurorobotics | www.frontiersin.org 4June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
incorporate the FT joint, the load sensory feedback must be
provided as inhibitory feedback to the “flexor” neuron of the
bCPG. This allows the “extensor” neuron to hold the stance phase
longer so that the insect does not enter swing phase when there is
a substantial load on the leg (Fukuoka et al., 2015). The FT joint
is held fixed in this study due to the lack of sensory feedback.
The joint coupling is weak, varying in a range from 0.5 to
1.0 nS while the sCPG, inter-leg, and inter-segment coupling is
a stronger coupling of 10 nS. Inter-segment coupling is shown
in Figure 1C, it also consists of a mutual inhibitory connection
between each joint and its neighboring ipsilateral counterpart.
The complete network diagram with all neuron populations and
connections can be found in the Supplementary Material.
The coupling of legs and joints to their immediate neighbors
is based on observations in nature where sensory feedback to a
single leg affects those closest to it (Mantziaris et al., 2017). The
design of the coupled network allows for frequency, amplitude,
TABLE 1 | Parameter comparison for adaptive exponential integrate and fire
neurons Equations (1) and (2).
Parameter name Symbol Naud et al. (2008) Actual value
Neuron model parameters
Capacitance C200 pA 200 pA
Leakage conductance gL10 ns 10 ns
Resting potential EL58 mV 58 mV
Spike threshold VT50 mV Varied
Slope factor 1T2 mV 2 mV
Adaptation time constant τw120 ms 120 ms
Sub-threshold adaptation conductance a2 ns 2 ns
Spike-triggered adaptation b100 pA 100 pA
Reset voltage Vr46 mV 46 mV
Bias current Ie210 pA 500 pA
Refractory period tref 0 ms 2 ms
and phase to be updated while running, relying on inter-leg and
inter-segment coupling to coordinate leg movements.
2.3. Manipulation of Network
Characteristics
The scaling parameter, VT, is used to update the inter-burst
interval frequency of the oscillators. VToffsets the membrane
potential as it defines the minimum of the V-nullcline (Naud
et al., 2008) thereby affecting frequency.
As the frequency increases, the amount of spikes per burst
decreases. Therefore, the coupling weight between joints must be
increased to compensate for the reduction in spiking. A desirable
coupling weight is found through testing of the network. In the
equation, frequency is represented by VTwhich has a linear
relationship to frequency, as shown in Figure 2B.
wcoupling =(VT_min VT)0.1 1 (4)
VT_min is the minimum threshold value discovered to give
meaningful results from the implemented network topology.
Equation (4) is derived through testing, results showing
frequency adaptation and the affect on joint coupling can be
found in section 3.
The amplitude (magnitude) of muscle activation during
stepping is usually updated in response to sensory feedback
(Schmitz et al., 2015). Biological research suggests that rhythmic
motor patterns are likely affected by short-term plasticity
(McDonnell and Graham, 2017). This is mimicked artificially
here by changing the synaptic weight. Due to the open-loop
nature of the network, the change in amplitude is prompted by
updating the excitatory weights to the motor neuron populations
by a set value. Once sensory feedback and synaptic learning are
incorporated into the network, this will replace the manually
selected weight values.
The phase shift between sCPGs relies on the ability to know
the frequency of the individual sCPGs. Furthermore, they must
FIGURE 2 | (A) Rate-coded sCPG output from each neuron, left and right, when using only two neurons in total. VT=56 mV. Rate-coding mimics the role of a
muscle which acts as a low pass filter (Hooper et al., 2007). (B) Observed frequency at each voltage threshold value using 1 mV steps.
Frontiers in Neurorobotics | www.frontiersin.org 5June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
run at the same frequency or at a whole number multiple relative
to each other in order to create a meaningful phase-lock. A whole
number ratio relating the period of each leg’s stepping cycle has
also been observed when studying walking in stick insects (Foth
and Bässler, 1985). The inter-leg and inter-segment phase shift
is a consequence of the coupling between oscillators. However,
the interjoint phase shift of 90between the body-coxa and coxa-
femur joints is chosen based on the requirements of the physical
robot used for validation (Thor, 2019). The delay to create a
90phase shift between the TC and CF joints in a single leg
is calculated using the output frequency of the sCPG as shown
in Equation (5).
delay =1
2(frequency 1) 1000 (5)
Using the delay calculation, when outputting a frequency of
3Hz, a delay of 250ms will be applied. Note, this is theoretically
a 90phase shift for 2Hz (period = 500ms Hhalf
period = 250ms), manual adjustment of the equation is found
through testing.
2.4. Experimentation
Simulation with NEST (Jordan et al., 2019) on a PC and
validation on a physical robot are used to confirm the sCPG
network design. The sCPG output plots use rate coding to plot
spikes in time bins of finite amounts. The filter is implemented
as a sliding time window. This is comparable to the muscles in
biology which act as low pass filters, filtering variations in spike
timing (Hooper et al., 2007). Figures 1F,G show how spike events
are added in order to provide an analog output for the motors.
The original code used to create the plots in section 3 is available
on GitLab (Strohmer, 2020). Initially, a simple two neuron sCPG
is simulated in order to confirm spiking and coupling behaviors
using the neuron parameters defined in Table 1 before moving
onto the full network simulation.
Manual testing revealed that the voltage threshold must be
constrained within a defined range from 56 to 51 mV when
using a current bias of 500 pA. A linear relationship between
frequency and spike threshold is observed when using the
defined parameters.
Real-time adaptation of amplitude is normally governed by
sensory feedback. For testing, the excitatory synapse to each
FIGURE 3 | (A,B) Rate-coded output of the left legs (A) and right legs (B) updating VTevery 1 s (10,000 time steps). VT=56 to 51 mV. Front and back legs on
the same side spike together indicating a tripod gait. (C,D) Amplitude of the rate-coded output is updated by increasing the excitatory synaptic weight exponentially,
multiplying the weight by 3 for each increase in VToccurring every 1 s. This shows the amplitude can be managed while the system is running.
Frontiers in Neurorobotics | www.frontiersin.org 6June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
motor neuron population is increased exponentially by a factor
of three to show that amplitude can be updated while running.
The excitatory synaptic weight starts at 2 nS, ending at 486 nS
after the six iterations defined by the voltage threshold range. The
weight range is determined through trial and error.
The delay between joints must be adapted when the frequency
changes in order to keep the phase. The delay is calculated
by Equation (5), ranging from 71 to 250 ms based on
attainable frequencies. It is updated each time the voltage
threshold changes. The delay is placed on the excitatory synaptic
connection from the coxa-femur joint to the corresponding
motor neurons.
A validation test is performed on the Modular Robot
Framework (MORF) (Thor, 2019) shown in Figure 1E. It is
both a simulated robot in V-REP (Rohmer, 2013) and a physical
hexapod robot with 18 degrees freedom. Each leg has three
movable joints corresponding to the TC, CF, and FT joints as
seen in Figure 1D. The MORF is fitted with an Intel NUC
which runs as a ROS node to send and receive data wirelessly
from a PC. The values produced in the NEST simulation by
the sCPG network are exported to comma delimited files, one
per joint. This data is used to send position values to the
relevant joint on the robot at a rate of 60 Hz as required
(Thor, 2019). A python script translates the original sCPG
output values to radians and enforces an upper and lower
bound to ensure the motors operate within their designed
range of motion. The python script is available on GitLab
(Strohmer, 2020), it handles reading and writing to the robot
through a robot operating system (ROS) interface. The network
is first simulated with VT= −56mV and again when
updating the threshold from VT= −55 to 54mV. The
joint positions for all 12 joints to be controlled are recorded
for each NEST simulation. These values are tested on the
simulated MORF in V-REP before confirming locomotion on the
physical robot.
Torque measurements are recorded from the Dynamixel
motors during robot locomotion in order to plot the swing and
stance phase of each leg. The torque feedback is actually a current
measurement converted to torque using the “torque to current
value ratio” defined by Dynamixel (ROBOTIS, 2018). The load of
the robot is mostly divided between the CF and FT joint during
walking as can be intuited from Figure 1D. Therefore, the torque
FIGURE 4 | Coupling comparison showing the stability added by fully coupling sCPGs to each other. (A,B) Leg coordination when only “first” neuron populations are
coupled to their corresponding counterpart in the neighboring sCPG. (C,D) Leg coordination when both neuron populations of each sCPG are coupled to their
corresponding counterpart in the neighboring sCPG. These simulations indicate that connectivity between sCPGs informs network stability.
Frontiers in Neurorobotics | www.frontiersin.org 7June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
values plotted are the sum of the absolute values of these two
joints. As torque is the rotational force measurement, negative
and positive only indicate a direction so plotting the absolute
values is valid.
The experiments performed on the MORF are offline as the
position values sent to the motors are pre-recorded data obtained
from the NEST simulations. However, the NEST simulations
are considered to be online manipulations as they calculate
updated network parameters and change neural characteristics
while running. This is achieved by dividing the simulation into
smaller time intervals of one second each and updating necessary
information between each interval. All rate-coded plots are based
on NEST simulation output data. The torque measurements
are recorded during MORF walking and the gait diagram
is based on data recorded during V-REP simulation of the
MORF walking.
3. RESULTS AND DISCUSSION
The output of a two neuron sCPG shown in Figure 2A
confirms spiking and anti-phasic outputs due to mutual
inhibitory coupling.
This result indicates that the neuron parameters producing
regular bursting can be used to create an sCPG in the style
of a pacemaker network. When expanding the network to six
legs consisting of two joints each, using a single neuron per
population is no longer viable. It does not allow for all spikes to
reach the motor neurons controlling the CF joints of the middle
or back legs. Further simulations (see Supplementary Material)
reveal that a minimum of five neurons per population yield a
more reliable result.
Figures 3A,B shows the frequency updating over time, VT
is incremented by 1mV every 1 second resulting in an increase
in frequency as well as a reduction in the amount of spikes
per time bin. Figure 2B shows the observed frequency for each
whole value of VT. The smallest voltage threshold shows a
frequency of 3 Hz, this increases to 8 Hz for the largest tested
value. The frequencies are also observable by counting the peaks
in Figures 3A,B.
The results show that the inter-leg coupling holds the phase
during the frequency adaptation so that after the initial transients,
the front and back legs on the same side move in-phase.
This indicates that a stable tripod gait is maintained with the
implemented network coupling.
FIGURE 5 | (A,B) sCPG output of front left (A) and front right (B) joints with a 90phase shift. (C,D) sCPG output of front left (C) and front right (D) joints with a 90
phase shift adapting to frequency. VTis incremented by 1 mV every 1 s (10,000 time steps). Delay is updated based on VTusing Equation (5).
Frontiers in Neurorobotics | www.frontiersin.org 8June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
Figures 3C,D show the adjustment of amplitude while the
frequency is increasing. This is in contrast to Figures 3A,B where
the number of spikes decreases significantly. The 1 s threshold
updates are clearly visible in Figures 3A,B whereas they are
evened out over the last 3 s in Figures 3C,D.
Even though the starting thresholds produce larger
amplitudes, they are still smaller than the initial amplitudes
shown in the frequency adaptation plot (Figures 3A,B)
indicating that updating the excitatory synaptic weight does
change the number of output spikes as expected. Even though
the initial transients are more apparent in Figures 3C,D, after
settling, the tripod gait returns due to the mutually inhibitory
connections between legs.
Coupling corresponding neurons in sCPGs added stability
to inter-leg and inter-segment phase differences. The legs are
able to move in a stable, coordinated pattern as can be seen in
Figures 4C,D. The single joint coupling shows that the front and
back legs do not always move in-phase (Figures 4A,B). However,
once coupling between corresponding neuron populations in
each sCPG is introduced, the front and back legs move in-
phase creating a steady tripod gait. The front left leg does not
have the same amplitude because of the sequential nature of the
simulation where the front sCPG joint outputs are calculated
first combined with the use of the minimal neuron population
size of 5. Once the neuron populations are increased to 6, the
amplitudes for the front legs are comparable to the other legs.
The joint output for each leg when using six neurons is shown in
the Supplementary Material.
Figures 5A,B show the 90phase shift between the TC and
CF joints in each of the front legs. When using a delay of 166 ms
for 3 Hz, the expected delay required, the outputs of the joints
are in-phase. Increasing the delay to 250 ms for 3 Hz produces
the desired 90phase shift. Figures 5C,D confirm the phase is
able to adapt to a change in frequency and keep a specified delay.
However, for the highest frequency, the phase takes an initial
adjustment period where the joints are in-phase before moving
FIGURE 6 | (A) Sum of the absolute values of torque measurements from the CF and FT joints for each leg on the left side of the MORF while walking at a single
frequency. The front and back legs move together while the middle leg is out of phase compared to both. This shows the tripod gait holds when testing on a physical
robot. (B) Sum of the absolute values of torque measurements from the CF and FT joints for the middle legs of the MORF while walking at a single frequency. The left
and right legs move out of phase, confirming a tripod gait. (C) Sum of the absolute values of torque measurements from the CF and FT joints for each leg on the left
side of the MORF while switching to a new frequency. After an initial transient, the legs are able to re-adjust and return to a tripod gait. (D) Sum of the absolute values
of torque measurements from the CF and FT joints for the middle legs of the MORF while switching to a new frequency. The left and right legs move out of phase
though the stance phase is visibly shorter for the left leg after the frequency transition. (E) Stills taken from the video of the MORF walking.
Frontiers in Neurorobotics | www.frontiersin.org 9June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
FIGURE 7 | Gait diagram based on force sensor feedback collected during simulation in V-REP. The force measurement from the CF and FT joints are rectified and
added together. If this absolute force measurement is above a specific threshold based on the extracted data, it is determined as foot contact with the ground.
Therefore, the colored points indicate stance phase and confirm a tripod gait.
toward a 90phase shift again. This indicates the delay and joint
coupling weights are not optimized.
Figure 6 confirms a tripod gait is maintained when the robot is
walking. The front and back legs move in-phase while the middle
leg moves out of phase. Likewise, left and right legs within the
same segment move out of phase. There are transients visible
at the end of the stance phase. The front and back legs seem to
compensate for each other during the end of the stance phase
where the front leg increases in torque as the back leg decreases
before both go to zero during their mutual swing phase. This
increase in torque is also visible in the original MORF torque
plots (Thor, 2019) indicating that this could be due to the robot’s
morphology. If the leg slips at the end of the stance phase and
then catches, this would show as a decrease in torque followed by
a spike as depicted in Figure 6.
The torque value when updating the voltage threshold half-
way through from VT= −55mV to 54mV shows a tripod
gait is maintained after a frequency transition during walking.
The transient in the middle of Figures 6C,D pinpoints when
the frequency is changing. After the initial transient, the robot
settles back into a steady tripod gait. However, the stance phase
is shorter for the left middle leg, this is most likely an artifact
of the frequency shift and mutual inhibition promoting anti-
phasic behavior. The lack of sensory feedback limits the ability
to re-adapt to an equal duty cycle for the left and right legs.
Instead, the weight of the connections and delay between sCPGs
will determine how the joints settle into anti-phasic behavior,
eventually defining the length of the swing vs. stance phase for
each. This result indicates that the selected weights and delays
are not optimal but the network is functional.
The gait diagram (Figure 7) confirms the tripod gait. The
overlap of stance phase can be seen. This is due to the slow
walking frequency which creates transition phases where all legs
are in contact with the ground. The stance phase is occasionally
interrupted, this is could be an effect of improper parameter
tuning of the FT joint’s position.
The remainder of the torque plots for the single
and transitioning frequency can be seen in the
Supplementary Material. A link to the video of the MORF
walking can also be found in the Supplementary Material.
The frequency, phase, and amplitude are confirmed to
be interdependent. Changing the frequency of a single joint
affects the phase difference in relation to other joints. This is
addressed by using the same frequency for all joints at any
given time, allowing the mutually inhibitory connections to
force anti-phasic behavior in neighboring legs and seems to
be a robust solution. However, the 90phase shift between
the intra-leg joints created through a delay to the motor
population is more susceptible to frequency changes since it
is a calculated parameter rather than a natural product of the
network topology. The amplitude is also directly influenced
by frequency as less spikes occur per time bin as frequency
increases. Increasing the number of neurons in the motor
population or increasing connectivity from the sCPG population
to adapt to this reduction are potential solutions. However,
these inter-dependencies require deeper investigation to
construct biologically-plausible network topologies with desired
output characteristics.
4. CONCLUSION
This paper introduces a decentralized spiking CPG network
inspired by insect neurobiology. The amplitude, frequency,
and phase can be manipulated while the network is
running indicating that these characteristics can be updated
online to further explore the role of sensory feedback
in shaping locomotion. A tripod gait is achieved based
on a biologically-inspired open-loop network topology
Frontiers in Neurorobotics | www.frontiersin.org 10 June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
indicating that topology does play a part in walking
coordination. This research lays the groundwork for further
investigation into online adaptable spiking networks and
the role of network topology compared to environmental
feedback.
Future work should incorporate sensory feedback into the
distributed sCPG network. Once this is implemented, the
network can begin reacting to its environment. Learning
algorithms should be explored to optimize synaptic strength
to the motor neurons as well as the coupling weights between
the sCPGs. The addition of the third leg joint using load
sensory feedback to mechanically couple this joint to the
existing network should be investigated. This will allow for
a more biologically accurate representation of walking where
the leg is extended and flexed during the swing and stance
phase (Bidaye et al., 2017). Additionally, sensory feedback could
help to adjust phase quicker during frequency adaptations
and may also provide stability through mechanical coupling
of joints.
The creation of a network capable of online, reactive
adaptation to the environment allows for biological hypotheses
to be explored in a controlled environment with detailed
feedback from sensors and actuators. This could provide more
information on the interaction of forces during locomotion
since there are limits when working with live insects and
deafferented samples. Online adaptation also lends itself to
online learning, providing the opportunity to examine stepping
patterns, both rhythmic and non-rhythmic. This leads to more
robust locomotion and exploration behaviors, allowing robots to
navigate rougher terrains.
DATA AVAILABILITY STATEMENT
Publicly available datasets were analyzed in this study. This
data can be found here: https://gitlab.com/esrl/scpg-network-
simulation.
AUTHOR CONTRIBUTIONS
BS and LL conceived of the presented idea. BS reviewed biology
and synthesized the findings to a complete, working sCPG model.
BS formulated the theory and performed the computations. The
manuscript was written by BS with support from LL and PM.
LL supervised the project and provided the funding. All authors
contributed to the article and approved the submitted version.
FUNDING
This research was funded by the SDU Biorobotics group at the
University of Southern Denmark.
ACKNOWLEDGMENTS
We would like to thank Mathias Thor for use of the MORF and
his help interfacing with the robot.
SUPPLEMENTARY MATERIAL
The Supplementary Material for this article can be found
online at: https://www.frontiersin.org/articles/10.3389/fnbot.
2020.00041/full#supplementary-material
REFERENCES
Aoi, S., Manoonpong, P., Ambe, Y., Matsuno, F., and Wörgötter, F. (2017).
Adaptive control strategies for interlimb coordination in legged robots: a
review. Front. Neurorobot. 11:39. doi: 10.3389/fnbot.2017.00039
Barikhan, S. S., Wörgötter, F., and Manoonpong, P. (2014). “Multiple decoupled
CPGs with local sensory feedback for adaptive locomotion behaviors of bio-
inspired walking robots,” in International Conference on Simulation of Adaptive
Behavior (Castellón: Springer), 65–75. doi: 10.1007/978-3-319-08864-8_7
Bidaye, S. S., Bockemühl, T., and Büschges, A. (2017). Six-legged walking in
insects: how CPGs, peripheral feedback, and descending signals generate
coordinated and adaptive motor rhythms. J. Neurophysiol. 119, 459–475.
doi: 10.1152/jn.00658.2017
Bing, Z., Meschede, C., Röhrbein, F., Huang, K., and Knoll, A. C. (2018). A survey
of robotics control based on learning-inspired spiking neural networks. Front.
Neurorobot. 12:35. doi: 10.3389/fnbot.2018.00035
Bohte, S. M. (2004). The evidence for neural information processing
with precise spike-times: a survey. Natural Comput. 3, 195–206.
doi: 10.1023/B:NACO.0000027755.02868.60
Brette, R., and Gerstner, W. (2005). Adaptive exponential integrate-and-fire model
as an effective description of neuronal activity. J. Neurophysiol. 94, 3637–3642.
doi: 10.1152/jn.00686.2005
Diehl, F., White, R. S., Stein, W., and Nusbaum, M. P. (2013). Motor circuit-specific
burst patterns drive different muscle and behavior patterns. J. Neurosci. 33,
12013–12029. doi: 10.1523/JNEUROSCI.1060-13.2013
Dürr, V., Theunissen, L. M., Dallmann, C. J., Hoinville, T., and Schmitz,
J. (2018). Motor flexibility in insects: adaptive coordination of limbs
in locomotion and near-range exploration. Behav. Ecol. Sociobiol. 72:15.
doi: 10.1007/s00265-017-2412-3
Faisal, A. A., Selen, L. P., and Wolpert, D. M. (2008). Noise in the nervous system.
Nat. Rev. Neurosci. 9, 292–303. doi: 10.1038/nrn2258
Foth, E., and Bässler, U. (1985). Leg movements of stick insects walking with five
legs on a treadwheel and with one leg on a motor-driven belt. Biol. Cybern. 51,
313–318. doi: 10.1007/BF00336918
Fukuoka, Y., Habu, Y., and Fukui, T. (2015). A simple rule for quadrupedal gait
generation determined by leg loading feedback: a modeling study. Sci. Rep.
5:8169. doi: 10.1038/srep08169
Goldschmidt, D., Hesse, F., Wörgötter, F., and Manoonpong, P. (2012).
“Biologically inspired reactive climbing behavior of hexapod robots,” in
2012 IEEE/RSJ International Conference on Intelligent Robots and Systems,
4632–4637. Algarve: IEEE. doi: 10.1109/IROS.2012.6386135
Graham, D. (1972). A behavioural analysis of the temporal organisation of walking
movements in the 1st instar and adult stick insect (carausius morosus). J.
Compar. Physiol. 81, 23–52. doi: 10.1007/BF00693548
Gutierrez-Galan, D., Dominguez-Morales, J. P., Perez-Pe na, F., Jimenez-
Fernandez, A., and Linares-Barranco, A. (2020). Neuropod: a real-time
neuromorphic spiking CPG applied to robotics. Neurocomputing 381, 10–19.
doi: 10.1016/j.neucom.2019.11.007
Han, B., Sengupta, A., and Roy, K. (2016). “On the energy benefits of
spiking deep neural networks: a case study,” in 2016 International Joint
Conference on Neural Networks (IJCNN), 971–976. Vancouver, BC: IEEE.
doi: 10.1109/IJCNN.2016.7727303
Hooper, S. L., Guschlbauer, C., von Uckermann, G., and Buschges, A. (2007).
Different motor neuron spike patterns produce contractions with very
similar rises in graded slow muscles. J. Neurophysiol. 97, 1428–1444.
doi: 10.1152/jn.01014.2006
Hughes, G. M. (1952). The co-ordination of insect movements: I the walking
movements of insects. J. Exp. Biol. 29, 267–285.
Frontiers in Neurorobotics | www.frontiersin.org 11 June 2020 | Volume 14 | Article 41
Strohmer et al. Flexible Spiking CPGs
Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Trans. Neural
Netw. 14, 1569–1572. doi: 10.1109/TNN.2003.820440
Jordan, J., Mørk, H., Vennemo, S. B., Terhorst, D., Peyser, A., Ippen, T., et al.
(2019). Nest 2.18.0. Available onine at: https://doi.org/10.5281/zenodo.2605422
(accessed April 02, 2020).
Maass, W. (1997). Networks of spiking neurons: the third generation
of neural network models. Neural Netw. 10, 1659–1671.
doi: 10.1016/S0893-6080(97)00011-7
Mantziaris, C., Bockemühl, T., Holmes, P., Borgmann, A., Daun, S., and
Büschges, A. (2017). Intra-and intersegmental influences among central pattern
generating networks in the walking system of the stick insect. J. Neurophysiol.
118, 2296–2310. doi: 10.1152/jn.00321.2017
Marder, E., and Bucher, D. (2001). Central pattern generators and
the control of rhythmic movements. Curr. Biol. 11, R986–R996.
doi: 10.1016/S0960-9822(01)00581-4
McDonnell, M. D., and Graham, B. P. (2017). Phase changes in neuronal
postsynaptic spiking due to short term plasticity. PLoS Comput. Biol.
13:e1005634. doi: 10.1371/journal.pcbi.1005634
Naud, R., Marcille, N., Clopath, C., and Gerstner, W. (2008). Firing patterns
in the adaptive exponential integrate-and-fire model. Biol. Cybern. 99:335.
doi: 10.1007/s00422-008-0264-7
Owaki, D., Goda, M., Miyazawa, S., and Ishiguro, A. (2017). A minimal model
describing hexapedal interlimb coordination: the tegotae-based approach.
Front. Neurorobot. 11:29. doi: 10.3389/fnbot.2017.00029
Owaki, D., Kano, T., Nagasawa, K., Tero, A., and Ishiguro, A. (2013). Simple robot
suggests physical interlimb communication is essential for quadruped walking.
J. R. Soc. Interface 10:20120669. doi: 10.1098/rsif.2012.0669
Pitchai, M., Xiong, X., Thor, M., Billeschou, P., Mailänder, P. L., Leung,
B., et al. (2019). “CPG driven RBF network control with reinforcement
learning for gait optimization of a dung beetle-like robot,” in International
Conference on Artificial Neural Networks (Munich: Springer), 698–710.
doi: 10.1007/978-3-030-30487-4_53
Reches, E., Knebel, D., Rillich, J., Ayali, A., and Barzel, B. (2019). The
metastability of the double-tripod gait in locust locomotion. iScience 12, 53–65.
doi: 10.1016/j.isci.2019.01.002
Ren, G., Chen, W., Dasgupta, S., Kolodziejski, C., Wörgötter, F., and Manoonpong,
P. (2015). Multiple chaotic central pattern generators with learning for
legged locomotion and malfunction compensation. Inform. Sci. 294, 666–682.
doi: 10.1016/j.ins.2014.05.001
ROBOTIS (2018). Robotis-Framework. Available online at: https://github.com/
ROBOTIS-GIT/ROBOTIS- Framework/blob/master/robotis_device/devices/
dynamixel/XM430-W350.device (accessed February 28, 2020).
Roeder, K. D. (1998). Nerve Cells and Insect Behavior, Vol. 4. Harvard University
Press.
Rohmer, E., Singh, S. P. N., and Freese, M. (2013). “Coppeliasim (formerly V-REP):
a versatile and scalable robot simulation framework, in Proceedings of The
International Conference on Intelligent Robots and Systems (IROS), (Tokyo).
Available online at: www.coppeliarobotics.com.
Rostro-Gonzalez, H., Cerna-Garcia, P. A., Trejo-Caballero, G., Garcia-Capulin, C.
H., Ibarra-Manzano, M. A., Avina-Cervantes, J. G., et al. (2015). A CPG system
based on spiking neurons for hexapod robot locomotion. Neurocomputing 170,
47–54. doi: 10.1016/j.neucom.2015.03.090
Sakurai, A., and Katz, P. S. (2016). The central pattern generator
underlying swimming in dendronotus iris: a simple half-center network
oscillator with a twist. J. Neurophysiol. 116, 1728–1742. doi: 10.1152/jn.
00150.2016
Schilling, M., Hoinville, T., Schmitz, J., and Cruse, H. (2013). Walknet, a
bio-inspired controller for hexapod walking. Biol. Cybern. 107, 397–419.
doi: 10.1007/s00422-013-0563-5
Schmitz, J., Gruhn, M., and Büschges, A. (2015). The role of leg touchdown for the
control of locomotor activity in the walking stick insect. J. Neurophysiol. 113,
2309–2320. doi: 10.1152/jn.00956.2014
Strohmer, B. (2020). Decentralized SCPG Network Simulation. Available online
at: https://gitlab.com/esrl/scpg-network- simulation (accessed February
28, 2020).
Sun, T., Shao, D., Dai, Z., and Manoonpong, P. (2018). “Adaptive neural
control for self-organized locomotion and obstacle negotiation of quadruped
robots,” in 2018 27th IEEE International Symposium on Robot and
Human Interactive Communication (RO-MAN), 1081–1086. Nanjing: IEEE.
doi: 10.1109/ROMAN.2018.8525645
Thor, M. (2019). MORF -Modular Robot Framework (Masters thesis). University
of Southern Denmark, Odense, Denmark.
Thor, M., and Manoonpong, P. (2019). A fast online frequency adaptation
mechanism for CPG-based robot motion control. IEEE Robot. Autom. Lett. 4,
3324–3331. doi: 10.1109/LRA.2019.2926660
Van Vreeswijk, C., Abbott, L., and Ermentrout, G. B. (1994). When inhibition
not excitation synchronizes neural firing. J. Comput. Neurosci. 1, 313–321.
doi: 10.1007/BF00961879
Walter, F., Röhrbein, F., and Knoll, A. (2016). Computation by time. Neural
Process. Lett. 44, 103–124. doi: 10.1007/s11063-015-9478-6
Yeldesbay, A., and Daun, S. (2020). Intra-and intersegmental neural
network architectures determining rhythmic motor activity in
insect locomotion. Commun. Nonlin. Sci. Numer. Simul. 82:105078.
doi: 10.1016/j.cnsns.2019.105078
Conflict of Interest: The authors declare that the research was conducted in the
absence of any commercial or financial relationships that could be construed as a
potential conflict of interest.
Copyright © 2020 Strohmer, Manoonpong and Larsen. This is an open-access article
distributed under the terms of the Creative Commons Attribution License (CC BY).
The use, distribution or reproduction in other forums is permitted, provided the
original author(s) and the copyright owner(s) are credited and that the original
publication in this journal is cited, in accordance with accepted academic practice.
No use, distribution or reproduction is permitted which does not comply with these
terms.
Frontiers in Neurorobotics | www.frontiersin.org 12 June 2020 | Volume 14 | Article 41
... Pure SNNs are also capable of manipulating output amplitude, frequency, and phase by updating different synaptic and neuronal characteristics (Strohmer et al., 2020). It was found that frequency could be changed by updating the value of the voltage threshold potential of the spiking central pattern generator (sCPG) neuron populations while amplitude was increased or decreased using the weight of the synaptic conductance to the motor neuron population (MNP). ...
... A diagram of the implemented network is shown in Figure 3A. The network consists of an NSI capable of injecting The sCPG populations and MNP consist of 5 neurons each in order to produce a smooth output with a minimum amount of neurons (Strohmer et al., 2020). The NSI is a single neuron so that all post-synaptic neurons connected to it receive the same inputs. ...
... The divisor in Equation (4) limits the voltage characteristic offset within a stable range. It is determined by dividing the biologically plausible 15mV NSI membrane potential fluctuation range (Burrows and Siegler, 1978) with the 5mV voltage characteristic manipulation range known to be stable for the sCPG network (Strohmer et al., 2020). ...
Article
Full-text available
Researchers working with neural networks have historically focused on either non-spiking neurons tractable for running on computers or more biologically plausible spiking neurons typically requiring special hardware. However, in nature homogeneous networks of neurons do not exist. Instead, spiking and non-spiking neurons cooperate, each bringing a different set of advantages. A well-researched biological example of such a mixed network is a sensorimotor pathway, responsible for mapping sensory inputs to behavioral changes. This type of pathway is also well-researched in robotics where it is applied to achieve closed-loop operation of legged robots by adapting amplitude, frequency, and phase of the motor output. In this paper we investigate how spiking and non-spiking neurons can be combined to create a sensorimotor neuron pathway capable of shaping network output based on analog input. We propose sub-threshold operation of an existing spiking neuron model to create a non-spiking neuron able to interpret analog information and communicate with spiking neurons. The validity of this methodology is confirmed through a simulation of a closed-loop amplitude regulating network inspired by the internal feedback loops found in insects for posturing. Additionally, we show that non-spiking neurons can effectively manipulate post-synaptic spiking neurons in an event-based architecture. The ability to work with mixed networks provides an opportunity for researchers to investigate new network architectures for adaptive controllers, potentially improving locomotion strategies of legged robots.
... On the other hand, refs. [55,56] used spiking neurons instead of oscillators for the design of a CPG architecture to improve the computational efficiency and due to the possibility of using temporal events as activation functions. Despite the method adopted, all these publications provided similar results in terms of generating tripod, wave and metachronal gaits and safely transitioning between them. ...
... The main interest of using bio-inspired architectures is to generate adaptive locomotion through the implementation of sensory feedback by adjusting the parameters and output signal of the oscillator or the spiking neuron when a perturbation occurs in the system [55,58]. The simplest method combined an IMU with a CPG model with three Matsuoka oscillators to control the attitude of a hexapod [40]. ...
Article
Full-text available
The static stability of hexapods motivates their design for tasks in which stable locomotion is required, such as navigation across complex environments. This task is of high interest due to the possibility of replacing human beings in exploration, surveillance and rescue missions. For this application, the control system must adapt the actuation of the limbs according to their surroundings to ensure that the hexapod does not tumble during locomotion. The most traditional approach considers their limbs as robotic manipulators and relies on mechanical models to actuate them. However, the increasing interest in model-free models for the control of these systems has led to the design of novel solutions. Through a systematic literature review, this paper intends to overview the trends in this field of research and determine in which stage the design of autonomous and adaptable controllers for hexapods is.
... Reference [81] presented the first implementation of a real-time neuromorphic spiking CPG (sCPG) that runs on the SpiNNaker to command a hexapod robot to perform a walk, trot, or run motion. Reference [82] implemented sCPG with an AdEx neuron model that exhibits a tripod-like gait. Their model can manipulate the amplitude, frequency, and phase while the network is running, indicating that these characteristics can be updated in an online manner to further explore the role of sensory feedback in shaping locomotion. ...
Article
Full-text available
The past decade has witnessed the great success of deep neural networks in various domains. However, deep neural networks are very resource-intensive in terms of energy consumption, data requirements, and high computational costs. With the recent increasing need for the autonomy of machines in the real world, e.g., self-driving vehicles, drones, and collaborative robots, exploitation of deep neural networks in those applications has been actively investigated. In those applications, energy and computational efficiencies are especially important because of the need for real-time responses and the limited energy supply. A promising solution to these previously infeasible applications has recently been given by biologically plausible spiking neural networks. Spiking neural networks aim to bridge the gap between neuroscience and machine learning, using biologically realistic models of neurons to carry out the computation. Due to their functional similarity to the biological neural network, spiking neural networks can embrace the sparsity found in biology and are highly compatible with temporal code. Our contributions in this work are: (i) we give a comprehensive review of theories of biological neurons; (ii) we present various existing spike-based neuron models, which have been studied in neuroscience; (iii) we detail synapse models; (iv) we provide a review of artificial neural networks; (v) we provide detailed guidance on how to train spike-based neuron models; (vi) we revise available spike-based neuron frameworks that have been developed to support implementing spiking neural networks; (vii) finally, we cover existing spiking neural network applications in computer vision and robotics domains. The paper concludes with discussions of future perspectives.
... In [12], the open-loop CPG is implemented on Loihi [13] using an astrocytic network, producing two different gaits with 24 motor neurons. In, [14], authors propose the implementation of a CPG with the possibility of changing the amplitude, frequency and phase online without any sensory input required. The authors also pointed out that the architecture should include any sensory feedback to modify the behavior of the CPG. ...
Article
In recent years, locomotion mechanisms exhibited by vertebrate animals have been the inspiration for the improvement in the performance of robotic systems. These mechanisms include the adaptability of their locomotion to any change registered in the environment through their biological sensors. In this regard, we aim to replicate such kind of adaptability through a sCPG. This sCPG generates different locomotion (rhythmic) patterns which are driven by an external stimulus, that is, the output of a FSR sensor to provide feedback. The sCPG consists of a network of five populations of LIF neurons designed with a specific topology in such a way that the rhythmic patterns can be generated and driven by the aforementioned external stimulus. Therefore, eventually, the locomotion of an end robotic platform could be adapted to the terrain by using any sensor as input. The sCPG with adaptation has been numerically validated at software and hardware level, using the Brian 2 simulator and the SpiNNaker neuromorphic platform for the latest. In particular, our experiments clearly show an adaptation in the oscillation frequencies between the spikes produced in the populations of the sCPG while the input stimulus varies. To validate the robustness and adaptability of the sCPG, we have performed several tests by variating the output of the sensor. These experiments were carried out in Brian 2 and SpiNNaker; both implementations showed a similar behavior with a Pearson correlation coefficient of 0.905.
... Additionally, in vitro studies with insects permit the analysis and characterization of their different subsystems, isolated from the influence of the rest of the body 34 . Insects have also inspired computational models to test various control strategies, making them a versatile model [35][36][37] . While different approaches and electrode designs have been proposed to interface with peripheral nerves in literature 38,39 , the small size of peripheral nerves limits their applicability in insects. ...
Article
Full-text available
Activity of an innervated tissue can be modulated based on an acquired biomarker through feedback loops. How to convert this biomarker into a meaningful stimulation pattern is still a topic of intensive research. In this article, we present a simple closed-loop mechanism to control the mean angle of a locust’s leg in real time by modulating the frequency of the stimulation on its extensor motor nerve. The nerve is interfaced with a custom-designed cuff electrode and the feedback loop is implemented online with a proportional control algorithm, which runs solely on a microcontroller without the need of an external computer. The results show that the system can be controlled with a single-input, single-output feedback loop. The model described in this article can serve as a primer for young researchers to learn about neural control in biological systems before applying these concepts in advanced systems. We expect that the approach can be advanced to achieve control over more complex movements by increasing the number of recorded biomarkers and selective stimulation units.
... The parameter calculations defined within the network architecture depend on frequency. In previous work, Strohmer et al. (26) found a linear relationship between V th and frequency. Therefore, all linear relationship calculations use V th as a substitute to approximate frequency instead of calculating frequency at each time step. ...
Article
Full-text available
The walking system of the stick insect is one of the most thoroughly described invertebrate systems. We know a lot about the role of sensory input in the control of stepping of a single leg. However, the neuronal organization and connectivity of the central neural networks underlying the rhythmic activation and coordination of leg muscles still remain elusive. It is assumed that these networks can couple in the absence of phasic sensory input due to the observation of spontaneous recurrent patterns (SRPs) of coordinated motor activity equivalent to fictive stepping-phase transitions. Here we sought to quantify the phase of motor activity within SRPs in the isolated and interconnected meso- and meta-thoracic ganglia. We show that SRPs occur not only in the meso-, but also in the metathoracic ganglia of the stick insect, discovering a qualitative difference between them. We construct a network based on neurophysiological data capable of reproducing the measured SRP phases to investigate this difference. By comparing network output to the biological measurements we confirm the plausibility of the architecture and provide a hypothesis to account for these qualitative differences. The neural architecture we present couples individual central pattern generators to reproduce the fictive stepping-phase transitions observed in deafferented stick insect preparations after pharmacological activation, providing insights into the neural architecture underlying coordinated locomotion.
... Spiking CPG are used in insect robots' locomotion, to coordinate single leg movements and the coordination of multiple legs. Spiking CPG show stable and coordinated locomotion pattern that can robustly adapt to external disturbances 157 and can be implemented on FPGA 158 . [80][81][82] . ...
Article
Full-text available
The design of robots that interact autonomously with the environment and exhibit complex behaviours is an open challenge that can benefit from understanding what makes living beings fit to act in the world. Neuromorphic engineering studies neural computational principles to develop technologies that can provide a computing substrate for building compact and low-power processing systems. We discuss why endowing robots with neuromorphic technologies – from perception to motor control – represents a promising approach for the creation of robots which can seamlessly integrate in society. We present initial attempts in this direction, highlight open challenges, and propose actions required to overcome current limitations. A grand challenge in robotics is realising intelligent agents capable of autonomous interaction with the environment. In this Perspective, the authors discuss the potential, challenges and future direction of research aimed at demonstrating embodied intelligent robotics via neuromorphic technology.
... In robotics control, oscillator models are usually used other than biophysical models. For some rare cases [43], neuron models are used the spiking neuron model instead of oscillator models and demonstrated that the model can improve computational efficiency together with neuromorphic hardware. As this research is primarily concerned with the gait designs, which are determined by the dynamic properties of the CPG network, oscillator models would be used. ...
Article
Full-text available
Legged robots have significant potential to operate in unstructured environments. The design of locomotion control is, however, still challenging. Currently, controllers must be manually designed for specific robots and tasks, or automatically designed via machine learning methods that require long training times and yield large opaque controllers. Drawing inspiration from animal locomotion, we propose a simple yet versatile modular neural control structure with fast learning. The key advantages of our approach are that behaviour-specific control modules can be added incrementally to obtain increasingly complex emergent locomotion behaviours, and that neural connections can be quickly and automatically learned. In a series of experiments, we show how eight modules can be quickly learned and added to a base control module to obtain emergent adaptive behaviours allowing a hexapod robot to navigate in complex environments. We also show that modules can be added and removed during operation without affecting the functionality of the remaining controller. Finally, the controller is successfully demonstrated on a physical robot. Taken together, our study reveals a significant step towards fast automatic design of versatile neural locomotion control. Controllers for robotic locomotion patterns deal with complex interactions and need to be carefully designed or extensively trained. Thor and Manoonpong present a modular approach for neural pattern generators that allows incremental and fast learning.
Thesis
Full-text available
The advantage of walking robots when compared to robots on wheels is that they can interact with generic physical environments that are either designed for legged motion or complex terrain filled with obstacles. Current solutions to adaptive locomotion for legged robots are promising, but often ineffective and far from able to compete with the behaviors of real animals. This is presumably because the benefits of using legs most often are overshadowed by the high design complexity. Hence, there is a need for a platform which enables researchers to start working on the actual locomotion controller faster. In this work, we present MORF, a MOdular Robot Framework. The framework is intended for a wide range of research studies and is aiming at being easy and convenient to use. The framework consists of a modular multi-legged robot and a software suite. The legged robot, also called MORF, is modular as it defines standards that can be used for reconfiguring, extending, and replacing parts (e.g., body shape). The software suite includes simulations of MORF and hardware interfacing software based on the Robot Operating System. The framework is developed and validated based on a thorough analysis of both existing methods and technical issues. When compared to other modular robot frameworks, MORF is advantageous in areas like processing power, mobility, controllability, completeness (includes a software suite), sensory feedback, and expandability, but lacks an easy mechanism for connecting parts together (e.g., using magnets or threaded collars). This will have to be improved upon in future revisions of MORF to stress and improve the modularity of MORF even more.
Article
Full-text available
In this letter, we present an online learning mechanism called the Dual Integral Learner for fast frequency adaptation in neural Central Pattern Generator (CPG) based locomotion control of a hexapod robot. The mechanism works by modulating the CPG frequency through synaptic plasticity of the neural CPG network. The modulation is based on tracking error feedback between the CPG output and joint angle sensory feedback of the hexapod robot. As a result, the mechanism will always try to match the CPG frequency to the walking performance of the robot, thereby ensuring that the entire generated trajectory can be followed with low tracking error. Real robot experiments show that our mechanism can automatically generate a proper walking frequency for energy-efficient locomotion with respect to the robot body as well as being able to quickly adapt the frequency online within a few seconds to deal with external perturbations such as leg blocking and a variation in electrical power. These important features will allow a hexapod robot to be more robust and also extend its operating time. Finally, the generality of the mechanism is shown by successfully applying it to a compliant robotic manipulator arm called GummiArm.
Code
Full-text available
NEST is a simulator for spiking neural network models that focuses on the dynamics, size and structure of neural systems rather than on the exact morphology of individual neurons. For further information, visit http://www.nest-simulator.org. The release notes for this release are available at https://github.com/nest/nest-simulator/releases/tag/v2.18.0
Article
Full-text available
Insect locomotion represents a fundamental example of neuronal oscillating circuits generating different motor patterns or gaits by controlling their phase coordination. Walking gaits are assumed to represent stable states of the system, often modeled as coupled oscillators. This view is challenged, however, by recent experimental observations, in which in vitro locust preparations consistently converged to synchronous rhythms (all legs oscillating as one), a locomotive pattern never seen in vivo. To reconcile this inconsistency, we developed a modeling framework to capture the trade-off between the two competing mechanisms: the endogenous neuronal circuitry, expressed in vitro, and the feedback mechanisms from sensory and descending inputs, active only in vivo. We show that the ubiquitously observed double-tripod walking gait emerges precisely from this balance. The outcome is a short-lived meta-stable double-tripod gait, which transitions and alternates with stable idling, thus recovering the observed intermittent bouts of locomotion, typical of many insects' locomotion behavior.
Article
Full-text available
Biological intelligence processes information using impulses or spikes, which makes those living creatures able to perceive and act in the real world exceptionally well and outperform state-of-the-art robots in almost every aspect of life. To make up the deficit, emerging hardware technologies and software knowledge in the fields of neuroscience, electronics, and computer science have made it possible to design biologically realistic robots controlled by spiking neural networks (SNNs), inspired by the mechanism of brains. However, a comprehensive review on controlling robots based on SNNs is still missing. In this paper, we survey the developments of the past decade in the field of spiking neural networks for control tasks, with particular focus on the fast emerging robotics-related applications. We first highlight the primary impetuses of SNN-based robotics tasks in terms of speed, energy efficiency, and computation capabilities. We then classify those SNN-based robotic applications according to different learning rules and explicate those learning rules with their corresponding robotic applications. We also briefly present some existing platforms that offer an interaction between SNNs and robotics simulations for exploration and exploitation. Finally, we conclude our survey with a forecast of future challenges and some associated potential research topics in terms of controlling robots based on SNNs.
Article
Full-text available
In recent years, research on insect motor behaviour―locomotion in particular―has provided a number of important new insights, many of which became possible because of methodological advances in motion capture of unrestrained moving insects. Behavioural analyses have not only backed-up neurophysiological analyses of the underlying mechanisms at work, they have also highlighted the complexity and variability of leg movements in naturalistic, unrestrained behaviour. Here, we argue that the variability of unrestrained motor behaviour should be considered a sign of behavioural flexibility. Assuming that variation of movement-related parameters is governed by neural mechanisms, behavioural analyses can complement neurophysiological investigations, for example by (i) dissociating distinct movement episodes based on functional and statistical grounds, (ii) quantifying when and how transitions between movement episodes occur, and (iii) dissociating temporal and spatial coordination. The present review emphasises the importance of considering the functional diversity of limb movements in insect behaviour. In particular, we highlight the fundamental difference between leg movements that generate interaction forces as opposed to those that do not. On that background, we discuss the spatially continuous modulation of swing movements and the quasi-rhythmic nature of stepping across insect orders. Based on examples of motor flexibility in stick insects, we illustrate the relevance of behaviour-based approaches for computational modelling of a rich and adaptive movement repertoire. Finally, we emphasise the intimate interplay of locomotion and near-range exploration. We propose that this interplay, through continuous integration of distributed, multimodal sensory feedback, is key to locomotor flexibility.
Article
Initially, robots were developed with the aim of making our life easier, carrying out repetitive or dangerous tasks for humans. Although they were able to perform these tasks, the latest generation of robots are being designed to take a step further, by performing more complex tasks that have been carried out by smart animals or humans up to date. To this end, inspiration needs to be taken from biological examples. For instance, insects are able to optimally solve complex environment navigation problems, and many researchers have started to mimic how these insects behave. Recent interest in neuromorphic engineering has motivated us to present a real-time, neuromorphic, spike-based Central Pattern Generator of application in neurorobotics, using an arthropod-like robot. A Spiking Neural Network was designed and implemented on SpiNNaker. The network models a complex, online-change capable Central Pattern Generator which generates three gaits for a hexapod robot locomotion in real-time. Reconfigurable hardware was used to manage both the motors of the robot and the low-latency communication interface with the Spiking Neural Networks. Real-time measurements confirm the simulation results, and locomotion tests show that NeuroPod can perform the gaits without any balance loss or added delay.
The coordinated movement of the extremities of an animal during locomotion is achieved by the interaction between groups of neurons called central pattern generators (CPGs). In the absence of any sensory input this network creates a stable rhythmic motor activity that is essential for a successful coordination between limbs. Studying the structure and the interaction between different parts of the CPG network is therefore of particular interest. This work is motivated by recent experimental results reported by Mantziaris et al. (2017) [12]. By chemically activating both isolated and interconnected deafferented thoracic segments (ganglia) of the stick insect Mantziaris et al. (2017) [12] analyzed the interactions between contralateral networks that drive the levator-depressor muscle pairs, which are responsible for the upward-downward movement of the legs. The results of the experimental analysis showed that intrasegmental phase relations differ between isolated segments. In particular, in isolated segments where the control networks of the middle and hind legs reside, i.e. in the meso- and metathoracic ganglia, the phase relations between activities of the contralateral depressor motoneurons were in-phase and anti-phase, respectively. Moreover, the phase relations switched to in-phase and stabilized when the ganglia were interconnected. Using the phase reduction of an intersegmental network model of stick insect locomotion presented in our previous work (Yeldesbay et al. (2017) [22]), we built a reduced model of the intra- and intersegmental network controlling levator-depressor activity in the meso- and metathoracic ganglia. By examining the intra- and intersegmental phase differences in the model we identified the properties of the network couplings that replicate the results observed in the experiments. We applied the theoretical analysis to escape type CPGs and revealed a set of possible contra- and ipsilateral synaptic connections. Finally, we defined general features of the couplings between CPGs of any type that maintain the phase relations observed in the experiments.
Chapter
In this paper, we employ a central pattern generator (CPG) driven radial basis function network (RBFN) based controller to learn optimized locomotion for a complex dung beetle-like robot using reinforcement learning approach called “Policy Improvement with Path Integrals (PI\(^2\))”. Our CPG driven RBFN controller is inspired by rhythmic dynamic movement primitives (DMPs). The controller can be also seen as an extension to a traditional CPG controller, which usually controls only the frequency of the motor patterns but not the shape. Our controller uses the CPG to control the frequency while the RBFN takes care of the shape of the motor patterns. In this paper, we only focus on the shape of the motor patterns and optimize those with respect to walking speed and energy efficiency. As a result, the robot can travel faster and consume less power than using only the CPG controller.