Conference PaperPDF Available

A Model for Foraging Ants, Controlled by Spiking Neural Networks and Double Pheromones

Authors:

Abstract and Figures

—A model of an Ant System where ants are controlled by a spiking neural circuit and a second order pheromone mechanism in a foraging task is presented. A neural circuit is trained for individual ants and subsequently the ants are exposed to a virtual environment where a swarm of ants performed a resource foraging task. The model comprises an associative and unsupervised learning strategy for the neural circuit of the ant. The neural circuit adapts to the environment by means of classical conditioning. The initially unknown environment includes different types of stimuli representing food (rewarding) and obstacles (harmful) which, when they come in direct contact with the ant, elicit a reflex response in the motor neural system of the ant: moving towards or away from the source of the stimulus. The spiking neural circuits of the ant is trained to identify food and obstacles and move towards the former and avoid the latter. The ants are released on a landscape with multiple food sources where one ant alone would have difficulty harvesting the landscape to maximum efficiency. In this case the introduction of a double pheromone mechanism (positive and negative reinforcement feedback) yields better results than traditional ant colony optimization strategies. Traditional ant systems include mainly a positive reinforcement pheromone. This approach uses a second pheromone that acts as a marker for forbidden paths (negative feedback). This blockade is not permanent and is controlled by the evaporation rate of the pheromones. The combined action of both pheromones acts as a collective stigmergic memory of the swarm, which reduces the search space of the problem. This paper explores how the adaptation and learning abilities observed in biologically inspired cognitive architectures is synergistically enhanced by swarm optimization strategies. The model portraits two forms of artificial intelligent behaviour: at the individual level This work has been accepted for presentation at the UK Workshop on Computational Intelligence — University of Exeter, September 2015 http: //www.ukci2015.ex.ac.uk/ the spiking neural network is the main controller and at the collective level the pheromone distribution is a map towards the solution emerged by the colony. The presented model is an important pedagogical tool as it is also an easy to use library that allows access to the spiking neural network paradigm from inside a Netlogo—a language used mostly in agent based modelling and experimentation with complex systems. If you are interested in the netlogo file please contact me at cristian.jimenez-romero@open.ac.uk
Content may be subject to copyright.
1
A Model for Foraging Ants, Controlled by Spiking
Neural Networks and Double Pheromones
Cristian Jimenez-Romero, David Sousa-Rodrigues, and Jeffrey H. Johnson
Faculty of Maths, Computing and Technology
The Open University
Milton Keynes
MK7 6AA, United Kingdom
Emails: cristian.jimenez-romero@open.ac.uk,
david.rodrigues@open.ac.uk, jeff.johnson@open.ac.uk
Vitorino Ramos
LaSEEB - Evolutionary Systems and Biomedical Engineering Lab.,
IST, Technical University of Lisbon,
Lisbon, Portugal
Email: vitorino.ramos@ist.utl.pt
F
Abstract—A model of an Ant System where ants are controlled by a
spiking neural circuit and a second order pheromone mechanism in a
foraging task is presented. A neural circuit is trained for individual ants
and subsequently the ants are exposed to a virtual environment where a
swarm of ants performed a resource foraging task. The model comprises
an associative and unsupervised learning strategy for the neural circuit
of the ant. The neural circuit adapts to the environment by means
of classical conditioning. The initially unknown environment includes
different types of stimuli representing food (rewarding) and obstacles
(harmful) which, when they come in direct contact with the ant, elicit
a reflex response in the motor neural system of the ant: moving towards
or away from the source of the stimulus. The spiking neural circuits
of the ant is trained to identify food and obstacles and move towards
the former and avoid the latter. The ants are released on a landscape
with multiple food sources where one ant alone would have difficulty
harvesting the landscape to maximum efficiency. In this case the in-
troduction of a double pheromone mechanism (positive and negative
reinforcement feedback) yields better results than traditional ant colony
optimization strategies. Traditional ant systems include mainly a positive
reinforcement pheromone. This approach uses a second pheromone
that acts as a marker for forbidden paths (negative feedback). This
blockade is not permanent and is controlled by the evaporation rate of
the pheromones. The combined action of both pheromones acts as a
collective stigmergic memory of the swarm, which reduces the search
space of the problem. This paper explores how the adaptation and
learning abilities observed in biologically inspired cognitive architectures
is synergistically enhanced by swarm optimization strategies. The model
portraits two forms of artificial intelligent behaviour: at the individual level
This work has been accepted for presentation at the UK Workshop on
Computational Intelligence — University of Exeter, September 2015 http:
//www.ukci2015.ex.ac.uk/
the spiking neural network is the main controller and at the collective
level the pheromone distribution is a map towards the solution emerged
by the colony. The presented model is an important pedagogical tool as
it is also an easy to use library that allows access to the spiking neural
network paradigm from inside a Netlogo—a language used mostly in
agent based modelling and experimentation with complex systems.
1 INTRODUCTION
The exploration of artificially constructed entities that
simulate biology counterparts has a long tradition in
the field of Artificial Intelligence, and in particular in
Artificial life. Langton proposed the study of artificial life
with celular automata in 1996 [1] where the author aimed
to “implement the ‘molecular logic of the living state’ in
an artificial biochemistry environment” via modeling the
artificial molecules as “virtual automata that were able
to roam in an abstract computer space and interact with
each other”.
Ant colony based algorithms have been applied suc-
cessfully to several domains, namely for the clustering
of web usage mining [2], image retrieval [3], newspaper
and document organisation [4], [5], and mainly to the
travelling salesman problem [6], [7], [8], [9], [10], [11],
among other combinatorial optimisation problems.
In nature, many ant species have used trail-laying
when foraging. They deposit pheromone, a chemical
substance that is volatile and is secreted by the ants
when returning to the nest carrying food, that acts as
a recruitment mechanism. This pheromone is detected
2
by other colony ants that can use it as an indicator
for the food source. This process of ant recruitment is
a positive reinforcement mechanism as more ants are
driven to the foraging task and subsequently deposit
more pheromone. This positive amplification allows the
colony to exploit the food source in optimal time. In the
model presented here the only process of recruitment is
based on chemicals trails and therefore is refereed as mass
recruitment [6].
Social insects are very well adapted to solve the for-
aging problem. The resilience of the species relies on
the flexibility presented by social insects to changing
problem landscapes. As sources of food are depleted
their strategy needs to allow them to find newer sources
and they need to adapt quickly to new situations. The
use of pheromones allows them to collectively adapt to
changing environments, and the colonies become robust
entities even if some individuals fail to perform their
tasks. This “swarm intelligence” reflects the fact that
these social insects are capable of self-organisation. Self-
organisation is a process, whereby the dynamics of the
parts of the system, a high order structure emerges. Self-
organisation systems like ant colonies present defining
features: they exhibit some kind of positive feedback
mechanism, they exhibit also negative feedback mecha-
nisms, they amplify fluctuations observed in nature and
the self-organisation relies on the existence of multiple
interactions. Although one single ant can deposit a trail
of pheromone, the benefit of these trails is only observed
when many ants interact together via the recruitment
process.
While ants can interact with other ants in a direct
way, via antennation or prophallaxis for example, in
this model we only model indirect interaction via the
environment. This process is known as stigmergy. One
ant deposits pheromones at a certain time and in a
certain location and other ants will interact with the
deposited pheromone at a latter time.
Following this introduction, section 2 presents the
model in its constituent parts. Subsections 2.1 and 2.2
discuss the use of spiking neural networks—compared
to traditional artificial neural networks and in the context
of controllers for autonomous systems, respectively. Sub-
section 2.3 presents the spiking neural network submodel
while subsection 2.4 shows the brain of the virtual ant.
It is followed in subsection 2.5 by the description of the
double pheromone mechanism and section 2 concludes
with details of the implementation in subsection 2.6.
Illustrative results are presented in section 3 and it is
followed by the conclusion in section 4.
2 A MODEL FOR INTERNAL+EXTERNAL ART I-
FIC IAL INTELLIGENCE
2.1 Spiking versus traditional Neural Networks
In traditional Artificial Neural Networks (ANNs) models
(e.g. McCulloch-Pitts and sigmoidal neurons) the neural
activity is represented by the neurons firing rates, thus
the timing between single pulses (Pulse code) is not
taken into account [12]. On the other hand, Spiking
Neural Networks (SNNs) are able to represent neural
activity in terms of both rate and pulse codes. The pulse
code which makes use of the information contained
in the interspike interval has been associated with fast
information processing in the brain in cases where the
time required for the integration and computation of
average rates (rate code) would take too long (i.e., a
housefly can change the direction of its flight in reaction
to visual stimuli in about 30 milliseconds)[12]. In addi-
tion to the enhanced computational capabilities added by
the time dimension, the resemblance of SNNs dynamics
with their biological counterparts is significantly more
accurate than in first and second generations ANNs.
SNNs are capable of simulating a broad range of learning
and spiking dynamics observed in biological neurons
including: Spike timing dependent plasticity (STDP),
long term potentiation and depression (LTP/LTD), tonic
and phasic spike, inter-spike delay (latency), frequency
adaptation, resonance and input accommodation [13].
The STDP rule mentioned above, is implemented in
the proposed neural circuit as the underlying mecha-
nism for associative and classical conditioning learning.
Experimental results have demonstrated that different
types of classical conditioning (i.e., Pavlovian, extinction,
partial conditioning, inhibitory conditioning) can be im-
plemented successfully in SNNs. [14], [15].
2.2 Suitability of Spiking neural networks for con-
trolling autonomous systems (robots and agents)
The Capabilities shown by insects (given their lower
neural complexity compared to vertebrates) to interact
and cope with the environment including: exploration,
reliable navigation, pattern recognition and interactions
with each other, are being considered as key features for
implementation in the design of autonomous robots [16].
Based on the fact that SNNs reproduce to some extent
the computational characteristics of biological neural sys-
tems, SNNs prove to be a potential computational instru-
ment to achieve the above mentioned features in artificial
systems. There is increasing research (e.g., [14], [16],
[17], [18] ) on the use of SNNs to control autonomous
systems which exhibit intelligent behaviour in terms of
learning and adaptation to the environment. Wang et
al. [18] compares the implementation of SNNs with tra-
ditional ANNs autonomous controllers highlighting the
following advantages in the SNN approach: (1) Espatio-
temporal information is used more efficiently in SNN
than in ANNs. (2) The topology of the SNNs controller is
much simpler than in ANNs. (3) The (hebbian) training
method in the SNNs was easier to implement than in
ANNs. SNNs are demonstrating that their application as
artificial neural controllers in autonomous systems is not
only advantageous in computational terms (when com-
pared with previous connectionists models) but it also
3
allows the implementation of biologically inspired neural
systems (e.g., [16], [15], [19]) to be used in machines.
2.3 The Spiking Neural Network (SNN) model
The SNN model implemented in Netlogo [20] by Jimenez
and Johnson [21] has been used for the simulation of the
neural circuit of the ants. This SNN model is a simpli-
fied but functional version of integrate and fire neuron
models [12] [22] aimed at pedagogical purposes and
experimentation with small spiking neural circuits. The
artificial neuron is implemented as a finite-state machine
where the states transitions depend on a variable which
represents the membrane potential of the cell. All the
characteristics of the artificial neuron including: (1) mem-
brane potential, (2) resting potential, (3) spike threshold,
(4) excitatory and inhibitory postsynaptic response, (5)
exponential decay rate and (6) absolute and refractory
periods, are enclosed in two possible states: open and
absolute-refractory.
Fig. 1. Modeling of the membrane potential in the imple-
mented SNN model
In the open-state the artificial neuron is receptive to
input pulses coming from presynaptic neurons. The am-
plitude of postsynaptic potentials elicited by presynaptic
pulses is given by the function psp() (see figure 1). The
membrane perturbations reported by psp() are added
(excitatory postsynaptic potential EPSP) or subtracted
(inhibitory postsynaptic potential IPSP) to the actual
value of the membrane potential u. If the neuron firing
threshold ϑis not reached by u, then ubegins to decay
(decay() function in figure 1) towards a fixed resting
potential. On the other hand, if the membrane potential
reaches a set threshold, an action potential or spiking
process is initiated. In the used model, when ureaches
the firing threshold ϑ, this triggers a state transition
from the open to the absolute-refractory state. During
the latter, uis set to a fixed refractory potential value av
and all incoming presynaptic pulses are neglected by u.
Fig. 1 illustrates the behaviour of the membrane potential
in response to incoming presynaptic spikes.
2.4 The brain for the virtual ant
Fig. 2. Neural circuit controller of the virtual ant
Learning with the Spike Timing Dependent Plasticity
(STDP) rule
In this paper, the STDP model proposed by Gerstner et
al. 1996 [23] has been implemented as the underlying
learning mechanism for the ants neural circuit. In STDP
the synaptic efficacy is potentiated or depressed accord-
ing to the difference between the arrival time of incoming
pre-synaptic spikes and the time of the action potential
triggered at the post-synaptic neuron.
The following formula [23] describes the weight
change of the synapse according to the STDP model for
pre-synaptic and post-synaptic neurons jand irespec-
tively: Here, the arrival time of the pre-synaptic spikes
is indicated by tf
jwhile tn
irepresents the firing time at
the post-synaptic neuron:
wj=
N
X
j=1
N
X
n=1
W(tn
itf
j)(1)
The weight change resulting from the combination of
a pre-synaptic spike with a post-synaptic action potential
is given by the function W(∆t)[23].
W(∆t) = A+exp (∆t/τ+),if t < 0
Aexp (t/τ),if t0(2)
where tis the time difference between the arriving
pre-synaptic spike and the post-synaptic action potential.
4
A+and Adetermine the amplitude of the weight
change when increasing or decreasing respectively. τ+
and τdetermine the reinforce and inhibitory interval
or size of the learning window.
Topology of the neural circuit
The neural circuit presented in this work (Fig. 2) enables
a simulated ant to move in a two dimensional world,
learning to identify and avoid noxious stimuli while
moving towards perceived rewarding stimuli. At the
beginning of the training phase, the ant is not aware of
which stimuli are to be avoided or pursued. Learning
occurs through reward-and-punishment classical condi-
tioning [24]. Here the ant learns to associate the informa-
tion represented by different colours with unconditioned
reflex responses. In terms of classical conditioning, learn-
ing can be described as the association or pairing of a
conditioned or neutral stimulus with an unconditioned
stimulus (one that elicits an innate or reflex response).
Thus, the neutral or conditioned stimulus acquires the
ability to elicit the same response or behaviour produced
by the unconditioned stimulus. The pairing of two un-
related stimuli usually occurs by repeatedly presenting
the neutral stimulus shortly before the unconditioned
stimulus that elicits the innate response. In the context
of classical conditioning in animals, the word ”shortly”
refers to a time interval of a few seconds (or in some
cases a couple of minutes). On the other hand, at the
cellular level and in the context of STDP, the association
of stimuli encoded as synaptic spikes occurs in short
milliseconds intervals [23].
Sensory system
The neural circuit of the ant is able to process three
types of sensorial information: (1) olfactory, (2) pain
and (3) pleasant or rewarding sensation. The olfactory
information is acquired through three olfactory receptors
(see figure 2) where each one of them is sensitive to one
specific smell represented with a different color (white,
red or green). Each olfactory receptor is connected with
one afferent neuron which propagates the input pulses
towards the Motoneurons. Pain is elicited by a nociceptor
whenever the insect collides with a wall or another
obstacle. Finally, a rewarding or pleasant sensation is
elicited when the insect gets in direct contact with a
positive stimulus (i.e. food).
Motor system
The motor system allows the virtual ant to move forward
or rotate in one direction according to the reflexive
behaviour described below in Fig. 2. In order to keep
the ant moving even in the absence of external stimuli,
the motoneuron M is connected to a neural oscillator
sub-circuit composed of two neurons H1 and H2 (Fig.
2) performing the function of a pacemaker which sends
a periodic pulse to M. The pacemaker is initiated by
a pulse from an input neuron which represents an ex-
ternal input current (i.e; intracellular electrode). Fig. 2
illustrates the complete neural anatomy of the ant.
Pheromone system
The positive and negative pheromone systems are con-
trolled by the two neurons Pp and Np respectively. The
neuron Pp is activated by the reward sensor F resulting
in the release of positive pheromone whenever the ants
gets in contact with food (or any other positive stimulus
associated with the activation of F). On the other hand,
the neuron Np is activated by the summation of pulses
coming periodically from the oscillator sub-circuit (neu-
rons H1 and H2). Np works as an energy-consumption
counter which fires unless it is inhibited by the reward
sensor F. Thus, whenever the ant gets in contact with
food the energy counter is reinitialized.
2.5 The Double Pheromone as the basis for collec-
tive intelligence
Traditional models of ant colony systems use mainly a
positive pheromone feedback mechanism, meaning that
they simulate a single chemical being secreted by the
ants and that this chemical acts as an attractor for other
ants to follow the pheromone trail. Recent findings on
colonies of monomorium pharaonis ants show that this
species uses a negative pheromone to help repel foragers
from unrewarding areas of the landscape[25], [26]. This
empirical evidence shows that this second chemical acts
as a ‘no entry’ signal that ants deposit when they find
unrewarding paths.
The application of a double pheromone mechanism
in artificial ant systems has been shown to improve
the performance of ant colony optimisation problems,
as the use of a negative ‘no entry’ pheromone reduces
the exploration space in symetric travelling salesman
problems [5], [11], [27], [28].
Following these ideas in the proposed model, ants
explore the landscape according to rules established
by the internal intelligence (or brain) dictated by the
responses of the spiking neural network. The deposition
of pheromones is controlled by the occurrence under two
situations:
Negative Pheromone deposition:
Occurs after a threshold time since food was
last found by the ant.
Positive Pheromone deposition:
Occurs immediately after an ant finds food and
the deposition of positive pheromone persists
for a parameterized amount of time after that.
2.6 Implementation in Netlogo
In Netlogo there are four types of agents: Turtles,
patches, links and the observer [29]. The virtual ants
are represented by turtle agents as well as each neuron
in the implemented SNN model. Synapses on the other
5
hand are represented by links. The produced pheromone
is represented by patches. All simulated entities includ-
ing the insect, neurons and synapses have their own
variables and functions that can be manipulated using
standard Netlogo commands. The Netlogo virtual world
consists of a two dimensional grid of patches where each
patch corresponds to a point (x, y)in the plane. In a simi-
lar way to the turtles, the patches own a set of primitives
which allow the manipulation of their characteristics and
also the programming of new functionalities and their
interaction with other agents. The visualization of the
ants and their environment is done through the Netlogo’s
world-view interface.
The virtual world of the ant is an ensemble of patches
of four different colours, where each one of them is
associated with a different type of stimulus. White and
Red patches are both used to represent harmful stimulus.
Thus, if the ant is positioned on a white or red patch,
this will trigger a reaction in the ant’s nociceptor (pain
sensor) and its corresponding neural pathway (Fig. 2).
On the other hand, green patches trigger a reaction in
the reward sensor of the ants whenever it is positioned
on one of them. Black patches represent empty spaces
and do not trigger any sensory information in the ant at
all.
Fig. 3. Short trajectories at the beginning of the training
phase. The ant collides and escapes the world repeatedly.
At the beginning of the training phase (Fig. 3) the ant
moves along the virtual-world colliding indiscriminately
with all types of patches. the ant is repositioned in its
initial coordinates every time it reaches the virtual-world
boundaries. As the training phase progresses it can be
seen that the trajectories lengthen as the ant learns to
associate the red and white patches with harmful stimuli
and consequently to avoid them (Fig. 4).
Once the training phase has been completed, the con-
ditioned aversion to white and red patches is exploited
by using the white patches to delimit the boundaries of
the virtual world of the ants ( using white as walls )
while the red patches are used to represent the negative
pheromone released by the ant.
Fig. 4. Long trajectory shows ant avoiding red and white
patches.
3 RE SULTS
Figures 5–8 illustrate a sequence of the ants’ movements
using the double pheromone.
Fig. 5. No Pheromone.
Fig. 5 shows the ants swarm moving through the
virtual world. Since the virtual ants can only react to
stimuli located directly in front of them , they manage to
avoid the obstacles delimiting the virtual world (brown
or white patches), however they are not able to detect
the food sources located inside the virtual world.
Fig. 6 shows that when the ants start releasing the
pheromone their otherwise monotonous trajectories are
affected as the pheromone constitutes a new obstacle
which is avoided thus creating several random trajec-
tories for the ants.
Fig. 7 and 8 shows that the ants follow new trajectories
which allow them to find the food and eat it. At the same
time the negative pheromones released, by occupying
previously empty spaces in the virtual world reduce the
areas where the ants move to find food (the search space).
Fig. 8 shows that after several iterations the food sources
have been exhausted. The foraging activity is shown in
6
Fig. 6. After 200 iterations with activated Pheromone.
Fig. 7. After 1000 iterations with activated Pheromone.
Fig. 8. After 8000 iterations with activated Pheromone
Fig. 9 that illustrates the available food over time in the
landscape.
Fig. 9. Amount of available food during simulation with
activated pheromone.
4 CONCLUSION
Combining internal and external forms of intelligence—
or at least forms of individual and societal decision
processes—is a challenging problem. It is one task that
benefits from building on top of biological findings. In
this work an agent based model of an Ant System was
presented were both the individual ants and the colonies
are organised based on biological principles.
The model encompasses a combined synergy between
internal individual intelligence and external collective
intelligence. The former is presented in the form of a
Spiking Neural Network while the latter is comprised of
a double pheromone based space exploration and mass
recruitment.
It was shown how a Spiking Neural Network can be
used to endow each ant with the ability to recognise
rewarding and harmful stimuli. In this paper we demon-
strated how associative learning in SNN can be used
to allow accurate navigation of virtual ants in a two
dimensional environment. Although the demonstrated
association tasks are based on simplified action-reaction
(sensor-actuator) relationships, it is possible to extend the
neural circuit to associate more complex input patterns
(i.e. bitmap images and other sensor arrays) with differ-
ent types of actuators in order to produce more complex
and intelligent behaviour.
It was also shown how the results obtained by the indi-
vidual ants are enhanced at the collective level by using
a double pheromone mechanism. This self-organisation
principle allows the collective—even in a small scale
model as the one presented—to exhibit emergent features
and to solve the problem of foraging by communicating
through pheromone deposited in the landscape. This
deposition acts as a memory—even if temporary, because
of the action of evaporation—that allows the colony to
improve their foraging task.
The model depicts two forms of intelligence in an easy
to use and easy to understand software package that
introduces two important paradigms of artificial life to a
vast community of scientists.
REFERENCES
[1] C. G. Langton, “Studying artificial life with cellular automata,”
Physica D: Nonlinear Phenomena, vol. 22, no. 1–3, pp. 120
– 149, 1986, proceedings of the Fifth Annual International
Conference. [Online]. Available: http://www.sciencedirect.com/
science/article/pii/016727898690237X
7
[2] A. Abraham and V. Ramos, “Web usage mining using artificial
ant colony clustering and linear genetic programming,” in Pro-
ceedings of the Congress on Evolutionary Computation. Australia:
IEEE Press, 2003, pp. 1384–1391.
[3] V. Ramos, F. Muge, and P. Pina, “Self-organized data and image
retrieval as a consequence of inter-dynamic synergistic relation-
ships in artificial ant colonies,” Hybrid Intelligent Systems, vol. 87,
2002.
[4] V. Ramos and J. J. Merelo, “Self-organized stigmergic document
maps: Environment as a mechanism for context learning,” in
Proceddings of the AEB, M´
erida, Spain, February 2002.
[5] D. Sousa-Rodrigues and V. Ramos, “Traversing news with ant
colony optimisation and negative pheromones,” in European Con-
ference in Complex Systems, Lucca, Italy, Sep 2014.
[6] E. Bonabeau, G. Theraulaz, and M. Dorigo, Swarm Intelligence:
From Natural to Artificial Systems, 1st ed., ser. Santa Fe Insitute
Studies In The Sciences of Complexity. 198 Madison Avenue,
New York: Oxford University Press, USA, Sep. 1999.
[7] M. Dorigo and L. M. Gambardella, “Ant colony system: A coop-
erative learning approach to the traveling salesman problem,”
Universit´
e Libre de Bruxelles, Tech. Rep. TR/IRIDIA/1996-5,
1996.
[8] M. Dorigo, G. Di Caro, and L. M. Gambardella, “Ant
algorithms for discrete optimization,” Artif. Life, vol. 5,
no. 2, pp. 137–172, Apr. 1999. [Online]. Available: http:
//dx.doi.org/10.1162/106454699568728
[9] L. M. Gambardella and M. Dorigo, “Ant-q: A reinforcement
learning approach to the travelling salesman problem,” in Pro-
ceedings of the ML-95, Twelfth Intern. Conf. on Machine Learning,
M. Kaufman, Ed., 1995, pp. 252–260.
[10] A. Gupta, V. Nagarajan, and R. Ravi, “Approximation
algorithms for optimal decision trees and adaptive tsp
problems,” in Proceedings of the 37th international colloquium
conference on Automata, languages and programming, ser. ICALP’10.
Berlin, Heidelberg: Springer-Verlag, 2010, pp. 690–701. [Online].
Available: http://dl.acm.org/citation.cfm?id=1880918.1880993
[11] V. Ramos, D. Sousa-Rodrigues, and J. Louc¸ ˜
a, “Second order
swarm intelligence,” in HAIS’13. 8th International Conference on
Hybrid Artificial Intelligence Systems, ser. Lecture Notes in Com-
puter Science, J.-S. Pan, M. Polycarpou, M. Wo´
zniak, A. Carvalho,
H. Quinti´
an, and E. Corchado, Eds. Salamanca, Spain: Springer
Berlin Heidelberg, Sep 2013, vol. 8073, pp. 411–420.
[12] W. Maass and C. M. Bishop, Pulsed Neural Networks. Cambridge,
Massachusetts: MIT Press, 1998.
[13] E. M. Izhikevich and E. M. Izhikevich, “Simple model of
spiking neurons.” IEEE transactions on neural networks / a
publication of the IEEE Neural Networks Council, vol. 14, no. 6,
pp. 1569–72, 2003. [Online]. Available: http://www.ncbi.nlm.
nih.gov/pubmed/18244602
[14] C. Liu and J. Shapiro, “Implementing classical conditioning
with spiking neurons,” in Artificial Neural Networks ICANN
2007, ser. Lecture Notes in Computer Science, J. de S,
L. Alexandre, W. Duch, and D. Mandic, Eds. Springer Berlin
Heidelberg, 2007, vol. 4668, pp. 400–410. [Online]. Available:
http://dx.doi.org/10.1007/978-3-540-74690- 4 41
[15] J. Haenicke, E. Pamir, and M. P. Nawrot, “A spiking
neuronal network model of fast associative learning in the
honeybee,” Frontiers in Computational Neuroscience, no. 149, 2012.
[Online]. Available: http://www.frontiersin.org/computational
neuroscience/10.3389/conf.fncom.2012.55.00149/full
[16] L. I. Helgadottir, J. Haenicke, T. Landgraf, R. Rojas, and M. P.
Nawrot, “Conditioned behavior in a robot controlled by a spiking
neural network,” in International IEEE/EMBS Conference on Neural
Engineering, NER, 2013, pp. 891–894.
[17] A. Cyr and M. Boukadoum, “Classical conditioning in different
temporal constraints: an STDP learning rule for robots controlled
by spiking neural networks,” pp. 257–272, 2012.
[18] X. Wang, Z. G. Hou, F. Lv, M. Tan, and Y. Wang, “Mobile robots’
modular navigation controller using spiking neural networks,”
Neurocomputing, vol. 134, pp. 230–238, 2014.
[19] C. Hausler, M. P. Nawrot, and M. Schmuker, “A spiking neu-
ron classifier network with a deep architecture inspired by the
olfactory system of the honeybee,” in 2011 5th International
IEEE/EMBS Conference on Neural Engineering, NER 2011, 2011, pp.
198–202.
[20] U. Wilensky, “Netlogo,” Evanston IL, USA, 1999. [Online].
Available: http://ccl.northwestern.edu/netlogo/
[21] C. Jimenez-Romero and J. Johnson, “Accepted abstract: Simula-
tion of agents and robots controlled by spiking neural networks
using netlogo,” in International Conference on Brain Engineering and
Neuro-computing, Mykonos, Greece, Oct 2015.
[22] W. Gerstner and W. M. Kistler, Spiking Neuron Models: Single Neu-
rons, Populations, Plasticity. Cambridge: Cambridge University
Press, 2002.
[23] J. v. H. W Gerstner, R Kempter and H. Wagner, “A neuronal
learning rule for sub-millisecond temporal coding,” Nature, vol.
386, pp. 76–78, 1996.
[24] I. P. Pavlov, “Conditioned reflexes: An investigation of the activity
of the cerebral cortex,” New York, 1927.
[25] E. J. H. Robinson, D. E. Jackson, M. Holcombe, and F. L. W.
Ratnieks, “Insect communication: ‘no entry’ signal in ant
foraging,” Nature, vol. 438, no. 7067, pp. 442–442, 11 2005.
[Online]. Available: http://dx.doi.org/10.1038/438442a
[26] E. J. Robinson, D. Jackson, M. Holcombe, and F. L. Ratnieks,
“No entry signal in ant foraging (hymenoptera: Formicidae):
new insights from an agent-based model,” Myrmecological News,
vol. 10, no. 120, 2007.
[27] D. Sousa-Rodrigues, J. Louc¸˜
a, and V. Ramos, “From standard
to second-order swarm intelligence phase-space maps,” in 8th
European Conference on Complex Systems, S. Thurner, Ed., Vienna,
Austria, Sep 2011.
[28] V. Ramos, D. Sousa-Rodrigues, and J. Louc¸ ˜
a, “Spatio-temporal
dynamics on co-evolved stigmergy,” in 8th European Conference
on Complex Systems, S. Thurner, Ed., Vienna, Austria, 9 2011.
[29] S. Tisue and U. Wilensky, “Netlogo: A simple environment
for modeling complexity,” in International conference on complex
systems. Boston, MA, 2004, pp. 16–21.
... There is increasing research (e.g., [41,42,43,44,45]) demonstrating that the third generation of artificial neural networks is emerging as a potential computational tool to control autonomous systems which exhibit intelligent behaviour in terms of learning and adaptation to the environment. ...
Thesis
Full-text available
Artificial neural systems for computation were first proposed three quarters of a century ago and the concepts developed by the pioneers still shape the field today. The first generation of neural systems was developed in the nineteen forties in the context of analogue electronics and the theoretical research in logic and mathematics that led to the first digital computers in nineteen forties and fifties. The second generation of neural systems implemented on digital computers was born in the nineteen fifties and great progress was made in the subsequent half century with neural networks being applied to many problems in pattern recognition and machine learning. Through this history there has been an interplay between biologically inspired neural systems and their implementation by engineers on digital machines. This thesis concerns the third generation of neural networks, Spiking Neural Networks, which is making possible the creation of new kinds of brain inspired computing architectures that offer the potential to increase the level of realism and sophistication in terms of autonomous machine behaviour and cognitive computing. This thesis presents the development and demonstration of a new theoretical architecture for third generation neural systems, the Integrate-andFire based Spiking Neural Model with extended Neuro-modulated Spike Timing Dependent Plasticity capabilities. This proposed architecture overcomes the limitation of the homosynaptic architecture underlying existing implementations of spiking neural networks that it lacks a natural spike timing dependent plasticity regulation mechanism, and this results in ‘run away’ dynamics. To overcome this ad hoc procedures have been implemented to overcome the ‘run away’ dynamics that emerge from the use of spike timing dependent plasticity among other hebbian-based plasticity rules. The new heterosynaptic architecture presented, explicitly abstracts the modulation of complex biochemical mechanisms into a simplified mechanism that is suitable for the engineering of artificial systems with low computational complexity. Neurons work by receiving input signals from other neurons through synapses. The difference between homosynaptic and heterosynaptic plasticity is that, in the former the change in the properties of a synapse (e.g. synaptic efficacy) depends on the point to point activity in either of the sending and receiving neurons, in contrast for heterosynaptic plasticity the change in the properties of a synapse can be elicited by neurons that are not necessary presynaptic or postsynaptic to the synapse in question. The new architecture is tested by a number of implementations in simulated and real environments. This includes experiments with a simulation environment implemented in Netlogo, and an implementation using Lego Mindstorms as the physical robot platform. These experiments demonstrate the problems with the traditional Spike timing dependent plasticity homosynaptic architecture and how the new heterosynaptic approach can overcome them. It is concluded that the new theoretical architecture provides a natural, theoretically sound, and practical new direction for research into the role of modulatory neural systems applied to spiking neural networks.
... Circuits of SNNs have been coupled with a double pheromone stigmergy process in a simulation of foraging ants enhancing the behaviour of the simulated swarm. [11]. ...
... Circuits of SNNs have been coupled with a double pheromone stigmergy process in a simulation of foraging ants enhancing the behaviour of the simulated swarm. [11]. ...
Conference Paper
Full-text available
This study explores the design and control of the behaviour of agents and robots using simple circuits of spiking neurons and Spike Timing Dependent Plasticity (STDP) as a mechanism of associative and unsupervised learning. Based on a " reward and punishment " classical conditioning, it is demonstrated that these robots learnt to identify and avoid obstacles as well as to identify and look for rewarding stimuli. Using the simulation and programming environment NetLogo, a software engine for the Integrate and Fire model was developed, which allowed us to monitor in discrete time steps the dynamics of each single neuron, synapse and spike in the proposed neural networks. These spiking neural networks (SNN) served as simple brains for the experimental robots. The Lego Mindstorms robot kit was used for the embodiment of the simulated agents. In this paper the topological building blocks are presented as well as the neural parameters required to reproduce the experiments. This paper summarizes the resulting behaviour as well as the observed dynamics of the neural circuits. The Internet-link to the NetLogo code is included in the annex.
Article
Full-text available
The past decade has seen the rapid development of the online newsroom. News published online are the main outlet of news surpassing traditional printed newspapers. This poses challenges to the production and to the consumption of those news. With those many sources of information available it is important to find ways to cluster and organise the documents if one wants to understand this new system. A novel bio inspired approach to the problem of traversing the news is presented. It finds Hamiltonian cycles over documents published by the newspaper The Guardian. A Second Order Swarm Intelligence algorithm based on Ant Colony Optimisation was developed that uses a negative pheromone to mark unrewarding paths with a "no-entry" signal. This approach follows recent findings of negative pheromone usage in real ants.
Conference Paper
Full-text available
Insects show a rich repertoire of goal-directed and adaptive behaviors that are still beyond the capabilities of today's artificial systems. Fast progress in our comprehension of the underlying neural computations make the insect a favorable model system for neurally inspired computing paradigms in autonomous robots. Here, we present a robotic platform designed for implementing and testing spiking neural network control architectures. We demonstrate a neuromorphic realtime approach to sensory processing, reward-based associative plasticity and behavioral control. This is inspired by the biological mechanisms underlying rapid associative learning and the formation of distributed memories in the insect.
Conference Paper
Full-text available
An artificial Ant Colony System (ACS) algorithm to solve general-purpose combinatorial Optimization Problems (COP) that extends previous AC models [21] by the inclusion of a negative pheromone, is here described. Several Travelling Salesman Problem (TSP) were used as benchmark. We show that by using two different sets of pheromones, a second-order co-evolved compromise between positive and negative feedbacks achieves better results than single positive feedback systems. The algorithm was tested against known NP-complete combinatorial Optimization Problems, running on symmetrical TSP's. We show that the new algorithm compares favourably against these benchmarks, accordingly to recent biological findings by Robinson [26,27], and Gruter [28] where "No entry" signals and negative feedback allows a colony to quickly reallocate the majority of its foragers to superior food patches. This is the first time an extended ACS algorithm is implemented with these successful characteristics.
Article
Autonomous navigation plays an important role in mobile robots. Artificial neural networks (ANNs) have been successfully used in nonlinear systems whose models are difficult to build. However, the third generation neural networks – Spiking neural networks (SNNs) – contain features that are more attractive than those of traditional neural networks (NNs). Because SNNs convey both temporal and spatial information, they are more suitable for mobile robots׳ controller design. In this paper, a modular navigation controller based on promising spiking neural networks for mobile robots is presented. The proposed behavior-based target-approaching navigation controller, in which the reactive architecture is used, is composed of three sub-controllers: the obstacle-avoidance SNN controller, the wall-following SNN controller and the goal-approaching controller. The proposed modular navigation controller does not require accurate mathematical models of the environment, and is suitable to unknown and unstructured environments. Simulation results show that the proposed transition conditions for sub-controllers are feasible. The navigation controller can control the mobile robot to reach a target successfully while avoiding obstacles and following the wall to get rid of the deadlock caused by local minimum.
Article
This work investigates adaptive behaviours for an intelligent robotic agent when subjected to temporal stimuli consisting of associations of contextual cues and simple reflexes. This is made possible thanks to a novel learning rule based on spike-timing-dependent plasticity and embedded in an artificial spiking neural network serving as a brain-like controller. The subsequent bio-inspired cognitive system carries out different classical conditioning tasks in a controlled virtual 3D-world while the timing and frequency of unconditioned and conditioned parameters are varied. The results of this simulated robotic environment are analysed at different stages from stimuli capture to neural spike generation and show extended behavioural capabilities by the robot in the temporal domain.
Article
This article presents an overview of recent work on ant algorithms, that is, algorithms for discrete optimization that took inspiration from the observation of ant colonies' foraging behavior, and introduces the ant colony optimization (ACO) metaheuristic. In the first part of the article the basic biological findings on real ants are reviewed and their artificial counterparts as well as the AGO metaheuristic are defined. In the second part of the article a number of applications of ACO algorithms to combinatorial optimization and routing in communications networks are described. We conclude with a discussion of related work and of some of the most important aspects of the ACO metaheuristic.
Article
Article
A paperback edition of the translation by Anrep, first published in 1927 by the Oxford University Press, containing a series of 23 lectures on the research of Pavlov's laboratory in the 1st quarter of the 20th century. From Psyc Abstracts 36:05:5CG30P. (PsycINFO Database Record (c) 2012 APA, all rights reserved)