Conference PaperPDF Available

Polyethism in a colony of artificial ants



We explore self-organizing strategies for role assignment in a foraging task carried out by a colony of artificial agents. Our strategies are inspired by various mechanisms of division of labor (polyethism) observed in eusocial insects like ants, termites, or bees. Specifically we instantiate models of caste polyethism and age or temporal polyethism to evaluated the benefits to foraging in a dynamic environment. Our experiment is directly related to the exploration/exploitation trade of in machine learning.
Polyethism in a colony of artificial ants
Chris Marriott and Carlos Gershenson
IIMAS, Universidad Nacional Aut´
onoma de M´
exico, M´
exico City, M´
We explore self-organizing strategies for role assignment in a
foraging task carried out by a colony of artificial agents. Our
strategies are inspired by various mechanisms of division of
labor (polyethism) observed in eusocial insects like ants, ter-
mites, or bees. Specifically we instantiate models of caste
polyethism and age or temporal polyethism to evaluated the
benefits to foraging in a dynamic environment. Our experi-
ment is directly related to the exploration/exploitation trade
of in machine learning.
The self-organizing strategies of eusocial insects are now
well known and well studied in biology (Beckers et al.
(1989); Traniello (1989); Robinson (1992); Th´
erauluz et al.
(1998); Th´
eraulaz and Bonabeau (1999); Gautrais et al.
(2002); Garnier et al. (2007)) and applications to compu-
tation are abundant (Bonabeau et al. (1999); Panait and
Luke (2004b,a); Schmickl and Crailsheim (2008); Gershen-
son (2010); Ducatelle et al. (2010)). One of the more re-
markable behaviors observed is the ability of rather sim-
ple, unintelligent agents (individual insects) to coordinate
their behavior to establish a rather fluid and adaptive be-
havior on the colony level. The phenomenon of stigmergy
(communication via the environment) has now been mod-
eled and applied in artificial simulations to achieve similar
results among rather simple artificial agents (Th´
eraulaz and
Bonabeau (1999); Bonabeau et al. (1999); Panait and Luke
(2004b,a); Schmickl and Crailsheim (2008)) cooperating in
multi-agent systems.
However, many of these applications focus on homoge-
neous colonies, where each agent has the same behavioral
capabilities. Nonetheless, observations of insects show that
in many colonies the individuals are not always homoge-
neous. Colonies consist of heterogeneous agents, whether
these agents display morphological differences (i.e. distinct
castes) or merely behavioral differences. The effects of this
stratification of agents in a colony is referred to as division
of labor (DOL) or by the term polyethism (Robinson (1992);
Traniello and Rosengaus (1997); Th´
erauluz et al. (1998);
Gautrais et al. (2002); Gordon (2003)). As artificial multi-
agents systems grow larger and involve agents with different
roles the problem of assigning roles to agents becomes in-
creasingly important (Campbell and Wu (2010); dos Santos
and Bazzan (2009)).
Biologists differentiate between at least two observable
means of dividing roles amongst individual workers in natu-
ral insect colonies. The means we select for study are called
caste polyethism and age polyethism. Other types of poly-
theism are also observed (e.g. elitism) and the two above
types have many possible underlying mechanisms though
these additional types and subtypes will not be explored
in detail in this article. Simulations have just begun ex-
ploring task assignment and heterogeneous agent popula-
tions (e.g. Schmickl and Crailsheim (2008); Ducatelle et al.
(2010)). Our experiment differs from these in that our agents
are assigned the same task (foraging), but must decide which
strategy to adopt to solve the task(between an individual ex-
ploratory strategy and a cooperative exploitative strategy).
In this sense, our experiment parallels attempts to solve the
well known exploration/exploitation trade off in machine
learning. Further, other experiments focus on simulations
of actual natural colony behavior in an attempt to assess
models of those behaviors, whereas while we are inspired by
these models our focus is on the self-organizing and adaptive
problem solving that these models make possible.
Caste polyethism occurs when distinct types of individ-
uals are bred by the colony. An individual is effectively
born into its role, often times displaying morphological dif-
ferences from individuals from other castes. The clearest
example of castes is the division between the reproductive
caste and the worker caste in eusocial insects. A single
or small group of reproductive females (called queens) are
responsible for all reproductive tasks in the colony while
non-reproductive workers carry out all other tasks required
by the colony (brood care, nest constructions and main-
tenance, waste removal, foraging, and defense). In some
species workers are further divided into sub-castes. Differ-
ences among workers from different castes are particular to
the worker’s role. For instance in some species of ants the
arXiv:1104.3152v1 [cs.AI] 15 Apr 2011
workers can be divided into majors and minors (occasion-
ally with an intermediate caste as well) where the majors are
larger than the minors, this size being helpful in the task they
carry out (primarily colony defense). Minors are smaller,
making them more energy efficient, and they are relegated
to less dangerous tasks like foraging and nest maintenance.
Only in rare occasions will a worker do a task that is typi-
cally assigned to a different caste.
Age or temporal polyethism is a type of division of la-
bor where the worker’s role is correlated with its age or
changes over time. Age polyethism is more common than
caste polyethism in natural insect colonies. In colonies dis-
playing age polyethism younger workers are commonly as-
signed less risky tasks (nursing or nest maintenance allowing
them to stay in the nest) whereas older workers are assigned
more dangerous tasks (foraging, defense, or raiding where
the agent must leave the nest). It is hypothesized that this
division of labor allows the colony to maximize the work
carried out by each individual worker (i.e. young workers
will be less likely to die and thus can live longer to carry out
more work). This will be beneficial to the colony since it will
have to breed fewer workers if each worker’s longevity (and
thus productivity) is maximized (Tofilski (2002, 2009)). In
certain cases this progressive role assignment may also al-
low younger and less experienced workers to gain the ex-
perience necessary to carry out more difficult tasks (say at
the very least allowing them to become familiar with the
layout of the nest and surrounding environment before hav-
ing to venture far from the nest) (Tofts and Franks (1992);
Tofts (1993); Franks and Tofts (1994)). Many mechanisms
have been suggested as the underlying reason for observed
age polyethism. The mechanism we employ is similar to
the response threshold model commonly studied (see e.g.
erauluz et al. (1998); Garnier et al. (2007)).
As systems of artificial agents grow larger (in population)
and become more heterogeneous the task of assigning roles
to agents becomes more critical. This article aims to ex-
plore models that might achieve the types of division of la-
bor observed in eusocial insects so that these models may
be exploited in engineering of multi-robot and multi-agent
Artificial Ants
The experiment detailed below involves a colony of artificial
ants engaged in a foraging task. The colony level task is
to maximize the food intake of the colony (allowing colony
sustenance and growth). On the individual worker level the
task is to explore the environment, find a food object, and
return to the nest with the object.
We consider two different strategies for individual work-
ers inspired by natural ant populations. The first, and sim-
pler, strategy is for workers to forage for the most part in-
dividually. We say “for the most part” here since individual
foragers cooperate at least insofar as they attempt to divide
the environment to be explored equally among them (see
Figure 1). We implement this strategy by having ants leave
a “seeker” trail as they leave the nest. While “seeking” the
ants will avoid other seeker trails, meaning they will travel
mostly straight away from the nest while avoiding the trail
they leave behind them, but they will also avoid trails left
by other ants, helping to divide the area somewhat evenly.
Other than this simple cooperation, workers leave the nest
and randomly explore until they find a food object (or reach
the range of their exploration) and return to the nest. We
will call this strategy the “individual” or “exploratory” strat-
egy, and ants following this strategy “explorers”. The seeker
path left by these ants also serves as the ants sole means of
returning to the den (i.e. they follow seeker paths back).
Ants that find a food source of sufficient size (i.e. they find
at least one food morsel to carry back to the nest and at least
one more food morsel they will recruit others to seek out) the
ant will leave a second type of trail we call the “carrier” trail.
The second strategy, which we call the “cooperative” strat-
egy or “exploitative” strategy, involves foragers that will fol-
low “carrier” trails to exploit food sources that were already
discovered by other ants. Both explorers and exploiters will
leave “carrier” trails under the conditions listed above, but
only exploiters will follow them to food sources (explorers
ignore them).
These strategies are inspired by those found in natural
populations, with a correlation of colony size to the strat-
egy used (Beckers et al. (1989); Traniello (1989)). In par-
ticular it has been observed that smaller colonies tend to
use the individual exploratory strategy whereas as the larger
the colony is the more likely the colony uses a cooperative
or exploitative strategy (and the more cooperative the strat-
egy used). Despite this correlation, upon closer examination
larger colonies have foragers carrying out both strategies,
that is, they engage in DOL or polyethism.
In preliminary experimentation it was found that these
strategies fare differently depending on the environment the
colony is situated in. If food objects are uniformly dis-
tributed around the nest then the individual strategy reaches
near optimal foraging. Over time the workers will clear a
disc shaped area of food around the nest, the radius of the
disc being determined by the frequency of food objects and
by the size of the population. This situation is presented in
Figure 1.
Interestingly, in larger colony sizes the cooperative strat-
egy also fares quite well in environments with uniform dis-
tribution of food, though the foragers carry out a more com-
plex foraging strategy. Cooperative foragers form an “arm”
leading from the nest into the environment and this arm has
been observed to swing in a circle around the nest, clearing
food objects as it goes, or spontaneously dissolving and re-
forming in a more lucrative direction. These strategies have
also been observed in natural ant colonies. While the coop-
erative strategy seems to approach the performance of the in-
Figure 1: Explorers in a uniform environment. The den is in
the center of the torus. Green squares are food. Red paths
are seeker paths. Blue paths are carrier paths. Recall that
explorers ignore the carrier paths.
dividual strategy in experimentation, the individual foragers
have an advantage in an environment with uniformly dis-
tributed food.
A second environment type we have investigated con-
tains food isolated in “patches”. For the sake of comparison
among simulation runs our food patches are always placed
equidistant from the nest, though in a random direction. In
this environment the cooperative foragers have a clear ad-
vantage. Once a forager finds a patch of food it recruits other
foragers to help it clear the patch and the colony quickly op-
timizes the path to the food patch. Figure 2 shows a typical
patch environment (with 2 patches) and a colony of exploita-
tive ants foraging from the patches.
Individual foragers are at a significant disadvantage when
faced with an environment with a single patch. Many in-
dividual foragers leave the nest in the wrong direction (re-
member they attempt to divide the environment equally) and
so return empty handed. Only a fraction of individual for-
agers leave the nest in the right direction and return with
Given the differential success of these strategies in these
environments it is our hypothesis that polyethism in a colony
will be beneficial if the colony is faced with either an un-
known environment (of one of these two types) or with a
dynamic environment consisting of either a combination of
these types or shifting between these types.
Figure 2: Exploiters in an environment with two patches.
The den is in the center of the torus. Green squares are food.
Red paths are seeker paths. Blue paths are carrier paths.
Exploiters use the carrier paths to cooperatively forage.
Experimental Setup
In our experiment we consider four different types of
colonies that we will expose to five different types of en-
vironment. We will consider how each colony fares in each
environment, as well as how the colony fares across all en-
A colony will consist of a queen (responsible for creating
new workers), a population of workers, a population of lar-
vae, and a store of food. Workers consume food at a constant
rate (about 1 food every 450 simulation rounds) and larvae
consume food at a constant rate (1 food for the 100 round
gestation period) until they are born as a new worker.
The queen lives for the duration of the experiment (or un-
til the colony dies of starvation), though workers and lar-
vae may die. Workers die under two conditions. If they
reach their maximum age (selected uniformly from the range
2750-3250 rounds), or if they run out of food energy. When
a worker consumes a piece of food it gains energy that will
sustain it for 450 simulation rounds. If while foraging the
worker’s food energy reaches 0 (i.e. after 450 rounds) then
the worker attempts to return to the nest (possibly without
food). Upon returning the worker will attempt to consume a
unit of food from the store. If there is no food in the store
the worker dies.
A larvae also consumes food, once upon creation by the
queen and again upon changing into a worker. The food
consumed when the larvae matures forms the initial energy
store of the worker. A queen will never create a larvae in an
instance where the food stores are empty, however, a larvae
may mature and find the store empty. In this case the new
worker dies.
Queens from different colonies have different profiles,
however, they all follow the same rule when deciding to re-
produce. A queen will only create a new larvae if the food
store exceeds the current population of workers plus the cur-
rent population of larvae.
The first two types of colony will form a control group
for comparison. These two types will not use polyethism
and queens in these colonies will create only explorers or
only exploiters respectively. From the earlier discussion we
know that these colonies will fare well in some environments
but not in others and will not be adaptive to the environment.
The third colony will engage in an adaptive caste
polyethism. Queens in this type of colony produce lar-
vae that can mature into either an individual or cooperative
worker. The queen chooses the type of worker to create in
proportion to the success rate of workers of that type. (The
queen keeps track of food returned by each type of forager
over the last 500 rounds, and of the number of each type of
forager. From this she estimates the efficiency of the aver-
age ant of each type and randomly selects to create a new
ant in proportion to the ratio of success rate.) Thus if ex-
plorers are more successful at foraging than exploiters then
a queen will make an explorer with higher probability (and
vice versa). Queens in this type of colony will ensure there
is always at least one worker of each type so success rates
can be properly estimated.
The fourth colony will engage in one type of age
polyethism. Workers in these colonies are homogeneous in
their behavioral repertoire, in that they can act as either ex-
plorers or exploiters. Which role a worker adopts depends
first on their age (for younger workers) and then on the de-
mands of the colony (for older workers). In this colony
new workers adopt an individual foraging strategy, and may
switch to a cooperative strategy (or back again) after reach-
ing a particular age (usually consisting of 1 or 2 full foraging
trips). Workers of this type choose to change roles based on
collective experience, that is, in proportion to the success
rate of workers in the colony similar to the mechanism used
in the third colony. While we do not use pheromones in our
model of this behavior we believe this mechanism is closely
related to response threshold models of behavior selection.
We expose these 4 colony types to 5 distinct environ-
ments: uniform, patch, roaming patch, seasonal, and mixed.
The rate at which food drops in each environment is the same
(1 food every 5 rounds) and each food will stay in the envi-
ronment for exactly 1000 rounds or until picked up by a for-
ager. The uniform and patch environments were described
above consisting of uniformly distributed food or an isolated
patch of food respectively.
The roaming patch environment as has a single patch but
this patch will change location every 1000 rounds (the new
location will be the same distance from the nest as the old
location). This means that after the patch has moved new
food will drop in the new patch location, though old food
is not removed unless foraged or it reaches its 1000 round
limit. As a result there will usually be two patches in the en-
vironment, one containing old food that is decaying and one
containing new food. Figure 2 displays a typical scenario
for this type of environment.
The seasonal environment is intended to simulate an en-
vironment that changes from a uniform distribution to an
isolated patch with regularity possibly corresponding to the
seasons. We simulate this idea by alternating between the
two distributions every 1000 rounds. Again there will be
a temporal overlap between these two environments mean-
ing that the environment will typically contain food dropped
uniformly and in a patch. Every time the season changes to
the patch distribution a new location for the patch is selected
so in this sense we see the patch as roaming as in the last
The mixed environment includes both uniform food drops
and an isolated patch at the same time, and the environment
is static (in that the patch does not move). In this environ-
ment the drop rate is the same as previous environments de-
spite there being two active food drop mechanisms operating
Observations and Data
We choose to analyze the worker population data of our
colonies. This data reflects the colonies’ ability to forage
for food efficiently. Each colony begins with an initial food
store of 32 food and zero ants. The queen will use this ini-
tial food to create 16 initial workers which mature on rounds
101-116 of the simulation. At this point the food stores will
be exhausted (each larvae will use one food when created
and another when maturing into a worker) and so the colony
must forage for food to sustain itself or grow in size. Also
by round 116, 23 food objects will exist in the environment,
with their location depending on the type of environment.
Parameters of the simulation determine a maximum colony
size, namely the food drop rate and the energy consump-
tion rate of the workers (as well to a lesser extent the size of
the environment). This maximum is just above 80 workers,
though due to the non-linear dynamics of the simulation this
maximum can be exceeded for short periods.
The initial stages of the simulation are occupied by rapid
growth of population as the foragers are able to bring in
more food than the colony needs so new workers are created
(exceptions to this are noted below). This rapid growth com-
monly results in too many workers and so is often followed
by a large dip in population and an oscillation is observed
until an equilibrium can be found. This equilibrium depends
on the type of colony and environment.
Figure 3 (left column) displays the worker population data
gathered from all experimental runs. The data presented in
the figure is the average worker population over time (N=
0 10000 20000 30000
0 10000 20000 30000
Uniform: Division of Labor
0 10000 20000 30000
0 10000 20000 30000
Patch: Division of Labor
0 10000 20000 30000
Roaming Patch
0 10000 20000 30000
Roaming Patch: Division of Labor
0 10000 20000 30000
0 10000 20000 30000
Season: Division of Labor
0 10000 20000 30000
0 10000 20000 30000
Mixed: Division of Labor
Figure 3: Worker Population Data. From the top row the data is presented for each environment: uniform, patch, roaming patch,
seasonal, and mixed. The left column displays worker population over time for the four colony types. The right column displays
the division of labor in the Caste and Age colonies. The worker population of these colonies is contrasted to the number of
workers in the colony assigned to the exploration task. Dotted lines in the seasonal environment indicate the changing seasons.
Please note we use ”Solo” to indicate explorers and ”Coop” to indicate exploiters in the charts.
Beginning with our control environments, we note that the
colonies perform as expected. In the uniform environment
the best performance is achieved by the explorers, and is
closely matched by the caste and age polyethistic colonies.
All three colonies settle around a population of 80 work-
ers after initial instability. While the exploitative colony
has no trouble surviving in this environment its sub-optimal
foraging strategy allows it to maintain only a population of
between 40-60 workers. It’s population is also subject to
greater instability as the foraging arm grows and shrinks in
size and changes location.
In the second control environment with a single station-
ary patch again we see expected results. The explorers are
unable to maintain even the low initial colony size and the
colony starves quickly. The cooperative foragers are the
quickest to exploit the isolated patch, whereas the polyethis-
tic colonies are able to quickly adapt to the environment by
producing exploiters instead of explorers. Both polyethistic
colonies still maintain a small population of explorers. The
dip in cooperative population observed near the end of the
simulation is caused by two anomalous colonies from the
simulation runs that starved to death. No such starvations
were observed among the polyethistic colonies. We observe
some population instability in this environment.
In the roaming patch environment we see that the
polyethistic colonies are able to maintain a higher pop-
ulation than the purely exploitative colony (the explorers
quickly starve in this environment as well). This implies
a better ability to adapt to the moving patch. The exploita-
tive colony also displays a greater instability in population
though all three successful colonies have greater instability
(than in the stationary patch environment). Also noteworthy
is that all colonies have trouble maintaining an optimal pop-
ulation (even though the polyethistic colonies occasionally
reach 80 workers).
In the seasonal environment we again observe better per-
formance from the polyethistic colonies than the purely ex-
plorer and purely exploiter colonies. Further there is greater
stability of population in the polyethistic colonies, where the
pure explorer and pure exploiter colonies suffer population
oscillations corresponding roughly to the changing seasons.
Note in the figures the dotted lines display the changing sea-
sons. The polyethistic colonies manage to maintain roughly
optimal populations in this environment while the explorer
colony suffers the most in the seasons when food becomes
isolated in a patch.
Finally, in the mixed environment, we again see a popu-
lation advantage to polyethism. While both the purely ex-
plorer and purely exploiter colonies survive in the mixed en-
vironment they are unable to reach the optimal populations
and display a slightly greater instability. The purely explorer
population also maintains a slight population advantage over
the purely exploiter population.
A secondary focus of our simulations was on the division
of labor in the polyethistic colonies. We gathered data on
how many workers of each type were deployed at a given
time by the polyethistic colonies. This data is presented in
Figure 3 (right column) for each environment. We display
only the number of explorer workers in the chart in contrast
to the total worker population, with the number of exploita-
tive workers being the difference. In the caste polyethism
colonies this corresponded to how many workers of each
caste were available. In the age polyethism colonies this
corresponded to how many workers were currently assigned
to each task, exploring or exploiting.
In the control environments the polyethistic colonies sta-
bilized around a constant number of explorers. For the uni-
form environment both colonies settled at just over half of
the workers (about 50 out of 80 workers) dedicated to ex-
ploring. It is worth noting that the colonies did not try to
maximize the number of explorers in this environment. In
the patch environment the caste colony settled at around 5
workers dedicated to exploring while the age colony main-
tained a slightly higher number of explorer, typically oscil-
lating between 5 and 15 workers. We note that in these envi-
ronments the age polyethistic colony displayed greater oscil-
lations of worker assignments whereas the caste polyethis-
tic colony tended to stabilize around a particular division of
workers assigned to each task.
In the roaming patch environment more explorers were
maintained than in the stationary patch environment. In the
caste colony just over 10 of the workers were assigned the
exploring role. The age colony still assigned more work-
ers to exploring on average than the caste colony, typically
above 15, but as high as 25. Again the age colony had greater
variation in its division of labor.
The seasonal environment sees distinct performance dif-
ferences among the two colonies displaying polyethism.
The caste colony settles on 30 to 35 workers dedicated to
exploring. This number is stable when compared to the age
colonies that attempted to adjust the worker base to the cur-
rent season. Thus we see the number of explorers oscillating
between about 25 workers to as high as 43 workers (except-
ing the early spike).
In the mixed environment both polyethistic colonies stabi-
lize their worker base by assigning roughly half the workers
to each task. The age colony again assigns slightly more
workers to exploration than the caste colony and displays
slight oscillations.
The data presented suggests that polyethism, regardless of
kind, offers benefits to the foraging task. While both of
the foraging methods studied in this experiment (exploring
and exploiting) can be seen as self-organizing methods, the
colonies benefit if the “higher-level” self-organizing method
of polyethism is applied to select which of the methods to
engage in (Gershenson (2010)).
In the control environments where the environment is
specifically created to favor one of the two basic strate-
gies, exploring or exploiting, we see that polyethism al-
lows the colony to adjust the worker base to the environ-
ment. Whereas the non-polyethistic colonies perform sub-
optimally when matched with an environment they are not
specialized for, the polyethistic colonies can modify their
behavior to either of the environments and perform near op-
timally. The only drawback in these environments to the
polyethistic colonies is that they require some time to adjust
to the environment.
In the more dynamic environments we see that polyethism
is necessary to get optimal or near-optimal performance. For
instance we see that in the roaming patch environment, while
exploiters are designed for this environment, maintaining a
small population of explorers allows the new patch location
to be found quicker, and more quickly exploited. The insta-
bility seen in these environments is likely due to the shifting
location of the food patch, and since the new patches are
placed randomly (independent from the old patch location)
each switch imposes a different level of difficulty upon the
colony. If the new patch is located close to the old one it
might be more easily found and exploited than one that is
far away. However, closer patches might be exploited by
extending the old path system to the new patch instead of
forming a new, shorter path. These dynamics affect forag-
ing time in the short and long term and thus we see greater
instability in population.
It is clear in the seasonal and mixed environment that
polyethism is necessary to have optimal foraging. In the
seasonal environment the non-polyethistic colonies suffer in
seasons where they are not favored. In the mixed environ-
ment the non-polyethistic colonies are unable to exploit all
the food drops and thus cannot maintain as high a popu-
lation. In the mixed environment the polyethistic colonies
settle on a division of workers among the two strategies that
allows for exploiting both food sources. It is interesting to
note that the polyethistic colonies still managed to reach op-
timal population levels in the mixed environment, implying
that the two strategies did not experience negative interfer-
We also note that the polyethistic colonies tackle the sea-
sonal environment in slightly different ways. In the seasonal
environment the caste colony maintains a constant number
of each worker. This can be seen as the colony being pre-
pared for either season, but not necessarily specializing for
the current season. This approach may be favored by the
caste colony because the season length (1000) is short com-
pared to the lifespan of a worker (selected uniformly from
the range 2750-3250 rounds). Thus the caste colony will
not have the opportunity to adjust the balance of workers
each season since workers from the previous season will still
be present in the work force. The age colony does adjust
its work force to the new season, albeit only slightly, since
the workers in this colony can switch tasks every round trip
which is about 300-400 rounds long, shorter than the season
length. Both strategies allow the colony to maintain fairly
stable and nearly optimal populations.
To test this analysis we conducted a follow up experiment
where the season size was extended to 3000 rounds (see Fig-
ure 4). In this run we saw that the caste colony adopted
the adjustment strategy as well, attempting to match work-
ers to the season instead of opting for an equal distribution.
We observed in this case that the age colony was able to
adapt its workers more rapidly than the caste colony, and
thus had a slightly more stable population. The stability of
both colonies’ populations suffered with the longer seasons
due to more polarization of the workforce and the lag be-
tween the season change and the ability of the colony to ad-
just its workforce.
The results of our experiment also have applica-
tions in machine learning in dealing with the explo-
ration/exploitation trade-off. The colonies engaged in
polyethism are able to organize their foraging strategy
around exploration or exploitation based on simple to form
estimates of the utility of these methods. We believe that the
methods displayed by these colonies can be easily adapted
to machine learning applications and have similarities to
some machine learning strategies for tackling the explo-
ration/exploitation trade-off.
We conclude that division of labor is beneficial to ant
colonies in that it adds a layer of dynamism to their problem
solving as well as makes the colony more robust. We suggest
that the simple self-organizing methods of assigning work-
ers to tasks can be adopted in artificial systems. These meth-
ods are simple to implement and require a minimal amount
of central planning or control. The methods are reactive and
dynamic and can likely be applied in a variety of situations,
this being the topic of future work.
While we found little evidence favoring one of age or
caste polyethism as a method of assigning workers to tasks
we did find that the caste polyethism appeared to be more
rigid in that it took longer for the workforce to adjust to
new conditions. However, the trade off is that in the age
polyethistic colonies there was a tendency to over adjust to
new conditions, which may not be favorable in all situations.
We believe that more work is required to determine the ben-
efits of each of these methods, given that the distribution of
these methods among natural colonies is not balanced (re-
call age polyethism is more common than caste polyethism).
One aspect that was not considered in this experiment, and
probably plays an important role in natural colonies, is the
variable costs to a colony or species (genetically and in terms
of energy expenditure) in producing workers that either are
specialized for their task (caste polyethism) or are general-
0 10000 20000 30000
Long Season
0 10000 20000 30000
Long Season Division of Labor
Figure 4: Long (3000 round) Season. The left chart displays population over time for the four colony types. The right chart
displays the division of labor in the Caste and Age colonies by contrasting the total population to the number of workers in the
exploration task (the legend has been removed for clarity though we follow the same format as Figure 1). Dotted lines indicate
the changing seasons.
ists able to take on any available task (age polyethism).
Beckers, R., Goss, S., Deneubourg, J., and Pasteels, J. (1989).
Colony size, communication, and ant foraging strategy. Psy-
che, 96:239–56.
Bonabeau, E., Dorigo, M., and Th´
eraulaz, G. (1999). Swarm Intel-
ligence: From natural to artificial systems. Oxford University
Campbell, A. and Wu, A. (2010). Multi-agent role allocation:
Issues, approaches, and multiple perspectives. Autonomous
Agents and Multi-Agent Systems, 22(2):317–355.
dos Santos, F. and Bazzan, A. (2009). An ant based algorithm for
task allocation in large-scale and dynamic multiagent scenar-
ios. In Proceedings of the 11th annual conference on Genetic
and evolutionary computation (GECCO’09).
Ducatelle, F., Di Caro, G., and Gambardella, L. (2010). Coopera-
tive self-organization in a heterogeneous swarm robotic sys-
tem. In Proceedings of the 12th annual conference on Genetic
and evolutionary computation (GECCO’10).
Franks, N. and Tofts, C. (1994). Foraging for work: how tasks
allocate workers. Animal Behavior, 48:470–472.
Garnier, S., Gautrais, J., and Th´
eraulaz, G. (2007). The biological
principles of swarm intelligence. Swarm Intelligence, 1:3–31.
Gautrais, J., Th´
eraulaz, G., Deneubourg, J., and Anderson, C.
(2002). Emergent polyethism as a consequence of increased
colony size in insect societies. Journal of Theoretical Biol-
ogy, 215:363–373.
Gershenson, C. (2010). Computing networks: A general frame-
work to contrast neural and swarm cognitions. Paladyn, Jour-
nal of Behavioral Robotics, 1(2):147–153.
Gordon, D. (2003). The organization of work in social insect
colonies. Complexity, 8(1):43–46.
Panait, L. and Luke, S. (2004a). Ant foraging revisited. In Proceed-
ings of the ninth international conference on the simulation
and synthesis of living systems (ALIFE 9).
Panait, L. and Luke, S. (2004b). Learning ant foraging behaviors.
In Proceedings of the ninth international conference on the
simulation and synthesis of living systems (ALIFE 9), pages
Robinson, G. (1992). Regulation of division of labor in insect
species. Annual Review of Entomology, 37:637–65.
Schmickl, T. and Crailsheim, K. (2008). Analysing honeybees’ di-
vision of labour in broodcare by a multi-agent model. In Pro-
ceedings of the 11th conference on the simulation and synthe-
sis of living systems (ALIFE11).
eraulaz, G. and Bonabeau, E. (1999). A brief history of stig-
mergy. Artificial Life, 5:97–116.
erauluz, G., Bonabeau, E., and Deneubourg, J.-L. (1998). Re-
sponse threshold reinforcements and division of labour in in-
sect societies. In Proceedings of the Royal Society, pages
Tofilski, A. (2002). Influence of age polyethism on longevity of
workers in social insects. Behavior Ecology and Sociobiol-
ogy, 51:234–237.
Tofilski, A. (2009). Shorter-lived workers start foraging earlier.
Insectes Sociau, 56(4):359–366.
Tofts, C. (1993). Algorithms for task allocation in ants (a study of
temporal polyethism theory). Bulletin of Mathematical Biol-
ogy, 55:891–918.
Tofts, C. and Franks, N. (1992). Doing the right thing: ants, honey-
bees and naked mole rats. Trends in Ecology and Evolution,
Traniello, J. (1989). Foraging strategies of ants. Annual Review of
Entomology, 34:191–210.
Traniello, J. and Rosengaus, R. (1997). Ecology, evolution and
division of labour in social insects. Animal Behavior, 53:209–
... Our experiment is embedded in a research program spanning a few decades that studies ant division of labor through simulation (Bae & Marriott, 2019;Campos et al., 2000;Duarte et al., 2012;Franks & Tofts, 1994;Gautrais et al., 2002;Gove et al., 2009;Jeanson et al., 2007;Marriott & Gershenson, 2011;Merkle & Middendorf, 2004;de Oliveira & Campos, 2019;Prabhakar et al., 2012;Theraulaz et al., 1998;Waibel et al., 2006). A common cognitive theory of division of labor is the response threshold theory (Bonabeau et al., 1996Duarte et al., 2011). ...
... The stark simplicity of these models make them easier to study analytically and computationally, though it also makes them susceptible to criticism that too much has been abstracted away to capture real world phenomena accurately. In contrast, our agent-based model (Bae & Marriott, 2019;Marriott & Gershenson, 2011) can test the same theory in a variety of structured environments in order to incorporate critical features of situated and embodied cognition (Clark, 1997;Almeida e Costa & Rocha, 2005;Ziemke, 2003). Within the environment we can define explicit tasks, like foraging and reproduction, that require complex patterns of behaviors to complete. ...
... Experience-Based Plasticity A curious observation of ant behavior for response threshold theories of division of labor is what is called age polyethism (Franks & Tofts, 1994;Marriott & Gershenson, 2011;Tripet & Nonacs, 2004). Among many ant colonies it is observed that roles selected correlate with the age of the ant. ...
Full-text available
We implement an agent-based simulation of the response threshold model of reproductive division of labor. Ants in our simulation must perform two tasks in their environment: forage and reproduce. The colony is capable of allocating ant resources to these roles using different division of labor strategies via genetic architectures and plasticity mechanisms. We find that the deterministic allocation strategy of the response threshold model is more robust than the probabilistic allocation strategy. The deterministic allocation strategy is also capable of evolving complex solutions to colony problems like niche construction and recovery from the loss of the breeding caste. In addition, plasticity mechanisms had both positive and negative influence on the emergence of reproductive division of labor. The combination of plasticity mechanisms has an additive and sometimes emergent impact.
... We have chosen to focus our experimentation on the third stage in which RDoL emerges and the fourth stage in which cooperative foraging strategies can evolve. We have previously studied the emergence of eusocial behavior in foraging agents capable of social learning (Marriott and Chebib, 2016) and division of labor in artificial ants in the fourth stage (Marriott and Gershenson, 2011). ...
... As a secondary experiment we have exposed our evolved colonies to a changing environment from one in which solitary exploration is adaptive into one in which cooperative foraging is adaptive. This is similar to the seasonal environments used in (Marriott and Gershenson, 2011). Our ants have a second gene which determines the response threshold for the two foraging strategies. ...
... We run our secondary experiment for 200,000 ticks to allow the colony first to evolve its RDoL strategy and then adapt to the new environment. Our second experiment reproduces some of the experiments in (Marriott and Gershenson, 2011) and tests the conditions under which a colony in stage three can move into stage four adapting from solo foragers to cooperative foragers. ...
... Agent-based simulation has been used to study role reassignment in other studies. In natural systems, like an ant colony, role reassignment is a emergent process (Marriott and Gershenson, 2011). In human organizations role reassignment can be emergent or be based on deliberate policies of individuals or organizations. ...
Full-text available
Examines foraging strategy as a system divisible into elements composed of individual and collective action, although such a distinction is often blurred because of the high degree of social integration of colony foraging activity. Changing nutritional demands associated with the maturation of reproductives may shift a colony's foraging activity among different food types. Communication mediates cooperation among foragers during search and resource retrieval and serves as a control mechanism of colony-level foraging responses. Major sections cover foraging system evolution, with particular reference to caste evolution, polymorphism, competition and foraging ecology; foraging theory and diet selection; search pattern, including the importance of territoriality; social regulation of foraging; and the role of learning. The chief ecological determinants of ant foraging strategy are the distribution of food resources (in size, time, space and quality); competition with sympatric ant species; and predation. -P.J.Jarvis
Full-text available
We describe a multi-agent model of a honeybee colony and show several applications of the model that simulate exper-iments that have been performed with real honeybees. Our special emphasis was on the decentralized, self-organized regulation of brood nursing, which we successfully simu-lated: We found that brood manipulations, food-deprivation experiments and colony-size manipulations can be explained by the mechanisms we implemented into our model described here. Our agents can perform various tasks (foraging, storing, nursing). The model is spatially resolved, and contains a des-ignated broodnest area as well as a designated honey/nectar storage area. All bees (and larvae) consume nectar/honey at a task-specific rate, allowing us to track the flow of nec-tar through the colony. Several kinds of stimuli, which are important for division of labour, were modelled in detail: dances, contact stimuli and chemical signals.
Full-text available
Age polyethism is widespread among social insects and, as a rule, safe tasks are performed by workers earlier in life than are risky ones. Mathematical models were used to compare expected longevity of workers in colonies with and without age polyethism. The results of the models suggest that if aging does not depend on activity then age polyethism is profitable when safer tasks are performed earlier in life. If, however, aging depends on activity, age polyethism is profitable when safer tasks are performed earlier in life and if they are associated with higher aging-related mortality. On the other hand, age polyethism is not profitable if safer tasks are performed later in life, and if they are associated with lower aging-related mortality. Furthermore, if there is no aging, then age polyethism does not bring any benefits. Electronic supplementary material to this paper can be obtained by using the Springer Link server located at
Full-text available
In social insects it is often observed that young workers perform tasks inside the nest and later switch to tasks outside the nest. By doing this the workers maximize their expected longevity, because tasks inside the nest are safe and tasks outside the nest are risky. The optimal strategy of workers should be expected to depend not only on their age but also on their health status if it is associated with reduction of longevity. Here a mathematical model is used to calculate the optimal time of switching between safe and risky tasks in a colony consisting of both healthy and unhealthy workers. The model predicts that unhealthy workers, with shorter longevity, should perform more risky tasks at an earlier age than their healthy nest mates should. The optimal time to switch between safe and risky tasks depends on the proportion of healthy and unhealthy workers in the colony, but the workers need not perceive the health status of their nest mates in order to adopt the optimal strategy. The workers need only perceive their own life expectancy, because the life expectancy of healthy and unhealthy workers should be the same at the time of switching from safe to risky tasks. The model predictions agree with a wide range of empirical data presented in this paper. Workers that are infected, poisoned, injured or affected by other harmful factors start to forage and perform other risky tasks at an earlier age than their healthy nest mates.
In social insect colonies, workers perform a variety of tasks, such as foraging, brood care and nest construction. As the needs of the colony change, and as resources become available, colonies adjust the numbers of workers engaged in each task. Task allocation is the process that results in specific workers being engaged in specific tasks, in numbers appropriate to the current situation.
Insects are good at cooperatively solving many complex tasks. For example, foraging for food far away from a nest can be solved through relatively simple behaviors in combi-nation with pheromones. As task complexity increases, how-ever, it may become difficult to find individual agent rules which yield a desired emergent cooperative behavior, or to know if any such rules exist at all. For such tasks, machine learning techniques like evolutionary computation (EC) may prove a valuable approach to searching the space of possi-ble rule combinations. This paper presents an application of genetic programming to search for foraging behaviors. The learned foraging behaviors use only pheromone information to find the path to the nest and to the food source.
Most previous artificial ant foraging algorithms have to date relied to some degree on a priori knowledge of the environ-ment, in the form of explicit gradients generated by the nest, by hard-coding the nest location in an easily-discoverable place, or by imbuing the artificial ants with the knowledge of the nest direction. In contrast, the work presented solves ant foraging problems using two pheromones, one applied when searching for food and the other when returning food items to the nest. This replaces the need to use complicated nest-discovery devices with simpler mechanisms based on pheromone information, which in turn reduces the ant sys-tem complexity. The resulting algorithm is orthogonal and simple, yet ants are able to establish increasingly efficient trails from the nest to the food in the presence of obstacles. The algorithm replaces the blind addition of new amounts of pheromones with an adjustment mechanism that resembles dynamic programming.
We present an algorithm for allocating individual ants to tasks that relies solely on task change being caused by the unavailability of work. We prove that such an algorithm will allocate the correct number of individuals to each job. Furthermore, we can demonstrate that if such an algorithm is used then an age structure emerges over the ants performing the various tasks. This matches closely with the weak temporal structure over tasks that is observed in Sendova-Franks and Franks (1993. Division of labour in ants nests within highly variable environments. (A study of temporal polyethism: experimental).Bull. math. Biol. 55, 75–96).