Divide and conquer: A four-fold docking experience of agent-based models

Abstract

Verification and validation (V&V) techniques are used in agent-based modeling (ABM) to determine whether the model is an accurate representation of the real system. Docking is a form of V&V that tries to align multiple simulation models. In a previous paper, we described the docking process of an ABM that simulates the life cycle of Anopheles gambiae. Results showed that the implementations were docked for adult but not for aquatic mosquito populations. In this paper, following the `Divide and Conquer' paradigm, we compartmentalize the simulation world to prohibit the propagation of errors between compartments. Using four separate implementations that sprung from the same core model, we describe a series of docking experiments, analyze the results, and show how they lead to a successful dock. The complete four-fold docking encompasses verification between the four implementations, as well as validation against the core model with respect to these implementations.

Full text

Page 1

Proceedings of the 2010 Winter Simulation Conference
B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Y¨ ucesan, eds.
DIVIDE AND CONQUER: A FOUR-FOLD DOCKING EXPERIENCE OF AGENT-BASED MODELS
S. M. Niaz Arifin
Gregory J. Davis
Steve Kurtz
James E. Gentile
Ying Zhou
Gregory R. Madey
Department of Computer Science & Engineering
University of Notre Dame
384 Fitzpatrick Hall
Notre Dame, IN 46556, USA
ABSTRACT
Verification and validation (V&V) techniques are used in agent-based modeling (ABM) to determine whether the
model is an accurate representation of the real system. Docking is a form of V&V that tries to align multiple
simulation models. In a previous paper, we described the docking process of an ABM that simulates the life cycle
of Anopheles gambiae. Results showed that the implementations were docked for adult but not for aquatic mosquito
populations. In this paper, following the ‘Divide and Conquer’ paradigm, we compartmentalize the simulation world
to prohibit the propagation of errors between compartments. Using four separate implementations that sprung from
the same core model, we describe a series of docking experiments, analyze the results, and show how they lead to
a successful dock. The complete four-fold docking encompasses verification between the four implementations, as
well as validation against the core model with respect to these implementations.
1INTRODUCTION
Verification and validation (V&V) techniques are used in agent-based modeling (ABM) to determine whether the
model is an accurate representation of the real system. The primary goal of verification is to compare multiple
models with each other in order to ensure that the implementations are correct. Validation ensures that the model
is semantically right, and it is indeed modeling the phenomena of the real system being simulated. Docking, also
known as alignment or replication, is a form of V&V that tries to align multiple models in order to investigate
whether they yield similar results. Docking is useful to confirm whether the claimed results of a given simulation
are reliable, and can be reproduced by someone starting from scratch.
1.1 Background
In a previous paper (Arifin, Davis, Zhou, and Madey 2010), we described the verification and validation (V&V)
process of an agent-based model (ABM) that simulates the life cycle of Anopheles gambiae. Anopheles gambiae is
regarded as the primary vector for transmission of Malaria, which is one of the top three pathogen-specific causes of
global mortality. Following a fixed version of the core (conceptual) malaria entomology model, we developed two
agent-based implementations using Java and C++, the former having different versions which were developed in
different phases. Both implementation simulated the life cycle of Anopheles gambiae by tracking attributes relevant
to the vector population dynamics for each individual mosquito. We described the verification performed between
the implementations, and the validation performed against the core model with respect to these implementations. We
also described how the major findings helped to clarify concepts and eliminate ambiguities by identifying differences
in model specification, interpretation, implementation and enhancement phases, and revealing semantic errors. The
importance of rigorous docking was illustrated by the discovery of some incorrect assumptions and programming
575978-1-4244-9864-2/10/$26.00 ©2010 IEEE

Page 2

Arifin, Davis, Kurtz, Gentile, Zhou and Madey
errors, which, being unnoticed, led to erroneous results. However, as evident from the results of (Arifin, Davis,
Zhou, and Madey 2010), the two implementations were not completely ‘docked’, especially in terms of the aquatic
mosquito populations. The reason for this is explained below.
In a complex ABM, synergies arising from seemingly insignificant differences in separate implementations
may lead to significant mismatch in overall model output. Hence, it is crucial, and sometimes necessary, to
‘compartmentalize’ the artificial simulation world, i.e. to separate it into isolated compartments so that errors in
one specific compartment are not propagated and thus cannot influence the discovery (and correction) of errors in
other compartments.
In this paper, we compartmentalize the mosquito world with respect to the adult and aquatic populations, and
perform some experiments to identify and fix the sources of output mismatches in order to achieve a complete
‘dock’. Following the ‘Divide and Conquer’ paradigm, we ensure that the pieces are working as intended, and
then combine them to perform some other, more complex experiments. Using four separate implementations that
sprung from the simplified core malaria entomology model, we describe the design of these experiments, analyze
their results, and show how they lead to a successful dock between all four implementations, and hence achieve a
complete four-fold docking of the core model.
This paper particularly addresses the following issues:
•
How the results from four agent-based implementations compare to each other and to the corresponding
results from theoretical models
How to verify the age-structure of mosquito populations and the age-specific mortality rates (for both adults
and aquatic populations)
How to verify the oviposition mechanism and the calculation of one-day old equivalent larval population
Neat each step of the simulation (see Section 3.5.3), by removing all randomness (e.g. with fixed number
of eggs to lay, single aquatic habitat, etc.)
•
•
The complete four-fold docking encompasses verification between the four implementations, as well as validation
against the core model with respect to these implementations. The major findings, as described in Tables 2, 3, and
4, help to clarify model concepts and eliminate ambiguities.
This paper also serves as a case-study to illustrate the importance of docking complex simulations, since even
the best simulation programs may imply dubious assumptions, leading to erroneous results. Rigorous docking can
detect such problems, uncovering these incorrect assumptions (Will 2009). The four-fold docking process serves the
dual purpose of increasing confidence to the core model and revealing conceptual errors in model implementations.
1.2 Paper Organization
The organization of this paper is as follows: Section 2 discusses some previous works involving docking. Section
3 discusses one fixed version of the core malaria model and its two slightly different variations used in this paper.
Section 4 briefly describes the four separate implementations and the V&V workflow. Section 5 discusses the design
of docking experiments, which are performed in three phases. For each phase, we analyze the results and identify
issues to be updated for experiments in subsequent phases. We also show how the results lead to a successful dock
between all four implementations, and include two additional experiments. Finally, section 6 concludes.
2LITERATURE REVIEW
In this section, we re-emphasize two recent works that show the importance of rigorous docking for agent-based
models. Edmonds and Hales (Edmonds and Hales 2003) replicated a published model involving co-operation
between self-interested agents in two independent implementations to align the results and the conceptual design.
The replication revealed a host of minor bugs and ill-defined implementation issues that otherwise appeared unnoticed.
Theyconcludedthataligningmodelscanbeverydifficult, butveryrevealing, becausesimplyimplementingsimulations
with respect to a conceptual model and then analyzing outputs for consistency with the conceptual model and data
series is insufficient to ensure the correctness of an implementation.
Will and Hegselmann (Will and Hegselmann 2008) showed the importance of replication by reporting a failure
to replicate the results presented on a published model. Obtaining the source code from the original authors, Will
(Will 2009) found that the simulation unintentionally implemented an assumption that was never mentioned in the
576

Page 3

Arifin, Davis, Kurtz, Gentile, Zhou and Madey
original model, and showed that the model crucially depended on that dubious assumption, and its removal leaded
to dramatically different results.
This paper serves as a case study of docking multiple ABMs. It shows how to achieve a complete dock
between multiple implementations, all of which are built on a sufficiently complex core model. It also highlights the
importance of docking by showing that a successful dock may reveal conceptual and/or programming errors, and
may eliminate dubious assumptions, reinforcing the findings in (Carley 2002), (Edmonds and Hales 2003), (Rand
and Wilensky 2006), (Will 2009), and (Will and Hegselmann 2008).
3 THE AGENT-BASED MODELS
In this section, we briefly describe the fixed version of the simplified core (conceptual) malaria entomology model,
and the four different implementations.
3.1 The simplified core models
The core model is primarily governed by the biology underlying Anopheles gambiae. However, for the purpose
of docking, some simplifying assumptions have been made, which may deviate from biological plausibility. For
different experiments performed in this paper (see Section 5), we use two slightly different versions of the core
model. The major difference, as shown in Figure 1, is whether to use a single vs. multiple aquatic habitats.
Emerging Adults
New Eggs
Aquatic
Habitat
Pupa
Egg
Larva
Aquatic States
Female
Gravid
Bloodmeal
Digesting
Bloodmeal
Seeking
Immature
Adult
Adult States
No Eggs Remaining
Emerging Adults
New Eggs
Aquatic
Habitat
1
Pupa
Egg
Larva
Aquatic States
Female
Gravid
Bloodmeal
Digesting
Bloodmeal
Seeking
Immature
Adult
Adult States
No Eggs Remaining
Aquatic
Habitat
2
Pupa
Egg
Larva
Aquatic
Habitat
3
Pupa
Egg
Larva
Figure 1: The simplified Core Model 1 (on left) and the simplified Core Model 2 (on right). Core Model 1 has a
single aquatic habitat and is used in experiments E1, E2 and E3. Core Model 2, designed for experiments E4 and
E5, has three aquatic habitats.
The state durations for mosquito agents are defined in Table 1. One important, though biologically unnatural,
assumption is illustrated in the state duration for the Gravid state: the duration (in days) that the female mosquito
is forced to remain as Gravid equals the number of oviposition attempts taken to successfully lay all her remaining
eggs.
3.2 The Agents
Anopheles gambiae mosquitoes are the only mosquito agents in the model. Like all other mosquito species,
anophelines go through four major states in their life cycle: egg, larva, pupa, and adult (see Figure 1). The first
three states are collectively termed as ‘aquatic’. The adult state of the female mosquito is when the female is able
to be a vector of the malaria parasite. Being autonomous agents, mosquitoes begin their life as eggs (except the
initial adults created at the start of the simulation) and progress through various life cycle states.
577

Page 4

Arifin, Davis, Kurtz, Gentile, Zhou and Madey
Table 1: State durations for the mosquito agents. States marked with * apply only to female mosquito agents.
State
Egg
Larva
Pupa
Immature Adult
Bloodmeal Seeking*
Bloodmeal Digesting*
Gravid*
Duration (in Days)
1
Temperature-dependent; see Equation (1).
1
2
1
2
Stay until all eggs are laid; then stay 1, 2 or 3 days depending on the
number of attempts taken that day.
3.2.1 Aquatic Mosquitoes
Eggs and pupae transition out of their current state after a 24-hour period. The growth and development of larvae,
however, is much slower and depends on daily temperature. Following the stochastic thermodynamic models of
Sharpe and DeMichele (Sharpe and DeMichele 1977) and Schoolfield et al. (Schoolfield, Sharpe, and Magnuson
1981), we use the equation
Devday= (Tempday∗0.000305−0.003285)∗24(1)
where Devdayis the larval development rate and Tempdayis the average temperature (in◦C) of that day. The
threshold for the Larva → Pupa transition is deterministically set to 1.0 removing any randomness (see Section
5.1). Each day, each larva approaches its threshold by developing at a rate characterized by equation (1).
3.2.2 Adult Mosquitoes
For this paper, only female mosquitoes are considered, and all male mosquitoes are omitted. This allows an unbiased,
uniform selection of agents from separate age-groups when the mosquito agents die out. The females feed on
bloodmeals, lay eggs, and go back to feed on bloodmeals again. This is repeated until a female dies. Once a female
enters the Gravid state, it remains there until all of her eggs are laid, which are then instantiated as new agents into
the system. The states are described below.
In Immature Adult, a female mosquito emerges as an immature adult after its aquatic development is complete.
This is the starting state of the ABM where new mosquitoes are initially placed into the system and also when
pupae turn into adults. In Bloodmeal Seeking, the female seeks and finds a bloodmeal. In Bloodmeal Digesting,
the female rests while the blood is digested and eggs are developed. In Gravid, the eggs are developed, and the
female is ready to lay them. The maximum number of eggs a female may oviposit, Eggsmax, is deterministically set
to 80, removing any randomness. All eggs are set to be females. Once a female enters the Gravid state, it remains
there until all of her eggs have been laid, and then transitions back to the Bloodmeal Seeking state.
3.3 The Environment
The environment provides an artificial realm in which agents operate. Variables such as temperature and humidity
factors, defined within the environment, govern how the mosquito agents develop. The aquatic states of An. gambiae
require an aquatic habitat environment to develop, and are often found in transient, sunlit and small pools. These
aquatic habitats control the development of aquatic mosquitoes and the oviposition rate for female adults, as described
below. The carrying capacity of an aquatic habitat represents the repulsive force sensed by a Gravid female and
limits the number of eggs she may oviposit in that habitat. Though not treated as a hard limit, this is an indication
to the female that the habitat is full, and it may be more beneficial to lay eggs elsewhere.
578

Page 5

Arifin, Davis, Kurtz, Gentile, Zhou and Madey
Verification
Internal
Verification
Validation
Java1
Java2
CPP1
CPP2
Core
Model
Start of Day
End of Day
Kill adult mosquitoes
Kill aquatic mosquitoes
Update adult mosquitoes
(age, state duration & state transition)
Update aquatic mosquitoes
(age, state duration & state transition)
Collect, save & print output data
Figure 2: The daily processing order of the simulation (on left) and the V&V workflow (on right). In the workflow,
bidirectional arrows indicate Verification relationships between the four implementations. The dashed arrow, between
the re-factored implementations Java2 and CPP2, indicates Internal Verification, since these two are docked with
respect to each other. Unidirectional, dashed arrows indicate Validation relationships between the Core Model and
the four implementations.
3.4 Age-specific Mortality Rates
In any ABM, the mortality of agents plays a crucial role in shaping the model’s characteristics. Traditionally,
ABMs that model malaria transmission assume age-independent vector mortality (i.e. non-senescence of the vector).
However, as shown by recent studies (Styer, Carey, Wang, and Scott 2007), this leads to misleading predictions,
and obscures age-dependent aspects of mosquito biology. This led us to use the concept of Age-Specific Mortality
Rates (ASMRs) for the adults and larvae states. For these states, ASMR refers to the total number of deaths per
day of a given age. We use a modified version of the Gompertz function, making the age-dependent component
increase exponentially with age.
3.5 Population Dynamics
Mosquito populations are dynamic in nature. The population dynamics is governed by three key operations performed
at each simulation day: killing adult mosquitoes, killing aquatic mosquitoes, and creating new mosquitoes.
3.5.1 Daily Processing Order
The present model discretizes time such that every agent is updated during a time step representing a single day
(24-hour period). The major daily processing steps, modified and hence different from (Arifin, Davis, Zhou, and
Madey 2010), are shown in Figure 2 (left).
3.5.2 Killing Adult Mosquitoes
As noted in Section 3.4, adult mosquitoes have a daily age-specific mortality rate. Newly emergent adults begin
with a daily mortality rate of α. As the mosquito ages, the age-specific mortality rate ASMRAge(adult)for that age
group changes based on the following equation:
ASMRAge(adult)=
α ∗e
Age
β
1+α ∗s∗β(e
Age
β−1)
(2)
579