Content uploaded by Christian Balkenius
Author content
All content in this area was uploaded by Christian Balkenius on Mar 04, 2015
Content may be subject to copyright.
http://adb.sagepub.com/
Adaptive Behavior
http://adb.sagepub.com/content/16/6/349
The online version of this article can be found at:
DOI: 10.1177/1059712308092955
2008 16: 349Adaptive Behavior
Anna Balkenius, Almut Kelber and Christian Balkenius
How Do Hawkmoths Learn Multimodal Stimuli? A Comparison of Three Models
Published by:
http://www.sagepublications.com
On behalf of:
International Society of Adaptive Behavior
can be found at:Adaptive BehaviorAdditional services and information for
http://adb.sagepub.com/cgi/alertsEmail Alerts:
http://adb.sagepub.com/subscriptionsSubscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://adb.sagepub.com/content/16/6/349.refs.htmlCitations:
What is This?
- Nov 13, 2008Version of Record >>
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
349
How Do Hawkmoths Learn Multimodal Stimuli?
A Comparison of Three Models
Anna Balkenius,
1
Almut Kelber,
2
Christian Balkenius
3
1
Department of Chemical Ecology, Swedish University of Agricultural Sciences, Sweden
2
Department of Cell and Organism Biology, Lund University, Sweden
3
Lund University Cognitive Science, Sweden
The moth Macroglossum stellatarum can learn the color and sometimes the odor of a rewarding food
source. We present data from 20 different experiments with different combinations of blue and yellow
artificial flowers and the two odors, honeysuckle and lavender. The experiments show that learning
about the odors depends on the color used. By training on different color–odor combinations and test-
ing on others, it becomes possible to investigate the exact relation between the two modalities during
learning. Three computational models were tested in the same experimental situations as the real
moths and their predictions were compared with the experimental data. The average error over all
experiments as well as the largest deviation from the experimental data were calculated. Neither the
Rescorla–Wagner model nor a learning model with independent learning for each stimulus component
were able to explain the experimental data. We present the new hawkmoth learning model, which
assumes that the moth learns a template for the sensory attributes of the rewarding stimulus. This
model produces behavior that closely matches that of the real moth in all 20 experiments.
Keywords learning · model · hawkmoth · vision · olfaction
1 Introduction
Flowers attract pollinators mainly by color and odor
stimuli. For newly eclosed moths and butterflies, it is
important to quickly recognize a rewarding flower.
Innate color and odor preferences contribute to this abil-
ity (Cunningham, Moore, Zalucki, & West, 2004; Weiss,
1997). By their innate preference for blue, naïve honey-
bees are guided to flowers with a large amount of nectar
(Giurfa, Núñez, Chittka, & Menzel, 1995). A prefer-
ence for blue is shared by other insects but innate color
preferences can differ between species (Weiss, 2001).
Rapid and flexible learning to associate color or odor
with a reward has been demonstrated in honeybees,
butterflies, and moths (Andersson, 2003; Kelber, Voro-
byev, & Osorio, 2003; Menzel, 1967; Srinivasan,
Zhang, & Zhu, 1998; von Frisch, 1914, 1919; Weiss,
1997).
The diurnal hummingbird hawkmoth, Macroglos-
sum stellatarum, uses color vision in searching for
food, and spontaneously forages from colored artifi-
cial flowers without any odor (Kelber, 1997; Kelber &
Hénique, 1999). M. stellatarum has a strong innate
preference for blue flowers as a food source and a
weaker preference for yellow (Kelber, 1997), but it can
easily and equally fast learn other colors including
green, which is not a color of a typical flower (Balken-
ius & Kelber, 2004; Kelber, 1997).
Copyright © 2008 International Society for Adaptive Behavior
(2008), Vol 16(6): 349–360.
DOI: 10.1177/1059712308092955
Correspondence to: Anna Balkenius, Department of Chemical Ecology,
Swedish University of Agricultural Sciences, Alnarp, Box 44, 230 53
Alnarp, Sweden. E-mail: anna.balkenius@ltj.slu.se
Tel.: +46 40 41 52 99; Fax: +46 40 46 19 91
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
350 Adaptive Behavior 16(6)
M. stellatarum has most probably evolved from a
nocturnal ancestor, and in nocturnal hawkmoths odor
is very important in searching for food (Brantjes, 1978;
Raguso & Willis, 2002). It has recently been shown
that the ability of M. stellatarum to learn an odor that
accompanies a color depends on the choice of color
(Balkenius & Kelber, 2006). When an innately pre-
ferred blue color is learned together with an odor, the
moth will not learn the odor. However, if the less pre-
ferred color yellow is used instead, the moths can read-
ily learn the odor.
Stimuli of one sensory modality can influence learn-
ing of stimuli of another modality in different ways. In
most cases, two stimuli are more effective than one, and
the advantages of multisensory integration are of great
importance in many animals (Luo & Kay, 1992). In
honeybees, colors attract attention before odor, while
odor attracts attention when the bees are very close to
the food source (Giurfa, Núñez, & Backhaus, 1994; von
Frisch, 1919). There is also evidence for increased
learning when two stimulus types are combined (Rowe,
1999). In bumblebees, it has been shown that the pres-
ence of odor enhances color discrimination, and
increases attention and memory formation (Kunze &
Gumbert, 2001). In honeybees, the similarity between
colors modulates odor learning (Giurfa et al., 1994;
von Frisch, 1919). The similarity between colors has
also been shown to modulate place learning in a hawk-
moth (Balkenius, Kelber, & Balkenius, 2004).
A special case of multimodal learning is config-
ural learning where an animal learns to respond to a
configuration of stimuli, but not to the single stimulus
modalities themselves (Mackintosh, 1974). The hawk-
moth Manduca sexta needs both an odor and a visual
stimulus to unroll the proboscis for feeding (Raguso &
Willis, 2002), which also might be a preference for a
configuration of both cues.
In contrast, two different situations have been found
where learning of one stimulus prevents the learning of
another stimulus. First, animals trained to a stimulus
compound consisting of, for instance, a color and an
odor, sometimes only learn one of the components. For
example, they learn the color but not the odor. This
effect is called overshadowing (Pavlov, 1927). Second,
when animals are first trained to one stimulus compo-
nent and later to the compound, they will not learn the
stimulus component that was initially absent. The first
component already predicts the reward and blocks
learning of the second component (Kamin, 1969).
Blocking and overshadowing were originally defined
for classical conditioning but have also been found in
instrumental conditioning (Couvillon, Campos, Bass,
& Bitterman, 2001; Couvillon, Mateo, & Bitterman,
1996; Mackintosh, 1974). A possible reason for the lack
of learning of the second stimulus may be that the ani-
mal directs its attention only to the first stimulus (Zen-
tall & Riley, 2000). The existence of blocking and
overshadowing in insects is controversial and experi-
ments have given mixed results (Couvillon et al.,
1996, 2001; Couvillon, Arakaki, & Bitterman, 1997;
Funayama, Couvillon, & Bitterman, 1995; Gerber &
Ullrich, 1999). In particular, it has been disputed
whether the learning of one stimulus modality depends
on the other.
To test this, we collected data from 20 different
learning experiments with M. stellatarum where mul-
timodal stimuli were used. Most of the animal data
have been previously published (Balkenius & Kelber,
2006), but experiments 8–12 are reported here for the
first time. We also tested a number of learning mod-
els on these experiments using computer simulations.
Although a number of models of insect learning exist
(e.g., Borisyuk & Smith, 2004; Linster & Smith, 1997;
Wessnitzer, Webb, & Smith, 2007), most of these are
not applicable to our data. Because we were specifi-
cally interested in the role of interaction between stimu-
lus components during learning, the primary models
tested were the following:
1. The Rescorla–Wagner model, which assumes that
learning depends on all stimulus components
present and the prediction error of the reward mag-
nitude.
2. The independence model, which assumes that learn-
ing of each stimulus component is independent of
the other.
3. The new hawkmoth learning model, which assumes
that the moth learns a template for the rewarded
stimulus.
The first two models were selected as they have previ-
ously been suggested to explain learning in moths
(Couvillon et al., 1997; Funayama et al., 1995). The
new model was developed to overcome some of the
limitations of the first two models.
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
Balkenius, Kelber, & Balkenius How Do Hawkmoths Learn Multimodal Stimuli? 351
2 Materials and Methods
M. stellatarum were bred in the laboratory throughout
the year. The larvae were fed their natural food plant,
and the pupae were kept at 20 °C. On the day after
eclosion, the naïve moths were released in the cage
with two feeders (Pfaff & Kelber, 2003). The experi-
mental cage measured 50 × 60 × 70 cm
3
and was illumi-
nated from above with four fluorescent tubes (Osram,
Biolux). Two feeders were placed 35 cm above the cage
floor and 30 cm apart from each other. To prevent place
learning (Balkenius et al., 2004), the feeders were ran-
domly shifted between four locations during learning.
During training, the rewarded feeder was filled with
sucrose solution and the unrewarded feeder contained
water. Groups of up to 25 moths were flying and feed-
ing in the same cage. The tests occurred after 4 days of
training. During tests, both feeders were filled with
water and each moth was tested on its own. In all
experiments, only the first artificial flower the moth
touched with its proboscis was recorded.
Two colors—blue (B) and yellow (Y)—and two
odors—artificial honeysuckle (H) and extract of lav-
ender oil (L)—were used in the experiments. We dis-
tributed 25 1l of the odor extract in 10 ml of water or
sucrose solution in the feeders and refilled every sec-
ond day to make sure odor as well as reward was
always available to the moths. Both honeysuckle and
lavender flowers are visited by M. stellatarum in the
wild (Herrera, 1992). In electroantennograms, M. stel-
latarum responded strongly to both odors (Balkenius,
Rosén, & Kelber, 2006).
We ran 20 different experiments with different
combinations of colors and odors. Experiments 1–5
were different preference tests (Figure 1). Untrained
moths were presented with two stimuli and their first
choice was recorded. The stimulus combinations used
were B/Y, YH/YL, BH/BL, BL/YH, and BH/YL. Note
that it is not possible to present an odor without a vis-
ual stimulus or to compare the preference for a color
with the preference for an odor. The results of the
preference tests were used to set the initial weights of
the different computational models. The numbers of
animals tested in the first five experiments were 25,
38, 21, 25, and 10, respectively.
In experiments 6 and 7, we tested the ability of
the moths to learn which color was rewarded. The
training used B+/Y and Y+/B, respectively, where +
indicates that this stimulus was rewarded. The tests
used the same stimuli, but without any reward. There
were 20 animals in each experiment.
In experiments 8–12, the moths were trained on
one combination of color and odor, and tested on
another. These combinations are shown in Figure 2.
The numbers of animals tested in these experiments
were 50, 18, 21, 10, and 18, respectively.
We also used additional data from eight experiments
previously reported by Balkenius and Kelber (2006),
summarized in Figures 3 and 4. The experiments shown
in Figure 4 started with a pretraining phase where the
Figure 1 The results of the preference tests (experiments 1–5). All models reproduce the results of the preference
tests very well (animal data from Balkenius & Kelber, 2006). B, blue; Y, yellow; H, honeysuckle; L, lavender.
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
352 Adaptive Behavior 16(6)
preference for a color was changed. In two of the
experiments, the weak preference for yellow was
strengthened by a pretraining procedure. In the two other
experiments, the strong preference for blue was weak-
ened to see how this would influence subsequent learn-
ing (for details, see Balkenius & Kelber, 2006).
3 Computational Models
The experiments run with the real moths were also
tested with three computational models to see if these
models were able to explain the behavior of the ani-
mals. These models were the Rescorla–Wagner
model, an independence model, and the new hawk-
moth learning model, which is described here for the
first time.
For all models, each flower stimulus was coded as
a vector s = with four components cod-
ing for blue color (s
0
), yellow color (s
1
), honeysuckle
odor (s
2
), and lavender odor (s
3
). Each of these compo-
nents was set to 1 when the corresponding stimulus
component was available, and 0 otherwise. For exam-
ple, the stimulus BL was coded as s = .
Figure 2 Results of experiment 12–16. Choices of the stimulus with the rewarded color after discrimination training in
five experiments for the real moth and the three models. In the experiments, the moths (and models) were first trained
on one combination of color and odor and later tested on another combination, to see how much of the learning involved
color and odor, respectively. The three stars indicate that the behavior of the model was significantly different from the
moth with p < .001 for Fisher's exact test. See Figure 3 for further explanation.
Figure 3 Choices of the stimulus with the rewarded odor after discrimination training in experiments 8–11 for the real
moth (data from Balkenius & Kelber, 2006) and the three models. With the yellow color, the moths learn the odors, but
with the blue color, they do not. This is predicted by the hawkmoth learning model, but not by the Rescorla–Wagner or
independence models.
s
0
s
1
s
2
s
3
22234
100122234
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
Balkenius, Kelber, & Balkenius How Do Hawkmoths Learn Multimodal Stimuli? 353
3.1 Rescorla–Wagner Model
As it appears that learning of one stimulus component
can block learning of another in the moth experi-
ments, it seems reasonable to test how well the Res-
corla–Wagner model is able to reproduce the results of
the experiments (Rescorla & Wagner, 1972). Let w(t)
be the current weight vector, s(t) the stimulus vector,
and R(t) the reward at time t. When the moth attempts
to forage, the weights are updated according to the
equation
w
i
(t + 1) = w
i
(t) + 56(t)s
i
(t), (1)
for both rewarded and unrewarded trials. Here, 5 is the
learning rate and 6(t) is the difference between the
actual and expected reward
(2)
where n = 3, as there were four different stimulus
components. This formulation of the Rescorla–Wag-
ner model is equivalent to the delta-rule commonly
used in neural network models (Widrow & Hoff,
1960). When the model moth senses a stimulus s(t), it
is selected according to the probability
(3)
This probability thus determines whether the
model moth will try to forage from a single flower at a
single point in time. Note that this is not the probabil-
ity of selecting one stimulus type in relation to
another.
3.2 Independence Model
There is not a single established mathematical formu-
lation of the idea that each stimulus component
acquires associations independently of other stimulus
components. We collectively call these models the
independence model. Here, we use a mathematical
formulation that is similar to the Rescorla–Wagner
model, except that the stimulus components do not
interact during learning. In this model, one stimulus
component is not able to block learning of another and
Equations 1 and 2 are replaced by
w
i
(t + 1) = w
i
(t) + 56
i
(t)s
i
(t)(4)
Figure 4 Choices of the rewarded color in the training phase in experiments 17–20 for the real moth (data from Balken-
ius & Kelber, 2006) and the three models. By pretraining the moths, their learning could be changed. (a), (b) When the in-
nate preference for blue was extinguished through discrimination learning, the moths could learn to discriminate between
the two odors. This behavior was predicted by all models. (c), (d) When the moths were pretrained to prefer yellow, they
lost their ability to learn a discrimination between the two odors. The hawkmoth learning and independence models pre-
dict this behavior, while the Rescorla–Wagner model fails. See Figure 3 for further explanation.
6 t78 Rt78 w
i
t78s
i
t782
i 0=
n
9
–=
pst78 w
i
t78s
i
t78.
i 0=
n
9
=
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
354 Adaptive Behavior 16(6)
and
6
i
(t) = R(t) – w
i
(t)s
i
(t). (5)
For the binary stimuli used here, this learning rule
is equivalent to the Bush–Mostellar model (Bush &
Mostellar, 1955). The probability with which a stimu-
lus is selected is calculated in the same manner as for
the Rescorla–Wagner model (Equation 3).
3.3 Hawkmoth Learning Model
The hawkmoth learning model assumes that the animal
learns a single template for the sensory attributes of the
flowers that are rewarded during foraging. The tem-
plate is updated when the moth is rewarded such that it
more closely matches the vector associated with the
current flower. If the moth is unrewarded, the template
is left unaltered. Let w(t) be the current weight vector
coding for the flower template, and R(t) the current
reward. When the moth is rewarded, the weights are
updated according to the following equation
(6)
where
(7)
and 6(t) is calculated as for the Rescorla–Wagner model
(Equation 2). Weights are not allowed to become nega-
tive. To function as a template, it is necessary that the
weight vector w is normalized as described by Equation
6. This makes the model sensitive to the stimulus pat-
tern and not to its magnitude. The match between the
template and the current external stimulus is thus used
to predict the reward. Unlike the Rescorla–Wagner
model, however, the learning attempts to move the
learned template towards the rewarded stimulus pattern
instead of directly decreasing the prediction error. As a
consequence, the prediction error will reach zero as
the template approaches the rewarded stimulus (see
the Appendix).
Because of 6 in Equation 7, excitatory learning
only occurs when there is a prediction error, which
makes blocking possible when the reward is already
predicted by the stimulus. As the template is normal-
ized, but the stimulus input is not, it is possible for a
stimulus component to completely block learning
even if the stimulus and the template are not identical.
In this case, learning will converge before the tem-
plate reaches the stimulus pattern.
The probability of selecting a stimulus is set to
p[s(t)] = . (8)
The sum describes the matching process and the
exponent q is a parameter that is used to derive selec-
tion probabilities from the matching. This parameter
was set to q = 2.00 to quantitatively fit the experimen-
tal data.
3.4 Simulations
The three models were tested on the 20 experiments
described above. During each simulation, the simu-
lated moth was randomly presented with one of two
stimuli and was allowed to select it with the probabil-
ity given by the selection functions described above.
Data from 100,000 simulated animals were recorded
for each experiment and each model. The simulated
moths were rewarded 50 times during each learning
phase to approximate the number of visits to flowers
by the real moths. The exact number of rewards are
not critical to our results as learning parameters for
each model were optimized to compensate for the
number of trials used.
To allow a fair comparison of the performance of
the different models, the initial weights for each model
were set to parallel the stimulus preferences of the
moth as closely as possible (Figure 1). The weights of
each model were optimized to two decimal places to
make the model perform as well as possible on the
preference data. There were no significant differences
between the results of the real moth and any of the
models on the preference tests (Fisher’s exact test:
hawkmoth learning model p = .37, Rescorla–Wagner
model p = .35, independence model p = .35).
The constants for each model were subsequently
numerically optimized to minimize the average error
over all experiments for each model. These constants
and initial weight values are given in Table 1.
w
i
t 1+78
u
i
t 1+78
u
i
t 1+78
i 0=
n
9
-----------------------------=
u
i
t 1+78
w
i
t78 56t78+when s
i
1=
w
i
t78 e–otherwise
=
w
i
s
i
t78
i 0=
n
9
q
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
Balkenius, Kelber, & Balkenius How Do Hawkmoths Learn Multimodal Stimuli? 355
Each experiment with each model was tested
against the data from the real moth using Fisher’s
exact test. The simulation results were used to calcu-
late the expected number of choices based on the
actual number of choices in the experiments with real
moths, and these were compared with the moth data.
Finally, we combined the results of each individual
test into a combined measure for the whole model.
4Results
The results of experiments 1–5 showed that the moth
had a marked preference for blue, but no clear prefer-
ence for any of the odors (Figure 1). Because the
parameters of each model were optimized to reflect
these preferences, all models behaved as the real moth
in these preference tests. Experiments 6 and 7 verified
that the moth could be trained to select either a blue or
yellow flower. The real moth selected yellow in 80%
of the trials after being trained on yellow, and blue in
95% of the trials after being trained on blue. All mod-
els, except the random selection, were able to learn
these discriminations.
Figure 3 shows the result of experiments 8–11 with
the same color but different odors (Balkenius & Kelber,
2006). In the real moth, the blue color prevents odor
learning from occurring, but with the yellow color, the
moth is able to learn which odor is rewarded. The
hawkmoth learning model gives almost the same result
as the real moth on all experiments. In contrast to the
real moth, the Rescorla–Wagner model learns the odor
in all experiments. The same is true about the inde-
pendence model, although the learning is less pro-
nounced for this model, regardless of which color was
used.
The behavior of the real moth and the different
models differ even more in experiments 12–16, shown
in Figure 2. Here, it is again evident that the real moth
learns odor when it is presented together with yellow
(Figure 2a and e). The fact that the test with color and
odor (Figure 2a) and the test with only color (Figure 2b)
differ, also shows that the animals must have learned
the odor. With the blue color, the animals did not learn
the odor and the result is the same with and without
odor (Figure 2c and d).
Again, the predictions of the hawkmoth learning
model were very close to the actual data, but the other
two models differed in different ways. In experiments
12 and 16 (Figure 2a and e), the Rescorla–Wagner
model did not make the correct discrimination, and
appears to select the correct odor and ignore the color.
The independence model does not take the color into
account when learning odor and learns the color in
experiment 10 (Figure 2c), when the other models and
the real moth does not.
For the real moth, the preference for the color
could be changed by pretraining (Balkenius & Kelber,
2006). In the experiments shown in Figure 4a and b,
the innate preference for blue is decreased during pre-
training. As a result, the moth can later learn odors
together with a blue artificial flower. The opposite sit-
uation is shown in Figure 4c and d where the less pre-
ferred yellow is made more attractive during pretraining.
As a consequence, the real moth no longer learns the
odor together with yellow.
Like the real moth, the hawkmoth learning model
behaves differently depending on which color is used
and whether it was pretrained or not. This is also true
of the independence model, although the difference in
the two cases is not as large. For the Rescorla–Wagner
model, however, the learning is almost the same
regardless of the color or pretraining.
Figure 5 shows the overall results of the simula-
tions for the different models. The average error of the
new hawkmoth learning model is clearly much lower
than that of the other models. Both the Rescorla–Wag-
ner and the independence models are much better than
Table 1 Optimal parameter for each of the models. Note that the sum of the weights for the hawkmoth learning model
equals 1.
Model 5 e w
0
w
1
w
2
w
3
Rescorla–Wagner 0.04 – 0.09 1.00 0.12 0.03
Independence 0.27 – 0.09 1.00 0.12 0.03
Hawkmoth learning 0.09 0.12 0.19 0.74 0.07 0.00
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
356 Adaptive Behavior 16(6)
random selection, where the two stimuli are both
selected with equal probability. Looking at the maxi-
mal error, the hawkmoth learning model reproduces
the data much more closely than the other models.
Surprisingly, both the Rescorla–Wagner and the inde-
pendence models perform at close to the random
model in the worst case. The Rescorla–Wagner model
is even worse than the random model on some experi-
ments.
The difference between the real moth and the dif-
ferent choices in the simulations for each model and
each experiment were statistically analyzed using
Fisher’s exact test. The average simulation results
from 100,000 trials were used as the expected values
and were adapted to the number of choices recorded
with the real moths.
The results of all experiments for each model
were combined to test whether the behavior of each
model differed significantly from the moth data. There
were no significant differences between the behavior
of the hawkmoth learning model and the real moth
(p = .12). The behavior of the Rescorla–Wagner
model differed significantly from the moth data (p <
.0001). This was also the case for the independence
model (p < .0001).
5 Alternative Models
The two critical assumptions of the hawkmoth learn-
ing model are that the moth learns a template for the
rewarded stimulus and that learning only occurs on
rewarded trials. To test if the other models would do
better if they incorporated these assumptions, we ran a
number of simulations with modified versions of the
independence and Rescorla–Wagner models.
We first tested the behavior of the two models
when they did not learn on unrewarded trials. For the
modified independence model, the average error
increased to 16% and the maximum error became
50%. The behavior of the modified model is signifi-
cantly different from the animal data (Fisher’s exact
test, p < .001). The average error of the Rescorla–
Wagner model decreased slightly to 15% but the max-
imum error increased to 68%. The behavior of the
modified model is still significantly different from the
animal data (Fisher’s exact test, p < .001).
As a second step, we tested the assumption that a
template is used. The Rescorla–Wagner model can be
modified to operate on stimulus configurations rather
than stimulus components as the hawkmoth learning
model. According to the configural theory of Pearce
(1994), new configurations are learned as new stimuli
are encountered, and associations are formed from
these configurational codes rather than from the indi-
vidual stimulus components. As this model does not
specify how innate preferences should be handled, we
tested two versions of this model.
In the first case, we used the original version of
the model without any preferences. For this model, the
average error was 18% and the maximum error 50%.
In the second case, we added direct innate associa-
tions from the stimulus vector to produce the same
preferences as for the Rescorla–Wagner model. The
Figure 5 Overall results of the three models and a random selection strategy (horizontal stripes). (a) Average error on
all 20 experiments. The new model clearly outperforms the other models with an average error of 4.14%. (b) The maxi-
mum error for each of the models and the random selection strategy. Again, the new model is much better than the two
alternatives. The behaviors of the Rescorla–Wagner and independence models and the random strategy differ signifi-
cantly from that of the real moth.
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
Balkenius, Kelber, & Balkenius How Do Hawkmoths Learn Multimodal Stimuli? 357
learned associations were added (or subtracted) from
these preferences. For this model, the average and
maximum errors were 16% and 61%, respectively.
The behavior of the two versions of the model differs
significantly from the animal data (Fisher’s exact test,
p < .001 in both cases).
None of the modified models came close to the
performance of the hawkmoth learning model. It is
thus not sufficient to add the assumption of learning
to only rewarded trials or the assumption of template
learning to the other models to account for the learn-
ing of the moth.
6Discussion
We have reported the results of 20 experiments with
moths in different discrimination tasks involving mul-
timodal stimuli with color and odor. Three compu-
tational models were tested on the data to try to
determine the mechanisms behind this type of learn-
ing in hawkmoths. This is the first time multimodal
learning in sphingids has been modeled and the results
show that the learning mechanisms in insects can be
far from trivial.
We observed behaviors that are reminiscent of
overshadowing (Figure 3a and b) and blocking (Fig-
ure 4c and d). In naïve moths, the degree of odor learn-
ing depended on the color used during training
(Figure 3). Although the Rescorla–Wagner model is
often proposed as an explanation for these phenomena
(Rescorla & Wagner, 1972), it was not able to repro-
duce our experimental results without changing the
parameters for each individual experiment. Because
the parameters were set to minimize the overall error
on all experiments, the model failed in some instances.
In fact, in one case, this model performed worse than
random selection (Figure 5). This parameter sensitivity
is a well-known problem with this model (Gallistel,
1990). Although the model is able to handle a wide
range of conditioning experiments, it cannot do so with
identical parameters. For example, in the experiments
shown in Figure 3a and b, the blue color is not able to
block odor learning as the association from the blue
color undergoes extinction on the non-rewarded trials,
and thus loses its ability to block odor learning. For the
same reason, the Rescorla–Wagner model fails to
reproduce the blocking-like situation in Figure 4c and d.
The influence of the non-rewarded trials on the result is
highly dependent on the precise learning rate and the
number of trials. In contrast, in the real moth, the
behavior does not critically depend on the number of
trials.
This was the motivation for the learning rule in the
hawkmoth learning model, where learning only occurs
during rewarded trials. This model is thus immune to
extinction during non-rewarded trials and can accu-
rately predict the behavior of the moth in all the exper-
iments in Figures 3 and 4. In particular, the model will
never learn the odor with a blue color as this color is
never extinguished as long as it is the only rewarded
color. Thus, blocking remains intact throughout the
experiment (Figure 3a and b). However, extinction of
blue can occur if another color is rewarded as in exper-
iments 17 and 18 (Figure 4a and b).
Although the hawkmoth learning model cannot
handle extinction through presentation of a single
non-rewarded stimulus in its current form, it can eas-
ily be extended with a non-specific extinction mecha-
nism that decreases the overall response probability
after a non-rewarded trial. In order not to interfere
with the blocking mechanism in the model, such
extinction would have to influence all stimuli and not
only the non-rewarded one. However, no experimental
data are currently available on the properties of extinc-
tion in moths in the free-flying paradigm.
To only learn on rewarded trials cannot by itself
explain the results of the hawkmoth learning model. A
modified Rescorla–Wagner model or independence
model that only learns at rewarded trials is not able to
predict the experimental results as well as the original
models. While extinction at unrewarded trials is nec-
essary for the Rescorla–Wagner and independence
models to avoid constantly increasing weights, the
hawkmoth learning model uses normalization to keep
the weights within bounds. This also results in the for-
mation of a template for the rewarded stimulus combi-
nation. This template acts as an adaptive search image
that can be used to lead the animal to rewarding flow-
ers (Goulson, 2000; Tinbergen, 1960).
Surprisingly, the independence model was slightly
better than the Rescorla–Wagner model, both on aver-
age and in the worst case (Figure 5). In particular, this
was the case in the blocking-like experiments in Fig-
ure 4. The reason for this is the interaction between the
initial preferences and the particular number of trials,
despite a fundamental disability to handle these learn-
ing situations. However, from the animal data, it is
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
358 Adaptive Behavior 16(6)
clear that the learning of one modality depends on the
other.
These experiments are in line with color prefer-
ence tests that have shown that M. stellatarum prefers
blue to yellow (Kelber, 1997). Blocking has been
demonstrated with free-flying honeybees. Experi-
ments have shown that the blocking effect depended
on the salience of the different stimuli, that is, how
easy it was for the animal to detect the stimulus (Cou-
villon et al., 1997). In honeybees, the salience of a
stimulus (e.g., the concentration of an odor) also influ-
enced its ability to overshadow other stimuli (Pelz,
Gerber, & Menzel, 1997).
The results of our simulations differ from the
results that would be obtained in the standard case
when the different models start out without any pref-
erences and all weights are zero. With initial weights
at zero, the Rescorla–Wagner model would be better
at handling the blocking-like experiment. However, its
overall score would decrease, as its ability to handle
many of the other experiments would be reduced. This
is a consequence of our methodology where the
parameters for each model were optimized in relation
to all experiments taken together. Given that the mod-
els claim to describe the learning of an animal in many
different situations, it is important that the parameters
are not tweaked to fit each experiment.
The different modalities could be handled in the
same way by the hawkmoth learning model. It was
sufficient to set the initial weights in accordance with
the observed preferences of the moths. This shows
that it is not necessary to assume that different modal-
ities are handled in different ways in hawkmoth learn-
ing.
As the initial weights are not zero, the behaviors
of the different models are not always what would be
expected. The hawkmoth is innately prepared for the
stimuli used in these experiments, which contrasts
with the stimuli often used in conditioning studies
(Seligman, 1970).
In the future, we would also like to further study
two of the assumptions of the model in experiments
with real moths. One is that extinction never occurs or
is non-specific. The other is that the moth can only
learn a single template.
In summary, we have presented experimental
results from 20 different experiments with the hawk-
moth M. stellatarum, which show that the particular
color of an artificial flower determines whether the
moth will learn its odor or not. Also, when the moth
has learned a combination of color and odor, color is
most important. By manipulating the preference for
the colors, its effect on odor learning could be
changed. Furthermore, we have shown that neither the
Rescorla–Wagner model nor the independence model
are able to explain the experimental results. Instead,
we have proposed a new model, the hawkmoth learn-
ing model, which is based on the idea that the moth
learns a template for the rewarded multimodal stimu-
lus when it is rewarded. This new model faithfully
reproduces all the experimental data.
Appendix
In this appendix, we show that learning in the hawk-
moth learning model converges when trained in a dis-
crimination task with two binary stimulus vectors a
and b, where a is rewarded, but b is not. As no learning
occurs for stimulus b, we only need to consider the
rewarded trials where s
i
= a
i
. Let X = {i|s
i
= 1} and Y =
{i|s
i
= 0}. According to Equation 7, the weights w
i
for
i Y trivially converge to zero. It remains to show that
the weights w
i
, for which i X, converge. It is sufficient
to show that the learning converges once w
i
(for which
i Y) have reached zero. At this stage, Equation 7 can be
simplified to 6(t) = 1 – w
i
(t). However, as the vec-
tor w is normalized, w
i
(t) = 1, which implies that
6(t) = 0. This proves that the learning converges.
Acknowledgments
We would like to thank Michael Pfaff for help with
breeding the M. stellatarum. We are grateful for the
financial support from the Swedish Research Council.
We would like to thank the three anonymous review-
ers for their insightful comments on the manuscript.
The code for the simulations is available at http://
www.lucs.lu.se/Downloads.
References
Andersson, S. (2003). Foraging responses in the butterflies Ina-
chis io, Aglais urticae (Nymphalidae), and Gonepteryx
rhamni (Pieridae) to floral scents. Chemoecology, 13, 1–
11.
iX
9
iX
9
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
Balkenius, Kelber, & Balkenius How Do Hawkmoths Learn Multimodal Stimuli? 359
Balkenius, A., & Kelber, A. (2004). Color constancy in diurnal
and nocturnal hawkmoths. Journal of Experimental Biol-
ogy, 207, 3307–3316.
Balkenius, A., & Kelber, A. (2006). Color preferences influ-
ence odor learning in the hawkmoth, Macroglossum stell-
atarum. Naturwissenschaften, 93, 255–258.
Balkenius, A., Kelber, A., & Balkenius, C. (2004). A model of
selection between stimulus and place strategy in a hawk-
moth. Adaptive Behavior, 12(1), 21–35.
Balkenius, A., Rosén, W., & Kelber, A. (2006). The relative
importance of olfaction and vision in a diurnal and a noctur-
nal hawkmoth. Journal of Comparative Physiology A: Sen-
sory, Neural, and Behavioral Physiology, 192(4), 431–437.
Borisyuk, A., & Smith, B. H. (2004). Odor interactions and
learning in a model of the insect antennal lobe. Neurocom-
puting, 58–60, 1041–1047.
Brantjes, N. B. M. (1978). Sensory responses to flowers in
night-flying moths. In A. J. Richards (Ed.), The pollinaton
of flowers by insects (pp. 13–19). Dorchester, UK: Dorset
Press.
Bush, R. R., & Mostellar, F. (1955). Stochastic models for
learning. New York: Wiley.
Couvillon, P. A., Arakaki, L., & Bitterman, M. E. (1997).
Intramodal blocking in honeybees. Animal Learning and
Behavior, 25, 277–282.
Couvillon, P. A., Campos, A. C., Bass, T. D., & Bitterman, M.
E. (2001). Intermodal blocking in honeybees. The Quar-
terly Journal of Experimental Psychology B, 54, 369–381.
Couvillon, P. A., Mateo, E. T., & Bitterman, M. E. (1996).
Reward and learning in honeybees: Analysis of an over-
shadowing effect. Animal Learning and Behavior, 24, 19–
27.
Cunningham, J. P., Moore, C. J., Zalucki, M. P., & West, S. A.
(2004). Learning, odour preference and flower foraging in
moths. Journal of Experimental Biology, 207, 87–94.
Funayama, E. S., Couvillon, P. A., & Bitterman, M. E. (1995).
Compound conditioning in honeybees: Blocking tests of
the independence assumption. Animal Learning and
Behavior, 23, 429–437.
Gallistel, R. C. (1990). The organization of learning. Cam-
bridge, MA: MIT Press.
Gerber, B., & Ullrich, J. (1999). No evidence for olfactory
blocking in honeybee classical conditioning. Journal of
Experimental Biology, 202, 1839–1854.
Giurfa, M., Núñez, J., & Backhaus, W. (1994). Odour and col-
our information in the foraging choice behaviour of the
honeybee. Journal of Comparative Physiology A, 175,
773–779.
Giurfa, M., Núñez, J., Chittka, L., & Menzel, R. (1995). Colour
preferences of flower-naïve honeybees. Journal of Com-
parative Physiology A, 177, 247–259.
Goulson, D. (2000). Are insects flower constant because they
use search images to find flowers? Oikos, 88(3), 547–552.
Herrera, C. M. (1992). Activity pattern and thermal biology of
a day-flying hawkmoth (Macroglossum stellatarum) under
mediterranean summer conditions. Ecological Entomol-
ogy, 17, 52–56.
Kamin, L. J. (1969). Predictability, surprise, attention and con-
ditioning. In B. A. Campbell & R. M. Church (Eds.), Pun-
ishment and aversive behavior (pp. 279–296). New York:
Appleton-Century-Crofts.
Kelber, A. (1997). Innate preferences for flower features in the
hawkmoth Macroglossum stellatarum. Journal of Experi-
mental Biology, 200, 826–835.
Kelber, A., & Hénique, U. (1999). Trichromatic color vision in
the hummingbird hawkmoth, Macroglossum stellatarum.
Journal of Comparative Physiology A, 184, 535–541.
Kelber, A., Vorobyev, M., & Osorio, D. (2003). Animal color
vision—behavioral tests and physiological concepts. Bio-
logical Reviews, 78, 81–118.
Kunze, J., & Gumbert, A. (2001). The combined effect of color
and odor on flower choice behavior of bumble bees in
flower mimicry systems. Behavioral Ecology, 12, 447–
456.
Linster, C., & Smith, B. (1997). A computational model of the
response of honey bee antennal lobe circuitry to odor mix-
tures: overshadowing, blocking and unblocking can arise
from lateral inhibition. Behavioural Brain Research,
87(1), 1–14.
Luo, R. C., & Kay, M. G. (1992). Data fusion and sensor inte-
gration: State-of-the-art 1990s. In M. A. Abidi & R. C.
Gonzalez (Eds.), Data fusion in robotics and machine
intelligence. Boston, MA: Academic Press.
Mackintosh, N. J. (1974). The physiology of animal learning.
London: Academic Press.
Menzel, R. (1967). Untersuchungen zum erlernen von spektral-
farben durch die honigbiene (Apis mellifica). Zeitschrift
für vergleichende Physiologie, 56, 22–62.
Pavlov, I. P. (1927). Conditioned reflexes. Oxford: Oxford Uni-
versity Press.
Pearce, J. M. (1994). Similarity and discrimination: A selective
review and a connectionist model. Psychological Review,
101(4), 587–607.
Pelz, C., Gerber, B., & Menzel, R. (1997). Odorant intensity as
a determinant for olfactory conditioning in honeybees:
Roles in discrimination, overshadowing and memory con-
solidation. Journal of Experimental Biology, 200, 837–
847.
Pfaff, M., & Kelber, A. (2003). Ein vielseitiger Futterspender
für anthophile Insekten. Entomologische Zeitschrift, 113,
360–361.
Raguso, R. A., & Willis, M. A. (2002). Synergy between visual
and olfactory cues in nectar feeding by naïve hawkmoths,
Manduca sexta. Animal Behaviour, 64, 685–695.
Rescorla, R., & Wagner, A. (1972). A theory of pavlovian con-
ditioning: variations in the effectiveness of reinforcement
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
360 Adaptive Behavior 16(6)
and non-reinforcement. In A. H. Black & W. F. Prokasy
(Eds.), Classical conditioning II: Current research and
theory (pp. 64–99). New York: Appleton-Century-Crofts.
Rowe, C. (1999). Receiver psychology and the evolution of
multicomponent signals. Animal Behaviour, 58, 921–931.
Seligman, M. E. P. (1970). On the generality of the laws of
learning. Psychological Review, 77, 406–418.
Srinivasan, M. V., Zhang, S. W., & Zhu, H. (1998). Honeybees
links sight to smell. Nature, 396, 637–638.
Tinbergen, N. (1960). The natural control of insects in pine
woods: Vol. I. Factors influencing the intensity of preda-
tion by songbirds. Archives Neelandaises de Zoologie, 13,
265–343.
von Frisch, K. (1914). Der Farbensinn und Formensinn der
Biene. Zoologisher Jahrbucher, Abteilung für Allgemeine
Zoologie und Physiologie der Tierre, 15, 193–260.
von Frisch, K. (1919). Über den Geruchssinn der Bienen und
seine blütenbiologische Bedeutung. Zoologisher Jahrbucher,
Abteilung für Allgemeine Zoologie und Physiologie der
Tierre, 37, 2–238.
Weiss, M. (1997). Innate colour preferences and flexible colour
learning in the pipevine swallowtail. Animal Behaviour,
53, 1043–1052.
Weiss, M. (2001). Vision and learning in some neglected polli-
nators: Beetles, flies, moths, and butterflies. In L. Chittka
& J. D. Thomson (Eds.), Cognitive ecology of pollination
(pp. 171–190). Cambridge: Cambridge University Press.
Wessnitzer, J., Webb, B., & Smith, D. (2007). A model of non-
elemental associative learning in the mushroom body
neuropil of the insect brain. In Proceedings of the Interna-
tional Conference on Adaptive and Natural Computing
Algorithms (LNCS Vol. 4431, pp. 488–497). Berlin:
Springer-Verlag.
Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits.
In IRE WESCON convention record (pp. 96–104). New
York: Institute of Radio Engineers.
Zentall, T. R., & Riley, D. A. (2000). Selective attention in ani-
mal discrimination learning. Journal of General Psychol-
ogy, 127, 45–66.
About the Authors
Anna Balkenius is a postdoc at the Department of Chemical Ecology at the Swedish
University of Agricultural Sciences. She received her Ph.D. in the Vision Group, at the
Department of Cell and Organism Biology, in Lund, Sweden. Her research focuses on
vision and learning in hawkmoths. For her thesis work, she compared the visual and
olfactory learning abilities of the two species Deilephila elpenor and Macroglossum stell-
atarum. She is currently working with optical imaging techniques to study multimodal
processing and learning in the mushroom body of the hawkmoth Manduca sexta.
Almut Kelber is a professor of sensory biology at the Vision Group, at the Department of
Cell and Organism Biology, in Lund, Sweden. She studied biology, psychology, and elec-
tronics, at the universities of Mainz and Tbingen, Germany. She received her Ph.D. from
Tbingen University in 1993, and was a postdoc at the Research School of Biological Sci-
ences, ANU, in Canberra, Australia. Her research interests include color vision and visu-
ally guided behavior and learning abilities in animals. Address: Department of Cell and
Organism Biology, Vision Group, Lund University, Helgonavägen 3, S-22362 Lund, Swe-
den. E-mail: almut.kelber@cob.lu.se
Christian Balkenius is an associate professor at Lund University Cognitive Science
(LUCS), in Lund, Sweden. He studied mathematics, computer science, linguistics, and
psychology at Lund University where he also received his Ph.D. in cognitive science in
1995. His research goal is to understand the cognitive and developmental processes
involved in learning and perception at both neural and computational levels. The research
ranges from models of classical and instrumental conditioning to learning processes in
the control of visual attention. Address: Lund University Cognitive Science, Kungshuset,
Lundagård, S-222 22 Lund, Sweden. E-mail: christian.balkenius@lucs.lu.se
at LUND UNIVERSITY LIBRARIES on March 9, 2012adb.sagepub.comDownloaded from
