Content uploaded by Miguel A. Vadillo
Author content
All content in this area was uploaded by Miguel A. Vadillo on Oct 08, 2014
Content may be subject to copyright.
http://adb.sagepub.com/
Adaptive Behavior
http://adb.sagepub.com/content/22/3/207
The online version of this article can be found at:
DOI: 10.1177/1059712314527005
2014 22: 207 originally published online 6 May 2014Adaptive Behavior
Helena Matute, Sara Steegen and Miguel A Vadillo
Outcome probability modulates anticipatory behavior to signals that are equally reliable
Published by:
http://www.sagepublications.com
On behalf of:
International Society of Adaptive Behavior
can be found at:Adaptive BehaviorAdditional services and information for
Immediate free access via SAGE ChoiceOpen Access:
http://adb.sagepub.com/cgi/alertsEmail Alerts:
http://adb.sagepub.com/subscriptionsSubscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://adb.sagepub.com/content/22/3/207.refs.htmlCitations:
What is This?
- May 6, 2014OnlineFirst Version of Record
- May 23, 2014Version of Record >>
by guest on October 8, 2014adb.sagepub.comDownloaded from by guest on October 8, 2014adb.sagepub.comDownloaded from
Original Paper
Adaptive Behavior
2014, Vol. 22(3) 207–216
ÓThe Author(s) 2014
Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/1059712314527005
adb.sagepub.com
Outcome probability modulates
anticipatory behavior to signals
that are equally reliable
Helena Matute
1
, Sara Steegen
2
and Miguel A Vadillo
3
Abstract
A stimulus is a reliable signal of an outcome when the probability that the outcome occurs in its presence is different
from in its absence. Reliable signals of important outcomes are responsible for triggering critical anticipatory or prepara-
tory behavior, which is any form of behavior that prepares the organism to receive a biologically significant event.
Previous research has shown that humans and other animals prepare more for outcomes that occur in the presence of
highly reliable (i.e., highly contingent) signals, that is, those for which that difference is larger. However, it seems reason-
able to expect that, all other things being equal, the probability with which the outcome follows the signal should also
affect preparatory behavior. In the present experiment with humans, we used two signals. They were differentially fol-
lowed by the outcome, but they were equally (and relatively weakly) reliable. The dependent variable was preparatory
behavior in a Martians video game. Participants prepared more for the outcome (a Martians’ invasion) when the out-
come was most probable. These results indicate that the probability of the outcome can bias preparatory behavior to
occur with different intensities despite identical outcome signaling.
Keywords
Anticipatory behavior, outcome signaling, contingency, reliability, preparatory behavior, outcome probability, outcome
density, predictive cue, prediction, learning, adaptive behavior, cognitive bias
1Introduction
Learning to predict important events is critical for the
survival of any animal species. Not surprisingly, organ-
isms have evolved the ability to detect signals in the
environment that may help them in this endeavor.
However, very few of those critical events that organ-
isms need to predict are clearly signaled by reliable and
infallible signals. Most signal–outcome relationships
are probabilistic and organisms are constantly sur-
rounded by a myriad of stimuli that may or may not
signal anything. To complicate things even further,
many of those potential signals are only weakly to
moderately reliable. Thus, one of the most serious chal-
lenges organisms face in their daily life consists of accu-
rately estimating the reliability of the countless stimuli
around them, or in other words, the predictive value of
those signals. If they succeed, they will prepare for
important events in the presence of reliable signals and
not in the presence of confusing ones, which would be
a tremendous waste of energy and resources. Their
probability of survival crucially depends on this ability.
Reliable signals of important outcomes are responsi-
ble for triggering critical anticipatory or preparatory
behavior, which is any form of behavior that prepares
the organism to receive a biologically significant event.
The salivation that occurs in anticipation of food,
documented in Pavlov’s (1927) experiments, is a good
example of this kind of behavior. Another example of
anticipatory behavior is Garcia’s food-aversion learn-
ing (Garcia & Koelling, 1966), which produces antici-
patory nausea in response to flavors that signal that a
meal might be poisonous and in this way prevents the
organism from ingesting it. Importantly, anticipatory
behavior may include different response systems and
may refer to either appetitive or aversive events.
Moreover, it may involve different degrees of intention-
ality. Anticipatory behavior may consist of automatic
Pavlovian responses, such as salivation or nausea, or it
can be intentional and instrumental behavior, such as
1
Deusto University, Spain
2
University of Leuven, Belgium
3
University College London, UK
Corresponding author:
Helena Matute, Departamento de Fundamentos y Me
´todos de la
Psicologı
´a, Universidad de Deusto, Apartado 1, 48080 Bilbao, Spain.
Email: matute@deusto.es
by guest on October 8, 2014adb.sagepub.comDownloaded from
pressing a lever to avoid foot-shock (e.g., Brogden,
Lipman, & Culler, 1938; Sidman, 1953).
Ideally, anticipatory behavior should only be trig-
gered by stimuli that are reliable signals of significant
outcomes. If organisms lacked the ability to tell reliable
from unreliable signals, they would have to be con-
stantly preparing for the imminent occurrence of
potentially critical events. The cost for such strategy, in
terms of energy and stress, would make it unfeasible.
Alternatively, organisms might have adopted the strat-
egy of not preparing at all in the absence of perfectly
reliable signals. This would certainly have been lethal
as well. Thus, erring from time to time in predicting the
important events and preparing for the wrong ones
might be as lethal as not preparing at all. However,
errors will often occur in the world of uncertain infor-
mation in which we all live. Thus, natural selection
must have biased this process of detecting and respond-
ing to reliable signals in one particular direction, that
in which behavioral errors are less costly. From this, it
follows that in addition to reliability triggering antici-
patory behavior, several other variables must be affect-
ing (and thus, biasing) this process. The present
research will look at one of these variables.
Previous research has already dealt with the question
of to what degree humans and other animals are sensi-
tive to what has been called cue–outcome contingency,
that is, the reliability with which a signal predicts the
occurrence of an outcome. A widely accepted statistical
measure of contingency is Dp(Allan, 1980; Jenkins &
Ward, 1965; Rescorla, 1968), which is defined as the
difference between the probability of the outcome (O)
in the presence and the absence of a signal (S). That is,
Dp=P(O|S)2P(O|:S). If the outcome occurs with the
same probability in the presence and in the absence of
the signal, contingency is zero, and thus, the signal
holds no predictive value.
Many studies with non-human animals have shown
that contingency modulates the degree with which ani-
mals prepare for a given outcome in the presence of a
signal (e.g., Hallam, Grahame, & Miller, 1992; Murphy
& Baker, 2004; Rescorla, 1968). Animals prepare less
when the outcome is signaled by a weakly reliable sig-
nal. Thus, among other factors that are also known to
influence learning and behavior (such as saliency or
attention), all current models of animal learning and
behavior we are aware of assume that anticipatory
behavior should be guided by contingency (e.g., Stout
& Miller, 2007; Rescorla, 1968; Rescorla & Wagner,
1972; Van Hamme & Wasserman, 1994).
The potential influence of signal–outcome contin-
gency has also been central in the study of causal, pre-
dictive and contingency learning with humans. In this
area, sensitivity to signal–outcome contingency has
often been reported when using dependent variables
such as subjective estimations of causality, of contin-
gency, or of predictive value (e.g., De Houwer,
Vandorpe, & Beckers, 2007; Vadillo, Miller, & Matute,
2005; Vadillo & Matute, 2007; Wasserman, Kao, Van
Hamme, Katagari, & Young, 1996). However, systema-
tic biases of these processes have often been reported as
well. A typical experiment in this literature may present
participants with a series of fictitious patients, one per
trial. The fictitious patients are sometimes taking a pill
(S) which may or may not be followed by relief from
back-pain (O). One group of participants observes a
condition in which the probability of recovery from
back-pain is 80% regardless of whether the fictitious
patient has or has not taken the pill, that is,
P(O|S)=P(O|:S)=.80; Dp=0. Another group of par-
ticipants receives a condition in which the probability
of the outcome (i.e., recovery) is 20% regardless of
whether the patient has or has not taken the pill, that is
P(O|S)=P(O|:S)=.20. Thus, contingency is zero in
both groups but the probability of the outcome differs.
What has been observed in these cases is that the higher
the probability of the outcome, the higher the degree of
contingency that participants estimate between the sig-
nal and the outcome. That is, if contingency is held
constant, judgments of contingency and of causal rela-
tionships increase as P(O|S) increases. This is known as
the outcome–density bias and has been reported in a
wide variety of procedures, conditions, and labora-
tories (e.g., Allan & Jenkins, 1983; Allan, Siegel, &
Tangen, 2005; Alloy & Abramson, 1979; Blanco,
Matute, & Vadillo, 2013; Buehner, Cheng, & Clifford,
2003; Matute, 1995; Msetfi, Murphy, Simpson, &
Kornbrot, 2005; Musca, Vadillo, Blanco, & Matute,
2010; Shanks, Lo
´pez, Darby, & Dickinson, 1996;
Wasserman et al., 1996).
Thus, human experiments have shown that contin-
gency judgments are often biased by P(O|S). However,
it is unclear to what extent the causal and contingency
judgments collected in these latter experiments are com-
parable with the preparatory responses measured in
animal conditioning experiments. Quite interestingly,
there are some studies in which human participants are
not asked to give a judgment of contingency or a judg-
ment about the predictive value of the signal. Instead,
they are simply asked to predict the outcome or to esti-
mate how important would it be to prepare for the out-
come when the signal is present. These tasks are more
similar to the ones faced by animals in conditioning
experiments, where animals do not have to judge the
signal–outcome contingency, but just prepare for the
incoming outcomes. The results of these human experi-
ments suggest that, if anything, the influence of P(O|S)
on these judgments of prediction and of importance of
preparation is even greater than on judgments of con-
tingency and causality (De Houwer et al., 2007; Vadillo
& Matute, 2007; Vadillo et al., 2005). Nevertheless, it is
well known that what people say that they would do is
often very different from what they really do, and there
have been numerous demonstrations of dissociations
208 Adaptive Behavior 22(3)
by guest on October 8, 2014adb.sagepub.comDownloaded from
between judgments and behavior (Perales, Catena,
Shanks, & Gonza
´lez, 2005; Perruchet, Cleeremans, &
Destrebecqz, 2006). Thus, it might still be the case that,
as predicted by learning theories, preparatory behavior
in humans as in other animals is actually guided by
contingency.
The only experiment that, as far as we know, has
been conducted with humans using anticipatory beha-
vior per se as the dependent variable was conducted by
Blanco, Matute, and Vadillo (2010). They reported that
anticipatory behavior in humans was driven by contin-
gency. That study used the Martians paradigm
(Arcediano, Ortega, & Matute, 1996; see also Costa &
Boakes, 2011; Franssen, Clarysse, Beckers, van
Vooren, & Baeyens, 2010; Matute, Vadillo, & Ba
´rcena,
2007). It is a simple videogame in which the task of the
participants is to prevent Martians from landing the
Earth. There are various cues that might signal a
Martian’s invasion (the outcome) or nothing. Thus,
this task assesses how people learn to predict the inva-
sions and how they prepare for them. In other words, it
assesses anticipatory behavior directly rather than ask-
ing participants provide a judgment. The results of
Blanco et al. showed that participants actually pre-
pared more for the outcome in the presence of a reli-
able than unreliable signals. However, because the
probability of the outcome was held constant in that
experiment, it provides no hints on how this variable
might have biased behavior. In the current experiment,
we will explore whether the probability of the outcome
may bias non-verbal anticipatory behavior when con-
tingency is kept constant.
2Method
2.1 Participants and ethical statement
Thirty-six students from Deusto University volunteered
to take part in the experiment and were paid e5 for
their collaboration. Their mean age was 22.22 years old
(SD=1.514). They were run in individual cubicles con-
taining a chair, a desk and a computer.
2.2 Procedure and design
Following Blanco et al.’s (2010) experiment we used the
Martians task (Arcediano et al., 1996; Costa & Boakes,
2011; Franssen et al., 2010; Matute et al., 2007). This
allows us to assess how P(O|S) affects the participants’
anticipatory behavior directly. In this task, the goal of
the participant is to prevent Martian spaceships from
landing the earth. Every 250 ms a Martian spaceship
appears on the screen and participants can destroy it by
pressing the spacebar, which causes a laser gun to fire.
Participants usually develop a stable baseline behavior
of approximately four responses per second. Once this
behavior is established, the actual experiment begins.
From this moment the Martians will occasionally acti-
vate an anti-laser shield, which appears as a series of
light flashes accompanied by a distinctive sound for 500
ms. If the participant fires the laser gun during the acti-
vation of a shield, an invasion of Martian spaceships
will follow. During an invasion, the laser gun becomes
deactivated and the Martian spaceships appear at a
very high rate (one spaceship every 50 ms). The activa-
tion of the shield is the outcome the participant has to
prepare for. Given that the shield activation lasts for
only 500 ms, there is no time to cancel ongoing beha-
vior and prevent the invasion unless the activation of
the shields is predicted and ongoing behavior is sup-
pressed in anticipation of the shield’s activation. Thus,
the suppression of ongoing behavior is our dependent
variable. It indicates the degree to which anticipatory
behavior for the outcome occurs in response to a given
signal. In order to effectively anticipate the outcome,
the participant can learn to make use of signals that
appear on the screen that might predict the activation
of the shield.
A within-participants design was used. The manipu-
lation of different outcome probabilities was between
two equally (and moderately) reliable signals of the out-
come. Signals A and B were abstract visual symbols.
1
Participants were told that these symbols were commu-
nication signals intercepted between Martian space-
ships. The two Martian signs differed in their shape and
colors, and were counterbalanced in their role as Signal
A and Signal B. They appeared on the screen for 1500
ms each time and could or could not be followed by the
outcome (the activation of the shield). As explained
below, the probability of the outcome given each signal
was different, although the signal–outcome contingency
was the same for both.
In line with earlier research (e.g., Alloy &
Abramson, 1979; Msetfi et al., 2005; Valle
´e-
Tourangeau, Baker, & Mercier, 1994), and because in
this experiment the two conditions were manipulated
within subjects, we used two different tones to mark
the different trial types belonging to each condition.
Note that this is particularly important in trials in
which neither signal is present, which could be in prin-
ciple computed for any of the two conditions, so they
need to be clearly separated. A low and a high tone
were counterbalanced in their role as Context 1 and
Context 2. Therefore, there were two different commu-
nication signals, A and B, which always occurred in dif-
ferent contexts and were characterized by different
background tones. There were a total of 80 training
trials. Of these, 40 trials occurred in Context 1. They
consisted of the presence or the absence of target
Signal A, and comprised the low outcome probability
condition. The other 40 trials occurred in Context 2
and consisted of the presence or the absence of Signal
B, and comprised the high outcome condition. The
tones lasted for 1500 ms, so whenever signals A and B
Matute et al. 209
by guest on October 8, 2014adb.sagepub.comDownloaded from
were presented they occurred simultaneously with their
respective contexts. For programming purposes, the 80
trials were divided in 10 blocks of eight trials each, so
that all different trial types could be presented in ran-
dom order within each block. The purpose of this pro-
gramming strategy was to avoid concentration of a
certain type of trial either at the initial or final part of
the training sequence. The 10 blocks of trials were pre-
sented without interruption. The inter-trial intervals
lasted between 6500 and 11,500 ms. During that time
participants were pressing the space bar for Martians
on a black (and silent) background.
Table 1 shows the number of trials for both condi-
tions. As can be seen, the probability that the outcome
followed the signal was .50 for Signal A and .90 for
Signal B, which was the one for which we expected
more preparatory behavior. Both signals had the same
contingency with the outcome, with Dpheld constant at
.40. Thus, the two of them were equally (and moder-
ately) reliable. It is important to note that the probabil-
ity of the outcome cannot be manipulated without
confounding the overall number of trials and the num-
ber of signal–outcome pairings in each condition. As
can be seen in Table 1, in the present experiment we
chose to keep the overall number of trials constant
across conditions. This is standard practice in experi-
ments that explore the effect of the probability of the
outcome, among other things because it is known that
outcome–probability effects are dramatically influenced
by the length of the training phase (Shanks et al., 1996).
Given that the target behavior consists of suppres-
sing ongoing behavior, a suppression ratio (SR) needs
to be calculated so that the number of responses (i.e.,
in this case, presses of the space bar) that occur during
the presentation of the signal can be compared
to the base rate of responding (Arcediano et al., 1996).
The dependent variable is thus computed by dividing
the number of responses during the signal (R
signal
)
by the number of responses during the signal plus the
number of responses during an equivalent time period
preceding the signal during the intertrial interval (R
pre
):
that is, SR=R
signal
/(R
signal
+R
pre
). If this ratio has a
value of 0.5, this means that the participant shows the
same amount of bar-pressing behavior during and just
before the signal presentation, and therefore he or she
is not preparing for the outcome. A suppression ratio
of 0, on the other hand, reflects that the participant
completely suppresses bar-pressing behavior during the
signal presentation, and thus he or she is strongly pre-
paring for the outcome.
3Results
Figure 1 shows the suppression ratios for Signals A and
B and for Contexts 1 and 2 across training. Because
trials at the end of training are the most likely to yield
asymptotic results, we compared the mean suppression
ratios of the last trial with each signal. Behavioral sup-
pression was stronger when B, the signal that was asso-
ciated to a greater probability of the outcome, was
present, than when A, which was associated to a lower
probability of the outcome, was present. A repeated
measures t-test on the mean suppression ratios during
these trials with signal (A vs. B) confirmed that, as
expected, participants suppressed their behavior more
for B (M=0.315, SD=0.078) than for A (M=0.356,
SD=0.116), t(35)=2.503, p\.05, d
z
=0.41. Therefore,
although both signals were equally weak predictors of
the outcome, the signal holding the highest P(O|S)
influenced behavior by yielding more preparation for
the outcome under that condition.
Even though the use of trial markers, or contexts, is
standard practice and should not be eliciting prepara-
tory behavior (because of the way they are described in
the instructions and the way they are used), it could be
argued that they might not be as neutral as we intended.
Indeed, it is possible they became the only informative
cues in this experiment because the outcome occurred
with greater frequency in Context 2 than in Context 1.
Figure 1 shows that this was not the case. As expected,
participants prepared for the outcome in response to
signals but not in response to trial markers or contexts.
Nevertheless, and in order to make sure that the dif-
ferences in responding between signals A and B were
not due to any differences in responding to the trial
markers (Context 1 vs. Context 2), we also analyzed
the suppression ratios in the last trial in which the con-
texts were presented alone, that is, the latest trials in
which Signal A and Signal B were absent. A repeated
measures t-test on the mean suppression ratios during
these context-alone trials revealed that suppression
ratios did not differ depending on context (M=0.439,
SD=0.061 for Context 1; M=0.427, SD=0.091 for
Context 2); t(35)=0.704, p=.486, d
z
=0.15. Thus, the
observed differences cannot be attributed to differences
in the contextual trial markers. Preparatory behavior in
the absence of the signals was very week and did not
differ between both contexts.
Table 1. Number of trials of each type presented for each
condition.
Context 1 Context 2
O noO
A1010
noA 2 18
O noO
B182
noB 10 10
Signals A and B were two different abstract visual symbols,
counterbalanced. Signal A (low outcome condition) was always
presented in Context 1. Signal B (high outcome condition) was always
presented in Context 2. Context 1 and Context 2 were a low and a
high tone, counterbalanced, which were used to mark the different
conditions. O and noO represent the presence or absence of light
flashes, which played the role of outcomes.
210 Adaptive Behavior 22(3)
by guest on October 8, 2014adb.sagepub.comDownloaded from
4 Discussion
Behavioral experiments with human (Blanco et al.,
2010) and non-human animals (e.g., Hallam et al, 1992;
Murphy & Baker, 2004; Rescorla, 1968) had already
shown that preparatory behavior is influenced by the
reliability of the signals or, in other words, by signal–
outcome contingency. Those experiments had typically
maintained P(O|S) constant, so that contingency
between the signal and the outcome could be manipu-
lated and its effect on behavior established.
By contrast, judgmental experiments with humans
had often manipulated P(O|S) while holding contin-
gency constant. In this case, the dependent variable
had been subjective judgments of contingency rather
than preparatory behavior. The results had shown that
the higher the P(O|S), the higher the judgments. In
addition, they had also shown that it was not only the
subjective judgments of contingency that was affected
by P(O|S). Experiments in which participants were
asked to predict an outcome based on the presence of a
signal showed that these prediction judgments are influ-
enced mainly by the probability of the outcome given
the signal, not by contingency (Vadillo & Matute,
2007; Vadillo et al., 2005). Moreover, when asked
whether they would prepare for the outcome, partici-
pants judged that it was most important to prepare for
an outcome when its occurrence was most probable
(De Houwer et al., 2007). Thus, we predicted that if
contingency was kept constant, preparatory behavior
might be biased when the outcome’s frequency was
manipulated. The results showed the predicted pattern.
Although the two signals in the present experiment had
the same contingency (i.e., Dp) with the outcome, parti-
cipants prepared more for the outcome when it was
most likely to occur.
In the context of human and animal research, the
effects of P(O|S) on causal and contingency judgments
have usually been considered a bias. If the participants’
goal is to detect whether two events are covariate, their
base rate should be irrelevant. Therefore, any demon-
stration that the base rate probability of the outcome
matters in those judgments can be considered a bias.
However, the situation might be the opposite for
experiments using outcome predictions, preparatory
judgments and anticipatory responses as dependent
variables. If an outcome is very likely to happen, it
makes sense to predict it very strongly, to prepare for it
and to adapt our behavior consequently, regardless of
the reliability of the cues that signal it. In other words,
it is not obvious that the effect of P(O|S) on predictive
and preparatory judgments, and on anticipatory beha-
vior, should be considered a bias.
This does not necessarily mean that the effect of
contingency on anticipatory behavior should be consid-
ered a mistake or a bias. Indeed, the question of what
is a bias is a hard question that might not have a simple
response. It all depends on how we look at it and what
aspects we pay attention to. Preparing for an outcome
only in the presence of highly reliable signals may make
sense because it is a waste of energy to prepare for out-
comes when they do not occur, and preparing in the
Figure 1. Mean suppression ratios for Signals A and B, as well as for Contexts 1 and 2 when presented alone during the 10 blocks
of eight trials in which the different trial types were presented. When a given signal (A or B) or context alone was presented more
than once in the same block of trials the figure shows the average suppression ratio for that cue or context in that block.
Matute et al. 211
by guest on October 8, 2014adb.sagepub.comDownloaded from
face of misleading and ambiguous signals would cer-
tainly lead to many cases in which the organism pre-
pares in vain. However, many signals hold weak or just
moderately reliable relationships with the important out-
comes we need to predict for survival. Thus, there is no
doubt we will often err. We may err in our estimations
and in preparing for outcomes when they will not actu-
ally occur; we will err in not preparing for critical out-
comes that will occur; and given that there will be many
errors and some may be lethal, evolution must have
favored the least costly error. On balance, not preparing
for a predator, or for a prey, is certainly more costly
than preparing in excess. Thus, any variable that, when
in doubt, makes the organism prepare in excess, has
probably developed the power to bias both perception
and behavior toward this direction of overestimation of
contingencies and over-preparing for the outcome.
Of course, it is not without problems to state that an
evolutionary framework can predict the results
observed in the present research. In a sense, anything
to happen should be a consequence of evolution. As
such, and according to some, evolution cannot be an
explanation, because it could serve to account for
whatever result is observed (Gould & Lewontin, 1979;
Plotkin, 2004). However, we find that sometimes it is
useful to think in generic evolutionary terms in addi-
tion to thinking on more focused predictions and on
previous data, which might help discriminate among
several more specific hypotheses. We believe the pres-
ent line of research is one of those cases that should
benefit from such combined approach.
In any case, and aside from the necessary speculative
nature of the discussion on whether it constitutes a bias
or an efficient strategy, our results are important in
that they show that non-verbal preparatory behavior in
humans does deviate from contingency, at least under
certain conditions. As we already noted, a shared prin-
ciple among theorists of learning has been that beha-
vior is strongly guided by contingency. Thus, it has
been typically assumed by most textbooks (though
often implicitly) that the probability with which the
outcome occurs should not significantly affect behavior
(e.g., Bouton, 2007; Domjan, 2003). The main contri-
bution of our experiment has been to uncover the effect
of the probability of the outcome on human prepara-
tory behavior. We can state that not only contingency
but also the probability of the outcome has an effect on
preparatory behavior, at least in humans. It seems that,
when preparing for an outcome, people, and possibly
other animals as well, tend to allocate resources as a
function of both, the reliability of the signals (i.e., con-
tingency) and the probability that the outcome will fol-
low. This probably minimizes the potential errors that
can take place in a world of high uncertainty, or at
least biases behavior toward the least costly errors.
That is, assuming two signals of similar reliability, it
makes sense to prepare more in the presence of the sig-
nal that is most frequently followed by the outcome.
There are some results in the literature that seem
contrary to the present ones, at least at first glance.
Those studies suggest that learning is faster when
P(O|S) is low than when P(O|S) is high, along the same
level of contingency (Dp). Wasserman, Elek, Chatlosh,
and Baker (1993) explored different levels of contin-
gency in a causal judgment task with humans manipu-
lating different values of P(O|S) and P(O|:S). They
compared, for instance, a condition in which P(O|S)
was 0.25 and P(O|:S) was 0 to a condition in which
P(O|S) was 1 and P(O|:S) was 0.75. These two cases
involve a contingency of 0.25. Wasserman and his col-
leagues (1993) observed that contingency judgments
were more accurate in the case in which the probabil-
ities were lower. This might seem to contradict our
results. However, it should be noted, first, that the sig-
nal Swas not an external signal, as in the present
research, but an action of the participant. Second, the
dependent variable was a contingency judgment, which
might also be a source for differences. And third, that
what Wasserman et al. (1993) found was that partici-
pants were more accurate when the probabilities were
lower. This means that their participants showed an
overestimation of contingency when the probabilities
where higher. This is perfectly compatible with our
finding that participants prepared more for the out-
come when the probability of the outcome was higher.
Hallam et al. (1992) showed a similar result with
rats. In Experiment 1, carried out with a licking sup-
pression paradigm, they measured the level of prepara-
tory behavior when varying P(O|S) and keeping
P(O|:S) equal to zero. They found that preparatory
behavior did not increase linearly with changes in Dp.
They observed a larger increment between 0.0 and 0.25
than between 0.50, 0.75 and 1. Note, however, that
these manipulations involve variations in Dp, from 0 to
0.25, 0.50, 0.75, 1. In our experiment, Dpwas kept con-
stant at 0.4, so Hallam et al.’s result might not be as
related to ours as it might seem at first. Similarly,
Morı
´s, Carnero, and Loy (2012) showed that rats can
anticipate the arrival of food to the magazine device
using an auditory stimulus when P(O|S) equals 0.1 and
P(O|:S) equals 0, but not when P(O|S) is 0.9 and
P(O|:S) is 0.8.
The differences between those two studies and the
present one are so many that it is difficult to conclude
what the crucial variables might be. For instance, in
Morı
´s et al.’s study, the fact that in one condition
P(O|:S) equals 0 adds information to the experimental
subjects by showing that the context-alone trials predict
no food and therefore the signal is the only possible
predictor for food. This does not occur in the other
condition, in which, because the two probabilities are
very high (0.9 and 0.8), food occurs at almost any trial
212 Adaptive Behavior 22(3)
by guest on October 8, 2014adb.sagepub.comDownloaded from
regardless of whether the signal is present or absent,
which means that the signal is not particularly relevant.
To sum up, we tested preparatory behavior in an
instrumental avoidance task and have shown that it
does not occur only in response to reliable signals.
Instead, relatively weakly reliable signals can trigger
preparatory avoidance behavior, and this behavior is
enhanced, as the outcome is more likely to follow them.
We used humans as subjects but given the enormous
similarities between humans and other species in their
detection of contingencies and conditioned behavior
(for reviews see Escobar & Miller, 2012; Lo
´pez &
Shanks, 2008; Miller & Matute, 1996; Wasserman et
al., 1996) we assume that the results can possibly be
extended to other animals as well. An interesting ques-
tion for further research would be whether this finding
can be extended to other, more autonomic response
systems, such as salivation, nausea or immune reac-
tions, to mention just a few. Do they occur only in
response to highly reliable signals or are they also
biased by the probability of the outcome? In principle,
there are no grounds to suspect differential influences
of these variables on different response systems or spe-
cies. Our preparatory behavior results in the present
experiment, along with the previously mentioned find-
ings in the human judgmental literature, suggest that
these more autonomous response systems could possi-
bly be also biased by signals that are frequently fol-
lowed by significant outcomes.
Acknowledgments
We thank Fernando Blanco, Jan de Houwer, Ion Yarritu and
three anonymous reviewers for very helpful comments and
suggestions on previous versions of this article.
Funding
Support for this research was provided by Direccio
´nGeneral
de Investigacio
´n of the Spanish Government (Grant PSI2011-
26965) and Departamento de Educacio
´n, Universidades e
Investigacio
´n of the Basque Government (Grant IT363-10).
The funders had no role in study design, data collection and
analysis, decision to publish, or preparation of the manuscript.
Note
1. We are using Arcediano et al.’s (1996) Martians task, but
the general layout and visual aspect of the Martians,
explosions, and outcome, is that of Franssen et al.’s
(2010) version, while the two signals are from Costa and
Boakes’ (2011) version. We wish to thank them for shar-
ing their programs with us.
References
Allan, L. G. (1980). A note on measurements of contingency
between two binary variables in judgment tasks. Bulletin of
the Psychonomic Society,15, 147–149.
Allan, L.G., & Jenkins, H. M. (1983). The effect of represen-
tations of binary variables on judgment of influence.
Learning and Motivation,14, 381–405.
Allan, L. G., Siegel, S., & Tangen, J. M. (2005). A signal
detection analysis of contingency data. Learning & Beha-
vior,33, 250–263.
Alloy, L. B., & Abramson, L. Y. (1979). Judgments of contin-
gency in depressed and nondepressed students: Sadder but
wiser? Journal of Experimental Psychology: General,108,
441–485.
Arcediano, F., Ortega, N., & Matute, H. (1996). A beha-
vioural preparation for the study of human Pavlovian con-
ditioning. Quarterly Journal of Experimental Psychology,
49B, 270–283.
Blanco, F., Matute, H., & Vadillo, M. A. (2010). Contingency
is used to prepare for outcomes: Implications for a func-
tional analysis of learning. Psychonomic Bulletin & Review,
17, 117–121.
Blanco, F., Matute, H., & Vadillo, M. A. (2013). Interactive
effects of the probability of the cue and the probability of
the outcome on the overestimation of null contingency.
Learning & Behavior. Advance online publication. DOI:
10.3758/s13420-013-0108-8
Bouton, M. E. (2007). Learning and behavior: A contemporary
synthesis. Sunderland, MA: Sinauer Associates, Inc.
Brogden, W. J., Lipman, E. A., & Culler, E. (1938). The role
of incentive in conditioning and extinction. American Jour-
nal of Psychology,51, 109–117.
Buehner, M. J., Cheng, P. W., & Clifford, D. (2003). From
covariation to causation: A test of the assumption of cau-
sal power. Journal of Experimental Psychology: Learning,
Memory, and Cognition,29, 1119–1140.
Costa, D. S. J., & Boakes, R. A. (2011). Varying temporal
contiguity and interference in a human avoidance task.
Journal of Experimental Psychology: Animal Behavior Pro-
cesses,37, 71–78.
De Houwer, J., Vandorpe, S., & Beckers, T. (2007). Statistical
contingency has a different impact on preparation judg-
ments than on causal judgments. Quarterly Journal of
Experimental Psychology,60, 418–432.
Domjan, M. (2003). Principles of learning and behavior (5th
edition). Belmont, CA: Thomson/Wadsworth.
Escobar, M., & Miller, R.R. (2012). Associative accounts of
causal judgments. In T.R. Zentall & E.A. Wasserman
(Eds.), The Oxford handbook of comparative cognition (pp.
157–174). New York, NY: Oxford University Press.
Franssen, M., Clarysse, J., Beckers, T., van Vooren, P., &
Baeyens, F. (2010). A free software package for a human
online-conditioned suppression preparation. Behavior
Research Methods,42, 311–317.
Garcia, J., & Koelling, R.A. (1966). Relation of cue to conse-
quence in avoidance learning. Psychonomic Science,4,
123–124.
Gould, S. J., & Lewontin, S. C. (1979). The spandrels of San
Marco and the panglossian paradigm: A critique of
the adaptationist programme. Proceedings of the Royal
Society B: Biological Sciences,205(1161), 581. doi:
10.1098/rspb.1979.0086
Hallam, S. C., Grahame, N. J., & Miller, R. R. (1992).
Exploring the edges of Pavlovian contingency space: An
assessment of contingency theory and its various metrics.
Learning and Motivation,23, 225–249.
Matute et al. 213
by guest on October 8, 2014adb.sagepub.comDownloaded from
Jenkins, H. M., & Ward, W. C. (1965). Judgment of contin-
gency between responses and outcomes. Psychological
Monographs,79, 1–17.
Lo
´pez, F.J., & Shanks, D.R. (2008). Models of animal learn-
ing and their relations to human learning. In R. Sun (Ed.),
Handbook of computational cognitive modelling (pp. 589–
611). Cambridge, MA: Cambridge University Press.
Matute, H. (1995). Human reactions to uncontrollable out-
comes: Further evidence for superstitions rather than help-
lessness. Quarterly Journal of Experimental Psychology,
48B, 142–157.
Matute, H., Vadillo, M. A., & Ba
´rcena, R. (2007). Web-based
experiment control software for research and teaching
on human learning. Behavior Research Methods,39,
689–693.
Miller, R. R., & Matute, H. (1996). Animal analogues of cau-
sal judgment. In D.R. Shanks, K. J. Holyoak, & D.
L. Medin (Eds.), The psychology of learning and motiva-
tion, Vol. 34: Causal learning (pp. 133–166). San Diego,
CA: Academic Press.
Morı
´s, J., Carnero, S., & Loy, I. (2012). A test of Rescorla
and Wagner’s prediction of non linear effects in contin-
gency learning. Learning & Behavior,40, 507–519.
Msetfi, R. M., Murphy, R. A., Simpson, J., & Kornbrot, D.
E. (2005). Depressive realism and outcome density bias in
contingency judgments: The effect of the context and inter-
trial interval. Journal of Experimental Psychology: General,
134, 10–22.
Murphy, R. A., & Baker, A. G. (2004). A role for CS–US
contingency in Pavlovian conditioning. Journal of Experi-
mental Psychology: Animal Behavior Processes,30,
229–239.
Musca, S. C., Vadillo, M. A., Blanco, F., & Matute, H.
(2010). The role of cue information in the outcome–
density effect: Evidence from neural network simulations
and a causal learning experiment. Connection Science,22,
177–192.
Pavlov, I.P. (1927). Conditioned reflexes. London: Clarendon
Press.
Perales, J. C., Catena, A., Shanks, D. R., & Gonza
´lez, J. A.
(2005). Dissociation between judgments and outcome
expectancy measures in covariation learning: A signal
detection theory approach. Journal of Experimental Psy-
chology: Learning, Memory, and Cognition,31, 1105–1120.
Perruchet, P., Cleeremans, A., & Destrebecqz, A. (2006) Dis-
sociating the effects of automatic activation and explicit
expectancy on reaction times in a simple associative learn-
ing task. Journal of Experimental Psychology: Learning,
Memory, and Cognition,32, 955–965.
Plotkin, H. C. (2004). Evolutionary thought in psychology: A
brief history. Malden, MA: Wiley-Blackwell.
Rescorla, R.A. (1968). Probability of shock in the presence
and absence of CS in fear conditioning. Journal of Com-
parative and Physiological Psychology,66, 1–5.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlo-
vian conditioning: Variations in the effectiveness of rein-
forcement and nonreinforcement. In A. H. Black & W.
F. Prokasy (Eds.), Classical conditioning II: Current
research and theory (pp. 64–99). New York: Appleton-
Century-Crofts.
Shanks, D. R., Lo
´pez, F. J., Darby, R. J., & Dickinson, A.
(1996). Distinguishing associative and probabilistic con-
trast theories of human contingency judgment. In D.
R. Shanks, K. J. Holyoak, & D. L. Medin (Eds.), The psy-
chology of learning and motivation, Vol. 34: Causal learning
(pp. 265–311). San Diego, CA: Academic Press.
Sidman, M. (1953). Avoidance conditioning with brief shock
and no exteroceptive warning signal. Science,118(3058),
157–158.
Stout, S. C., & Miller, R. R. (2007). Sometimes-competing
retrieval (SOCR): A formalization of the comparator
hypothesis. Psychological Review,114, 759–783.
Vadillo, M. A., & Matute, H. (2007). Predictions and causal
estimations are not supported by the same associative
structure. Quarterly Journal of Experimental Psychology,
60, 433–447.
Vadillo, M. A., Miller, R. R., & Matute, H. (2005). Causal
and predictive-value judgments, but not predictions, are
based on cue–outcome contingency. Learning & Behavior,
33, 172–183.
Valle
´e-Tourangeau, F., Baker, A. G., & Mercier, P. (1994).
Discounting in causality and covariation judgements.
Quarterly Journal of Experimental Psychology, 47B,
151–171.
Van Hamme, L.J., & Wasserman, E.A. (1994). Cue competi-
tion in causality judgments: The role of nonpresentation of
compound stimulus elements. Learning and Motivation,25,
127–151
Wasserman, E. A., Elek, S. M., Chatlosh, D. L., & Baker, A.
G. (1993). Rating causal relations: Role of probability in
judgments of response–outcome contingency. Journal of
Experimental Psychology: Learning, Memory, and Cogni-
tion,19, 174–188.
Wasserman, E. A., Kao, S.-F., Van Hamme, L. J., Katagari,
M., & Young, M. E. (1996). Causation and association. In
D. R. Shanks, K. J. Holyoak, & D. L. Medin (Eds.), The
psychology of learning and motivation, Vol. 34: Causal
learning (pp. 207–264). San Diego, CA: Academic Press.
About the Authors
Helena Matute is a professor of psychology at the University of Deusto (Bilbao, Spain), where
she is the director of the experimental psychology laboratory. Her research is concerned with
human and animal associative learning, causal and predictive learning, contingency judgments,
cognitive biases, causal illusions and internet-based research. Her work has been published in
the main journals of her area of expertise, including among others the Journal of Experimental
Psychology: General; Journal of Experimental Psychology: Learning, Memory and Cognition;
The Quarterly Journal of Experimental Psychology; and Psychological Science.
214 Adaptive Behavior 22(3)
by guest on October 8, 2014adb.sagepub.comDownloaded from
Sara Steegen is now a graduate student at the University of Leuven (Belgium) at the research
group quantitative psychology and individual differences. Her research is concerned with cogni-
tive models and model selection. She ran an internship at the Experimental Psychology
Laboratory at the University of Deusto (Bilbao, Spain) about contingency learning and pre-
paratory behavior.
Miguel A Vadillo is research associate at University College London. His current research
explores how associative learning mechanisms contribute to the guidance of attention in visual
search. He has also conducted research on causal learning, causal reasoning, interference,
retrieval-induced forgetting and internet-based research. His work has been published in the
main journals of his area of expertise, including among others Journal of Experimental
Psychology: General; Journal of Experimental Psychology: Learning, Memory, and Cognition;
and Psychonomic Bulletin & Review.
Appendix 1. Instructions
Initial instructions
In this experiment, your goal is to prevent Martians
from landing on our planet. Every 250 milliseconds one
new Martian spaceship will try to land. You will watch
them appear one by one in rows on the screen. In order
to destroy them, you must use your laser-gun, firing by
pressing the spacebar, but you must do it before they
can see you (that is, just before you can see them). Do
not shoot too early, however, because you only have
one shot per Martian spaceship. You do not need to
aim your gun; the only thing that matters is to fire at
the right moment (pressing the space bar). If you hit a
Martian spaceship, you will see an explosion appearing
where a Martian would have appeared otherwise. At
the end of this training phase, we will tell you the per-
centage of Martian spaceships you destroyed.
As soon as you click the mouse, the Martians will
immediately start trying to land, thus you must start
firing.
The planet depends on you!! Do not allow them to
land!!
[Participants are exposed to a demonstration phase
of 25 seconds where they have to shoot at the Martians,
by quickly pressing the spacebar.]
Instructions after demonstration
Now the Martians have developed a powerful anti-laser
shield that they will activate from time to time. You
must continue to use your laser-gun to prevent their
landing. BUT BE CAREFUL, because, if you shoot
your laser-gun when the anti-laser shield is connected,
your shot will reflect back to you, and then thousands of
Martians will land safely immediately, and you will not
be able to stop that invasion. You will know that the
anti-laser shield is connected when you see a white
intermittent flashing on the computer screen; also, you’ll
clearly hear the sound produced by the magnetic field.
Remember well: just a single shot as soon as the
anti-laser shield is connected (WHITE FLASING
SCREEN & MAGNETIC FIELD SOUND), and the
Martians will make a successful invasion!!
Now you will see it .
[Participants are presented to another demonstration
phase, where they learn how the shield and the invasion
work.]
Final instructions
As you may have deduced, it will be really useful to
anticipate the anti-laser shield activation, thus you
would be able to stop firing your laser-gun just before
the shield is activated.
To activate the anti-laser shield, the Martian space-
ships need to communicate with each other. They com-
municate with different signs that are ALWAYS paired
with a beep sound. In the Earth command post, we can
always intercept this beep sound whenever they com-
municate, but we can only see some of the signs that are
paired with the beep.
So every time when the Martians communicate, you
will hear the beep sound and some of the times, if we
can intercept the sign, you will see the sign on the screen
as well. If you only hear the beep sound, this means that
the Martians are communicating with a sign that we
cannot intercept.
Unfortunately, we cannot decode the content of the
signs we intercept, and thus we cannot know whether
they have anything to do with the anti-laser shield acti-
vation or with anything else.
To sum up: an intercepted communication, which
consists of a beep sound and sometimes a beep sound
with a signal, may appear just before the anti-laser
shield is activated. However, not all the times we inter-
cept communications, the anti-laser shield is activated.
Matute et al. 215
by guest on October 8, 2014adb.sagepub.comDownloaded from
Thus, it is possible that sometimes a communication
will NOT be followed by the activation of the anti-laser
shield, thus becoming a false alarm.
If you learn how to decode the signals properly, you
will be able to stop firing just before the Martians
activate the anti-laser shield, at least most of the times.
Otherwise, each time the Martians activate the shield
you will be firing and thus you will not be able to avoid
an invasion of hundreds of Martians .
216 Adaptive Behavior 22(3)
by guest on October 8, 2014adb.sagepub.comDownloaded from
