The partial reinforcement extinction effect (PREE) in female Roman high- (RHA-I) and low-avoidance (RLA-I) rats.
ABSTRACT The present experiment was designed with the goal of studying the partial reinforcement extinction effect (PREE) in female inbred Roman high- (RHA-I) and low-avoidance (RLA-I) rats. Two groups of RHA-I and two of RLA-I food-deprived animals were placed in a straight alley where they were partially or continuously reinforced. Once the animals reached the acquisition criterion, they were exposed to an extinction phase where the reinforcement was omitted. During the extinction phase RHA-I animals continuously reinforced during acquisition exhibited more resistance to extinction than their RLA-I counterparts, whereas only RLA-I rats partially reinforced during acquisition showed an increased resistance to extinction in comparison to continuously reinforced control RLA-I rats, this PREE being absent in RHA-I animals. These results are discussed within the framework of PREE theories that account for this effect by using emotional mechanisms, as pertains to the repeatedly observed RHA-RLA differences in emotional reactivity.
- SourceAvailable from: Rocio Donaire[Show abstract] [Hide abstract]
ABSTRACT: Frustration is an emotional response that can be induced by the sudden devaluation of a reinforcer in the presence of greater reinforcement expectancies (e.g. instrumental successive negative contrast, iSNC). This emotional response seems to be similar to anxiety and can be attenuated by previous experiences of reward loss (e.g. partial reinforcement, PR, as opposed to continuous reinforcement, CR). In this study we used iSNC and PR procedures in order to compare the performance of two strains of rats psychogenetically selected on the basis of their emotional reactivity: the inbred Roman High- (RHA-I, low anxiety) and Low- (RLA-I, high anxiety) Avoidance rats. Animals were exposed to a straight alley, where they were changed from 12 pellets in the preshift phase (presented in 100% of trials-CR vs. 50% of trials-PR) to 2 pellets in the postshift phase, or exposed to 2 pellets throughout the training. The results indicated that the iSNC only appeared in RLA-I rats exposed to CR, as opposed to RLA-I animals exposed to PR and to RHA-I rats exposed to PR or CR. These data seem to support the implication of emotional responses in both iSNC and PR situations, and indicate that the behavioral reactivity to reward loss experiences is modulated by genetic variables.Physiology & Behavior 12/2011; 105(5):1112-6. · 3.16 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Two recent microarray and qRT-PCR studies showed that inbred Roman High- (RHA-I, low anxiety and frustration vulnerability) and Low-Avoidance (RLA-I, high anxiety and frustration vulnerability) rats, psychogenetically selected on the basis of their divergence in two-way avoidance performance, differed in basal whole-brain and hippocampal expression of genes related to neurotransmission, emotion, stress, aversive learning, and drug seeking behavior. We have extended these studies by analyzing strain differences in hippocampal gene expression following a frustrative experience involving reward downshift, i.e. instrumental successive negative contrast (iSNC), a phenomenon in which the sudden reduction of an expected reward induces frustration/anxiety. Food-deprived male Roman rats were exposed to a reduction in the amount of solid food presented in the goal of a straight alley (from 12 pellets in "training" trials-i.e. preshift trials- to 2 pellets in "frustration testing" trials-i.e. postshift trials-). The iSNC effect, as measured by response latencies in the "postshift" trials, appeared only in RLA- I rats (i.e. higher response latencies in the 12-2 RLA-I group as compared to the 2-2 RLA-I control group in postshift trials). Two and a half hours after the "postshift" behavioral test, hippocampi were removed and stored (-80°C) until analysis. Microarray analysis of these hippocampi showed that four differentially-expressed, and qRT-PCR-validated genes (TAAR2, THAP1, PKD2L1, NANOS), have relevance for brain function and behavior, including schizophrenia, depression, anxiety, and drug addiction, thus showing the usefulness of Roman strains as a genetic model for research on the neurogenetic basis of frustration.Behavioural brain research 09/2013; · 3.22 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Abstract One-way avoidance learning: Differences between genetically selected males and females: The Roman high-(RHA) and low-(RLA) avoidance rats were selected, respectively, for good vs. poor acquisition of two-way active avoidance, these behavioral differences being modulated by sex and environmental influences. In this study, inbred male and female Roman rats were exposed to a one-way avoidance task in which the time spent in the safe compartment was manipulated in two phases. In the pre-shift phase, animals were exposed to 30 s vs. 1 s in safety. In the post-shift phase, the time in safety was devaluated for Groups 30-1 (successive negative contrast). The results showed that, in both phases, strain differences were observed only in female rats, the RLA-I strain being poorer than the RHA-I strain. The present data show an interaction among genes, sex, and time in safety that influences one-way avoidance behavior.Revista Argentina de Ciencias del Comportamiento. 03/2013; 5(nº 2):40-46.
Behavioural Brain Research 194 (2008) 187–192
Contents lists available at ScienceDirect
Behavioural Brain Research
journal homepage: www.elsevier.com/locate/bbr
The partial reinforcement extinction effect (PREE) in female
Roman high- (RHA-I) and low-avoidance (RLA-I) rats
MaJosé Gómeza, Lourdes de la Torrea, José Enrique Callejas-Aguileraa,
José Manuel Lerma-Cabrerab, Juan M. Rosasa, MaDolores Escarabajala,
Ángeles Agüeroa, Adolf Tobe˜ nac, Alberto Fernández-Teruelc, Carmen Torresa,∗
aDepartment of Psychology, University of Jaén, Paraje de Las Lagunillas s/n, Edif. D-2, 23071 Jaén, Spain
bDepartment of Neurociencia y Ciencias de la Salud, University of Almería, 04120 Almería, Spain
cMedical Psychology Unit, Department of Psychiatry and Forensic Medicine, Institute of Neurosciences, School of Medicine,
Autonomous University of Barcelona, 08193 Bellaterra, Barcelona, Spain
a r t i c l ei n f o
Received 7 May 2008
Received in revised form 30 June 2008
Accepted 10 July 2008
Available online 18 July 2008
a b s t r a c t
of RLA-I food-deprived animals were placed in a straight alley where they were partially or continuously
reinforced. Once the animals reached the acquisition criterion, they were exposed to an extinction phase
where the reinforcement was omitted. During the extinction phase RHA-I animals continuously rein-
forced during acquisition exhibited more resistance to extinction than their RLA-I counterparts, whereas
only RLA-I rats partially reinforced during acquisition showed an increased resistance to extinction in
comparison to continuously reinforced control RLA-I rats, this PREE being absent in RHA-I animals. These
mechanisms, as pertains to the repeatedly observed RHA–RLA differences in emotional reactivity.
© 2008 Elsevier B.V. All rights reserved.
One of the most important organisms’ adaptive capabilities
when coping with changing environments is to adjust to unex-
pected omissions of reward by changing its previous response. This
adaptive function facilitates a switch from previously successful
behaviors that are no longer effective, to new responses that may
less of the selected behavior (ceasing the current behavior, trying
a novel response, continuing the current behavior or moving to
a new location and trying the behavior there), it has been shown
are partially responsible for the overt behavior . In this regard,
experiences of loss constitute one of the major sources of distress
in human subjects, interfering with the normal autonomic, physi-
ological, immunological and psychological functioning (for review
∗Corresponding author. Tel.: +34 953 21 22 92; fax: +34 953 21 18 81.
E-mail address: firstname.lastname@example.org (C. Torres).
Several experimental procedures enable us to systematically
study the emotions associated with aversive nonreward in the lab-
oratory. The term nonreward refers to the surprising omission,
reduction in magnitude, or quality degradation of an expected
appetitive reinforcer . Aversive reward reduction paradigms
include extinction, the partial reinforcement extinction effect
(PREE), the magnitude of reinforcement extinction effect (MREE)
and the consummatory and instrumental successive negative con-
trast (SNC) effect, these paradigms being often conceptualized as
. All of these phenomena share a violation of the expectancies
of reward, triggering an aversive emotional state that is function-
such as painful stimuli or novelty, as suggested by the results of
several experimental lines (for review see [23,30,39]). Animals
show similar responses when they are exposed to stimuli paired
to either surprising nonreward or innate fear events, including
escape , aggression , potentiated startle reflex  or jump-
ing . An additional source of evidence is provided by the effects
of GABAergic anxiolytic drugs in reward reduction/omission sit-
uations. Benzodiazepines, barbiturates and ethanol attenuate the
0166-4328/$ – see front matter © 2008 Elsevier B.V. All rights reserved.
M.J. Gómez et al. / Behavioural Brain Research 194 (2008) 187–192
SNC effect, increase resistance to extinction, and abolish the PREE,
these results being explained on the basis of the antifrustrative
effects of these drugs (for review see [13,29,33,47]). Additionally,
it has been found that exposure to incentive downshift activates
the hypothalamic–pituitary–adrenal (HPA) axis, increasing plasma
levels of stress-related hormones such as ACTH and corticosterone
[36,41]. Brain studies indicate that the lesion of structures related
to emotion and motivation (such as the hippocampus, the amyg-
dala or the nucleus accumbens) interfere with the occurrence of
reward loss phenomena such as the instrumental and consumma-
tory SNC effects [7,24,34]. Finally, it has been recently observed
that experience of reward loss in humans is correlated with an
increased activity in brain structures involved in physical pain
hedonic processing, such as the insular, prefrontal and cingulate
A different perspective to further prove the implication of neg-
ative emotional processes on reward loss situations involves the
study of performance differences among strains of animals genet-
ically selected on the basis of emotional reactivity. Examples of
strains of rats developed through this psychogenetic selection pro-
and Roman high-avoidance (RHA/Verh) rats. Using a stock from the
original RHA and RLA rats developed by Broadhurst and Bignami
, the outbred Swiss sub-lines have been genetically selected on
the basis of their rapid vs. extremely poor acquisition of a two-way
active avoidance behavior in the shuttlebox , but some of them
also being continued as inbred starting in 1993 at the Autonomous
University of Barcelona (designated as Roman high- (RHA-I) and
low-avoidance (RLA-I) rats; [15,17,18,20,31]). A large body of evi-
dence shows that, as a consequence of this selection, RLA/Verh rats
are characterized by more pronounced, anxious reactions to novel,
conflict and threatening situations, less novelty-seeking behaviors,
and a blunted response to addictive drugs when they are com-
pared with their RHA/Verh rats counterparts (see [26,44,45]). This
evidence suggests that the Roman rats constitute a valid experi-
mental approach to explore the genetic basis of emotions induced
by stressful and anxiety-provoking events.
Recent research in our laboratory has analyzed the behavior
of these strains of animals when they are exposed to reward loss
experiences as the SNC or the extinction of a previously learned
response. SNC is defined as a temporary reduction in respond-
ing to a smaller reward by animals previously exposed to a larger
reward, compared to the response observed in a control group that
is always exposed to the smaller reward . Aversive instrumen-
tal SNC effect has been found to be greater in female inbred RLA-I
rats as compared to RHA-I animals when the time of exposure to
the reinforcing cues associated to the safe compartment was sud-
denly reduced from 30s to 1s in a one-way avoidance task .
Similar RHA–RLA differences were obtained by Rosas et al. 
in an appetitive instrumental SNC situation induced by surpris-
ingly reducing the amount of solid food received by animals in the
goal of a straight alley (from 12 pellets to 2 pellets). The RHA–RLA
differences exhibited in instrumental SNC paradigms have been
also observed during the extinction of an appetitive instrumental
response such as the runway behavior, the RHA-I rats showing a
greater resistance to extinction in comparison to RLA-I rats [27,28].
Although the explanation of the obtained results in terms of strain
differences in emotional reactivity to surprising nonreward seems
to be plausible, additional studies that investigate the performance
of RHA-I and RLA rats-I in alternative reward loss tests are needed
in order to further assess the utility of these strain of animals for
the study of incentive downshift related emotions.
The PREE is defined as an increased resistance to extinction that
is observed after training with partial reinforcement as compared
to continuous reinforcement . This paradoxical learning effect
constitutes an experimental example of response-outcome uncer-
tainty during acquisition that results in an increased, rather than
in a decreased, behavioural persistence during extinction . The
PREE has been shown in a variety of instrumental conditioning sit-
uations (runway behavior, lever pressing, key pecking) and animal
species (rats, pigeons; for review see ), and it has been often
considered as an emotional-based phenomenon. Thus, according
to the frustration theory proposed by Amsel , when a response
is nonrewarded in the presence of the expectancy of reward (as
occurring in animals receiving partial reinforcement in the non-
reinforced trials during acquisition) an aversive internal state of
primary frustration is induced. The pairing of initially neutral con-
textual stimuli with this emotional reaction would enable these
stimuli to trigger an expectancy of frustration, called secondary
frustration that would interfere with the performance of the previ-
ously learned response. Finally, the occurrence of a reinforced trial
in presence of secondary frustration would increase the tolerance
to frustration through a counterconditioning process. Therefore,
counterconditioning may be the mechanism underlying response
persistence during extinction after chronically uncertain condi-
tions of reinforcement (for review see ). The emotional nature
both the administration of anxiolytic drugs and the lesion of brain
sites related to emotion abolish the PREE by preventing animals
from learning to tolerate frustration during acquisition (for review
see [33,37]). Therefore, given the emotional reactivity divergences
repeatedly observed in RHA and RLA rats, the study of the PREE in
of the emotional explanation of this paradoxical phenomenon.
The present experiment was designed with the aim of study-
ing the PREE in female inbred RHA-I and RLA-I rats. With this goal
in mind, groups of RHA-I and RLA-I food-deprived animals were
exposed to a straight alley where they were partial or continu-
ously reinforced (RHA/PRF and RLA/PRF vs. RHA/CRF and RLA/CRF).
According to the emotional view of PREE and related phenomena
described above, it was expected that the more fearfulness RLA-I
rats would show more robust behavioral effects induced by frus-
tration, in comparison to the less emotional RHA-I strain (whose
behavior could be compared to animals receiving the injection of
an anxiolytic drug). That is, a greater PREE effect was expected in
RLA rats than in RHA rats.
tively). Their weigh ranged from 250g to 310g at the start of the experiment.
Animals were individually housed with water continuously available (ad libitum),
and deprived to 80% of ad lib feeding weight via daily feedings of lab chow approxi-
20◦C. Animals were maintained under a 12L–12D cycle with lights on at 8:00 a.m.
All testing sessions were performed between 9:00 a.m. and 14:00 p.m. Counter-
balanced groups across batches of 16 rats makes any influence of the oestrus cycle
on the observed results unlikely. The experiment was carried out according to E.U.
guidelines on the use of animals for research (86/609/EU).
The test apparatus was a straight 120 length×11 width×14 high cm runway
divided into three sections separated by cardboard guillotine doors. The “start” sec-
tion measured 20cm; the running section measured 80cm; and the goal section
measured 20cm. The walls and floor of the runway were made of painted wood
(dark green) and two guillotine doors separated the start and goal sections from
the running section when closed. The entire length of the runway was covered by
clear Plexiglas lids. The food reward was 45-mg pellets (formula P; Research Diets,
Inc., Noyes Precision Pellets, Lancaster, NH). Pellets were placed on the floor at the
distal end of the goal box. Time to run through the runway was manually recorded
by using a chronometer. Trials began as soon as the start door was raised, and the
chronometer was stopped when the rat entered the goal section with its four paws.
M.J. Gómez et al. / Behavioural Brain Research 194 (2008) 187–192
Rats were moved from the colony room to the adjacent experimental room in
their home cages in sets of sixteen; accordingly, trials were always spaced about
18min apart, and this feature was maintained throughout the experiment (even
though some animals of the same batch had finished their training). They were kept
in their home-cage within the experimental room between trials. The floor of the
apparatus was vacuumed and wiped down with 5% ethanol solution after every set
of rats finished its session. A white noise was always present in the background. The
experiment was conducted in three phases: Pre-training, Training, and Extinction.
Three days of habituation to the apparatus preceded training. On the first day,
rats were placed in the start box with both doors open and given five 1-min access
box feedings. Rats were given a maximum of 30s to consume the food reward and
then they were removed from the goal box. The rats were given six Noyes pellets
in the home cage 30min after the third habituation session along with their daily
ration of lab chow.
Training began on the fourth day. Each animal was placed in the start box with
the start box door closed and the goal box door opened. The start box door was then
opened and the rat was allowed to run down the runway to obtain the food reward.
On reinforced trials, 12 pellets were used as reward. On nonreinforced trials, the
rat was left in the goal box for 30s and was then returned to its home cage until
the next trial. The sequence of reinforced and nonreinforced trials was randomly
arranged by using those Gellermann  sequences with a similar number of RN
and NR transitions (R: reinforcement; N: nonreinforcement). Each of the CRF groups
was reinforced in the goal box on every acquisition trial. A maximum time of 20s
was allowed for the rat to complete the trial. If the rat did not reach the goal box
before 20s has elapsed, it was gently pushed down the runway by the experimenter
and 20s was recorded as running time. When the rat reached the goal box, the goal
box door was quietly closed by the experimenter and a stopwatch was started. The
rat was given a maximum of 30s to eat the food. As soon as either the rat finished
eating or the 30s elapsed, the rat was removed from the goal-box and placed back
into its home cage until the next trial began. The number of pellets left uneaten, if
5h of the L/D cycle. A learning criterion of two consecutive sessions with a mean
latency equal or under 3s was used. Each individual rat continued training until it
reached the acquisition criterion.
Each individual animal entered the extinction phase on the following day to the
one in which it reached the acquisition criterion. Extinction phase was identical to
the training phase with the exception that no reward was provided in the goal-box.
Once the rat reached the goal-box it remained there for 30s before going back to its
home-cage to wait for the following trial. The extinction phase lasted 8 days.
2.4. Dependent variable and statistical analysis
Sessions to criterion and latency on reaching the goal section were used as
dependent variables on acquisition and extinction, respectively. Prior to statistical
analysis, all latencies were converted to log (base 10) scores to better approximate
within-group normal distributions of scores, as required for the use of parametric
statistical analysis. Mean values in each experimental session were subjected to a
three-factor analysis of variance, with strain (RHA-I vs. RLA-I), reinforcement (CRF
were conducted for acquisition and extinction data, respectively. Where appropri-
ate, post hoc comparisons were made using the Bonferroni test. For all statistical
analyses, alpha was set at 0.05.
One rat belonging to the RLA/CRF group and two rats belonging
to the RLA/PRF group did not reach the acquisition crite-
rion after 21 days of training, so they were discarded from
the statistical analysis. The mean±S.E.M. number of sessions
needed to reach the acquisition criterion was as follows: group
RHA/CRF=6.13±1.11; RHA/PRF=5.88±0.44; RLA/CRF=8±0.84
and RLA/PRF=8.17±1.55. A 2 (strain)×2 (reinforcement) ANOVA
conducted with sessions to criterion data found a significant main
action was significant, F’s<1. These results showed that the RLA-I
Fig. 1. Mean latency (log) to reach the goal-box during the last two days of training
(T1 and T2) and the eight days of extinction in groups RHA/CRF, RHA/PRF, RLA/CRF
and RLA/PRF. Bars denote standard errors of the mean. *: RLA/CRF vs. RHA/CRF,
p<0.05. **: RLA/CRF vs. RLA/PRF, p<0.05.
strain needed more sessions to reach the acquisition criterion than
the RHA-I strain, regardless of the reinforcement schedule used
(continuous or partial).
Fig. 1 presents the mean latency (log) to reach the goal-box dur-
ing last 2 days of training (T1 and T2) and the 8 days of extinction in
groups RHA/CRF, RHA/PRF, RLA/CRF and RLA/PRF. With respect to
found a significant main effect of strain, F(1, 25)=29.08, p<0.001.
to indicate that, in spite of being trained to an acquisition criterion,
the RLA-I rats were slower than RHA-I rats, that is, they needed
more time to reach the goal box, even during the last sessions of
the training phase.
The most interesting results refer to the extinction phase. A 2
(strain)×2 (reinforcement)×8 (day) ANOVA conducted with the
extinction log latency data found a significant main effect of strain,
F(1, 25)=11.18, p<0.003, reinforcement, F(1, 25)=15.98, p<0.0001,
and day, F(7, 175)=49.40, p<0.0001. The strain×day interaction
was also statistically significant, F(7, 175)=2.82, p<0.008 as well as
importantly, there was a significant strain×reinforcement×day
interaction, F(7, 175)=2.10, p<0.046. In order to further explore
the source of this triple interaction, the following analyses were
conducted. First, to evaluate the differences in PREE in RHA-I and
RLA-I rats, the reinforcement×day interaction was analyzed in
each strain of animals. In RLA-I rats, a day×reinforcement inter-
action was obtained, F(7, 77)=5.22, p<0.005, suggesting that RLA-I
animals exposed to continuous reinforcement were slower than
those exposed to partial reinforcement. The simple effect of rein-
forcement in RLA-I rats was significant in sessions 1, 2 and 3,
smallest F(1, 11)=12.07, p<0.005. By contrast, in RHA-I animals
F<1. These results suggest that the PREE appeared in RLA-I rats, as
opposed to RHA-I rats.
Finally, the strain×day interaction was explored within each
within extinction of CRF and PRF training. A significant strain×day
These results indicate that RLA-I rats trained under CRF extin-
guished faster than equally trained RHA-I rats. The simple effect
of strain in the CRF groups was significant in sessions 1, 2, 4 and
M.J. Gómez et al. / Behavioural Brain Research 194 (2008) 187–192
5, smallest F(1, 13)=5.18, p<0.040 and fell just short of signifi-
cance in session 3, F(1, 13)=4.52, p<0.053. By contrast, in the PRF
condition the strain×day interaction did not reach statistical sig-
nificance, F<1. Summarizing these results, the RLA-I strain showed
less resistance to extinction than the RHA-I strain when they were
The present study analyzed the performance of RHA-I and RLA-I
rats in an appetitive instrumental task in which animals were con-
tinuously or partially reinforced with a 12 pellets-reward, and then
animals needed more trials to reach the acquisition criterion and
were slower as compared to RHA-I rats regardless of the type of
reinforcement. During extinction, PREE was found only in RLA-I
rats, with faster extinction in RLA/CRF rats than in RLA/PRF rats. By
contrast, RHA-I groups extinguished at the same rate, regardless of
the reinforcement schedule used during acquisition.
The effect of partial reward on extinction has been seen as a
paradoxical form of learning, and the search for an answer to that
question has generated many theories and a very large number
of experiments (for review see ). Some authors have found
evidence suggesting the implication of memory processes, rather
emotional mechanisms, in the PREE and reward omission related
phenomena [9,10]. In this regard, RLA (outbred and inbred) rats
have been shown to be superior to their RHA counterparts in a
variety of spatial and working memory tasks (see [3,50,51] and
unpublished results from our laboratory). This could explain that
RLA-I rats show faster extinction under the CRF condition (i.e.,
better between session transfer), but such a memory-based expla-
nation cannot account for the fact that RLA-I and RHA-I rats do no
differ in extinction under the PRF condition. Therefore, additional
processes seem to be required in order to explain the observed
Alternatively, Amsel [4,5] has argued that the partial reinforce-
reinforcement conditions learn to persist in the face of frustra-
tion. He assumes that during the initial runway trials the animal
develop an expectancy of receiving a reward in the goal-box of the
straight alley, this expectancy being mediated by a Pavlovian asso-
ciation. Once this reward expectancy is in effect, the occurrence
of nonreward triggers an aversive emotional response of frustra-
in partially reinforced subjects) they should come to expect frus-
tration as well as reward when they reach the goal area. This
anticipatory frustration would induce a state of conflict generated
by opposing tendencies to approach and/or withdrawal from the
goal. Finally, since the approach response is rewarded in some tri-
als in partially reinforced rats, the expectancy of both reward and
frustration should become associated with making the approach
response. Both internal states would act as a signalling stimulus for
the animal to make the approach response, facilitating the behav-
ioral persistence observed during the extinction phase. Therefore,
animals with a history of partial reward tend to approach longer
in extinction because of a counterconditioning of the anticipatory
frustration (for review see [22,35]).
Several items of evidence support the idea that the PREE can
be considered as a paradoxical learning effect based on emotional
mechanisms. Firstly, previous experience of partial reinforcement
can increase subsequent behavioral resistance to other frustrative
nonreward situations such as the consummatory SNC, an effect
that can be attenuated by the administration of the anxiolytic
drug chlordiazepoxide . In addition, it has been repeatedly
found that lesions of the septohippocampal system (a brain circuit
that regulates anxiety-related emotions) abolish the PREE [19,32].
Finally, the chronic administration of anxiolytic GABAergic com-
pounds in both acquisition and extinction phases tends to decrease
persistence in partially reinforced subjects, abolishing the PREE in
spaced-trial procedures (for review see [29,33]).
Applied to the present study, this theory can explain the whole
pattern of results found in it. First, less emotional RHA-I rats would
be expected to develop less frustration than RLA-I rats in the
presence of the unexpected omission of reinforcement. These dif-
ferences on emotionality across strains can explain why extinction
after continuous reinforcement was slower in RHA-I rats than in
RLA-I rats. Lower frustration raised by the unexpected omission
of the reward would lead RHA-I rats to a weaker withdrawal-
from-the-goal reaction and, subsequently, a greater behavioural
persistence during extinction. This increased resistance to extinc-
in our laboratory [27,28], and could be related to the innate genetic
emotional divergences repeatedly observed in several fear/anxiety
tests (for review see ).
Secondly, this lower emotionality would also explain the lack
of PREE in RHA-I rats. If frustration by the unexpected omission
of the outcome is low, there is no opportunity for countercondi-
proceed at the same rate regardless of the type of reinforcement
schedule (partial or continuous) used during training. In a par-
allel effect, decreased persistence after partial reward experience
has been observed when Wistar rats are trained and extinguished
under the effect of anxiolytic drugs .
Thirdly, the frustration triggered by the unexpected omission of
reinforcement would be greater in the more emotional RLA-I rats
than in RHA-I rats. Subsequently, extinction should proceed faster
in the former as opposed to the latter under the CRF condition.
acquisition that were sometimes followed by reinforced trials and,
according to Amsel  this experience would had allowed these
rats to persist responding in the presence of frustration.
Finally, the RHA–RLA differences in acquisition of the runway
response replicate performance divergences previously obtained
in our laboratory using a similar instrumental runway task. Thus,
sented reinforcer (12 pellets, 2 pellets or 1 pellet), RLA-I rats spent
counterparts. Similar RHA–RLA differences in acquisition of a run-
way response were obtained in an extinction study where sessions
to criterion were used as dependent variable [27,28]. Therefore, the
results obtained in our laboratory suggest the existence of consis-
tent strain performance differences in this appetitive instrumental
task. As already discussed [27,42], these training strain divergences
could be dependent on RHA–RLA differences in response to the
reinforcing properties of natural rewards. In this regard, it has been
found that RHA-I rats show better performance than RLA-I rats
in experimental paradigms where different appetitive reinforcers
are used, including saccharin solutions and drugs of abuse such as
psychostimulants [11,20]. These differences seem to be related to
strain divergences in mesolimbic dopaminergic transmission (for
review see ). It can be also argued that, as opposed to the freez-
ing response usually observed in RLA-I rats, RHA-I animals tend to
show high levels of locomotor activity when coping with challeng-
could alternatively explain the slower runway behavior and the
greater number of sessions needed to reach the acquisition crite-
rion observed in the RLA-I strain in comparison to the RHA-I strain.
M.J. Gómez et al. / Behavioural Brain Research 194 (2008) 187–192
ison to the RHA line during the acquisition of the running response
may be explained in terms of motor or response-cost strain diver-
gences. These differences may also explain the faster extinction of
RLA rats. However, the differential PREE effect size across strains
cannot be explained by motor differences. Finally, runway perfor-
mance could have been influenced by strain differences in food
palatability sensitivity, although previous RHA–RLA hyponeopha-
gia studies conducted under different novelty conditions seem to
discard this possibility [17,21].
In summary, present results suggest that the behavioral differ-
of an appetitive instrumental response were dependent on the
particular conditions of reinforcement experienced during the
acquisition phase. Thus, when a continuous reinforcement contin-
gency was employed, the RHA-I strain showed a greater resistance
to extinction as compared to the RLA-I strain. By contrast, the
experience of nonreward during the partial reinforcement train-
ing seemed to increase the tolerance to frustration exhibited by
the more emotionally reactive RLA-I rats, when their performance
was compared to that of the less anxious RHA-I animals exposed to
continuous or partial reinforcement.
Many sources of suffering and stress in human subjects involve
reinforcement downshift, and the understanding of their physical
and mental consequences has been, and is being, made possible in
recent years through the use of aversive reward reduction animal
models. Partial reinforcement may be conceptualized as a chronic
exposure to psychological pain that facilitates the development
of tolerance against reward loss . The results obtained in this
experiment suggest that RHA-I and RLA-I animals may be used
as useful tools for exploring the genetic variables that modulate
the individual vulnerability to develop behavioral, emotional and
physiological disorders induced by reward loss.
This research was supported by MCYT, Ministerio de Ciencia y
019015). The authors thank Toni Ca˜ nete for his continous and
 Abler B, Walter H, Erk S. Neural correlates of frustration. Neuroreport
 Adelman R, Maatsch JL. Learning and extinction based upon frustration, food
reward and exploratory tendency. J Exp Psychol Anim B 1956;52:311–5.
reactivity and fear of heights: a factor analytic map from a large F2 intercross
of Roman rat strains. Brain Res Bull 2002;57:17–26.
Psychol Bull 1958;55:102–19.
Memory. Psychon B Rev 1994;1(3):280–96.
 Azrin NH, Hutchinson RR, Hake DF. Extinction-induced aggression. J Exp Anal
 Becker HC, Jarvis M, Wagner G, Flaherty CF. Medial and lateral amygdala
lesions differentially influence contrast with sucrose solutions. Physiol Behav
 Broadhurst PL, Bignami G. Correlative effects of psychogenetic selection: a
study of the Roman high and low avoidance strains of rats. Behav Res Ther
 Capaldi EJ. Effect of nonrewarded and partially rewarded placements on resis-
tance to extinction in the rat. Comp Physiol Psychol 1971;76(3):483–90.
 Capaldi EJ. The relation between memory and expectancy as revealed
by percentage and sequence of reward investigations. Psychon Bull Rev
 Corda MG, Piras G, Lecca D, Fernández-Teruel A, Driscoll P, Giorgi O. The psy-
chogenetically selected Roman rat lines differ in the susceptibility to develop
amphetamine sensitization. Behav Brain Res 2005;157:147–56.
 Daly HB. Learning to escape cues paired with reward reductions following
single- or multiple-pellet reward. Psychon Sci 1972;26:49–52.
ish CT, editors. Experimental psychopharmacology. Clifton, NJ: Human Press;
1987. p. 263–97.
 Driscoll P, Bättig K. Behavioral, emotional and neurochemical profiles of rats
selected for extreme differences in active, two-way active performance. In:
Lieblich I, editor. Genetics of the brain. Amsterdam: Elsevier; 1982. p. 95–
 Driscoll P, Escorihuela RM, Fernández-Teruel A, Giorgi O, Schwegler H, Steimer
Th, et al. Genetic selection and differential stress responses: the Roman
lines/strains of rats. In: Csermely P, editor. Stress of life: from molecules to
man. New York, NY, USA: New York Academy of Sciences; 1998. p. 501–
 Eisenberger NI, Lieberman MD. Why rejection hurts: a common neural alarm
system for physical and social pain. Trends Cogn Sci 2004;8:294–300.
 EscorihuelaRM,Fernández-TeruelA,GilL,AguilarR,Tobe˜ naA,DriscollP.Inbred
roman high- and low-avoidance rats: differences in anxiety, novelty-seeking
and shuttlebox behaviours. Physiol Behav 1999;67(1):19–26.
 Escorihuela RM, Fernández-Teruel A, Tobe˜ na A, Langhans W, Bättig K, Driscoll
P. Labyrinth exploration, emotional reactivity, and conditioned fear in young
Roman/Verh inbred rats. Behav Genet 1997;27(6):573–8.
 Feldon J, Rawlins JNP, Gray JA. Fornixfimbria section and the partial reinforce-
ment extinction effect. Exp Brain Res 1985;58:435–9.
 Fernández-TeruelA,DriscollP,GilL,AguilarR,Tobe˜ naA,EscorihuelaRM.Endur-
ing effects of environmental enrichment on novelty seeking, saccharin and
ethanol intake in two rat lines (RHA/Verh and RLA/Verh) differing in incentive-
seeking behavior. Pharmacol Biochem Behav 2002;73:225–31.
 Ferré P, Fernández-Teruel A, Escorihuela RM, Driscoll P, Corda MG, Giorgi
O, et al. Behavior of the Roman/Verh high- and low-avoidance rat lines in
anxiety tests: relationship with defecation and self-grooming. Physiol Behav
 Flaherty CF. Animal learning and cognition. NY: McGraw-Hill; 1985.
 Flaherty CF. Incentive relativity. New York: Cambridge University Press; 1996.
disrupt runway but not consummatory contrast. Behav Brain Res 1998;93:1–9.
 Gellermann LW. Chance orders of alternating stimuli in visual discrimination
experiments. J Genet Psychol 1933;42:206–8.
 Giorgi O, Piras G, Corda MG. The psychogenetically selected Roman high- and
low-avoidance rat lines: a model to study the individual vulnerability to drug
addiction. Neurosci Biobehav Rev 2007;31:148–63.
 Gómez MJ, de la Torre L, Callejas-Aguilera JE, Rosas JM, Escarabajal MD, Agüero
itive instrumental response in female Roman high (RHA) and low avoidance
(RLA) rats, Psicológica, in press.
 Gómez MJ, Rosas JM, de la Torre L, Callejas-Aguilera JE, Escarabajal MD, Agüero
A, et al. Extinction of an appetitive instrumental response in Roman female rats
of high (RHA) and low avoidance (RLA). In: The XIX congress of the Spanish
society of comparative psychology. 2007.
 Gray JA. The psychology of fear and stress. Cambridge: CUP; 1987.
 Gray JA, McNaughton N. The neuropsychology of anxiety: an enquiry into the
 Guitart-Masip M, Giménez-Llort L, Fernández-Teruel A, Ca˜ nete T, Tobe˜ na
A, Ögren SO, et al. Reduced ethanol response in the alcohol-preferring
RHA rats and neuropeptide mRNAs in relevant structures. Eur J Neurosci
 Henke PG. Dissociation of the frustration effect and the partial reinforce-
ment extinction effect after limbic lesions in rats. J Comp Physiol Psych
 Leslie JC, Shaw D, McCabe C, Reynolds DS, Dawson GR. Effects of drugs that
rosci Biobehav Rev 2004;28:229–38.
not consummatory negative contrast in rats. Behav Brain Res 2000;116:61–79.
 Mackintosh NJ. The psychology of animal learning. London: Academic Press;
 Mitchell C, Flaherty CF. Temporal dynamics of corticosterone elevation in suc-
cessive negative contrast. Physiol Behav 1998;64:287–92.
 Papini MR. Comparative psychology of surprising nonreward. Brain Beha Evol
 Papini MR, Wood M, Daniel AM, Norris JN. Reward loss as psychological pain.
Int J Psych Psychol Ther 2006;6(2):189–213.
 Pellegrini S, Muzio RN, Mustaca AE, Papini MR. Successive negative contrast
after partial reinforcement in the consummatory behavior of rats. Learn Motiv
 Romero LM, Levine S, Sapolsky RM. Adrenocorticotropine secretation release:
stimulation by frustration and paradoxically by reward presentation. Brain Res
 Rosas JM, Callejas-Aguilera JE, Escarabajal MD, Gómez MJ, de la Torre L, Agüero
A, et al. Successive negative contrast effect in instrumental runway behaviour:
a study with Roman high- (RHA) and Roman low- (RLA) avoidance rats. Behav
Brain Res 2007;185:1–8.
M.J. Gómez et al. / Behavioural Brain Research 194 (2008) 187–192
 Siegrist J, Menrath I, Stöcker T, Klein M, Kellermann T, Shah NJ, et al. Differential
brain activation according to chronic social reward frustration. Neuroreport
 Steimer T, Driscoll P. Divergent stress responses and coping styles in
psychogenetically selected Roman high- (RHA-I) and low- (RLA) avoid-
ance rats: behavioural, neuroendocrine and developmental aspects. Stress
 Steimer T, Driscoll P. Inter-individual vs line/strain differences in psy-
chogenetically selected Roman high- (RHA-I) and low- (RLA) avoidance
rats: neuroendocrine and behavioural aspects. Neurosci Biobehav Rev
 Stout SC, Muzio RN, Boughner RL, Papini MR. Aftereffects of the surprising pre-
sentation and omission of appetitive reinforcers on key-pecking performance
in pigeons. J Exp Psychol Anim Behav Process 2002;28:242–56.
 Thomas BL, Papini MR. Mechanisms of spaced-trial runway extinction in
pigeons. Learn Motiv 2003;34:104–26.
 Torres MC, Cándido A, Escarabajal MD, de la Torre L, Maldonado A, Tobe˜ na A, et
al. Successive negative contrast effect in one-way avoidance learning in female
roman rats. Physiol Behav 2005;85:377–82.
 Wagner AR. Frustration and punishment. In: Haber RN, editor. Current research
on motivation. New York: Holt, Rinehart & Winston; 1966. p. 229–39.
 Willig F, M’Harzi M, Bardelay C, Viet D, Delacour J. Roman strains as a psycho-
genetic model for the study of working memory: behavioral and biochemical
data. Pharmacol Biochem Behav 1991;40(1):7–16.
 Willig F, Van de Velde D, Laurent J, M’Harzi M, Delacour J. The Roman strains of
ory: example with RU 41656. Psychopharmacology Berl 1992;107(2–3):415–