ArticlePublisher preview available

Behavioral Momentum and the Partial Reinforcement Effect

American Psychological Association
Psychological Bulletin
Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

Free-operant behavior is more resistant to change when the rate of reinforcement is high than when it is low. The usual partial reinforcement extinction effect, demonstrating greater resistance to extinction after intermittent than after continuous reinforcement, seems to contradict this generalization. However, most free-operant extinction data are reported as response totals, which confound the initial levels of responding and the rate at which responding decreases over the course of extinction. A reanalysis shows that after extended training, the slope of the extinction curve is shallower after continuous reinforcement than after intermittent reinforcement, suggesting greater rather than less resistance to change. These results, which hold for both independent-groups and within-subject comparisons, support the general finding that resistance to change of free-operant behavior is a positive function of the rate of reinforcement. This generalization does not, however, hold for discrete-trial performance. I discuss some consequences of these analyses for applications of behavioral research results.
Psychological
Bulletin
1988.
VolT
103,
No.
1,44-56
Copyright
1988
by the
American
Psychological
Association, Inc.
0033-2909/88/J00.75
Behavioral
Momentum
and the
Partial
Reinforcement
Effect
John
A.
Nevin
University
of
New
Hampshire
Free-operant
behavior
is
more
resistant
to
change when
the
rate
of
reinforcement
is
high
than
when
it
is
low.
The
usual
partial
reinforcement
extinction
effect,
demonstrating
greater
resistance
to
extinc-
tion
after
intermittent
than
after
continuous
reinforcement, seems
to
contradict
this
generalization.
However, most
free-operant
extinction
data
are
reported
as
response
totals,
which confound
the
initial
levels
of
responding
and the
rate
at
which
responding
decreases
over
the
course
of
extinction.
A
reanalysis
shows
that
after
extended
training,
the
slope
of the
extinction
curve
is
shallower after
continuous
reinforcement
than
after
intermittent
reinforcement, suggesting
greater
rather
than
less
resistance
to
change.
These
results, which
hold
for
both
independent-groups
and
within-subject
com-
parisons,
support
the
general
finding
that
resistance
to
change
of
free-operant
behavior
is a
positive
function
of the
rate
of
reinforcement.
This
generalization
does
not,
however,
hold
for
discrete-trial
performance.
I
discuss
some
consequences
of
these
analyses
for
applications
of
behavioral
research
results.
A
series
of
experiments conducted over
the
past dozen years
supports
a
simple generalization:
The
greater
the
rate
of
rein-
forcement
for a free-operant
response,
the
more
it
resists change
(Nevin,
1974,
1979; Nevin, Mandell,
&
Atak,
1983;
Nevin,
Mandell,
&
Yarensky,
1981).
A
standard experiment uses
pi-
geons
as
subjects, deprived
to 80% of
their
free-feeding
weights.
After
brief training
to eat
from
the
grain magazine
in a
conven-
tional
pigeon chamber
and
then
to
peck
the
key, with
food
after
each peck,
the
birds
are
exposed
to a
multiple schedule
of
rein-
forcement
in
which
two
distinctive lights
on the key
signal
two
different
schedule components.
For
example,
the key
might
be
lighted
red for 1
min,
during which pecks would
be
reinforced
at
variable
intervals
averaging
1 min (60
reinforcers
per
hr),
and
then green
for the
next
1
min,
during which pecks would
be
reinforced
at
variable intervals averaging
3 min (20
reinforcers
per
hr). (Such
a
schedule would
be
denoted
mult
VI
1-min,
VI
3-min.)
Training would continue until response rates
in
both
components became
stable—perhaps
40
daily
1-hr
sessions.
Then
the
resistance
to
change
of
these asymptotic response
rates would
be
assessed
by
introducing
a new
variable
uni-
formly
with respect
to
both components, such
as
free
food
dur-
ing
dark-key periods between schedule components (e.g.,
Nevin,
1974)
or
prefeeding
in the
home cage (e.g.,
Eckerman,
1968).
Both procedures have
a
common result:
The
rate
of re-
sponding changes less, relative
to its
baseline training level,
in
the
component with
the
higher rate
of
reinforcement (red,
VI
1-min
in
this case).
[ am
indebted
to
L.
Slade
for
assistance
with
literature
searches
and
data
analyses,
to E.
Fickett
for
help
with
the
conduct
of the
study
re-
ported
in
this
article,
to W.
Baum
and V.
Benassi
for
many stimulating
discussions,
and to two
reviewers
for
helpful
comments
on an
earlier
version
of
this
article.
Correspondence
concerning
this
article
should
be
addressed
to
John
A.
Nevin,
Department
of
Psychology, University
of New
Hampshire,
Durham,
New
Hampshire
03824.
Generality
and
Consistency
In
addition
to
pigeons, rats
and
monkeys have served
as
sub-
jects
in
experiments
of
this sort,
and
many other procedures
for
assessing resistance
to
change have been used: introducing
concurrent
reinforcement
for an
alternative response, either sig-
naled
(Nevin
et
al.,
1981)
or
unsignaled
(Pliskoff,
Shull,
&
Gol-
lub,
1968);
varying
hours
of
deprivation
(Carlton,
1961)orbody
weight
(Herrnstein
&
Loveland,
1974); presenting stimuli
sig-
naling
unavoidable electric shock (Blackman,
1968;
Lyon,
1963);
punishing
the
reinforced response
(Bouzas,
1978);
in-
creasing response
effort
(Elsmore,
1971);
and
simply tracking
response rates through
circadian
changes
in
activity (Elsmore
&Hursh,
1982).
Whatever
the
assessment procedure,
it
must
be
applied
equally
to the two
components:
For
example,
if
punishment
is
used
to
assess resistance
to
change,
it
must
be
arranged
so
that
experienced shock rates
are the
same
in
both components, even
when
response rates
differ.
Prefeeding
and
deprivation proce-
dures
are
especially
useful
in
this
respect,
because motivation
levels
and
internal stimulation
are the
same regardless
of
which
component
is in
effect.
Also,
all
these procedures leave
the
base-
line
schedule
of
reinforcement unchanged. Under these condi-
tions,
the
results
are
uniform
and
clear: Relative
to
their base-
line levels,
the
rate
of
responding
in the
component with
the
higher
rate
of
reinforcement
is
less
affected
than
the
rate
of re-
sponding
in the
component
with
the
lower
rate
of
reinforce-
ment.
Normally,
baseline response rate
is
higher
in the
component
with
the
higher rate
of
reinforcement. However, resistance
to
change
is
independent
of
baseline response rates.
For
example,
Nevin
(1974,
Experiment
5)
established
low
response
rates
in a
VI
1-min component
and
high rates
in a VI
3-min component
by
means
of
tandem
interresponse-time
requirements. These
rate
differences
are
opposite
to
those usually
obtained
with
standard
VI
schedules. Nevertheless, resistance
to
change rela-
tive
to
baseline remained greater
in the VI
1-min component.
44
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
... Unless they graduated from graduate school recently and studied with a faculty member who has kept up with these advances, this material is out of date, and their ideas need to be updated. Several researchers have recently made similar assertions in regard to current conceptions of both operant (Nevin, 1988) and classical conditioning (Rescorla, 1988) within the field of psychology. ...
... There are numerous exciting areas that may interface between behavioral science and behavioral medicine and that illustrate the importance for using behavioral theories for work that focuses on behavior. These include new approaches to choice and decision making (Rachlin, 1989) or new approaches to maintenance of behavior (Nevin, 1988). ...
... One reason for this state of affairs may be an incorrect or inadequate conceptualization of the variables that are required to prevent extinction of newly acquired behavior and to maintain behavior over time. One striking example of this misunderstanding is the role of continuous and intermittent or partial reinforcement in maintaining behavior (Nevin, 1988). Common to conceptualizations of most behavioral interventions for habit change is the notion that variables responsible for acquisition and maintenance of a behavior are separate (Bandura, 1969). ...
Article
Full-text available
Behavioral medicine is a multidisciplinary field that combines research methods and findings from behavioral and biomedical sciences. Many investigators in the field have tended to emphasize the contribution of the biomedical more than the behavioral sciences. This is evident in the emphasis on biological rather than behavioral outcomes and on the reductionist approach within the field to reduce mechanisms responsible for behavioral effects and disease to biological influences. There has been a similar shift in psychology toward mechanistic, bottom-up approaches to understanding mechanisms responsible for integrated and dynamic behavior. These shifts in emphasis have stimulated investigators to examine the use of biomedical methods and findings as causes and explanations for behavior, rather than to utilize newer findings in behavioral sciences. New advances in basic research on learning are used to illustrate that findings from behavioral science have implications for the field of contemporary behavioral medicine. Finally, the importance of developing new technologies for measuring behavior is presented.
... Response rates during the testing sessions were divided by this baseline rate to arrive at a proportional measure, standardized around 1. Adult rats generally responded at higher rates than adolescents, but this is likely because of the fact that adults were larger and pressing the lever was easier for them. POB measures are used extensively in research interested in changes in response rate as a function of changes in other variables (Nevin, 1988). Within-session rates of responding in 5 min bins served as another dependent variable. ...
... The partial-reinforcement extinction effect (PREE) predicts that partially reinforced (i.e., "PRF" or intermittently reinforced) responses will show greater resistance to extinction than a continuously reinforced responses because exposure to PRF necessarily involves occasions where responses did not produce reinforcement. Behavioral Momentum Theory ("BMT, " Nevin, 1988) suggests that, all else being equal, the response associated with the higher reinforcement rate will be more resistant to extinction or change. In the current experiments, the adults had earned a greater number of reinforcers for lever-pressing, had higher rates of responding, and had higher rates of reinforcement, but their behavior changed more quickly when exposed to extinction, DRL, and DRO, suggesting that habit-strength theories, PREE and BMT cannot account for the present results. ...
Article
Poor self-control, lack of inhibition, and impulsivity contribute to the propensity of adolescents to engage in risky or dangerous behaviors. Brain regions (e.g., prefrontal cortex) involved in impulse-control, reward-processing, and decision-making continue to develop during adolescence, raising the possibility that an immature brain contributes to dangerous behavior during adolescence. However, very few validated animal behavioral models are available for behavioral neuroscientists to explore the relationship between brain development and behavior. To that end, a valid model must be conducted in the relatively brief window of adolescence and not use manipulations that potentially compromise development. The present experiments used three operant arrangements to assess whether adolescent rats differ from adults in measures of learning, behavioral inhibition, and impulsivity, within the aforementioned time frame without substantial food restriction. In Experiment 1, separate squads of rats were trained to lever-press and then transitioned to two types of extinction. Relative to their baselines, adolescent rats responded more during extinction than adults, suggesting that they were less sensitive to the abolishment of the reinforcement contingency. Experiment 2 demonstrated similar age-related differences during exposure to a differential reinforcement of low rates schedule, a test of behavioral inhibition. Lastly, in Experiment 3, adolescent's responding decreased more slowly than adults during exposure to a resetting delay of reinforcement schedule, suggesting impaired self-control. Results from these experiments suggest that adolescents exhibit impaired learning, behavioral inhibition and self-control, and in concert with recent reports, provide researchers with three behavioral models to more fully explore neurobiology of risk-taking behavior in adolescence.
... In fact, the FREE seems to occur rather rarely in free-operant experiments. We have not been able to find a choice experiment of the appropriate sort, but Nevin (1988) has reviewed a number of similar experiments with multiple schedules (i.e., successive rather than simultaneous discriminations). For example, in one experiment (Nevin, 1979, Fig. 4.1), pigeons were trained on multiple VI 28-s-VI 86-s, VI 86-s-VI 360-s, or VI 28-s-VI 360-s schedules. ...
... The CE model implies that the most important variable determining resistance to extinction in a choice situation is the number of reinforcers received for each alternative. Nevin (1988) noted that when it is possible to compare the effects of number of reinforcers, resistance to extinction increases with the number of reinforcers received in training. ...
Article
Full-text available
Recurrent choice has been studied for many years. A static law, matching, has been established, but there is no consensus on the underlying dynamic process. The authors distinguish between dynamic models in which the model state is identified with directly measurable behavioral properties (performance models) and models in which the relation between behavior and state is indirect (state models). Most popular dynamic choice models are local, performance models. The authors show that behavior in different types of discrimination-reversal experiments and in extinction is not explained by 2 versions of a popular local model and that the nonlocal cumulative-effects model is consistent with matching and that it can duplicate the major properties of recurrent choice in a set of discrimination-reversal experiments. The model can also duplicate results from several other experiments on extinction after complex discrimination training.
... The temporal primacy of θ slope i estimation could provide a prediction tool for future preference modification in CAT, without tainting the results with a direct evaluation of preferences using a choice task. An interesting hypothesis could anticipate that the 50% cue-contingency would resemble partial reinforcement learning schedule, and thus will have potentially higher long-term sustainability and resistance to extinction (Nevin, 1988). While the current work did not examine long-term maintenance of CAT effect and its association cue-contingency, this could be an interesting subject to test in future studies. ...
Article
Full-text available
Cue-Approach Training (CAT) is a paradigm that enhances preferences without external reinforcements, suggesting a potential role for internal learning processes. Here, we developed a novel Bayesian computational model to quantify anticipatory response patterns during the training phase of CAT. This phase includes individual items, and thus, this marker potentially reflects internal learning signals at the item level. Our model, fitted to meta-analysis data from 28 prior CAT experiments, was able to predict individual differences in nonreinforced preference changes using a key computational marker. Crucially, two new experiments manipulated the training procedure to influence the model’s predicted learning marker. As predicted and preregistered, the manipulation successfully induced differential preference changes, supporting a causal role of our model. These findings demonstrate powerful potential of our computational framework for investigating intrinsic learning processes. This framework could be used to predict preference changes and opens new avenues for understanding intrinsic motivation and decision making.
... As with any area of science, comparisons can be described as absolute differences or proportional differences-neither is fundamentally superior to the other because both approaches answer different questions about the subject matter. For example, proportionally comparing the strength of a human to an ant (i.e., the ratio of mass lifted to the lifter's body mass) can generate questions and extend scientific theory (Entomology Today, 2014), as can proportionally comparing resistance to extinction following continuous reinforcement versus intermittent reinforcement (Nevin, 1988), but proportional comparisons might be less advantageous when determining who is best suited to lift heavy furniture, ensuring caregivers understand how to avoid prolonging dangerous behavior via intermittent reinforcement, or visually comparing absolute amounts of change across multiple graphs bound from 0 to 100%. ...
Article
Full-text available
Visual analysis is the primary method to detect functional relations in single-case experimental design (SCED) research. Meta-Visual-Analysis (MVA) is a novel approach used to synthesize SCED data where the estimated effect size measures are principally anchored to primary aspects of visual analysis: change in the magnitude of level, trend, variability, and trend-adjusted level of projected trends. For each of these aspects, percentage point differences between baseline and intervention conditions are estimated and quantified for every participant across studies. MVA effect sizes are standardized, and their aggregates are graphically displayed in a manner similar to individual SCED graphs. MVA graphs are compared and visually analyzed with the aim of better understanding the effectiveness and generality of interventions across SCED studies. In this discussion paper we provide general steps to conduct an MVA and describe MVA’s utility in reviewing, organizing, and directing future SCED research syntheses.
... The early primacy of cumulative records in behavioral psychology explicitly reflects the emphasis of response rate as a primary measure of behavior, especially how it changes in time at a molecular level (Ferster & Skinner, 1957;Kangas & Cassidy, 2011;. Response rate is fundamental to our understanding of behavior as choice (Herrnstein, 1961), behavioral momentum (Nevin, 1988), relapse and resurgence (Podlesnik & Shahan, 2010), and the theoretical system known as Mathematical Principles of Reinforcement (Killeen, 1994). Response rate is also a fundamental measure of behavior maintenance and change central to many areas of applied behavior analysis (Johnston & Pennypacker, 1993). ...
Article
Full-text available
Operant behavior typically occurs in bouts and pauses. The microstructural analysis of bouts and pauses reveals important and separable information about the physical characteristics of the operant and the motivation behind it. An analysis of interresponse times (IRTs) often reveals a mixture of two exponential distributions. One corresponds to short IRTs within ongoing response bouts, reflecting motor properties of the operant, and the other corresponds to longer intervals between bouts, reflecting the motivation behind the response. Partitioning responses into bout initiations and within-bout responses via this two-mode framework reveals the mechanisms underlying behavior maintenance and change. This approach is used in the fields of neurotoxicology, behavioral pharmacology, and behavioral neuroscience to disentangle the contribution of motivational and motoric variables to the pattern of operant behavior. In this paper, we present a primer aimed at providing essential concepts related to the analysis of response bouts and temporal dynamics of operant performance. *This paper is in press*
... Variações assistemáticas da intermitência dos reforços em contingências complexas podem dificultar o contato do comportamento com a extinção, o que pode ocasionar um responder persistente sem produzir consequências mantenedoras(Nevin, 2020).Em uma contingência, os intervalos entre reforços não só têm a função de consequências mantenedoras do comportamento, mas também exercem controle discriminativo. ParaNevin (1988) e Pierce e Cheney(2004), os efeitos do reforçamento parcial pode ser fruto de uma generalização de estímulo considerando que contingências cujo intervalo entre reforços é maior tende a ser mais similar à extinção do que contingências em CRF. Segundo os autores, a literatura relata uma série de evidências de que comportamentos mantidos por reforçamento intermitente são mais resistentes à mudança do que aqueles mantidos por reforçamento contínuo. ...
... Variações assistemáticas da intermitência dos reforços em contingências complexas podem dificultar o contato do comportamento com a extinção, o que pode ocasionar um responder persistente sem produzir consequências mantenedoras(Nevin, 2020).Em uma contingência, os intervalos entre reforços não só têm a função de consequências mantenedoras do comportamento, mas também exercem controle discriminativo. ParaNevin (1988) e Pierce e Cheney(2004), os efeitos do reforçamento parcial pode ser fruto de uma generalização de estímulo considerando que contingências cujo intervalo entre reforços é maior tende a ser mais similar à extinção do que contingências em CRF. Segundo os autores, a literatura relata uma série de evidências de que comportamentos mantidos por reforçamento intermitente são mais resistentes à mudança do que aqueles mantidos por reforçamento contínuo. ...
... Others have attempted to adjust for the differences in responding associated with the different baseline conditions by calculating the rate of change in responding during extinction or the proportion-of-baseline response rates (cf. Nevin, 1988). Results of these studies indicated that CRF schedules were associated with greater resistance to extinction than were INT schedules. ...
Chapter
This chapter covers the mechanics of the discussion section of a scholarly manuscript. The discussion section is intended to tie together the study and leave a meaningful, perhaps inspiring, impression on the reader. In this chapter, we review writing style, structure, common components, and common errors found in discussion sections. A variety of examples and tips are provided to aid the construction of a factual and impactful discussion section.
Article
Full-text available
An attempt was made to produce a within-S PRE by running the PRF and CRF trials to patterns of stimulation which differed very greatly. Under these within-S conditions there was still no evidence for a PRE; however, between-S groups run under these conditions showed a clear PRE.
Article
Resistance to extinction and resistance to satiation were examined with rats after training on random-ratio 1,5, 10, 20, 30, and 40 schedules of food reinforcement. During training, response rate was an increasing function of the random ratio. Relative to baseline, extinction responding increased and responding under reduced deprivation decreased as functions of the random ratio. A formula based on reinforcement frequency and responses per reinforcer during training was devised to describe extinction responding. Further support for the formula was obtained from data on responding during extinction after training on fixed-ratio (Boren, 1961) and fixed-interval (Wilson, 1954) schedules of reinforcement.
Article
Rats deprived to 80% or 90% body weight received either 50% or 100% food reinforcement in a runway. They were then satiated and run on the same schedule. A history of partial reinforcement, or of more severe deprivation, produced faster response speeds during satiation. The greater resistance of partially reinforced responses to the decremental effects of satiation may be related to their high resistance to the effects of extinction and punishment.
Article
Four pigeons trained on a 1-min. VI schedule were reinforced, extinguished, and reconditioned under each of two stimulus conditions for 20 daily sessions. The average rate in extinction decreased significantly as a function of the number of extinctions. A concurrent increase in rate occurred during VI reinforcement periods. Extinction occurred more rapidly as it was repeated. Assuming an exponential decay in rate of responding during extinction, the effects of repetition upon the parameters of fitted curves was examined. The decay constant (ratio of deceleration to rate) increased linearly as a function of the number of extinctions, while the log initial rate decreased linearly. These changes in extinction parameters were in the same direction as previously obtained in an experiment involving another species and procedure, but which did not permit determination of the form of relations between extinction parameters and the number of extinctions. The empirical grounds for the exponential decay assumption are discussed in connection with the effects of the principal independent variables influencing the course of operant extinction.
Article
Number of responses and time to extinction were measured after 3, 10, 1000, 3000, 5000, and 10,000 reinforced key-peck responses during conditioning. Each response was reinforced with a 045-gm. food pellet. The number of responses in extinction was a monotonically increasing function which became asymptotic beyond 1000 reinforced responses. Number of reinforced responses during conditioning significantly affected the number of responses in extinction ( p < .001) but not the time to extinction. The results support the findings of previous free-operant bar-press studies with rats. Free-operant animal studies of extinction after continuous reinforcement have consistently produced monotonically increasing functions and have typically employed relatively small amounts of reinforcement. Amount of reward may be an important parameter determining the shape of the extinction function in the free-operant studies.
Article
Pigeons were trained to peck a key for food on a variable-interval 1-min schedule. They were then given a choice between responding for food or eating freely available food from a cup in the rear of the chamber. Four pigeons were given consecutive choice tests after 6, 12, 24, 48, and 96 sessions of training, four pigeons were tested after 48 and 96 sessions, and four pigeons with extensive experience from a previous experiment were tested once. Response rates in the presence of free food increased as a function of previous training. The data suggest that amount of training affects contrafreeloading and that this variable may be partly responsible for the large variability in responding observed among studies examining responding in the presence of free food.
Article
Rats given acquisition and extinction training in discrete lever-pressing trials in lieu of the conventional runway were required to complete a fixed ratio of responses on a retractable lever to end each trial. Magnitude of reward (large or small) and schedule of reinforcement (continuous or partial) was signaled during each trial. Contrary to the classic effects of these variables between subjects, there was greater resistance to extinction with continuous than partial reinforcement (a reversed effect of partial reinforcement) at both magnitudes of reward; greater resistance to extinction with large than small magnitudes at both schedules; and no interaction of magnitude and schedule in resistance to extinction.