ArticlePDF Available

Factors controlling the micro-structure of human free-operant behaviour: Bout-initiation and within-bout responses are effected by different aspects of the schedule

Authors:

Abstract

Two experiments examined factors controlling human free-operant performance in relation to predictions based on the nature of bout-initiation and within-bout responding. Overall, responding was higher for a random ratio (RR) than a random interval (RI) schedule, with equal rates of reinforcement. Bout-initiation rates were not different across the two schedules, but within-bout rates were higher on the RR schedule. Response cost reduced overall rates of responding, but tended to suppress bout-initiation responding more than within-bout responding (Experiments 1 & 2). In contrast, reinforcement magnitude increased all forms of responding (Experiment 2). One explanation consistent with these effects is that bout-initiation responses are controlled by overall rates of reinforcement through their impact on the context (i.e. are stimulus-driven), but that within-bout responses are controlled by response reinforcement (i.e. are goal-directed). These current findings are discussed in the light of these theoretical suggestions.
Human schedule performance - 1
Factors controlling the micro-structure of human free-operant behaviour:
Bout-initiation and within-bout responses are effected by different aspects
of the schedule
Xiaosheng Chen and Phil Reed
Swansea University, UK.
Correspondence address: Phil Reed,
Department of Psychology,
Swansea University,
Singleton Park,
Swansea, SA2 8PP, UK.
e-mail: p.reed@swansea.ac.uk
Short title: Human schedule performance.
Cite as: Chen, X., & Reed, P. (in press). Factors controlling the micro-structure of human
free-operant behaviour: Bout-initiation and within-bout responses are effected by different
aspects of the schedule. Behavioural Processes. Doi: 10.1016/j.beproc.2020.104106
Human schedule performance - 2
Abstract
Two experiments examined factors controlling human free-operant performance in
relation to predictions based on the nature of bout-initiation and within-bout responding.
Overall, responding was higher for a random ratio (RR) than a random interval (RI) schedule,
with equal rates of reinforcement. Bout-initiation rates were not different across the two
schedules, but within-bout rates were higher on the RR schedule. Response cost reduced
overall rates of responding, but tended to suppress bout-initiation responding more than
within-bout responding (Experiments 1 & 2). In contrast, reinforcement magnitude increased
all forms of responding (Experiment 2). One explanation consistent with these effects is that
bout-initiation responses are controlled by overall rates of reinforcement through their impact
on the context (i.e. are stimulus-driven), but that within-bout responses are controlled by
response reinforcement (i.e. are goal-directed). These current findings are discussed in the
light of these theoretical suggestions.
Keywords: schedules of reinforcement; bout-initiation; within-bout responding; response
cost; reinforcement magnitude; actions and habits; humans.
Human schedule performance - 3
Response rates are typically higher on random ratio (RR) than random interval (RI)
schedules of reinforcement when the rates of reinforcement on these schedules are equated
(Ferster & Skinner, 1957; Peele, Casey, & Silberberg, 1984; Zuriff, 1970). This finding has
formed the basis of numerous investigations of the factors that control free-operant
responding, and has underpinned several theoretical accounts of schedule-maintained
behaviour (Peele et al., 1984; Reed, 2015; Zuriff, 1970). Recent analysis of schedule-
controlled responding has suggested that two forms of responding are present: ‘bout-
initiation’ responding and ‘within-bout’ responding’ (Killeen, Hall, Reilly, & Kettle, 2002;
Shull, 2011; Shull, Gaynor, & Grimes, 2001). ‘Bout-initiation’ responding refers to the first
responses in any particular bout of behavioural engagement (e.g., the first response from a set
of responses made to a lever); whereas, ‘within-bout’ responding comprises all of the
responses that follow in that particular engagement.
These aspects of responding are controlled by different aspects of contingencies.
‘Bout-initiation responding co-varies with overall rates of reinforcement (Killeen et al., 2002;
Shull, 2011; Shull et al., 2001), and is hypothesised to be stimulus-driven – that is, dependent
on eliciting cues (Reed, 2020), and controlled by factors that influence the degree to which
the context gains strength – that is, the degree to which it is associated with the delivery of
the reinforcer (Reed, Smale, Owens, & Freegard, 2018). ‘Within-bout’ responding is
influenced by the shaping effects of reinforcement on responding (Reed et al., 2018; Shull,
2011), such as the reinforcement of particular inter-response times (Peele et al., 1984), and is
hypothesised to be goal-directed (Reed, 2020).
Previous explorations have shown that human schedule performance also comprises
these forms of responding, which are controlled by some of the same factors that control such
responding in nonhuman subjects (Reed, 2015; Reed et al., 2018). In addition, it has been
suggested that human ‘bout-initiation’ responding is ‘habitual’ and ‘automatic’ in nature, and
Human schedule performance - 4
not under conscious control; whereas, ‘within-bout’ responding is goal-directed and under
conscious control (Chen & Reed, 2020; Reed, 2020). For example, Chen and Reed (2020)
found that instructions, known to impact human responding (Hayes, Brownstein, Zettle,
Rosenfarb, & Korn, 1986), and act on consciously controlled but not habitual automatic
responses (Aarts & Dijksterhuis, 2000), affected ‘within-bout’, but not ‘bout-initiation’,
responding.
The current series of studies develops the exploration of the nature of the micro-
structure of human schedule responding (e.g., on ‘bout-initiation’ and ‘within-bout’
responding) by examining how factors known to be important for free-operant responding
impact bout-initiation and within-bout responding. Response cost (Kazdin, 1972), and
reinforcer magnitude (Bonem & Crossman, 1988), have both been suggested as important in
the control of schedule performance, but have not been explored deeply in the context of
human free-operant behaviour or its micro-structure. This examination will be conducted in
the light of theoretical predictions regarding the likely source of impact of such factors – i.e.
on stimulus-driven or goal-directed behaviours, and their hypothesised impact on ‘bout-
initiation’ and ‘within bout’ responding (Reed, 2020). Such an examination should not only
elucidate the nature of the control over these schedule micro-behaviours, but also extend
theoretical understanding of these response types.
Response cost is an important factor in the maintenance of schedule-controlled
behaviour (Azrin & Holz, 1966; McMillan, 1967; Pietras, Brandt, & Searcy, 2010; Weiner,
1964), and its effects are easy to quantify (Bennett & Cherek, 1990; Bradshaw, Szabadi, &
Bevan, 1978; Pietras et al., 2010). It refers to factors associated with making a response,
such as required force of the response, or whether making the response involves reducing the
obtained value of a reinforcer. Research with nonhuman subjects has found that response
cost decreases rates of responding, irrespective of its effect on reinforcement rate (Pietras &
Human schedule performance - 5
Hackenberg, 2005; Raiff, Bullock, & Hackenberg, 2008). Such a response cost manipulation
on an RI schedule suppressed human bout-initiation responding more than within-bout
responding, although the impact on an RR schedule was unclear (Reed et al., 2018). This
differential suppressive effect on the microstructure of schedule responding was taken to
reflect that response may impact the Pavlovian value of the context by adding a negative
outcome into the context, separable from the value of reinforcer (Raiff et al., 2008), and, thus,
impact stimulus-driven bout-initiation responses (Reed et al., 2018).
Reinforcer magnitude has been suggested to impact patterns of human schedule
responding. Larger reinforcer magnitudes tend to lead to higher response rates (Bonem &
Crossman, 1988; Gentry & Eskew, 1984; Hendry, 1962). Buskist, OliveiraCastro, and
Bennett (1988) examined the effect of response-correlated increases in the reinforcer
magnitude on human behaviour, and noted that a positive correlation between reinforcer
magnitude and response rates led to higher response rates than a fixed-correlation procedure.
However, the impact of reinforcement magnitude on responding is not always noted
(Dougherty & Cherek, 1994; Reed, 1991). In terms of the predicted effect of reinforcement
magnitude on the micro-structure of schedule-controlled performance, reinforcer magnitude
has been suggested to have dual impacts on responding (Killeen, 1985). Firstly, it is
suggested to energise general levels of activity (Killeen, 1985), perhaps through conditioning
goal-related cues (Pereboom & Crawford, 1958), which should impact bout-initiation
responding (Reed et al., 2018; Reed, 2020). Secondly, it has been taken to act directly on
goal-directed behaviour by strengthening to tendency to emit the preceding responses (Reed,
1991). Given such theoretical speculation, in contrast to response cost (which is taken to act
more on bout-initiation than within-but responses), reinforcement magnitude should act on
both types of responses.
Human schedule performance - 6
A number of different procedures have been adopted to explore the micro-structure of
free-operant responding (Killeen et al., 2002; Mellgren & Elsmore, 1991; Reed, 2011; Shull,
2011; Sibley, Nott, & Fletcher, 1990). As these approaches tend to produce the same pattern
of results (Chen & Reed, 2020; Reed et al., 2018), the more commonly adopted ‘log survivor
method’ was used for the current set of studies. This method calculates the number of inter-
response times (IRTs) emitted in particular time-bins, and turns these into a percentage of all
responses not yet emitted. This percentage is then turned into a log. The slope of a resulting
log survivor plot is an indicator of the response rate: the steeper the slope, the higher the rate
of responding. The slope of log survival plots is not uniform, but comprises an initially steep
slope (bout-initiation responses), followed by a shallow slope (within-bout responses),
indicating the presence of two different types of responding. A double exponential equation
can be fitted, where the equation fits the two distributions of IRTs (i.e. those prior to the
‘break’, taken to represent response initiations; and those after the break, taken to represent
within-bout responses). This equation takes the form: Ppred = a*exp(-bt)+(1-a)*e(-dt); where
b and d represent the rates of within-bout and bout-initiation, respectively.
The present study aimed to examine the relationship between micro-responding
observed on RR and yoked RI schedules, and both response cost and reinforcer magnitude.
This investigation was aimed at further understanding the factors controlling human schedule
behaviour, and at elucidating the mechanisms that might underpin this behaviour.
Experiment 1
Previous studies have demonstrated that overall responding is higher on RR than RI
schedules with equal rates of reinforcement (Peele et al., 1984; Reed et al., 2018). When the
micro-structure of responding is analysed, within-bout, but not bout-initiation, responding
Human schedule performance - 7
follows this same pattern (Chen & Reed, 2020; Reed et al., 2018). Response cost
manipulations reduce the overall rates of responding for nonhuman subjects (McMillan,
1967; Pietras et al., 2010) and human participants (Reed, 2001; Weiner, 1962). Such a
manipulation on an RI schedule suppressed bout-initiation responding more than within-bout
responding (Reed et al., 2018). This differential suppressive effect was taken to reflect that
response cost would tend to impact the Pavlovian value of the context, and, thus, impact
stimulus-driven, bout-initiation responses (Reed et al., 2018). One aim of Experiment I was
to replicate and extend the previous explorations of the effects of response cost on human
schedule behaviour.
The effects of response cost on RR schedules have not been investigated extensively.
One theoretical possibility is that such a manipulation may not greatly impact responding on
RR schedules, as this schedule tends to produce more goal-directed responding, due to the
stronger relationship between responding and reinforcement (Pérez, Aitken, Zhukovsky, Soto,
Urcelay, & Dickinson, 2016). If response costs tend to have their impact through effecting
the value of the context driving bout-initiation responding (Reed, 2020), then the effects of
response cost may not be as pronounced on an RR schedule as it is on an RI schedule.
To test these possibilities, participants were randomly split into two experimental
groups: a 1-point response cost group, and a 10-point response cost group (following Reed et
al., 2018). Additionally, procedures highlighted as important in previous studies were
adopted to bring the human schedule performance under greater schedule control; that is by
using: a response cost (Raia et al., 2000), a verbal suppression task (Bradshaw & Reed,
2012), and screening for aberrant personality types as individuals high in depression and
schizotypy show atypical schedule performance (Dack, McHugh, & Reed, 2009; Randell,
Ranjith-Kumar, Gupta, & Reed, 2009).
Human schedule performance - 8
Method
Participants
A sample of 48 adult participants (36 male, 12 female) was recruited from a Chinese
company. The participants were all Chinese, and aged between 18 and 54 years (mean=
37.29 + 9.93 SD). Participants received a financial payment (50 RMB per hour). No
participant reported any previous history of mental illness. However, 4 participants were
excluded on this basis of high depression and schizotypy scores (Dack et al., 2009; Randell et
al., 2009), leaving 44 participants in the study.
Materials
Beck’s Depression Inventory (BDI; Beck et al., 1961, Chinese version from Wu &
Chang, 2008) is a 21-item questionnaire that measures the clinical symptoms of depression
through asking about feelings during past few weeks. The score ranges from 0 to 63, with an
internal reliability (α) between .73 and .92 for a non-psychiatric population (Beck et al.,
1988). The reliability and validity of the scale are supported in Chinese translation (Wu &
Chang, 2008). A score of greater than 9 is taken as showing some level of depression (Beck
et al., 1961)
Oxford Liverpool Inventory of Feelings and Experiences-Brief Version (O-LIFE
(B); Mason, Linney, & Claridge, 2005, translated into Chinese for this study) is designed to
assess schizotypy in a healthy population, and contains 43 questions, under four subscales
(unusual experiences, UE; cognitive disorganisation, CD, introverted anhedonia, IA, and
impulsive nonconformity, IN). The internal reliability (Cronbach α) of the scales is: UE = .
80; CD = .77; IA = .62; IN =.63 (Mason et al., 2005, Randell et al., 2009). A score one
standard deviation above the mean on the UE scale has been taken as indicating some degree
of deviation from the norm in terms of schedule behaviour, in that response rates on RR
Human schedule performance - 9
schedules are not demonstrably greater than those on matched RI schedules (Randall et al.,
2009).
Apparatus
The experimental task was presented on a standard desktop computer. Visual Basic
(6.0) was used to programme the task. The computer task was presented on a white screen,
with a stimulus box placed in the centre upper portion of the screen. The box was
approximately 8cm wide × 3cm high, and was blocked with a single colour (either blue or
pink), to indicate the schedule type (each schedule was associated with a particular colour for
each participant). Underneath the colour stimulus box, the word “POINTS” (in capital
letters) was positioned, and below this, the running total of the points accumulated appeared
in figures.
Procedure
Participants were tested individually in a quiet room, which contained a desk and
computer, with the monitor situated approximately 60cm from them. Participants gave
written consent, and read the study information and instructions for the task. Participants
commenced the task in their own time, and were required to fill in basic demographic details
about themselves, along with the psychometric questionnaires, before the schedule task was
presented.
Each schedule presentation (trial) was 4min long, and a RR schedule trial was always
presented immediately prior to the yoked RI schedule trial. The procedure of yoking RI trials
to preceding RR trials ensured that reinforcement in the RI schedule was delivered after a
similar elapse of time that it had taken for the corresponding reinforcers to be awarded on the
RR trial.
Human schedule performance - 10
On the RR-30 schedule, points were awarded after each space bar response with a
1/30 probability. On the following RI schedule, points were awarded following the first
response after a specified amount of time had elapsed. The RI schedule was yoked to the
preceding RR schedule, so that each successive reinforcement in the RI schedule was
delivered only after the elapse of time that it had taken for the corresponding reinforcer to be
awarded on the RR trial. There were 4 such RR-RI pairs of schedule presentations (i.e. 8
trials in total). A new schedule was indicated by the colour in the box changing. For the first
trial (RR) it was blue (for half the participants), followed by pink for the second trial (RI),
and alternated, in this manner, for the subsequent trials. Participants were informed that the
box would change colour when a new trial commenced but were not informed of which
schedule type the colour indicated.
Each reinforcer in each condition consisted of 40 points being added to the
participant’s total. The total started at 100 points for all participants at the start of each new
schedule presentation. Participants also lost 1 or 10 points for each space bar response,
regardless of whether the response was reinforced. This response cost procedure has been
adopted in previous studies (Bradshaw & Reed, 2012; Raia et al., 2000).
Participants were randomly allocated to one of two groups: Cost 1 or Cost 10 (both ns
= 22). Prior to the task beginning, all participants were presented with instructions on the
computer screen (in Chinese):
When the task begins, use the space bar to score as many points as possible. There
are eight games in total. The first game is identified with a large blue [pink] rectangle at the
top of the screen. When the first game is over, the rectangle will change to blue [pink] to
indicate the start of the next game. The rectangles alternate between blue and pink to
indicate the changing games for the remainder of the task. Your goal in each game is to
reach the highest score possible. You will see that the points reduce according to the way in
Human schedule performance - 11
which you play, but will rise again every so often, according to the pattern of space bar hits
that you use. All you need to do is to find the best pattern of space bar hits to score as highly
as possible in each game. It may be a good idea to respond quickly sometimes and slowly at
other times, but you need to discover this for yourself!
The participants were then instructed to click a start button to continue with the
experiment. Participants in the Cost 1 group lost one point for each space bar response,
regardless of whether the response was reinforced; and participants in the Cost 10 group lost
10 points for each space bar response, regardless of whether the response was reinforced.
During the time in which they were performing on the schedules, the participants had
to perform a counting backwards task throughout the entire experiment (Andersson, Hagman,
Talianzadeh, Svedberg, & Larsen, 2002). They were each given one random five-digit
number at the start of the procedure (different for each participant), and were asked to count
backwards from that number, out-loud, in 7s. This procedure was adopted in an attempt to
minimize the potential role of verbal rule formation in influencing participants’ performance
on the schedule (Bradshaw & Reed, 2012; Raia et al., 2000). In order to enhance task
adherence, a recording device was placed prominently on the desk in front of the participant,
and they were told that their answers to the counting task would be analysed and scored later.
Results and Discussion
On the first trial of training, the schedule means for the Cost 1 group were: RR =
165.03 (+ 97.50); RI = 152.86 (+ 98.36); and these means for the Cost 10 group were: RR =
61.57 (+ 52.51); RI = 47.93 (+ 69.55). A two-factor mixed-model analysis of variance
(ANOVA), with cost (1 versus 10) as a between-subject factor, and schedule (RR versus RI)
as a within-subject factor, was conducted on these data. This analysis revealed a statistically
significant main effect of cost, F(1,42) = 21.32, p < .001, η2p = .337[95%CI = .121:.502],
Human schedule performance - 12
pH1/D = .999, but not of schedule, F(1,42) = 1.67, p = .203, η2p = .004[.000:.012], pH0/D = .
912, or interaction between cost and schedule, F < 1, η2p = .001[.000:.00] pH0/D = .999.
------------------------------
Figure 1 about here
-------------------------------
Figure 1 shows the group-mean overall responses rates for final trial, for each
schedule, for each group. Analyses were conducted on the final trial, because it represents
the terminal performance. Inspection of these data shows that, for both schedules, responding
in Cost 1 group was higher than that for Cost 10 group. Moreover, for both groups, RR
schedule response rates were higher than RI schedule response rates. A two-factor mixed-
model ANOVA (cost x schedule) revealed statistically significant main effects of cost,
F(1,42) = 23.62, p < .001, η2p = .360[95%CI = .135:.531], pH1/D = .999, and schedule,
F(1,42) = 3.87, p = .050, η2p = .084[.000:.263], pH1/D = .545. There was no significant
interaction between cost and schedule, F(1,42) = 2.91, p = .095, η2p = .065[.000:.236] pH0/D
= .581. These results indicate that participants responded in a differentiated manner
according to schedule types (Chen & Reed, 2020; Peele et al., 1984), and that the response
cost decreased levels of responding (Pietras et al., 2010; Reed et al., 2018).
-----------------------------
Figure 2 about here
------------------------------
Figure 2 shows the group-mean bout-initiation (top panel) and within-bout (bottom
panel) rates for two schedules, for the last trial, for both groups. These rates were calculated
using the log survivor method (Shull, 2011), by fitting the double exponential equation: Ppred
= a*exp(-bt)+(1-a)*e(-dt), for each participant individually. Each individual’s IRTs were
entered into the spreadsheet developed by Peter Killeen (available on the SQAB website, and
Human schedule performance - 13
later modified by Richard Shull). The worksheet fits the data by minimizing the summed
squared differences between the logs of obtained and predicted survivor proportions. It also
excludes the longest 1% of IRTs, as very long IRTs may result from extra-experimental
factors, thus the programme forces a better fit to the right tail – the portion relevant to bout-
initiation rate.
Inspection of the group-mean bout-initiation data (top panel) shows that both
schedules produced similar rates of responding, but that responding in the Cost 1 group was
greater than that to the Cost 10 group. A two-factor mixed-model ANOVA (cost x schedule)
revealed a significant main effect of cost, F(1,42) = 27.23, p <.001, η2p = .393[.164:.558],
pH1/D = .736, but not of schedule, F(1,42) = 1.62, p =.210, η2p = .037[.000:.193], pH0/D = .
999, and no interaction between schedule and cost, F < 1, η2p = .000[.000:.000], pH0/D = .999.
These data replicate previous demonstrations that when the rate of responding is equated on
two different schedules, the rate of bout-initiation responding is equal (Chen & Reed, 2020;
Killeen et al., 2002; Shull et al., 2001). However, response cost had a pronounced effect on
both RR and RI bout-initiations, in line with previous demonstrations of this effect (Reed et
al., 2018).
The bottom panel of Figure 2 shows the group-mean within-bout rates for the two
groups, for last trial, and reveals response rates in RR schedule are higher than the RI
schedule for both groups. The effect of response cost was more pronounced for the RI than
for the RR schedule. A two-factor mixed-model ANOVA (group x schedule) revealed
statistically significant main effects of schedule, F(1,42) = 31.02, p < .001, η2p = .
425[.194:.583], pH1/D = .999, but not of cost, F < 1, η2p = .002[.000:.064], pH0/D = .999.
There was a significant interaction between cost and schedule, F(1,42) = 10.14, p < .004, η2p
= .195[.026:.384], pH0/D = .962. Simple effect analyses revealed no effect of cost on the RR
schedules, F(1,42) = 1.92, p = .219, η2p = .044[.000:.204], pH0/D = .694, but an effect of cost
Human schedule performance - 14
on the RI schedule, F(1,42) = 3.37, p = .045, η2p = .074[.000:.249], pH1/D = .503. These
results demonstrate a higher within-bout rate for the RR compared to the RI schedule (Chen
& Reed, 2020; Reed et al., 2018), but that response cost had an effect only on the RI
schedule.
Overall, these results replicated the effects of RR and RI schedules on human
responding, and also replicated the effects of these schedules on the micro-structure of human
responding (Chen & Reed, 2020; Reed et al., 2018). They also confirmed the effect of the
response cost condition noted in the current Experiment 1 (Pietras et al., 2010; Reed et al.,
2018; Weiner, 1962). It was also noted in the current experiment that the cost manipulation
appeared to differentially impact bout-initiation responding, in comparison with within-bout
responding. The manipulation tended to suppress bout-initiation responses more than it
suppressed within-bout responses (Reed et al., 2018), although this was more true for the RI
than the RR schedule. It has been suggested that the response cost manipulation would
impact the Pavlovian strength of the context, by associating the context with negative
outcomes, as well as positive ones, and, thus, impact bout-initiation responding (Reed et al.
2018; Reed, 2020). Moreover, this effect was stronger for the within-bout responses on the
RI than the RR schedule; the former is taken (overall) to be less goal-directed in nature (Perez
et al., 2016), and perhaps more sensitive to context-driven effects.
Experiment 2
Experiment 2 investigated the effects of both response cost and reinforcement
magnitude on the micro-structure of human schedule behaviour. Experiment 1 demonstrated
that higher response costs impacted bout-initiation responding to a much greater extent than
within-bout responding (see also Reed et al., 2018). This was in line with predictions based
Human schedule performance - 15
on the assumption that response cost would impact context conditioning and stimulus-driven
responding more than goal-directed responding (Reed et al., 2018). Experiment 2 examined
whether this effect of response costs on bout-initiation responding could be replicated.
Magnitude of reinforcement also has been suggested to impact instrumental
responding (Gentry & Eskew, 1984; Hendry, 1962; but see Bonem & Crossman, 1988; Reed,
1991). However, there are very few studies that investigate the effects of reinforcement
magnitude for human participants (see Blakely, Starin, & Poling, 1988), or which explore the
interaction between the response cost and reinforcement magnitude on the micro-structure of
responding for human participants or, indeed, any species. In terms of the predicted effect of
reinforcement magnitude, this factor has been suggested to both energise overall levels of
activity (Killeen, 1985) through conditioning goal-related/contextual cues (Pereboom &
Crawford, 1958), and to strengthen goal-directed behaviour (Reed, 1991). Given this, in
contrast to response cost (which is taken to act more on bout-initiation than within-bout
responses), reinforcement magnitude should act on both types of responses.
To this end, participants were randomly split into four experimental groups: Cost 1
Reinforcement 40; Cost 1 Reinforcement 600; Cost 10 Reinforcement 40; and Cost 10
Reinforcement 600. They then responded on a RR-30 RI-y schedules, as in Experiment 1,
and their responses analysed using the log survivor method (Killeen et al., 2002).
Method
Participants
A sample of 105 students (39 males, 65 females) were recruited from two universities
(55 from China and 50 from the UK). None of the participants were involved in Experiment
1. They were aged between 18 to 36 years (mean= 19.76 + 1.99). Participants received
credits from the University subject pool. No participant reported any previous history of
Human schedule performance - 16
mental illness. However, 7 participants were excluded on this basis of their psychometric
depression or schziotypy scores (as described in Experiment 1), leaving 98 in the study (Cost
1 Rein 40 = 25; Cost 1 Rein 600 = 22; Cost 10 Rein 40 = 24; Cost 10 Rein 600 = 27). The
apparatus and materials were as described in Experiment 1.
Procedure
The procedure was as described in Experiment 1, except that the participants were
randomly divided into four groups: (Cost 1 Rein 40; Cost 1 Rein 600; Cost 10 Rein 40; Cost
10 Rein 600). All groups responded on the RR-30 RI-y schedule, and experienced 4 pairs of
RR and RI training. Each trial lasted 4 min (i.e. there were 8 x 4-min trials in total). All
participants initially started with 100 points, which was reset at the start of each trial. For
Group Cost 1 Rein 40, each response subtracted 1 point from their total, and a reinforcer
consisted of 40 points being added to the total. For Group Cost 1 Rein 600, each response
subtracted 1 point from their total, and a reinforcer consisted of 600 points being added to the
total. For Group Cost 10 Rein 40, each response subtracted 10 points from their total, and a
reinforcer consisted of 40 points being added to the total. For Group Cost 10 Rein 600, each
response subtracted 10 points from their total, and a reinforcer consisted of 40 points being
added to the total.
Results and Discussion
On the first trial of training, the schedule means for the Cost 1 Rein 40 group were:
RR = 136.12 (+ 57.21), RI = 129.76 (+ 72.61); Cost 1 Rein 600: RR = 165.32 (+ 82.54), RI =
154.34 (+ 42.11); Cost 10 Reinf 40: RR = 68.21 (+ 52.61), RI = 63.43 (+ 49.18); Cost 10
Rein 600: RR= 147.32 (+ 54.34), RI = 132.45 (+ 44.76). A three-factor mixed-model
ANOVA (schedule x cost x reinforcement) revealed a statistically significant main effect of
Human schedule performance - 17
cost, F(1,93) = 11.14, p < .001, η2p = .102[.030:.220], pH1/D = .976, and reinforcement,
F(1,93) = 8.86, p = .021, η2p = .089[.005:.123], pH1/D = .856, but not schedule, F < 1, η2p = .
007[.000:.023], pH0/D = .970. There were no two-way or three-way interactions, all, Fs < 1,
largest η2p = .006[.000:.069], all pH0/D > .990.
-----------------------------
Figure 3 about here
------------------------------
Figure 3 shows the group-mean overall responses rates for final trial, for each
schedule, for each of the four groups. Inspection of these data shows that responding to the
RR schedule was higher than that to the RI schedule, responding in 1-point cost groups was
higher than that for 10-point cost groups, and responding in the 600-point reinforcement
groups was higher than that in the 40-point reinforcement groups. A three-factor mixed-
model ANOVA (schedule x cost x reinforcement) revealed statistically significant main
effects of schedule, F(1,93) = 14.04, p < .001, η2p = .132[.030:.260], pH1/D = .990, cost,
F(1,93) = 10.32, p = .002, η2p = .100[.015:.223], pH1/D = .946, and reinforcement, F(1,93) =
13.79, p < .001, η2p = .129[.029:.257], pH1/D = .989. There were no two-way interactions:
schedule and cost, F < 1, η2p = .009[.000:.079] pH0/D = .865; schedule and reinforcement,
F(1,93) = 1.40, p = .238, η2p = .014[.000:.094], pH0/D = .826; cost and reinforcement, F(1,93)
= 2.85, p = .095, η2p = .029[.000:.123], pH0/D = .692, or three-way interaction, F < 1, η2p = .
001[.000:.013], pH1/D = .907.
These results indicate that participants responded in a differentiated manner according
to schedule types (Chen & Reed, 2020; Peele et al., 1984; Zuriff, 1970), and that the response
cost decreased levels of responding (Reed et al., 2018; Weiner, 1964; and the current
Experiment 1). In addition, reinforcement magnitude increased rates of responding. This
Human schedule performance - 18
latter effect has been noted for human participants (Blakely et al., 1988), but is not always
observed (Bonem & Crossman, 1988; Reed, 1991).
-----------------------------
Figure 4 about here
------------------------------
Figure 4 shows the group-mean bout-initiation (top panel) and within-bout (bottom
panel) rates for two schedules, for the last trial, for all groups, using the survivor method
(Killeen et al., 2002), as described in Experiment 1. Inspection of the group-mean bout-
initiation data (top panel) shows that both schedules produced similar rates of responding, but
that responding in 1-point cost groups was greater than that in the 10-point cost groups.
Rates of bout-initiation were higher in the 600-point reinforcement groups, than in the 40-
point reinforcement groups.
A three-factor mixed-model ANOVA (schedule x cost x reinforcement) revealed
statistically significant main effects of cost, F(1,93) = 12.65, p < .001, η2p = .120[.023:.246],
pH1/D = .981, and reinforcement, F(1,93) = 13.90, p < .001, η2p = .130[.029:.258], pH1/D = .
989, but not of schedule, F < 1, η2p = .004[.000:.066], pH0/D = .889. There were no two-way
interactions: schedule and cost, F(1,93) = 1.55, p = .215, η2p = .016[.000:.098] pH0/D = .814;
schedule and reinforcement, F < 1, η2p = .016[.000:.098], pH0/D = .892; cost and
reinforcement, F(1,93) = 1.82, p = .181, η2p = .019[.000:.104], pH0/D = .793, or three-way
interaction, F(1,93) = 2.97, p = .088, η2p = .031[.000:.125], pH1/D = .679.
These data replicate previous demonstrations that when the rate of responding is
equated on two different schedules, the rate of bout-initiation responding is equal (Chen &
Reed, 2020; Reed, 2015; Shull, 2011). However, increasing the response cost decreased the
rate of bout-initiations, consistent with the results of Experiment 1 (see also Reed et al.,
2018). Increasing the level of reinforcement increased the bout-initiation rate. This is a novel
Human schedule performance - 19
finding, but is predictable on the basis of reinforcement magnitude increasing the level of
context conditions (Killeen, 1985; Pereboom & Crawford, 1958).
The bottom panel of Figure 4 shows the group-mean within-bout rates for all groups,
and reveals within-bout response rates in RR schedule were higher than the RI schedule for
all groups. Increasing the magnitude of reinforcement increased within-bout rates, and
increasing response costs decreased rates, but not in as strong a manner as reinforcement
magnitude increased these rates.
A three-factor mixed-model ANOVA (schedule x cost x reinforcement) revealed
statistically significant main effects of schedule, F(1,93) = 18.74, p < .001, η2p = .
168[.051:.300], pH1/D = .788, reinforcement, F(1,93) = 14.00, p < .001, η2p = .
131[.030:.259], pH1/D = .990, and a smaller effect of cost, F(1,93) = 4.49, p = .037, η2p = .
046[.000:.152], pH1/D = .505. There were no two-way interactions: schedule and cost, F < 1,
η2p = .003[.000:.061] pH0/D = .893; schedule and reinforcement, F < 1, η2p = .008[.000:.079],
pH0/D = .864; cost and reinforcement, F < 1, η2p = .008[.000:.079], pH0/D = .903, or three-
way interaction, F(1,93) = 2.32, p = .129, η2p = .024[.000:.113], pH1/D = .745.
These results demonstrate a higher within-bout rate for the RR compared to the RI
schedule (Chen & Reed, 2020; Shull, 2011). They also show that reinforcement magnitude
appears to control within-bout rates, which might be expected if it is assumed that this form
of response is goal-directed, and magnitudes of reinforcement not only impact stimulus-
driven responding, but also impact goal-directed behaviour (Killeen, 1985; Reed, 1991).
Although numerically similar to the data reported in Experiment 1, there was no statistical
differentiation between the effect of response cost on RI and RR schedules.
General Discussion
Human schedule performance - 20
This current series of experiments examined the effects of a number of factors, taken
to control free-operant performance, on the micro-structure of human schedule behaviour. In
particular, response cost and reinforcer magnitude were examined in relation to predictions
based on the nature of bout-initiation and within-bout responding. It was hypothesised that
bout-initiation responding would be controlled by factors affecting context-conditioning that
could control bout-initiation responding through stimulus-driven means. In contrast, it was
suggested that within-bout responses were goal-directed, and would be impacted by factors
strengthening responding. In addition to demonstrating the empirical effects of manipulating
the above aspects of the contingency on human schedule performance, which is itself novel,
this investigation explored some theoretical underpinnings to the micro-structure of human
responding.
The current results demonstrated that human responding on RR and RI schedules of
reinforcement, yoked in terms of reinforcement rate, followed the same pattern as noted for
nonhuman subjects (Peele et al., 1984; Zuriff, 1970); overall responding was higher for the
RR than the RI schedules (see also Chen & Reed, 2020; Reed et al., 2018). It was noted that
bout-initiation rates were not different across the two schedules in either Experiment 1 or 2.
This replicates previous findings with nonhuman subjects (Reed, 2011; Shull et al., 2001)
with human participants (Chen & Reed, 2020; Reed et al., 2018). In contrast, within-bout
rates were higher on the RR than the RI schedule. Again, replicating previous findings from
nonhuman subjects (Reed, 2011; Shull, 2011), and human participants (Reed et al., 2018).
One explanation for this differential impact of schedules on the micro-structure of
schedule-controlled responding is that bout-initiation responses are controlled by the rate of
reinforcement received in the context (Shull, 2011), but that within-bout responses are
controlled by factors like IRT reinforcement (Peele et al., 1984). Reed et al. (2018) suggested
Human schedule performance - 21
that these facts could be accommodated by suggesting that bout-initiation responding was
stimulus-driven, and the strength of the context was a prime determinant of this action; but
within-bout responding is goal-directed and that response-strengthen effects control this type
of responding. The current series of studies attempt to determine if a series of manipulations
would support this suggestion.
Response cost was examined in both Experiments 1 and 2, and was found to reduce
overall rates of responding (Pietras & Hackenberg, 2005; Raiff et al., 2008). This is in line
with the results from previous studies for nonhuman subjects (McMillan, 1967; Pietras et al.,
2010) and human participants (Reed, 2001; Weiner, 1964). However, this manipulation
tended to suppress bout-initiation responding more than within-bout responding (Reed et al.,
2018); although there was some impact on the latter in Experiment 2. This differential
suppressive effect is consistent with response cost effecting the Pavlovian value of the
context, and, thus, impacting responses on more stimulus-driven, bout-initiation responses
(Reed et al., 2018).
The effect of reinforce magnitude was to increase rates of responding (Experiment 2).
Many previous researchers have found a positive correlation between reinforcer magnitude
and human participants’ response rates (Buskist et al., 1988; Hendry, 1962; Gentry & Eskew,
1984), although this is not always noted (Bonem & Crossman, 1988; Reed, 1991). At the
level of the micro-structure of responding, the effect of reinforcement magnitude was
suggested to be both on bout-initiation and within-bout responding. This effect was noted in
Experiment 2. Reinforcement magnitude has been suggested to have dual impacts on
responding (Killeen, 1985). Firstly, to increase the likelihood of activity (Killeen, 1985)
through conditioning goal-related cues (Pereboom & Crawford, 1958), which should impact
bout-initiation responding. Secondly, to act directly on goal-directed behaviour to strengthen
Human schedule performance - 22
to tendency to emit the preceding responses (Reed, 1991). These predictions were in line
with the current findings.
It should be noted that the log survivor procedure avoids arbitrary selection of cut-off
values to categorise responses into bout-initiation and within-bout classes (Reed, 2020), but it
relies on assumptions about the fit of the equation to the data (Bowers, Hill, & Palya, 2008),
and it is not clear if there may be other equations more appropriate for human responding.
Additionally, the double exponential method (Shull, 2011) requires many IRTs to get very
precise parameter estimates. The current samples were somewhat smaller than those
typically used from nonhuman subjects, and the resulting imprecision is likely to
underestimate within-bout response rate is acknowledged. Nevertheless, this method has the
best documented association with the factors that influence the ‘bout-initiation’ and ‘within-
bout’ responding – the main aim of the current study.
It should be acknowledged that while the reverse-counting task may have mitigated
the possibility that behavior was rule-governed, this cannot be discounted entirely. For
example, the last statement of the instructions advised: “It may be a good idea to respond
quickly sometimes and slowly at other times”, and this mirrored the order of the schedule
presentation. If participants followed the rules in the order that they were given, they would
come into contact with the fact that higher rates of responding would produce higher rates of
point delivery under the RR schedule, which was the first “game” presented. As they were
also instructed that there were two games, when the rectangle changed colours, they could
follow the rule that it was the next game, and maybe “slowly at other times” was now in
effect. Varying the instructions may serve to further explore these possibilities. It may also
be the case that longer exposure to the schedules might have impacted the results, although
that there was little impact of schedule control on the first trials, and there subsequently was
such an impact, suggests some learning had occurred.
Human schedule performance - 23
Thus, the current data were consistent with the suggestion that bout-initiation
responding co-varies with overall rates of reinforcement (Killeen et al., 2002; Shull, 2011;
Shull et al., 2001), and is stimulus-driven (Reed, 2020), controlled by the degree to which the
context gains strength (Reed et al., 2018). ‘Within-bout’ responding is influenced by the
shaping effects of reinforcement on responding (Reed et al., 2018; Shull, 2011), and is goal-
directed (Reed, 2020).
Human schedule performance - 24
References
Aarts, H., & Dijksterhuis, A. (2000). Habits as knowledge structures: Automaticity in goal
directed behaviour. Journal of personality and social psychology, 78(1), 53.
Andersson, G., Hagman, J., Talianzadeh, R., Svedberg, A., & Larsen, H. C. (2002). Effect of
cognitive load on postural control. Brain research bulletin, 58(1), 135-139.
Azrin, N. H., & Holz, W. C. (1966). Punishment. In W. K.Honig (Ed.), Operant behavior:
Areas of research and applications (pp. 380–447). New York: Appleton-Century
Crofts.
Beck, A. T., Steer, R. A., Ball, R., & Ranieri, W. F. (1996). Comparison of Beck Depression
Inventories-IA and-II in psychiatric outpatients. Journal of personality assessment,
67(3), 588-597.
Beck, A.T., Ward, C.H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for
measuring depression. Archives of General Psychiatry, 4, 561–571.
Bennett, R. H., & Cherek, D. R. (1990). Punished and nonpunished responding in a multiple
schedule in humans: A brief report. The Psychological Record, 40(2), 187-196.
Blakely, E., Starin, S., & Poling, A. (1988). Human performance under sequences of fixed
ratio schedules: Effects of ratio size and magnitude of reinforcement. The
Psychological Record, 38(1), 111-119.
Bonem, M., & Crossman, E. K. (1988). Elucidating the effects of reinforcement magnitude.
Psychological Bulletin, 104(3), 348.
Bowers, M.T., Hill, J., & Palya, W.L. (2008). Interresponse time structures in variable-ratio
and variable-interval schedules. Journal of the Experimental Analysis of Behavior, 90,
345-62.
Bradshaw, C. A., Freegard, G., & Reed, P. (2015). Human performance on random ratio and
random interval schedules, performance awareness and verbal instructions. Learning
Human schedule performance - 25
& behaviour, 43(3), 272-288.
Bradshaw, C. A., & Reed, P. (2012). Relationship between contingency awareness and
human performance on random ratio and random interval schedules. Learning and
Motivation, 43(1-2), 55-65.
Bradshaw, C. M., Szabadi, E., & Bevan, P. (1978). Effect of variable-interval punishment on
the behaviour of humans in variable-interval schedules of monetary renforcement.
Journal of the Experimental Analysis of Behavior, 29(2), 161-166.
Buskist, W., OliveiraCastro, J., & Bennett, R. (1988). Some effects of responsecorrelated
increases in reinforcer magnitude on human behavior. Journal of the experimental
analysis of behavior, 49(1), 87-94.
Chen, X., & Reed, P. (2020). Effect of instructions on the micro-structure of human
schedule performance: A differentiation between habits and actions. Under review.
Dack, C., McHugh, L., & Reed, P. (2009). Generalization of causal efficacy judgments after
evaluative learning. Learning & Behaviour, 37, 336-348.
Dougherty, D. M., & Cherek, D. R. (1994). Effects of social context, reinforcer probability,
and reinforcer magnitude on humans’ choices to compete or no to compete. Journal
of the experimental analysis of behavior, 62(1), 133-148.
Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York; Appleton
Century Crofts.
Gentry, G. D., & Eskew Jr, R. T. (1984). Graded differential reinforcement: Response
dependent reinforcer amount. Journal of the experimental analysis of behavior, 41(1),
27-34.
Hayes, S. C., Brownstein, A. J., Zettle, R. D., Rosenfarb, I., & Korn, Z. (1986). Rule
governed behavior and sensitivity to changing consequences of responding. Journal
of the experimental analysis of behavior, 45(3), 237-256.
Human schedule performance - 26
Hendry, D. P. (1962). The effect of correlated amount of reward on performance on a fixed
schedule of reinforcement. Journal of Comparative and Physiological Psychology,
55(3), 387.
Kazdin, A. E. (1972). Response cost: The removal of conditioned reinforcers for therapeutic
change. Behavior Therapy, 3(4), 533-546.
Killeen, P. R. (1985). Incentive theory: IV. Magnitude of reward. Journal of the experimental
analysis of behavior, 43(3), 407-417.
Killeen, P.R., Hall, S.S., Reilly, M.P., & Kettle, L.C. (2002). Molecular analyses of
the principal components of response strength. Journal of the Experimental
Analysis of Behaviour, 78, 127-160.
Mason, O., Linney, Y., & Claridge, G. (2005). Short scales for measuring schizotypy.
Schizophrenia research, 78(2-3), 293-296.
McMillan, D.E. (1967). A comparison of the punishing effects of response-produced shock
and response-produced time out. Journal of the Experimental Analysis of Behavior,
10, 439—449.
Mellgren, R. L., & Elsmore, T. F. (1991). Extinction of operant behavior: An analysis based
on foraging considerations. Animal Learning & Behavior, 19(4), 317-325.
Peele, D. B., Casey, J., & Silberberg, A. (1984). Primacy of interresponse-time reinforcement
in accounting for rate differences under variable-ratio and variable-interval schedules.
Journal of experimental psychology: Animal behavior processes, 10(2), 149.
Pereboom, A. C., & Crawford, B. M. (1958). Instrumental and competing behavior as a
function of trials and reward magnitude. Journal of Experimental Psychology, 56(1),
82.
Pietras, C. J., Brandt, A. E., & Searcy, G. D. (2010). Human responding on randominterval
schedules of responsecost punishment: The role of reduced reinforcement density.
Human schedule performance - 27
Journal of the experimental analysis of behavior, 93(1), 5-26.
Pietras, C. J., & Hackenberg, T. D. (2005). Response-cost punishment via token loss with
pigeons. Behavioural Processes, 69(3), 343-356.
Pérez, O.D., Aitken, M.R., Zhukovsky, P., Soto, F.A., Urcelay, G.P., & Dickinson, A. (2016).
Human instrumental performance in ratio and interval contingencies: A challenge for
associative theory. The Quarterly Journal of Experimental Psychology, 1-13.
Raia, C. P., Shillingford, S. W., Miller Jr, H. L., & Baier, P. S. (2000). Interaction of
procedural factors in human performance on yoked schedules. Journal of the
Experimental Analysis of Behavior, 74(3), 265-281.
Raiff, B. R., Bullock, C. E., & Hackenberg, T. D. (2008). Response-cost punishment with
pigeons: Further evidence of response suppression via token loss. Learning &
Behavior, 36(1), 29-41.
Randell, J., Ranjith-Kumar, A. C., Gupta, P., & Reed, P. (2009). Effect of schizotypy on
responding maintained by free-operant schedules of reinforcement. Personality and
Individual Differences, 47, 783-788.
Reed, P. (1991). Multiple determinants of the effects of reinforcement magnitude on free
operant response rates. Journal of the Experimental Analysis of Behavior, 55(1),
109-123.
Reed, P. (2001). Schedules of reinforcement as determinants of human causality judgments
and response rates. Journal of Experimental Psychology: Animal Behavior Processes,
27(3), 187.
Reed, P. (2011). An experimental analysis of steady-state response rate components on
variable ratio and variable interval schedules of reinforcement. Journal of
Experimental Psychology: Animal Behavior Processes, 37(1), 1.
Reed, P. (2015). The structure of random ratio responding in humans. Journal of
Human schedule performance - 28
Experimental Psychology: Animal Learning and Cognition, 41(4), 419.
Reed, P. (2020). Human free-operant performance varies with a concurrent task: Probability
learning without a task and schedule-consistent with a task. Learning and Behavior.
Reed, P., Smale, D., Owens, D., & Freegard, G. (2018). Human performance on random
interval schedules. Journal of Experimental Psychology: Animal Learning and
Cognition, 44(3), 309.
Shull, R. L., Gaynor, S. T., & Grimes, J. A. (2001). Response rate viewed as engagement
bouts: Effects of relative reinforcement and schedule type. Journal of the
Experimental Analysis of Behavior, 75, 247–274.
Shull, R. L. (2011). Bouts, changeovers, and units of operant behavior. European Journal of
Behavior Analysis, 12(1), 49-72.
Sibley, R.H., Nott, H.M.R., & Fletcher, D.J. (1990). Splitting behaviour into bouts. Animal
Behaviour, 39, 63-69.
Weiner, H. (1964). Response cost effects during extinction following fixedinterval
reinforcement in humans. Journal of the Experimental Analysis of Behavior, 7(4),
333-335.
Wu, P. C., & Chang, L. (2008). Psychometric Properties of the Chinese Version of the Beck
Depression Inventory-ll Using the Rasch Model. Measurement and Evaluation in
Counseling and Development, 41(1), 13-31.
Zuriff, G. E. (1970). A comparison of variable-ratio and variable-interval schedules of
reinforcement. Journal of the Experimental Analysis of Behavior, 13(3), 369-374.
Human schedule performance - 29
Figure 1: Experiment 1: Group-mean overall response rates for RR and RI schedules
for both groups (1 point response cost and 10 point response cost) on the final trial of
training. Error bars = 95% confidence intervals.
RR RI
0
50
100
150
200
250
Cost 1 Cost 10
Response per min
Human schedule performance - 30
Figure 2: Experiment 1: Group-mean response rates for RR and RI schedules both
groups (1 point response cost and 10 point response cost) on the last trial, using the log
survivor method. Top panel = response initiation rates. Bottom panel = within-bout
rates. Error bars = 95% confidence intervals.
Human schedule performance - 31
RR RI
0
5
10
15
20
25
30
Cost 1
Cost 10
Response per min
RR RI
0
50
100
150
200
250
300
350 Cost 1
Cost 10
Response per min
Human schedule performance - 32
Figure 3: Experiment 2: Group-mean overall response rates for RR and RI schedules
for all groups (cost = response cost; Rein = reinforcement magnitude) on the final trial
of training. Error bars = 95% confidence intervals.
0
50
100
150
200
250
RR RI
Responses per min
Human schedule performance - 33
Figure 4: Experiment 2: Group-mean response rates for RR and RI schedules both
groups (cost = response cost; Rein = reinforcement magnitude) on the last trial, using
the log survivor method. Top panel = response initiation rates. Bottom panel = within-
bout rates. Error bars = 95% confidence intervals.
Human schedule performance - 34
0
10
20
30
40
RR RI
Responses per min
100
150
200
250
300
RR RI
Responses per min
... To enhance stimulus control, clicks on one button dimmed the other button (see Podlesnik et al., 2020;Ritchey et al., 2021a). Across all phases and experiments, every click on either the target or alternative button immediately produced a response cost to simulate costs of engaging in responding under natural conditions (see Chen and Reed, 2020;Ritchey et al., 2021a;Shanks & Dickinson, 1995), indicated by (1) a 0.4-s presentation of red text -"− 1 ′′ -below the button, (2) a switch in the point-bar color from gray to red for 0.4 s, and (3) a deduction of $0.00005 (Experiments 1 and 2) or $0.000005 (Experiments 3 and 4) per point lost from total earnings. Clicks on other parts of the interface were recorded but resulted in no programmed consequencessee supplemental materials for details. ...
... Moreover, these findings are also in contrast to our findings with differences in reinforcer rate in Experiment 1. Therefore, the present findings are generally consistent with studies demonstrating that operant behavior is less sensitive to manipulations of reinforcer magnitude than rate in both humans and nonhumans (e.g., Bonem and Crossman, 1988;Catania, 1968;Kuroda et al., 2021;Wurster and Griffiths, 1979; but see Chen and Reed, 2020). ...
... Finally, it is possible that our methods for manipulating reinforcer magnitude were not appropriate for examining this reinforcer variable under the present conditions. The 6-to 30-fold differences in reinforcement magnitude arranged across the present experiments were larger than, for example, the 15-fold differences arranged by Chen and Reed (2020). Chen and Reed nevertheless observed differences in button pressing rate across groups as a result of these differences in reinforcer magnitude. ...
Article
Resurgence occurs when a previously reinforced and then extinguished target response increases due to a worsening of reinforcement conditions for an alternative response. We conducted four crowdsourcing experiments to evaluate effects of alternative-reinforcer rate and magnitude on resurgence with humans. Contingent on an alternative response, we manipulated across groups either the rate of point delivery (Experiment 1) or number of points delivered per reinforcer (Experiments 2-3). Experiment 4 arranged combinations of high- or low-rate and high- or low-magnitude alternative reinforcement across four groups. When extinguishing alternative responding across experiments, we observed resurgence of target responding with relatively high rates of alternative reinforcement but differences in reinforcer magnitude did not influence resurgence. A quantitative model based on the concatenated matching law, Resurgence as Choice in Context (RaC²), provided a poor fit to the data, generally underpredicted target responding, and could not account for data from control groups experiencing extinction of target responding in the absence of alternative reinforcement. We then fit a modified version of RaC² borrowing an assumption from theories of choice that suggest some proportion of reinforcers are misallocated between responses – this modified version of RaC² provided a better account of these findings.
... We arranged a variable-ratio (VR) 6 schedule for one group, consistent with Robinson & Kelley, 2020, and a yoked variable-interval (VI) schedule for the other group. We also examined whether response rates on the VR schedules were greater than the VI schedules in Phase 1 (e.g., Chen & Reed, 2020;Matthews, Shimoff, Catania, & Sagvolden, 1977) and whether there were differences in sensitivity to transitioning to the extinction contingency between VR and VI groups in Phase 2 (e.g., Nevin, McLean, & Grace, 2001). ...
... We observed greater response rates with the VR than the yoked-VI schedule (e.g., Chen & Reed, 2020;Matthews et al., 1977;Reed, Smale, Owens, & Freegard, 2018) and further demonstrate the generality of this finding to adult participants recruited via MTurk. During extinction, we found (1) that 180 s of extinction was effective in significantly reducing responding relative to the preceding reinforcement phase and (2) that levels of responding were not significantly different between the VR and VI groups by the end of Phase 2. In contrast to the findings of Robinson and Kelley (2020), only 40 % of our participants (11 in the VR group and 5 in the VI group) met the 80 %-reduction criterion in Phase 2. Other studies have also shown that humans' responding stabilizes above near-zero levels when extinction is arranged for even longer durations (e.g., 18− 50 min, Novak et al., 2020;Weiner, 1964). ...
... In the present experiment, we examined one approach to potentially facilitate greater sensitivity to an extinction contingency with human participantsresponse cost. Several researchers arranging points as reinforcers also arranged a cost for every response (e.g., Chen & Reed, 2020;Reed, 2019;Shanks & Dickinson, 1991;Weiner, 1964). Several studies have shown response rates were negatively related to the rate (Pietras et al., 2010) and amount (Chen & Reed, 2020) of point loss with ongoing reinforcement contingencies. ...
Article
Full-text available
Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace providing researchers with the opportunity to collect behavioral data from remote participants at a low cost. Recent research demonstrated reliable extinction effects, as well as renewal and resurgence of button pressing with MTurk participants. To further examine the generality of these findings, we replicated and extended these methods across six experiments arranging reinforcement and extinction of a target button press. In contrast to previous findings, we did not observe as reliable of decreases in button pressing during extinction (1) after training with VR or VI schedules of reinforcement, (2) in the presence or absence of context changes, or (3) with an added response cost for button pressing. However, we found that that a 1-point response cost for all button presses facilitated extinction to a greater extent than the absence of response cost. Nevertheless, we observed ABA renewal of button pressing when changing background contexts across phases and resurgence when extinguishing presses on an alternative button. Our findings suggest that MTurk could be a viable platform from which to ask and address questions about extinction and relapse processes, but further procedural refinements will be necessary to improve the replicability of control by experimental contingencies.
... Phase 3 consisted of a 60-s period during which no responses resulted in reinforcement. Across all three phases, clicks on either button also resulted in a 1-point response cost to facilitate performance regulation (Chen and Reed, 2020). ...
Article
Resurgence occurs when a worsening of conditions for an alternative response (e.g., extinction) increases a previously reinforced and subsequently extinguished target response. In contrast, renewal is an increase in a response previously eliminated by extinction following a contextual change. Moreover, arranging contextual changes during resurgence tests has enhanced relapse compared with the absence of contextual changes. Several laboratory studies evaluating resurgence in humans have included interruptions of operant tasks by presenting instructions or “blackouts” immediately prior to phase changes in which reinforcement contingencies change. Such interruptions could enhance relapse during testing similarly to changes in contextual stimuli. We tested this possibility in the present study with human participants recruited via Amazon Mechanical Turk by interrupting the task across groups with 1-s blackouts every 12 s, 60 s, between phases, or by arranging no blackouts in a control group. We found that blackouts prior to phase changes increased resurgence effects relative to no blackouts, suggesting that blackouts between phases are a type of contextual manipulation serving to increase the salience of contingency changes.
... Within the boundary of each workplace, the response button randomly moved in one of four directions Figure 2). There was a response cost for each response (e.g., Chen & Reed, 2020), which was indicated by a 0.4-s presentation of a red "-1" text under the just-clicked button and a simultaneous 0.4-s flash of the bar with red [session, 245, 513-535] (see also Ritchey et al., 2021). ...
Preprint
Full-text available
This is the draft of an article submitted for publication of a tutorial with a demonstration experiment to study operant behavior with crowdsourcing. We describe in detail the procedures used in our research on extinction and relapse of operant behavior recruiting participants through Amazon Mechanical Turk (Ritchey et al., 2021, Learning & Motivation): https://doi.org/10.1016/j.lmot.2021.101728 Therefore, the tutorial can be used both to develop an online operant task and/or interface with Amazon Mechanical Turk. Note the online task can be used with other methods of recruiting (e.g., Prolific, SONA).
... 2017b; Johnson, Pesek, & Christopher Newland, 2009;Shull, 2004;Shull, Gaynor, & Grimes, 2001) and rate of reinforcement (Brackney et al., 2017;Cheung, Neisewander, & Sanabria, 2012;Reed, 2011Reed, , 2015Reed, Smale, Owens, & Freegard, 2018;Shull et al., 2001;Shull & Grimes, 2003;Shull, Grimes, & Bennett, 2004), and w and L are sensitive to changes in contingency requirements (Brackney et al., 2011;Brackney et al., 2017;Brackney & Sanabria, 2015;Chen & Reed, 2020;Reed, 2011;Reed et al., 2018;Shull et al., 2001;Shull & Grimes, 2003;Shull et al., 2004;Tanno, 2016). Thus, the parameters of the microstructure of operant behavior appear to index two of the three necessary conditions for operant performance (Killeen, 1994;Killeen & Sitomer, 2003;Sanabria, 2019): incentive motivation (b) and response-outcome association learning (w and L). ...
Article
Operant behavior is organized in bouts that are particularly visible under variable-interval (VI) schedules of reinforcement. Previous research showed that increasing the work required to produce a response decreases the rate at which bouts are emitted and increases the minimum interresponse time (IRT). In the current study, the minimum effective IRT was directly manipulated by changing the minimum duration of effective lever presses reinforced on a VI 40-s schedule. Contrary to assumptions of previous models, response durations were variable. Response durations were typically 0.5 s greater than the minimum duration threshold; durations that exceeded this threshold were approximately log-normally distributed. As the required duration threshold increased, rats emitted fewer but longer bouts. This effect may reflect an effort-induced reduction in motivation and a duration-induced facilitation of a response–outcome association.
... They are also called bout components. The bout length, the within-bout response rate, and the bout initiation rate are affected by motivational and schedule-type manipulations [4,[6][7][8][9][10][11]. Motivational manipulations include the reinforcement rate, the response-reinforcement contingency, and the deprivation level. ...
Article
Full-text available
Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice—an agent makes a choice between operant and other behaviors. The second mechanism is cost—a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.
Article
Full-text available
Two experiments examined the impact of depression and anxiety on the microstructure of human schedule responding. Human participants responded by pressing a computer key for points on a multiple random ratio (RR) random interval (RI) schedule. The RI schedule was yoked to the RR schedule in terms of reinforcement rate within subject. Overall response rates were higher on the RR compared to the RI schedule. In both experiments, the presence of psychometrically-measured depressive traits reduced overall levels of responding. Depressive traits also decreased within-bout responding, but increased levels of bout-initiation responding. These findings regarding the microstructure of responding were not noted for anxiety traits. These results suggest that depression impacts learning by working through a number of different mechanisms – both impacting the perceived value of the reinforcer, and possibly by creating a bias to attend and process external cues associated with reinforcement which are taken to control response-initiation rates.
Article
Full-text available
An experiment with adult humans investigated the effects of response-contingent money loss (response-cost punishment) on monetary-reinforced responding. A yoked-control procedure was used to separate the effects on responding of the response-cost contingency from the effects of reduced reinforcement density. Eight adults pressed buttons for money on a three-component multiple reinforcement schedule. During baseline, responding in all components produced money gains according to a random-interval 20-s schedule. During punishment conditions, responding during the punishment component conjointly produced money losses according to a random-interval schedule. The value of the response-cost schedule was manipulated across conditions to systematically evaluate the effects on responding of response-cost frequency. Participants were assigned to one of two yoked-control conditions. For participants in the Yoked Punishment group, during punishment conditions money losses were delivered in the yoked component response independently at the same intervals that money losses were produced in the punishment component. For participants in the Yoked Reinforcement group, responding in the yoked component produced the same net earnings as produced in the punishment component. In 6 of 8 participants, contingent response cost selectively decreased response rates in the punishment component and the magnitude of the decrease was directly related to the punishment schedule value. Under punishment conditions, for participants in the Yoked Punishment group response rates in the yoked component also decreased, but the decrease was less than that observed in the punishment component, whereas for participants in the Yoked Reinforcement group response rates in the yoked component remained similar to rates in the no-punishment component. These results provide further evidence that contingent response cost functions similarly to noxious punishers in that it appears to suppress responding apart from its effects on reinforcement density.
Article
Full-text available
Electric shock and time out were compared as punishers in the squirrel monkey. At the parameters investigated, both suppressed responding to about the same degree. Scheduling punishment intermittently or administering pentobarbital reduced the effectiveness of both punishers. The effects of the punishers were different in that responding suppressed by shock recovered more within a session than responding suppressed by time out. Responding was suppressed after some shock-punishment components, but less often after time-out-punishment components. The similarities of the two punishers were more striking than the differences.
Article
Full-text available
The rate of a reinforced response is conceptualized as a composite of engagement bouts (visits) and responding during visits. Part I of this paper describes a method for estimating the rate of visit initiations and the average number of responses per visit from log survivor plots: the proportion) of interresponse times (IRTs) longer than some elapsed time (log scale) plotted as a function of elapsed time. In Part 2 the method is applied to IRT distributions from rats that obtained food pellets by nose poking a lighted key under various multiple schedules of reinforcement. As expected, total response rate increased as a function of (a) increasing the rate of reinforcement (i.e., variable-interval [VI] 4 min vs. VI 1 mi), (b) increasing the amount of the reinforcer (one food pellet vs. four pellets), (c) increasing the percentage of reinforcers that were contingent on nose poking (25% vs. 100%), and (d) requiring additional responses after the end of the VI schedule (i.e., adding a tandem variable-ratio [VR] 9 requirement). The first three of these variables (relative reinforcement) increased the visit-initiation rate. The tandem VR, in contrast, increased the number of responses per visit. Thus, variables that have similar effects on total response rate can be differentiated based on their effects on the componemts of response rate.
Article
Full-text available
Killeen and Hall (2001) showed that a common factor called strength underlies the key dependent variables of response probability, latency, and rate, and that overall response rate is a good predictor of strength. In a search for the mechanisms that underlie those correlations, this article shows that (a) the probability of responding on a trial is a two-state Markov process; (b) latency and rate of responding can be described in terms of the probability and period of stochastic machines called clocked Bernoulli modules, and (c) one such machine, the refractory Poisson process, provides a functional relation between the probability of observing a response during any epoch and the rate of responding. This relation is one of proportionality at low rates and curvilinearity at higher rates.
Article
Full-text available
Humans were presented with a task that required moving a light through a matrix. Button presses could produce light movements according to a multiple fixed-ratio 18/differential-reinforcement-of-low-rate 6-s schedule, with components alternating every 2 min. Moving the light through the maze earned points worth chances on money prizes. In Experiment 1 four conditions were assessed through between-subject comparisons: minimal instructions, instructions to press rapidly, instructions to press slowly, and instructions that sometimes rapid responding would work while at other times a slow rate would work best. Subjects responded in three successive sessions of 32 min each. The results suggested that instructions affected the nature of the contact made with the programmed consequences and thus subsequent performance. In some cases, responding seemed to result from added contingencies introduced by stating rules. In Experiment 2 the relative contribution of these two effects was assessed by presenting and then withdrawing two lights that had been paired with two specific instructions: "Go Fast" or "Go Slow." There were three conditions. In one condition, only the Go Fast light was on; in a second, only the Go Slow light was on; and in a third, the lights alternated each minute. In each condition, half the subjects had all instruction lights turned off after the first session. The results once again showed an effect of instructions on contact with the programmed consequences. However, responding sometimes continued in a manner consistent with added contingencies for rule-following even when the programmed consequences had been contacted and would have controlled a different type of responding in the absence of instructions. The relevance of added contingencies for rule-following in determining the effects of explicitly programmed consequences is emphasized.
Article
The interresponse-time structures of pigeon key pecking were examined under variable-ratio, variable-interval, and variable-interval plus linear feedback schedules. Whereas the variable-ratio and variable-interval plus linear feedback schedules generally resulted in a distinct group of short interresponse times and a broad distribution of longer interresponse times, the variable-interval schedules generally showed a much more continuous distribution of interresponse times. The results were taken to indicate that a log survivor analysis or double exponential fit of interresponse times may not be universally applicable to the task of demonstrating that operant behavior can be dichotomized into bouts of engagement and periods of disengagement.
Article
In the first two experiments, subjects' choices to earn points (exchangeable for money) either by competing with a fictitious opponent or by not competing were studied. Buskist, Barry, Morgan, and Rossi's (1984) competitive fixed-interval schedule was modified to include a second response option, a noncompetitive fixed-interval schedule. After choosing to enter either option, the opportunity for reinforcers became available after the fixed-interval's duration had elapsed. Under the no-competition condition, points were always available after the interval had elapsed. Under the competition condition, points were available based on a predetermined probability of delivery. Experiments 1 and 2 examined how reinforcer probabilities and reinforcer magnitudes affected subjects' choices to compete. Several general conclusions can be made about the results: (a) Strong preferences to compete were observed at high and moderate reinforcer probabilities; (b) competing was observed even at very low reinforcer probabilities; (c) response rates were always higher in the competition component than in the no-competition component; and (d) response rates and choices to compete were insensitive to reinforcer-magnitude manipulations. In Experiment 3, the social context of this choice schedule was removed to determine whether the high levels of competing observed in the first two experiments were due to a response preference engendered by the social context provided by the experimenters through instructions. In contrast to the first two experiments, these subjects preferred the 60-s fixed-interval schedule (formerly the no-competition option), indicating that the instructions themselves were responsible for the preference to compete. This choice paradigm may be useful to future researchers interested in the effects of other independent variables (e.g., drugs, social context, instructions) on competitive behavior.
Article
The differential effects of reinforcement contingencies and contextual variables on human performance were investigated in two experiments. In Experiment 1, adult human subjects operated a joystick in a video game in which the destruction of targets was arranged according to a yoked variable-ratio variable-interval schedule of reinforcement. Three variables were examined across 12 conditions: verbal instructions, shaping, and the use of a consummatory response following reinforcement (i.e., depositing a coin into a bank). Behavior was most responsive to the reinforcement contingencies when the consummatory response was available, responding was established by shaping, and subjects received minimal verbal instructions about their task. The responsiveness of variable-interval subjects' behavior varied more than that of variable-ratio subjects when these contextual factors were altered. Experiment 2 examined resistance to instructional control under the same yoked-schedules design. Conditions varied in terms of the validity of instructions. Performance on variable-ratio schedules was more resistant to instructional control than that on variable-interval schedules.
Article
This study examined the effects of response-produced cost upon human observer responses during extinction following FI reinforcement. Relative to a no-cost condition, cost produced marked and rapid response attenuation.
Article
Four pigeons responded under a two-component multiple schedule of reinforcement. Responses were reinforced in one component under a variable-ratio schedule and in the other component under a variable-interval schedule. It was found that when rates of reinforcement were equal in the two components, the rate of response in the variable-ratio component was nearly twice that in the variable-interval component. Furthermore, for three of the four subjects, the function relating response rate to relative rate of reinforcement in the variable-ratio component had a slope 2.5 to 3 times the slope of the corresponding function for the variable-interval component.