Learning and Motivation

Published by Elsevier
Online ISSN: 1095-9122
Print ISSN: 0023-9690
The olfactory span task (OST) uses an incrementing non-matching to sample procedure such that the number of stimuli to remember increases during the session. The number of consecutive correct responses (span length) and percent correct as a function of the memory load have been viewed as defining rodent working memory capacity limitations in several studies using the OST. However, the procedural parameters of the OST vary across experiments and their effects are not well understood. For example, in several studies, the number of stimuli to remember is confounded with the number of comparison stimuli displayed in the test arena. Experiment 1 addressed whether performance is influenced by the number of comparison choices available on any given trial (2, 5, 10) as well as the number of odor stimuli to remember during a session (12, 24, 36). Performance was most accurate when the number of stimuli to remember was low, as would be expected from a working memory interpretation of OST. However, accuracy was also affected by the number of comparison stimulus choices. High levels of accuracy were seen even with 36 odors, suggesting that the capacity for odor memory in rats was greater than suggested by previous research. Experiment 2 attempted to define this capacity by programming sessions with 36, 48 or 72 stimuli to remember in a group of rats that had previously received extensive OST training. Highly accurate performance (80% correct or better) was sustained throughout the session at even the greatest memory loads, arguing strongly against the notion that the OST models the limited capacity of human working memory. Experiment 3 explored the possibility that stimulus control in the OST is based on relative stimulus familiarity, rather than recognition of stimuli not yet presented during the current session. Number of odor cups visited increased with the number of comparisons in the arena, but rats rarely sampled all of the comparison odors before responding. However, on probe trials which included only stimuli that had been presented during the session, latency to respond and number of comparisons sampled was sharply increased. These data suggest that responding in the OST is determined not just by relative familiarity, but rather by a more specific "what-when" or perhaps "how long ago" form of stimulus control.
People talk to be understood, and so they should produce utterances that are easy for their listeners to understand. I begin this chapter by describing evidence showing that speakers rarely avoid sentences that are ambiguous, even though ambiguity is a factor that is well known to cause difficulty for listeners. Instead, speakers seem to choose utterances that are especially easy for them to say, specifically by producing more accessible, easy-to-think-of material sooner, and less accessible, harder-to-think-of material later. If speakers produce utterances that are easy to say but not utterances that are easy to understand, how is it that we understand each other? A third line of evidence shows that even when sentences are structurally ambiguous, they're likely to include enough information for comprehenders to figure out what they mean. This suggests that speakers produce ambiguous utterances simply because they can -- because the grammar of their language will only let them produce utterances that are unambiguous enough to be understood most of the time anyway. And so, we understand each other because speakers produce utterances efficiently even if they're not optimally understandable; addressees do what they need to to understand their speakers; and the grammar makes sure everything works out properly.
Recovery-from-extinction effects (e.g., spontaneous recovery, renewal, reinstatement, and facilitated reacquisition) have become the focus of much research in recent years. However, despite a great deal of empirical data, there are few theoretical explanations for these effects. This paucity poses a severe limitation on our understanding of these behavioral effects, impedes advances in uncovering neural mechanisms of response recovery, and reduces our potential to prevent relapse after exposure therapy. Towards correcting this oversight, this review takes prominent models of associative learning that have been used in the past and continue to be used today to explain Pavlovian conditioning and extinction, and assesses how each model can be applied to account for recovery-from-extinction effects. The models include the Rescorla-Wagner (1972) model, Mackintosh's (1975) attentional model, Pearce and Hall's (1980) attentional model, Wagner's (1981) SOP model, Pearce's (1987) configural model, McLaren and Mackintosh's (2002) elemental model, and Stout and Miller's (2007) SOCR (comparator hypothesis) model. Each model is assessed for how well it explains or does not explain the various recovery-from-extinction phenomena. We offer some suggestions for how the models might be modified to account for these effects in those instances in which they initially fail.
In two conditioned lick suppression experiments using water-deprived rats, we examined the effects of following Pavlovian conditioned inhibition training (i.e., A-US/AX-NoUS) with pairings of the training excitor (A) and the unconditioned stimulus (US). Experiments 1 and 2 assessed the effects of this posttraining inflation treatment on Pavlovian conditioned inhibition using a summation test and a retardation test, respectively. Both experiments revealed that subjects exposed to inflation treatment demonstrated behavior indicative of enhanced conditioned inhibition compared to subjects that did not receive inflation treatment. The results in conjunction with other recent findings suggest that posttraining associative deflation (i.e., extinction of A) and inflation effects are more symmetrical than has previously been realized.
Three Pavlovian fear conditioning experiments with rats as subjects explored the effect of extinction in the presence of a concurrent excitor. Our aim was to explore this particular treatment, documented in previous studies to deepen extinction, with novel control groups to shed light on the processes involved in extinction. Relative to subjects extinguished on the target CS alone, Experiments 1 and 2 found across a range of parameters that any appreciable effect of facilitated extinction due to the concurrent excitor was submerged by generalization decrement going from extinction to testing. In Experiment 3 we used different durations for the target and concurrent stimuli in order to discourage configuring and an ABC renewal design to increase sensitivity, and observed diminished renewal resulting from extinction in the presence of a second excitor. Our findings suggest that there are distinct limits to the observation of extinction in the presence of an excitor and identifies some of the sources of these limitations.
Rats pressed levers for food delivered by several fixed interval schedules. A drinking spout or running wheel was also available during some conditions, but not during others. The rate of lever pressing, drinking and running often changed within experimental sessions. The within-session patterns of lever pressing did not differ when drinking or running was available and when it was not. The correlation between the amount of lever pressing and the amount of drinking or running at a particular time in the session was inconsistently positive or negative. Finding within-session changes in responding for adjunctive behaviors implies that the factors that produce these changes are present for both adjunctive and instrumental behavior. Finding inconsistent correlations between instrumental responding and adjunctive behaviors questions arousal and interference from adjunctive behaviors as explanations for within-session changes in instrumental responding.
Pigeons show a preference for an alternative that provides them with discriminative stimuli (sometimes a stimulus that predicts reinforcement and at other times a stimulus that predicts the absence of reinforcement) over an alternative that provides them with non discriminative stimuli, even if the non discriminative stimulus alternative is associated with 2.5 times as much reinforcement (Stagner & Zentall, 1910). In Experiment 1 we found that the delay to reinforcement associated with the non discriminative stimuli could be reduced by almost one half before the pigeons were indifferent between the two alternatives. In Experiment 2 we tested the hypothesis that the preference for the discriminative stimulus alternative resulted from the fact that, like humans, the pigeons were attracted by the stimulus that consistently predicted reinforcement (the Allais paradox). When the probability of reinforcement associated with the discriminative stimulus that predicted reinforcement was reduced from 100% to 80% the pigeons still showed a strong preference for the discriminative stimulus alternative. Thus, under these conditions, the Allais paradox cannot account for the sub-optimal choice behavior shown by pigeons. Instead we propose that sub-optimal choice results from positive contrast between the low expectation of reinforcement associated with the discriminative stimulus alternative and the much higher obtained reinforcement when the stimulus associated with reinforcement appears. We propose that similar processes can account for sub-optimal gambling behavior by humans.
During training trials, some rats were injected with apomorphine as the unconditioned stimulus immediately before (Group Forward) or 15 min before (Group Backward) exposure to the conditioned stimulus, a circular alley where locomotor activity was measured for 30 min. Rats in the control condition (Group Unpaired) were not injected until 15 min after their removal from the alley. When the rats were later tested in the circular alley without prior administration of the drug, both Groups Forward and Backward were more active than Group Unpaired and thus showed conditioned hyperactivity. During a final test in which all the rats were injected with apomorphine prior to placement in the circular alley, Groups Forward and Backward showed greater apomorphine-induced hyperactivity than Group Unpaired. Although Group Forward was more active than Group Backward during both tests, these differences were not significant. These findings provide further evidence for the modulation of drug effects by Pavlovian conditioning.
Previous research with the radial maze has found evidence that rats can remember both places that they have already been (retrospective coding) and places they have yet to visit (prospective coding; Cook, Brown, & Riley, 1985). Such dual coding also has been found in pigeons using a radial maze analog in which insertion of a delay at different points during a trial affects performance differentially depending on where in the trial it is inserted. When a delay is interpolated either early or late in a trial, there is minimal disruption of performance compared with when it is interpolated in the middle of the trial. However, the analysis required with this procedure requires the assumption that if errors made on control trials are subtracted from errors made in delay trials, the remaining errors can be directly attributed to the delay. But errors may also be attributed to the changing criterion for making a response as the trial proceeds. Furthermore, the animal's tendency to choose alternatives in a systematic order may also affect its need to remember the sequence of choices made (and yet to be made) on each trial. In the present research, we avoided having to make this assumption by giving the pigeons a two-alternative choice at the time of testing and by randomly determining for the pigeon the order of predelay choices on each trial. This change in procedure resulted in comparable performance as a function of where in the trial the test occurred on both control and delay trials. The effect of the delay was to produce a general decrement in performance independent of where it occurred in the trial.
When animals code stimuli for later retrieval they can either code them in terms of the stimulus presented (as a retrospective memory) or in terms of the response or outcome anticipated (as a prospective memory). Although retrospective memory is typically assumed (as in the form of a memory trace), evidence of prospective coding has been found when response intentions and outcomes are particularly salient. At a more abstract level is the question of whether animals are able figuratively to travel back in time to recover memories of past events (episodic memory) and forward in time to predict future events (future planning). Although what would constitute adequate evidence of episodic memory and future planning is controversial, preliminary evidence suggests that animals may be capable of both forms of subjective time travel.
In experiments that measured food consumption, Holland (1981; Learning and Motivation, 12, 1-18) found that food aversions were formed when an exteroceptive associate of food was paired with illness, but not when such an associate was paired with shock. By contrast, measuring the ability of food to reinforce instrumental responding, Ward-Robinson and Hall (1999; Quarterly Journal of Experimental Psychology, 52B, 335-350) found that pairing an associatively-activated representation of food with shock readily established an aversion to that food. Two experiments considered the origins of these apparently discrepant results. The results did not support either the possibility that instrumental reinforcement power is a more sensitive measure of aversion learning than consumption, nor the hypothesis that illness particularly devalues properties of food representations that determine consumption (such as palatability) whereas shock devalues more general properties critical to reinforcement. The results suggested instead that whereas the effects of pairings of a food associate with illness are mediated by changes in the value of the food itself, the effects of pairings with shock are mediated by the conditioning of fear or other competing responses to the site of food delivery, and not by modification of the value of food itself.
Four experiments with rats assessed conditioning to contextual cues after the delivery of footshocks that were either signaled by a discrete stimulus or unsignaled. Two different tests were used. The first was a context preference test in which subjects were allowed to move freely in a brightly lit, unconditionally aversive context and the former shock context. The second test consisted of scoring freezing behavior while the animals were confined to the former conditioning context. During context preference tests, signaled-shock animals spent more time in the conditioning context and/or entered that context more frequently than did unsignaled-shock subjects. However, freezing tests largely failed to detect a difference between groups. These results were discussed in terms of possible interactions between the formation of context-shock, signal-shock, and context-signal associations and their effect on performance in each of the two types of tests.
Mean volume of SAC consumption (± SEM) after either three CS+US pairings (CTA) or three explicitly unpaired CS/US exposures (NO CTA). The CTA group showed a significant decrease in the amount of SAC consumed over the three exposures. The NO CTA group showed a significant increase in SAC consumption over the same three periods. This indicates that the CTA groups had acquired the CTA, whereas the NO CTA group did not. * = significantly different from the CTA animals (α = 0.05).  
Mean days (± SEM) for animals to reach asymptotic extinction. Animals either underwent the CTA+EU-EXT or CTA+CSO-EXT procedure. The CTA+EU-EXT group took significantly fewer days to extinguish the learned fear than the CTA+CSO-EXT group. * = significantly different from the CTA+CSO-EXT animals (α = 0.05).  
Mean volume of SAC consumption (± SEM) on the day of asymptotic extinction and on the subsequent SR test day for both the CTA+EU-EXT and CTA+CSO-EXT animals. On the day of the final extinction test, the CTA+EU-EXT and CTA+CSO-EXT groups drank comparable amounts of SAC (p > 0.05; see text). The CTA+CSO-EXT group drank significantly more SAC on the last day of extinction than on the day of the SR test, indicating a spontaneous recovery of the CTA. However, the CTA+EU-EXT animals drank nearly the same amount of SAC on the day of extinction as they did on the SR test day, suggesting that the EU-EXT procedure may be effective in blocking SR. * = groups indicated are significantly different, α= 0.05.  
Conditioned taste aversions (CTAs) may be acquired when an animal consumes a novel taste (CS) and then experiences the symptoms of poisoning (US). This aversion may be extinguished by repeated exposure to the CS alone. However, following a latency period in which the CS is not presented, the CTA will spontaneously recover (SR). In the current study we employed an explicitly unpaired extinction procedure (EU-EXT) to determine if it could thwart SR of a CTA. Sprague-Dawley rats acquired a strong CTA after 3 pairings of saccharin (SAC the CS) and Lithium Chloride (LiCl the US). CTA acquisition was followed by extinction (EXT) training consisting of either (a) CS-only exposure (CSO) or, (b) exposure to saccharin and Lithium Chloride on alternate days (i.e., explicitly unpaired: EU). Both extinction procedures resulted in >/= 90% reacceptance of SAC, although the EU extinction procedure (EU-EXT) significantly decreased the time necessary for rats to reach this criterion (compared to CSO controls). Rats were subsequently tested for SR of the CTA upon re-exposure to SAC following a 30-day latency period of water drinking. Rats that acquired a CTA and then underwent the CSO extinction procedure exhibited a significant suppression of SAC drinking during the SR test (as compared to their SAC drinking at the end of extinction). However, animals in the EU-EXT group did not show such suppression in drinking compared to CSO controls. These data suggest that the EU-EXT procedure may be useful in reducing both time to extinction and the spontaneous recovery of fears.
Seventy-two Sprague-Dawley rats were used to investigate the multidimensional features of the context shift effect and the forgetting of stimulus attributes in a Pavlovian differential fear conditioning paradigm. One day after training, a change in either conditioning box or room context resulted in substantial decrements in performance similar to that induced by a shift of both box and room contexts. Two weeks after training, a shift of either box or room context exerted no significant detrimental effects on responding. However, a combined shift of both box and room contexts still induced a severe performance deficit. These results suggest that changes in various components of the stimulus context can result in similar retention deficits, that several aspects of stimulus attributes are forgotten over a delay, and that changing several aspects of the stimulus complex can synergistically impair performance.
Pigeons prefer a positive discriminative (S+) stimulus that follows a less preferred event (a large number of required responses, a longer delay, or the absence of food) over a different S+ with a similar history of reinforcement that follows a more preferred event (a single required response, no delay, or food). We proposed that this phenomenon results from contrast (referred to as within-trial contrast) between the less preferred initial event and the signal for reinforcement. Delay reduction theory (Fantino, 1969) can account for these results by proposing that the less preferred initial event lengthens the duration of the trial, thereby allowing the S+ stimulus to occur later in the trial and thus become a better predictor of reinforcement. In the present experiments, we further explored this effect. In Experiment 1, we controlled for trial duration by using a fixed ratio response (30 pecks) as one initial event and the absence of pecking for the same duration as the other initial event (0 pecks). The pigeons showed a reliable preference for the positive stimulus that followed the least preferred initial event. In Experiment 2, we controlled for trial duration by using 30 pecks as one initial event and 1 peck followed by a delay that matched the duration of the preceding 30-peck trial. (Group Time Same). For Group Time Different, there was no delay following the 1-peck initial event. For Group Time Same, preference for the initial event negatively predicted the pigeons' preference for the S+ stimulus that followed, supporting the contrast account. A somewhat greater preference for the discriminative stimulus that followed the least preferred initial event was found for Group Time Different suggesting that in addition to contrast, delay reduction also may play a small role. However, the greater initial-event preference found for Group Time Different suggests that contrast can account for the group difference as well.
Specifications of dogs participating in the study.
Number of sessions required to reach criterion (see text) or until the termination (in parentheses).
Sketch of the experimental setup (for details see text), seen from above.
All stimulus pairs used in the Stage 2 (2a) and Stage 3 (2b) of the study (see text).
Faces are an important visual category for many taxa, and the human face is no exception to this. Because faces differ in subtle ways and possess many idiosyncratic features, they provide a rich source of perceptual cues. A fair amount of those cues are learned through social interactions and are used for future identification of individual humans. These effects of individual experience can be studied particularly well in hetero-specific face perception. Domestic dogs represent a perfect model in this respect, due to their proved ability to extract important information from the human face in socio-communicative interactions. There is also suggestive evidence that dogs can identify their owner or other familiar human individuals by using visual information from the face. However, most studies have used only dogs' looking behavior to examine their visual processing of human faces and it has been demonstrated only that dogs can differentiate between familiar and unknown human faces. Here, we examined the dog's ability to discriminate the faces of two familiar persons by active choice (approach and touch). Furthermore, in successive stages of the experiment we investigated how well dogs discriminate humans in different representations by systematically reducing the informational richness and the quality of the stimuli. We found a huge inter-individual and inter-stage variance in performance, indicating differences across dogs in their learning ability as well as their selection of discriminative cues. On a group level, the performance of dogs significantly decreased when they were presented with pictures of human heads after having learned to discriminate the real heads, and when - after relearning - confronted with the same pictures showing only the inner parts of the heads. However, as two dogs quickly mastered all stages, we conclude that dogs are in principle able to discriminate people on the basis of visual information from their faces and by making active choices.
We (Lazareva, Freiburger, & Wasserman, 2004) previously trained four pigeons to classify color photographs into their basic-level categories (cars, chairs, flowers, or people) or into their superordinate-level categories (natural or artificial). Here, we found that brief stimulus durations had the most detrimental effect on the basic-level discrimination of natural stimuli by the same pigeons. Increasing the delay between stimulus presentation and choice responding had greater detrimental effect on the basic-level discrimination than the superordinate-level discrimination. These results suggest that basic-level discriminations required longer stimulus durations and were more subject to forgetting than were superordinate-level discriminations. Additionally, categorization of natural stimuli required longer stimulus durations than categorization of artificial stimuli, but only at the basic level. Together, these findings suggest that basic-level categorization may not always be superior to superordinate-level categorization and provide additional evidence of a dissociation between natural and artificial stimuli in pigeons' categorization.
The rate of habituation to food is inversely related to energy intake, and overweight children may habituate slower to food and consume more energy. This study compared patterns of sensitization, as defined by an initial increase in operant or motivated responding for food, and habituation, defined by gradual reduction in responding, for macaroni and cheese and pizza in overweight and non-overweight 8-12 year-old children. Non-overweight children habituated faster to both foods than overweight children (p = 0.03). All children recovered motivated responding for a new food (chocolate). Overweight children consumed more energy than non-overweight children (p = 0.0004). Children who showed a sensitization in responding consumed more food (p = 0.001), and sensitization moderated the effect of overweight on habituation, with slower habituation for overweight children who sensitized (p < 0.0001). This study replicates previous data on overweight/non-overweight differences in habituation of food and of energy intake, and provides new information that individual differences in sensitization and habituation of motivated responding to food cues may be associated with a sustained motivation to eat, resulting in greater energy intake.
In three experiments, four chimpanzees made choices between two visible food options to assess the validity of the selective value effect (the assignment of value to only the most preferred type of food presented in a comparison). In Experiment 1, we established that all chimpanzees preferred single banana pieces to single apple pieces before presenting the critical test. In this test two chimpanzees preferred a mix of one banana piece and one apple piece to a single banana piece when both banana piece were approximately the same size, but two chimpanzees were indifferent between the two options, exhibiting the selective value effect. In Experiment 2, when the banana pieces in both options were more closely equated in size the chimpanzees then were biased to choose the single banana piece over the mixed array even though this was the smaller total amount of food. However, in Experiment 3, when we introduced longer intervals between each trial, the chimpanzees preferred the mixed set and thus the larger total amount of food. The results demonstrate that only some chimpanzees exhibit the choice pattern indicative of the selective value effect, and they do so only when item size is not carefully controlled and trials are presented quickly in succession. Thus, the behavior pattern originally labeled the selective value effect may actually be explained by a combination of chimpanzees' sensitivity to small differences in preferred food amount and chimpanzees tendency to avoid less preferred foods that would delay the acquisition of further preferred food items.
The purpose of the present experiment was to determine whether repeated cocaine exposure differentially affects sucrose-reinforced operant responding in rats raised in an enriched condition (EC) or an isolated condition (IC). Specifically, the performance of EC and IC rats pressing a lever for sucrose under a high fixed-ratio schedule (FR 30) prior to and after 10 days of exposure to cocaine (15 mg/kg, i.p.) or saline was compared. Regardless of rearing condition, rats repeatedly exposed to cocaine had shorter reacquisition latencies to complete a sucrose-reinforced FR 30 task than saline controls. The results suggest that cocaine exposure may have cross-sensitized both EC and IC rats to the reinforcing effects of sucrose or sucrose-associated cues, thus facilitating reacquisition of operant responding.
The present experiment compared the effects of a food-based conditioned inhibitor on food seeking vs. cocaine seeking behavior. In two groups of rats, the A+/AB- Pavlovian conditioned inhibition procedure was used to create a conditioned inhibitor for food. Then, for one group of rats (Food-Food Group), a click stimulus was established as an operant discriminative stimulus (S(D)) for food-reinforced lever pressing. In the other group (Food-Cocaine Group), the click was established as an S(D) for cocaine self-administration. In testing, the putative inhibitor for food was simultaneously presented with the click for the first time in both groups. In the Food-Food Group, the food-based inhibitor suppressed responding occasioned by the click significantly more than did a neutral control stimulus. In contrast, in the Food-Cocaine Group, there was no difference in the amount of suppression produced by the food-based inhibitor and the control stimulus. These results suggest that the effects of food-based Pavlovian conditioned inhibitors are specific for food-motivated behavior and do not easily transfer to cocaine-motivated behavior.
Efforts to develop animal models of memory are critical for understanding the neural substrate of memory. Memory is essential for daily life and enables information to be stored and retrieved after seconds to years. The ability to remember episodes from the past is thought to be related to the ability to plan for the future. Here we focus on a particular aspect of prospective cognition, namely the ability to remember to take action when a future scenario occurs. This review focuses on a recently developed method to evaluate prospective memory in the rat. Available evidence suggests that rats remember to take action in the future, but little is known about the temporal specificity of such memories or about the flexibility and limitations of prospective memories. Recent studies that suggest that rats remember a specific past episode are reviewed to underscore potential approaches that may be used to explore the range and limits of prospective cognition. The review highlights some directions to explore, including the temporal specificity of prospective cognition, the range of flexibility or creativity within prospective cognition, and the constraints imposed by multiple motivational systems.
Three experiments examined the processes mediating rat serial pattern learning for rule-consistent versus rule-violating pattern elements ("violation elements"). In all three experiments, rats were trained to press retractable levers in a circular array in a specific sequence for brain stimulation reward (BSR). Experiment 1 examined the role of lever location (L) and element serial position (SP) cues in rats' ability to learn to anticipate a violation element positioned at the end of a 24-element serial pattern. Rats with L cues either alone or in combination with SP cues learned to anticipate the violation element, whereas those with SP cues alone did not. Rats in groups L and L+SP underwent a series of transfers designed to remove various cues that might have controlled their performance on the violation element. Results indicated that intra-chamber lever location cues mediated performance on the violation element whereas performance on rule-consistent elements within pattern chunks was mediated by an internal mnemonic representation that was insensitive to changes in lever location cues. Experiment 2 examined whether rats could learn to use SP cues alone to anticipate a violation element if it was positioned earlier in a serial pattern. Rats learned to anticipate the violation element based on SP cues alone when it was located in SP6 in a 24-element pattern, but not when it was in SP12. Experiment 3 examined whether or not rats spontaneously encode information about chunk length and the serial position of phrasing cues in serial patterns. Rats were trained to a high criterion on the serial pattern used in Experiment 1, then were challenged with three probe patterns that manipulated both chunk length and overall pattern length. Results indicated that rats spontaneously encoded information regarding the serial position of phrasing cues in relation to chunk length. Thus, rats appear to use at least three cognitive processes concurrently in serial pattern learning tasks, namely, item memory involving external discriminative cues, counting- or timing-like processes for encoding serial position, and rule abstraction for encoding an internal representation of pattern structure.
The variables of delay and effort have been found to influence self-control predictably and in similar fashion when tested independently, but it is unclear how they influence self-control interactively. In the present study, I tested these 2 variables simultaneously to gain better understanding of their combined influence on self-control. A computerized task was employed in which monkey participants could sequence 1 or more digital images before "cashing-out," after which they would receive their accumulated rewards. Delay was manipulated by adjusting the speed of the cursor used to select images. Cognitive effort was manipulated by presenting image sets that appeared in either a constant or a randomized configuration. For most monkeys, an interaction was found between the effects of delay and effort on the number of images selected before cashing-out. The results suggest that, when combined, these 2 variables have a complex influence on self-control.
Two lick suppression studies were conducted with water-deprived rats to investigate the influence of spatial similarity in cue interaction. Experiment 1 assessed the influence of similarity of the spatial origin of competing cues in a blocking procedure. Greater blocking was observed in the condition in which the auditory blocking cue and the auditory blocked cue originated at the same spatial location. Recent investigations have demonstrated that manipulations that impact competition between cues trained together have similar effects on interference between cues trained apart. Therefore, Experiment 2 investigated the influence of similarity of the spatial origin in proactive interference of Pavlovian conditioning by separately pairing two auditory cues with a common outcome, originating at the same spatial location or different spatial locations. Greater proactive interference was observed in the condition in which the interfering cue and target cue originated at the same spatial location. The results are considered in light of the possibility that a similar mechanism may underlie interference between cues trained apart and cue competition between cues trained together.
Four experiments used a conditioned taste aversion procedure to examine the potential for CS-alone extinction treatment to produce a conditioned stimulus that possesses inhibitory properties. In Experiment 1, saccharin was paired with LiCl, and then saccharin was presented alone for several trials to produce extensive behavioral extinction. Animals receiving this treatment were retarded in reacquiring conditioned responding to saccharin relative to control subjects receiving conditioning to the flavor for the first time. In Experiment 2, the extinguished saccharin stimulus was shown to decrease conditioned responding to a known excitor when the two stimuli were presented in compound as a summation test. Experiments 3A and 3B replicated the findings of Experiments 1 and 2 while providing evidence that the effects were not due to the differential effects of neophobia during testing. These three experiments revealed that an extinguished conditioned excitor passes retardation and summation tests for conditioned inhibition. Experiment 4 found that extinction of a known excitor was slowed when the excitor was extinguished in compound with a previously extinguished conditioned stimulus. That is, an extinguished CS provided protection from extinction to another CS, a finding also consistent with the view that extinction produces conditioned inhibition.
Three experiments were conducted to examine the role of novel contextual stimuli in producing the unconditioned stimulus (US) preexposure effect. Experiment 1 demonstrated that novel contextual stimuli produce a significantly stronger US preexposure effect than familiar or "latently inhibited" contextual stimuli. Moreover, subjects preexposed in the presence of latently inhibited contextual cues failed to show a significant US preexposure effect. Experiments 2 and 3 attempted to provide evidence that the addition of a single novel stimulus to the latently inhibited context would result in a significantly stronger US preexposure effect than when no such novel cue was present. Experiment 3 was able to demonstrate this effect. Results are consistent with the Rescorla-Wagner (1972) model of conditioning.
Three experiments with rat subjects examined resurgence of an extinguished instrumental response using the procedure introduced by Epstein (1983) with pigeons. There were three phases: (1) initial acquisition of pressing on a lever (L1) for pellet reward, (2) extinction of L1, and (3) a test session in which a second lever (L2) was inserted, briefly reinforced, and then extinguished. Experiment 1 confirmed that if pressing L2 delivered 20 pellets followed by extinction, rats would resume L1 responding in the final test. Experiment 2 compared the effects of response-contingent and non-contingent rewards delivered upon insertion of L2. Although insertion of L2 alone did not increase L1 responding, response-contingent and non-contingent rewards led to comparable increases in L1 responding. Experiment 3 found that the delivery of non-contingent pellets during extinction of L1, which would be expected to reduce the ability of pellets to set the occasion for the L1 response, also reduced the effects of both response-contingent and non-contingent rewards during the final test. The results indicate that in this method, the resurgence treatment leads to an increase in L1 pressing due to simple presentation of the pellet; delivering the reinforcer after extinction of L1 reinstates L1 responding by setting the occasion for the L1 response.
The present studies examined whether the retrieval of an old 'reactivated' memory could be brought under the control of new contextual cues. In Experiment 1 rats trained in one context were exposed to different contextual cues either immediately, 60 min, or 120 min after a cued reactivation of the training memory. When tested in the shifted context, subjects exposed shortly after reactivation treated the shifted context as the original context. This transfer diminished with longer post-reactivation delays. Experiment 2 replicated the basic finding and demonstrated that the transfer of the old retrieval cues was specific to the contextual cues present during exposure. These findings are consistent with previous research (i.e., Briggs, Fitz, & Riccio, in press) showing the transfer of retrieval cues for a new memory, and demonstrating a similarity (in this case) between newly acquired and old reactivated memories.
Recent studies have demonstrated that the expectation of reward delivery has an inverse relationship with operant behavioral variation (e.g., Stahlman, Roberts, & Blaisdell, 2010). Research thus far has largely focused on one aspect of reinforcement - the likelihood of food delivery. In two experiments with pigeons, we examined the effect of two other aspects of reinforcement: the magnitude of the reward and the temporal delay between the operant response and outcome delivery. In the first experiment, we found that a large reward magnitude resulted in reduced spatiotemporal variation in pigeons' pecking behavior. In the second experiment, we found that a 4-s delay between response-dependent trial termination and reward delivery increased variation in behavior. These results indicate that multiple dimensions of the reinforcer modulate operant response variation.
Rats were trained on a delayed successive matching-to-stimulus modality task consisting of onset of chamber lights or a tone for the sample stimulus, S1, and a comparison stimulus, S2. Lever pressing to S1 was reinforced as was lever pressing to its matching S2 (light-light or tone-tone pairs) but not to its mismatching S2 (light-tone, tone-light pairs). The interval between S1 and S2 within a trial, the retention interval (RI), was varied between 1 and 6 s within sessions while the interval between S2 and the next S1, the intertrial interval (ITI), was reduced from 24 to 12 s and finally to 6 s over blocks of sessions. In Experiment 1, where S1 was kept at 2 s and S2 at 10 s, rats' matching accuracy declined over the longer RI, was slightly disrupted as ITIs were reduced to 6 s only over 1-s RIs, and was generally poorer to the tone than light S1. In Experiment 2, where both stimuli were 10 s increasing RIs caused steeper declines in matching accuracy to the tone S1 than light S1 and decreasing ITIs to 6 s disrupted rats performance over both RIs. Matching accuracy to the tone but not to the light S1 was also poorer when a preceding trial's S2 was a light S2 than when it was a tone S2 only during Experiment 2. This intertrial stimulus disagreement effect was not influenced by ITI duration. These results suggest that intertrial proactive interference in delayed matching tasks consists of two separate and independent processes in rats similar to those found in pigeons (Edhouse & White, 1988).
Individual vocal recognition behaviors in songbirds provide an excellent framework for the investigation of comparative psychological and neurobiological mechanisms that support the perception and cognition of complex acoustic communication signals. To this end, the complex songs of European starlings have been studied extensively. Yet, several basic parameters of starling individual vocal recognition have not been assessed. Here we investigate the temporal extent of song information acquired by starlings during vocal recognition learning. We trained two groups of starlings using standard operant conditioning techniques to recognize several songs from two conspecific male singers. In the first experiment we tested their ability to maintain accurate recognition when presented with (1) random sequences of 1-12 motifs (stereotyped song components) drawn from the training songs, and (2) 0.1 - 12-s excerpts of continuous song drawn from the training songs. We found that song recognition improved monotonically as more vocal material is provided. In the second experiment, we systematically substituted continuous, varying length regions of white noise for portions of the training songs and again examined recognition accuracy. Recognition remained above chance levels for all noise substitutions tested (up to 91% of the training stimulus) although all but the smallest substitutions led to some decrement in song recognition. Overall, above chance recognition could be obtained with surprisingly few motifs, short excerpts of song, and in the absence of large portions of the training songs. These results suggest that starlings acquire a representation of song during individual vocal recognition learning that is robust to perturbations and distributed broadly over large portions of these complex acoustic sequences.
Reinforcing value and habituation are two processes that have been used to study eating behaviors, but no research has examined their relationship, how they relate to energy intake, and whether they respond in a similar manner to food deprivation. Twenty-two female subjects were randomized to food deprived or non-deprived conditions, and assessed for food reinforcement, habituation to food and ad libitum eating. Results showed food reinforcement and habituation are correlated (r = 0.62, p = 0.002) and both independently predict energy intake. Hierarchical regression showed that the rate of habituation accounted for 30 percent of the variance in eating (p = 0.008), and adding food reinforcement increased the amount of variance accounted for up to 57.5 percent (p < 0.05). This suggests that both processes may influence energy intake in a meal.
In resurgence, an operant behavior that has undergone extinction can return ("resurge") when a second operant that has replaced it itself undergoes extinction. The phenomenon may provide insight into relapse that may occur after incentive or contingency management therapies in humans. Three experiments with rats examined the impact of several variables on the strength of the resurgence effect. In each, pressing one lever (L1) was first reinforced and then extinguished while pressing a second, alternative, lever (L2) was now reinforced. When L2 responding was then itself extinguished, L1 responses resurged. Experiment 1 found that resurgence was especially strong after an extensive amount of L1 training (12 as opposed to 4 training sessions) and after L1 was reinforced on a random ratio schedule as opposed to a variable interval schedule that was matched on reinforcement rate. Experiment 2 found that after 12 initial sessions of L1 training, 4, 12, or 36 sessions of Phase 2 each allowed substantial (and apparently equivalent) resurgence. Experiment 3 found no effect of changing the identity of the reinforcer (from grain pellet to sucrose pellet or sucrose to grain) on the amount of resurgence. The results suggest that resurgence can be robust; in the natural world, an operant behavior with an extensive reinforcement history may still resurge after extensive incentive-based therapy. The results are discussed in terms of current explanations of the resurgence effect.
Three experiments with rats examined reacquisition of an operant response after either extinction or a response-elimination procedure that included occasional reinforced responses during extinction. In each experiment, reacquisition was slower when response elimination had included occasional reinforced responses, although the effect was especially evident when responding was examined immediately following each response-reinforcer pairing during reacquisition (Experiments 2 and 3). An extinction procedure with added noncontingent reinforcers also slowed reacquisition (Experiment 3). The results are consistent with research in classical conditioning (Bouton, Woods, & Pineño, 2004) and suggest that rapid reacquisition after extinction is analogous to a renewal effect that occurs when reinforced responses signal a return to the conditioning context. Clinical implications are also discussed.
Four rhesus monkeys (Macaca mulatta) were tested on joystick-based computer tasks in which they could choose to be reinforced either with pellets-only or with pellets + video. A variety of videotapes were used to reinforce task performance. The monkeys significantly preferred to be rewarded with a pellet and 10 s of a blank screen than a pellet plus 10 s of videotape. When they did choose to see videotaped images, however, they were significantly more likely to view video of themselves than video of their roommate or of unfamiliar conspecifics. These data support earlier findings of individual differences in preference for video reinforcement, and have clear implications for the study of face-recognition and self-recognition by nonhuman primates.
We investigated the time course of spatial-memory decay in rats using an eight-arm radial maze. It is well established that performance remains high with retention intervals as long as 4 hr, but declines to chance with a 24-hr retention interval (e.g., Beatty & Shavalia, 1980b). It is possible that 24 hr reflects a genuine retention limitation of rat spatial memory. Alternatively, it may be possible to identify factors that might support memory performance even after very long delays. The current experiment was conducted to test the above two hypotheses. We evaluated performance using two intertrial intervals (24 and 48 hr) and two retention intervals (1 and 25 hr). Increasing the intertrial interval produced an approximately constant increase in performance for both retention intervals. This improvement is consistent with a trial-spacing effect (i.e., the superiority of spaced over massed trials). Rat spatial memory apparently lasts at least 25 hr.
Three experiments examined the effect of reinforcement magnitude on free-operant response rates. In Experiment 1, rats that received four food pellets responded faster than rats that received one pellet on a variable ratio 30 schedule. However, when the food hopper was illuminated during reinforcer delivery, there was no difference between the rates of response produced by the two magnitudes of reward. In Experiment 2, there was no difference in response rates emitted by rats receiving either one or four pellets of food as reward on a random interval (RI) 60-s schedule. In Experiment 3, rats responding on an RI 30-s schedule did so at a lower rate with four pellets as reinforcement than with one pellet. This effect was abolished by the illumination of the food hopper during reinforcement delivery. These results indicate that the influence of magnitude is obscured by manipulations which signal the delivery of reinforcement.
Rats' learning about visual patterns was studied in a computerized Y-maze where wide-angle stimuli were viewed from a distance. Many patterns were available; some were spatially complex and others were more homogeneous figures. Experiment 1 used a discrimination paradigm in which a single S+ could be paired with any one of 15 different S-s. Hooded rats learned successively six such discrimination problems. Their learning rate improved across the series, and comparison with controls suggested that the learning-set did not merely reflect simple habituation. Experiments 2 and 3 employed Dark Agouti rats, again learning many discrimination problems. Each problem comprised a constant stimulus which was paired with stimuli which varied in trial-unique fashion. The version in which the constant stimulus was nonrewarded (S-) and the varying stimuli rewarded was performed better than the converse, constant S+ and varying nonrewarded, reflecting rats' preference for relatively unfamiliar stimuli. In the constant S- task, rats showed substantial within-problem learning when three novel problems were given per day for 20 trials each. Rats are capable of rapid learning about complex visual displays if we engage their natural dispositions to use vision for distal stimuli and to approach relatively unfamiliar cues.
In a transfer-of-control experiment with rats, Pavlovian CSs were tested for the specificity of their effects. The instrumental behavior consisted of a discriminative, conditional two-lever choice task in which qualitatively different appetitive reinforcers were contingent upon the two correct choices. In a Pavlovian phase, subjects experienced conditioning to establish either a CS+ or CS− for one reinforcer or a CS+ or CS− for the other reinforcer. Finally, in a test, these CSs were presented when there was the opportunity to make choice responses. The CS+s evoked choices of the lever which had eventuated in the reinforcer that had served as the Pavlovian US, while the CS−s showed only a slight tendency to evoke the other choice responses. When the CSs were compounded with the original SDs, the CS+s had little effect upon the vigor of responding while the CS−s reduced the vigor of responding to the SD for the reinforcer that was the same as the US used in establishing the CS−. The results are discussed in terms of associative mediational theory and the reinforcer specificity of Pavlovian conditioned excitation and inhibition.
The present experiments investigated how fluid aversions conditioned with LiCl may differ from those conditioned with shock. In the first experiment, different groups received a saccharin/vanilla fluid either paired with an injection of LiCl or paired with tongue shocks. Following aversive conditioning, rats then received the saccharin/vanilla fluid pitted against water in repeated nonreinforced two-bottle tests for an aversion. Prior to each extinction test, different groups received injections either of an anxiolytic agent, i.e., chlordiazepoxide (CDP; 9 mg/kg), or the physiological saline vehicle. Only the group conditioned with tongue shock evidenced a taste aversion that was weakened by chlordiazepoxide. The second experiment included nonassociative control groups and increased the number of CS preexposures prior to conditioning with LiCl and the CDP extinction test, in an attempt to rule out some interpretations of this result. Data from this experiment indicated that CDP enhanced the level of aversion shown in the LiCl-conditioned group; it also demonstrated that CDP unconditionally suppressed intake. The nature of these selective attentuating and enhancing effects of CDP upon different fluid aversions was discussed in light of the assumption that shock and LiCl activate motivationally distinct aversive states.
Field and Davey (1997) claimed that evaluative conditioning (EC), rather than being a genuine type of Pavlovian associative learning, represents nothing but an experimental artifact. They come to this conclusion by first identifying the prototypical experimental procedure of EC research, by next pointing to the potential methodological shortcomings of this prototypical procedure in order to infer associative learning, and by finally demonstrating that when the appropriate controls are added to this prototypical procedure, it becomes clear that apparent EC results represent nothing but an experimental artifact. In this paper, we first demonstrate that what Field and Davey identify as the “prototypical procedure” is, as a matter of fact, the exception rather than the rule in EC research, such that the potential scope of their criticism is rather limited. Next, we show how the results of their own experiment are actually due to an artifact that is not present in any EC study. Finally, we discuss how Field and Davey's criticism of EC research has its roots in a misconception of the appropriate within or between subject controls for Pavlovian associative learning.
The present report considers alternative measures of sensitivity and response bias for the discrimination learning paradigm. The classical signal detection measures, d′ and β, were compared with their nonparametric equivalents, A′ and B″, with theoretical measures derived from threshold theory and with empirical measures derived from the ROC graph. Differential rabbit eyelid conditioning data from three experiments were analyzed with these measures, and the results of these analyses were used along with other information to determine which measures of sensitivity and response bias are most useful for the analysis of discrimination learning data.
An unsignaled, escapable shock was presented contingent on an avoidance response. Fischer 344 rats responded less to the warning signal in proportion to its temporal distance from the avoidance response. Partial contingency effects were further obtained by variation in the instrumental conditioning space for an aversive stimulus. However, the arbitrary omission of an imminent shock on half the trial in which the rats failed to avoid a shock, led to little avoidance acquisition, and shock-frequency reduction was thus not sufficient to produce the acquisition of the avoidance response. Because early avoidance responses were initiated by escape from shock, a stimulus contingency may be essential for response initiation, and an explicit response contingency is important in maintaining successful avoidance responses.
Group mean suppression in Experiment 2. The data are shown in the same six groupings as in Fig. 1, except that the last trial of extinction is depicted as Trial 48. Group designations O, OB, and OSB (odor, side of room, and box) represent features that distinguished Contexts A and B. Error bars show AE 1 SEM.
Mean suppression in Experiment 4. The first five groupings of data resemble those in Figs. 1–3. From left to right, the last three groupings show, respectively, the renewal data for Groups AAC, ABC, and CAC. Here, open symbols show the results of testing for renewal in the extinction context, where no renewal was expected, and solid symbols show the results of testing for renewal in Context C, where renewal was expected. For each group, the difference between its open and solid symbols provides a within-subject measure of renewal. Error bars show AE 1 SEM.  
Using barpress conditioned suppression, we studied the renewal of conditioned fear in rats, an animal model for the relapse of human fears and phobias. We demonstrated ABA renewal when the only differences between Contexts A and B included (1) their odor, (2) their location (i.e., side of room), and (3) unintended differences between copies of the same box at the two sites. Removing either the odor or location cues abolished the renewal effect. We then directly compared the effects of ABA and AAB procedures under two levels of context similarity. Although AAB renewal occurred, ABA renewal was stronger. Adding multiple context distinctions to the three listed above did not significantly enhance either form of renewal. Finally, we directly compared the strengths of AAB, ABC, and ABA renewal. AAB renewal, though again significant, was weaker than ABA and ABC renewal, which did not differ significantly. Fear renewal (relapse) can thus be reduced by extinguishing the fear in the acquisition context, regardless of the nature of the test context.
Transfer summation results of Experiment 3. Error bars show AE 1 SEM.
Group mean suppression on the last trial of extinction (L) and Test Trials 1-3 in Experiments 4a and 4b combined. Error bars show AE 1 SEM.
In four experiments using albino rats in an ABA fear renewal paradigm, we studied conditioned fear in the A test context following extinction in Context B. Conditioned suppression of operant responding was the index of fear. In Experiments 1–3, we found that extinguishing a feared cue in compound with a putative conditioned inhibitor of fear led to more fear in the test context than did a conventional extinction procedure. In Experiments 4a and 4b, we found that extinguishing three feared cues in compound required one third the time and generally led to less fear to the cues in the test context than did the extinction of each cue separately. Thus, fear in the test context seems to vary inversely with the values of co-present cues during extinction in Context B. Results imply that cue value is actually reduced by extinction procedures rather than merely being opposed by a growing inhibitory process. Implications for theories of renewal and for clinical practice are discussed.
A continuous chain of homogeneous responding was established in rats by training animals to hold a lever down for 10 sec or longer before releasing it for food reinforcement. When criterion releases were subsequently punished, completed holding chains were greatly suppressed, aborted chains increased markedly, while the rate of chain initiations remained unchanged.
In Experiment 1, rats were given a 1-pellet reward for 48 preshift trials. During a subsequent 20-trial postshift phase, one group was shifted to a 12-pellet reward on Trial 1, a second was shifted on Trial 11, and a third was given 1 more pellet each trial and then 12 pellets for the last 10 trials. The speeds of all three groups increased to a level above that of a control group given a 12-pellet food reward throughout training (positive contrast). In experiment 2, rats were shifted from 1 to 12 pellets either gradually or abruptly following either abbreviated training (9 trials) or extended training (20 trials). One group of control subjects received 12 pellets throughout training. The results revealed a positive contrast effect for gradually shifted subjects following extended training but not following abbreviated training. The abrupt shift procedure produced positive contrast following abbreviated training but only a marginal effect following extended training. These results indicate that, contingent upon the amount of preshift training, either gradual or abrupt reward increases may produce positive contrast.
Mean skin conductance responses (left panel) and blink magnitude modulation (right panel) during CS and CS for verbalizers and nonverbalizers (vertical bars represent standard errors of the means).  
Mean blink magnitude modulation (left panel) and latency shortening (right panel) during CS and CS in verbalizers before and after verbalization (vertical bars represent standard errors of the means).  
The proposal that affective learning, the learning of likes and dislikes, can exist in the absence of contingency awareness, whereas signal learning, the learning of stimulus relationships, cannot, was investigated in a differential conditioning paradigm that was embedded in a visual masking task. Startle magnitude modulation and changes in verbal ratings served as measures of affective learning, whereas skin conductance was taken to reflect signal learning. Awareness was assessed online with an expectancy dial and in a postexperimental questionnaire. Both between-subject comparisons of verbalizers and nonverbalizers and within-subject comparisons of verbalizers before and after verbalization failed to reveal any evidence for learning, whether affective or otherwise, in the absence of knowledge of the stimulus contingencies.
Top-cited authors
Robert A Boakes
  • The University of Sydney
John Prescott
  • (1) TasteMatters Research & Consulting
Peter D Balsam
  • Barnard College and Columbia University
Manuel M. Ramos Alvarez
  • Universidad de Jaén
Anja Dieckmann
  • Nuremberg Institute for Market Decisions