Figure - uploaded by Jeffrey Podos
Content may be subject to copyright.
The subjective value of a delayed reward. The discounted value of a delayed reinforcer is modeled by a hyperbolic function. Here f(delay) = (1 + K × delay)−1 where “K” defines the shape of the discounting curve (green: K = 0.25, black: K = 0.5, red: K = 2.0). Higher K-values (red) are characteristic of greater impulsive choice. In each case, the average of the values of the variable options—shown, for the black function, in dotted lines—is greater than the value of their average which is shown as a solid line. This difference (Jensen's inequality) holds for any concave up function, but the difference is greater for functions with higher K-values. In other words, the difference between the variable and fixed delay options is greatest for the red line, moderate for the black line, and least for the green line.
Source publication
Impulsive choice-the preference for small immediate rewards over larger delayed rewards-has been linked to various psychological conditions ranging from behavioral disorders to addiction. These links highlight the critical need to dissect the various components of this multifaceted behavioral trait. Delay discounting tasks allow researchers to stud...
Contexts in source publication
Context 1
... specifically, while the value of a reward continues to decrease as the delay increases, the greatest decrease in value is during the initial delay. This delay dis- counting relationship can be seen in Figure 1 where the greatest decrease in reward value occurs on the left side of the plot. For any non-linear function such as this, the average result of the func- tion applied to two delay values differs from the result of the same function applied to the average of the two delays. ...Context 2
... any non-linear function such as this, the average result of the func- tion applied to two delay values differs from the result of the same function applied to the average of the two delays. To illustrate, the average discounted value of a delay alternating between 2 and 18 s (dotted lines in Figure 1) is greater than the discounted value of the same reward delayed 10 s (solid line). This mathematical property, known as Jensen's inequality (Jensen, 1906), is relevant to any study of variable properties and has been finding increas- ingly broad applicability within biology (Smallwood, 1996;Ruel and Ayres, 1999). ...Context 3
... there is a general trend to prefer variable delays, the strength of this preference can differ greatly between individu- als. Differences in the sharpness of the delay discounting curve (Figure 1) can be used to describe these individual differences. For illustrative purposes, Figure 1 shows the delay functions of three hypothetical subjects: red, black, and green. ...Context 4
... in the sharpness of the delay discounting curve (Figure 1) can be used to describe these individual differences. For illustrative purposes, Figure 1 shows the delay functions of three hypothetical subjects: red, black, and green. All three sub- jects would value a variable delay over the fixed delay, but the red subject presents a strong preference for the variable option (more impulsive), while the green subject presents a weaker preference for variability (less impulsive). ...Context 5
... described earlier, delay discounting follows a hyperbolic decay (Figure 1) modeled by Equation 1 ( Bateson and Kacelnik, 1998). The above method of testing impulsive choice is modeled by Equation 2: the delay on the stable lever is reduced by an off- set (x) such that the animal assigns an equal value to this stable delay as to the average of the variable delay options. ...Context 6
... VTA relays this signal to the nucleus accumbens (NAcc) (Glimcher, 2011) where the higher levels of DAT activ- ity are associated with greater delay discounting. This dopamine reuptake has been suggested to underlie decaying reward value curves-such as those in Figure 1-and thus individual differ- ences in DAT in the NAcc could create differences in the resulting delay discounting curves. Many components of the dopamine system in addition to DAT correlate with impulsive choice in adolescent populations, but DAT and the dopamine D4 recep- tor show the most consistent associations ( Nemoda et al., 2011). ...Similar publications
Nowadays the smartphone plays an important role in our lives. While it brings us convenience and efficiency, its overuse can cause problems. Although a great number of studies have demonstrated that people affected by substance abuse, pathological gambling, and internet addiction disorder have lower self-control than average, scarcely any study has...
Citations
... The relation between the subjective value of a reinforcer and the delay until its receipt can be described by a discounting function. Such behavior may be based on the peculiarities of time perception [125][126][127]. In situations requiring a waiting period, impulsive behavior, as an increase in SS reward preference, can be provoked by a tendency to overestimate forthcoming time intervals. ...
Time perception is a fundamental cognitive function essential for adaptive behavior and shared across species. The neural mechanisms underlying time perception, particularly its neuromodulation, remain debated. In this review, we examined the role of the serotonergic system in time perception (at the scale of seconds and minutes), building a translational bridge between human and non-human animal studies. The literature search was conducted according to the PRISMA statement in PubMed, APA PsycINFO, and APA PsycARTICLES. Sixty papers were selected for full-text review, encompassing both human (n = 10) and animal studies (n = 50). Summarizing the reviewed literature, we revealed consistent evidence for the role of serotonin in timing behavior, highlighting its complex involvement across retrospective, immediate, and prospective timing paradigms. Increased serotonergic activation appears to accelerate internal time speed, which we interpret through the dual klepsydra model as accelerated discharge of the temporal accumulator. However, some findings challenge this framework. Additionally, we link impulsivity—associated with decreased serotonergic functioning in our review—to a slower internal time speed. Variability in prospective timing tasks underscores the need for further research into how serotonin modulates reward-based temporal decisions, using novel approaches to disentangle internal time speed, response inhibition, and other factors.
... lead to reduced temporal discounting. Indeed, rats with more precise timing have been shown to also be less impulsive (McClure et al., 2014). Exposure to tasks that necessitate waiting also appears to decrease impulsivity (Bailey et al., 2018;Fox et al., 2019;Smith et al., 2015;Stuebing et al., 2018). ...
When foraging for food, animals must balance levels of exploitation and exploration. Too much exploitation leads to an exhaustion of the source and means one will never know if there are better sources of food elsewhere. Too much exploration and one will never reap the benefits from a source. Recently, substantial evidence has implicated this tradeoff in human sequential decision-making, such as when choosing which restaurant to dine at. Computational models of optimal foraging have been used to describe both foraging and sequential decision-making, suggesting a common substrate. Importantly, these models factor in the opportunity cost of (travel) time by assuming perfect temporal cognition. Here, we apply a recently proposed intertemporal choice model that mixes rate distortion theory and hyperbolic discounting to estimate people’s expected opportunity cost of time. Assuming that the adaptive control of temporal cognition is crucial for balancing exploration and exploitation, we hypothesized that model parameters would be able to predict childhood adversity, as the type of environment one was raised in has been shown to explain differences in explore-exploit behavior.
... Adolescence is a sensitive period for the onset of mental disorders (Fuhrmann et al., 2015). Changes on a behavioral level during the time of adolescence, such as investing in peer bonding and involvement in high-risk behavior (McClure et al., 2014;Romer, 2010;Whelan et al., 2012), are associated with the rise in sex hormones (Forbes and Dahl, 2010) One explanation for this change in behavior can be linked to brain development. In this line, adolescence is a vulnerable time for brain development involving synaptic pruning and maturation of the brain, especially the prefrontal cortex which is involved in cognitive functioning (Herting et al., 2015). ...
... D2R upregulation in NAc CINs does not alter timing. The ability to accurately represent the time it takes to receive a reward following a press is a key behavioral sub-component in delay discounting tasks [48]. Thus, it is conceivable that CIN D2R upregulation results in an overestimation of time intervals, thereby reducing tolerance of delays compared to controls. ...
... Indeed, individuals deemed impulsive on delay discounting tasks are more prone to timing errors compared to control subjects [85]. Similarly, rats that showed higher timing precision in peak interval and temporal discrimination tasks also show reduced delay-based impulsivity [48,86]. Multiple studies also support a role of dopamine in timing [87][88][89]. ...
Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.
... Even with the robust trait-like characteristics of DD, comparisons of DD across the lifespan have yielded mixed results. While several studies show a decline in DD as individuals age from adolescence to adulthood and from young adulthood to older adulthood [29][30][31][32][33][34][35][36], other research demonstrates no difference in DD across the lifespan [32,34,[37][38][39][40], or even an increase in DD [41]. Further, very few reports test the same subjects across development, making it difficult to determine if DD is a stable lifelong trait within subjects and thus if DD phenotypes (e.g., high vs. low levels of choice impulsivity) convey similar risk to the development of SUD across life stages. ...
... Subjects were male Long-Evans rats bred at the University of Maryland School of Medicine. Many studies investigating sex differences in intertemporal choice show that male and female rodents and humans exhibit similar levels of DD [78,79], have conserved corticostriatal activity during task performance [47], and express indistinguishable test-retest reliability [41,80]. Adolescent rats (PND 30-55) were group-housed, and adult rats (PND 55+) were pair-housed until stereotaxic surgery in adulthood (PND~120-130); they were then singly housed to protect surgical implants. ...
... Relative stability of DD from adolescence to adulthood aligns with a wide body of research demonstrating long-term, stable "trait-like" levels of impulsive choice in humans and animals [14,19,20,41]. Although it is possible that DD behavior observed here is not the result of trait-like choice impulsivity but rather due to fixed patterns of responding on the task established through extended training, the propensity for DD behavior to become fixed or habitual over training is currently unexplored. ...
Impulsive choice has enduring trait-like characteristics and is defined by preference for small immediate rewards over larger delayed ones. Importantly, it is a determining factor in the development and persistence of substance use disorder (SUD). Emerging evidence from human and animal studies suggests frontal cortical regions exert influence over striatal reward processing areas during decision-making in impulsive choice or delay discounting (DD) tasks. The goal of this study was to examine how these circuits are involved in decision-making in animals with defined trait impulsivity. To this end, we trained adolescent male rats to stable behavior on a DD procedure and then re-trained them in adulthood to assess trait-like, conserved impulsive choice across development. We then used chemogenetic tools to selectively and reversibly target corticostriatal projections during performance of the DD task. The prelimbic region of the medial prefrontal cortex (mPFC) was injected with a viral vector expressing inhibitory designer receptors exclusively activated by designer drugs (Gi-DREADD), and then mPFC projections to the nucleus accumbens core (NAc) were selectively suppressed by intra-NAc administration of the Gi-DREADD actuator clozapine-n-oxide (CNO). Inactivation of the mPFC-NAc projection elicited a robust increase in impulsive choice in rats with lower vs. higher baseline impulsivity. This demonstrates a fundamental role for mPFC afferents to the NAc during choice impulsivity and suggests that maladaptive hypofrontality may underlie decreased executive control in animals with higher levels of choice impulsivity. Results such as these may have important implications for the pathophysiology and treatment of impulse control, SUDs, and related psychiatric disorders.
... The ability to accurately represent the time it takes to receive a reward following a press is a key behavioral sub-component in delay discounting tasks [52]. Thus, it is conceivable that CIN D2R upregulation results in an overestimation of time intervals, thereby reducing tolerance of delays compared to controls. ...
... Indeed, individuals deemed impulsive on delay discounting tasks are more prone to timing errors compared to control subjects [72]. Similarly, rats that showed higher timing precision in peak interval and temporal discrimination tasks also show reduced delay-based impulsivity [52,73]. Multiple studies also support a role of dopamine in timing [74][75][76]. ...
Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.
... In an investigation of time-based interventions, Smith et al. (2015;Experiment 2) reported that exposing rats to fixed-interval (FI; rats exposed to consistent delays across trials) and variable-interval (rats exposed to variable delays across trials) schedules of reinforcement increased self-controlled choices with corresponding increases in timing precision (i.e., decreased variance in the peak-interval procedure; Roberts, 1981). This outcome demonstrated that improvements in timing correspond with improvements in self-control, consistent with the observed correlation between individual differences in timing abilities and LL choices (Marshall et al., 2014;McClure et al., 2014; however see, . These time-based interventions may have improved self-control by improving timing, or by increasing delay tolerance (Smith et al., 2019). ...
Interventions exposing rats to delayed-reward contingencies attenuate suboptimal impulsive choices, a preference for a smaller-sooner (SS) over a larger-later (LL) reward. Interventions may potentially improve delay-tolerance, timing of delays, and/or discrimination of reward magnitudes. Generalization from the intervention to impulsive choice under different procedures can provide insights into the processes that underlie the intervention effects. Experiment 1 tested intervention effects on systematic-delay (SYS) and adjusting-delay (ADJ) procedures, predicting that intervention effects would be more effective on the SYS procedure with predictable delays. The ADJ procedure did not benefit significantly from intervention, but the SYS procedure, unexpectedly, showed greater impulsive choices following intervention. Experiment 2 tested whether short (5 s) SS intervention delays may have promoted greater impulsivity in the SYS impulsive choice procedure in Experiment 1. Short SS delays in choice and intervention procedures increased impulsive choices in comparison to longer (10 s) delays. Incongruent SS delays in the intervention/choice procedures resulted in negative intervention effects. The results suggest that short SS delays are detrimental to self-control and that specific temporal information generalizes from the intervention to the SYS choice task, but not the ADJ choice task.
... Finally, having examined DA's effects on behaviors in interval timing and measures of impulsivity, we can also examine how the behavioral phenotypes covary with each other. Our model predicts that-due to natural differences in DA levels within a species-animals that are more precise timers should also appear less impulsive in ITCs, as has indeed been observed [98,99] (Supplementary Text 6). ...
Bayesian models successfully account for several of dopamine (DA)’s effects on contextual calibration in interval timing and reward estimation. In these models, tonic levels of DA control the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice and probability discounting tasks. In intertemporal choice tasks, agents must choose between a small reward delivered soon and a large reward delivered later, whereas in probability discounting tasks, agents must choose between a small reward that is always delivered and a large reward that may be omitted with some probability. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings in both tasks. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases ‘impulsivity,’ and promote selection of the large/risky option, often taken to imply that DA increases ‘risk-seeking.’ However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect—promoting selection of the smaller/sooner option (higher impulsivity) and the small/safe option (lower risk-seeking). Second, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later and large/risky options. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts correlations among behavioral phenotypes: Animals that are better timers will also appear less impulsive.
... Imprecision is the amount of variation (noise) when an individual repeatedly estimates the same time interval; it is thus synonymous with the 'endogenous variability' discussed by Balci et al. (2009). Two correlational studies found that those individuals whose timing was more imprecise were also more impulsive (Marshall et al. 2014;McClure et al. 2014). In these studies, impulsivity was measured by repeated choices between smaller sooner and larger later rewards, and timing imprecision was measured by either repeated reproduction of a fixed interval (McClure et al. 2014), or a temporal discrimination task (Marshall et al. 2014). ...
... Two correlational studies found that those individuals whose timing was more imprecise were also more impulsive (Marshall et al. 2014;McClure et al. 2014). In these studies, impulsivity was measured by repeated choices between smaller sooner and larger later rewards, and timing imprecision was measured by either repeated reproduction of a fixed interval (McClure et al. 2014), or a temporal discrimination task (Marshall et al. 2014). In the former case, animals are trained that a response is available after a fixed interval (FI). ...
... This principle has been used to make the prediction that greater timing imprecision should produce less impulsivity (McClure et al. 2014). The study in fact found the opposite pattern. ...
Impulsivity, in the sense of the extent rewards are devalued as the time until their realization increases, is linked to various negative outcomes in humans, yet understanding of the cognitive mechanisms underlying it is limited. Variation in the imprecision of interval timing is a possible contributor to variation in impulsivity. We use a numerical model to generate predictions concerning the effect of timing imprecision on impulsivity. We distinguish between fixed imprecision (the imprecision that applies even when timing the very shortest time intervals) and proportional imprecision (the rate at which imprecision increases as the interval becomes longer). The model predicts that impulsivity should increase with increasing fixed imprecision, but decrease with increasing proportional imprecision. We present data from a cohort of European starlings (Sturnus vulgaris, n = 28) in which impulsivity had previously been measured through an intertemporal choice paradigm. We tested interval timing imprecision in the same individuals using a tri-peak temporal reproduction procedure. We found repeatable individual differences in both fixed and proportional imprecision. As predicted, birds with greater proportional imprecision in interval timing made fewer impulsive choices, whilst those with greater fixed imprecision tended to make more. Contradictory observations in the literature regarding the direction of association between timing imprecision and impulsivity might be clarified by distinguishing between fixed and proportional components of imprecision.
... Infusion and time-out stimuli were identical to FR1. This schedule was selected to remove the constraint of a fixed number of presses per reinforcement in order to allow for more behavioural variability (McClure et al. 2014). After 16 1-h sessions on FI1, the schedule was shifted to progressive ratio (PR) with a gradual increase in the number of presses necessary to deliver each successive infusion (i.e., 1, 2, 3, 4, etc.) for 4 sessions. ...
Rationale
Over the past decade, adolescent cigarette smoking has been declining. However, adolescent nicotine consumption via electronic cigarettes is rapidly gaining popularity. Earlier onset nicotine use is associated with increased risk of dependence. A bidirectional relationship between nicotine and stress exists; perceived stress is a predictor for nicotine use, and stress reduction is a commonly reported reason for using nicotine.
Objectives
We assessed the prolonged impact of adolescent high-dose nicotine and/or footshock exposure on adult nicotine self-administration, anxiety-like behaviour, and hormonal responsivity.
Methods
During adolescence (postnatal day [P]28-56) male Sprague-Dawley rats were assigned to one of five groups: saline (SALPRE: 1 ml/kg, SC, every day), nicotine (NICPRE: 1 mg/kg, SC, alternating daily with saline; 14 total nicotine injections), footshock (SHOCKPRE: 8 of 0.5 s, 0.8 mA alternating sessions; saline every day), or combination nicotine and footshock (NIC+SHOCK: concurrent and alternating daily with saline, or NIC–SHOCK: alternating with saline on shock sessions). On P70, one cohort underwent spontaneous intravenous nicotine self-administration (0.03 mg/kg/infusion); another cohort was assessed for open-field behaviour (P71), then corticosterone (CORT) response to nicotine or footshock in adulthood (P72-73).
Results
Intermittent adolescent nicotine or footshock alone (NICPRE and SHOCKPRE) did not potentiate adult spontaneous nicotine intake compared to SALPRE. However, both combination groups (NIC+SHOCK, NIC–SHOCK) showed increased adult nicotine consumption without associated differences in baseline anxiety-like behaviour or CORT response.
Conclusions
Adolescent nicotine and footshock stressors have a synergistic effect on adult nicotine consumption, enhancing nicotine intake. Avenues toward reducing stress in adolescent nicotine users may provide opportunities to reduce vulnerability to adult nicotine consumption.
