No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Rapid Communication Prediction error as a linear function of reward probability is coded in human nucleus accumbens
Abstract
Reward probability has been shown to be coded by dopamine neurons in monkeys. Phasic neuronal activation not only increased linearly with reward probability upon expectation of reward, but also varied monotonically across the range of probabilities upon omission or receipt of rewards, therefore modeling discrepancies between expected and received rewards. Such a discrete coding of prediction error has been suggested to be one of the basic principles of learning. We used functional magnetic resonance imaging (fMRI) to show that the human dopamine system codes reward probability and prediction error in a similar way. We used a simple delayed incentive task with a discrete range of reward probabilities from 0% to 100%. Activity in the nucleus accumbens of human subjects strongly resembled the phasic responses found in monkey neurons. First, during the expectation period of the task, the fMRI signal in the human nucleus accumbens (NAc) increased linearly with the probability of the reward. Second, during the outcome phase, activity in the NAc coded the prediction error as a linear function of reward probabilities. Third, we found that the Nac signal was correlated with individual differences in sensation seeking and novelty seeking, indicating a link between individual fMRI activation of the dopamine system in a probabilistic paradigm and personality traits previously suggested to be linked with reward processing. We therefore identify two different covariates that model activity in the Nac: specific properties of a psychological task and individual character traits.
... Finally, during the outcome phase, participants receive feedback about whether they successfully hit the target, which in turn informs them of whether they received a reward for that trial. During the cue phase, the ventral striatum not only preferentially responds to reward-predictive cues, but also positively scales with the magnitude and probability of attaining the cued reward (Abler, Walter, Erk, Kammerer, & Spitzer, 2006;Knutson et al., 2001;Knutson, Taylor, Kaufman, Peterson, & Glover, 2005), suggesting that ventral striatal activity in response to predic tive cues represents an index of expectancy. ...
... In contrast to expectancy, reward prediction error tests an individual's neural response to reward outcome feedback. More specifically, it is examined by contrasting neural re sponses to unexpected (low-probability) reward feedback against neural responses to ex pected (high-probability) reward feedback (Abler et al., 2006). Research examining pre diction error response indicates that expectancy violations during reward receipt reliably elicit neural activity in the ACC, anterior insula, and ventral striatum (Abler et al., 2006;Behrens, Hunt, Woolrich, & Rushworth, 2008;Garrison et al., 2013). ...
... More specifically, it is examined by contrasting neural re sponses to unexpected (low-probability) reward feedback against neural responses to ex pected (high-probability) reward feedback (Abler et al., 2006). Research examining pre diction error response indicates that expectancy violations during reward receipt reliably elicit neural activity in the ACC, anterior insula, and ventral striatum (Abler et al., 2006;Behrens, Hunt, Woolrich, & Rushworth, 2008;Garrison et al., 2013). While earlier stages of reward processing reflect the preference for reward and behaviors that align with this preference, this stage of reward processing represents how the individual's reward-based desires establish expectancies that may or may not be realized. ...
This book provides an overview of key processes relevant to disturbances in positive valence systems; discusses cutting-edge advances on positive emotion disturbance in key clinical disorders, translational applications, and targeted treatment foci; discusses conceptualizations of psychopathology and models of positive emotion disturbances; and suggests future research to better understand the nature of positive emotion. The book covers cutting-edge scientific work and theoretical perspectives from a renowned group of psychologists. Their expertise spans a diverse array of methodological and theoretical approaches applied to the study of positive valence disturbances across the life span and across a range of psychiatric disorders. In doing so, this book demonstrates how examining populations characterized by positive emotion disturbance enables a better understanding of both psychiatric course and risk factors and informs claims about the basic function of positive emotion.
... Early research in neuroeconomics established that in line with economic theory 9 , the brain encodes a utility-like signal that guides choice 10 . At the same time, to establish a biologically viable, unified framework explaining economic decision-making under uncertainty, neuroeconomists aimed to incorporate not only the reward magnitude but also probability into the framework and searched for evidence of inverse-S subjective reward probability weighting in human brain activity using neuroimaging techniques [11][12][13][14][15] . Focusing on the gain domain 13,15 , previous studies found that the activity of brain regions in the reward circuitry correlates with individual subjective valuations as proposed by the prospect theory 13,15,16 . ...
... Recordings of single-neuron activity in monkeys while receiving risky rewards [17][18][19][20][21] may offer substantial progress over existing neuroimaging studies [11][12][13][14] . Specifically, utility coding without probability weighting was tested on the activity of dopamine neurons 22 . ...
... This is consistent with the idea suggested by Tobler et al. (2007) 34 that the striatum integrates the reward magnitude and probability via multiplication into an expected value signal. Previous human fMRI studies found a nonlinear response to probability in striatal regions 12,15 and the dorsolateral prefrontal cortex 34 , while some studies concluded that the probability coding in the striatum is linear 11,13,14 . The probability weighting that we estimated in behavior and recovered from neural activity is concave. ...
Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals’ choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.
... In primates, the motor division is located in posterior dorsolateral portions of the putamen, while the associative and limbic divisions encompass dorsal and ventral portions of the anterior caudate nucleus and putamen, respectively . Several fMRI studies in humans have reported that the processing of reward-related information, including RPE, is dominant in the ventral rather than the dorsal striatum O'Doherty, 2004;Abler et al., 2006;Hare et al., 2008). In addition, neuronal recordings in rats have shown that the nucleus accumbens is important for updating choice behaviors (Ito and Doya, 2009). ...
... The role of midbrain dopamine neurons in RPE encoding is well established (Fiorillo et al., 2003;Abler et al., 2006;Bray and O'Doherty, 2007;Fujiyama et al., 2015). ...
... In particular, a number of studies have highlighted the role of the ventral striatum in the computation of RPEs (Abler et al., 2006;Bray and O'Doherty, 2007;Calderon et al., 2021 (Silvetti et al., 2014;. . ...
In the field of instrumental learning, mammals are able to implement two different behavioral strategies to interact with the environment: goal directed behavior (GDB), computationally flexible but slow, suitable to learn new tasks and adapt to changing environments; and habitual behavior, hard-coded, but suitable for faster motor responses and facing recurrent tasks. The advantage of GDB resides in the use of an inner representation of the environment, a ‘model of the world’, to encode stimuli-actions-outcomes associations, and its exploitation to choose future actions, in a process called planning. GDB is supported by large-scale networks involving both cortical and subcortical regions. Nevertheless, several open questions still remain. The aim of this thesis is to contribute to the understanding of three open questions (declined in three studies) that pertain to the neural and computational mechanisms of GDB.In the first study, we investigated how complex computations, such as learning the model of the world and planning, can emerge from simple neural activity. To achieve that, we built a spiking neural network, able to encode stimulus-actions-outcomes associations as a hidden Markov model (HMM), using biologically inspired mechanisms such as spike-timing dependent plasticity (STDP), and to test this model to correctly plan actions in order to solve a visuomotor goal directed task. The performance of the model was validated on behavioral data from human participants that performed the same task.In the second study, we assessed the importance of striatum in encoding the reward prediction error (RPE) signals, a relevant update signal in most instrumental learning models. To do so, we analysed local field potentials (LFPs) recorded in rhesus macaque striatum while performing a probabilistic goal-directed learning task. Then, we computed the trial-by-trial RPE using a Q-learning model fitted on monkeys’ behavior. Our results showed a significant increase of mutual information (MI) between the beta-band (15-30Hz) oscillatory activity and the RPE after the outcome presentation. Moreover, such correlates of RPE signals form an anatomo-functional gradient in the striatum, showing stronger effects toward the rostro-ventral part and vanishing toward the caudo-dorsal part.In the third study, we investigated the neural correlates of GDB at the whole-brain cortical level in humans. To do so, we recorded the brain activity of human participants using magnetoencephalography (MEG) while they were performing a goal-directed causal learning task. We exploited cortical high-gamma activity (HGA, 60-120Hz) to map the spatio-temporal dynamics during learning. In particular, we used an ideal observer Bayesian model to estimate the trial-by-trial evolution of relevant behavioral variables, such as action-outcome probabilities and contingency values. We used MI and group-level cluster-based statics between HGA and those variables to obtain a whole brain profile of behavioral-dependent regions of interests’ activity, confirming some results from the literature.
... One identified study with MDD participants utilized simultaneous fMRI and positron emission tomography (PET) alongside the MID and observed lower VS and right dorsal striatum dopamine activity alongside lower connectivity with cortical targets (Hamilton et al., 2018). The MID involves components of various reward processes, but striatal dopamine activity is associated with reward coding during anticipation (Abler, Walter, Erk, Kammerer, Spitzer, 2006). Other studies using the MID in MDD participants have also found patterns of striatal hypoactivation during reward anticipation, in both the NAc (Misaki, Suzuki, Savitz, Drevets, & Bodurka, 2016) and the putamen (Pizzagalli et al., 2009;Takamura et al., 2017), with one study using an unmedicated MDD sample (Pizzagalli et al., 2009). ...
... Striatal hypoactivation in reward wanting deficits may be associated with abnormal dopamine signalling in this region in MDD, as modulating dopamine transmission using amisulpride in healthy individuals is associated with striatal alterations-for example, putamen, NAc (Metzger, Wiegers, Walter, Abler, & Graf, 2015), and dopaminergic activity in the striatum is involved in coding reward expectancy during anticipation (Abler et al., 2006), and driving incentive motivation (Bardgett, Depenbrock, Downs, Points, & Green, 2009;Denk et al., 2005;Salamone, Correa, Farrar, & Mingote, 2007;Salamone, Correa, Nunes, Randall, & Pardo, 2012). Indeed, midbrain dopamine function is dysfunctional in MDD (Dailly, Chenu, Renard, & Bourin, 2004;Dunlop & Nemeroff, 2007;Nestler & Carlezon, 2006), and amisulpride enhancement of dopamine transmission in MDD normalizes striatal hypoactivation during reward processing (Admon et al., 2017). ...
... Striatal hypoactivation may represent blunted dopamine signalling, because midbrain dopamine neurons projecting to the striatum code prediction errors when reward feedback is better or worse than expected (Abler et al., 2006;Bayer & Glimcher, 2005;Schultz, 1997Schultz, , 1998, which is essential for reinforcement learning (Glimcher, 2011). Two other studies did not compute prediction error signals, but did report striatal hyposensitivity for unexpected positive feedback (Robinson et al., 2012;Segarra et al., 2016). ...
Anhedonia is a key symptom of major depressive disorder (MDD) and comprises behavioural deficits in three reward processing subtypes: reward liking, reward wanting, and reward learning. However, neuroimaging findings regarding the neural abnormalities underpinning these deficits are complex. We have conducted a systematic review to update, reframe and summarize neuroimaging findings across the three subtypes of anhedonia in MDD. Using PubMed, The Cochrane Library, PsycINFO, and Web of Science databases, we identified 59 fMRI studies comparing participants with current or remitted MDD with controls, using reward processing tasks. For reward liking and wanting, striatal hypoactivation was observed, alongside hypoactivation and hyperactivation across frontal regions. For reward learning, blunted frontostriatal sensitivity to positive feedback was observed. These findings highlight the importance of studying anhedonia not only as a clinical manifestation but also as a neurobiological mechanism underlying depressive disorder and other broader psychiatric conditions.
... Previous research on brain processing related to sensation-seeking has typically involved activation studies with relatively low numbers of participants. In one study, activation in the ventral striatum in a delayed incentive task was related to individual differences in sensation-seeking and novelty seeking (Abler et al., 2006). In another study, reactivity to reward within the nucleus accumbens had different relations to sensation-seeking at different ages (Hawes et al., 2017). ...
... A previous task-related fMRI study showed that high sensation seekers have a high activation to arousing images from the IAPS set in the posterior medial orbitofrontal cortex (Abler et al., 2006;Joseph et al., 2009). In addition, activations in the nucleus accumbens (which receives from the orbitofrontal cortex) are high in sensation-seekers in a monetary incentive delay task (Abler et al., 2006). ...
... A previous task-related fMRI study showed that high sensation seekers have a high activation to arousing images from the IAPS set in the posterior medial orbitofrontal cortex (Abler et al., 2006;Joseph et al., 2009). In addition, activations in the nucleus accumbens (which receives from the orbitofrontal cortex) are high in sensation-seekers in a monetary incentive delay task (Abler et al., 2006). There is also extensive evidence that the human medial OFC areas, including BA13, are activated by rewarding stimuli that are subjectively pleasant (including pleasant odours, pleasant touch, pleasant flavour, and monetary reward) (Grabenhorst and Rolls, 2011;O'Doherty et al., 2001;Rolls, 2014Rolls, , 2019c. ...
Sensation-seeking is a multifaceted personality trait with components that include experience-seeking, thrill and adventure seeking, disinhibition, and susceptibility to boredom, and is an aspect of impulsiveness. We analysed brain regions involved in sensation-seeking in a large-scale study with 414 participants and showed that the sensation-seeking score could be optimally predicted from the functional connectivity with typically (in different participants) 18 links between brain areas (measured in the resting state with fMRI) with correlation r=0.34 (p=7.3×10⁻¹³) between the predicted and actual sensation-seeking score across all participants. Interestingly, 8 of the 11 links that were common for all participants were between the medial orbitofrontal cortex and the anterior cingulate cortex and yielded a prediction accuracy r=0.30 (p=4.8×10⁻¹⁰). We propose that this important aspect of personality, sensation-seeking, reflects a strong effect of reward (in which the medial orbitofrontal cortex is implicated) on promoting actions to obtain rewards (in which the anterior cingulate cortex is implicated). Risk-taking was found to have a moderate correlation with sensation-seeking (r=0.49, p=3.9×10⁻²⁶), and three of these functional connectivities were significantly correlated (p<0.05) with the overall risk-taking score. This discovery helps to show how the medial orbitofrontal and anterior cingulate cortices influence behaviour and personality, and indicate that sensation-seeking can involve in part the medial orbitofrontal cortex reward system, which can thereby become associated with risk-taking and a type of impulsiveness.
... In experiment 1, it was possible to assume a priori an effect in the nucleus accumbens/ventral striatum at the appearance of the cue based on the existing literature. We therefore carried out inference aided by an anatomical region of interest for this region, defined as a box at MNI coordinates x: −12-+12, y: 0-+12; z: −12-+6 (5184 mm 3 ), based on the data reported by ref. 64 . In contrast, because no a priori hypothesis was made for the effect of reward in the foraging patches, we conducted the analysis of the respective contrast in the whole volume. ...
... The findings concerning the signal associated with the cue replicated the observations of a now fairly extensive body of functional neuroimaging literature. In the first experiment, differences in reward rates at the cue activated the ventral striatum 40,64,[69][70][71][72][73][74] . This activity was present from the first trial, confirming that the cue was predictive from the outset of the experiment. ...
... However, in the second experiment, where the cue carried no information on the level of reward of the impeding trial, we did not observe the same activation of the ventral striatum in association with the presentation of the cue. This finding is consistent with previous observations in similar studies that no ventral striatal signal is present at the presentation of cues or reward if their occurrence is predicted 64 , as when the cues follow a predictable sequence 40 , and with the prediction of the temporal difference model 34 . Together, the findings about the cue characterize the activation in the ventral striatum as the known correlates of prediction errors about available rewards. ...
Theoretical models of dopamine function stemming from reinforcement learning theory have emphasized the importance of prediction errors, which signal changes in the expectation of impending rewards. Much less is known about the effects of mean reward rates, which may be of motivational significance due to their role in computing the optimal effort put into exploiting reward opportunities. Here, we used a reinforcement learning model to design three functional neuroimaging studies and disentangle the effects of changes in reward expectations and mean reward rates, showing recruitment of specific regions in the brainstem regardless of prediction errors. While changes in reward expectations activated ventral striatal areas as in previous studies, mean reward rates preferentially modulated the substantia nigra/ventral tegmental area, deep layers of the superior colliculi, and a posterior pontomesencephalic region. These brainstem structures may work together to set motivation and attentional efforts levels according to perceived reward opportunities.
... Two fMRI phenotypes were chosen as representative of two known important aspects of depressive functioning: lack of sensitivity to reward (impairment in appetitive motivation, leading to an anhedonic symptom domain, (Treadway and Zald, 2011)) and hyperreactivity to negative emotional stimuli (Siegle et al., 2002;Stuhrmann et al., 2011;Whalen et al., 2002). These two fMRI phenotypes refer to assessment of ventral striatal activation during anticipation of reward (Abler et al., 2006), and of amygdala activity when exposed to highly arousing negative emotional stimuli (Whalen et al., 2002). ...
... The total duration of this paradigm was 8:07 min. The contrast of interest to assess reactivity to appetitive incentives was the difference in brain signal elicited by cues announcing high and low reward rates (Abler et al., 2006). ...
... The results section reports cluster sizes k in number of voxels of 2 × 2 × 2 mm. Correction was obtained for the region of interest in the nucleus accumbens/ventral striatum in experiment by defining a box at MNI coordinates x: −12 ± +12, y: 0 ± +12; z: −12 ± +6, based on the data reported by (Abler et al., 2006). For region of interest analysis of the amygdala, we used maps from the Jülich atlas based on (Amunts et al., 2005). ...
Treatment with interferon (IFN) has been associated with depressive side effects. Previous neuroimaging studies have provided information about changes in brain activation patterns in patients under treatment with IFN-alpha, but the effect of other IFNs, or the role of the underlying disease, has yet to be clarified. In the present fMRI study, we looked at brain changes after 8 days of IFN-beta treatment in N = =17 healthy volunteers, thus avoiding the possible confound of the effects of underlying pathology in studies of IFN-treated patients with neurological or other medical disorders. We followed a symptom dimensional approach by simultaneously investigating two distinct symptom domains of depressiveness: negative affect (amygdala) and appetitive motivation (ventral striatum). In these early phases of IFN treatment we detected a selective change in neural substrates of appetitive motivation, consistent with the predominant symptomatic change recorded in psychopathology ratings. In contrast, the fMRI phenotype of negative affect, which is known to characterize disorders of affect involving anxiety and depressiveness as well as individual vulnerability to depression, was unchanged after treatment. These findings suggest that IFN may induce an affective syndrome through a mechanism involving down-regulation of appetitive motivation.
... Previous studies have identified significant differences in regional brain activity between high and low self-reported sensation-seeking individuals during passive viewing of 'high arousal' and emotional stimuli (Joseph et al., 2009;Straube et al., 2010), reward anticipation (Abler et al., 2006), and risky choice (Freeman and Beer, 2010;Kruschwitz et al., 2012) -with differences in activity in the orbitofrontal cortex (OFC), ventral striatum (vS), and insula being commonly implicated. ...
... Based on studies cited above, in conjunction with previous work indicating a role for the vS and OFC/ventromedial prefrontal cortex (vmPFC) in representing the expected value of choice options (e.g. Knutson et al., 2001;Abler et al., 2006;Levy and Glimcher, 2012), we hypothesized that these regions may encode common responses to MES-associated stimuli and economic reward in behavioural high sensation-seekers. ...
... A large number of functional imaging studies have related variation in BOLD signal in the OFC/vmPFC and ventral striatum (including head of the caudate nucleus) to the expected value of choice options (Knutson et al., 2001;Abler et al., 2006;Levy and Glimcher, 2012). Thus our finding of increased signal in these regions when choosing MES-associated options, as a linear function of the additional value participants assigned to opportunity to receive the MES (positive or negative), is consistent with the proposal that participants used a common valuation currency for the points and sensory value of different choice options. ...
Sensation-seeking is a personality trait concerned with motivation for intense and unusual sensory experiences, that has been identified as risk factor for a variety of psychopathologies with high social cost; in particular gambling and substance addictions. It has previously proved difficult to tease out neural mechanisms underlying sensation-seeking in humans, due to a lack of cognitive-behavioural paradigms probing sensation-seeking-like behaviour in the lab. The first aim of this thesis was to develop such a behavioural paradigm. Within, we present evidence from this novel task and a combination of psychopharmacological, functional imaging and computational approaches to argue that sensation-seeking behaviour in humans is driven by inter-individual differences in the activation of dopaminergic approach-withdrawal tendencies, when faced with the opportunity to experience intense and unusual sensory stimulation. In a parallel research stream, we investigate the relationship between self-reported sensation-seeking, D2-type dopamine receptor function and risky decision-making, motivated by the common implication of sensation-seeking personality and D2ergic drugs in disorders involving excessive risk-taking. Together, the findings presented here may aid investigation of various psychopathologies for which more extreme sensation-seeking scores constitute a vulnerability factor. In particular, a more precise understanding of sensation-seeking behaviour might aid in the identification of at-risk individuals and the development of individualised therapies and prevention strategies.
... Thus, our findings indicate that the detection of reward may not be affected by pain. However, we cannot conclude that the NAcc neural activity is unaffected by pain when it is responsible for other aspects of reward processing, as the NAcc is also found to encode the reward probability during reinforcement learning (Abler, Walter, Erk, Kammerer, & Spitzer, 2006;Knutson, Adams, Fong, & Hommer, 2001). We admit that the win probability in the card-guessing game were just 50% in our study, in which no reinforcement learning was involved. ...
... We admit that the win probability in the card-guessing game were just 50% in our study, in which no reinforcement learning was involved. Therefore, further investigation is needed to determine whether pain alters NAcc neural activities that process reward probability, by adopting a learning task with prediction errors, such as a monetary incentive task (Abler et al., 2006). ...
Pain has been found to promote reward-seeking behaviors, which might be a consequence of modulated brain activities in the reward neural circuitry in a painful state. The present study investigated how pain affected reward processing and reward-related neural activities using fMRI technique. A total of 50 healthy participants were recruited and used for data analyses, with half being treated with topical capsaicin cream and the other half with hand cream (treatment: pain or control). The participants were asked to perform a card-guessing game when their brain activities responding to feedbacks (outcome: win or loss) were recorded. Behavioral results showed that participants in pain group overestimated their correct choices in the card-guess game. Whole-brain fMRI analysis revealed that the main effect of outcome (win vs. loss) activated a typical network of the reward neural circuitry, including the medial prefrontal cortex (mPFC) and the bilateral nucleus accumbens (NAcc). Importantly, the region of interest analysis revealed a significant interaction of treatment and outcome in the mPFC, with increased mPFC neural activity responding to win outcome in pain condition. Moreover, the functional connectivity between the mPFC and the NAcc was decreased in pain condition. We conclude that the pain-induced modulation of the mPFC activity could result in alterations of both the emotional response to and the cognitive evaluation of reward.
... LEDD was included as a covariate in subsequent analyses. Although dopamine agonists as a class are associated with an elevated risk of ICBs (Weintraub et al., 2010), we included all dopaminergic medication in the calculation of LEDD, given that ICBs are also more prevalent in individuals on levodopa, as well as the substantial prior literature linking dopamine to changes in reward learning (and thus impulsivity) in preclinical models (Schultz et al., 1997), healthy individuals (Abler et al., 2006) and individuals with Parkinson's disease (Frank et al., 2004). ...
... The greater the connectivity of the reward evaluation network, the more explorative and the higher the bet size used by participants, modulated by LEDD. Bilateral tracts connecting the VS to vmPFC were weighted most heavily in this model, upholding much prior work linking the VS with reinforcement learning and reward evaluation (Schultz et al., 1997;Abler et al., 2006;Daw et al., 2006;Tanaka et al., 2008;Wittmann et al., 2008;Basar et al., 2010;Haber and Knutson, 2010;de Wit et al., 2012;Kishida et al., 2016;Hampton et al., 2017). The amount wagered in a gamble is a parsimonious way to obtain a behavioural readout of impulsivity and it is interesting that this measure correlated significantly with our reward evaluation network measures. ...
See O’Callaghan (doi:10.1093/brain/awz349) for a scientific commentary on this article.
Mosley et al. examine impulsivity and naturalistic gambling behaviours in patients with Parkinson’s disease. They link within-patient differences to the structural connectivity of networks subserving reward evaluation and response inhibition, and reveal pivotal roles for the ventral striatum and subthalamic nucleus within these networks.
... Également, l'erreur de prédiction d'une récompense se retrouve souvent dans une corrélation avec le signal BOLD du SV mais pas du COFvm (Abler et al., 2006;Hare et al., 2008;Lin et al., 2012;McClure et al., 2003;Pessiglione et al., 2006). Cette activation du SV est probablement due aux afférences dopaminergiques du SV. ...
Les mécanismes cérébraux engagés dans la prise de décision sont loin d’être compris. Nos choix sont le plus souvent loin d’être rationnels et cela est lié à la valeur subjective attribuée aux options présentées par notre cerveau. Le travail réalisé dans cette thèse vise à comprendre plus en profondeur le système de valuation cérébral grâce à la technique d’IRMf. Nous avons étudié notamment les biais de prise de décision autour de trois études portant sur la distorsion des probabilités pour une récompense immédiate ou retardée, l’effet de cadre dans un contexte de gains ou de pertes pour une décision prise pour soi ou pour un proche et enfin le phénomène d’aversion à la perte pour des récompenses primaires ou secondaires. Ces études ont été menée à la fois chez une population saine et une population souffrant d’addiction comportementale, une addiction aux jeux d’argent pour les deux premières études et l’anorexie mentale pour la dernière étude.
... Activation in ventral striatum is associated with both value and uncertainty (Knutson et al., 2005). Thus, activation in the ventral striatum is consistent with aspects of the utility (both value and uncertainty, e.g., valuation; Bartra et al., 2013) and narrative model (related to the probability of an event and prediction error; Abler et al., 2006;Rodriguez et al., 2006). These findings support a role for uncertainty or probability estimation but do not support a role for value in evidence accumulation. ...
Efforts to explain complex human decisions have focused on competing theories emphasizing utility and narrative mechanisms. These are difficult to distinguish using behavior alone. Both narrative and utility theories have been proposed to explain juror decisions, which are among the most consequential complex decisions made in a modern society. Here, we asked jury-eligible male and female subjects to rate the strength of a series of criminal cases while recording the resulting patterns of brain activation. We compared patterns of brain activation associated with evidence accumulation to patterns of brain activation derived from a large neuroimaging database to look for signatures of the cognitive processes associated with different models of juror decision-making. Evidence accumulation correlated with multiple narrative processes, including reading and recall. Of the cognitive processes traditionally viewed as components of utility, activation patterns associated with uncertainty, but not value, were more active with stronger evidence. Independent of utility and narrative, activations linked to reasoning and relational logic also correlated with increasing evidence. Hierarchical modeling of cognitive processes associated with evidence accumulation supported a more prominent role for narrative in weighing evidence in complex decisions. However, utility processes were also associated with evidence accumulation. These complementary findings support an emerging view that integrates utility and narrative processes in complex decisions.SIGNIFICANCE STATEMENT The last decade has seen a sharply increased interest in narrative as a central cognitive process in human decision-making and as an important factor in the evolution of human societies. However, the roles of narrative versus utility models of decision-making remain hotly debated. While available models frequently produce similar behavioral predictions, they rely on different cognitive processes and so their roles can be separated using the right neural tests. Here, we use brain imaging during mock juror decisions to show that cognitive processes associated with narrative, and to a lesser extent utility, were engaged while subjects evaluated evidence. These results are consistent with interactions between narrative and utility processes during complex decision-making.
... On one hand, reward expectancy modulates reward experience, and a mismatch between expected and actual outcomes causes RPE signals. Human fMRI studies have found that the ventral striatum could represent linear variations of RPE (Abler, Walter, Erk, Kammerer, & Spitzer, 2006;Bayer & Glimcher, 2005;Preuschoff, Bossaerts, & Quartz, 2006); however, contradictory results have been observed in other studies (Hsu, Krajbich, Zhao, & Camerer, 2009;Tobler, Christopoulos, O'Doherty, Dolan, & Schultz, 2008). In contrast, the expectancy of events influences causal attribution (Hastie, 1984;Weiner, 1985). ...
Individual success and failure in social cooperation matter not only to oneself but also to teammates. However, the common and distinct neural activities underlying salient success and failure in social cooperation are unclear. In this functional magnetic resonance imaging (fMRI) study, participants in the social group (Experiment one) cooperated with two human beings during a dice-gambling task, whereas those in the nonsocial group (Experiment two) cooperated with two computers. The social group reported more pride in success and more guilt in failure. The fMRI results in Experiment one demonstrated that left temporoparietal junction (LTPJ) activation increased exclusively with linearly changing unexpected success, whereas increasing anterior cingulate cortex (ACC) activation was only coupled with increasing unexpectedness of failure. Moreover, the dorsal medial prefrontal cortex (dMPFC) and left anterior insula were recruited in both success and failure feedback conditions. Dynamic causality model analysis suggested that the dMPFC first received information from the LTPJ and ACC separately and then returned information to these regions. The between-experiment comparison showed more dMPFC activity in social versus nonsocial contexts irrespective of success and failure feedback. Our findings shed light on the common and distinct neural substrates involved in processing success and failure feedback in social cooperation.
... Furthermore, another recent study found that sensation seeking moderated the link between sleep quality, measured using the Pittsburgh Sleep Quality Index (PSQI; Buysse et al., 1989), and risk-taking among adolescents, such that better quality sleep was associated with reduced risk taking, but only for youth with low and average sensation seeking (Baker et al., 2020). In addition, studies have found evidence that greater ventral striatum (VS) activity, which is involved in reward processing and positively correlated with impulsive traits (Abler et al., 2006), may buffer the link between sleep problems and both depression and risk taking (Avinun et al., 2017;Baker et al., 2020). ...
Heavier drinking and depression are common mental health concerns in the USA, yet few studies have sought to understand transdiagnostic risk factors for both. Two health-focused risk factors are impulsive personality traits and sleep duration, but research typically separates the two, precluding additive and interactive relations. The current study sought to test a theoretical model where risk conferred from impulsive traits is heightened when individuals have reduced sleep. Public-access data from the National Longitudinal Study on Adolescent to Adult Health (Add Health) were used to test study hypotheses. Participants reported on impulsive traits (i.e., lack of premeditation, sensation seeking), sleep duration, depression, and drinking across three waves spanning adolescence, emerging adulthood, and adulthood. Multilevel models distinguished risk processes at the between- vs. within-person level. At the between-person level, sensation seeking predicted drinking whereas premeditation predicted depression. Additionally, within-person deviations in both traits were associated with drinking, whereas within-person deviations in premeditation were associated with depression. Sleep duration was protective against outcomes at both levels. However, main effects were qualified by interactions at both levels, such that having below average sleep duration heightened the effects of premeditation at the between-person level, whereas within-person decreases in sleep heightened the effects of sensation seeking at the within-person level. Findings support a theoretical model where poor sleep exacerbates risk conferred from impulsive traits. Risk conferred from impulsive traits diverged based upon level of analysis, suggesting that global and just-in-time interventions may benefit from targeting specific impulsive traits as well as sleep.
... In humans, parts of the midbrain that send dopaminergic projections to the NAc respond to stimulus uncertainty, and activity of these dopaminergic cells correlates with reward probability (Dreher et al. 2006). Moreover, increasing reward probability is associated with increased blood flow in the striatum in humans (Abler et al. 2006). Consistent with these findings in humans, lesions of the NAc in rats promote risk-aversive behavior by biasing choices away from large rewards with a low probability of occurrence and toward small rewards with a high probability of occurrence, while discrimination of the reward value of different choices remains largely intact (Cardinal and Howes 2005). ...
Despite the prominence of anhedonic symptoms associated with diverse neuropsychiatric conditions, there are currently no approved therapeutics designed to attenuate the loss of responsivity to previously rewarding stimuli. However, the search for improved treatment options for anhedonia has been reinvigorated by a recent reconceptualization of the very construct of anhedonia, including within the Research Domain Criteria (RDoC) initiative. This chapter will focus on the RDoC Positive Valence Systems construct of reward learning generally and sub-construct of probabilistic reinforcement learning specifically. The general framework emphasizes objective measurement of a subject’s responsivity to reward via reinforcement learning under asymmetrical probabilistic contingencies as a means to quantify reward learning. Indeed, blunted reward responsiveness and reward learning are central features of anhedonia and have been repeatedly described in major depression. Moreover, these probabilistic reinforcement techniques can also reveal neurobiological mechanisms to aid development of innovative treatment approaches. In this chapter, we describe how investigating reward learning can improve our understanding of anhedonia via the four RDoC-recommended tasks that have been used to probe sensitivity to probabilistic reinforcement contingencies and how such task performance is disrupted in various neuropsychiatric conditions. We also illustrate how reverse translational approaches of probabilistic reinforcement assays in laboratory animals can inform understanding of pharmacological and physiological mechanisms. Next, we briefly summarize the neurobiology of probabilistic reinforcement learning, with a focus on the prefrontal cortex, anterior cingulate cortex, striatum, and amygdala. Finally, we discuss treatment implications and future directions in this burgeoning area.
... D'Ardenne et al. (2008) used primary rewards (liquid delivered to thirsty participants) and found decreased activation in the ventral striatum for negative prediction errors relative to positive prediction errors. The pattern of increased activation for positive prediction errors relative to negative prediction errors in the ventral striatum and VTA has been replicated numerous times (e.g., Abler et al., 2006;see Wang et al., 2016 for review); however, error-related processing in response to informational rewards such as online solutions has received less attention. These findings of differential activation for positive and negative prediction error are consistent with the results of our Neurosynth meta-analysis that identified the ventral striatum and VTA as preferentially associated with contrasts of "prediction error," indicating that activation in the ventral striatum and VTA should distinguish between positive and negative prediction errors. ...
... Les études IRMf chez l'humain ont confirmé ce rôle en étudiant les corrélations entre le signal cérébral enregistré et l'erreur de prédiction de récompense pendant une variété de tâches. Ces études ont identifié un engagement du striatum ventral dans une grande diversité de paradigmes expérimentaux, comme des tâches d'effort, d'apprentissage ou de choix (Abler, Walter, Erk, Kammerer, & Spitzer, 2006;Hare, O'Doherty, Camerer, Schultz, & Rangel, 2008;Knutson, Fong, Adams, Varner, & Hommer, 2001;Lin, Adolphs, & Rangel, 2011;McClure, Berns, & Montague, 2003;Mathias Pessiglione, Seymour, Flandin, Dolan, & Frith, 2006;Schmidt, Lebreton, Cléry-Melin, Daunizeau, & Pessiglione, 2012). ...
... In the context of a loss trial, a no loss outcome can be considered a successful, and therefore rewarding, trial (i.e., avoidance of expected loss). The increased reactivity in the caudate and putamen during no loss trials may indicate a positive prediction error response to a 'better than expected' outcome (Abler, Walter, Erk, Kammerer, & Spitzer, 2006;Schultz, 2017). ...
Antisocial Behavior (AB), is associated with persistence in risky, reward-driven behaviors despite severe potential consequences such as incarceration. Neuroimaging research has the potential to elucidate the biological mechanisms of these behaviors and to contribute to our understanding of the etiology of AB. However, little is known about the reward-related neural mechanisms that contribute to AB. The few neuroimaging studies that have investigated associations between reward processing and AB often report conflicting findings, perhaps due to methodological and sample heterogeneity across studies. Furthermore, research on adolescent reward-related brain function has typically examined these processes in small, primarily Caucasian, middle-class samples. These samples may have limited generalizability to more diverse or low-income populations that are at increased risk for AB. Finally, studies of typically developing adolescents and studies of youth and adults high on AB fail to adequately investigate specific phases of reward processing in racially-diverse samples. This dissertation is comprised of three studies that examine the neural correlates of reward and loss processing and how variability in these reinforcement circuits is related to AB. Study 1 addresses questions regarding the direction of the relationship between AB and neural response to reward and loss by conducting a systematic review of functional magnetic resonance imaging (fMRI) studies of reward and loss processing in adult AB. To better understand factors contributing to the development of reward processing in adolescence, Study 2 examines the effects of age, pubertal status, race, and gender on neural response to reward and loss in a relatively large, well-sampled cohort of primarily low-income adolescents from urban environments. This study uses a novel version of the Monetary Incentive Delay (MID) task, which has been modified to allow for sufficient temporal separation of phases of reward and loss processing and comparisons to neutral conditions. Finally, Study 3 builds on the results of Study 2 by examining the relationship between AB and neural response to reward and loss using the same sample and task. This study aimed to elucidate the relationship between AB reward- and loss-related neural functioning by investigating whether individual differences in traits such as disinhibition and sensation-seeking explain associations and whether the callous-unemotional traits moderate associations between AB and neural response to reward and loss.
... This pattern may reflect the fact that, although bilateral striatum was identified in our term-based meta-analysis as being related to the term "error," these structures are not typically associated with error monitoring or the salience network. Their identification in the meta-analysis may have been an artifact of the inclusion of the word "error" in other constructs associated with striatum (e.g., "reward prediction error") (Abler et al. 2006). As error-related activations in ROIs previously associated with error monitoring and the salience network were strongly related to PC1, we concluded that this component would operate as an effective summary measure of error-related activation. ...
RationaleSubstance use peaks during the developmental period known as emerging adulthood (ages 18–25), but not every individual who uses substances during this period engages in frequent or problematic use. Although individual differences in neurocognition appear to predict use severity, mechanistic neurocognitive risk factors with clear links to both behavior and neural circuitry have yet to be identified. Here, we aim to do so with an approach rooted in computational psychiatry, an emerging field in which formal models are used to identify candidate biobehavioral dimensions that confer risk for psychopathology.Objectives
We test whether lower efficiency of evidence accumulation (EEA), a computationally characterized individual difference variable that drives performance on the go/no-go and other neurocognitive tasks, is a risk factor for substance use in emerging adults.Methods and resultsIn an fMRI substudy within a sociobehavioral longitudinal study (n = 106), we find that lower EEA and reductions in a robust neural-level correlate of EEA (error-related activations in salience network structures) measured at ages 18–21 are both prospectively related to greater substance use during ages 22–26, even after adjusting for other well-known risk factors. Results from Bayesian model comparisons corroborated inferences from conventional hypothesis testing and provided evidence that both EEA and its neuroimaging correlates contain unique predictive information about substance use involvement.Conclusions
These findings highlight EEA as a computationally characterized neurocognitive risk factor for substance use during a critical developmental period, with clear links to both neuroimaging measures and well-established formal theories of brain function.
... Previous studies have proposed an important role for the NAc, and more generally, the reward/ motivational system, in chronification-or "stickiness"-of pain (6,7,(47)(48)(49)(50)(51), suggesting that patients with altered reward or aversion processing may be at a higher risk for developing chronic pain and more resistant to treatment. Moreover, NAc (part of the ventral striatum) is a key region for analgesia (52,53), including placebo analgesia (i.e., pain relief related to treatment context/expectancies) (54), potentially through its involvement in (reward-related) prediction errors (55,56). In our study, less pain relief was achieved by those individuals with stronger connectivity between the reward/ motivational and the sensorimotor network before treatment. ...
Background
A subset of patients with chronic pain who receive Exposure in vivo (EXP) treatment experience clinically-relevant pain intensity relief. Although pain relief is no explicit therapeutic target, it is important to understand how and why this concomitant effect occurs in some but not others. This longitudinal study therefore aims to characterize brain plasticity as well as to explore pre-treatment factors related to pain relief.
Methods
Resting-state fMRI data were acquired in 30 patients with chronic pain. Twenty-three patients completed EXP, and six-month follow-up data was available in n=20 (MRI data in n=17). Pain-free control data were acquired at two time-points (n=29, n=21). Seed-based resting-state functional connectivity (rsFC) analyses were performed, with seeds in amygdala, hippocampus and nucleus accumbens.
Results
Pain relief following EXP was highly variable, with 60% of patients reporting a clinically-relevant improvement. Amygdala rsFC with middle frontal gyrus decreased significantly over time in patients, not associated with pain relief. In contrast, greater pain relief was associated with greater decreases over time in hippocampus rsFC with precuneus, which was related to reductions in catastrophizing (EXP therapeutic target) too. Greater pain relief was also associated with lower pre-treatment nucleus accumbens rsFC with postcentral gyrus.
Conclusion
While changes in hippocampus rsFC were associated with pain relief following EXP, pre-treatment nucleus accumbens rsFC showed potential prognostic value. Our findings further support the importance of cortico-limbic circuitry in chronic pain, emphasizing its relation to pain relief and identifying potential underlying mechanisms and prognostic factors, warranting further testing in independent samples.
... The modulation of risk preferences according to energy reserves may be crucial for the adaptation to changes in the environment, in particular when resources are scarce (Houston, 1991;Kacelnik and Bateson, 1997;Stephens, 1981). Experiential decision-making relies on subcortical brain areas such as the striatum and the dopaminergic midbrain (Abler et al., 2006;Knutson et al., 2001;Niv et al., 2012;Tobler et al., 2007) that are targeted by circulating hormones that signal current energy reserves (Elmquist et al., 1998;Zigman et al., 2006). In particular, leptin inhibits and ghrelin activates dopaminergic neurons in the ventral tegmental area, and could therefore modulate learning and decision-making via the mesolimbic pathway (Abizaid et al., 2006;Figlewicz et al., 2007;Hommel et al., 2006). ...
We assess risks differently when they are explicitly described, compared to when we learn directly from experience, suggesting dissociable decision-making systems. Our needs, such as hunger, could globally affect our risk preferences, but do they affect described and learned risks equally? On one hand, explicit decision-making is often considered flexible and context-sensitive, and might therefore be modulated by metabolic needs. On the other hand, implicit preferences learned through reinforcement might be more strongly coupled to biological drives. To answer this, we asked participants to choose between two options with different risks, where the probabilities of monetary outcomes were either described or learned. In agreement with previous studies, rewarding contexts induced risk-aversion when risks were explicitly described, but risk-seeking when they were learned through experience. Crucially, hunger attenuated these contextual biases, but only for learned risks. The results suggest that our metabolic state determines risk-taking biases when we lack explicit descriptions.
... Broad evidence suggests that the brain implements computations similar to RL: dopamine neurons generate rewardprediction errors (19,20), and a widespread network of frontal cortical regions (21) and basal ganglia (22,23) represents actionvalues. Specific brain circuits thereby form "RL loops" (17,24), in which learning is implemented through the continuous updating of action-values (11,25). ...
Humans have the fascinating ability to achieve goals in a complex and constantly changing world, still surpassing modern machine-learning algorithms in terms of flexibility and learning speed. It is generally accepted that a crucial factor for this ability is the use of abstract, hierarchical representations, which employ structure in the environment to guide learning and decision making. Nevertheless, how we create and use these hierarchical representations is poorly understood. This study presents evidence that human behavior can be characterized as hierarchical reinforcement learning (RL). We designed an experiment to test specific predictions of hierarchical RL using a series of subtasks in the realm of context-based learning and observed several behavioral markers of hierarchical RL, such as asymmetric switch costs between changes in higher-level versus lower-level features, faster learning in higher-valued compared to lower-valued contexts, and preference for higher-valued compared to lower-valued contexts. We replicated these results across three independent samples. We simulated three models—a classic RL, a hierarchical RL, and a hierarchical Bayesian model—and compared their behavior to human results. While the flat RL model captured some aspects of participants’ sensitivity to outcome values, and the hierarchical Bayesian model captured some markers of transfer, only hierarchical RL accounted for all patterns observed in human behavior. This work shows that hierarchical RL, a biologically inspired and computationally simple algorithm, can capture human behavior in complex, hierarchical environments and opens the avenue for future research in this field.
... For example, functional magnetic resonance imaging (fMRI) studies have demonstrated that the VS responses not only are consistently greater for immediate compared with delayed rewards [9][10][11] but also explicitly track the delay-discounted subjective value rather than the objective magnitude of rewards [12][13][14] . Similarly, VS activity is strongly associated with expected reward such that less probable rewards elicit decreased responses 15,16 , and tracks with the probability-discounted subjective value 13 . Moreover, it has been proposed that the VS encodes subjective-value signals that are domain general, such that different types of cost/benefits are incorporated and represented on a common scale 17 . ...
The ventral striatum is believed to encode the subjective value of cost–benefit options; however, this effect has notably been absent during choices that involve physical effort. Previous work in freely moving animals has revealed opposing striatal signals, with greater response to increasing effort demands and reduced responses to rewards requiring effort. Yet, the relationship between these conflicting signals remains unknown. Using functional magnetic resonance imaging with a naturalistic maze-navigation paradigm, we identified functionally segregated regions within the ventral striatum that separately encoded effort activation, movement initiation and effort discounting of rewards. In addition, activity in regions associated with effort activation and discounting oppositely predicted striatal encoding of effort during effort-based decision-making. Our results suggest that the dorsomedial region hitherto associated with action may instead represent the cost of effort and raise fundamental questions regarding the interpretation of striatal ‘reward’ signals in the context of effort demands. This has implications for uncovering the neural architecture underlying motivated behaviour.
... Evidence from human molecular imaging demonstrated abnormal reward processing in the nucleus accumbens in OCD (26): it reported attenuated activations for reward anticipation, but did not examine learning or prediction error signaling. The nucleus accumbens has notably been associated with coding prediction error in healthy controls (27)(28)(29)(30). The concept of negative prediction error is closely related to that of error monitoring, which is reliably abnormal in OCD. ...
Background: Patients with obsessive-compulsive disorder (OCD) have been found to show exaggerated error responses and prediction error learning signals in a variety of EEG and fMRI tasks, with data converging on the anterior cingulate cortex as a key locus of dysfunction. Although there is considerable evidence linking prediction error processing to dopaminergic function, this has not been studied in the context of OCD.
Methods: We studied OCD patients (n=18) and controls (n=18) whilst they learned probabilistic associations between abstract stimuli and monetary rewards in the fMRI scanner involving administration (on separate visits) of: a dopamine receptor agonist, pramipexole 0.5mg; a dopamine receptor antagonist, amisulpride 400mg, and placebo. We fitted a Q-learning computational model to fMRI prediction error responses; group differences were examined in anterior cingulate and nucleus accumbens regions of interest.
Results: There were no significant group, drug or interaction effects in number of correct choices; computational modeling suggested a marginally significant difference in learning rates between groups. OCD patients showed abnormally strong signaling of prediction errors during omission of an expected reward, with normalization by pramipexole and amisulpride. Exaggerated cingulate prediction error signaling to omitted reward in placebo was related to trait subjective difficulty in self-regulating behavior in OCD.
Conclusions: These data support cingulate dysfunction during reward processing in OCD, and remediation by dopaminergic modulation, suggesting that exaggerated cingulate error signals in OCD may be of dopaminergic origin. The results help to illuminate the mechanisms through which dopamine receptor antagonists achieve therapeutic benefit in OCD.
... In particular, the computational neuroscience of perception, learning, and decision making has now reached a stage of maturity, both in terms of its methods and models and in terms of the reproducibility of the ensuing results. For example, neuroscientific evidence that basal ganglia encode the reward prediction error that enables reinforcement learning (i.e., learning from reward feedbacks) has been found repetitively in monkeys (Fiorillo et al., 2003;Schultz et al., 1997) and humans (Abler et al., 2006;Diederen et al., 2016;Garrison et al., 2013). From a methodological standpoint, this line of study is remarkable for two reasons. ...
Computational investigations of learning and decision making suggest that systematic deviations to adaptive behavior may be the incidental outcome of biological constraints imposed on neural information processing. In particular, recent studies indicate that range adaptation, i.e., the mechanism by which neurons dynamically tune their output firing properties to match the changing statistics of their inputs, may drive plastic changes in the brain’s decision system that induce systematic deviations to rationality. Here, we ask whether behaviorally-relevant neural information processing may be distorted by other incidental, hard-wired, biological constraints, in particular: Hebbian plasticity. One of our main contributions is to propose a simple computational method for identifying (and comparing) the neural signature of such biological mechanisms or constraints. Using ANNs (i.e., artificial neural network models) and RSA (i.e., representational similarity analysis), we compare the neural signatures of two types of hard-wired biological mechanisms/constraints: namely, range adaptation and Hebbian plasticity. We apply the approach to two different open fMRI datasets acquired when people make decisions under risk. In both cases, we show that although peoples’ apparent indifferent choices are well explained by biologically-constrained ANNs, choice data alone does not discriminate between range adaptation and Hebbian plasticity. However, RSA shows that neural activity patterns in bilateral Striatum and Amygdala are more compatible with Hebbian plasticity. Finally, the strength of evidence for Hebbian plasticity in these structures predicts inter-individual differences in choice inconsistency.
... It is well-suited to practical experimental manipulation, allowing modification of valence magnitude and distinction between the behaviourist concepts of reward and punishment through delivery or deduction of reward. The MIDT (and modified variants) is capable of capturing multiple aspects of reward processing including initial presentation of an incentive, approach/avoidance behaviour, reward learning, delayed reward discounting (Lutz and Widmer 2014) and even reward prediction error (Abler, Walter et al. 2006). ...
Thesis, MD (res) Psychological Medicine
... Reward/Reinforcement -The level of reward associated with a particular branded stimulus is essential information. fMRI studies have shown that activation of the dopaminereleasing Nucleus Accumbens can give valuable insight into consumer preference (i.e., items that are more desired or are expected to lead to a gain (Knutson et al., 2008;Abler et al., 2006;Cohen et al., 2009). Commercial access to fMRI machines remains limited, but there is a potential proxy for striatal dopamine: spontaneous eye blink rate (EBR). ...
Dual-process theories of consumer decision making, comprising both intuitive (System 1) and trained (System 2) processes, now dominate the consumer neuroscience marketplace. Although this has been beneficial, in practice this has led to confusion around the use of implicit and explicit measures. Dual-process theories also lack specificity around the diverse array of neural systems that are lumped together under the term System 1. Here we propose an enhancement called System N. System N provides a holistic framework for integrating explicit (verbal) responses and the diverse array of implicit brain and body measures that are now available (e.g., EEG, heart rate, eye blink rate, pupillometry, electrodermal response, automated facial affect coding, etc.). System N is based on our current understanding of arousal, attention, emotion, reward, habit, internal monologue, learning and effort, and includes state and trait information as relevant priors that shape consumer response. System N is considered iterative and offers a practical means to integrate priors, moment-by-moment brain and body measures, and post-hoc measures to better understand consumer decision making.
... This theory suggests that phasic bursts of neural DA activity and the transient DA release events they give rise to, encode the perceived value of an outcome that is signaled by a predictive cue (Hart et al., 2014;Hollerman and Schultz, 1998). The RPE theory has been supported by converging lines of evidence using multiple experimental approaches, including invasive neural monitoring in animal models, neuroimaging in humans, and computational predictions (Abler et al., 2006;Daw and Doya, 2006;Hart et al., 2014;Pagnoni et al., 2002). Additional fast-scan cyclic voltammetry studies confirm that transient DA release events in the NAc are evoked by cues predicting a valued outcome (Hamid et al., 2016;Sackett et al., 2017). ...
Endocannabinoids (eCBs) are neuromodulators that influence a wide range of neural systems and behaviors. In the current review, we describe our recent research showing how eCBs, particularly 2-arachidonoylglycerol (2-AG), concurrently shape mesolimbic dopamine (DA) release and associated behavior. We will restrict our discussion by emphasizing three distinct behaviors: reward seeking, interval timing, and active avoidance. During reward seeking we find that 2-AG is necessary to observe cue-evoked DA release events that are thought to represent the value of a rewarding outcome. We then describe data showing that 2-AG modulates unique patterns of DA release and behavior observed under conditions of periodic reinforcement. These data are discussed within the context of interval timing and adjunctive behavior. eCB modulation of DA release is also implicated in defensive behavior, including the avoidance of harm. As in reward seeking, our data suggest that the concentration of DA that is evoked by a warning signal can represent the value of an avoidance outcome. And, disrupting eCB signaling concomitantly reduces the concentration of the avoidance value signal and active avoidance. Disruptions in reward seeking, interval timing, and defensive behavior are commonly observed in a variety of movement disorders (e.g., Parkinson’s and Huntington’s disease) and disorders of motivation (e.g., addiction). We believe our data on eCB-DA interactions have implications for the development of novel pharmacotherapies to treat these disorders. Thus, we conclude by discussing how eCB pharmacology might be harnessed to treat disorders of movement and motivation.
... In human functional MRI (fMRI) studies, VS activations scale with RPE (3,8,21). VS blood-oxygen level-dependent (BOLD) responses and midbrain dopaminergic neuronal firing are correlated (22), supporting an RPE interpretation of the VS. ...
Do dopaminergic reward structures represent the expected utility of information similarly to a reward? Optimal experimental design models from Bayesian decision theory and statistics have proposed a theoretical framework for quantifying the expected value of information that might result from a query. In particular, this formulation quantifies the value of information before the answer to that query is known, in situations where payoffs are unknown and the goal is purely epistemic: That is, to increase knowledge about the state of the world. Whether and how such a theoretical quantity is represented in the brain is unknown. Here we use an event-related functional MRI (fMRI) task design to disentangle information expectation, information revelation and categorization outcome anticipation, and response-contingent reward processing in a visual probabilistic categorization task. We identify a neural signature corresponding to the expectation of information, involving the left lateral ventral striatum. Moreover, we show a temporal dissociation in the activation of different reward-related regions, including the nucleus accumbens, medial prefrontal cortex, and orbitofrontal cortex, during information expectation versus reward-related processing.
... The RPE represents the difference between expected and actual received reward that support an error-dependent update of value estimates for better prediction of future rewards. In humans, RPE signal was found to be encoded by VS (Abler, Walter, Erk, Kammerer, & Spitzer, 2006;Berns, McClure, Pagnoni, & Montague, 2001;McClure, Berns, & Montague, 2003), and is likely complemented and enhanced by striatumamygdala interactions (Ernst et al., 2005;Watanabe, Sakagami, & Haruno, 2013). ...
Background
Depressive episodes experienced in unipolar (UD) and bipolar (BD) disorders are characterized by anhedonia and have been associated with abnormalities in reward processes related to reward valuation and error prediction. It remains however unclear whether these deficits are associated with familial vulnerability to mood disorders.
Methods
In a functional magnetic resonance imaging study, we evaluated differences in the expected value (EV) and reward prediction error (RPE) signals in ventral striatum (VS) and prefrontal cortex between three groups of monozygotic twins: affected twins in remission for either UD or BD ( n = 53), their high-risk unaffected co-twins ( n = 34), and low-risk twins with no family history of mood disorders ( n = 25).
Results
Compared to low-risk twins, affected twins showed lower EV signal bilaterally in the frontal poles and lower RPE signal bilaterally in the VS, left frontal pole and superior frontal gyrus. The high-risk group did not show a significant change in the EV or RPE signals in frontostriatal regions, yet both reward signals were consistently lower compared with low-risk twins in all regions where the affected twins showed significant reductions.
Conclusion
Our findings strengthen the notion that reduced valuation of expected rewards and reduced error-dependent reward learning may underpin core symptom of depression such as loss of interest in rewarding activities. The trend reduction in reward-related signals in unaffected co-twins warrants further investigation of this effect in larger samples and prospective follow-up to confirm possible association with increased familial vulnerability to mood disorders.
... A number of explanations might account for this finding. First, striatal dopaminergic activity in response to reward has been shown to reflect not only the absolute value of an outcome, but whether it is better or worse than expected, i.e., a prediction error (32,33). Given our hypothesis about an internal focus of attention in high- ruminators, it is possible that a tendency for greater rumination interferes with calculation of expected values, leading to greater reward prediction errors. ...
Background
Ruminative responding involves repetitive and passive thinking about one’s negative affect. This tendency interferes with initiation of goal-directed rewarding strategies, which could alleviate depressive states. Such reward-directed response selection has been shown to be mediated by ventral striatum/nucleus accumbens (VS/NAcc) function. However, to date, no study has examined whether trait rumination relates to VS/NAcc functionality. Here, we tested whether rumination moderates VS/NAcc function both in response to reward and during a ruminative state.Methods
Trait rumination was considered dimensionally using Rumination Response Scale (RRS) scores. Our sample (N = 80) consisted of individuals from a community sample and from patients diagnosed with major depressive disorder, providing a broad range of RRS scores. Participants underwent fMRI to assess two modes of VS/NAcc functionality: 1) in response to reward, and 2) during resting-state, as a proxy for ruminative state. We then tested for associations between RRS scores and VS/NAcc functional profiles, statistically controlling for overall depressive symptom severity.ResultsRRS scores correlated positively with VS/NAcc response to reward. Furthermore, we noted that higher RRS scores were associated with increased ruminative-dependent resting-state functional connectivity of the VS/NAcc with the left orbitofrontal cortex.Conclusions
These findings suggest that ruminative tendencies manifest in VS/NAcc reward- and rumination-related functions, providing support for a theoretical-clinical perspective of rumination as a habitual impairment in selection of rewarding, adaptive coping strategies.
Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.
Humans can think about possible states of the world without believing in them, an important capacity for high-level cognition. Here, we use fMRI and a novel "shell game" task to test 2 competing theories about the nature of belief and its neural basis. According to the Cartesian theory, information is 1st understood, then assessed for veracity, and ultimately encoded as either believed or not believed. According to the Spinozan theory, comprehension entails belief by default, such that understanding without believing requires an additional process of "unbelieving." Participants (N = 70) were experimentally induced to have beliefs, desires, or mere thoughts about hidden states of the shell game (e.g., believing that the dog is hidden in the upper right corner). That is, participants were induced to have specific "propositional attitudes" toward specific "propositions" in a controlled way. Consistent with the Spinozan theory, we found that thinking about a proposition without believing it is associated with increased activation of the right inferior frontal gyrus. This was true whether the hidden state was desired by the participant (because of reward) or merely thought about. These findings are consistent with a version of the Spinozan theory whereby unbelieving is an inhibitory control process. We consider potential implications of these results for the phenomena of delusional belief and wishful thinking.
Introduction:
Disorders of arousal (DOA) are parasomnias that emerge from incomplete arousal out of Non-Rem Sleep (NREM) and lead to a broad variety of emotional and motor behaviours. Increasing evidence supports the hypothesis that specific psychopathological traits contribute to the multifactorial origin of these phenomena. The aim of the current multicenter study was to compare the personality profile of children and adolescents with and without DOA using the Junior Temperament and Character Inventory (JTCI).
Methods:
We enrolled 36 patients with a diagnosis of DOA (mean age of 11 ± 3 years, 64% males), and 36 healthy age and gender matched control subjects (mean age of 11.2 ± 3.6, years, 67% males). Their parents completed the Paris Arousal Disorder Severity Scale (PADSS), the Sleep Disturbance Scale for Children (SDSC) and the JTCI.
Results:
Patients with DOA reached significantly higher levels compared to their control group in total PADSS (p < 0.0001) and in total SDSC (p < 0.0001). They also displayed higher scores in novelty seeking (p = 0.005), harm avoidance (p = 0.01), self-transcendence (p = 0.006) JTCI subscales, and lower scores on the self-directedness subscale (p = 0.004).
Conclusion:
Our pediatric sample with DOA exhibited specific psychobiological personality traits compared to age and gender matched subjects without DOA. These results shed light on new possible etiopathogenetic mechanisms, as TCI traits have been linked to specific genetic variants and brain circuits, like the reward system. Prospective studies are required to assess the effect of targeted psychological/psychiatric treatment on DOA symptomatology.
The traditional approaches to classifying Human Error have provided a strong and beneficial approach to understanding the roles of humans in safety occurrences. However, as the demand for human performance has increased and with the rise of more automation, the line between man and machine continues to become increasingly blurred. Traditional Human Error analysis techniques are no longer sufficient for extracting the most valuable learning data from occurrences. Furthermore, there is a continuing need to strengthen Safety and Just culture. One means to achieving this is through developing ever greater trust in the ability of organisations to accurately investigate and report occurrences. This paper outlines a new 10 step approach to investigating human contribution to occurrences which is based on a version of predictive situational awareness in the brain that is derived from hierarchical predictive coding approaches. Utilising statistically based induction, the steps allow investigators to make an objective reconstruction of sub-conscious human error analysis and integrate it with traditional subjective feedback on conscious decisions made. Examples of system data that can provide further evidence to support the reconstructions is also discussed. This approach has been tested on more than 50 human factors related occurrences in Air Traffic Control in 2021/2022 and the results are introduced. It is further argued that using a refined approach and educating operators on the approach can significantly benefit Just Culture and lead to increases in trust and frequency of reporting because it will allow for the removal of biases by individual investigators. Future research should focus on validating the approach against current methods and testing it for occurrences further away from front line operators such as in non-safety related decision making.
When faced with uncertainty, individuals’ value-based decisions are influenced by the expected rewards and risks. Understanding how reward and risk are processed and integrated at the behavioral and neural levels is essential for building up utility theories. Using a modified monetary incentive delay task in which the mean of two possible outcomes (expected reward) and the standard deviation (SD) of the possible outcomes (risk) were parametrically manipulated and orthogonalized, we measured eye movements, response times (RTs), and brain activity when participants seek to secure a reward. We found that RTs varied as a function of the mean but not the SD of the potential reward, suggesting that expected rewards are the main driver of RTs. Moreover, the difference between gazes focused on high vs. low value rewards became smaller when the magnitude of the potential reward (mean of possible outcomes) was larger and when risk (SD of possible outcomes) became smaller, highlighting that reward and risk have different effects on attention deployment. Processing the mean reward activated the striatum. The positive striatal connectivity to the amygdala and negative striatal connectivity to the superior frontal gyrus were correlated with individuals’ sensitivity to the expected reward. In contrast, processing risk activated the anterior insula. Its positive connectivity to the ventromedial prefrontal cortex and negative connectivity to the anterior midcingulate cortex were correlated with individual differences in risk sensitivity, further suggesting the functional dissociation of reward and risk at the neural level. Our findings, based on several different measures, delineate the distinct representations of reward and risk in non-decision contexts and provide insight into how these utility parameters modulate attention, motivation, and brain networks.
Choices between smaller certain reward and larger riskier reward are referred to as risky decision making. Numerous functional magnetic resonance imaging (fMRI) studies have investigated the neural substrates of risky decision making via conventional univariate analytical approaches , revealing dissociable activation of decisions involving certain rewards and risky rewards. However, it is still unclear how the patterns of brain activity predict the choice that the individual will make. With the help of multi-voxel pattern analyses, which is more sensitive for evaluating information encoded in spatially distributed patterns, we showed that fMRI activity patterns represent viable signatures of certain and risky choice and individual differences. Notably, the regions involved in representation of value and risk and cognitive control play prominent roles in differentiating certain and risky choices as well as individuals with distinct risk preference. These results deepen our understanding of the neural correlates of risky decision making as well as emphasize the important roles of regions involved in representation of value and risk cognitive control in predicting risky decision making and individual differences.
The development of treatments for impaired cognition in schizophrenia has been characterized as the most important challenge facing psychiatry at the beginning of the twenty-first century. The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) project was designed to build on the potential benefits of using tasks and tools from cognitive neuroscience to better understanding and treat cognitive impairments in psychosis. These benefits include: (1) the use of fine-grained tasks that measure discrete cognitive processes; (2) the ability to design tasks that distinguish between specific cognitive domain deficits and poor performance due to generalized deficits resulting from sedation, low motivation, poor test taking skills, etc.; and (3) the ability to link cognitive deficits to specific neural systems, using animal models, neuropsychology, and functional imaging. CNTRICS convened a series of meetings to identify paradigms from cognitive neuroscience that maximize these benefits and identified the steps need for translation into use in clinical populations. The Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRaCS) Consortium was developed to help carry out these steps. CNTRaCS consists of investigators at five different sites across the country with diverse expertise relevant to a wide range of the cognitive systems identified as critical as part of CNTRICs. This work reports on the progress and current directions in the evaluation and optimization carried out by CNTRaCS of the tasks identified as part of the original CNTRICs process, as well as subsequent extensions into the Positive Valence systems domain of Research Domain Criteria (RDoC). We also describe the current focus of CNTRaCS, which involves taking a computational psychiatry approach to measuring cognitive and motivational function across the spectrum of psychosis. Specifically, the current iteration of CNTRaCS is using computational modeling to isolate parameters reflecting potentially more specific cognitive and visual processes that may provide greater interpretability in understanding shared and distinct impairments across psychiatric disorders.KeywordsCNTRaCSCNTRICSCognitive neurosciencePositive valence systemsSchizophrenia
Treating and preventing chronic pain require our understanding of how it develops and maintains. Löffler et al. (2022)¹ demonstrate that distinct operant learning signals in the vmPFC-NAc pathway predict the development and maintenance of chronic back pain.
Connectivity between the nucleus accumbens (NAc) and ventromedial prefrontal cortex (vmPFC) and reward learning independently predict the transition from acute to chronic back pain (CBP). However, how these predictors are related remains unclear. Using functional magnetic resonance imaging, we investigate NAc- and vmPFC-dependent reward learning in 50 patients with subacute back pain (SABP) and follow them over 6 months. Additionally, we compare 29 patients with CBP and 29 pain-free controls to characterize mechanisms of reward learning in the chronic stage. We find that the learning-related updating of the value of reinforcement (prediction error) in the NAc predicts the transition to chronicity. In CBP, compared with controls, vmPFC responses to this prediction error signal are decreased, but increased during a discriminative stimulus. Distinct processes of reward learning in the vmPFC and NAc characterize the development and maintenance of CBP. These could be targeted for the prevention and treatment of chronic pain.
Background
Bipolar disorder (BD), and especially the mania phenotype, is characterized by heightened reward responsivity and aberrant reward processing. In this longitudinal fMRI study, we investigated neuronal response during reward anticipation as the computed expected value (EV) and outcome evaluation as reward prediction error (RPE) in recently diagnosed patients with BD.
Methods
Eighty remitted patients with BD and 60 healthy controls (HC) underwent fMRI during which they performed a card guessing task. Of these, 41 patients and 36 HC were re-scanned after 16 months. We compared reward-related neural activity between groups at baseline and longitudinally and assessed the impact of mood relapse.
Results
Patients showed lower RPE signal in areas of the vlPFC than HC. In these regions, the HC showed decrease in RPE signal over time, which was absent in patients. Patients further exhibited decreased EV signal in the occipital cortex across baseline and follow-up. Patients who remained in remission showed normalization of the EV signal at follow-up. Baseline activity in the identified regions was not associated with subsequent relapse.
Limitations
Follow-up scans were only available in a relatively small sample. Medication status, follow-up time and BD illness duration prior to diagnosis varied.
Conclusions
Lower RPE signal in the vlPFC in patients with BD at baseline and its lack of normative reduction over time may represent a trait marker of dysfunctional reward-based learning or habituation. The increase in EV signal in the occipital cortex over time in patients who remained in remission may indicate normalization of reward anticipation activity.
Neuroplasticity follows nervous system injury in the presence or absence of rehabilitative treatments. Rehabilitative interventions can be used to modulate adaptive neuroplasticity, reducing motor impairment and improving activities of daily living in patients with brain lesions. Learning principles guide some rehabilitative interventions. While basic science research has shown that reward combined with training enhances learning, this principle has been only recently explored in the context of neurorehabilitation. Commonly used reinforcers may be more or less rewarding depending on the individual or the context in which the task is performed. Studies in healthy humans showed that both reward and punishment can enhance within-session motor performance; but reward, and not punishment, improves consolidation and retention of motor skills. On the other hand, neurorehabilitative training after brain lesions involves complex tasks (e.g., walking and activities of daily living). The contribution of reward to neurorehabilitation is incompletely understood. Here, we discuss recent research on the role of reward in neurorehabilitation and the needed directions of future research.
The subthalamic nucleus (STN) is a subcortical, glutamatergic, excitatory, relay nucleus that increases the inhibitory drive of the basal ganglia and suppresses action. It is of central relevance to the neuropsychological construct of inhibition, as well as the pathophysiology of Parkinson's disease (PD). Deep brain stimulation (DBS) of the STN (STN-DBS) is an established surgical treatment for PD that can be complicated by adverse neuropsychiatric side effects, most commonly characterized by impulsivity and mood elevation, although depression, anxiety, apathy, and cognitive changes have also been reported. Notwithstanding these adverse neuropsychiatric effects in PD, STN-DBS may also have a role in the treatment of refractory psychiatric disorders, as more is understood about the physiology of this nucleus and techniques in neuromodulation are refined. In this chapter, we link neuropsychiatric symptoms after STN-DBS for PD to the biological effects of electrode implantation, neurostimulation, and adjustments to dopaminergic medication, in the setting of neurodegeneration affecting cortico-striatal connectivity. We then provide an overview of clinical trials that have employed STN-DBS to treat obsessive-compulsive disorder and discuss future directions for subthalamic neuromodulation in psychiatry.
Adolescence has been linked to an enhanced tolerance of uncertainty and risky behavior and is possibly connected to an increased response toward rewards. However, previous research has produced inconsistent findings. To investigate whether these findings are due to different reward probabilities used in the experimental design, we extended a monetary incentive delay (MID) task by including three different reward probabilities. Using functional magnetic resonance imaging, 25 healthy adolescents and 22 adults were studied during anticipation of rewards in the VS. Differently colored cue stimuli indicated either a monetary or verbal trial and symbolized different reward probabilities, to which the participants were blinded. Results demonstrated faster reaction times for lower reward probabilities (33%) in both age groups. Adolescents were slower through all conditions and had less activation on a neural level. Imaging results showed a three-way interaction between age group x condition x reward probability with differences in percent signal change between adolescents and adults for the high reward probabilities (66%, 88%) while adolescents demonstrated differences for the lowest (33%). Therefore, previous inconsistent findings could be due to different reward probabilities, which makes examining these crucial for a better understanding of adolescent and adult behavior.
Introduction
: Reduced reactivity to pleasurable stimulation is a defining symptom of post-traumatic stress disorder (PTSD), but trauma exposure also increases the severity of many anxiety and mood disorders, including depression, social anxiety, and panic disorder, suggesting that reward system dysfunction might be pervasive in the internalizing disorders. The ventromedial prefrontal cortex (vmPFC) and ventral striatum are core components of the reward circuit and the current study assesses functional activity and connectivity in this circuit during emotional picture viewing in anxiety and mood disorder patients.
Method
: Functional brain activity (fMRI) and functional coupling in the fronto-striatal circuit were measured in a large sample of patients diagnosed with anxiety and mood disorders (n=155) during affective scene viewing as it varied with trauma exposure and temperament.
Results
: In women, but not men, blunted fronto-striatal connectivity was associated with increased posttraumatic anhedonic symptoms, whereas the amplitude of functional activity was not related to trauma exposure. In both men and women, reduced fronto-striatal coupling was associated with decreases in temperamental positive affect. When predicting fronto-striatal connectivity, temperament and posttraumatic symptomology accounted for independent proportions of variance.
Limitations
: In this civilian sample of anxiety disorder patients, men reported very little trauma-related symptomology.
Conclusions
: Because dysfunctional reward processing due to trauma and temperament is pervasive across the internalizing disorder spectrum, assessing the integrity of the fronto-striatal reward circuit could provide important information in diagnostic and treatment protocols.
Introduction:
Phenomena related to reward responsiveness have been extensively studied in their associations with substance use and socioemotional functioning. One important task in this literature is the Monetary Incentive Delay (MID) task. By cueing and delivering performance‐contingent reward, the MID task has been demonstrated to elicit robust activation of neural circuits involved in different phases of reward responsiveness. However, systematic evaluations of common MID task contrasts have been limited to between‐study comparisons of group‐level activation maps, limiting their ability to directly evaluate how researchers’ choice of contrasts impacts conclusions about individual differences in reward responsiveness or brain‐behavior associations.
Methods:
In a sample of 104 participants (Age Mean = 19.3, SD = 1.3), we evaluate similarities and differences between contrasts in: group‐ and individual‐level activation maps using Jaccard's similarity index, region of interest (ROI) mean signal intensities using Pearson's r, and associations between ROI mean signal intensity and psychological measures using Bayesian correlation.
Results:
Our findings demonstrate more similarities than differences between win and loss cues during the anticipation contrast, dissimilarity between some win anticipation contrasts, an apparent deactivation effect in the outcome phase, likely stemming from the blood oxygen level‐dependent undershoot, and behavioral associations that are less robust than previously reported.
Conclusion:
Consistent with recent empirical findings, this work has practical implications for helping researchers interpret prior MID studies and make more informed a priori decisions about how their contrast choices may modify results.
Various stimuli have been employed as reinforcers in preclinical rodent models to elucidate the underpinnings of reward at a molecular and circuit level, with the release of dopamine (DA) in the nucleus accumbens (NAc) as a well-replicated, physiological correlate. Many factors, however, including strain differences, sex, prior stress, and reinforcer administration protocols can influence reward responding and DA release. Although previous evidence indicates that access to the home cage can be an effective reinforcer in behavioral tasks, whether this simple environmental manipulation can trigger DA release in the NAc has not been demonstrated. Here, using fiber photometric recordings of in vivo NAc dopamine release from a genetically-encoded DA sensor, we show that the movement of animals from the home cage to a clear, polycarbonate recording chamber evokes little to no DA release following initial exposure whereas returning animals from the recording chamber to a clean, home-like cage or to the home cage robustly triggers the release of DA, comparable in size to that observed with a 10 mg/kg i.p. Cocaine injection in the recording chamber. Although DA release can be evoked in moving mice to a clean cage, this release was significantly augmented when moving animals from the clean cage to the home cage. Our data provide direct evidence that home cage return from a foreign environment results in a biochemical change consistent with that of a rewarding stimulus. This simple environmental manipulation provides a minimally invasive approach to study the reward circuitry underlying an ethologically relevant reinforcer, return to the safe confines of “home”. The home cage – DA release paradigm may also represent a biomarker-driven paradigm for the evaluation of genetic and experiential events that underlie anhedonic states, characteristic of major mood disorders, and to present new opportunities to identify their treatments.
Rationale
Substance use peaks during the developmental period known as emerging adulthood (ages 18–25), but not every individual who uses substances during this period engages in frequent or problematic use. Although individual differences in neurocognition appear to predict use severity, mechanistic neurocognitive risk factors with clear links to both behavior and neural circuitry have yet to be identified. Here we aim to do so with an approach rooted in computational psychiatry, an emerging field in which formal models are used to identify candidate biobehavioral dimensions that confer risk for psychopathology.
Objectives
We test whether lower efficiency of evidence accumulation (EEA), a computationally-characterized individual difference variable that drives performance on the go/no-go and other neurocognitive tasks, is a risk factor for substance use in emerging adults.
Methods and Results
In an fMRI substudy within a sociobehavioral longitudinal study ( n =106), we find that lower EEA and reductions in a robust neural-level correlate of EEA (error-related activations in salience network structures) measured at ages 18–21 are both prospectively related to greater substance use during ages 22–26, even after adjusting for other well-known risk factors. Results from Bayesian model comparisons corroborated inferences from conventional hypothesis testing and provided evidence that both EEA and its neuroimaging correlates contain unique predictive information about substance use involvement.
Conclusions
These findings highlight EEA as a computationally-characterized neurocognitive risk factor for substance use during a critical developmental period, with clear links to both neuroimaging measures and well-established formal theories of brain function.
The ventral striatum is believed to encode the subjective value of cost/benefit options; however, this effect has strikingly been absent during choices that involve physical effort. Prior work in freely-moving animals has revealed opposing striatal signals, with greater response to increasing effort demands and reduced responses to rewards requiring effort. Yet, the relationship between these conflicting signals remains unknown. Using fMRI with a naturalistic maze-navigation paradigm, we identified functionally-segregated regions within ventral striatum that separately encoded action initiation, effort activation, and effort discounting of rewards. Additionally, activity in regions associated with effort activation and discounting oppositely predicted striatal encoding of effort during effort-based decision-making. Our results suggest that the dorsomedial region hitherto associated with action may instead represent the cost of effort, and raises fundamental questions regarding the interpretation of striatal reward signals in the context of effort demands. This has implications for uncovering the neural architecture underlying motivated behavior.
The dopamine system is associated with reward processes in both gambling disorder and substance use disorder, and may constitute a common neurobiological underpinning in addiction. The present review examines differences and similarities of dopaminergic reward processes in gambling disorder and substance use disorder. First, it is suggested that baseline binding potentials of the dopamine system may not be a common pathway, since substance use disorder is associated with reduced binding potentials, whereas gambling disorder is not. Second, it is suggested that dopaminergic reward response may be not a common pathway, since substance use disorder is associated with a blunted dopamine response toward drugs, while conflicting findings of reward response has been reported in gambling disorder. Instead, it is suggested that the anticpatory dopamine response may constitute a common underpinning of gambling disorder and substance use disorder, which may be associated with increased dopamine activity in both types of disorder, and does not involve the intake of substances. The notion of the anticipatory dopamine response as a common underpinning of gambling disorder and substance use disorder is consistent with dopaminergic models of addictions such as the incentive-sensitization model, the ingrative neurodevelopmental model of vulnerability toward addiction and the reward prediction error model.
ResearchGate has not been able to resolve any references for this publication.