ArticlePublisher preview available

Surprise, value and control in anterior cingulate cortex during speeded decision-making

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

Activity in the dorsal anterior cingulate cortex (dACC) is observed across a variety of contexts, and its function remains intensely debated in the field of cognitive neuroscience. While traditional views emphasize its role in inhibitory control (suppressing prepotent, incorrect actions), recent proposals suggest a more active role in motivated control (invigorating actions to obtain rewards). Lagging behind empirical findings, formal models of dACC function primarily focus on inhibitory control, highlighting surprise, choice difficulty and value of control as key computations. Although successful in explaining dACC involvement in inhibitory control, it remains unclear whether these mechanisms generalize to motivated control. In this study, we derive predictions from three prominent accounts of dACC and test these with functional magnetic resonance imaging during value-based decision-making under time pressure. We find that the single mechanism of surprise best accounts for activity in dACC during a task requiring response invigoration, suggesting surprise signalling as a shared driver of inhibitory and motivated control. The role of the anterior cingulate cortex (ACC) in decision-making and cognitive control is the subject of a long-standing debate. Vassena et al. tested the dominant accounts in the same paradigm and found that the ACC signals the difference between predicted and actual outcomes.
Factors influencing Akaike Weight estimates Our analysis of fMRI data using polynomial functions fit to beta estimates from BOLD data is substantially different from classical or model-based fMRI analyses. In order to explore what factors may influence our results, as well as how nested polynomial functions independently account for observed patterns of activity in our data, we carried out additional simulations and analyses. First, we asked what factors affect the ability of our analysis approach to identify quartic effects using Akaike Weights. To answer this, we conducted six simulations of synthetic data generated by a quartic polynomial equation while varying the noise, number of subjects, and shape of the function. 10,000 simulation runs were conducted for each condition, and Akaike Weights calculated to obtain a distribution of the likelihood of Akaike Weights. The results of these simulations suggest that, as in classical univariate analyses, the likelihood of obtaining an Akaike Weight >0.999 for the quartic polynomial function (when the data was in fact generated by a quartic polynomial function) depends both on the quantity and noisiness of the data itself. That is, with less data and more noise, it becomes less likely that our analyses will assign the quartic polynomial function an Akaike Weight of 0.999 or higher. Additionally, changes in the shape of the quartic function that render it more similar to other (quadratic) functions reduce the likelihood of calculating an Akaike Weight >0.999 for the quartic function. These results suggest our analysis approach parallels typical fMRI analyses in terms of the factors that influence the likelihood of observing significant effects.
… 
Behavioural strategies and reactive and proactive control processes a–c, Logistic regression against left-vs-right choice data indicates that subjects adopted a strategy that weighted the maximum (Max.) available outcome of an option more than the minimum (Min.) possible outcome. This strategy is observed when the maximum and minimum values of each option are used as predictor variables (a), as well as when the difference between maximum and minimum values and expected values is used to predict choice behaviour (b). Subjects are slightly risk-averse when considering only the maximum value of each option (c). EV, expected value; a.u., arbitrary units. Error bars are s.e.m. See additional simulations in Supplementary Methods. d, In cognitive control, repetitions of trials with high and low control demands tend to produce more rapid RTs relative to trials with markedly different control demands from previous trials. Our behavioural data reproduce the effect of trial sequences (conflict adaptation), a widely replicated effect in studies of cognitive control. e, Unsigned feedback prediction errors contributed to longer RTs on trials following immediately, consistent with post-surprise slowing accounts of reactive control³². f, Drift-diffusion modelling of decision parameters suggests both bottom-up and top-down influences on responses. Estimated drift rates (top) are higher for trials in which the difference in expected value between the two options is higher than for those with similarly valued options (posterior probability of equal drift rates < 0.001), suggesting that identification of high-valued options is easier. Estimates for the decision threshold parameter (bottom) are lower for hard trials than for easy trials (posterior probability of equal decision thresholds = 0.022), implying that control is required in hard trials in adjusting the decision threshold to ensure a timely response. Decision threshold adjustment is a marker of proactive control processes14,28.
… 
This content is subject to copyright. Terms and conditions apply.
Articles
https://doi.org/10.1038/s41562-019-0801-5
1Donders Institute for Brain Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands. 2Department of Experimental Psychology, Ghent
University, Ghent, Belgium. 3Center for Complex Systems & Brain Sciences, Florida Atlantic University, Boca Raton, FL, USA. *e-mail: walexander@fau.edu
Activity in the dorsal anterior cingulate cortex (dACC)
and surrounding regions in the medial prefrontal cortex
(mPFC) is routinely observed in neuroimaging studies of
cognitive control and decision-making1. Consequently, a number of
theoretical and computational accounts have been developed in the
past two decades to describe the role and function of dACC in cog-
nitive control2. Generally, cognitive control entails the need to sup-
press an incorrect, prepotent response to generate a correct, but less
automatic, response. Early computational models of dACC function
therefore principally addressed tasks thought to involve response
selection and inhibition, assigning the region roles in signalling
behavioural error, detecting and resolving response conflict3, select-
ing appropriate motor responses4 or predicting the likelihood of an
incorrect response5.
Although cognitive control research has traditionally focused on
response inhibition, recent work has highlighted the interactions
between control and motivation2,6. Motivation refers to the drive
to pursue specific behavioural goals to obtain desired outcomes
(such as rewards (extrinsic motivation) or other states perceived as
rewarding (intrinsic motivation)). This line of work defines motiva-
tion as the “invigorating impact, on both behaviour and cognition,
of prospective reward (both extrinsic reward such as money and
instrinsic reward tied to the satisfaction of self-relevant behavioural
goals”7. Here, we refer to motivated control as the process promot-
ing successful selection and invigoration of a behavioural response
leading to a valuable desired outcome. In this view, exerting con-
trol is costly, but also valuable, as it allows the securing of a pro-
spective reward8,9. Individuals consistently tend to avoid exerting
mental effort when possible10, and preparing for a cognitively effort-
demanding task is associated with increased dACC activity11,12.
Interestingly, activity in the same region appears to correlate with
the expectation of higher reward following task completion13. The
overlap of cognitive effort and reward signals within dACC has led
to development of new accounts of the region’s function, assigning
it a role in computing benefits and costs of actions, and integrating
these in a ‘net-value’ driving adaptive behavioural selection insitu-
ations involving exertion of cognitive control or physical effort9.
In line with these findings, an influential theoretical framework
has been proposed, the expected value of control account (EVC;
ref. 14). The EVC posits that activity in dACC reflects ‘expected
value of control’—a trade-off between cost and benefits resulting in
the selection of an optimal control signal.
In parallel, a growing amount of evidence supports a key role of
the dACC in tracking the likelihood of events (such as responses
and outcomes given a certain stimulus), and computing the dis-
crepancy between predicted and actual events (that is, prediction
errors), formalized in the predicted response outcome model (PRO;
ref. 15), which posits that the dACC signal reflects an ongoing com-
parison between expected and observed events: any unexpected,
and therefore ‘surprising’, event will produce increased activity in
dACC, and this signal contributes to updating of future predictions.
This type of surprise could be termed epistemic control: dACC
activity reflects predictive signals and error signals when predic-
tions are not met. These signals may trigger behavioural adaptation
when necessary1620. This line of work successfully explains classical
inhibitory control effects as a function of likelihood of responses
and outcomes (errors, incongruent options and non-prepotent
responses are generally less likely, and therefore surprising). More
recently, this approach has also been applied to motivated control,
suggesting that motivationally relevant variables (for example, effort
requirements or potential reward amounts) may be monitored in a
similar fashion. In this framework, deciding to engage in effortful
behaviour is generally less likely and therefore the choice to engage
is associated with greater dACC activity (PRO–effort)21. Even in the
absence of subsequent invigoration, the choice itself to accept more
effortful tasks is infrequent (humans are generally effort-avoidant),
and would therefore elicit increased dACC activity (which, in epis-
temic control terms, reflects the likelihood of engaging in a task
given its specific effort and reward properties). The PRO–effort
proposal outlines how a likelihood-monitoring account of dACC
function holds potential for generalization to motivated invigora-
tion of behaviour. However, this account is yet to be tested against
the wide array of effort-related effects on behaviour and brain activ-
ity in the mPFC–dACC, especially considering that previous work
Surprise, value and control in anterior cingulate
cortex during speeded decision-making
Eliana Vassena 1,2, James Deraeve2 and William H. Alexander 2,3*
Activity in the dorsal anterior cingulate cortex (dACC) is observed across a variety of contexts, and its function remains intensely
debated in the field of cognitive neuroscience. While traditional views emphasize its role in inhibitory control (suppressing
prepotent, incorrect actions), recent proposals suggest a more active role in motivated control (invigorating actions to obtain
rewards). Lagging behind empirical findings, formal models of dACC function primarily focus on inhibitory control, highlight-
ing surprise, choice difficulty and value of control as key computations. Although successful in explaining dACC involvement in
inhibitory control, it remains unclear whether these mechanisms generalize to motivated control. In this study, we derive pre-
dictions from three prominent accounts of dACC and test these with functional magnetic resonance imaging during value-based
decision-making under time pressure. We find that the single mechanism of surprise best accounts for activity in dACC during
a task requiring response invigoration, suggesting surprise signalling as a shared driver of inhibitory and motivated control.
NATURE HUMAN BEHAVIOUR | VOL 4 | APRIL 2020 | 412–422 | www.nature.com/nathumbehav
412
Content courtesy of Springer Nature, terms of use apply. Rights reserved
... The vmPFC and OFC are predominantly associated with the evaluation of the relative value of alternative offers [15][16][17][18][19][20][21] , while the ACC plays a role in monitoring conflicting information and actions [22][23][24][25][26] , as well as the estimation of expected outcomes and the cognitive cost of the decision performance [27][28][29][30] . In particular, the dorsal ACC (dACC) has been linked to reward anticipation and cognitive effort computation 31,32 , and to the processing of delayed rewards across various decision-making contexts [33][34][35][36][37][38] . Neural signals in ACC have been previously associated with multi-trial 39 and virtual reward expectation 40 , with remarked impact on behavioral adjustments 41,42 . ...
... Finally, we found that higher accumulated tokens count, and easier Furthermore, the involvement of the dACC in token accumulation reinforcement is particularly noteworthy. Previous research has implicated the dACC in cognitive control 7,53 , conflict monitoring [22][23][24] , and value-based decision-making 12,14,25,37 . Our findings further support its role in encoding the subjective value of outcomes in a dynamic, multi-trial context. ...
Preprint
Full-text available
Normatively, our decisions ought to be made relative to our total wealth, but in practice, we make our decisions relative to variable, decision-time-specific set points. This predilection introduces a major behavior bias that is known as reference-point dependence in Prospect Theory, and that has close links to mental accounting. Here we examined neural activity in the dorsal anterior cingulate cortex (dACC) of macaques performing a token-based risky choice task, in which the acquisition of 6 tokens (accumulated over several trials) resulted in a jackpot reward. We find that subjects make faster and more accurate choices as the jackpot reward becomes more likely to be achieved, suboptimal behavior that can readily be explained by reference dependence. This biased behavior systematically covaries with the neural encoding of corresponding offer values. Moreover, we found significant enhancement in speed, accuracy and neural encoding strength for easier levels of difficulty in detecting the offer with the best expected value. These results suggest a neural basis of reference dependence biases in shaping decision-making behavior and highlight the critical role of value representations in dACC in driving those biases.
... The vmPFC and OFC are predominantly associated with the evaluation of the relative value of alternative offers [15][16][17][18][19][20][21] , while the ACC plays a role in monitoring conflicting information and actions [22][23][24][25][26] , as well as the estimation of expected outcomes and the cognitive cost of the decision performance [27][28][29][30] . In particular, the dorsal ACC (dACC) has been linked to reward anticipation and cognitive effort computation 31,32 , and to the processing of delayed rewards across various decision-making contexts [33][34][35][36][37] . Neural signals in ACC have been previously associated with multi-trial 38 and virtual reward expectation 39 , with remarked impact on behavioral adjustments 40,41 . ...
... Furthermore, the involvement of the dACC in token accumulation reinforcement is particularly noteworthy. Previous research has implicated the dACC in cognitive control 7,52 , conflict monitoring [22][23][24] , and value-based decision-making 12,14,25,36 . Our findings further support its role in encoding the subjective value of outcomes in a dynamic, multi-trial context. ...
Preprint
Full-text available
Normatively, our decisions ought to be made relative to our total wealth, but in practice, we make our decisions relative to variable, decision-time-specific set points. This predilection introduces a major behavior bias that is known as reference-point dependence in Prospect Theory, and that has close links to mental accounting. Here we examined neural activity in the dorsal anterior cingulate cortex (dACC) of macaques performing a token-based risky choice task, in which the acquisition of 6 tokens (accumulated over several trials) resulted in a jackpot reward. We find that subjects make faster and more accurate choices as the jackpot reward becomes more likely to be achieved, suboptimal behavior that can readily be explained by reference dependence. This biased behavior systematically covaries with the neural encoding of corresponding offer values. Moreover, we found significant enhancement in speed, accuracy and neural encoding strength for easier levels of difficulty in detecting the offer with the best expected value. These results suggest a neural basis of reference dependence biases in shaping decision-making behavior and highlight the critical role of value representations in dACC in driving those biases.
... Despite their many strengths, these past value-based experiments have been limited by their inability to determine whether purported integrator regions are accumulating evidence or instead representing unchosen values (Boorman et al., 2011;Kolling et al., 2016;Wittmann et al., 2016), decision conflict (Frömer et al., 2024;Hunt et al., 2018;Kaanders et al., 2021;Kolling et al., 2012;Shenhav et al., 2014Shenhav et al., , 2016Vassena et al., 2020), or time on task (Holroyd et al., 2018). In most experiments, these variables are highly correlated and difficult to distinguish. ...
... These results provide novel evidence for the neural mechanisms underlying the SSM process, as exemplified by the DDM, which appears to govern many types of decisions (Busemeyer, 1985;Ratcliff, 1978). The gaze modulation of the accumulated value signals in the pre-SMA provides critical evidence that this region indeed represents accumulated evidence, as opposed to unchosen values (Boorman et al., 2011;Kolling et al., 2016;Wittmann et al., 2016), decision conflict (Hunt et al., 2018;Kaanders et al., 2021;Kolling et al., 2012;Shenhav et al., 2014Shenhav et al., , 2016Vassena et al., 2020), or time on task (Holroyd et al., 2018). While accumulated evidence is typically correlated with these other measures, we were able to dissociate them by taking advantage of the fact that accumulated evidence, but not the other measures, are modulated by gaze location. ...
Preprint
Full-text available
When indicating a preference between two options, decision makers are thought to compare and accumulate evidence in an attention-guided process. Little is known about this process's neural substrates or how visual attention affects the representations of accumulated evidence. We conducted a simultaneous eye-tracking and fMRI experiment in which human subjects gradually learned about the value of two food-lotteries. With this design we were able to extend decisions over a prolonged time-course, manipulate the temporal onset of evidence, and therefore dissociate sampled and accumulated evidence. Consistent with past work, we found correlates of sampled evidence in ventromedial prefrontal cortex (vmPFC), and correlates of accumulated evidence in the prefrontal and parietal cortex. We also found that more gaze at an option increased its choice probability and that gaze amplified sampled-value signals in the vmPFC and ventral striatum. Most importantly, we found that gaze modulated accumulated-value signals in the pre-supplementary motor area (pre-SMA), providing novel evidence that visual attention has lasting effects on decision variables and suggesting that activity in the pre-SMA reflects accumulated evidence and not decision conflict. These results shed new light on the neural mechanisms underlying gaze-driven decision processes.
... Prior human and laboratory animal studies have repeatedly linked motivational anhedonia to dysfunction of corticostriatal reward networks [6][7][8] that support effortful behavior [9][10][11] and include dopamine (DA)-rich areas such as the striatum as well as brain regions encompassing the dorsomedial prefrontal cortex including the dorsal anterior cingulate cortex and surrounding paracingulate and presupplementary motor areas (herein referred to collectively as "dmPFC"). Human functional imaging studies as well as animal lesion studies have repeatedly implicated the dmPFC as a critical hub for effort-based decision-making [12][13][14][15][16], which appears to encode multiple decision-variables related to effort cost [17], choice difficulty [18] and effort-related expectation violation [12,19]. Taken together, the dmPFC and striatum have been found to encode distinct decision-variables related to effort-based choice and appear to be causally involved in the willingness to expend effort for rewards. ...
... As anticipated, a single dose of infliximab was associated with a significant group (placebo vs. infliximab) by time (baseline vs. 14 days) interaction (F (1,34) = 5.751, p = 0.022, η 2 = 0.145) such that patients receiving infliximab exhibited a significant reduction in CRP (t (19) ...
Article
Full-text available
Chronic, low-grade inflammation has been associated with motivational deficits in patients with major depression (MD). In turn, impaired motivation has been linked to poor quality of life across psychiatric disorders. We thus determined effects of the anti-inflammatory drug infliximab–a potent tumor necrosis factor (TNF) antagonist–on behavioral and neural measures of motivation in 42 medically stable, unmedicated MD patients with a C-reactive protein >3 mg/L. All patients underwent a double-blind, placebo-controlled, single-dose, randomized clinical trial with infliximab (5 mg/kg) versus placebo. Behavioral performance on an effort-based decision-making task, self-report questionnaires, and neural responses during event-related functional magnetic resonance imaging were assessed at baseline and 2 weeks following infusion. We found that relative to placebo, patients receiving infliximab were more willing to expend effort for rewards. Moreover, increase in effortful choices was associated with reduced TNF signaling as indexed by decreased soluble TNF receptor type 2 (sTNFR2). Changes in effort-based decision-making and sTNFR2 were also associated with changes in task-related activity in a network of brain areas, including dorsomedial prefrontal cortex (dmPFC), ventral striatum, and putamen, as well as the functional connectivity between these regions. Changes in sTNFR2 also mediated the relationships between drug condition and behavioral and neuroimaging measures. Finally, changes in self-reported anhedonia symptoms and effort-discounting behavior were associated with greater responses of an independently validated whole-brain predictive model (aka “neural signature”) sensitive to monetary rewards. Taken together, these data support the use of anti-inflammatory treatment to improve effort-based decision-making and associated brain circuitry in depressed patients with high inflammation.
... Unlike previous neuroimaging and pupil studies in ASD that focused on the post-feedback period, we concentrated on neural responses during the time window preceding the termination of the decision process (i.e., before a button press). Specifically, we analyzed decision-related neural changes in selected regions of interest (ROIs) previously shown to be involved in decision-making under uncertainty, including the medial prefrontal and adjacent anterior cingulate cortex (mPFC & ACC), as well as the anterior insula (AIC) and the ventral visual stream areas (VVS) (Kucyi & Parvizi, 2020;Monosov, 2017;Passingham & Toni, 2001;Payzan-LeNestour et al., 2013;Vassena et al., 2020). ...
Preprint
Full-text available
The ability to adjust brain resources to manage expected uncertainty is hypothesized to be impaired in autism spectrum disorder (ASD), though the evidence remains limited. To investigate this, we studied 23 neurotypical (NT) and 23 high-functioning adults with ASD performing a probabilistic two-alternative value-based task while undergoing magnetoencephalography (MEG) and pupillometry. The task comprised five sequential blocks with stable reward probabilities (70%:30%), but varying stimulus pairs and reward values, enabling the assessment of behavioral and neural adaptation to expected uncertainty. We analyzed the hit rate of advantageous choices, response times, and computational measures of prior belief strength and precision. To examine cortical activation during decision-making, we used magnetoencephalographic (MEG) source reconstruction to quantify α-β oscillation suppression in decision-relevant cortical regions within the pre-decision time window. Linear mixed models were applied to assess trial-by-trial effects. Behaviorally, ASD participants exhibited lower overall belief precision but intact probabilistic rule generalization, showing gradual performance improvement and strengthening of prior beliefs across blocks. However, unlike NT individuals, they did not show a progressive downscaling of neural activation during decision-making or a reduction in neural response to feedback signals as performance improved. Furthermore, on a trial-by-trial basis, increased belief precision in ASD was not associated with reduced cortical activation, a pattern observed in NT individuals. These findings suggest an atypically rigid and enhanced allocation of neural resources to advantageous decisions that individuals with ASD rationally judge as optimal. This pattern may reflect an aversive response to the irreducible uncertainty inherent in probabilistic decision-making.
... The different aspects of the error are processed in distinct neural circuits that operate on different timescales (Gabitov et al. 2020). Error-specific signals arise in an extended neural network that includes the anterior cingulate cortex (ACC; Carter et al. 1998;Holroyd et al. 2004, Alexander andBrown 2011;Vassena et al. 2020), basal ganglia (Bech et al. 2023) and cerebellum (Thach et al. 1992;Blakemore et al. 2001;Maschke et al. 2004;Smith and Shadmehr 2005;Xu-Wilson et al. 2009;Criscimagna-Hemminger et al. 2010;Hanajima et al. 2015). ...
Article
A substantial body of literature has focused on neural signals evoked by errors emerging during the execution of goal-directed actions. It is still unclear how motor cortex activity during movement execution relates to feedback error processing. To investigate this, we recorded primary motor cortex (M1) single-unit activity in rats during a grasping task. About half of the recorded neurons showed modulation of their firing activity that did not depend on success or failure, which we termed outcome-independent neurons. Other neurons showed a difference in their discharge profile when comparing successful and unsuccessful trials, which we called outcome-dependent neurons. Among both outcome-dependent and -independent neurons, we further distinguished neurons presenting their maximum firing rate in specific epochs as defined by the task. We compared the cortical distribution of outcome-independent and outcome-dependent neurons to cortical maps of complex forelimb movements evoked by intracortical microstimulation in additional animals. The majority of outcome-independent neurons was localized within the limb extension and paw open-closure movement representations. Outcome-dependent neurons were not clearly associated to particular motor representations. Cortical arrangement of neurons, both outcome-independent and outcome-dependent, and their correlation with distinct movement representations, can serve as indicator for anticipating potential outcomes before the conclusion of an action.
... Auditory stimulation can regulate the rhythm of the hippocampus, holding significance for enhancing memory and alleviating cognitive decline [130,131] . Studies on humans and nonhuman primates have revealed that there exists a common effective connectivity signal that directly connects the AC to the ventrolateral prefrontal cortex (VLPFC) and indirectly projects to the hippocampus [132] . The tendency of energy allocation is important for the metabolic balance of the brain. ...
Article
Full-text available
With the increasing aging of the population, there has been a growing focus on the association between age-related hearing loss (ARHL) and Alzheimer’s disease (AD). Given the extended latency period of AD, it has been assumed that mechanisms contributing to its pathogenesis may also contribute to ARHL. Abnormal deposition of beta-amyloid (Aβ) and tau protein in the brain serves as the primary etiological marker for AD, being produced long before the symptomatic onset of AD. The neurotoxicity induced by their unique oligomers not only exerts significant effects on the central nervous system but also exhibits similar impacts on certain peripheral organs. In this review, we analyze the factors leading to β-amyloid and tau production, explore their cascading effects and roles in AD progression, discuss the possible influence of auditory deprivation on cognitive functions, and investigate the potential causes of hearing loss by examining other peripheral organs in AD patients and model animals as examples. These findings provide further evidence supporting ARHL as an early indicator of AD.
Article
We propose a cognitive and neurobiological model by which curiosity regulates affective memory, by positively biasing memory encoding through the promotion of emotion regulation. We begin with a brief overview of curiosity's observed emotional effects. Then we introduce three prominent models of affective memory encoding to suggest that the dopaminergic modulation of encoding associated with curiosity may positively bias memory processes. We situate the role of curiosity role in emotion regulation relative to its promotion of abstract thinking and cognitive flexibility. We then identify the neural processes associated with abstraction and flexibility observed in the left inferior frontal gyrus, the dorsal anterior cingulate cortex, and the lateral prefrontal cortex as the neurobiological mechanisms underlying our framework.
Article
Full-text available
A neglected question regarding cognitive control is how control processes might detect situations calling for their involvement. The authors propose here that the demand for control may be evaluated in part by monitoring for conflicts in information processing. This hypothesis is supported by data concerning the anterior cingulate cortex, a brain area involved in cognitive control, which also appears to respond to the occurrence of conflict. The present article reports two computational modeling studies, serving to articulate the conflict monitoring hypothesis and examine its implications. The first study tests the sufficiency of the hypothesis to account for brain activation data, applying a measure of conflict to existing models of tasks shown to engage the anterior cingulate. The second study implements a feedback loop connecting conflict monitoring to cognitive control, using this to simulate a number of important behavioral phenomena.
Article
Full-text available
The frontal lobes are essential for human volition and goal-directed behavior, yet their function remains unclear. While various models have highlighted working memory, reinforcement learning, and cognitive control as key functions, a single framework for interpreting the range of effects observed in prefrontal cortex has yet to emerge. Here we show that a simple computational motif based on predictive coding can be stacked hierarchically to learn and perform arbitrarily complex goal-directed behavior. The resulting Hierarchical Error Representation (HER) model simulates a wide array of findings from fMRI, ERP, single-units, and neuropsychological studies of both lateral and medial prefrontal cortex. By reconceptualizing lateral prefrontal activity as anticipating prediction errors, the HER model provides a novel unifying account of prefrontal cortex function with broad implications for understanding the frontal cortex across multiple levels of description, from the level of single neurons to behavior.
Article
Full-text available
Human behavior is strongly driven by the pursuit of rewards. In daily life, however, benefits mostly come at a cost, often requiring that effort be exerted to obtain potential benefits. Medial PFC (MPFC) and dorsolateral PFC (DLPFC) are frequently implicated in the expectation of effortful control, showing increased activity as a function of predicted task difficulty. Such activity partially overlaps with expectation of reward and has been observed both during decision-making and during task preparation. Recently, novel computational frameworks have been developed to explain activity in these regions during cognitive control, based on the principle of prediction and prediction error (predicted response–outcome [PRO] model [Alexander, W. H., & Brown, J. W. Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14, 1338–1344, 2011], hierarchical error representation [HER] model [Alexander, W. H., & Brown, J. W. Hierarchical error representation: A computational model of anterior cingulate and dorsolateral prefrontal cortex. Neural Computation, 27, 2354–2410, 2015]). Despite the broad explanatory power of these models, it is not clear whether they can also accommodate effects related to the expectation of effort observed in MPFC and DLPFC. Here, we propose a translation of these computational frameworks to the domain of effort-based behavior. First, we discuss how the PRO model, based on prediction error, can explain effort-related activity in MPFC, by reframing effort-based behavior in a predictive context. We propose that MPFC activity reflects monitoring of motivationally relevant variables (such as effort and reward), by coding expectations and discrepancies from such expectations. Moreover, we derive behavioral and neural model-based predictions for healthy controls and clinical populations with impairments of motivation. Second, we illustrate the possible translation to effort-based behavior of the HER model, an extended version of PRO model based on hierarchical error prediction, developed to explain MPFC–DLPFC interactions. We derive behavioral predictions that describe how effort and reward information is coded in PFC and how changing the configuration of such environmental information might affect decision-making and task performance involving motivation.
Article
Full-text available
In the last two decades the anterior cingulate cortex (ACC) has become one of the most investigated areas of the brain. Extensive neuroimaging evidence suggests countless functions for this region, ranging from conflict and error coding, to social cognition, pain and effortful control. In response to this burgeoning amount of data, a proliferation of computational models has tried to characterize the neurocognitive architecture of ACC. Early seminal models provided a computational explanation for a relatively circumscribed set of empirical findings, mainly accounting for EEG and fMRI evidence. More recent models have focused on ACC's contribution to effortful control. In parallel to these developments, several proposals attempted to explain within a single computational framework a wider variety of empirical findings that span different cognitive processes and experimental modalities. Here we critically evaluate these modeling attempts, highlighting the continued need to reconcile the array of disparate ACC observations within a coherent, unifying framework.
Article
Full-text available
Sometime in the past two decades, neuroimaging and behavioral research converged on pFC as an important locus of cognitive control and decision-making, and that seems to be the last thing anyone has agreed on since. Every year sees an increase in the number of roles and functions attributed to distinct subregions within pFC, roles that may explain behavior and neural activity in one context but might fail to generalize across the many behaviors in which each region is implicated. Emblematic of this ongoing proliferation of functions is dorsal ACC (dACC). Novel tasks that activate dACC are followed by novel interpretations of dACC function, and each new interpretation adds to the number of functionally specific processes contained within the region. This state of affairs, a recurrent and persistent behavior followed by an illusory and transient relief, can be likened to behavioral pathology. In Journal of Cognitive Neuroscience, 29:10 we collect contributed articles that seek to move the conversation beyond specific functions of subregions of pFC, focusing instead on general roles that support pFC involvement in a wide variety of behaviors and across a variety of experimental paradigms.
Article
According to prominent models in cognitive psychology, neuroscience, and economics, effort (be it physical or mental) is costly: when given a choice, humans and non-human animals alike tend to avoid effort. Here, we suggest that the opposite is also true and review extensive evidence that effort can also add value. Not only can the same outcomes be more rewarding if we apply more (not less) effort, sometimes we select options precisely because they require effort. Given the increasing recognition of effort's role in motivation, cognitive control, and value-based decision-making, considering this neglected side of effort will not only improve formal computational models, but also provide clues about how to promote sustained mental effort across time.
Article
The frontal lobes are important for cognitive control, yet their functional organization remains controversial. An influential class of theory proposes that the frontal lobes are organized along their rostrocaudal axis to support hierarchical cognitive control. Here, we take an updated look at the literature on hierarchical control, with particular focus on the functional organization of lateral frontal cortex. Our review of the evidence supports neither a unitary model of lateral frontal function nor a unidimensional abstraction gradient. Rather, separate frontal networks interact via local and global hierarchical structure to support diverse task demands.
Article
Many everyday choices are based on personal, subjective preferences. When choosing between two options, we often feel conflicted, especially when trading off costs and benefits occurring at different times (e.g., saving for later versus spending now). Although previous work has investigated the neurophysiological basis of conflict during inhibitory control tasks, less is known about subjective conflict resulting from competing subjective preferences. In this pre-registered study, we investigated subjective conflict during intertemporal choice, whereby participants chose between smaller immediate versus larger delayed rewards (e.g., 15todayvs.15 today vs. 22 in 30 days). We used economic modeling to parametrically vary eleven different levels of conflict, and recorded EEG data and pupil dilation. Midfrontal theta power, derived from EEG, correlated with pupil responses, and our results suggest that these signals track different gradations of subjective conflict. Unexpectedly, both signals were also maximally enhanced when decisions were surprisingly easy. Therefore, these signals may track events requiring increased attention and adaptive shifts in behavioral responses, with subjective conflict being only one type of such event. Our results suggest that the neural systems underlying midfrontal theta and pupil responses interact when weighing costs and benefits during intertemporal choice. Thus, understanding these interactions might elucidate how individuals resolve self-control conflicts.