ChapterPDF Available

Abstract and Figures

One key aspect of motivation is the ability of agents to overcome excessive weighting of intrinsic subjective costs. This contribution aims to analyze the subjective cost of effort and assess its neural correlates in sedentary volunteers. We recruited a sample of 57 subjects who underwent a decision-making task using a prospective, moderate, and sustained physical effort as devaluating factor. Effort discounting followed a hyperbolic function, and individual discounting constants correlated with an indicator of sedentary lifestyle (global physical activity questionnaire; R = − 0.302, P = 0.033). A subsample of 24 sedentary volunteers received a functional magnetic resonance imaging scan while performing a similar effort-discounting task. BOLD signal of a cluster located in the dorsomedial prefrontal cortex correlated with the subjective value of the pair of options under consideration (Z > 2.3, P < 0.05; cluster corrected for multiple comparisons for the whole brain). Furthermore, effort-related discounting of reward correlated with the signal of a cluster in the ventrolateral prefrontal cortex (Z > 2.3, P < 0.05; small volume cluster corrected for a region of interest including the ventral prefrontal cortex and striatum). This study offers empirical data about the intrinsic subjective cost of effort and its neural correlates in sedentary individuals.
Content may be subject to copyright.
Bernacer et al.
1
[This is a preprint version of the paper: Bernacer J, Martinez-Valbuena I, Martinez M, Pujol N,
Luis EO, Ramirez-Castillo D, Pastor MA. Progress in Brain Research 229:103-123
https://doi.org/10.1016/bs.pbr.2016.05.003
Please cite reference above if necessary.
It can be also cited as a book chapter in Motivation: theory, neurobiology and applications,
Edited by Bettina Studer and Stefan Knecht. Elsevier]
Running title: Subjective cost of effort in the brain
Brain correlates of the intrinsic subjective cost of effort in
sedentary volunteers
Javier Bernacer1, Ivan Martinez-Valbuena1, Martin Martinez2, Nuria
Pujol3, Elkin Luis2, David Ramirez-Castillo1, Maria A Pastor1,2,3
1Mind-Brain Group (Institute for Culture and Society, ICS), University of Navarra
2Neuroimaging Laboratory, Center for Applied Medical Research (CIMA), University of
Navarra
3Department of Neurology, Clínica Universidad de Navarra, University of Navarra
Contact information: Javier Bernacer (jbernacer@unav.es)
Mind-Brain Group, Institute for Culture and Society (ICS)
Edificio Biblioteca. Campus Universitario s/n. Universidad de Navarra
Pamplona (Navarra), 31008 Spain
Abstract
One key aspect of motivation is the ability of the agents to overcome an excessive
weighting of intrinsic subjective costs. This contribution aims to analyze the subjective
cost of effort and assess its neural correlates in sedentary volunteers. We recruited a
sample of 57 subjects who underwent a decision making task using a prospect and
aerobic physical effort as devaluating factor. Effort discounting followed a hyperbolic
function, and individual discounting constants correlated with an indicator of sedentary
lifestyle (GPAQ questionnaire; R=-0.334, P=0.019). A subsample of 24 sedentary
volunteers received an fMRI scan while performing a similar effort discounting task.
BOLD signal of a cluster located in the dorsomedial prefrontal cortex correlated with
the subjective value of the pair of options under consideration (z>2.3, P<0.05 corrected
for multiple comparisons, whole brain). Furthermore, effort-related discounting of
rewards correlated with the signal of a cluster in the ventrolateral prefrontal cortex
(z>2.3, P<0.05 small volume corrected for a region of interest including the ventral
Bernacer et al.
2
prefrontal cortex and striatum). In all, our contribution offers empirical data about the
intrinsic subjective cost of effort and its neural correlates in sedentary individuals.
Keywords: decision making, effort discounting, GPAQ, risk discounting, sedentary
lifestyle, subjective value, utility.
Bernacer et al.
3
1. Introduction
Decision making and action performance depend on the balance between costs
and benefits undertaken by the agent. As it is explained in the first section of this
Volume, both factors have a dual contribution, namely intrinsic and extrinsic. Let us
consider the case of a one-hour jogging session for a usual runner. On the side of
benefits, there is an intrinsic value of physical exercise centered on the positive feeling
that it causes on the runner; besides, there could be an extrinsic subjective benefit which
may include, for example, the increase of the runner’s probabilities to win the next race
and achieve an economic reward. On the side of costs, there is an obvious intrinsic cost
due to the energy expenditure that the physical exercise requires; in addition, there
could be other intrinsic factors such as the temporal cost related to the delay to achieve
the expected reward (i.e. improving personal records, winning the next race, etc), or the
high price of running clothes. Finally, extrinsic costs mainly refer to the loss of putative
benefits that alternative activities (such as going out with friends or watching TV at
home) may entail. Considering all these factors, we may assume that the usual runner
could be motivated for his or her running session, because the subjective benefits
overcome the subjective costs. However, if we substitute the usual runner for a beginner,
subjective benefits would be lower because the intrinsic value of exercise and extrinsic
value of instrumental outcomes are unfamiliar to the agent. Furthermore, the intrinsic
cost of effort, as well as the cost associated to forgoing alternative activities, would be
extremely high. Thus, it should not come as a surprise that the beginner would be poorly
motivated for the running session.
This chapter aims to study the intrinsic subjective cost of effort at both behavioral
and neural levels. We are interested in the variable subjective weighing of effort
Bernacer et al.
4
depending on the habitual physical exercise performed by the agent. For that reason, we
analyzed effort discounting in a sample of volunteers with various levels of physical
activity, from sedentary to highly active. We then studied the brain correlates of effort
weighing in a subsample of sedentary volunteers.
Peters and Büchel (2010) described a brief taxonomy of value types in decision
making, including outcome, goal, decision and action values. Whereas outcome and
goal values are unrelated to costs, decision value depends on the subjective discounting
of the objective value of a reward. Action value reflects the pairing of an action with
any of the other types of values, and thus it could be either related or unrelated to costs.
Therefore, decision value is the only type strictly related to subjective costs. In general
terms, as it is described by prospect theory, subjective value (SV) is the expected
objective outcome of the actions discounted by different factors, namely risk, time and
effort (see, for example, Kable and Glimcher, 2007; Prévost et al., 2010; Weber and
Huettel, 2008). This theoretical and experimental framework was first described in the
field of economics (Kahneman and Tversky, 1979), although it has been also
extrapolated to behavioral psychology (Green and Myerson, 2004). Recently, it has
become a productive field of research in neuroscience. Due to the main interest of this
chapter, we will focus on the literature in neuroscience to explain the background of the
topic.
The point of studying value-based decision making in neuroscience aims to
describe the brain correlates of SV or, in other words, the brain area that encodes the
subjective discounting of a reward. Thirty euros are objectively better than ten euros,
but they could be less valuable if: 1) they are not immediately available; 2) we are not
sure about obtaining them; or 3) we have to exert some effort to win them. The actual
weight of these discounting factors is subjective and state-dependent, but it seems clear
Bernacer et al.
5
that their neural correlates are common in all humans. Based on a meta-analysis of
functional magnetic resonance imaging (fMRI) studies, Levy and Glimcher proposed
that the ventromedial prefrontal cortex (VMPFC) encodes SV, irrespective to the nature
of the reward (Levy and Glimcher, 2012). This valuation is carried out by integrating
sensory inputs (from parietal and occipital cortices), information about the internal state
of the agent (subcortical inputs) and personal preferences in terms of discounting factors
(from other regions of the prefrontal cortex). Then, the value signal is conveyed to
motor-related cortical areas which, in association with the basal ganglia, will produce
the behavioral output. The engagement of VMPFC in value coding is extensively
known (see, for example Bartra et al., 2013; Dreher, 2013; Montague et al., 2006;
O’Doherty, 2011). Pharmacologically, this valuation seems to depend on
monoaminergic signaling (Arrondo et al., 2015; Bernacer et al., 2013; Jocham et al.,
2011). In the following paragraphs, we will briefly summarize the main findings about
intrinsic subjective costs in decision making in the fields of psychology and
neuroscience.
As we mentioned above, the main discounting factors in decision making (i.e.
factors that determine intrinsic costs) are time, risk and effort. In 2004, Green and
Myerson published an integrative review on temporal and probabilistic discounting in
human behavior (Green and Myerson, 2004). Experimentally, the intrinsic cost of
temporal delay is assessed with a very simple task, where volunteers are asked to
choose between a relatively small immediate reward and a larger delayed reward (for
example, $150 now vs $1000 in 6 months). Considering the responses of each volunteer
a discounting curve is calculated, showing the subjective devaluation of a reward (Y
axis) with increasing delays (X axis). As Green and Myerson explain, even though
temporal discounting curves were first described as exponential, they seem to follow a
Bernacer et al.
6
hyperbola-like shape. This has been extensively replicated in psychology and
neuroscience (see, for example, Estle et al., 2006; Kable and Glimcher, 2007;
Kobayashi and Schultz, 2008; McKerchar et al., 2009; Peters and Büchel, 2009; Pine et
al., 2010; Wittmann et al., 2007). Concerning risk discounting, the procedure and results
are very similar. In this case the experimenter offers two options that differ in the
probability to obtain the reward (for example, $150 for sure versus $1000 with a 30%
chance). Once again, the hyperbola-like function produces the best fit to the
experimental data (Estle et al., 2006; Green and Myerson, 2004; Weber and Huettel,
2008), being the X axis the odds against winning the reward. Finally, the
characterization of the effort discounting curve is quite recent (Hartmann et al., 2013).
Hartmann and collaborators reported that effort discounting is best defined by a
parabolic curve, instead of hyperbolic as it was suggested by other authors (Mitchell,
2004; Prévost et al., 2010). The task proposed in all these reports consists on choosing a
small non-effortful reward, or a larger reward that involves squeezing a handle with a
variable intensity. Thus, the value of the reward is discounted by increasing levels of
effort.
At the neural level, the main brain area whose activity correlates with discounting
functions is the ventral prefrontal cortex. Kable and Glimcher followed the behavioral
approach explained above to calculate the SV of the option that volunteers chose in the
fMRI scanner (Kable and Glimcher, 2007). For example, if a volunteer chooses $30
with a temporal delay of 30 days, the SV is the objective value (30) multiplied by the
subjective intrinsic cost (or individual temporal discounting factor) of waiting for 30
days (say, for example, 0.25). Thus, an objective reward of $30 is reduced to 7.5. These
authors found that the BOLD signal of VMPFC and ventral striatum correlated with the
SV of the chosen option. These results have been replicated by others (Gregorios-Pippas,
Bernacer et al.
7
2009; Prévost et al., 2010; Wittmann et al., 2007). SV discounted by probability has
been described to have similar brain correlates, although other areas such as the
intraparietal sulcus have been also included (Peters and Büchel, 2009). With respect to
physical effort discounting, the main brain areas involved in SV are the striatum,
supplementary motor area, anterior cingulate, VMPFC and motor cortex (Burke et al.,
2013; Croxson et al., 2009; Kurniawan et al., 2011, 2010; Prévost et al., 2010;
Treadway et al., 2012). In the next paragraph we will discuss in more detail the tasks
employed in these effort discounting experiments, in order to highlight the novelty of
the research that we present in this chapter.
The investigation of the brain correlates of effort discounting has shown a
growing interest in the last years. There are several fMRI experiments aimed to assess
the brain areas associated with effort-based decision making, their interaction with other
discounting factors, and the influence of dopamine in this process, for example. The
theoretical background of these experiments is based on Salamone’s research carried
out in rats (see Salamone, 2009 for a review). The cornerstone of his studies is the
relationship between effort, decision making, dopamine and nucleus accumbens. To our
knowledge, one of the first translational articles that attempted to assess the neural
correlates of effort discounting in humans was the work by Botvinick et al (2009).
However, the type of effort involved in their task was mental effort. Previously,
although in a different context, Pessiglione et al (2007) studied the motivational role of
subliminal images and the brain activity associated to the process. Remarkably, they
measured motivation as the grip force exerted when squeezing a handle, and reported
that the ventral pallidum encoded both conscious and subliminal motivation. This type
of task (hand grip) has been used in most of the studies on physical effort-discounting
(Bonnelle et al., 2015; Burke et al., 2013; Kurniawan et al., 2010; Meyniel et al., 2013;
Bernacer et al.
8
Prévost et al., 2010; Skvortsova et al., 2014). Others have used different paradigms
involving button presses per time unit (Kroemer et al., 2014; Scholl et al., 2015;
Treadway et al., 2009). The important issue we would like to stress at this point is that
all these experiments involve a decision about an immediate effort. In addition, both
hand gripping and button pressing could not be optimal to evaluate the actual
willingness of a subject to make an effort in real life: even though volunteers may have
a strong effort discounting in their everyday decisions (i.e. driving a car instead of
walking, or using the elevator instead of the stairs), they could be extra-motivated and
willing to make a brief effort within the experiment.
Considering all this, we decided to use a paradigm where the effort at stake was
prospective and aerobic, and therefore with potential ecological validity: running in a
treadmill. Hence, we first recruited a large sample of volunteers which undertook a
decision making task where they had to decide between a small non-effortful reward or
a larger reward in exchange for some time running in a treadmill. We collected
information about their active lifestyle with the intention of testing the ecological
validity of our task, that is, the correlation between effort discounting and the level of
physical activity in a normal week. We then recruited a subsample of sedentary
volunteers who received an fMRI scan while doing a similar decision making task.
Using neurocomputational methods, we investigated the activity of which brain areas
correlated with effort discounting-related signals. In the next sections we will explain
these two experiments in detail, and we will then discuss the implications of the results
for a better understanding of motivation.
Bernacer et al.
9
2. Methods
In this chapter we report the results from two experiments. The first aimed to
calculate individual and group effort discounting curves when the effort at stake is
prospective and aerobic. In addition, it was intended to test whether the decaying
constants of individual curves correlated with a lifestyle indicator, assessed by the
administration of the GPAQ questionnaire (World Health Organization, WHO). The
second experiment aimed to assess brain activity in sedentary subjects, when effort is
the main devaluating factor in a decision making task. We used neurocomputational
methods to evaluate the neural correlates of SV and effort discounting. These two
parameters were estimated from the individual curves obtained in the first experiment.
2.1. Subjects
The protocol of the experiment was approved by the Committee of Ethics in
Research of the University of Navarra. A sample of 57 subjects (26 female, 18-25 yr)
was recruited within the environment of the university. Hence, they all had a similar
profile in terms of age, income and educational level; however, they were not asked to
fulfil any special requirements in terms of sedentary lifestyle prior to the study, in order
to ensure certain degree of diversity to correlate the data with the effort discounting
constants. A subsample of volunteers (24, 14 female, 18-25 yr) was recruited from the
initial sample for the second experiment. Inclusion criteria were: 1) a low score in
GPAQ; 2) no fMRI scan incompatibilities; 3) availability to follow a physical exercise
program for the following three months; 4) no neurological or psychiatric disorders,
assessed by the Mini International Neuropsychiatric Interview (Cummings et al., 1994).
The third criterion was part of an additional project not reported here. They all provided
signed informed consent before the scan.
Bernacer et al.
10
2.2. Tasks
The tasks of both experiments were coded in Cogent 2000 (Wellcome Department
of Imaging Neuroscience, UCL, London, UK) and Matlab (Mathworks, Natick, MA).
For the first experiment, we used a modified version of the most common task used for
temporal and risk discounting (Kable and Glimcher, 2007), which has been also
employed to assess effort discounting (Hartmann et al., 2013) (Figure 1). Subjects were
instructed about the general framework of the project, and they were presented
sequentially several pairs of options from which they had to choose one: one of the
options (randomly presented on the left or right side of the screen) was always present,
and involved a 5 € reward in exchange for no effort. The other option entailed a higher
amount of money (5.25, 9, 14, 20, 30 or 50 €) and different effort levels to win it (5, 10,
15, 20, 25 and 30 min running in the treadmill). Therefore, there were 36 different pairs
of options presented and each of them was randomly displayed 4 times (144 trials in
total, divided into 2 sessions of 72 trials). Subjects had to respond by pressing the left or
right arrow of the keyboard. They were not informed about the structure of the task, and
they were told that both rewards and effort were hypothetical (see discussion about this
below). We used a similar task to calculate risk discounting, another devaluating factor
used in the fMRI task (see below). The task and data analysis were identical to the effort
discounting task, substituting effort levels for probability of winning the reward (90, 75,
50, 33, 10 and 5%).
The fMRI task was similar to the one used in the behavioral study, although there
were key differences (Figure 2). Again, two options were presented at the same time,
and volunteers had to choose one of them by pressing a left or right button with the
index or middle finger (respectively) of their right hand. In this case, both options
entailed the possibility of winning 30 € (fixed reward). Besides, they included a
Bernacer et al.
11
probability of winning the reward (30, 40, 50, 60 or 70%), and an effort to be exerted in
exchange for it (10, 15, 20, 25 or 30 min running in the treadmill). Subjects were
explained that after the scan one of the trials would be picked at random, and the actual
option they chose would be recorded. Then, they would play the lottery according to the
probability of the chosen option, and in the case of winning they had to do the physical
exercise in exchange for the money during the following week. If they lose the lottery,
they would not get any money nor do any exercise. Payments were given as vouchers
for the university’s book shop.
Pairs of options were selected individually for each volunteer, guaranteeing 7
“difficult” (SV of both options were nearly identical), 6 “easy” (SV were very different)
and 7 of medium “difficulty” (SV were similar). Therefore, in total, 20 different pairs of
options (“task pairs”) were presented. As it was explained above, SV corresponds to the
actual reward (30 €) multiplied by the discounting factors of effort and risk, obtained in
the first experiment.
Each of the 20 task pairs were presented 9 times. In addition to these 180 trials, 45
motor control trials were included (Figure 2). There were also 45 trials where subjects
could choose a certain non-effortful reward (30 €, 100 %, 0 min vs 30 €, 0%, 0 min),
and 45 additional trials involving a certain reward with a maximum effort (30 €, 100%,
35 min vs 30 €, 0%, 35 min). In total, 315 trials were presented to each volunteer,
divided into 3 sessions of 105 trials each (about 12 min). The options stayed on the
screen up to 4 seconds or until the subject responded. The order and position of the
options (left or right) were randomly arranged. Trials were separated by a fixation cross
of random variable duration (2-6 s).
Bernacer et al.
12
2.3. Behavioral data analysis and curve fitting
Data processing and curve fitting were performed using Matlab, and statistical
analyses were carried out using SPSS 15.0 (SPSS Inc., Chicago). We first calculated the
function that best described the behavior of each participant. To do so, for each subject
and effort level, we looked for the situation at which the SV of the effortless option was
equal to the SV of a particular effort level (i.e. indifference point). This was inferred by
plotting, for each effort level, the number of times (out of four) that each reward was
preferred instead of the 5 € (effortless option). For example, for the effort level of 10
min running, one particular subject may have the following behavior: 5.25, 0 times
chosen (0/4); 9 €, 1/4; 14 and 20 €, 3/4; 30 and 50 €, 4/4. This data was then fitted to a
logistic function (Eq. 1) to calculate which amount of money corresponded to a 2/2
behavior, that is, the indifference point (Figure 1):
Eq.1:
()
0
()
1
G Money r
k
y Money e
−−
=+
Curve fitting was performed by a script that tested all the possible combinations
of 100 different values of the constants in the logistic function [k(0.5,1.5); G(0.1,10);
r0(1,100)]. The best fitting was the maximum value after calculating the sum of least
squares for each combination. After this, the discounting factor of each effort level was
calculated by dividing each indifference point by the money corresponding to the
effortless option (5 €). Finally, these discounting factors were plotted and four different
fittings were evaluated, according to the literature: hyperbolic (Eq. 2), exponential (Eq.
3), double exponential (Eq. 4) (Green and Myerson, 2004; Prévost et al., 2010) and
parabolic (Eq. 5) (Hartmann et al., 2013):
Bernacer et al.
13
Eq. 2:
1
()
1*
y Effort K Effort
=+
Eq. 3:
*
()
c Effort
y Effort
e
=
Eq. 4:
**
() 2
Effort Effort
y Effort ee
β
− −∂
+
=
Eq. 5:
2
() *y Effort A H Effort
= −
Again, curve fitting was carried out by a script that tested different combinations
of the constants included in each formula, and the best fitting was chosen by sum of
least squares. In this case, 1000 different values of each constant were tested.
In order to evaluate the best fitting for the whole sample, we calculated the
median of the indifference points for each effort level, obtained the discounting factors
as before, plotted them and assessed the same fitting functions.
2.4. GPAQ
We estimated the active lifestyle of the volunteers with the Spanish version of the
Global Physical Activity Questionnaire published by the WHO
(http://www.who.int/chp/steps/resources/GPAQ_Analysis_Guide.pdf). This test inquires the
volunteers about the physical activity they perform in a normal week. It is divided into 4
sections: at work, traveling to and from places, in recreational activities and sedentary
behavior. In each of the first three sections, they have to disclose the amount of time (in
hours and minutes) they spend doing moderate or vigorous physical activity. In the last
section, they have to report the number of hours they spend sitting or reclining in a
Bernacer et al.
14
typical day. The dependent variable is the number of METs (metabolic equivalents),
which is the ratio of a person’s working metabolic rate relative to the resting metabolic
rate. One MET corresponds to a consumption of 1 kcal/kg/hour. According to WHO’s
GPAQ guidelines, 4 METs get assigned to the time spent in moderate activities, and 8
METs to the time spent in vigorous activities. Time spent traveling to or from places is
considered a moderate activity.
2.5. fMRI setting
We used a 3T fMRI scanner (Siemens TRIO, Erlangen, Germany) and a 32-
channel head coil. Between 170 and 274 volumes (depending on the subjects’ reaction
times) were acquired in each of the 3 sessions, using an echo-planar imaging sequence
to measure BOLD contrast (or “activity”) (resolution=3x3x3 mm3; TR/TE=3000/30 ms;
FOV=192x192 mm2, Flip angle=90º; 64, 48 and 48 volumes acquired in the coronal,
sagittal and axial planes respectively). The first five volumes were discarded for T1
equilibration effects. An anatomical T1 MPRAGE image was also collected (TR = 1620
ms; TE = 3.09ms; inversion time (TI) = 950ms; FOV = 256x192x160 mm3; flip angle =
15º; image resolution=1 mm isotropic).
fMRI data were analyzed with FSL (created by the Analysis Group, FMRIB,
Oxford, UK, http://fmrib.ox.ac.uk/fsl) (Jenkinson et al., 2012). Prior to any data
processing, the skull was removed from all T1 images using the BET tool included in
FSL package. Individual T2* images were processed with FEAT (FMRI Expert
Analysis Tool). They were realigned, motion corrected and spatially smoothed with a
Gaussian kernel of 5 mm (full-width half maximum). Each time series was high-pass
filtered (100 s cut off). Images were registered to the corresponding T1 image and
finally normalized to MNI template.
Bernacer et al.
15
2.5.1. General Linear Model for the fMRI data
Each individual time series was fitted to a general linear model (GLM) with the
following 10 explanatory variables (EVs): 1) task pair presentation; 2) uncertainty
measured as Shannon’s entropy; 3) SV of the pair; 4) maximum effort pair; 5) non-
effortful maximum reward pair; 6) effort discounting factor of the chosen option; 7) risk
discounting factor of the chosen option; 8) effort discounting factor of the pair; 9) risk
discounting factor of the pair; 10) motor control. EVs 1, 2, 3, 6, 7, 8 and 9 were time-
locked to the onset of the task pair. EVs 4, 5 and 10 were time-locked to the onset of
their respective trial type. The height of the regressor was uniform for EVs 1, 4, 5 and
10. Due to the neurocomputational nature of our approach, the following EVs had a
parametric modulator: 2) the values of Shannon’s entropy (not reported here); 3)
absolute value of the difference between the SV of both options within the pair (see
below); 6) values of effort discounting factors for the physical exercise involved in the
chosen option (see below); 7) values of risk discounting factors for the probability
involved in the chosen option; 8) absolute value of the difference between the effort
discounting factors of both options; 9) absolute value of the difference between the risk
discounting factors of both options. The duration of all regressors was set to 2 s, since
this was the average reaction time of all participants.
The main interest of this experiment was to assess the brain correlates of effort
discounting in SV. For that reason, the contrasts of interest that are presented here are
EV3 vs EV10, and EV8 vs EV10. Once the individual statistical parametric maps were
calculated for each session, a second level analysis was performed to average all three
individual sessions; then, the whole sample statistical map was calculated in a third
level analysis.
Bernacer et al.
16
SV of the pair or ‘difference SV’. The SV of each option was calculated by
multiplying the actual reward (30 €) by the discounting factors of effort and risk, which
were estimated in the first experiment. Since both options were simultaneously
presented on screen, we calculated the SV of the pair, that is, the absolute value of the
difference between the SV of both options. This approach has been used by other
authors (FitzGerald et al., 2009). This was the parametric modulator of EV3.
Discounting factors. Effort and risk discounting factors were calculated from the
discounting curves of the first experiment. Please consider that discounting factors close
to 1 involve a low discounting, that is, a SV close to the objective reward; when
discounting factors are close to 0, they have a maximum effect reducing SV. Therefore,
discounting factors and effort- or risk-related SV are equivalent from a
neurocomputational point of view. EVs 6 and 7 refer to the discounting factor of effort
and risk (respectively) of the chosen option, whereas EVs 8 and 9 represent the
discounting factors of the pair of options (|DFchosenDFnot_chosen|) for effort and risk,
respectively. We will focus our analyses on the brain correlates of effort discounting
(EV 8 vs motor control).
3. Results
3.1. Experiment 1: effort-discounting and correlation with
lifestyle
GPAQ data was not collected from one volunteer (male). As expected, the sample
(N=56) showed a high variability in terms of physical activity measured in METs:
mean=1597, SEM=247.9, min=0, max=10080). Median values differed between male
(1360 METs) and female (840 METs), and this difference was statistically significant
(Mann-Whitney U=243.5; Nmale=29; Nfemale=27; P=0.015, two-tailed).
Bernacer et al.
17
About effort discounting, the behavior of the whole sample was explained best by
a hyperbolic function, according to the adjustment values (R2 adjusted for the number of
variables in each function): hyperbolic=0.9694; exponential=0.9024; double
exponential=0.9628; parabolic=0.5297) (Figure 1). Please note that the double
exponential curve was also a good predictor of the sample’s behavior, whereas the
parabolic fitting was the poorest. Interestingly, in terms of individual fitting, the
hyperbolic curve was the best predictor for the same number of subjects as the double
exponential (N=20). The behavior of 16 subjects adjusted to an exponential curve,
whereas the parabolic function was optimal for 1. Since the best fitting for the sample
was hyperbolic, the subsequent analyses were done taking the individual constants (K)
from the hyperbola-like discounting function. When comparing male and female
participants, there were no statistical differences in hyperbolic K (Mann-Whitney
U=398.5; Nmale=30; Nfemale=27; P=0.917, two-tailed) or R2 goodness of fit (Mann-
Whitney U=328; Nmale=30; Nfemale=27; P=0.218, two-tailed).
Once achieved the first goal of this part of the study, we focused on the validation
of our task as an ecological model for effort discounting. To do so, we correlated the
individual hyperbolic decaying constants with the individual METs value, controlling
for the individual adjustment (R2) to the hyperbolic curve. This partial instead of a
bivariate correlation was carried out to consider the fact that the hyperbolic was not the
best fitting for all subjects. Statistical analyses revealed a significant correlation
between both variables: r=-0.334, P=0.019 (N=51, after discarding outliers). This means,
as predicted, that the effort discounting is higher (higher values of K) for subjects with a
sedentary lifestyle (lower METs values). Interestingly, this statistically significant result
was similar when correlating the METs value with the main decaying constant of the
Bernacer et al.
18
double exponential function (β), controlling for the double exponential adjustment: r=-
0.283, P=0.049 (N=52 after discarding outliers).
In conclusion, this first experiment demonstrates that: 1) effort discounting in a
sample of University students is explained by a hyperbolic function; and 2) a task
including a prospective aerobic effort as devaluating factor is a proper indicator of the
active or sedentary lifestyle of the subjects.
[Insert Figure 1 here]
3.2. Experiment 2: brain correlates of effort discounting in
sedentary subjects
3.2.1. Behavioral results of the fMRI task
According to the design of our task, the effect of effort as discounting factor is
revealed by the selection of the lower probability option of the pair (i.e. if effort were
not a discounting factor for the participants they would always choose the high
probability option, which is obviously more advantageous). The sedentary lifestyle of
the volunteers was reflected in their choices during the fMRI task. Focusing on the nine
times that each “difficult” and “medium” pair was presented, subjects chose on average
5.7 (±0.56) times the high probability option, irrespective to the demanded effort.
“Easy” pairs were discarded from this analysis because some of them involved a high
probability/low effort option vs low probability/high effort option. Even though all
participants of the second experiment were sedentary, there was some variability in the
degree of habitual physical activity and the individual value of the hyperbolic decaying
constants. Interestingly, there was a negative correlation between the number of times
that the high probability option was chosen and the hyperbolic decaying constant
(Spearman’s rho=-0.414, P=0.05, N=23 after discarding one outlier). This result
Bernacer et al.
19
confirmed that subjects with a higher effort discounting (higher K) tended to prefer the
low probability/low effort option.
3.2.2. Imaging results
In this section we report those areas whose activity correlated with 1) SV of the
pair or ‘difference SV and 2) effort-related discounting factor of the pair, when they
were contrasted with the motor control.
With respect to the SV of the pair, we performed a whole brain analysis that
revealed a large cluster in the dorsomedial prefrontal cortex, as well as a cluster located
in the right ventrolateral prefrontal cortex (VLPFC) and different aspects of the parietal
cortex (Figure 2; Table 1). In other words, these brain areas had a higher BOLD signal
for those pairs of options with a high difference SV (i.e. a very different SV between
option A and B), and a low BOLD signal for those options with a low difference SV (i.e.
both options with similar SV).
Finally, based on the previous literature discussed above, we restricted our
analysis on the neural correlates of effort-discounting to a large region of interest
including the ventral prefrontal cortex and striatum (12186 voxels in total) (Figure 2).
The analysis revealed a cluster located in the left VLPFC (Figure 2; Table 1). Therefore,
BOLD signal in this area correlated with effort discounting of the pair, which is
equivalent to effort-based difference SV (i.e. SV excluding the effect of risk
discounting).
In summary, our neurocomputational imaging results suggest that the DMPFC is
associated with the SV of the pair of options under consideration taking into account
both effort and risk discounting, and the VLPFC is related with effort discounting in
decision making.
[Insert Figure 2 here]
Bernacer et al.
20
4. Discussion
In this contribution we present the main results of two experiments aimed to
advance in the understanding of the intrinsic cost of effort in decision making. First, we
have described the hyperbola-like discounting function of effort, using for the first time
a prospective and aerobic physical exercise. We have demonstrated the ecological
validity of our approach by proving the association between the decaying constants and
the level of physical activity of the volunteers. Second, we have shown the neural
correlates of two different effort-related neurocomputational parameters, namely SV
and effort discounting of the pair: DMPFC and VLPFC, respectively.
Even though the role of effort in decision making has been largely studied in the
last years both at behavioral and neural levels, the tasks always shared two common
characteristics with respect to the demanded effort: immediate and anaerobic (see, for
example, Bonnelle et al., 2015; Burke et al., 2013; Croxson et al., 2009; Hartmann et al.,
2013; Kurniawan et al., 2011; Prévost et al., 2010; Skvortsova et al., 2014; Treadway et
al., 2012). Due to this, to our knowledge, the relationship between the experimental
intrinsic cost of effort and the active or sedentary lifestyle of the participants had not
been analyzed yet. We decided to apply a task commonly used in this kind of
experiments, although including an exercise that could inform about the weight of effort
on the participants’ daily lives. In our opinion, the validity of our approach is confirmed
by the correlation between the individual effort discounting constants and the metabolic
consumption of the participants measured as METs, as recommended by the WHO.
This correlation held for the two discounting functions that best explained the data,
namely hyperbolic and double exponential. In the next lines we will comment on the
implications of these functions to explain the intrinsic cost of effort.
Bernacer et al.
21
The hyperbolic curve has been reported to explain the role of other discounting
factors, such as temporal delay and risk (Green and Myerson, 2004; Kable and
Glimcher, 2007). Considering the shape of this curve, a mild initial contribution of the
devaluator rapidly lowers the SV of the expected reward; this steep decrease turns
milder at some point, since an increase in the intensity of the devaluator (longer
temporal delay, higher odds against winning or higher effort) does not have such a
strong impact on the subjective discounting of the reward. The dynamics of the
hyperbolic function are mainly explained by the constant K: a high K involves a very
steep decrease of value, whereas K values closer to zero give rise to milder curves. Thus,
the intrinsic cost of effort goes in parallel with K values. We found that the decisions of
a large amount of the participants, as well as the whole sample’s behavior, was also
explained by a double exponential discounting function, as it has been proposed by
some authors in temporal discounting (Mcclure et al., 2007). In this case, the utility
function is decomposed into two processes, each accounted by a different constant: β
and δ. Depending on the actual values, the former usually relates to a quicker and abrupt
decay of the function, whereas the latter explains a more harmonic exponential trend for
higher amounts of the devaluator. McClure and collaborators, in the context of temporal
discounting and primary rewards, termed β as the “impatient” component, whereas they
related the δ component with planning and deliberation (Mcclure et al., 2007). Applying
the analogy to our task, the β may be understood as the “passive” component, since it
accounts for the initial decay of the SV with low levels of effort. In turn, the level of
weekly physical activity of our sample also correlated negatively with β, reinforcing the
ecological validity of our approach. This goes also in line with a day-to-day fact: it is
more costly to start jogging than to keep jogging. In our opinion, this initial strong
devaluating effect of physical effort is the reason why the parabolic function provided
Bernacer et al.
22
the worst fitting, contrariwise to recent research (Hartmann et al., 2013). The work by
Hartmann and collaborators included a hand grip task, where an initial low effort does
not have such as a strong intrinsic cost as a prospective aerobic exercise. Their approach,
however, may provide useful information about actual immediate efforts and rewards.
Our neuroimaging analyses reveal the brain correlates of effort discounting in
decision making. To our knowledge, this is the first time that a prospective aerobic
effort has been used in this kind of experiments. One of the main advantages of our task
is to remove the effects of motor preparation and vigor from the decision itself. When
assessing the neural correlates of effort discounted decision making with an immediate
intense effort, brain activity may be associated with the decision, preparation of the
movement, immediate motivation or vigor, among other factors. Another possible
strategy to overcome this limitation is to separate choice and execute periods during the
hand grip task (Kurniawan et al., 2010). Rather, we decided to use an ecologically valid
and generalizable task, as proven in the first experiment. The key brain areas tagged by
our neurocomputational analyses are the DMPFC and VLPFC.
The whole brain analysis of the SV of the pair revealed a significant cluster in the
DMPFC. The neurocomputational methods that assess the neural bases of SV in
decision making allow two different approaches depending on the task. On the one hand,
if only one option is displayed on the screen (being the other fixed and implicit), the
variable of interest is usually the SV of the chosen option (for example Kable and
Glimcher, 2007). On the other hand, if both options are displayed on the screen, the best
strategy is to model the absolute value of the pair (FitzGerald et al., 2009). This reflects
more accurately the subject’s weighing of both options. These authors report a cluster in
the VMPFC (or subgenual area) as the neural correlate of difference value. In our study,
the brain correlates include the DMPFC. The discrepancy between FitzGerald et al’s
Bernacer et al.
23
study and ours may be due to the absence or presence of discounting factors in the
decision making process. Whereas their task is a direct valuation of items, we demand
our volunteers to employ more resources in evaluating the willingness to make an effort
in exchange for a higher probability to win. According to a recent meta-analysis carried
out on over 200 neuroimaging articles on SV, the DMPFC seems to be part of a network
whose activity correlates with the salience of SV rather than SV itself (Bartra et al.,
2013). Thus, BOLD signal would increase with both subjective rewards and
punishments, and would decrease with neutral values. In light of our results, the
interpretation could be somewhat different: DMPFC’s BOLD signal is higher when the
difference value of the choice is large and lower when it is small. Depending on the task,
a high difference value may be a consequence of either a reward (vs neutral) or a
punishment (vs neutral). The meta-analysis by Bartra et al includes several different
tasks and the foci in DMPFC could be understood as difference value when two options
are presented simultaneously, as well as a value-based salience signal.
Another intriguing result of our experiment is the description of the VLPFC as a
neural correlate of difference effort discounting: it tracks the effort-discounted value of
the pair, being very active for those pairs with disparate values of effort discounting and
poorly active for those pairs with similar effort discounting. The involvement of this
brain region in effort-related processing has been suggested by other authors. Schmidt et
al (2009) presented a series of arousing pictures prior to effort exertion in exchange for
a reward. They found that activity in VLPFC correlated with the level of arousal,
interpreting VLPFC function as a “motivating” signal which facilitates effort exertion to
obtain a reward. Although we did not include any motivating stimulus in our task, pairs
with a higher difference effort discounting might involve a need for motivation to
overcome the negative affect of high effort. It should be taken into account that a high
Bernacer et al.
24
difference effort discounting always means a high versus a low effort level in our task.
However, a low difference effort discounting could be due to similar effort levels,
irrespective to the magnitude of the demanded effort. In this case the motivation signal
could be irrelevant, since choosing any of the options would not make a big difference
in terms of effort exertion. With respect to the literature on decision making, a recent
experiment suggest a role of VLPFC in temporal discounting: it would process a state-
dependent cognitive control signal to determine the adequacy of waiting for delayed
rewards (Wierenga et al., 2015). The authors found that VLPFC was especially active in
sated volunteers, interpreting it as a cognitive control signal which would help them to
wait for larger rewards. Applying this to our results, pairs with a high difference effort
discounting would require a control signal to evaluate whether the effortful option is
worthy, considering that the other option would be much easier.
One of the possible limitations of our first experiment is that rewards and
prospective efforts were hypothetical. However, a within-subject experiment on
temporal discounting including hypothetical versus real rewards revealed that both
approximations account for the subject’s behavior in a similar way (Johnson and Bickel,
2002). Within-subjects experiments in this context have been criticized because they do
not consider the fact that volunteers may remember their responses to the previous
condition of the task, although the key results (no differences between real and
hypothetical rewards) have been replicated with other methods (Lagorio and Madden,
2005; Madden et al., 2004). In turn, many behavioral studies on temporal and risk
discounting have used hypothetical instead of real rewards (Estle et al., 2006; Green and
Myerson, 2004; Green et al., 2013; McKerchar et al., 2009 among others). In any case,
this potential limitation would not affect our second experiment, where subjects were
informed about the random selection of one of the presented pairs and winning the
Bernacer et al.
25
reward in exchange for the demanded effort. Another possible limitation of our task is
whether we are assessing effort or temporal discounting, since effort load is measured as
time (minutes running in the treadmill). Conceptually, however, the influence of
temporal delay on our task is negligible. In the first experiment subjects were instructed
to imagine they were ready to start the exercise, and then make the decision between the
fixed option (5 € reward and going back home) and the more rewarding effortful option.
Thus, other factors such as the time spent going to the gym or changing clothes were
explicitly attenuated, since they were previous to the decision. In the second experiment,
where actual efforts and rewards were at stake, the effect of temporal delay was
lessened even more strongly because subjects were told they would receive the reward
(and make the required effort) during the next week after the scan. Therefore, the actual
point in time of getting the reward did not covariate with the load of the exerted effort.
5. Conclusions
In this chapter we have analyzed behaviorally and at a neural level the intrinsic
cost of effort in economic decision making. This is one of the main factors that
contribute negatively to motivation for a specific exercise. We have designed a task to
calculate individual and group effort discounting, and we have proven its validity and
generalizability to inform about the sedentary lifestyle of volunteers. Finally, we have
shown that different aspects of the prefrontal cortex (dorsomedial and ventrolateral) are
associated with the subjective weighing of effort in decision making. Overall, we
hopefully contribute for a better understanding of the subjective costs that affect
motivation.
6. References
Arrondo, G., Aznarez-Sanado, M., Fernandez-Seara, M.A., Goñi, J., Loayza, F.R.,
Bernacer et al.
26
Salamon-Klobut, E., Heukamp, F.H., Pastor, M.A., 2015. Dopaminergic
modulation of the trade-off between probability and time in economic decision-
making. Eur. Neuropsychopharmacol. 25, 817–827.
Bartra, O., McGuire, J.T., Kable, J.W., 2013. The valuation system: a coordinate-based
meta-analysis of BOLD fMRI experiments examining neural correlates of
subjective value. Neuroimage 76, 412–27.
Bernacer, J., Corlett, P.R., Ramachandra, P., McFarlane, B., Turner, D.C., Clark, L.,
Robbins, T.W., Fletcher, P.C., Murray, G.K., 2013. Methamphetamine-induced
disruption of frontostriatal reward learning signals: relation to psychotic symptoms.
Am. J. Psychiatry 170, 1326–34.
Bonnelle, V., Manohar, S., Behrens, T., Husain, M., 2015. Individual Differences in
Premotor Brain Systems Underlie Behavioral Apathy. Cereb. Cortex 1–13.
Botvinick, M.M., Huffstetler, S., McGuire, J.T., 2009. Effort discounting in human
nucleus accumbens. Cogn. Affect. Behav. Neurosci. 9, 16–27.
Burke, C.J., Brünger, C., Kahnt, T., Park, S.Q., Tobler, P.N., 2013. Neural integration
of risk and effort costs by the frontal pole: only upon request. J. Neurosci. 33,
1706–13a.
Croxson, P.L., Walton, M.E., O’Reilly, J.X., Behrens, T.E.J., Rushworth, M.F.S., 2009.
Effort-based cost-benefit valuation and the human brain. J. Neurosci. 29, 4531–41.
Cummings, J.L., Mega, M., Gray, K., Rosenberg-Thompson, S., Carusi, D., Gornbein,
J., 1994. The Neuropsychiatric Inventory: comprehensive assessment of
psychopathology in dementia. Neurology 44, 2308–2314.
Dreher, J.-C., 2013. Neural coding of computational factors affecting decision making.
Prog. Brain Res. 202, 289–320.
Estle, S.J., Green, L., Myerson, J., Holt, D.D., 2006. Differential effects of amount on
temporal and probability discounting of gains and losses. Mem. Cognit. 34, 914–28.
FitzGerald, T.H.B., Seymour, B., Dolan, R.J., 2009. The role of human orbitofrontal
cortex in value comparison for incommensurable objects. J. Neurosci. 29, 8388–95.
Green, L., Myerson, J., 2004. A discounting framework for choice with delayed and
probabilistic rewards. Psychol. Bull. 130, 769–92.
Green, L., Myerson, J., Oliveira, L., Chang, S.E., 2013. Delay discounting of monetary
rewards over a wide range of amounts. J. Exp. Anal. Behav. 100, 269–281.
Gregorios-Pippas, L., 2009. Short-term temporal discounting of reward value in human
ventral striatum. J. Neurophysiol. 101, 1507–1523.
Bernacer et al.
27
Hartmann, M.N., Hager, O.M., Tobler, P.N., Kaiser, S., 2013. Parabolic discounting of
monetary rewards by physical effort. Behav. Processes 100, 192–6.
Jenkinson, M., Beckmann, C.F., Behrens, T.E.J., Woolrich, M.W., Smith, S.M., 2012.
FSL. Neuroimage 62, 782–790.
Jocham, G., Klein, T.A., Ullsperger, M., 2011. Dopamine-Mediated Reinforcement
Learning Signals in the Striatum and Ventromedial Prefrontal Cortex Underlie
Value-Based Choices. J. Neurosci. 31, 1606–1613.
Johnson, M.W., Bickel, W.K., 2002. Within-subject comparison of real and
hypothetical money rewards in delay discounting. J. Exp. Anal. Behav. 77, 129–
146.
Kable, J.W., Glimcher, P.W., 2007. The neural correlates of subjective value during
intertemporal choice. Nat. Neurosci. 10, 1625–33.
Kahneman, D., Tversky, A., 1979. Prospect theory: an analysis of decision under risk.
Econometrica 47, 263–291.
Kobayashi, S., Schultz, W., 2008. Influence of reward delays on responses of dopamine
neurons. J. Neurosci. 28, 7837–46.
Kroemer, N.B., Guevara, A., Ciocanea Teodorescu, I., Wuttig, F., Kobiella, A., Smolka,
M.N., 2014. Balancing reward and work: Anticipatory brain activation in NAcc
and VTA predict effort differentially. Neuroimage 102, 510–519.
Kurniawan, I.T., Guitart-Masip, M., Dolan, R.J., 2011. Dopamine and effort-based
decision making. Front. Neurosci. 5, 81.
Kurniawan, I.T., Seymour, B., Talmi, D., Yoshida, W., Chater, N., Dolan, R.J., 2010.
Choosing to make an effort: the role of striatum in signaling physical effort of a
chosen action. J. Neurophysiol. 104, 313–21.
Lagorio, C.H., Madden, G.J., 2005. Delay discounting of real and hypothetical rewards
III: Steady-state assessments, forced-choice trials, and all real rewards. Behav.
Processes 69, 173–187.
Levy, D.J., Glimcher, P.W., 2012. The root of all value: a neural common currency for
choice. Curr. Opin. Neurobiol. 22, 1027–38.
Madden, G.J., Raiff, B.R., Lagorio, C.H., Begotka, A.M., Mueller, A.M., Hehli, D.J.,
Wegener, A. a, 2004. Delay discounting of potentially real and hypothetical
rewards: II. Between- and within-subject comparisons. Exp. Clin.
Psychopharmacol. 12, 251–261.
Mcclure, S.M., Ericson, K.M., Laibson, D.I., Loewenstein, G., Cohen, J.D., 2007. Time
Bernacer et al.
28
Discounting for Primary Rewards. J. Neurosci. 27, 5796–5804.
McKerchar, T.L., Green, L., Myerson, J., Pickford, T.S., Hill, J.C., Stout, S.C., 2009. A
comparison of four models of delay discounting in humans. Behav. Processes 81,
256–259.
Meyniel, F., Sergent, C., Rigoux, L., Daunizeau, J., Pessiglione, M., 2013.
Neurocomputational account of how the human brain decides when to have a break.
Proc. Natl. Acad. Sci. U. S. A. 110, 2641–6.
Mitchell, S.H., 2004. Effects of short-term nicotine deprivation on decision-making:
Delay, uncertainty and effort discounting. Nicotine Tob. Res. 6, 819–828.
Montague, P.R., King-Casas, B., Cohen, J.D., 2006. Imaging valuation models in
human choice. Annu. Rev. Neurosci. 29, 417–48.
O’Doherty, J.P., 2011. Contributions of the ventromedial prefrontal cortex to goal-
directed action selection. Ann. N. Y. Acad. Sci. 1239, 118–29.
Pessiglione, M., Schmidt, L., Draganski, B., Kalisch, R., Lau, H., Dolan, R.J., Frith,
C.D., 2007. How the brain translates money into force: a neuroimaging study of
subliminal motivation. Science (80-. ). 316, 904–6.
Peters, J., Büchel, C., 2010. Neural representations of subjective reward value. Behav.
Brain Res. 213, 135–41.
Peters, J., Büchel, C., 2009. Overlapping and distinct neural systems code for subjective
value during intertemporal and risky decision making. J. Neurosci. 29, 15727–34.
Pine, A., Shiner, T., Seymour, B., Dolan, R.J., 2010. Dopamine, time, and impulsivity
in humans. J. Neurosci. 30, 8888–96.
Prévost, C., Pessiglione, M., Météreau, E., Cléry-Melin, M.-L., Dreher, J.-C., 2010.
Separate valuation subsystems for delay and effort decision costs. J. Neurosci. 30,
14080–90.
Salamone, J.D., 2009. Dopamine, Behavioral Economics, and Effort. Front. Behav.
Neurosci. 3, 1–12.
Schmidt, L., Clery-Melin, M.-L., Lafargue, G., Valabregue, R., Fossati, P., Dubois, B.,
Pessiglione, M., 2009. Get Aroused and Be Stronger: Emotional Facilitation of
Physical Effort in the Human Brain. J. Neurosci. 29, 9450–9457.
Scholl, J., Kolling, N., Nelissen, N., Wittmann, M.K., Harmer, C.J., Rushworth, M.F.S.,
2015. The good, the bad, and the irrelevant: Neural mechanisms of learning real
and hypothetical rewards and effort. J. Neurosci. 35, 11233–11251.
Skvortsova, V., Palminteri, S., Pessiglione, M., 2014. Learning To Minimize Efforts
Bernacer et al.
29
versus Maximizing Rewards: Computational Principles and Neural Correlates. J.
Neurosci. 34, 15621–15630.
Treadway, M.T., Buckholtz, J.W., Cowan, R.L., Woodward, N.D., Li, R., Ansari, M.S.,
Baldwin, R.M., Schwartzman, A.N., Kessler, R.M., Zald, D.H., 2012.
Dopaminergic mechanisms of individual differences in human effort-based
decision-making. J. Neurosci. 32, 6170–6.
Treadway, M.T., Buckholtz, J.W., Schwartzman, A.N., Lambert, W.E., Zald, D.H.,
2009. Worth the “EEfRT”? The effort expenditure for rewards task as an objective
measure of motivation and anhedonia. PLoS One 4, 1–9.
Weber, B.J., Huettel, S. a, 2008. The neural substrates of probabilistic and intertemporal
decision making. Brain Res. 1234, 104–15.
Wierenga, C.E., Bischoff-Grethe, A., Melrose, A.J., Irvine, Z., Torres, L., Bailer, U.F.,
Simmons, A., Fudge, J.L., McClure, S.M., Ely, A., Kaye, W.H., 2015. Hunger
does not motivate reward in women remitted from anorexia nervosa. Biol.
Psychiatry 77, 642–652.
Wittmann, M., Leland, D.S., Paulus, M.P., 2007. Time and decision making: differential
contribution of the posterior insular cortex and the striatum during a delay
discounting task. Exp. brain Res. 179, 643–53.
Bernacer et al.
30
Table 1: Clusters surviving the statistical threshold (Z>2.3, P<0.05 corrected) for the
two contrasts of interest. Coordinates are given in standard space. See text for details
about the region of interest. DMPFC, dorsomedial prefrontal cortex; L, left; R, right;
ROI, region of interest; VLPFC, ventrolateral prefrontal cortex.
Cluster Voxels Z max P
Coordinates
(X,Y,Z) Area
Difference Subjective Value vs Motor Control (whole brain)
1
1174
3.75
1.91e-6
0,44,40
DMPFC
2
768
3.98
0.00016
-20,-38,12
L parietal
3
551
3.8
0.00236
36,26,-18
R VLPFC
4
364
3.45
0.0321
-52,-64,38
L Angular gyrus
Difference effort discounting vs Motor Control (ROI)
1
158
3.72
0.0358
-54,28,6
L VLPFC
Bernacer et al.
31
Figures
Figure 1: Behavioral task and main results of the first experiment. A. Task used to
assess effort discounting in the whole sample (N=57). A fixed option (winning 5 €
without effort) was presented simultaneously with an effortful option that entailed a
larger reward and different levels of effort. See “2.2. Tasks” for details. B. Example of
logistic fitting to the actual behavior of one participant for 30 min running in the
treadmill. X axis represent money (in €), and Y axis is the fraction of the effortful
choice. The intersection of the dashed line with the X axis represents the indifference
point (IP). C. Two examples of hyperbolic effort-discounting curves for two individuals,
showing a low (left) and a high (right) effort discounting. D. Group hyperbolic and
double exponential fitting to effort discounting. Data points represent the median and
error bars the SEM. R2 indicates goodness of fit after sum of least squares, adjusted for
the number of constants in each formula. E. Scatterplot to illustrate the partial
correlation of individual hyperbolic K and habitual physical activity (METs),
controlling for the individual R2 values. Unstandardized residuals are calculated by a
linear regression considering K (or METs) as dependent variable, and R2 as independent
variable.
Bernacer et al.
32
Figure 2: fMRI task and neuroimaging results. A. Left, The decision-making task
included pairs of options involving the probability (30-70%) of winning a fixed reward
(30 €) in exchange for some effort (10-30 min running in a treadmill). Right, Display of
the motor control used in the task. Subjects were instructed to select the option with the
“O”. B. Clusters surviving the statistical threshold (Z>2.3, P<0.05 whole brain
corrected) for the contrast difference subjective value vs motor control. C. Region of
interest used to assess the neural correlates of effort-related subjective value, including
the striatum and ventral prefrontal cortex. D. Clusters surviving the statistical threshold
(Z>2.3, P<0.05 small volume corrected) for the contrast difference effort discounting vs
motor control. Right side of the brain is displayed on the left side of the image for
coronal and axial views.
... Thus, every decision is a trade-off between costs and benefits where the agent decides the convenience of selecting an expected outcome or doing an action. SV can be computed as the objective expected reward multiplied by the subjective discounting rate (ranging from 0 to 1) of the corresponding devaluating factor (time, risk or effort) (see, for example, Arrondo et al., 2015;Bernacer et al., 2016;Green & Myerson, 2004;Kable & Glimcher, 2007;Peters & Büchel, 2010). It is widely accepted that activity in ventromedial regions of the prefrontal cortex correlates with SV of the chosen option (Levy & Glimcher, 2012); beside this, other regions are involved in SV comparison when two options are presented, such as the posterior cingulate (Bartra, McGuire, & Kable, 2013;FitzGerald, Seymour, & Dolan, 2009) and dorsomedial prefrontal cortex (Rushworth, Kolling, Sallet, & Mars, 2012). ...
... The effort-and risk-discounting tasks follow the general structure of wellestablished psychological paradigms (Green & Myerson, 2004;Kable & Glimcher, 2007), and have been described elsewhere (Bernacer et al., 2016). Briefly, pairs of options were sequentially presented to each volunteer. ...
... The fMRI H-related task has been also described in a previous report (Bernacer et al., 2016). The goal is, by randomly presenting the same pair of options several times, to provoke choice inconsistency. ...
Article
Full-text available
According to the theory of value-based decision making, subjects tend to choose the most valuable among a set of options. However, agents may not be consistent when facing the same decision several times. In this paper, Shannon’s entropy (H) is employed as a measure of behavioral inconsistency: it is a central measure of information theory that, applied to decision making, allows the estimation of behavioral preferences among a set of options. We scanned (functional magnetic resonance imaging, fMRI) 24 young (18-25 year) subjects (14 female) while performing a decision-making task, where monetary rewards were devalued by physical effort (minutes running in the treadmill) and risk. Twenty different pairs of options were presented nine times each, and H was calculated for each pair and subject. Behavioral analyses showed that subjective value (SV) significantly explained agents’ preferences only in pairs with a low inconsistent response. Averaged response time positively correlated with H, confirming entropy as an indicator of choice difficulty. Group analyses on fMRI data revealed a cluster in the paracingulate cortex as the neural correlate of H. Besides, BOLD signal in the posterior cingulate correlated with the SV of the pair only in consistent decisions, confirming that SV loses its explanatory power on highly inconsistent decisions. Finally, the anterior and central cingulate were especially recruited when predicting a secured effortless reward, compared with a secured reward that involved a maximum effort. Our study shows that different regions of the cingulate cortex are involved in choice inconsistency, SV and processing effort costs
... Maladaptive sensitivity to effort is known to play a key role in multiple disorders that feature symptoms of impaired motivation such as apathy, major depression and schizophrenia Cléry-Melin et al., 2011;Gold et al., 2013;Salamone et al., 2006). Furthermore, overweighting of effort costs was found to be linked with physical inactivity and sedentary lifestyle (Bernacer et al., 2016), a major risk factor for diseases and millions of premature deaths worldwide each year (Lee et al., 2012). Understanding how humans evaluate energetic costs to form decisions and the underlying neural processes offers great potential for developing treatments and interventions for impaired motivation. ...
... This suggests that ACC indeed might be involved in expectation rather than effort cost encoding (Arulpragasam et al., 2018). Furthermore, this interpretation is supported by similar neuroimaging studies that did not find significant evidence for ACC modulation by effort cost in prospective effort choice (Bernacer et al., 2016) and voluntary effort-based choices, either physical (Kurniawan et al., 2013) or cognitive (Schouppe et al., 2014). Interestingly, a study that examined the brain correlates of cognitive effort discounting found that only nucleus accumbens activity represented effort demand but that preceding activation in the dorsal anterior cingulate cortex correlated with nucleus accumbens activity (Botvinick et al., 2009). ...
Article
How is effort integrated in value-based decision-making? Animal models and human neuroimaging studies primarily linked the anterior cingulate cortex (ACC) and ventral striatum (VS) to the integration of effort in valuation. Other studies demonstrated the role of these regions in invigoration to effort demands, thus it is hard to separate the neural activity linked to anticipation and subjective valuation from actual performance. Here, we studied the neural basis of effort valuation separated from performance. We scanned forty participants with fMRI, while they were asked to accept or reject monetary gambles that could be resolved with future performance of a familiar grip force effort challenge or a fixed risk prospect. Participants’ willingness to accept prospective gambles reflected discounting of values by physical effort and risk. Choice-locked neural activation in contralateral primary sensory cortex and ventromedial prefrontal cortex (vmPFC) tracked the magnitude of prospective effort the participants faced, independent of choice time and monetary stakes. Estimates of subjective value discounted by effort were found to be tracked by the activation of a network of regions common to valuation under risk and delay, including vmPFC, VS and sensorimotor cortex. Together, our findings show separate neural mechanisms underlying prospective effort and actual effort performance.
... Precommitment schemes were shown to raise retirement saving rates [19], chances of smoking cessation [20], healthy food shopping [21] and choices of delayed rewards over instant gratification [22]. But, it remains unexplored whether precommitment can also help to increase choices of effortful actions over effortless gratification, even though this would be highly relevant for interventions targeting physical inactivity and sedentary lifestyle in the still healthy and patient populations (see [23]). Our goal was to test the effectiveness of precommitment in the context of effort-requiring rewards and thereby provide an empirical cornerstone for clinical application of precommitment schemes in rehabilitation and exercise interventions. ...
Article
Full-text available
Letting effort-free gratification derail us from effort-requiring goals is one reason why we fail to realize health-relevant intentions like ‘exercise regularly’. We tested the effectiveness of the self-control strategy precommitment in such effort-related conflicts, using a novel laboratory choice paradigm, where participants could precommit to an effort-requiring large reward by pre-eliminating an effort-free small reward from their choice set. Our participants used precommitment frequently and effectively, such that they reached effort-requiring large rewards more often. Using computational modelling and Bayesian model comparisons, we assessed whether participants employed precommitment to avoid anticipated willpower failures (i.e. as a self-regulatory measure) or to maximize their motivation to choose the effort-requiring option (i.e. as a self-motivational measure). Observed choices and precommitment decisions were consistent with the motivation maximization hypothesis, but not the willpower hypothesis. Our findings show that offering precommitment is effective in helping individuals optimize their motivation and choice behaviour and thereby achieve effort-requiring goals, and strongly encourage application of precommitment schemes in exercise and rehabilitation interventions. This article is part of the theme issue ‘Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications’.
Article
Full-text available
Motivational deficits play a central role in disability due to negative symptoms of schizophrenia (SZ), but limited pathophysiological understanding impedes critically needed therapeutic development. We applied an fMRI Effort Discounting Task (EDT) that quantifies motivation using a neuroeconomic decision-making approach, capturing the degree to which effort requirements produce reductions in the subjective value (SV) of monetary reward. An analyzed sample of 21 individuals with SZ and 23 group-matched controls performed the EDT during fMRI. We hypothesized that ventral striatum (VS) as well as extended brain motivation circuitry would encode SV, integrating reward and effort costs. We also hypothesized that VS hypoactivation during EDT decisions would demonstrate a dimensional relationship with clinical amotivation severity, reflecting greater suppression by effort costs. As hypothesized, VS as well as a broader cortico-limbic network were activated during the EDT and this activation correlated positively with SV. In SZ, activation to task decisions was reduced selectively in VS. Greater VS reductions correlated with more severe clinical amotivation in SZ and across all participants. However, these diagnosis and amotivation effects could not be explained by the response to parametric variation in reward, effort, or model-based SV. Our findings demonstrate that VS hypofunction in schizophrenia is manifested during effort-based decisions and reflects dimensional motivation impairment. Dysfunction of VS impacting effort-based decision-making can provide a target for biomarker development to guide novel efforts to assess and treat disabling amotivation.
Article
Full-text available
[Free access until November 9, 2019: https://reader.elsevier.com/reader/sd/pii/S1053811919307724?token=31B33EDB338C7BA7F6C32D21AB937631E9578837DACD8AD36036B326B3750B172D98893D39DEDF8FDAAD224BD5F8BCAC] When humans make decisions, objective rewards are mainly discounted by delay, risk and effort. Whereas recent research has demonstrated that several brain areas process costs and code subjective value in effort-based decision making, it remains obscure how neural activity patterns change when effort costs are reduced due to the acquisition of healthy habits, such as moving from sedentary to active lifestyles. Here, a sample of sedentary volunteers was behaviorally assessed and fMRI-scanned before and after completing a 3-month fitness plan. The impact of effort cost on decisions, measured as the constant defining a hyperbolic decaying function, was reduced after the plan. A logistic mixed model demonstrated that the explanatory power of effort decreased with time. At a neural level, there was a marginally significant disruption of effort-cost related functional activity in the anterior cingulate after the plan. Functional connectivity between the amygdala and anterior cingulate cortex was strengthened after habit acquisition. In turn, this interaction was stronger in those participants with lower effort discounting. Thus, we show for the first time changes in value-based decision making after moving from a sedentary to an active lifestyle, which points to the relevance of the amygdala-cingulate interplay when the impact of effort on decisions fades away.
Article
Full-text available
Scientific research leans on the theoretical assumptions that have been taken for granted through decades of research. Experimental psychology, mostly rooted in experiments with rodents, defines habits as rigid, unconscious and non-teleological behaviors opposed to goal-directed actions. This definition has been transferred to human research as such, and habits are thus viewed as compulsions, obsessions, slips-of-action and addictive behaviors. From an experiential point of view, however, humans possess habits that go beyond these behaviors. According to Aristotle, habits are dispositions of thought and behavior, usually acquired by repetition, which predispose our future behavior. This ‘new’ understanding of human habits would be associated with a brain configuration that goes beyond the rigid carving of motor routines in certain areas. An empirical application of this interpretation is explained. In conclusion, a novel perspective is proposed to study the neural correlates of habits and their impact on behavior.
Article
Full-text available
Lack of physical engagement, productivity, and initiative-so-called "behavioral apathy"-is a common problem with significant impact, both personal and economic. Here, we investigate whether there might be a biological basis to such lack of motivation using a new effort and reward-based decision-making paradigm, combined with functional and diffusion-weighted imaging. We hypothesized that behavioral apathy in otherwise healthy people might be associated with differences in brain systems underlying either motivation to act (specifically in effort and reward-based decision-making) or in action processing (transformation of an intention into action). The results demonstrate that behavioral apathy is associated with increased effort sensitivity as well as greater recruitment of neural systems involved in action anticipation: supplementary motor area (SMA) and cingulate motor zones. In addition, decreased structural and functional connectivity between anterior cingulate cortex (ACC) and SMA were associated with increased behavioral apathy. These findings reveal that effort sensitivity and translation of intentions into actions might make a critical contribution to behavioral apathy. We propose a mechanism whereby inefficient communication between ACC and SMA might lead to increased physiological cost-and greater effort sensitivity-for action initiation in more apathetic people.
Article
Full-text available
Unlabelled: Natural environments are complex, and a single choice can lead to multiple outcomes. Agents should learn which outcomes are due to their choices and therefore relevant for future decisions and which are stochastic in ways common to all choices and therefore irrelevant for future decisions between options. We designed an experiment in which human participants learned the varying reward and effort magnitudes of two options and repeatedly chose between them. The reward associated with a choice was randomly real or hypothetical (i.e., participants only sometimes received the reward magnitude associated with the chosen option). The real/hypothetical nature of the reward on any one trial was, however, irrelevant for learning the longer-term values of the choices, and participants ought to have only focused on the informational content of the outcome and disregarded whether it was a real or hypothetical reward. However, we found that participants showed an irrational choice bias, preferring choices that had previously led, by chance, to a real reward in the last trial. Amygdala and ventromedial prefrontal activity was related to the way in which participants' choices were biased by real reward receipt. By contrast, activity in dorsal anterior cingulate cortex, frontal operculum/anterior insula, and especially lateral anterior prefrontal cortex was related to the degree to which participants resisted this bias and chose effectively in a manner guided by aspects of outcomes that had real and more sustained relationships with particular choices, suppressing irrelevant reward information for more optimal learning and decision making. Significance statement: In complex natural environments, a single choice can lead to multiple outcomes. Human agents should only learn from outcomes that are due to their choices, not from outcomes without such a relationship. We designed an experiment to measure learning about reward and effort magnitudes in an environment in which other features of the outcome were random and had no relationship with choice. We found that, although people could learn about reward magnitudes, they nevertheless were irrationally biased toward repeating certain choices as a function of the presence or absence of random reward features. Activity in different brain regions in the prefrontal cortex either reflected the bias or reflected resistance to the bias.
Article
Full-text available
The mechanisms of reward maximization have been extensively studied at both the computational and neural levels. By contrast, little is known about how the brain learns to choose the options that minimize action cost. In principle, the brain could have evolved a general mechanism that applies the same learning rule to the different dimensions of choice options. To test this hypothesis, we scanned healthy human volunteers while they performed a probabilistic instrumental learning task that varied in both the physical effort and the monetary outcome associated with choice options. Behavioral data showed that the same computational rule, using prediction errors to update expectations, could account for both reward maximization and effort minimization. However, these learning-related variables were encoded in partially dissociable brain areas. In line with previous findings, the ventromedial prefrontal cortex was found to positively represent expected and actual rewards, regardless of effort. A separate network, encompassing the anterior insula, the dorsal anterior cingulate, and the posterior parietal cortex, correlated positively with expected and actual efforts. These findings suggest that the same computational rule is applied by distinct brain systems, depending on the choice dimension— cost or benefit—that has to be learned.
Article
Full-text available
Previous theories predict that human dorsal anterior cingulate (dACC) should respond to decision difficulty. An alternative theory has been recently advanced that proposes that dACC evolved to represent the value of 'non-default', foraging behavior, calling into question its role in choice difficulty. However, this new theory does not take into account that choosing whether or not to pursue foraging-like behavior can also be more difficult than simply resorting to a default. The results of two neuroimaging experiments show that dACC is only associated with foraging value when foraging value is confounded with choice difficulty; when the two are dissociated, dACC engagement is only explained by choice difficulty, and not the value of foraging. In addition to refuting this new theory, our studies help to formalize a fundamental connection between choice difficulty and foraging-like decisions, while also prescribing a solution for a common pitfall in studies of reward-based decision making.
Article
Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions—particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex—tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.
Article
Numerous experiments have recently sought to identify neural signals associated with the subjective value (SV) of choice alternatives. Theoretically, SV assessment is an intermediate computational step during decision making, in which alternatives are placed on a common scale to facilitate value-maximizing choice. Here we present a quantitative, coordinate-based meta-analysis of 206 published fMRI studies investigating neural correlates of SV. Our results identify two general patterns of SV-correlated brain responses. In one set of regions, both positive and negative effects of SV on BOLD are reported at above-chance rates across the literature. Areas exhibiting this pattern include anterior insula, dorsomedialprefrontal cortex, dorsal and posterior striatum, and thalamus. The mixture of positive and negative effects potentially reflects an underlying U-shaped function, indicative of signal related to arousal or salience. In a second set of areas, including ventromedial prefrontal cortex and anterior ventral striatum, positive effects predominate. Positive effects in the latter regions are seen both when a decision is confronted and when an outcome is delivered, as well as for both monetary and primary rewards. These regions appear to constitute a “valuation system,” carrying a domain-general SV signal and potentially contributing to value-based decision making.
Article
Studies on animals and humans have demonstrated the importance of dopamine in modulating decision-making processes. In this work, we have tested dopaminergic modulation of economic decision-making and its neural correlates by administering either placebo or metoclopramide, a dopamine D2-receptor antagonist, to healthy subjects, during a functional MRI study. The decision-making task combined probability and time delay with a fixed monetary reward. For individual behavioral characterization, we used the Probability Time Trade-off (PTT) economic model, which integrates the traditional trade-offs of reward magnitude-time and reward magnitude-probability into a single measurement, thereby quantifying the subjective value of a delayed and probabilistic outcome. A regression analysis between BOLD signal and the PTT model index permitted to identify the neural substrate encoding the subjective reward-value. Behaviorally, medication reduced the rate of temporal discounting over probability, reflected in medicated subjects being more prone to postpone the reward in order to increase the outcome probability. In addition, medicated subjects showed less activity during the task in the postcentral gyrus as well as frontomedian areas, whereas there were no differences in the ventromedial orbitofrontal cortex (VMOFC) between groups when coding the subjective value. The present study demonstrates by means of behavior and imaging that dopamine modulation alters the probability-time trade-off in human economic decision-making. Copyright © 2015 Elsevier B.V. and ECNP. All rights reserved.
Article
Background Hunger enhances sensitivity to reward, yet individuals with anorexia nervosa (AN) are not motivated to eat when starved. This study examined whether diminished response to reward could underlie food restriction in AN by investigating brain response to rewards during hunger and satiated states. Methods Using a delay discounting monetary decision task known to discriminate brain regions contributing to processing of immediate rewards and cognitive control important for decision making regarding future rewards, we compared 23 adults remitted from AN (to reduce the confounding effects of starvation [RAN]) to 17 healthy women (CW). Monetary rewards were used because the rewarding value of food may be confounded by anxiety in AN. Results Interactions of group (RAN, CW) x visit (hunger, satiety) revealed that, for CW, hunger significantly increased activation in reward salience circuitry (ventral striatum, dorsal caudate, anterior cingulate cortex) during processing of immediate reward, whereas satiety increased activation in cognitive control circuitry (ventrolateral prefrontal cortex, insula) during decision-making. In contrast, brain response in reward and cognitive neurocircuitry did not differ during hunger and satiety in RAN. A main effect of group revealed elevated response in the middle frontal gyrus for RAN. Conclusions RAN failed to increase activation of reward valuation circuitry when hungry and showed elevated response in cognitive control circuitry independent of metabolic state. Decreased sensitivity to hunger’s motivational drive may explain AN individuals’ ability to restrict food when emaciated. Moreover, difficulties in valuating emotional salience may contribute to inabilities to appreciate the risks inherent in this deadly disorder.
Article
Complex decision-making involves anticipation of future rewards to bias effort for obtaining it. Using fMRI, we investigated 50 participants employing an instrumental-motivation task that cued reinforcement levels before onset of the motor-response phase. We extracted timecourses from regions of interest (ROI) in the mesocorticolimbic system and used a three-level hierarchical model to separate anticipatory brain responses predicting value and subsequent effort on a trial-by-trial basis. Whereas all ROIs scaled positively with value, higher effort was predicted by higher anticipatory activation in nucleus accumbens (NAcc) but lower activation in ventral tegmental area/substantia nigra (VTA/SN). Moreover, anticipatory activation in the dorsal striatum predicted average effort whereas higher activation in the amygdala predicted above-average effort. Thus, anticipatory activation entails the appetitive drive towards reinforcement that requires effort in order to be obtained. Our results support the role of NAcc as main hub supported by the salience network operating on a trial-by-trial basis, while the dorsal striatum incorporates habitual responding.