Working Memory Capacity Predicts Effects of Methylphenidate on Reversal Learning

Abstract

Increased use of stimulant medication, such as methylphenidate, by healthy college students has raised questions about its cognitive enhancing effects. Methylphenidate acts by increasing extracellular catecholamine levels and is generally accepted to remediate cognitive and reward deficits in patients with attention deficit hyperactivity disorder. However, the cognitive enhancing effects of such 'smart drugs' in the healthy population are still unclear. Here, we investigated effects of methylphenidate (Ritalin®, 20 mg) on reward and punishment learning in healthy students (N=19) in a within-subjects, double-blind, placebo-controlled cross-over design. Results revealed that methylphenidate effects varied both as a function of task demands and as a function of baseline working memory capacity. Specifically, methylphenidate improved reward versus punishment learning in high working memory subjects, while it impaired reward versus punishment learning in low working memory subjects. These results contribute to our understanding of individual differences in the cognitive enhancing effects of methylphenidate in the healthy population. Moreover, they highlight the importance of taking into account both inter- and intra-individual differences in dopaminergic drug research.Neuropsychopharmacology accepted article preview online, 23 April 2013; doi:10.1038/npp.2013.100.

Full text

Working Memory Capacity Predicts Effects of
Methylphenidate on Reversal Learning
Marieke E van der Schaaf*
,1
, Sean J Fallon
2
, Niels ter Huurne
1
, Jan Buitelaar
3,4
and Roshan Cools
1,2
1
Department of Psychiatry, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands;
2
Centre for Cognitive Neuroimaging,
Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, The Netherlands;
3
Department of Cognitive
Neuroscience, Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands;
4
Karakter Child and Adolescent Psychiatry University Center, Nijmegen, The Netherlands
Increased use of stimulant medication, such as methylphenidate, by healthy college students has raised questions about its cognitive-
enhancing effects. Methylphenidate acts by increasing extracellular catecholamine levels and is generally accepted to remediate cognitive
and reward deficits in patients with attention deficit hyperactivity disorder. However, the cognitive-enhancing effects of such ‘smart drugs’
in the healthy population are still unclear. Here, we investigated effects of methylphenidate (Ritalin, 20 mg) on reward and punishment
learning in healthy students (N¼19) in a within-subject, double-blind, placebo-controlled cross-over design. Results revealed that
methylphenidate effects varied both as a function of task demands and as a function of baseline working memory capacity. Specifically,
methylphenidate improved reward vs punishment learning in high-working memory subjects, whereas it impaired reward vs punishment
learning in low-working memory subjects. These results contribute to our understanding of individual differences in the cognitive-
enhancing effects of methylphenidate in the healthy population. Moreover, they highlight the importance of taking into account
both inter- and intra-individual differences in dopaminergic drug research.
Neuropsychopharmacology (2013) 38, 2011–2018; doi:10.1038/npp.2013.100; published online 8 May 2013
Keywords: cognition; cognitive enhancement; dopamine; learning & memory; psychostimulants; punishment; reversal
learning; reward; Ritalin

INTRODUCTION
Psychostimulants such as methylphenidate (MPH) are the
first-line medication treatment of attention deficit hyper-
activity disorder (ADHD). In the clinical context, MPH
decreases symptoms of inattention and hyperactivity
(Faraone and Buitelaar, 2010) and may improve attentional
and academic performance (Marcus and Durkin, 2011;
Wigal et al, 2011). However, recent surveys report that
healthy college students also increasingly use these stimu-
lants to enhance concentration and study performance
(Bundt et al, 2012; Smith and Farah, 2011). This raises
questions about their cognitive-enhancing effects in the
healthy population (Greely et al, 2008). MPH primarily
acts by blocking the dopamine transporter (DAT),
which removes excessive dopamine from the synaptic cleft,
resulting in the increase of extracellular dopamine levels
(Volkow et al, 2002). In ADHD, MPH is thought to be
effective by restoring deficient catecholamine levels. How-
ever, effects are still unclear in the healthy population, in
which catecholamine levels are generally assumed to be
relatively optimized in comparison to individuals with
disorders that implicate dopamine.
There is a large variability in the cognitive-enhancing
effects of psychostimulants in healthy adults (Smith and
Farah, 2011) and there are numerous factors that may
contribute to this variability. First, dopaminergic drug
effects can vary as a function of task demands, so that some
tasks are improved while other tasks are impaired (Cools
and D’Esposito, 2011). For example, administration of
dopamine receptor agonists improves learning from reward
while impairing learning from punishment in the same
subjects (Bodi et al, 2009; Cools et al, 2009; Frank and
O’Reilly, 2006). This concurs with observations that
psychostimulant treatment such as methylphenidate im-
proves reward but not punishment learning in patients
with ADHD (Frank et al, 2007). Such differential effects
likely reflect distinct optimal levels of striatal dopamine,
with high dopamine levels facilitating learning from
reward and low dopamine levels facilitating learning from
punishment (Frank, 2005). Here we assessed whether
methylphenidate also has opposite effects on reward and
punishment learning in healthy adults, thus extending
methylphenidate’s therapeutic effects in ADHD (Frank et al,
2007) to effects of so-called cognitive enhancers in the
*Correspondence: Dr ME van der Schaaf, Donders Centre for
Cognitive Neuroimaging, PO Box 9101, 6500 HB Nijmegen,
The Netherlands. Tel: +31 24 366 8292, Fax: +31 24 361 0989,
E-mail: Marieke.vanderschaaf@donders.ru.nl
Received 21 December 2012; revised 5 April 2013; accepted 8 April
2013; accepted article preview online 23 April 2013
Neuropsychopharmacology (2013) 38, 2011 – 2018
&
2013 American College of Neuropsychopharmacology. All rights reserved 0893-133X/13
www.neuropsychopharmacology.org

healthy population. A second factor is the large inter-
individual variability in drug response (Cools and
D’Esposito, 2011). The same drug and doses can have
different, even opposite, effects between different indivi-
duals depending on clinical condition (Mehta et al, 2004;
Voon et al, 2010), baseline working memory capacity
(Frank and O’Reilly, 2006; Kimberg et al, 1997; Mehta et al,
2000; Mehta et al, 2001; van der Schaaf et al, 2012), and
baseline levels of dopamine function as measured with
positron emission tomography (PET) (Cools et al, 2009).
For example, bromocriptine was shown to improve reward
learning in subjects with low baseline levels of dopamine
synthesis capacity, whereas impairing it in subjects with
high baseline levels of dopamine synthesis capacity (Cools
et al, 2009). Accordingly, it has been suggested that the
effects of dopamine receptor agents depend on the baseline
state of the system (Cools and D’Esposito, 2011). By
analogy, effects of MPH might also depend on the baseline
state of the system.
To test these hypotheses, we administered a single dose of
methylphenidate and placebo to healthy controls in a
double-blind, placebo-controlled cross-over design and we
stratified our drug effects by baseline working memory
capacity; a measure that has previously been shown to be
positively associated with striatal dopamine synthesis
capacity (Cools et al, 2008; Landau et al, 2009). The
direction of the relationship between MPH effects and
baseline working memory capacity might parallel that seen
for dopamine receptor agonists (Cools et al, 2009; Frank
and O’Reilly, 2006; Mehta et al, 2000, 2001), with greater
improvements in reward vs punishment learning in low-
than high-working memory subjects. Alternatively, it has
been proposed that DAT blockade leads to larger increases
in extracellular dopamine in subjects with a higher rate of
dopamine release (Volkow et al, 2002). This is supported by
animal work showing baseline-dependent effects of DAT
blockade on extracellular dopamine concentrations (Hooks
et al, 1992). Accordingly, together with the literature
reviewed above, an alternative hypothesis would be that
MPH would induce larger improvements on reward vs
punishment learning in subjects with high-working memory
capacity.
MATERIALS AND METHODS
Subjects
Twenty-four subjects, recruited via campus advertisements,
gave written informed consent approved by the local
research ethics committee (‘Commissie Mensgebonden
Onderzoek’, Arnhem-Nijmegen, number 2010/283) and
were compensated for participation. The current task was
part of a larger protocol and was preceded by a functional
magnetic resonance imaging (fMRI) experiment and
followed by two behavioral experiments (reported else-
where). Five non-native Dutch speakers (ie, German) were
excluded from analysis because good practice of the Dutch
language was essential for optimal assessment of baseline
working memory capacity as measured with the listening
span and digit span. The remaining 19 subjects (mean
age: 20.9 years, range: 19.0–24.4) were healthy, right-handed
men (9) and women (10) with no relevant medical/
psychiatric history or a history of drug abuse and/or
dependence. Other exclusion criteria included fulfillment of
ADHD criteria, family history of schizophrenia, bipolar
disorder, depression or neurological abnormalities, alcohol
use of more than 20 units per week, habitual smoking (420
cigarettes per week), use of prescribed or over-the-counter
medication within the last month (with the exception of
occasional paracetamol and anti-conceptive medication),
use of recreational drugs within 2 weeks before testing, and
a history of frequent use of recreational drugs or
psychotropic medication (cannabis: more than biweekly
on average or periods using more than weekly; other
recreational drugs (eg cocaine, amphetamines): more than
five times ever; psychotropic medication: more than
biweekly or periods of using more than five times weekly).
Intake Procedure
During a first intake, all subjects were screened by a medical
doctor (NtH) and psychologist (MvdS), which included
physical examination of weight, pulse rate, and blood
pressure, medical examination and administration of the
Mini-International Neuropsychiatric Interview (M.I.N.I.)
(Sheehan et al, 1998), to exclude psychiatric, neurological,
and medical history. Subjects were requested to complete
questionnaires including the Beck Depression Inventory
(Beck et al, 1961), Trait Anxiety Inventory (STAI)
(Spielberger et al, 1970), and Barratt Impulsiveness Scale
(BIS) (Patton et al, 1995). Symptoms of hyperactivity and
attention deficits were assessed with the ADHD self-report
screening questionnaire (DuPaul et al, 1998). Verbal
Intelligence was assessed with the Dutch Adult Reading
Test (NLV) (Schmand et al, 1991). A baseline measure of
working memory capacity was assessed during intake with a
Dutch version of the listening span (Daneman and
Carpenter, 1980) and the digit span (Groth-Marnat, 2001).
Pharmacological Design and Session Procedures
A within-subject, double-blind, placebo-controlled cross-
over design was employed. Subjects were tested after
administration of MPH (Ritalin, 20 mg) or placebo on two
different occasions, separated by at least 1 week. All subjects
abstained from alcohol or over-the-counter medication 24 h
before testing and caffeine on the day of testing. They were
asked to have a light breakfast one hour before arrival,
similar across both sessions. The reversal learning task was
assessed B165 min after drug intake for B20 min, directly
followed by the digit span. Ritalin is effective for B4 h with
peak plasma levels 90 min after dosing (Swanson et al,
2003). Although time of testing was optimized for the
preceding fMRI paradigm, assessment of the current task
coincided with the active time window of drug effects.
Physiology and mood were assessed B15 min prior (T1),
B30 min after (T2) and B210 min (T3) after drug intake.
See Supplementary Materials and Methods for analyses and
results of physiology and mood.
Baseline Working Memory Capacity
Baseline working memory capacity was assessed with the
digit span (Groth-Marnat, 2001) during intake and both
Effects of methylphenidate on reversal learning
ME van der Schaaf et al
2012
Neuropsychopharmacology

drug sessions. As in our previous report (van der Schaaf
et al, 2012), the average total digit span across all three
assessments was selected for drug stratification, because it
was thought to provide a more reliable estimate of working
memory capacity owing to the fact that is was administered
repeatedly. For each session, the individual total score on
the digit span forward and digit span backward was
calculated. These total scores were averaged across the
three assessments and used as a covariate of interest in
the behavioral analysis (see further below). In addition to
the digit span, the listening span (Daneman and Carpenter,
1980) was assessed during intake (two missing values).
To confirm our baseline-dependent results, supplementary
analyses were done with listening span and the digit span
assessed during intake (Supplementary Materials and
Methods).
Reversal Learning Paradigm
We employed a reversal learning task used previously
(Cools et al, 2006) (Figure 1). Two stimuli, a face and a
scene, were presented simultaneously on the screen
(location randomized). One of these stimuli was associated
with reward and the other with punishment (or reward
omission, note that we cannot disentangle the two).
Subjects were required to learn these deterministic
stimulus–outcome associations. Unlike standard (probabil-
istic) reversal paradigms, subjects did not choose between
the two stimuli. Instead, one of the stimuli was already
selected by the computer (highlighted with a black border)
and subjects were asked to predict the outcome of this
preselected stimulus. After the prediction, indicated with a
right index or middle finger button press (counterbalanced
between subjects), the actual outcome was presented after a
1000-ms delay for 500 ms at the location of the stimulus.
There was no time limit to provide a response. Reward
consisted of a green smiley with a ‘ þ$100’ sign. Punish-
ment consisted of a red sad smiley and a ‘ $100’ sign.
Note that this outcome did not depend on subjects’
responses but was directly coupled to the stimulus. After
4–6 consecutive correct predictions the stimulus–outcome
contingency reversed. This was either signaled by an
unexpected reward, presented after the previously pun-
ished stimulus was highlighted, or by an unexpected
punishment, presented after the previously rewarded
stimulus was highlighted. Accuracy on the trials directly
following these unexpected outcomes (reversal trials)
represents how well subjects updated their stimulus–
outcome associations. After unexpected outcomes, the
same stimulus was highlighted again, such that behavioral
and cognitive requirements were matched between valence
conditions.
Each participant performed four experimental blocks that
contained 120 trials: two blocks in which reversals were
signaled by unexpected rewards (reward condition) and two
blocks in which reversals were signaled by unexpected
punishment (punishment condition). Each block consisted
of one acquisition stage until the first reversal and a variable
number of reversal stages, depending on the participant’s
accuracy. On average, each participant performed 23.5
(±5.4) and 23.3 (±4.7) reversal stages in the punishment
and reward condition, respectively. On each session,
subjects performed two practice blocks to familiarize them
with the paradigm. Performance in the experimental blocks
was above chance level (470% correct) for all subjects and
there were no test–retest effects. See Supplementary
Materials and Methods for test–retest effects and informa-
tion about randomization and practice blocks.
Figure 1 Task design. (a) Example of a reward trial. Two stimuli, a face
and a scene, were presented simultaneously. One of the two stimuli was
highlighted with a black border and the task was to predict whether this
stimulus was followed by reward or punishment, after which the actual
outcome was presented (100% deterministic). (b) Example of a trial
sequence for the unexpected reward and unexpected punishment
condition. The participant learned to predict rewards (rw) and punishments
(pn) for the scene and face. Stimulus–outcome associations reversed after
4–6 consecutive correct predictions, signaled by either unexpected reward
or unexpected punishment. Measure of interest was the accuracy on
reversal trials immediately following the unexpected outcomes.
Effects of methylphenidate on reversal learning
ME van der Schaaf et al
2013
Neuropsychopharmacology

Behavioral Data Analysis
There were three trial-types per valence condition: reversal,
non-reversal reward, and non-reversal punishment. Reversal
trials were defined as those trials following an unexpected
outcome. Non-reversal trials were defined as those trials
following expected outcomes and preceding expected
rewards (non-reversal reward) or expected punishments
(non-reversal punishment). Only trials from the reversal
stages (after the first unexpected outcome) and trials
following correct predictions were included in the analysis.
Proportions of correct responses per trial-type were
arcsine transformed (2xarcsine(Ox)) as is appropriate
when the variance is proportional to the mean (Howell,
1997). To investigate whether effects of MPH on reversal
learning depend on baseline working memory, we employed
a repeated measures ANOVA with the within-subject
factors: drug (placebo, MPH), valence (reward, punish-
ment), and trial-type (reversal, non-reversal reward, non-
reversal punishment); and baseline working memory
capacity (mean centered) as covariate of interest. Green-
housse-Geiser correction was applied when sphericity
assumption was violated. Linear relationships between
valence-dependent learning and baseline working memory
capacity were further assessed using Pearson correlation
analysis. To this end, valence-dependent reversal learning
scores were calculated by computing the difference between
the proportion of correct responses on reward and punish-
ment reversal trials. This measure was then correlated with
individual measures of baseline working memory. We
specifically focused our analyses on relative scores because
this measure controls for nonspecific drug effects such as
changes in effort, attention, or alertness. The only measure of
interest that was not normally distributed was the relative
(valence-dependent) reversal score in the MPH condition
(Shapiro-Wilk test: P¼0.016, uncorrected for multiple
comparisons). Therefore we also report the nonparametric
(Spearman’s rho) correlation for our primary effect of interest.
Supplementary Win-Stay Lose-Shift Analysis
Both reward and punishment reversal trials required
response alternation. Therefore, it could be argued that
changes on valence-dependent learning reflect changes
in the adoption of a win-stay/lose-shift strategy, ie, the
tendency to maintain responding after reward (win-stay)
and to alternate responses after punishment (lose-shift).
Thus, improvements on reward relative to punishment
learning could reflect a bias away from a win-stay/lose-shift
strategy or a bias toward a win-shift/lose-stay strategy,
rather than increased ability to update reward relative to
punishment predictions. To test this alternative hypothesis,
we measured the adoption of a win-stay/lose-shift strategy
on the non-reversal trials. We calculated accuracy scores for
the following four trial-types averaged across valence
conditions: non-reversal reward trials after correctly
predicted rewards (win-stay) and correctly predicted
punishment trials (lose-shift), and non-reversal punishment
trials after correctly predicted rewards (win-shift) and
correctly predicted punishments (lose-stay). Drug effects
were assessed with repeated measures ANOVA with the
within-subject factors: drug (placebo, MPH), strategy
(stay, shift), and outcome (reward, punishment); and
working memory capacity as covariate. Direct associations
between drug effects on valence-dependent reversal learning
and strategy were assessed with correlation analyses. To this
end, individual levels of a win-stay/lose-shift strategy
was calculated as ((win-stay þlose-shift)–(win-shift þ
lose-stay)), where higher values reflect a greater adoption
of a win-stay/lose-shift strategy.
RESULTS
Subjects
All 19 subjects were healthy and none met DSM-IV criteria
for ADHD as measured with the self-report symptom que-
stionnaire (symptoms of inattention child: 1.3±1.6, range:
0–6; adult: 0.6±0.8, range: 0–3; symptoms of hyperactivity
child: 1.7±1.6, range: 0–5; adult: 1.3±1.3, range: 0–5) or
depression as measured with the Beck Depression Inventory
(1.4±2.0, range: 0–5). All had normal levels of trait anxiety
(STAI: 30.1±5.5, range: 22–42) and impulsivity (BIS:
61.1±8.0, range: 41–74). There were no associations
between these baseline measures and MPH-induced changes
on valence-dependent reversal learning (all P40.1).
Effects of MPH on Reward vs Punishment Learning
Varied as a Function of Baseline Working Memory
Capacity
MPH was predicted to alter reward vs punishment reversal
learning as a function of baseline working memory
capacity. This was confirmed statistically by a significant
drug valence trial-type span interaction (F
16,2
¼12.8,
Po0.001). A breakdown of this interaction confirmed that
the drug valence span interaction was significant for the
reversal trials (F
17,1
¼23.77, Po0.001), but not for the non-
reversal reward (F
17,1
¼0.02, ns) or non-reversal punishment
trials (F
17,1
¼2.36, P¼0.2). Correlation analysis revealed a
positive relationship between working memory capacity and
MPH effects on valence-dependent reversal learning scores
(r
19,Pearson
¼0.76, Po0.001; r
19,Spearman’s rho
¼0.72, Po0.001).
Thus, we show that MPH improved reward vs punishment
learning in high-working memory subjects, while the opposite
wasseeninlow-workingmemorysubjects(Figure2;Table1;
Supplementary Figure S1). All baseline-dependent effects were
replicated with the digit span and listening span assessed
during intake (Supplementary Materials and Methods). MPH
did not affect digit span itself (drug span: F
18,1
¼0.11, ns;
main effect of drug: F
18,1
¼1.12, P¼0.3), also not as a function
of digit span assessed during intake (drug baseline span:
F
18,1
¼0.11, ns) (Table 2).
Subjective mood and physiology effects were as predicted
with higher reports of subjective alertness and positive
affect as well as heart rate and systolic and diastolic blood
pressure increases after administration of MPH relative to
placebo across all subjects (Supplementary Table S1). These
MPH-induced changes on mood and physiology were not
associated with MPH-induced changes on valence-dependent
reversal learning (all P40.1) (see Supplementary Materials
and Methods for details).
Effects of methylphenidate on reversal learning
ME van der Schaaf et al
2014
Neuropsychopharmacology

Supplementary Win-Stay, Lose-Shift Analysis
Effects on valence-dependent reversal learning could not be
attributed to overall MPH-induced changes on the adoption
of a win-stay/lose-shift strategy. Thus, analysis of MPH
effects on win-stay/lose-shift strategy did not reveal any
span-dependent effect. There was no drug strategy 
outcome span interaction (F
17,1
¼0.004, ns). Furthermore,
correlation analysis revealed that MPH-induced changes on
valence-dependent reversal learning were not associated
with MPH-induced changes on a win-stay/lose-shift strategy
(r
19
¼0.015, ns).
An additional analysis was conducted to assess whether
there were MPH effects on response strategy irrespective
of working memory capacity. This analysis revealed
a significant drug strategy outcome interaction
(F
17
¼6.77, P¼0.019). Post-hoc pair-wise comparisons for
each of the four trial-types separately revealed that MPH
selectively decreased accuracy on win-shift trials
(T
18
¼4.17, Po0.001), while having no effect on win-
stay (T
18
¼0.053, P¼0.95), lose-stay (T
18
¼0.25, P¼0.8),
and lose-shift trials (T
18
¼5, P¼0.63) (Table 3).
DISCUSSION
The increased use of licensed stimulants like MPH by
students in educational settings has raised questions about
its cognitive-enhancing effects in the healthy population
(Greely et al, 2008). One major issue is the large variability
in the effects of such smart drugs on learning and cognition,
both within and across individuals. Here we show that
effects of MPH on reversal learning vary as function of
baseline working memory capacity. Moreover, as predicted,
effects of MPH were valence-dependent, so that MPH
altered reward relative to punishment learning. Specifically,
it improved reward vs punishment learning in high-working
memory subjects, while impairing it in low-working
memory subjects. These effects could not be accounted for
by nonspecific drug effects, for example on alertness or
other subjective effects. These results elucidate two factors
that contribute to the high variability of smart drug efficacy
in the healthy population. First, they demonstrate that
effects of MPH on learning vary within individuals
as a function of the specific demands of the task, with
differential effects on reward and punishment learning.
Second, they demonstrate that effects of MPH on learning
vary between individuals as a function of baseline working
memory capacity, with opposite MPH effects in high- and
Figure 2 Linear relationship between baseline working memory capacity
(digit span) and methylphenidate (MPH) effects on reward vs punishment
learning (r¼0.69, Po0.001). Data on the y-axis reflect arcsine-transformed
valence-dependent reversal scores (ie accuracy on reward relative to
punishment reversal trials) after administration of MPH relative to placebo.
Diamonds represent individual data points.
Table 1 Behavioral Data on the Reversal Learning Task
Placebo MPH
Reward condition
Reversal 0.95 (0.01) 0.98 (0.01)
Non-reversal reward 0.96 (0.01) 0.96 (0.01)
Non-reversal punishment 0.95 (0.01) 0.97 (0.01)
Punishment condition
Reversal 0.95 (0.02) 0.96 (0.01)
Non-reversal reward 0.94 (0.01) 0.97 (0.01)
Non-reversal punishment 0.94 (0.01) 0.96 (0.01)
Abbreviation: MPH, methylphenidate.
Values represent raw accuracy scores (SE) per trial-type and drug condition
averaged across all subjects (N¼19). Reversal trials are presented in bold.
Table 2 Digit Span
Intake Placebo MPH Average
Forward 8.37 (1.92)
a
9.32 (2.31) 9.79 (1.93) 9.16 (1.62)
Backward 7.58 (1.54)
a
8.21 (2.1) 8.47 (1.74) 8.09 (1.57)
Total 15.95 (2.91)
ab
17.53 (3.73) 18.26 (3.14) 17.25 (2.84)
Abbreviation: MPH, methylphenidate.
a
Differs significantly (Po0.05) from the methylphenidate session.
b
Differs significantly (Po0.05) from the placebo session.
Values represent forward, backward, and total span scores (SD) averaged across
all subjects (N¼19) measured during intake, placebo, and methylphenidate
session, and averaged across all measurements.
Table 3 Raw Accuracy Scores on Win-Stay, Lose-Stay, Win-Shift,
and Lose-Shift Trials
MPH Placebo
Win-stay 0.86 (0.03) 0.86 (0.03)
Lose-stay 0.86 (0.04) 0.85 (0.03)
Win-shift 0.82 (0.04)
a
0.85 (0.04)
Lose-shift 0.82 (0.04) 0.81 (0.06)
Abbreviation: MPH, methylphenidate.
a
Differs significantly (Po0.001) from the placebo session.
Values represent raw accuracy scores (SE) per trial-type and drug condition
averaged across all subjects (N¼19).
Effects of methylphenidate on reversal learning
ME van der Schaaf et al
2015
Neuropsychopharmacology

low-working memory subjects. These results help under-
stand the nature of the large inter- and intra-individual
differences in the response to smart drugs like MPH.
Our results are generally consistent with previous work
demonstrating opposite effects of dopaminergic drugs on
reward and punishment learning in healthy subjects (Cools
et al, 2009; Frank and O’Reilly, 2006; van der Schaaf et al,
2012) and further support computational modeling work,
which suggests that striatal dopamine shifts the balance
between reward and punishment learning (Frank, 2005;
Maia and Frank, 2011). More specifically, we observed that
MPH induced a larger improvement in reward vs punish-
ment learning in subjects with higher baseline working
memory capacity. This observation is particularly relevant
in the context of smart drug use in universities (Greely et al,
2008; Maher, 2008; Smith and Farah, 2011), given the large
body of evidence supporting a substantial relationship
between working memory capacity and general fluid
intelligence (eg, Engle et al, 1999). Thus, general fluid
intelligence might help predict whether smart drugs help or
hurt. Baseline working memory capacity has been pre-
viously shown to be a putative proxy of baseline dopamine
synthesis capacity in the striatum (Cools et al, 2008; Landau
et al, 2009). Accordingly, the dependency on working
memory capacity might reflect dependency on dopamine
synthesis capacity, with larger improvements in reward vs
punishment learning in subjects with higher dopamine
synthesis capacity. Our results are therefore consistent with
the dopamine cell-activity hypothesis (Volkow et al, 2002),
suggesting that DAT blockade induces larger dopamine
increases, and thus larger improvements in reward vs
punishment learning (Frank, 2005; Maia and Frank, 2011),
in subjects with high relative to low dopamine cell activity
(Volkow et al, 2002). Furthermore, it is also in line, albeit
indirectly, with prior work revealing larger MPH-induced
impairments on punishment-based reversal learning in
subjects with larger MPH-induced increases in dopamine
release (Clatworthy et al, 2009).
It might be noted that the present study revealed MPH-
induced impairments in reward vs punishment learning,
presumably associated with decreases in dopamine (Frank,
2005), in subjects with low-working memory capacity. This
aspect of our findings is not easily accounted for by the
above-described cell-activity hypothesis, unless MPH acts
more readily via autoregulatory (D2) systems in low- than
in high-working memory/dopamine synthesis subjects.
Further investigation is needed to elucidate the involvement
of these additional mechanisms in the observed span-
dependent effects of MPH. Accordingly, we refrain here
from definitively interpreting the MPH-induced impair-
ments in mechanistic terms. Instead, we emphasize the
relevance of the findings in the context of smart drug use by
healthy adults, by suggesting that baseline working memory
capacity may provide a valuable prediction measure
for individual MPH effects.
At first sight, our findings might appear inconsistent with
recent work showing beneficial effects of MPH on various
cognitive tasks, in subjects who perform poorly at baseline
(Eagle et al, 2007; Finke et al, 2010), consistent with the
well-known inverted-U-shaped relationship between
dopamine and cognitive performance (Arnsten, 1997).
However, it should be noted that our study did in fact also
reveal improvement in performance in low-span subjects,
but only if performance is calculated in terms of punish-
ment vs reward learning. This illustrates one important
take-home message of the present study, that effects of MPH
improve or impair cognition depending on current task
demands.
We did not find any MPH effect on working memory
capacity itself, as measured with the digit span.
This is consistent with our previous report (van der
Schaaf et al, 2012) and various other reports (Oken et al,
1995; Schmedtje et al, 1988; Silber et al, 2006; but see Agay
et al, 2010), but in apparent contrast with other studies
reporting stimulant effects on more complex spatial work-
ing memory tasks (Clatworthy et al, 2009; Mehta et al,
2000). In addition, the direction of our (positive) associa-
tion between digit span and MPH’s effects on reward vs
punishment learning is also in contrast with that observed
previously for spatial working memory (Mehta et al, 2000).
Thus, Mehta et al (2000) have reported a negative rather
than a positive association between digit span and MPH’s
effects on spatial working memory. One explanation for this
apparent discrepancy could be that the tasks have different
cognitive requirements. Indeed, together with prior work by
Clatworthy et al (2009), the current study indicates that the
nature of the relationship between the cognitive effects of
MPH and the baseline state of the system depends critically
on task demands.
MPH sustained subjective feelings of alertness and
positive affect over time and increased heart rate and blood
pressure across all subjects. Importantly, these nonspecific
effects of MPH could not explain the effects of interest on
valence-dependent learning, because, unlike our effects of
interest, they did not depend on working memory capacity.
One possibility is that these effects reflect modulation of
(prefrontal) noradrenalin, known to be involved in sympa-
thetic control, attention, and executive functioning
(Arnsten, 1997).
Effects on valence-dependent reversal learning could not
be attributed to MPH-induced changes on the adoption of a
win-stay/lose-shift strategy as measured on the non-reversal
trials. Supplementary analysis did reveal that MPH selec-
tively decreased the tendency to shift responding (on non-
reversal punishment trials) after non-reversal reward trials,
suggesting enhanced stickiness or response perseveration
after reward. Although these results are potentially
interesting, our task was not designed to directly assess
MPH effects on instrumental response strategy. Accord-
ingly, these results should be considered as preliminary, and
future studies specifically designed to assess instrumental
response strategy should further explore these potential
effects of MPH. Stimulant medication has been shown
to ameliorate reward vs punishment learning deficits in
patients with ADHD (Frank et al, 2007). ADHD has been
associated with low (spatial) working memory (Barkley,
1997) and striatal dopamine deficiency (Volkow et al, 2009).
Thus, the MPH-induced improvement in ADHD seems at
odds with the present finding that MPH impaired reward
relative to punishment learning in healthy adults with low-
working memory capacity. This discrepancy might reflect
the fact that beneficial effects of MPH on reward learning
have been shown only in ADHD patients who have received
long-term stimulant treatment. Long-term stimulant
Effects of methylphenidate on reversal learning
ME van der Schaaf et al
2016
Neuropsychopharmacology

treatment might have effects that are quite different from
those of acute administration (Robbins, 2002). Indeed long-
term stimulant treatment might induce changes in DAT
expression (Fusar-Poli et al, 2012) and/or dopamine signaling
(Grace, 1991; Volkow et al, 2012). Accordingly, the current
study of acute MPH administration in the healthy population
provides a better model of MPH use in healthy adults, who
likely take smart pills only on particular occasions (Smith
and Farah, 2011) and not on the longer term.
The finding that MPH has differential effects on reward
and punishment learning might have relevance to other task
domains, given that many tasks load on either reward or
punishment learning. Thus, improved reward (relative to
punishment) learning might well translate to enhanced
attribution of positive value to study material and thus
increase student interest and motivation in schoolwork. In
line with that, mesolimbic dopamine is thought to modulate
the attribution of incentive salience to stimuli that drive
behavior and become the focus of goal-directed behavior
(Berridge and Robinson, 1998). This is further supported by
a positive correlation between MPH-induced extracellular
dopamine increases and subjective ratings of interest,
excitement, and motivation for a mathematical task
(Volkow et al, 2004). Conversely, improved punishment
(relative to reward) learning might translate to enhanced
attribution of negative value, something characteristic of
mood disorders like depression (Clark et al, 2009; Robinson
et al, 2011). This also raises concerns about possible
negative side effects of MPH and highlights the need for
further evaluation of both potential risks and benefits
of cognitive enhancement in the healthy population.
Moreover, future studies are needed to further unravel
how changes in reward and punishment learning might
contribute to academic performance.
ACKNOWLEDGEMENTS
RC is supported by a VIDI grant from the innovational
Incentives Scheme of the Netherlands Organization
for Scientific Research (NWO), a research grant to Kae
Nakamura, RC, and Nathaniel Daw from the Human
Frontiers Science Program (RGP0036/2009-C), and the
2012 James McDonnell Scholar Award.
DISCLOSURE
RC has been a consultant to Abbott Laboratories, but she is
not an employee or a stock shareholder. JB has been in the
past 3 years a consultant to/ member of advisory board of/
and speaker for Janssen Cilag BV, Eli Lilly, Bristol-Myer
Squibb, Shering Plough, UCB, Shire, Novartis and Servier,
but he is not an employee or a stock shareholder of any of
these companies. He has no other financial or material
support, including expert testimony, patents or royalties.
DISCLAIMER
All listed authors concur in submission and the final
manuscript has been approved by all. Experimental
procedures have been conducted in conformance with the
policies and principles contained in the Federal Policy for
the Protection of Human Subjects and in the Declaration of
Helsinki. The paper has not been and is not intended to be
published anywhere in any language except in Neuropsycho-
pharmacology. No similar paper has been, or will be,
simultaneously submitted for publication elsewhere.
REFERENCES
Agay N, Yechiam E, Carmel Z, Levkovitz Y (2010). Non-specific
effects of methylphenidate (Ritalin) on cognitive ability and
decision-making of ADHD and healthy adults. Psychopharma-
cology 210: 511–519.
Arnsten A (1997). Catecholamine regulation of the prefrontal
cortex. J Psychopharmacol 11: 151–162.
Barkley R (1997). Behavioral inhibition, sustained attention, and
executive functions: constructing a unifying theory of ADHD.
Psychol Bull 121: 65–159.
Beck A, Ward C, Mendelson M, Mock J, Erbaugh J (1961).
An inventory for measuring depression. Arch Gen Psychiatry 4:
561–632.
Berridge K, Robinson T (1998). What is the role of dopamine in
reward: hedonic impact, reward learning, or incentive salience?
Brain Res Brain Res Rev 28: 309–369.
Bodi N, Keri S, Nagy H, Moustafa A, Myers CE, Daw N et al (2009).
Reward-learning and the novelty-seeking personality: a between-
and within-subjects study of the effects of dopamine agonists on
young Parkinson’s patients. Brain 132(Pt 9): 2385–2395.
Bundt C, van Mil A, Olthof B, Reintjes W (2012). Presteren onder
druk of drugs? Een interdisciplinaire academische verkenning
naar cognitive enhancement. In: Radboud Honours Academy
(eds) The Wider Implications of Cognitive Neuroscience,
Reflections on Science 2011-2012. Radboud Honours Academy:
Nijmegen.
Clark L, Chamberlain S, Sahakian B (2009). Neurocognitive
mechanisms in depression: implications for treatment.
Annu Rev Neurosci 32: 57–74.
Clatworthy P, Lewis S, Brichard L, Hong Y, Izquierdo D, Clark L
et al (2009). Dopamine release in dissociable striatal subregions
predicts the different effects of oral methylphenidate on reversal
learning and spatial working memory. J Neurosci 29: 4690–4696.
Cools R, Altamirano L, D’Esposito M (2006). Reversal learning in
Parkinson’s disease depends on medication status and outcome
valence. Neuropsychologia 44: 1663–1673.
Cools R, D’Esposito M (2011). Inverted-u-shaped dopamine
actions on human working memory and cognitive control.
Biol Psychiatry 69: e113–e125.
Cools R, Frank MJ, Gibbs SE, Miyakawa A, Jagust W, D’Esposito M
(2009). Striatal dopamine predicts outcome-specific reversal
learning and its sensitivity to dopaminergic drug administration.
J Neurosci 29: 1538–1543.
Cools R, Gibbs SE, Miyakawa A, Jagust W, D’Esposito M (2008).
Working memory capacity predicts dopamine synthesis capacity
in the human striatum. J Neurosci 28: 1208–1212.
Daneman M, Carpenter PA (1980). Individual differences
in working memory and reading. J Verb Learn Verb Beh 19:
450–916.
DuPaul G, Power T, Anastopoulos A, Reid R (1998). ADHD Rating
Scale – IV: Checklists, Norms, and Clinical Interpretation.
Eagle D, Tufft M, Goodchild H, Robbins T (2007). Differential
effects of modafinil and methylphenidate on stop-signal reaction
time task performance in the rat, and interactions with the
dopamine receptor antagonist cis-flupenthixol. Psychopharma-
cology 192: 193–206.
Engle R, Tuholski S, Laughlin J, Conway A (1999). Working
memory, short-term memory, and general fluid intelligence: a
latent-variable approach. J Exp Psychol Gen 128: 309–331.
Effects of methylphenidate on reversal learning
ME van der Schaaf et al
2017
Neuropsychopharmacology

Faraone S, Buitelaar J (2010). Comparing the efficacy of stimulants
for ADHD in children and adolescents using meta-analysis.
Eur Child Adolesc Psychiatry 19: 353–417.
Finke K, Dodds C, Bublak P, Regenthal R, Baumann F, Manly T
et al (2010). Effects of modafinil and methylphenidate on visual
attention capacity: a TVA-based study. Psychopharmacology 210:
317–329.
Frank M, O’Reilly R (2006). A mechanistic account of striatal
dopamine function in human cognition: psychopharmacological
studies with cabergoline and haloperidol. Behav Neurosci 120:
497–1014.
Frank MJ (2005). Dynamic dopamine modulation in the basal
ganglia: a neurocomputational account of cognitive deficits in
medicated and nonmedicated Parkinsonism. J Cogn Neurosci 17:
51–72.
Frank MJ, Santamaria A, O’Reilly RC, Willcutt E (2007). Testing
computational models of dopamine and noradrenaline
dysfunction in attention deficit/hyperactivity disorder. Neurop-
sychopharmacology 32: 1583–1599.
Fusar-Poli P, Rubia K, Rossi G, Sartori G, Balottin U (2012).
Striatal dopamine transporter alterations in ADHD: pathophy-
siology or adaptation to psychostimulants? A meta-analysis. Am
J Psychiatry 169: 264–272.
Grace A (1991). Phasic versus tonic dopamine release and the
modulation of dopamine system responsivity: a hypothesis for
the etiology of schizophrenia. Neuroscience 41: 1–25.
Greely H, Sahakian B, Harris J, Kessler R, Gazzaniga M, Campbell P
et al (2008). Towards responsible use of cognitive-enhancing
drugs by the healthy. Nature 456: 702–705.
Groth-Marnat G (ed) (2001). The Wechsler Intelligence Scales.
Cambridge University Press: Cambridge.
Hooks M, Colvin A, Juncos J, Justice J (1992). Individual
differences in basal and cocaine-stimulated extracellular dopa-
mine in the nucleus accumbens using quantitative microdialysis.
Brain Res 587: 306–318.
Howell DC (1997). Statistical Methods for Psychology. Wadsworth
Publishing Company.
Kimberg DY, D’Esposito M, Farah MJ (1997). Effects of
bromocriptine on human subjects depend on working memory
capacity. Neuroreport 8: 3581–3585.
Landau SM, Lal R, O’Neil JP, Baker S, Jagust WJ (2009). Striatal
dopamine and working memory. Cereb Cortex 19: 445–454.
Maher B (2008). Poll results: look who’s doping. Nature 452: 674–675.
Maia TV, Frank MJ (2011). From reinforcement learning models
to psychiatric and neurological disorders. Nat Neurosci 14:
154–162.
Marcus S, Durkin M (2011). Stimulant adherence and academic
performance in urban youth with attention-deficit/hyperactivity
disorder. J Am Acad Child Adolesc Psychiatry 50: 480–489.
Mehta M, Goodyer I, Sahakian B (2004). Methylphenidate
improves working memory and set-shifting in AD/HD: relation-
ships to baseline memory capacity. J Child Psychol Psychiatry 45:
293–598.
Mehta M, Owen A, Sahakian B, Mavaddat N, Pickard J, Robbins T
(2000). Methylphenidate enhances working memory by
modulating discrete frontal and parietal lobe regions in the
human brain. J Neurosci 20:6.
Mehta MA, Swainson R, Ogilvie AD, Sahakian J, Robbins TW
(2001). Improved short-term spatial memory but impaired
reversal learning following the dopamine D(2) agonist bromo-
criptine in human volunteers. Psychopharmacology 159: 10–20.
Oken B, Kishiyama S, Salinsky M (1995). Pharmacologically
induced changes in arousal: effects on behavioral and electro-
physiologic measures of alertness and attention. Electroence-
phalogr Clin Neurophysiol 95: 359–371.
Patton J, Stanford M, Barratt E (1995). Factor structure of the
Barratt impulsiveness scale. J Clin Psychol 51: 768–842.
Robbins T (2002). ADHD and addiction. Nat Med 8: 24–29.
Robinson O, Cools R, Carlisi C, Sahakian B, Drevets W (2011).
Ventral striatum response during reward and punishment
reversal learning in unmedicated major depressive disorder.
Am J Psychiatry 169: 152–159.
Schmand B, Bakker D, Saan R, Louman J (1991). The Dutch
Reading Test for Adults: a measure of premorbid intelligence
level. Tijdschr Gerontol Geriatr 22: 15–24.
Schmedtje J, Oman C, Letz R, Baker E (1988). Effects of
scopolamine and dextroamphetamine on human performance.
Aviat Space Environ Med 59: 407–410.
Sheehan D, Lecrubier Y, Sheehan K, Amorim P, Janavs J, Weiller E
et al (1998). The Mini-International Neuropsychiatric Interview
(M.I.N.I.): the development and validation of a structured
diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin
Psychiatry 59(Suppl 20): 22–33.
Silber B, Croft R, Papafotiou K, Stough C (2006). The acute
effects of d-amphetamine and methamphetamine on attention
and psychomotor performance. Psychopharmacology 187:
154–169.
Smith M, Farah M (2011). Are prescription stimulants ‘‘smart
pills’’? The epidemiology and cognitive neuroscience of pre-
scription stimulant use by normal healthy individuals. Psychol
Bull 137: 717–741.
Spielberger G, Gorsuch RL, Lushene RE (1970). Manual for the
State-Trait Anxiety Inventory. Consulting Psychologists Press:
Palo Alto, CA, USA.
Swanson J, Gupta S, Lam A, Shoulson I, Lerner M, Modi N et al
(2003). Development of a new once-a-day formulation of
methylphenidate for the treatment of attention-deficit/hyper-
activity disorder: proof-of-concept and proof-of-product studies.
Arch Gen Psychiatry 60: 204–215.
van der Schaaf M, van Schouwenburg M, Geurts D, Schellekens A,
Buitelaar J, Verkes R et al (2012). Establishing the dopamine
dependency of human striatal signals during reward and
punishment reversal learning. Cereb Cortex (e-pub ahead of
print, doi:10.1093/cercor/bhs344).
Volkow N, Wang G-J, Fowler J, Logan J, Franceschi D, Maynard L
et al (2002). Relationship between blockade of dopamine
transporters by oral methylphenidate and the increases in
extracellular dopamine: therapeutic implications. Synapse 43:
181–188.
Volkow N, Wang G-J, Fowler J, Telang F, Maynard L, Logan J et al
(2004). Evidence that methylphenidate enhances the saliency of a
mathematical task by increasing dopamine in the human brain.
Am J Psychiatry 161: 1173–1180.
Volkow N, Wang G-J, Kollins S, Wigal T, Newcorn J, Telang F et al
(2009). Evaluating dopamine reward pathway in ADHD: clinical
implications. JAMA 302: 1084–1091.
Volkow N, Wang G-J, Tomasi D, Kollins S, Wigal T, Newcorn J
et al (2012). Methylphenidate-elicited dopamine increases in
ventral striatum are associated with long-term symptom
improvement in adults with attention deficit hyperactivity
disorder. J Neurosci 32: 841–850.
Voon V, Pessiglione M, Brezing C, Gallea C, Fernandez HH,
Dolan RJ et al (2010). Mechanisms underlying dopamine-
mediated reward bias in compulsive behaviors. Neuron 65:
135–142.
Wigal S, Wigal T, Schuck S, Brams M, Williamson D, Armstrong R
et al (2011). Academic, behavioral, and cognitive effects of
OROSsmethylphenidate on older children with attention-
deficit/hyperactivity disorder. J Child Adolesc Psychopharmacol
21: 121–131.
Supplementary Information accompanies the paper on the Neuropsychopharmacology website (http://www.nature.com/npp)
Effects of methylphenidate on reversal learning
ME van der Schaaf et al
2018
Neuropsychopharmacology