Modulating fluid intelligence performance through combined cognitive training and
Anna-Katharine Brema, b, 1, Jessamy Norton-Ford Almquistc, Karen Mansfielda, Franziska
Plessowb,2, Francesco Sellaa, Emiliano Santarnecchib, Umut Orhanc, James McKannad, Misha
Paveld, Santosh Mathanc, Nick Yeunga, Alvaro Pascual-Leoneb, Roi Cohen Kadosha, on
behalf of Honeywell SHARP Team authorsa,b,c,d,e
aDepartment of Experimental Psychology, University of Oxford, Oxford, United Kingdom
bBerenson-Allen Center for Noninvasive Brain Stimulation and Division for Cognitive
Neurology, Department of Neurology, Beth Israel Deaconess Medical Center, Harvard
Medical School, Boston, MA, United States
cHoneywell Labs, Honeywell Aerospace, Redmond, WA, United States
dElectrical and Computer Engineering Department, Northeastern University, Boston, MA,
eSimcoach Games, Pittsburgh PA, USA
Honeywell SHARP Team Authors: Oxford: Anna-Katharine Brem, Roi Cohen Kadosh,
Karen Mansfield, Nick Yeung; Harvard Medical School: Franziska Plessow, Emiliano
Santarnecchi, Alvaro Pascual-Leone; Honeywell: Jessamy Norton-Ford Almquist, Michael
Dillard, Umut Orhan, Santosh Mathan; Northeastern University: James McKanna, Deniz
Erdogmus, Misha Pavel; Simcoach Games: Garrett Kimball, Eben Myers
1"Present address: Max-Planck Institute of Psychiatry, Munich, Germany
2 Present address: Neuroendocrine Unit, Massachusetts General Hospital, Boston, MA, USA;
Department of Medicine, Harvard Medical School, Boston, MA, USA
Corresponding authors: Anna-Katharine Brem and Roi Cohen Kadosh
Max-Planck Institute of Psychiatry, Department of Neuropsychology, Kraepelinstrasse 2-10,
80804 Munich, Germany
Fax: +49 89 30622 605 Phone: +49 89 30622 285
E-mail address: email@example.com
Roi Cohen Kadosh
Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
Phone: +44 1865 271 385
E-mail address: firstname.lastname@example.org
Words in abstract: 247
Words in the manuscript: 4658
Number of figures: 3
Number of tables: 1
It is debated whether cognitive training of specific executive functions leads to far transfer
effects, such as improvements in fluid intelligence (Gf). Within this context, transcranial
direct current stimulation and recently also novel protocols such as transcranial random noise
and alternating current stimulation are being investigated with regards to their ability to
enhance cognitive training outcomes.
We compared the effects of four different transcranial electrical brain stimulation protocols in
combination with nine daily computerized training sessions on Gf.
82 participants were randomly assigned to receive transcranial direct current stimulation
(tDCS), random noise stimulation (tRNS), multifocal alternating current stimulation at 40 Hz
(mftACS), or multifocal tDCS (mftDCS) in combination with an adaptive and synergistic
executive function (EF) training, or to a no-contact control group. EF training consisted of
gamified tasks drawing on isolated as well as integrated executive functions (working
memory, inhibition, cognitive flexibility). Transfer was assessed with a combined measure of
Gf including three established tests (Bochumer Matrizentest - BOMAT, Raven’s Advanced
Progressive Matrices - RAPM, and Sandia Matrices). We found significant improvements in
Gf for the tDCS, mftDCS, and tRNS groups when compared with the no-contact group. In
contrast, the mftACS group did not improve significantly and showed a similar pattern as the
no-contact group. Mediation analyses indicated that the improvement in Gf was mediated
through game progression in the mftDCS and tRNS group. Electrical brain stimulation in
combination with sustained EF training can lead to transfer effects in Gf, which are mediated
by training progression.
Keywords (max 6 keywords): Transcranial electrical stimulation (tES); cognitive training;
fluid intelligence; cognitive enhancement; executive functions.
Fluid intelligence (Gf), first defined by Cattell (1963), is the ability to cope with novelty, to
think rapidly and flexibly, to see relations amongst items independent of acquired knowledge
and is predictive of important life outcomes such as income, work performance, and health
(Au et al., 2014; Sternberg, 2012). Gf is theorized to draw on neural processes that overlap
with executive functions (EFs) such as working memory, inhibition, and cognitive flexibility
(Bastian and Oberauer, 2014; Burgess et al., 2011; Colzato et al., 2006; Diamond, 2013;
Witthöft et al., 2009) as well as the broader concept of creativity (Benedek et al., 2014), and
has hence inspired researchers to explore interventions aimed at improving Gf via executive
Cognitive training of individual EFs such as response inhibition (Enge et al., 2014) or
working memory (e.g., Jaeggi et al., 2008; Redick et al., 2013; Rudebeck et al., 2012) has
yielded diverging results with regards to transfer effects to Gf in healthy subjects. Recent
meta-analyses mirror these inconsistent findings in the domain of working memory. While
some (Au et al., 2014) support far transfer effects of working memory training to Gf, others
conclude that working memory training does not “generalize to measures of “real-world”
cognitive skills” (Melby-Lervåg et al., 2016). Inconsistent findings have been suggested to
partly result from methodological issues (Shipstead et al., 2012) as well as the aggregation of
a broad range of test and training tasks. Notably, reported effect sizes for far transfer to Gf are
mostly small, which leaves the question of how to optimize cognitive training in order to
achieve greater transfer effects. One approach is the adaptation of training protocols to imitate
more life-like tasks, which commonly require the engagement of several cognitive functions
simultaneously. For example, specific EFs are rarely used singularly, but mostly in
conjunction with other cognitive functions (Taatgen, 2013). Therefore, training regimes
should ideally train multiple functions concomitantly in an effort to imitate real-life
One potential tool to enhance training outcomes and promote transfer effects is transcranial
electrical stimulation (tES). The most prominent form of tES is transcranial direct current
stimulation (tDCS), and the latest additions are transcranial alternating current (tACS) and
random noise stimulation (tRNS). These techniques do not induce neuronal firing, but rather
modulate brain excitability and task-related neuronal activity, which in turn is thought to lead
to neuroplastic changes, and hence physiological mechanisms that are similar to those
involved in learning (Cooke and Bliss, 2006; Paulus, 2004; Pelletier and Cicchetti, 2015).
Synergistic effects might therefore arise through the combination of cognitive training and
tES. A number of studies support the assumption that noninvasive brain stimulation during
cognitive training can enhance cognitive functions in healthy subjects (for an overview, see
Krause and Cohen Kadosh, 2013; Santarnecchi et al., 2015). These effects might be attributed
to stimulation of brain regions that become active during the cognitive training, and therefore
to the ability to impact the neuronal substrates of the desired cognitive function more
effectively based on the idea of state-dependent brain stimulation (Cohen Kadosh et al., 2010;
Feurra et al., 2013; Romei et al., 2016; Silvanto et al., 2007). Moreover, a recent meta-
analysis stated that stimulation effects are stronger for training than for performance
(Simonsmeier et al., 2018), and theoretical frameworks have been proposed to account for
functional enhancement as well as costs (Brem et al., 2014).
Most tES approaches so far have used two stimulation electrodes. However, to increase focus
on single brain regions as well as targeting multiple regions of a network, devices are
available that allow simultaneous stimulation of multiple brain areas with multiple electrodes
(Ruffini et al., 2014). These multifocal protocols differ from recently developed high-
definition protocols in that they are not trying to increase focality by suppressing activity in
surrounding areas of the region of interest. Furthermore, in comparison to “classic” protocols
such as the bifocal tDCS montage, they do not only focus on one brain area that is known to
be implicated in a targeted function, but try to modulate a whole network associated with it
and therefore imitate “natural” network activation. The optimization of stimulation protocols
poses a challenge in this research field. Here we have exploited these recent developments
and investigated the combined effects of different tES protocols in combination with
cognitive training on Gf.
We compared four different stimulation protocols in their efficiency in enhancing Gf
performance relative to a no-contact group (NC): bifocal tDCS (tDCS), bifocal tRNS (tRNS),
multifocal tDCS (mftDCS), and multifocal tACS (mftACS), the latter two being novel
protocols developed on the basis of functional imaging literature related to EF (see Methods).
Note that our main goal was to compare the different stimulation protocols. The tES methods
were applied in combination with a long-term (9 sessions) cognitive training approach
integrating several EFs concomitantly. We hypothesised that progression in the game would
predict Gf post, however, we did not preview a hypothesis regarding the predominance of any
of the stimulation protocols.
2. MATERIAL AND METHODS
2.1. Subjects and study design
In this randomized, controlled, single-blind study eighty-seven healthy subjects (out of
ninety-eight enrolled) completed all study procedures. Eleven subjects were excluded from
the analysis due to non-compliance with the test administration on at least two independent
measures (i.e., >2 standard deviations). The attrition rate (n =11) was similar across
stimulation groups (before intervention: n =6; during intervention: mftACS: n=1, mftDCS: n
=1, bitDCS: n =2; bitRNS: n =1). The study took place at two difference sites (University of
Oxford and Harvard). The 82 (Oxford N=45; Harvard N=37) were randomized to receive
tDCS [n =17, age 28.41±11.58; 11 males], mftDCS [n =15, age 27.88±11.58; 9 males], tRNS
[n = 16, age 29.19±10.39; 9 males], or mftACS [n = 17, age 30.73±13.17; 9 males] combined
with cognitive training. In addition we included a NC group [n = 17, age 30.88±13.30; 8
males]. A stratified randomization process taking into account age, education, and gender was
applied to control for the influence of these baseline characteristics. Subjects were
remunerated for their participation (£10 or $15 per hour in the UK and the US respectively).
Subjects with any current or past history of psychiatric illness, unstable medical condition,
epilepsy or family history of epilepsy, and active or past substance abuse considered a
potential hazard for the application of tES were precluded from participating in the study.
Subjects were instructed to sleep at least 6 hours each night, abstain from alcohol during the
entire study duration, and refrain from caffeine for 1 hour before study visits. The respective
ethics committees (Oxford: NRES Committee South Central – Berkshire; Harvard: CCI/IRB,
Beth Israel Deaconess Medical Center) approved the study and all participants gave written
informed consent prior to study onset according to the Declaration of Helsinki.
The study design is depicted in Fig. 1. We compared multiple types of stimulation protocols
with a NC group in order to assess the most effective regimen. Each training participant
received 9 cognitive training sessions on consecutive days (except weekends), each lasting 30
minutes, combined with one of the four tES protocols. Transfer effects were assessed with a
combined measure of three established Gf tests, which were assessed before and after the
training in a non-randomized approach with the following test sequence: the Bochumer
Matrizentest (BOMAT; Hossiep et al., 1999), Raven’s Advanced Progressive Matrices
(RAPM(Raven et al., 1998), and Sandia (www.sandia.gov; Matzen et al., 2010).
Subjects were pretested within 2 weeks before the start of the intervention and post-tested on
the day after the last training visit.
Figure 1. Cognitive training intervention and study design. A: Cognitive training
intervention: The intervention setup is depicted in the top middle panel. Easier cognitive
training blocks (top left panel) focus on cues for a single EF task (in this case, matching n-
back on numbers). The most difficult blocks (top right panel) contain three EF tasks; in this
case, using the background colour to determine when to switch between tasks (switching
component) involving matching n-back on number (working memory component) and
inhibiting on pictures of flying things that are not birds (inhibition component with a semantic
operator). B: Study Design: After consenting and randomization, subjects came in for the pre-
test followed by 9 sessions of combined cognitive training and stimulation and the posttest.
Subjects randomized to the no-contact control group attended the pre- and posttest sessions
- Fluid intelligence:
Cognitive training combined
- 9 sessions (30 minutes)
- 5 groups: tRNS, tDCS,
mftDCS, mftACS, NC
- Fluid intelligence:
2.2. Transcranial electrical stimulation
Current was delivered via gel-filled pi-electrodes (3.14 cm2), which were inserted into a
neoprene cap (Starstim®, Neuroelectrics, Barcelona, Spain) in accordance to the international
10-20 EEG system. In all protocols 8 electrodes were mounted (Fig. 2) and spare positions
were used to collect EEG data. Stimulation onset coincided with training onset and the
current was ramped up and down for the first and last 30 s of stimulation. Participants were
monitored at all times to ensure their safety.
TDCS and tRNS are already established stimulation protocols. The chosen parameters were
hence based on previous literature that has shown enhancement of various cognitive functions
with the specific montages used in the present study (i.e., F3-Fp2 for tDCS and F3-F4 for
tRNS), including effects on functions that are strongly related to Gf (see Figure 2 for
stimulation details and (Santarnecchi et al., 2015) for a review on tES to enhance cognition).
The amplitude of 1 mA for tRNS implies that 99% of the amplitude values were between +/-
500 µA(Terney et al., 2008). Alternating current stimulation for mftACS was sinusoidal and
set to 40 Hz as this frequency was previously shown to modulate complex problem solving
(Santarnecchi et al., 2016). The multifocal stimulation templates (mftDCS, mftACS; Fig. 2)
were newly developed for the present study through Activation Likelihood Estimation (ALE)
analysis (Eickhoff et al., 2012; Turkeltaub et al., 2012) and based on a meta-analysis of
imaging literature investigating the three major EFs engaged during FAST with the following
tasks: 1) inhibition (go-nogo task, stop-signal task), cognitive flexibility (task switching), and
working memory (n-back task, AX version of the continuous performance test). The literature
on these functions suggests a more distributed pattern of activation, usually involving both
frontal and parietal lobes bilaterally. Studies that did not report MNI/Talairach coordinates, or
provided an insufficient description of results or experimental paradigms, were excluded. An
average map of all activation foci for each EF was created and a threshold was applied using
a False Discovery Rate of .05 (cluster mm3=1000), assuming independence of foci/studies.
This average map was used as a seed-region in a following functional connectivity analysis.
As expected, a fronto-parietal network was obtained by averaging the fMRI activation maps
for each EF and provided the basis for the mftACS and mftDCS protocols (Fischer et al.,
2017). These two montages were therefore optimized to focus on bilateral frontal and parietal
regions, hereby targeting the entire executive function network, using a genetic algorithm
testing any possible combination of electrode positions in the 10-10 EEG system. For the
multifocal stimulation protocols we chose tDCS, being the most used enhancement technique,
as well as tACS in the gamma band, which was previously shown to be effective in
modulating abstract reasoning by our group (Santarnecchi et al. 2013, 2016).
mftACS (40 Hz)
F3 (anode, 703µA)
F3 (1000µA, 100-500Hz)
F3 (anode, 1250µA)
Fz (cathode, -1000µA)
Fz (850µA, 180°)
F4 (1000µA, 100-500Hz)
AF8 (cathode, 1250µA)
F4 (anode, 860µA)
T7 (cathode, -341µA)
PO7 (925µA, 180°)
T8 (cathode, -412µA)
PO8 (925µA, 180°)
P3 (anode, 603µA)
P4 (anode, 587µA)
Oz (cathode, -1000µA)
Figure 2. Stimulation parameters. In all four stimulation protocols (mftDCS, mftACS,
tDCS, tRNS) 8 electrodes were mounted and spare positions were used to collect EEG.
Stimulation onset coincided with training onset. StimWeaver (Neuroelectrics, Barcelona,
Spain) was used to optimize the stimulation montages according to the specified targets
(Ruffini et al., 2013). Electric field calculations (component of the electric field orthogonal to
the cortical surface En [V/m]) were performed using a realistic head model (Miranda et al.,
2013). Positive values indicate that the field is directed into the cortical surface. Note that the
tRNS and mftACS protocols only depict one instance of stimulation given that polarity
changes continually during stimulation.
2.3. Cognitive training
Cognitive training (Fig. 1) was presented on a laptop (screen size 17”) while subjects listened
to custom-designed music and game-specific sounds via earphones. The training consisted of
gamified tasks drawing on isolated as well as integrated EFs (working memory, inhibition,
and cognitive flexibility). The task setting was a robot factory, in which the participants were
employed. They had to work through 2-minute task blocks (15 blocks per session), which
were preceded by time-limited (30 s) instruction screens, and received feedback on every trial
as well as at the end of each block. Difficulty levels were increased whenever subjects passed
80% of a training block, which ensured an individually adapted and constant level of
challenge. Reaching a level of 80% correct in a block was considered a successful completion
of that block. When a participant reached 50-80% correct in a given block, difficulty would
remain the same, while difficulty was decreased when a subject reached <50% correct.
Subjects started with the training of single sub-functions (working memory or inhibition or
cognitive flexibility) and continued with combinations of two or all three sub-functions.
Subjects never performed the same task twice in a row. At level 1, the trials were
differentiated according to stimulus material (numbers, pictures, words, spatial locations) and
EF component (working memory, cognitive flexibility, inhibition). At level 2, the trials
involved unique pairwise combinations of all three EF components. At level 3, tasks from
Levels 1 and 2 were extended to include four logical (identity, AND and (exclusive) OR). As
an example, a task involving exclusive OR might require that participants respond only to
numbers less than 10 or to purple shapes, but not both. The most difficult training blocks (4a)
contained all three EF tasks. For example, one block might consist of two different tasks
(switching component), each involving n-back operations (working memory component), as
well as a separate cue to not respond (inhibition component). A final, slightly different
complexity level (4b) required subjects to partially deduce the rules themselves, as the
instructions intentionally obfuscated some information so that subjects had to use a trial-and-
error approach. For example, the instruction would state that the task contains an inhibition
cue, but would not state what this cue consisted of. This regime forces participants to learn
general skills rather than training-specific stimulus-response relationships. Difficulty was
furthermore increased by reducing the time available to respond, or by increasing the number
of items to remember, while changing training parameters in the opposite direction decreased
difficulty. Progress in the game was defined as the number of tasks passed of which the
participant achieved 80% or better accuracy on the most challenging difficulty level. The
development of the cognitive training was effected in collaboration with Simcoach Games
(Pittsburgh, PA, USA) and is reported elsewhere (Almquist Norton-Ford et al., in press).
2.4. Gf tests
Gf tests are thought to capture domain-independent nonverbal abilities underlying
performance on various cognitive tasks. In the current study, three of the most common tests
were administered, which specifically assess logical-deductive reasoning: BOMAT (Hossiep
et al., 1999), RAPM (Raven et al., 1998), and Sandia (www.sandia.gov; Matzen et al., 2010).
They have a common structure in that stimuli are presented as matrices of patterns in which
one pattern is missing. Subjects then have to point out the missing pattern from an array of
patterns. The BOMAT presents a 5x3 matrix with a choice of 6 possible answers, whereas the
RAPM and Sandia each present 3x3 matrices with a choice of 8 possible answers. Each test
was presented for 15 minutes and subjects were instructed to solve as many of the presented
problems as possible.
For the current experiment both the BOMAT and the RAPM were divided into 2 parallel
versions with 14 and 17 stimuli, respectively. Odd and even numbered stimuli were used for
BOMAT, while we divided the RAPM stimuli by approximating an even-odd distribution
taking into account item difficulty provided in the test manual. A similar approach has been
adopted in previous studies (Jaeggi et al., 2008; Rudebeck et al., 2012; Thompson et al.,
2013). The Sandia overcomes the issue of a limited number of stimuli by providing the option
to choose from a pool of approximately 3000 matrices, obtained through the combination of
different stimulus features such as shape, colour and orientation (Matzen et al., 2010).
Experimental matrices belong to 4 different classes based on the type and number of
analogical operations required for a correct solution (1-, 2-, 3- relations and logic matrices).
Parallel versions of the Sandia with 42 stimuli each were based on stimuli classes and
difficulty levels (Santarnecchi et al., 2013, 2016), and presentation time was limited to 1
minute per stimulus.
2.5. Statistical analyses
Data was analyzed using SPSS (21.0 for Mac) and the statistical computing software R
(2015). Given the nature of our data as a series of successes (correct responses) and failures
(incorrect responses or items not attempted), we modeled subjects’ performance on Gf tests as
a binomial process, employing a logistic regression analysis approach. In this analysis,
outcome variables were the binomial distribution of correct and incorrect responses from each
participant at posttest defined by the total number of items on the tests. We modeled subjects’
accuracy across all Gf tests at posttest, given their (1) experimental condition (mftACS, tDCS,
mftDCS, tRNS, NC), (2) baseline ability in all Gf tests, and (3) age. We used the general
linear model (GLM) function to estimate the logistic regression model. In this case, the linear
predictor was associated to the outcome with the link function logit. In the first equation the
reference category for Condition was NC. In a post-hoc analysis we used the stimulation
group with the weakest effect as the reference category to further shed light on the current
effect, and the differences between the groups.
We further clarify whether progression in the game was mediating Gf post in the different
stimulation groups by conducting a mediation analysis [bootstrapped with 10,000 samples
and 90% confidence intervals (CIs) given the directional hypothesis between progress in the
game and improved Gf] in each of the stimulation groups using the PROCESS module in
SPSS (Hayes, 2013). In order to integrate the different Gf measures, we aggregated the
standardized scores of the three Gf tests at pre- and posttest. The mediation analysis verified
whether the effect of an independent variable (X, in this case Gf pre) on a dependent variable
(Y, Gf post), denominated c path, is mediated by a mediator variable (M, game progression).
The c path corresponds to the beta regression coefficient of the linear regression with Y as
dependent variable and X as predictor. The connection between the independent variable (X)
and the mediator (M) is a denominated a path, whereas the connection between the mediator
(M) and the dependent variable (Y) is a denominated b path. The a path corresponds to the
beta coefficient of the linear regression with the mediator (M) as outcome variable and the
independent variable (X) as predictor. The b path, instead, corresponds to the beta coefficient
of the linear regression with the dependent variable (Y) as outcome and the mediator (M) as
predictor when the contribution of the independent variable (X) is controlled by including it
into the model. Therefore the mediation analysis can elegantly demonstrate a link between
several variables in our experiment. The product of the a and b paths represents the indirect
effect of the independent variable (X) on the dependent variable (Y) through the mediator (M)
and its significance is evaluated adopting a resampling technique to obtain a bootstrapped
distribution of ab products. Despite the a and b paths may be themselves statistically
significant, a robust way to establish a specific indirect effect of the independent variable (X)
on the dependent variable (Y) through the mediator (M) is to observe when the 90% CIs of
the bootstrapped distribution of the ab products does not overlap zero (Hayes, 2013; Hayes
and Scharkow, 2013). The mediation model can elucidate the role of game progression, which
was hypothesized to act as a mediator in the relationship between the predictor X (Gf pre) and
the outcome Y (Gf post). In this respect mediation analyses allow a better view of the
mechanisms of change due to the intervention (Fairchild and MacKinnon, 2009). In addition,
the mediation analysis allows us to examine several hypotheses within one analytic
framework. That is, the a path allows us to examine the link between pre Gf and training
progression, as would be predicted by current theories (Diamond, 2013), unless stimulation
might alter this expected relationship. The b path allows us to examine if progression in the
cognitive training is transferred to post Gf scores, while controlling for baseline (pre) Gf.
At baseline, groups did not differ significantly with regards to age (F(4,81)=0.21, p=.934, η2p
=.01), gender (χ2(4, N=82)=1.68, p =.795), and education (F(4,81)=0.50, p =.736, η2p =.03).
As expected, education was positively associated with baseline Gf [controlling for age]
(r=.261, p =.038, 95% CI [.03, .48]). However, education was negatively correlated with the
change in Gf [controlling for age] (r=-.327, p =.008, 95% CI [-.57, -.01]) indicating that
subjects with lower education showed a higher increase in Gf over all cognitive training
No major side effects were reported and all stimulation groups showed similar progress
(number of tasks where subjects achieved 80% or better accuracy) in the training (F(3,63)<1,
p =.818, η2p =.08).
3.1. Logistic regression
In a first model, the reference category for Condition was NC (Table 1, model 1). We found
that all stimulation protocols, except the mftACS (p=.510) protocol, improved significantly
more in Gf after the training intervention (all ps <.05).
In a second model we then chose the weakest stimulation group (mftACS) as reference
category (Table 1, model 2). This model showed that the mftACS group was performing
significantly weaker than the tDCS group (p=.029) and the mftDCS group (p=.043), and
marginally weaker than the tRNS group (p=.067). Furthermore, we found a significant
negative correlation between age and Gf scores at pretest (r=-.592, p <.001, 95% CI [-.71, -
.44]) as well as posttest (r=-.562, p <.001, 95% CI [-.70, -.39]). However, the change in Gf
score was not correlated with age (r=-.150, p =.178, 95% CI [-.40, .11]).
Model 1a (Compared to NC)
Model 2 b (Compared to mftACS)
a Gf baseline: b = .050 (SE = .005), p<.001; Age: b = -.025 (SE = .003), p<.001
b Gf baseline: b = .050 (SE = .005), p<.001; Age: b = -.025 (SE = .003), p<.001
Table 1. Logistic regression analysis of subjects’ accuracy across all Gf tests.
3.2. Mediation analyses
Mediation analyses elucidate the mechanisms that are hypothesised to underlie the
relationship between independent and dependent variables (changes from pre- to post-training
Gf scores), via the inclusion of an explanatory variable - the mediator variable (progression in
the game). As expected, regressing Gf post onto the Gf pre (c path) resulted in a significant
association between the variables in all groups (all ps<.001). Participants with a high Gf pre
score were therefore more likely to achieve a high Gf post score. We then entered the
mediator, progression in the game, into the model.
For the mftACS group (Fig. 3), Gf pre was not significantly associated with progression (p
=.125). Similarly, progression was not significantly associated with Gf post (p =.29). The
bias-corrected bootstrapped confidence interval for the specific indirect effect of the mapping
of Gf pre on Gf post through progression crossed zero (0.02, 90% CI [-.02 to .24]).
For the tDCS group (Fig. 3), Gf pre was significantly associated with progression (p <.001).
However, progression was not significantly associated with Gf post (p =.14). The bias-
corrected bootstrapped confidence interval for the specific indirect effect of the mapping of
Gf pre on Gf post through progression crossed zero (0.22, 90% CI [-.23 to .64]).
For the mftDCS group (Fig. 3), Gf pre was significantly associated with progression (p
=.045). Progression, in turn, was significantly associated with Gf post (p =.003). The bias-
corrected bootstrapped confidence interval for the specific indirect effect of the mapping of
Gf pre on Gf post through progression remained above zero (0.21, 90% CI [.02 to .46])
indicating a significant mediation effect.
For the tRNS group (Fig. 3), Gf pre was significantly associated with progression (p =.01).
Progression was also significantly associated with Gf post (p =.028). The bias-corrected
bootstrapped confidence interval for the specific indirect effect of the mapping of Gf pre on
Gf post through progression remained above zero (0.16, 90% CI [.03 to .47]) indicating,
again, a significant mediation effect.
These results support an indirect effect of Gf pre on Gf post through the progression in the
game for the groups receiving mftDCS and tRNS. Interestingly, mediation effects in the tDCS
group did not reach significance, even though coefficients were high and their pattern similar
to the mftDCS and tRNS groups. This could be explained by increased variance in the b path
in this group compared to the others.
Figure 3. Mediation analyses to examine whether the relation between pre- and posttest
Gf is mediated by game progression in each stimulation group. Standardized regression
coefficients, standard errors, and 90% confidence intervals (in parentheses), bootstrapped
with 10,000 resamples are reported. Solid lines indicate significant paths. Due to our
directional hypothesis based on previous literature (Au et al., 2014; Jaeggi et al., 2008;
Taatgen, 2013) these are one-tailed tests.
Gf PostGf Pre
=.57, SE=.22[.18, .96]
=.73, SE=.13[.49, .97]
=.28, SE=.13[.04, .52]
Gf PostGf Pre
=.22, SE=.29[-.31, .75]
=.64, SE=.14[.39, .90]
=.50, SE=.14[.24, .76]
Gf PostGf Pre
=.80, SE=.15[.53, 1.07]
=.60, SE=.24[.18, 1.03]
=.28, SE=.24[-.15, .70]
Gf PostGf Pre
=.31, SE=.25[-.14, .75]
=.86, SE=.14[.61, 1.10]
=.08, SE=.14[-.16, .32]
This study compared four tES protocols with sustained 9-session integrative EF training in
terms of transfer to Gf. Although we observed greater improvements in Gf for three protocols
(tRNS, tDCS, mftDCS) compared to NC, mftACS did not demonstrate similar benefits.
In order to elucidate underlying mechanisms, we investigated the mediating role of game
progression in Gf outcome. Notably, game progression only mediated Gf outcome in the
groups that received mftDCS or tRNS; though the tDCS group showed a similar pattern with
similar effect size.
This study did not include a sham group or an active control group, which could have
elucidated the specific contribution of cognitive training alone, which was not the aim of the
current study, but rather to compare the differential effects of the four stimulation protocols.
However, it is known that cognitive training alone shows small transfer effects to Gf (Jaeggi
et al., 2011). Given that transfer effects were observed for the tRNS and mftDCS groups, we
doubt that such transfer effects might be epiphenomena, induced by other factors, such as
motivation, remuneration, or the so-called ‘Hawthorne effect’ (the interaction with the
experimenters). Previous studies have already revealed long-term transfer with tRNS
(Cappelletti et al., 2013; Snowball et al., 2013). Using a similar stimulation protocol,
Snowball and colleagues (Snowball et al., 2013) demonstrated that behavioural improvements
were associated with hemodynamic responses specifically within the left DLPFC, although
the exact mechanisms of tRNS-related effects in the current study might differ.
As the mftDCS protocol is a novel approach, we can only speculate about the underlying
mechanisms. We suspect that simultaneous stimulation of brain areas that are relevant for the
trained EF, specifically frontal and parietal areas, buttressed any advantageous effects.
Electric field modelling was used to ensure that indeed the intended areas were stimulated
whilst reducing shunting effects due to mounting several (though small) electrodes in
relatively close proximity. The application of only one frequency (40 Hz) of alternating
currents over several brain regions in our mftACS protocol, however, may not be sufficient to
account for the complex interplay of oscillatory activity in these brain areas and possibly even
had a detrimental effect on training outcome. It is premature, based on one stimulation
protocol, to conclude that mftACS cannot be effective for cognitive learning and transfer, and
future studies will need to further examine the effect of alternative montages, different timing
of stimulation with regard to the training (online vs. offline), frequencies, or amplitudes on
behavioural outcomes. Importantly, even though the present montage was optimized to
homogeneously distribute current intensity across the electrode array, the phase of oscillatory
stimulation was not completely balanced due to physical constraints induced by the proximity
of some electrodes. Given the relevance of phase information for tACS applications (Polanía
et al., 2012), this might have constituted an undesired source of noise and should be carefully
addressed in future investigations.
It should be noted that multifocal solutions allow for stimulation of a larger number of
regions, approximating stimulation of entire cortical networks. However, given the limitation
in overall stimulation intensity imposed by tES safety guidelines, solutions including more
than two electrodes usually result in a more distributed but lower stimulation intensity, which
might in turn result in a sub-threshold stimulation. This could apply to both the mftDCS and
the mftACS protocol. Moreover, given that mftACS effects might be driven by both local
entrainment of specific oscillatory activity as well as by synchronization of different areas via
in-phase stimulation (i.e., 0 degrees phase difference between stimulation electrodes), the
mftACS protocol used in the present study might have elicited desynchronization of the
targeted fronto-parietal network, thus inducing segregation of activity in frontal and parietal
Finally, age was negatively correlated with pretest as well as posttest Gf, but the change in Gf
was independent of age. Previous studies in young healthy adults have shown that subjects
with lower baseline abilities may profit more from brain stimulation interventions aiming to
improve cognitive functions (Foroughi et al., 2014; Hsu et al., 2014; Liang et al., 2014; Looi
et al., 2016; Tseng et al., 2012). Lower functioning subjects should have a higher margin for
improvement, as there is room for optimization of cognitive processes, while higher
functioning subjects already perform at an optimal physiological level, which might prevent
further improvement (Brem et al., 2014; Hsu et al., 2014; Krause et al., 2013). However, with
advancing age, functional and structural changes might, in return, modify underlying
mechanisms and lead to differential outcomes (Berryhill and Jones, 2012; Learmonth et al.,
2015; Li et al., 2015). Recent research suggests that individual traits and momentary state
predict behavioural outcomes after stimulation (Benwell et al., 2015; London and Slagter,
2015; Sarkar et al., 2014). Future studies should address the question of whether cognitive
training combined with brain stimulation might lead to an assimilation of high and low
functioning individuals by targeting low functioning individuals more effectively.
In sum, after nine sessions of brain stimulation combined with cognitive training we found
transfer effects to Gf measures for the mftDCS and tRNS groups. Future studies should
explore whether individualized intervention protocols based on trait (e.g., age, education,
genetic polymorphisms) and momentary state can enhance individual training effects.
This research was supported by the Intelligence Advanced Research Projects Activity
(IARPA) via contract #2014413121700007. The United States Government is authorized to
reproduce and distribute reprints for Governmental purposes notwithstanding any copyright
annotation thereon. The funders had no role in study design, data collection and analysis,
decision to publish, or preparation of the manuscript. The views and conclusions contained
herein are those of the authors and should not be interpreted as necessarily representing the
official policies or endorsements, either expressed or implied, of IARPA, the Office of the
Director of National Intelligence, or the U.S. Government.
CONFLICT OF INTEREST STATEMENT
RCK serves on the scientific advisory boards for InnoSphere Inc., The Cognitive
Enhancement Foundation (non-profit organization), and Neuroelectrics Inc. APL serves on
the scientific advisory boards for Nexstim, Neuronix, Starlab Neuroscience, Neuroelectrics
Inc., Axilum Robotics, Magstim Inc., and Neosync; and is listed as an inventor on several
issued and pending patents on the real-time integration of transcranial magnetic stimulation
with electroencephalography and magnetic resonance imaging. The authors declare no
competing interests and do not hold any interests in the training software or Simcoach Games
beyond scientific interest.
We would like to thank Giulia Maistrello, Amar Sarkar, Ruairidh Battleday, and Kalaiyashni
Puvanendran from University of Oxford; Dennis Cornhill, Sandi White, Zane Thimmesch-
Gill, and Forrest Olsen from Honeywell Aerospace; Erica Levenbaum and Alexandra
Emmendorfer from Harvard Medical School; Asieh Ahani, Sadegh Salehi, Yeganeh Marghi,
and Seyhmus Güler from Northeastern University; Adam Chizmar, Michael Wagner, and
Jessica Trybus from Simcoach Games for their valuable help.
Almquist Norton-Ford, J., Brem, A.-K., Plessow, F., McKanna, J., Santarnecchi, E.,
Pascual-Leone, A., Cohen Kadosh, R., Pavel, M., Yeung, N., n.d. FAST: A
novel, executive function-based approach to cognitive enhancement. PLoS
Au, J., Sheehan, E., Tsai, N., Duncan, G.J., Buschkuehl, M., Jaeggi, S.M., 2014.
Improving fluid intelligence with training on working memory: a meta-
analysis. Psychon. Bull. Rev. https://doi.org/10.3758/s13423-014-0699-x
Bastian, C.C. von, Oberauer, K., 2014. Effects and mechanisms of working memory
training: a review. Psychol. Res. 78, 803–820. https://doi.org/10.1007/s00426-
Benedek, M., Jauk, E., Sommer, M., Arendasy, M., Neubauer, A.C., 2014.
Intelligence, creativity, and cognitive control: The common and differential
involvement of executive functions in intelligence and creativity. Intelligence
46, 73–83. https://doi.org/10.1016/j.intell.2014.05.007
Benwell, C.S.Y., Learmonth, G., Miniussi, C., Harvey, M., Thut, G., 2015. Non-linear
effects of transcranial direct current stimulation as a function of individual
baseline performance: Evidence from biparietal tDCS influence on lateralized
attention bias. Cortex J. Devoted Study Nerv. Syst. Behav. 69, 152–165.
Berryhill, M.E., Jones, K.T., 2012. tDCS selectively improves working memory in
older adults with more education. Neurosci. Lett. 521, 148–151.
Brem, A.-K., Fried, P.J., Horvath, J.C., Robertson, E.M., Pascual-Leone, A., 2014. Is
neuroenhancement by noninvasive brain stimulation a net zero-sum
proposition? NeuroImage 85 Pt 3, 1058–1068.
Burgess, G.C., Gray, J.R., Conway, A.R.A., Braver, T.S., 2011. Neural mechanisms
of interference control underlie the relationship between fluid intelligence and
working memory span. J. Exp. Psychol. Gen. 140, 674–692.
Cappelletti, M., Gessaroli, E., Hithersay, R., Mitolo, M., Didino, D., Kanai, R., Cohen
Kadosh, R., Walsh, V., 2013. Transfer of Cognitive Training across
Magnitude Dimensions Achieved with Concurrent Brain Stimulation of the
Parietal Lobe. J. Neurosci. 33, 14899–14907.
Cattell, R.B., 1963. Theory of fluid and crystallized intelligence: A critical
experiment. J. Educ. Psychol. 54, 1–22. https://doi.org/10.1037/h0046743
Cohen Kadosh, R., Soskic, S., Iuculano, T., Kanai, R., Walsh, V., 2010. Modulating
neuronal activity produces specific and long-lasting changes in numerical
competence. Curr. Biol. CB 20, 2016–2020.
Colzato, L.S., van Wouwe, N.C., Lavender, T.J., Hommel, B., 2006. Intelligence and
cognitive flexibility: fluid intelligence correlates with feature “unbinding”
across perception and action. Psychon. Bull. Rev. 13, 1043–1048.
Cooke, S.F., Bliss, T.V.P., 2006. Plasticity in the human central nervous system.
Brain J. Neurol. 129, 1659–1673. https://doi.org/10.1093/brain/awl082
Diamond, A., 2013. Executive functions. Annu. Rev. Psychol. 64, 135–168.
Eickhoff, S.B., Bzdok, D., Laird, A.R., Kurth, F., Fox, P.T., 2012. Activation
likelihood estimation meta-analysis revisited. NeuroImage 59, 2349–2361.
Enge, S., Behnke, A., Fleischhauer, M., Küttler, L., Kliegel, M., Strobel, A., 2014. No
evidence for true training and transfer effects after inhibitory control training
in young healthy adults. J. Exp. Psychol. Learn. Mem. Cogn. 40, 987–1001.
Fairchild, A.J., MacKinnon, D.P., 2009. A general model for testing mediation and
moderation effects. Prev. Sci. Off. J. Soc. Prev. Res. 10, 87–99.
Feurra, M., Pasqualetti, P., Bianco, G., Santarnecchi, E., Rossi, A., Rossi, S., 2013.
State-dependent effects of transcranial oscillatory currents on the motor
system: what you think matters. J. Neurosci. Off. J. Soc. Neurosci. 33, 17483–
Fischer, D.B., Fried, P.J., Ruffini, G., Ripolles, O., Salvador, R., Banus, J.,
Ketchabaw, W.T., Santarnecchi, E., Pascual-Leone, A., Fox, M.D., 2017.
Multifocal tDCS targeting the resting state motor network increases cortical
excitability beyond traditional tDCS targeting unilateral motor cortex.
NeuroImage 157, 34–44. https://doi.org/10.1016/j.neuroimage.2017.05.060
Foroughi, C.K., Blumberg, E.J., Parasuraman, R., 2014. Activation and inhibition of
posterior parietal cortex have bi-directional effects on spatial errors following
interruptions. Front. Syst. Neurosci. 8, 245.
Hayes, A.F., 2013. Introduction to mediation, moderation, and conditional process
analysis: A regression-based approach. Guilford Press.
Hayes, A.F., Scharkow, M., 2013. The relative trustworthiness of inferential tests of
the indirect effect in statistical mediation analysis: does method really matter?
Psychol. Sci. 24, 1918–1927. https://doi.org/10.1177/0956797613480187
Hossiep, R., Turck, D., Hasella, M., 1999. Bochumer Matrizentest: BOMAT–
Advanced–Short Version. Hogrefe, Göttingen.
Hsu, T.-Y., Tseng, P., Liang, W.-K., Cheng, S.-K., Juan, C.-H., 2014. Transcranial
direct current stimulation over right posterior parietal cortex changes
prestimulus alpha oscillation in visual short-term memory task. NeuroImage
98, 306–313. https://doi.org/10.1016/j.neuroimage.2014.04.069
Jaeggi, S.M., Buschkuehl, M., Jonides, J., Perrig, W.J., 2008. Improving fluid
intelligence with training on working memory. Proc. Natl. Acad. Sci. U. S. A.
105, 6829–6833. https://doi.org/10.1073/pnas.0801268105
Jaeggi, S.M., Buschkuehl, M., Jonides, J., Shah, P., 2011. Short- and long-term
benefits of cognitive training. Proc. Natl. Acad. Sci. U. S. A. 108, 10081–
Krause, B., Cohen Kadosh, R., 2013. Can transcranial electrical stimulation improve
learning difficulties in atypical brain development? A future possibility for
cognitive training. Dev. Cogn. Neurosci. 6, 176–194.
Krause, B., Márquez-Ruiz, J., Cohen Kadosh, R., 2013. The effect of transcranial
direct current stimulation: a role for cortical excitation/inhibition balance?
Front. Hum. Neurosci. 7, 602. https://doi.org/10.3389/fnhum.2013.00602
Learmonth, G., Thut, G., Benwell, C.S.Y., Harvey, M., 2015. The implications of
state-dependent tDCS effects in aging: Behavioural response is determined by
baseline performance. Neuropsychologia 74, 108–119.
Li, L.M., Uehara, K., Hanakawa, T., 2015. The contribution of interindividual factors
to variability of response in transcranial direct current stimulation studies.
Front. Cell. Neurosci. 9, 181. https://doi.org/10.3389/fncel.2015.00181
Liang, W.-K., Lo, M.-T., Yang, A.C., Peng, C.-K., Cheng, S.-K., Tseng, P., Juan, C.-
H., 2014. Revealing the brain’s adaptability and the transcranial direct current
stimulation facilitating effect in inhibitory control by multiscale entropy.
NeuroImage 90, 218–234. https://doi.org/10.1016/j.neuroimage.2013.12.048
London, R.E., Slagter, H.A., 2015. Effects of Transcranial Direct Current Stimulation
over Left Dorsolateral pFC on the Attentional Blink Depend on Individual
Baseline Performance. J. Cogn. Neurosci. 27, 2382–2393.
Looi, C.Y., Duta, M., Brem, A.-K., Huber, S., Nuerk, H.-C., Cohen Kadosh, R., 2016.
Combining brain stimulation and video game to promote long-term transfer of
learning and cognitive enhancement. Sci. Rep. 6, 22003.
Matzen, L.E., Benz, Z.O., Dixon, K.R., Posey, J., Kroger, J.K., Speed, A.E., 2010.
Recreating Raven’s: software for systematically generating large numbers of
Raven-like matrix problems with normed properties. Behav. Res. Methods 42,
Melby-Lervåg, M., Redick, T.S., Hulme, C., 2016. Working Memory Training Does
Not Improve Performance on Measures of Intelligence or Other Measures of
“Far Transfer”: Evidence From a Meta-Analytic Review. Perspect. Psychol.
Sci. J. Assoc. Psychol. Sci. 11, 512–534.
Miranda, P.C., Mekonnen, A., Salvador, R., Ruffini, G., 2013. The electric field in the
cortex during transcranial current stimulation. NeuroImage 70, 48–58.
Paulus, W., 2004. Outlasting excitability shifts induced by direct current stimulation
of the human brain. Suppl. Clin. Neurophysiol. 57, 708–714.
Pelletier, S.J., Cicchetti, F., 2015. Cellular and molecular mechanisms of action of
transcranial direct current stimulation: evidence from in vitro and in vivo
models. Int. J. Neuropsychopharmacol. Off. Sci. J. Coll. Int.
Neuropsychopharmacol. CINP 18. https://doi.org/10.1093/ijnp/pyu047
Polanía, R., Nitsche, M.A., Korman, C., Batsikadze, G., Paulus, W., 2012. The
importance of timing in segregated theta phase-coupling for cognitive
performance. Curr. Biol. CB 22, 1314–1318.
Raven, J.C., Raven, J., Court, J.H., 1998. Manual for Raven’s progressive matrices
and vocabulary scales. Section 4: Advanced progressive matrices. Pearson,
San Antonio, TX.
Redick, T.S., Shipstead, Z., Harrison, T.L., Hicks, K.L., Fried, D.E., Hambrick, D.Z.,
Kane, M.J., Engle, R.W., 2013. No evidence of intelligence improvement after
working memory training: a randomized, placebo-controlled study. J. Exp.
Psychol. Gen. 142, 359–379. https://doi.org/10.1037/a0029082
Romei, V., Thut, G., Silvanto, J., 2016. Information-Based Approaches of
Noninvasive Transcranial Brain Stimulation. Trends Neurosci. 39, 782–795.
Rudebeck, S.R., Bor, D., Ormond, A., O’Reilly, J.X., Lee, A.C.H., 2012. A potential
spatial working memory training task to improve both episodic memory and
fluid intelligence. PloS One 7, e50431.
Ruffini, G., Fox, M.D., Ripolles, O., Miranda, P.C., Pascual-Leone, A., 2014.
Optimization of multifocal transcranial current stimulation for weighted
cortical pattern targeting from realistic modeling of electric fields.
NeuroImage 89, 216–225. https://doi.org/10.1016/j.neuroimage.2013.12.002
Ruffini, G., Wendling, F., Merlet, I., Molaee-Ardekani, B., Mekonnen, A., Salvador,
R., Soria-Frisch, A., Grau, C., Dunne, S., Miranda, P.C., 2013. Transcranial
current brain stimulation (tCS): models and technologies. IEEE Trans. Neural
Syst. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. 21, 333–345.
Santarnecchi, E., Brem, A.-K., Levenbaum, E., Thompson, T., Kadosh, R.C., Pascual-
Leone, A., 2015. Enhancing cognition using transcranial electrical stimulation.
Curr. Opin. Behav. Sci. 4, 171–178.
Santarnecchi, E., Muller, T., Rossi, S., Sarkar, A., Polizzotto, N.R., Rossi, A., Cohen
Kadosh, R., 2016. Individual differences and specificity of prefrontal gamma
frequency-tACS on fluid intelligence capabilities. Cortex J. Devoted Study
Nerv. Syst. Behav. 75, 33–43. https://doi.org/10.1016/j.cortex.2015.11.003
Santarnecchi, E., Polizzotto, N.R., Godone, M., Giovannelli, F., Feurra, M., Matzen,
L., Rossi, A., Rossi, S., 2013. Frequency-dependent enhancement of fluid
intelligence induced by transcranial oscillatory potentials. Curr. Biol. CB 23,
Sarkar, A., Dowker, A., Cohen Kadosh, R., 2014. Cognitive enhancement or
cognitive cost: trait-specific outcomes of brain stimulation in the case of
mathematics anxiety. J. Neurosci. Off. J. Soc. Neurosci. 34, 16605–16610.
Shipstead, Z., Redick, T.S., Engle, R.W., 2012. Is working memory training
effective? Psychol. Bull. 138, 628–654. https://doi.org/10.1037/a0027473
Silvanto, J., Muggleton, N.G., Cowey, A., Walsh, V., 2007. Neural adaptation reveals
state-dependent effects of transcranial magnetic stimulation. Eur. J. Neurosci.
25, 1874–1881. https://doi.org/10.1111/j.1460-9568.2007.05440.x
Simonsmeier, B.A., Grabner, R.H., Hein, J., Krenz, U., Schneider, M., 2018.
Electrical brain stimulation (tES) improves learning more than performance: A
meta-analysis. Neurosci. Biobehav. Rev. 84, 171–181.
Snowball, A., Tachtsidis, I., Popescu, T., Thompson, J., Delazer, M., Zamarian, L.,
Zhu, T., Cohen Kadosh, R., 2013. Long-Term Enhancement of Brain Function
and Cognition Using Cognitive Training and Brain Stimulation. Curr. Biol.
23, 987–992. https://doi.org/10.1016/j.cub.2013.04.045
Sternberg, R.J., 2012. Intelligence. Dialogues Clin. Neurosci. 14, 19–27.
Taatgen, N.A., 2013. The nature and transfer of cognitive skills. Psychol. Rev. 120,
Terney, D., Chaieb, L., Moliadze, V., Antal, A., Paulus, W., 2008. Increasing human
brain excitability by transcranial high-frequency random noise stimulation. J.
Neurosci. Off. J. Soc. Neurosci. 28, 14147–14155.
Thompson, T.W., Waskom, M.L., Garel, K.-L.A., Cardenas-Iniguez, C., Reynolds,
G.O., Winter, R., Chang, P., Pollard, K., Lala, N., Alvarez, G.A., Gabrieli,
J.D.E., 2013. Failure of working memory training to enhance cognition or
intelligence. PloS One 8, e63614.
Tseng, P., Hsu, T.-Y., Chang, C.-F., Tzeng, O.J.L., Hung, D.L., Muggleton, N.G.,
Walsh, V., Liang, W.-K., Cheng, S., Juan, C.-H., 2012. Unleashing potential:
transcranial direct current stimulation over the right posterior parietal cortex
improves change detection in low-performing individuals. J. Neurosci. Off. J.
Soc. Neurosci. 32, 10554–10561. https://doi.org/10.1523/JNEUROSCI.0362-
Turkeltaub, P.E., Eickhoff, S.B., Laird, A.R., Fox, M., Wiener, M., Fox, P., 2012.
Minimizing within-experiment and within-group effects in Activation
Likelihood Estimation meta-analyses. Hum. Brain Mapp. 33, 1–13.
Witthöft, M., Sander, N., Süss, H.-M., Wittmann, W.W., 2009. Adult age differences
in inhibitory processes and their predictive validity for fluid intelligence.
Neuropsychol. Dev. Cogn. B Aging Neuropsychol. Cogn. 16, 133–163.