ArticlePDF Available

Abstract and Figures

Generation of hierarchical structures, such as the embedding of subordinate elements into larger structures, is a core feature of human cognition. Processing of hierarchies is thought to rely on lateral prefrontal cortex (PFC). However, the neural underpinnings supporting active generation of new hierarchical levels remain poorly understood. Here, we created a new motor paradigm to isolate this active generative process by means of fMRI. Participants planned and executed identical movement sequences by using different rules: a Recursive hierarchical embedding rule, generating new hierarchical levels; an Iterative rule linearly adding items to existing hierarchical levels, without generating new levels; and a Repetition condition tapping into short term memory, without a transformation rule. We found that planning involving generation of new hierarchical levels (Recursive condition vs. both Iterative and Repetition) activated a bilateral motor imagery network, including cortical and subcortical structures. No evidence was found for lateral PFC involvement in the generation of new hierarchical levels. Activity in basal ganglia persisted through execution of the motor sequences in the contrast Recursive versus Iteration, but also Repetition versus Iteration, suggesting a role of these structures in motor short term memory. These results showed that the motor network is involved in the generation of new hierarchical levels during motor sequence planning, while lateral PFC activity was neither robust nor specific. We hypothesize that lateral PFC might be important to parse hierarchical sequences in a multi‐domain fashion but not to generate new hierarchical levels.
Content may be subject to copyright.
1
1
Recursion in Action: An fMRI study on the Generation of new Hierarchical
2
Levels in Motor Sequences
3
4
Abbreviated title: An fMRI study on motor hierarchy generation
5
6
Mauricio J.D. Martins1,2,3,*, Roberta Bianco2,4,*, Daniela Sammler2, Arno Villringer1,2,3
7
8
1Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
9
2Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
10
3Clinic for Cognitive Neurology, University Hospital Leipzig, Germany
11
4UCL Ear Institute, University College London, London, UK
12
*These authors contributed equally to this work
13
Correspondence should be addressed to:
14
Mauricio Dias Martins
15
Berlin School of Mind and Brain, Humboldt Universität zu Berlin
16
Luisenstrasse 56, 10117 Berlin, Germany
17
Telephone: +4915112090828
18
Email: diasmarm@hu-berlin.de
19
Number of figures: 7
20
Number of tables: 2
21
2
Abstract
22
Generation of hierarchical structures, such as the embedding of subordinate elements into
23
larger structures, is a core feature of human cognition. Processing of hierarchies is thought to
24
rely on lateral prefrontal cortex (PFC). However, the neural underpinnings supporting active
25
generation of new hierarchical levels remain poorly understood. Here, we created a new motor
26
paradigm to isolate this active generative process by means of fMRI. Participants planned and
27
executed identical movement sequences by using different rules: a ‘recursive’ hierarchical
28
embedding rule, generating new hierarchical levels; an ‘iterative’ rule linearly adding items to
29
existing hierarchical levels, without generating new levels; and a ‘repetition’ condition tapping
30
into short term memory, without a transformation rule. We found that planning involving
31
generation of new hierarchical levels (‘recursive’ condition vs. both ‘iterative’ and ‘repetition’)
32
activated a bilateral motor imagery network, including cortical and subcortical structures. No
33
evidence was found for lateral PFC involvement in the generation of new hierarchical levels.
34
Activity in basal ganglia persisted through execution of the motor sequences in the contrast
35
‘recursive’ vs. ‘iteration’, but also ‘repetition’ vs ‘iteration’, suggesting a role of these
36
structures in motor short term memory. These results showed that the motor network is
37
involved in the generation of new hierarchical levels during motor sequence planning, while
38
lateral PFC activity was neither robust nor specific. We hypothesize that lateral PFC might be
39
important to parse hierarchical sequences in a multi-domain fashion but not to generate new
40
hierarchical levels.
41
Keywords: fMRI, Prefrontal Cortex, Hierarchy, Motor, Recursion
42
43
3
Significance Statement
44
Processing of hierarchical structures often activates lateral PFC, across several domains.
45
However, it remains unclear whether this region supports the generation of new hierarchical
46
levels, or other peripheral mechanisms supporting structure encoding and externalization
47
components, such as motor execution. Using fMRI, here, we isolated (i) generative processes
48
by inspecting the planning of identical motor sequences based on different (hierarchical and
49
non-hierarchical) rules and (ii) externalization processes by inspecting their execution. The
50
generation of motor hierarchies via application of recursive hierarchical embedding rules was
51
supported by a neural system used for motor imagery and planning. No evidence was found for
52
lateral PFC involvement in the generation of new hierarchical levels. While lateral PFC might
53
be important to parse hierarchical sequences in a multi-domain fashion, it might not be
54
necessary to generate new hierarchical levels.
55
56
57
4
Introduction
58
Much of what differentiates human behaviour from that of other species is related to an
59
increased ability to represent and generate complex hierarchies (Conway & Christiansen, 2001;
60
Dehaene, Meyniel, Wacongne, Wang, & Pallier, 2015; Everaert, Huybregts, Chomsky,
61
Berwick, & Bolhuis, 2015; Fitch, Friederici, & Hagoort, 2012). This is evident across several
62
domains, including language (Chomsky, 1957) and music (Jackendoff & Lerdahl, 2006;
63
Lerdahl & Jackendoff, 1977), but also actions (Fitch & Martins, 2014; Lashley, 1951). While
64
animal behaviour is for the most part serially structured, i.e., can be described as a sequence of
65
movements in which each element (e.g., gesture) is maximally connected to one preceding and
66
one successive element, humans can organize movements into superordinate clusters (Koechlin
67
& Jubault, 2006), entailing several processing advantages. Humans, like other animals, use
68
serial representations, for example, when we learn a motor sequence (e.g., a dance) involving
69
multiple steps: A B C D. It frequently occurs that, when one step is forgotten (say C),
70
we need to repeat the sequence from the beginning, because we are incapable of transitioning
71
from B to D. This is because each node in the sequence is connected only to its adjacent
72
successor but not beyond (Udden, Martins, Fitch, & Zuidema, n.d.). Hierarchy overcomes this
73
limitation because individual behavioural units can be connected to more than one successor
74
(or in hierarchical terminology, each parent node can be connected to more than one child).
75
This allows elements in a motor sequence to be connected via a tree-like structure containing
76
non-expressed hidden nodes. For instance, take a procedure containing 6 steps A B C
77
D E F where two steps always form a structural unit [X[A,B], Y[C,D], Z[E,F]]. In a
78
coffee-making procedure, X could be adding water to the machine (A. taking the pot out and
79
B. pour water in it), Y adding coffee to the machine (C. get the coffee out and D. put coffee in
80
the filter), and Z operating the machine (E. turn it on and F. turn it off). When we already put
81
coffee in the filter the day before, we can easily transition from B to E because beyond the
82
5
simple serial relations between steps (from A to F) we also represent the higher-level structural
83
units containing these steps (X, Y, Z) and the connections between these units. This capacity
84
to represent hierarchies seems to be particularly well-developed in humans, who can generate
85
an unbounded number of hierarchical levels (Hauser, Chomsky, & Fitch, 2002) in a number of
86
different domains.
87
Numerous studies in the domains of language, music, and vision have investigated the
88
discrimination of hierarchical structures by asking participants to evaluate whether sequences
89
of items are well-formed according to a previously learned system of rules (Bahlmann,
90
Schubotz, & Friederici, 2008; Bahlmann, Schubotz, Mueller, Koester, & Friederici, 2009;
91
Bianco et al., 2016; Friederici, Bahlmann, Friedrich, & Makuuchi, 2011; Maess, Koelsch,
92
Gunter, & Friederici, 2001; Sammler, Koelsch, & Friederici, 2011). These studies suggest that
93
lateral prefrontal cortex (PFC, particularly Inferior Frontal Gyrus, IFG) might contribute multi-
94
domain resources to the processing of hierarchies (Fadiga, Craighero, & D’Ausilio, 2009;
95
Friederici et al., 2011; Patel, 2003), in interaction with areas along ventral visual/auditory
96
streams that may store domain-specific schematic information (Bianco et al., 2016; Martins et
97
al., 2014; Oechslin, Gschwind, & James, 2017; Pallier, Devauchelle, & Dehaene, 2011;
98
Sammler et al., 2013). Recent anatomical research has shown that IFG is humans is expanded
99
(Schenker et al., 2010) and more strongly connected with the posterior temporal regions in
100
comparison to other primates (Neubert, Mars, Thomas, Sallet, & Rushworth, 2014; Rilling et
101
al., 2008). If IFG would support the generation of hierarchies, these anatomical differences
102
would explain why this ability is enhanced in humans. However, activity in this region across
103
domains has also been argued to reflect general processes such as cognitive control and
104
working memory, which might assist (but not be central to) hierarchical generativity
105
(Fedorenko, Duncan, & Kanwisher, 2012; Matchin, Hammerly, & Lau, 2017; Novick,
106
Trueswell, & Thompson-Schill, 2005)
107
6
In the present study, we went beyond discrimination and investigated the generation of
108
hierarchically organised behaviours and their neurocognitive bases in the motor domain, where
109
the generation of structures can be explicitly measured at the behavioural level. Previous
110
research mostly relied on comparing classes of stimuli (hierarchical vs. non-hierarchical) that
111
differ in their underlying generative rule system but also in their surface serial structure. This
112
hampers the ability to separate processes involved in hierarchy generation from those involved
113
in stimulus peripheral encoding. To solve this issue, our novel motor design experimentally
114
separated generative act from execution: We asked participants to execute identical motor
115
sequences, hence having identical surface structure, but to generate these sequences using
116
different rules.
117
Crucially, the identical surface structure of the motor sequences could be represented
118
as either being hierarchical or not. This enabled us to focus on the generative process, i.e., in
119
how far participants specifically represented a rule that generated new hierarchical levels vs. a
120
flat rule that merely transformed preformed hierarchies without generating new levels, and vs.
121
a mere repetition procedure without a transformation rule. As first rule, we used a recursive
122
hierarchical embedding rule in which participants were required to generate connections
123
between a parent node and a set of 3 children nodes (such as X X[A1, A2, A3]), thus
124
generating a new hierarchical level. In the second rule (Iteration), successor elements were
125
added serially, without generating new levels (A1 A1-A2 A1-A2-A3). Hence, these trials
126
could be solved without representing cross-level relations. The third rule simply required to
127
repeat the full sequence presented in a previous step condition, hence tapping into short term
128
memory. The respective rule was revealed to the participants in a series of initial generative
129
steps that they had to generalize to complete the trial with the correct motor sequence. We
130
compared BOLD activity elicited by the use of recursive hierarchical embedding vs. non-
131
7
hierarchical iterative rules vs. repetition separately during planning and execution of these
132
motor sequences.
133
So far, behavioural and neural markers of action production have been extensively
134
studied in contexts of (i) fast production of simple movement sequences without hierarchical
135
relations (Elsinger, Harrington, & Rao, 2006; Hardwick, Rottschy, Miall, & Eickhoff, 2013;
136
Hétu et al., 2013) and (ii) representation of hierarchical relations within action sequences but
137
without active generation of new hierarchical levels (Fazio et al., 2009; Koechlin & Jubault,
138
2006).
139
The first stream of research demonstrated that learning to execute simple motor
140
sequences is supported by a network including premotor and motor cortices (PMC and M1),
141
superior parietal lobe (SPL), supplementary motor area (SMA), pre-SMA, left thalamus and
142
right cerebellum (Hardwick et al., 2013). Furthermore, “planning” (keeping a given motor
143
sequence in short-term memory) recruited a bilateral network comprising sensorimotor (M1S1)
144
and premotor cortices, cerebellum and basal ganglia (Boecker, Jankowski, Ditter, & Scheef,
145
2008; Elsinger et al., 2006).
146
The second stream of research has focused on the neural bases of hierarchical action
147
processing (when acts are encoded as parts of higher-order actions) (Fazio et al., 2009;
148
Koechlin & Jubault, 2006). These studies typically compare higher vs. lower levels of given
149
hierarchical action structures and have revealed posterior-to-anterior activation gradients along
150
lateral PFC with increasing hierarchy level. For instance, sequences of simple movements (left
151
and right button presses) activate more anterior regions in lateral PFC when the movements are
152
organized in superordinate clusters compared to when they are un-clustered (Koechlin &
153
Jubault, 2006).
154
Overall, these studies provide neural evidence for (i) generative capacity but restricted
155
to linear motor sequences and (ii) hierarchical processing of actions, but without active
156
8
generation of new hierarchical levels. Here, we combined these two approaches to go a step
157
further and probe the mechanisms underlying the internally driven generation of hierarchically
158
organised motor sequences.
159
Previous research on hierarchical representations in music, language or goal-directed
160
actions (Fadiga et al., 2009; Fitch & Martins, 2014) points to the lateral PFC and IFG as a
161
candidate region that may also be involved in the generation of new hierarchical levels in motor
162
sequences, even though these studies did not isolate the generative act from the processing of
163
given hierarchies. On the other hand, planning and imagery of complex motor sequences
164
(independently of execution) are known to be supported by a motor network comprising M1S1,
165
PMC, Basal ganglia and cerebellum. In this study we will test whether the latter areas also
166
support the generation of motor hierarchies, or whether this capacity requires dedicated systems
167
in PFC hypothesized to support generation of hierarchies across multiple domains.
168
169
Methods
170
Participants
171
20 healthy participants (11 males and 9 females, age range 21-35 years, M = 26.5) took
172
part in the study. All participants were non-musicians. None had more than 2 years of music
173
training, and none practiced regularly with a musical instrument. All had normal or corrected-
174
to-normal vision and audition, no history of neurological or psychiatric disease. All participants
175
were right-handed German native speakers. Participants were recruited from a pool of subjects
176
able to perform all behavioral tasks successfully (see below). They gave written informed
177
consent before the experiment in accordance with the local ethics committee that had approved
178
this study (016-15-26012015) and were paid 8 Euros/hour for their participation.
179
180
9
Task and Stimuli
181
Our approach involved the comparison of brain activity during active generation of
182
motor sequences following two different rules ‘Recursion’ and ‘Iteration- which can be
183
used to generate or transform hierarchies (Martins, 2012; Martins, Martins, & Fitch, 2015; van
184
der v.d. Hulst, 2010). A third rule ‘Repetition’ – controlled for simple action repetition and
185
working memory without any transformation rule. This framework (discussed in detail in
186
Fischmeister, Martins, Beisteiner, & Fitch, 2017; Martins, 2012; and Martins & Fitch, 2014)
187
focuses on the contrast between “recursive hierarchical embedding” rules which can generate
188
new levels at different subsequent steps and “iterative” insertions rules that do not. Importantly,
189
our framework is orthogonal to some previous formalizations (e.g. Formal Language Theory)
190
which do not specifically target the difference between hierarchical embedding and linear
191
operations (Martins, 2012, for a discussion).
192
Here, the ‘Iterative’ rule adds items linearly within levels of a given hierarchy without
193
generating new levels. For example, with the rule “add 1 B subordinate to A”, we can start
194
from A[B] and, in the subsequent steps, generate A[BB]], [A[BBB]], and so forth. Conversely,
195
the Recursiverule allows the generation of new hierarchical levels at each following step.
196
For instance, with the rule “add 2 subordinate to we start with and generate the
197
hierarchy []’ but also [[] []]’, and so forth. These fractal principles can be
198
described with a recursive notation [], as in Lindenmayer systems (Lindenmayer,
199
1968). This kind of re-write rules have been used to describe visual (Martins, Martins, & Fitch,
200
2016), melodic (Martins, Gingras, Puig-Waldmueller, & Fitch, 2017) and rhythmic fractals
201
(Geambasu, Ravignani, & Levelt, 2016). To control for general task-effects involving visual-
202
motor planning, working memory and movement sequence execution, our approach included
203
a third “rule” - ‘Repetition’ in which complete motor sequences had to be repeated, without
204
10
necessity to add any items or levels. This control task taxed the processes involved in acquiring,
205
buffering and executing a complete motor sequence.
206
In the motor task, participants were asked to produce sequences of finger movements
207
that followed these rules on a keyboard with 16 keys: They had to press a correct set of keys in
208
the correct order, and with the correct timing. The temporal structure (see Figure 1A) was given
209
by an auditory metronome sounding at 240 bpm (4 beats per second delivered through MR-
210
compatible headphones). The typical trial was composed of three steps (I, II and III) that
211
unfolded according to one of the three rules: Recursion, Iteration and Repetition (Figure 1B-
212
D). In steps I and II, participants executed the motor sequences by following visual cues
213
appearing on the keys of a virtual keyboard displayed on the screen. The visual cues, used only
214
in steps I and II, were colored circles with different colors denoting the different fingers that
215
participants had to use to press the keys (red: thumb, green: index and blue: middle finger).
216
The relation between steps I and II revealed the underlying rule that participants had to apply
217
in order to actively generate a final motor sequence in step III without visual guidance and
218
following the temporal structure (Figure 1A). Crucially, in the recursive rule, only the first two
219
hierarchical levels were present at step II, and participants had to generate a third level
220
consistent with the previous ones. This required the simultaneous representations of the first
221
two levels, and of the relations between them. In particular, each set of three keypresses [K-1,
222
K, K+1] at level n+1 derived from a particular key Kn from which it was hierarchically
223
dependent (see Figure 1D and caption for detailed illustration). The extraction of these cross-
224
level relations was essential to generate step III using the proper set of parameters.
225
The application of any of our three rules always generated a final motor sequence of
226
the kind [[K - 2s, K - s, K], [K - s, K, K + s], [K, K + s, K + 2s]]. To increase stimulus variability,
227
we introduced the changeable parameters s (contour) and K (reference key, which in the
228
Recursive rule was also the initial key k1). The parameter s could be a value within the set {-2,
229
11
-1, 1, 2}. If s was positive (1 or 2), the sequences were ascending, meaning that they unfolded
230
from left to right on the keyboard (e.g. [K-1, K, K+1]). If s was negative (-1 or -2), the
231
sequences were descending, meaning that they unfolded from right to left on the keyboard (e.g.
232
[K+1, K, K-1]). When s = 1 or -1, the sequence (within each cluster) was formed by adjacent
233
keys ([K+1, K, K-1]), and when s = 2 and -2, the sequence was formed by non-adjacent keys,
234
meaning that there was a space of one key between a pair of elements within the cluster (e.g.
235
[K-2, K, K+2]). The reference key K (which is the spatial center to the pattern) could be one
236
of the middle four keys of the keyboard {7, 8, 9, 10}. Overall, these variations produced 16
237
different sequences, which were perfectly balanced across conditions. Crucially, parameters s
238
and K had to be recognized during steps I and II, in order to correctly generate the sequence in
239
step III.
240
241
242
Figure 1. Task principles. In this task, participants were asked to generate sequences of 9
243
finger movements (ordered from 1 to 9) by pressing keys on the keyboard with the thumb,
244
index and middle finger (red, green and blue). These sequences were formed in three steps (I,
245
II and III) which followed one of three rules: Simple Repetition (B), Iteration (C) and Recursion
246
(D). During steps I and II, participants executed the motor sequence guided by visual cues
247
displayed on the screen. In step III, they were asked to generate the final sequence of 9 finger
248
movements without visual support.
249
12
A) Temporal structure: In step III, all rules resulted in the same complete sequence of 9
250
movements, grouped in 3 clusters, as here [[K-2, K-1, K], [K-1, K, K+1], [K, K+1, K+2]], of
251
4 s duration each (total sequence duration = 12 s). K is the key in the spatial center of the
252
pattern. Hierarchical clustering within the sequence (3 clusters of 3 key presses) was given by
253
the fingering pattern (red, green and blue) and the temporal structure (1s break after each
254
cluster). To mark the temporal structure, the sequence was in fact aligned with a metronome
255
with 4 beats per second (1 strong and 3 weak) with key presses starting on the strong beat and
256
being released at the onset of the third weak beat (duration of each key press (d) of .75 s).
257
B) Repetition: Consisted of the repetition of the complete sequence of 9 finger movements 3
258
times.
259
C) Iteration: Step I was composed of 3 key presses executed with the thumb, each with d =
260
.75s, on the first (strong) beat of each cluster [[K-2, _, _ ], [K-1, _ , _ ], [K, _ , _ ]. In step II, a
261
second key press with the index was added to each chunk: [[K-2, K-1, _ ], [K-1, K, _ ], [K,
262
K+1, _ ]]. Thus, the iterative rule added elements to pre-existing hierarchical levels, without
263
generating new levels. Step III was simply the serial completion of the pattern with the middle
264
finger [[K-2, K-1, K], [K-1, K, K+1], [K, K+1, K+2]].
265
D) Recursion: Step I was a single key press with the index finger (first finger, or 1) on key K
266
with d = 12s. Step II was a sequence of three key presses [K - 1, K, K + 1] executed with the
267
thumb (1), index (2), and middle finger (3), respectively, each with d = 3s and 1s break after
268
each key press. The underlying Recursive rule was the substitution of each key press a(k, f)n
269
(on key k and with finger f) in step n, with a sequence of three key presses [a(k, 1)n+1 = a(k-1,
270
f)n, a(k, 2)n+1 = a(k, f)n, a(k, 3)n+1 = a(k+1, f)n], in step n+1. In the time domain, each key press
271
with duration dn was substituted by 3 key-presses each with duration dn/4 and followed by a
272
break dn/12. For simplification, we will refer to this rule as kn → [(k - 1)n+1, (k)n+1, (k + 1)n+1].
273
Step III was obtained by applying the same transformation rule to each key press in step II thus
274
obtaining the complete sequence [[K-2, K-1, K], [K-1, K, K+1], [K, K+1, K+2]]. Each set of
275
key presses at level n+1 was clearly subordinate to one key press at level n. Therefore, the
276
representation of the underlying hierarchical structure was a necessary condition to solve the
277
task.
278
279
Pretest
280
All participants took part in a behavioral session up to one week before the fMRI
281
experiment. The goal of this session was to instruct participants explicitly about the task rules,
282
to assess their understanding of those rules and to train them in the execution of the motor
283
sequences. Note that none of the participants had experience in playing music, particularly not
284
in playing on a piano. The session lasted approximately 2 hours. Participants started by
285
performing a beat perception task (part of the battery used in Müllensiefen, Gingras, Musil, &
286
Stewart, 2014), to evaluate whether they were able to understand the synchrony between two
287
temporal events (a metronome and a music piece). Then, they were shown a slideshow
288
13
explaining the task rules, and video examples with the motor sequences they had to perform
289
(see supplementary material).
290
After the instructions, participants performed a supervised session comprising 10 trials
291
following the Recursive rule. They executed the sequences as depicted on the screen, i.e., did
292
not have to mentally generate the sequences by themselves, as steps I, II and III were all
293
presented visually on screen (unlike in the task used in the MR scanner). A researcher was in
294
the room supervising this session, incentivizing the participants to follow the temporal structure
295
and to use the visual landmarks to find the correct keys. The goal of this phase was to train the
296
participants with correct exemplars. We repeated this procedure for the Iterative rule.
297
If participants were able to execute the Recursive and Iterative sequences adequately, they
298
proceeded to a final session, similar to the one used in the fMRI experiment. This session was
299
composed of 20 trials including 8 trials following the Recursive rule, 8 trials following the
300
Iterative rule, and 4 trials following the simple Repetition. In this last session, step III was not
301
cued visually, and participants had to generate the sequence by themselves, without visual
302
support, but following the metronome, as later in the MR scanner.
303
Participants with accuracy >80% in the last session, for all rules, were invited to participate in
304
the fMRI experiment. Accuracy was measured as the number of correct keys pressed at the
305
correct time within the interval [-0.25s, 1s] locked to the onset of the appropriate beat. Each
306
trial contained 9 expected key presses in step III. 21 out of 39 participants performing the
307
pretest fulfilled this criterion. Note that, although only about 50% of our participants passed
308
our selection threshold for the MRI experiment, those 50% were able to acquire a keypress
309
accuracy above >80% after only 1h of training (roughly 64 trials). Despite the task being
310
relatively difficult to learn in one training session, this is in line with previous research showing
311
that more extensive training is necessary to reach adequate performance in simple, non-
312
hierarchical motor tasks (Taubert et al., 2010; Taubert, Lohmann, Margulies, Villringer, &
313
14
Ragert, 2011), and in artificial grammar learning in language (Opitz & Friederici, 2003).
314
Recent behavioral data from our laboratory (unpublished) also suggests that more training, in
315
terms of number of training sessions, increases keypress accuracy in our motor task.
316
In those participants who succeeded in the pretest, the “generation of new hierarchical levels”
317
(i.e., in the Recursive rule condition) hence pertains to the application of well-learned
318
hierarchical rules, independently on how these were acquired (Lungu et al., 2014). This may
319
entail either the execution of combinatorial computations specifically involved in the
320
generation of new hierarchical levels, or the retrieval of previously formed hierarchical
321
representations (Figure 2), which remain stored as “schemas” within the motor network
322
(Wiestler & Diedrichsen, 2013). Note that in case our task hinges on schema retrieval rather
323
than on-the-fly generativity, these schemas are still not rigidly automatized: (1) they also entail
324
a generative act since their internal spatio-temporal hierarchical organization needs to be
325
constructed from the set of free parameters (s and K), and (2) the transformations within the
326
recursive rule are bounded by a hierarchical temporal clustering which is strictly scale invariant
327
i.e., the key press duration at a subordinate level is exactly ¼ of that of a key press at the
328
dominant level (Figure 1A).
329
330
15
331
332
Figure 2. With our design, we explicitly separated the processes underlying the generation of
333
hierarchical levels (left) from those used to externalize and execute motor programs (right).
334
While the generation of new hierarchical levels in the Recursive rule involves hierarchical
335
branching (left) and then serialization (right), iterative completion of motor sequences is strictly
336
serial. It should be mentioned that activations referring to the generation of new hierarchical
337
levels can potentially involve either de novo combinatorial operations (upper cascade), or the
338
retrieval of previously formed hierarchical representations (lower transparent box). The
339
products of hierarchy-generating rules (e.g. [[K-2s, K-s, K] [K-s, K, K+s] [K, K+s, K+2s]])
340
might become schematized and stored in domain-specific networks from which they are
341
retrieved during sequence generation. The schema would retain the clustered hierarchical
342
structure and a set of free parameters binding different levels (in this study the reference key
343
K, and contour variable s). Importantly, even if the latter were the underlying mechanism,
344
participants would have to extract and apply the parameters from the second step of each trial,
345
and obey the same hierarchically organized temporal cluster boundaries. Thus, irrespective of
346
whether processing is based on combinatorial operations or retrieval of schemas, only
347
Recursion would entail flexible generation of hierarchical motor sequences.
348
349
fMRI Procedure
350
On the day of fMRI data acquisition, participants were again briefed on the task rules,
351
then positioned in the scanner and asked to perform a short test session of 6 trials. If they were
352
able to perform adequately, we proceeded with the anatomical and functional data acquisition.
353
One participant was excluded due to inability to replicate the experiment within the MR
354
16
experimental apparatus. At the end of the procedure, participants were given a questionnaire
355
on their cognitive strategies used to generate step III across the different tasks. The whole
356
procedure (briefing, scanning, and questionnaire) had a duration of approximately 2:30 hours.
357
The fMRI scan included 4 sessions, each with an approximate duration of 15 minutes and
358
composed of 20 trials - 8 Recursion’, 8 ‘Iteration’ and 4 ‘Repetition’ trials. For the sake of
359
maximizing the number of trials in the main tasks of interest, we kept the number of Repetition
360
trials to half. Trials following different rules were inter-mixed within each session and pseudo-
361
randomized. The trial sequence was determined using Optseq2
362
(https://surfer.nmr.mgh.harvard.edu/optseq/) to maximize the efficiency of fMRI signal
363
acquisition.
364
Trial structure is depicted in Figure 3. We were interested in two periods within each trial,
365
namely the transition between step II and III the planning phase and in step III the
366
execution phase. The planning phase was important to capture the computations necessary to
367
transform step II into step III, and the neural systems instantiating these transformations.
368
369
370
17
Figure 3. Trial structure (Recursion example). All trials had the same structure: First, a letter
371
indicated the trial type. Then, steps I and II of the sequence were shown on screen, which
372
participants had to execute simultaneously on a keyboard (coloured circles indicated which
373
finger to use). This was followed by a 6 second ‘planning phase’ composed of a 4 second blank
374
screen and a 2 second crosshair during which participants planned execution of step III. Finally,
375
in the ‘execution phase’, participants performed the correct continuation of the sequence
376
without visual cues. Throughout all steps, a metronome sound at 60bpm guided participants’
377
pace and the sequence’s temporal structure.
378
379
380
The experimental apparatus is depicted in Figure 4. Participants performed the task while lying
381
in the MR scanner, using a silent 16-key MR-compatible piano (Figure 4A). The keyboard
382
contained visual and tactile markers on keys 3, 5, 7, 10, 12 and 14 (from left-to-right) for spatial
383
reference (see Figure 1). We used a dual mirror system, so that participants were able to see
384
both the virtual keyboard projected on the screen, and the physical keyboard on which they
385
executed the motor sequences (Figure 4B). The position of the mirrors was adjusted
386
individually for each participant. Both keyboards had visual markers on specific keys for visuo-
387
spatial reference (Figure 4C). On the physical keyboard, these references could also be detected
388
by touch. All participants used their right hand to perform the motor sequences.
389
390
391
18
Figure 4. fMRI apparatus. A. The keyboard was placed on a custom-made wood stand.
392
This stand provided a degree of inclination that increased the visibility of the keyboard. The
393
metronome sound was delivered through MR compatible headphones. B. We used a double
394
mirror system mounted on the head coil, which allowed participants to see both the virtual
395
keyboard on screen (top mirror, left arrow), and the physical keyboard under their right hand
396
(bottom mirror, right arrow). We adjusted the position of the mirrors for each participant to
397
maximize visibility and comfort. C. The keyboard was an adapted MR compatible piano in
398
which the black keys were covered. We added visual and tactile cues on specific keys that the
399
participants could use for reference. Importantly, pressing the keys on the keyboard did not
400
generate any sound, and therefore key-tone associations could not be used in our task, which
401
was purely visuo-motor.
402
403
404
Data Acquisition
405
The experiment was carried out in a 3.0-Tesla Siemens SKYRA whole body magnetic
406
resonance scanner (Siemens AG, Erlangen, Germany) using a 32-radiofrequency-channel head
407
coil. During the 4 sessions, functional magnetic resonance images were acquired using a T2*-
408
weighted 2D echo planar imaging (EPI) sequence with TE = 30 ms and TR = 2000 ms. For
409
each session, we acquired 450 volumes with a square FOV of 192 mm, with 31 interleaved
410
slices of 3 mm thickness and 30% gap (3 x 3 x 3 mm3 voxel size) aligned to the AC-PC plane,
411
and a flip angle of 90°. T1-weighted images for anatomical co-registration were either selected
412
from the database of the institute or acquired using a 3D MP2RAGE sequence (TI1 = 700 ms,
413
TI2 = 2500 ms, TE = 2.03 ms, TR = 5000 ms) with a matrix size of 240 x 256 x 176, with 1
414
mm isotropic voxel size, flip angle1 of 4°, flip angle2 of 8°, and GRAPPA acceleration factor
415
of 3.
416
Data Analysis
417
Task-based fMRI. fMRI data of 20 participants were analysed with statistical
418
parametric mapping (SPM8; Welcome Trust Centre for Neuroimaging;
419
http://www.fil.ion.ucl.ac.uk/spm/software/spm8/). Anatomical data from high-resolution T1-
420
weighted images were obtained by masking the uniform tissue-contrast image with the 2nd
421
19
inversion image from the MP2RAGE sequence. Functional data were pre-processed by
422
following standard spatial pre-processing procedures. They consisted of: slice time correction
423
(by means of cubic spline interpolation method), spatial realignment, co-registration of
424
functional and anatomical data. Then, we performed spatial normalisation into the MNI
425
(Montreal Neurological Institute) stereotactic space that included resampling to 2x2x2 mm
426
voxel size. Finally, data were spatially low-pass filtered using a 3D Gaussian kernel with full-
427
width at half-maximum (FWHM) of 8 mm and temporally high-pass filtered with a cut-off of
428
1/128 Hz to eliminate low-frequency drifts.
429
Statistical parametric maps were generated for the whole brain data in the context of the general
430
linear model (GLM). For each rule, we modelled 3 ‘trial phase’ regressors at first level: 1) steps
431
I and II together, 2) planning and 3) execution phase of step III. This way we could control for
432
potential activity spill-over between phases. The evoked hemodynamic response to the onset
433
of each phase was modelled for the Recursive rule, Iteration rule and simple Repetition
434
conditions as boxcars convolved with a hemodynamic response function (HRF). We added
435
estimated motion realignment parameters as covariates of no interest to this design to regress
436
out residual motion artefacts and increase statistical sensitivity. Furthermore, we added two
437
regressors to account for potential differences in difficulty between ‘Recursion’, ‘Iteration’,
438
and ‘Repetition’: As a measure of ‘planning difficulty’, we modelled the average asynchrony
439
between the first metronome beat (when participants should press the key) and the actual first
440
key presses of step III. As a measure of ‘execution difficulty’, we modelled the average
441
asynchronies between the metronome beats and the actual key presses across all 9 key presses
442
of step III (mean response times for each key press are depicted in Supplementary Figure S1).
443
If participants pressed an incorrect or no key, we assigned the value 1s, which is the highest
444
possible value, i.e., reflects maximal difficulty.
445
20
For random effects group analyses, two within-subject flexible factorial ANOVAs (with the
446
factor RULE) were performed (for step III planning and execution, separately) on whole brain
447
data with binary grey matter masks thresholded at intensity value of 0.25. A main effect of
448
RULE (‘Recursion’, ‘Iteration’, ‘Repetition’) was detected in both planning and execution
449
phases of step III. To resolve these effects, statistical parametric maps with t-contrasts between
450
each RULE were calculated. We controlled family-wise error rate (FWER) of clusters below
451
0.05 with a cluster-forming height-threshold of 0.001.
452
To test for the involvement of lateral PFC, particularly IFG, we performed Small
453
Volume Corrected (SVC) analyses within an ROI comprising the left and right Brodmann
454
Areas (BA) 44 and 45 based on the Harvard-Oxford probability maps (thresholded at 50%).
455
Anatomical labels are based on Harvard-Oxford cortical and sub-cortical structural atlas
456
implemented in FSL (http://neuro.debian.net/pkgs/fsl-harvard-oxford-atlases.html). In
457
addition, we used REX toolbox (http://web.mit.edu/swg/software.htm) to extract the mean of
458
the single-subject beta values across each ROI mask and calculated the t-contrasts between
459
each RULE (‘Recursion’, Iteration’, ‘Repetition’), separately in the planning phase and the
460
execution phase.
461
462
21
Results
463
In the present fMRI study, participants generated sequences of finger movements in 3
464
steps (I, II, III) following one of three rules (Figure 1): (1) a linear Iterative rule, (2) a Recursive
465
Hierarchical Embedding rule, or (3) simple Repetition. In the first 2 steps (I, II) participants
466
executed sequences guided by visual cues on the screen. Then, they were asked to generate
467
step III according to the respective rule without visual support (Figure 3). The fMRI analysis
468
focused on the transition between step II and III, the ‘planning phase’ reflecting the generative
469
act, and on step III, the execution phase’ reflecting the externalization. Importantly, motor
470
sequences were identical in their surface structure across tasks. Hence, any difference in brain
471
activation during execution is likely to derive from the different outcomes of the generative
472
phase.
473
Overall, we found that during action planning, the generation of new hierarchical levels
474
in Recursion, compared to both Iteration and simple Repetition, yielded significantly stronger
475
activity in a bilateral network of brain areas involved in motor planning and imagery (Hardwick
476
et al., 2013; Hétu et al., 2013), including M1S1 and PMC, cerebellum, lateral occipital cortex
477
(LOC), and left putamen (Figure 5). Crucially, regions of interest (ROI) analyses within left
478
and right IFG lent no evidence for involvement of lateral PFC in the generation of new
479
hierarchical levels (Figure 6). In the execution phase, no activation was specific for Recursion
480
(i.e., stronger than in Iteration and Repetition). Instead, execution of sequences formed by both
481
Recursion and simple Repetition rules showed similarities when compared with Iteration in
482
form of bilateral basal ganglia and thalamus activity (Figure 7). This suggests that other than
483
planning, which required specific additional resources for Recursion, sequence representation
484
during execution was not Recursion specific.
485
486
22
Generation of new Hierarchical Levels is Supported by General Networks of
487
Motor Planning
488
By measuring brain activity in the ‘planning phase’, we sought to identify neural
489
networks underlying the cognitive processes that are relevant for the transition between step II
490
and III, i.e., the generative act as such. In simple Repetition, this process consisted in holding
491
the full sequence of step II in memory until its repeated execution in step III (Figure 1B). In
492
the Iterative rule, this process required the serial addition of one key press to each cluster within
493
a fixed hierarchical level (Figure 1C), without generation of new levels. The Recursive
494
embedding rule entailed the generation of new hierarchical levels (Figure 1D) by recursively
495
substituting each key press kn in step n with a new sequence of three key presses [(k - 1)n+1,
496
(k)n+1, (k + 1)n+1] in step n+1 (see Figure 1 for a detailed explanation). The parameters of the
497
transformation rule to be applied in each trial’s planning phase, s and initial key k1, could be
498
inferred from the transition between steps I and II (see Methods for details).
499
500
Behavioural data. According to the post-experiment questionnaires (see
501
Supplementary Table S1), participants considered it equally difficult to extract the rule
502
parameters in Recursion (mean ± SD: 6.15±0.88) and Iteration (6.40±0.75) (Wilcoxon signed-
503
ranks: z = -1.30, p = .19), while it was easier to do so in Repetition (6.80±0.41), than in the
504
other two conditions (Wilcoxon signed-ranks: z = -2.51, p = .01, and z = -2.13, p = .03,
505
respectively).
506
In addition, in Recursion trials, participants relied more on step II for the generation of
507
step III (6.00±1.52) than in both Iteration (5.20±1.74) and Repetition (4.90±1.75) (Wilcoxon
508
signed-ranks: z = -2.35, p = .02, and z = -2.32, p = .02, respectively). Finally, in comparison
509
with Repetition, during Recursion participants (i) imagined more where the hand should go in
510
key space, (ii) prepared the sequence more consciously and (iii) thought more explicitly about
511
23
the rule (Wilcoxon signed-ranks: all ps < .05). For detailed means and pairwise comparisons
512
see Supplementary table S1.
513
To account for potential differences in task difficulty, we included the asynchrony
514
between first metronome beat and first key press of step III as a planning difficulty measure in
515
our first level model (see Methods). Averages of these asynchronies did not differ between
516
tasks (Iteration: 0.30 ± 0.10s; Recursion: 0.31 ± 0.09s; Repetition: 0.30 ± 0.09s; F(2,38) = 0.14,
517
p = .868, np2 = .01).
518
519
fMRI. Whole-brain results of the planning phase are depicted in Figure 5 and Table 1.
520
We found increased activity in Recursion in comparison with both Repetition and Iteration.
521
The generation of new hierarchical levels in motor sequences using the Recursive rule was
522
supported by a bilateral network known to be involved in motor learning (Hardwick et al.,
523
2013), motor planning (Elsinger et al., 2006) and imagery of motor sequences (Hétu et al.,
524
2013). More precisely, this network included a large bilateral cluster with peaks in the
525
cerebellum and extending through LOC, Superior Parietal Lobe, M1S1 and left PMC (see
526
Supplementary Table S2 for more extensive enumeration and labelling of the peaks within this
527
cluster). Further clusters included left putamen and pallidum, and right PMC (all clusters p <
528
.05, FWE corrected). These activations were present in both contrasts ‘Recursion > Iteration’
529
and ‘Recursion > Repetition’, and despite correcting for planning difficulty. Additionally, the
530
contrast ‘Recursion > Iteration’ but not ‘Recursion > Repetition’ yielded stronger activation in
531
right Pallidum and Putamen.
532
Conversely, no activations were found for the Iterative rule or Repetition, i.e., the
533
contrasts ‘Iteration > Repetition’, ‘Iteration > Recursion’ and ‘Repetition > Recursion’ did not
534
yield significantly active clusters. Only ‘Repetition > Iteration’ revealed activity in bilateral
535
inferior lateral temporo-occipital cortex and frontal pole (Table 1), which may support mental
536
24
practice and working memory for motor sequence (Jackson, Lafleur, Malouin, Richards, &
537
Doyon, 2003).
538
539
540
Figure 5. Brain activations during the planning phase (between steps II and III).
541
Application of the Recursive rule yielded stronger activations compared to both Simple
542
Repetition and Iteration in a bilateral network known to be involved in motor learning, planning
543
and imagery, including sensorimotor and premotor cortices, cerebellum and lateral occipital
544
cortex. The reverse contrasts (‘Iteration > Recursionand ‘Repetition > Recursion’) did not
545
yield significant activations.
546
547
548
25
Region
Hem.
BA
k
x
y
z
Z-value
Cerebellum VI
R
-
23865
26
-50
-30
5.97
-
32
-42
-30
5.60
-
8
-70
-34
5.44
Putamen
L
-
1199
-26
-6
8
4.95
-
-12
-18
2
4.79
-
-24
-14
4
4.56
Pallidum
R
-
879
18
-2
0
4.71
-
26
-8
10
4.67
-
24
0
10
4.59
Precentral Gyrus
R
6
1108
42
-6
60
4.63
6
24
-2
54
4.63
6
32
-4
66
4.13
Brain stem
-
461
-6
-28
-22
4.01
-
-4
-30
-30
3.69
-
10
-26
-24
3.65
Recursion > Repetition
Cerebellum VI
R
-
1764
26
-52
-30
5.27
L
-
-24
-50
-26
4.32
R
32
-42
-30
4.10
Precentral Gyrus
R
6
6173
44
-6
56
5.00
L
6
-24
-4
54
4.94
L
3
-40
-20
54
4.85
Putamen
L
-
471
-20
12
-2
4.16
-
-24
-4
6
3.93
Postcentral Gyrus
R
3
1057
32
-32
48
4.15
2
54
-18
42
3.88
2
40
-30
48
3.79
Lateral Occipital Cortex
R
V5
819
44
-62
8
4.13
V5
52
-64
6
4.02
-
34
-68
22
3.83
Cerebellum I
R
2466
48
-66
-22
5.31
48
-56
-14
5.27
26
-42
-42
4.75
Frontal Pole
R
10
1291
18
38
-16
5.22
10
26
42
-8
4.63
10
34
48
-4
4.36
Lateral Occipital
L
V5
4760
-42
-66
-10
4.81
R
-
8
-70
-36
4.53
V3
-28
-90
2
4.52
Occipital Pole
R
V4
704
30
-90
2
4.42
17
14
-94
12
3.87
V4
38
-80
0
3.78
549
ROI analyses. To test whether there were specific activations for the Recursive rule
550
within lateral PFC, particularly IFG, we performed four Small Volume Corrected (SVC)
551
26
analyses within left Brodmann Area (BA) 44, left BA 45, right BA 44 and right BA 45. Planned
552
contrasts between rules yielded no significant differences (with uncorrected p < .01). The
553
comparison of global activity within each ROI (see Figure 6) revealed stronger right BA 44
554
activity in both ‘Recursion > Iteration (T = 2.15, p-uncorrected = .04) and ‘Repetition >
555
Iteration(T = 2.41, p-uncorrected = .02). However, these effects did not survive FDR
556
threshold at p < .05.
557
558
559
Figure 6. Global activity within the 4 IFG ROIs. Percent Signal change (globally scaled)
560
was higher in right BA 44 during planning in both Recursion and Repetition vs. Iteration.
561
However, this activity did not survive FDR threshold at p < .05. No significant differences
562
were found during execution.
563
564
565
27
We assumed participants engaged in specific computations to transform step II into the
566
final step III in the Recursive condition, using an explicit motor-spatial rule. We found these
567
specific computations were supported by general networks associated with motor planning and
568
imagery but did not recruit IFG. The trend of greater activity in right IFG in both Recursion
569
and Repetition vs. Iteration suggests that these conditions may pose greater strain on working
570
memory and motor control system (Aron, Robbins, & Poldrack, 2014).
571
572
Execution of Recursion- and Repetition-based Sequences (vs. Iteration-based)
573
Recruits Thalamus and Basal Ganglia
574
575
Behavioural data. In the execution phase, sequences were motorically identical across
576
all conditions. Importantly, key press accuracy did not differ between conditions (Recursion:
577
mean ± SD = 87% ± 20%; Iteration: 89% ± 18%; Repetition: 87% ± 23%; generalized χ2 score
578
= 1.8, p = .400), suggesting that the execution was equally difficult. In addition, participants
579
reported similar confidence in the correctness of their performance (regarding rhythm, keys
580
pressed, and fingers used) in Recursion and Iteration (Wilcoxon signed ranks: ps > .400; see
581
Supplementary Table S1 for full details on means and pairwise comparisons).
582
Finally, we measured the asynchrony between metronome beats and key presses in step
583
III. Average asynchronies across the 9 key presses (Iteration: 0.21 ± 0.13s; Recursion: 0.24 ±
584
0.14s; Repetition: 0.30 ± 0.12s) did not differ between the three tasks (F(2,38) = 1.27, p = .291,
585
np2 = .06). To account for residual difficulty differences between tasks, we included the mean
586
asynchronies for each trial as parameter into our fMRI statistical model.
587
588
fMRI. In the ‘execution phase’, we found clear similarities between Recursion and
589
Repetition that both dissociated from Iteration (Figure 7 and Table 2). We found significant
590
28
activations in subcortical clusters including Pallidum, Putamen, and Thalamus in both
591
‘Recursion > Iteration’ and ‘Repetition > Iteration’ contrasts. These clusters extended
592
posteriorly into Hippocampus and Parahippocampus, and anteriorly into right Orbitofrontal
593
Cortex (including BA10 and BA47). In Recursion > Iteration an additional cluster was found
594
in left LOC. Finally, Iteration > Recursion did not yield significant activations that survived
595
cluster level correction. However, one cluster in left Primary Somatosensory cortex (BA1) was
596
significant with FWE-correction at voxel level (Z = 4.53, voxel p-FWE = .04, x=-50, y = -16,
597
z = 48).
598
ROI. Similar to the planning phase, we performed Small Volume Corrected (SVC)
599
analyses within an IFG ROI comprising left BA 44, left BA 45, right BA 44 and right BA 45.
600
We found no significant differences between rules (with uncorrected p < .01). Global activity
601
within each area was also not significantly different across rules (all uncorrected p > .10)
602
(Figure 6).
603
604
29
605
606
Figure 7. Brain activations during the execution phase (step III). Participants executed
607
sequences of nine key presses that were identical at the motor output but were generated
608
according to different rules (Recursion, Iteration and Repetition). (1) Compared to Iteration,
609
both Recursion and Repetition (C and D) activated the Pallidum, Putamen and Thalamus
610
bilaterally. These clusters extended posteriorly into Hippocampus and Parahippocampus (left
611
panel), and anteriorly into right Orbitofrontal Cortex (right panel; BA10 and BA47). In the
612
contrast Recursion > Iteration we found an additional cluster in left LOC.
613
614
30
615
Discussion
616
To our knowledge, the present study is the first to investigate the neural systems involved in
617
the generation and overt production of motor hierarchies, which clearly separates these two
618
phases (generative act and externalization components). To do so, we developed a novel
619
paradigm that contrasted (1) sequences of finger movements formed according to a hierarchy-
620
generating Recursive rule with (2) identical sequences formed according to rules that did not
621
require generation of new hierarchical levels (Iteration and Repetition). Each trial was
622
composed of two initial steps (I and II) that established the rules and a set of parameters which
623
participants had to apply to correctly generate step III. Thus, during planning, Repetition
624
implied buffering of the given motor sequence [[K2s, K-s, K] [K-s, K, K+s] [K, K+s, K+2s]]
625
and Iteration required the completion of a pattern [[K2s, K-s, __ ] [K-s, K, __ ] [K, K+s, __
626
]] using within-level transformations. Only the Recursive rule entailed the generation of new
627
hierarchical levels through the recursive substitution of each finger movement kn with a
628
sequence of three finger movements [(k - s)n+1, (k)n+1, (k + s)n+1]. Accordingly, participants
629
Table 2. Rule effect in the execution phase.
Region
Hem.
BA
k
x
y
z
Z-value
Recursion > Iteration
Pallidum
R
-
2008
16
-6
-6
5.53
-
26
6
14
5.02
-
22
14
10
4.69
Putamen
L
-
432
-22
4
14
4.95
-
-22
12
12
4.73
-
-22
-10
-4
3.86
Lateral Occipital Cortex
L
19
1438
-30
-90
6
4.92
19
-18
-94
-12
4.58
18
-20
-94
8
4.29
Thalamus
L
-
1117
-12
-6
2
4.17
-
-26
-32
-32
4.03
-
-4
-30
-28
4.02
Repetition > Iteration
-
Pallidum
R
-
2229
16
-6
-6
5.01
-
26
0
14
4.69
-
28
-28
-4
4.68
Putamen
L
-
1848
-24
4
12
4.61
-
-10
-22
-18
4.48
-
-22
-10
-4
4.26
Whole-brain activation cluster sizes (k), MNI coordinates (x, y, z), and Z-scores for the Rule
contrast in the planning phase (pvoxel < .001; pcluster < .05, FWE corrected). BA: Brodmann area,
Hem.: hemisphere.
31
reported mostly for the Recursive condition that they relied on step II to consciously prepare
630
the final sequence and imagined the sequence prior to execution. This entails that in the
631
Iteration condition participants may have engaged less in active planning of Step III.
632
Nevertheless, Recursive and Iterative conditions did not differ in correctness of their execution
633
or in subjective reports of general difficulty.
634
Our first important finding was that the generation (i.e., planning) of new hierarchical
635
levels using the Recursive (compared to Iterative) rule was supported by a network of areas
636
involved in motor learning, planning and imagery (Elsinger et al., 2006; Hardwick et al., 2013;
637
Hétu et al., 2013). This bilateral network included M1S1 and PMC, cerebellum, LOC,
638
pallidum, putamen, and thalamus. Interestingly, in this study, which focused on the motor
639
domain, we did not find evidence that these generative processes recruited IFG, an area thought
640
to play an important role in the processing of hierarchies across domains (Fadiga et al., 2009;
641
Fitch & Martins, 2014; Jeon, 2014). Although there was higher global activity within right BA
642
44 in Recursion compared with Iteration, this activity neither survived multi-comparison p-
643
value correction, nor was it specific for Recursion, being present also in the Repetition >
644
Iteration contrast. Therefore, right BA44 activity, if any, is not likely caused by computations
645
specific to the generation of hierarchical levels using recursive rules.
646
Our second relevant finding was that execution of identical motor sequences generated
647
by the recursive rule or by simple repetition of a given sequence, both involved bilateral
648
subcortical areas including Putamen, Pallidum and Thalamus, extending posteriorly to
649
Hippocampus and Parahippocampus, and in the right hemisphere anteriorly to Orbitofrontal
650
cortex, including BA47 and BA10. This suggests that identical sequences might be represented
651
differently depending on their generative process. Notably, the similarity between Recursion
652
and Repetition (vs. Iteration) suggests that these representations are not specific to the
653
processing of hierarchical relations, but to some other processes, which we discuss below.
654
32
According to the discrete sequence production framework (Verwey, 2001; Verwey,
655
Shea, & Wright, 2014), performance involves (1) sequence generation and motor loading
656
during planning, followed by (2) fast execution of the motor buffer content by effector-specific
657
motor processors. The generation of new hierarchical levels in the Recursive rule puts
658
particular strain on stage (1), the planning of the final sequence, by strongly relying on cortical
659
resources. Unlike in Repetition and Iteration where the motor program is (partly) available
660
already in step II, performers have to use their rule knowledge in the Recursive condition to
661
construct or retrieve motor schemas (see legend of figure 2 about these alternatives) for
662
appropriate sequence continuation. Interestingly, they seem to do so by means of general
663
mechanisms of visuo-motor imagery and planning, as shown by stronger activity in bilateral
664
visuo-motor networks (Hardwick et al., 2013).
665
Once formed, these motor programs are buffered in striatal areas and sent to the motor
666
effectors for execution (Doyon et al., 2009; Miyachi, Hikosaka, Miyashita, Kárádi, & Rand,
667
1997). Our activity patterns in the execution phase speak for a similar buffering during
668
Recursion and simple Repetition, but not during Iteration. Both Recursion and Repetition
669
activated a fronto-striatal-thalamic circuit with the additional contribution of
670
Hippocampus/Parahippocampus, and right Orbitofrontal cortex (BA47 and BA10). The fronto-
671
striatal-thalamic circuit supports motor control and working memory during sequence
672
production (Humphries & Gurney, 2002; Schroll, Vitay, & Hamker, 2012; Vitay, 2010), and
673
fronto-hippocampal areas have been associated with a global vs. incremental representation of
674
motor sequences (Lungu et al., 2014). According to these previous findings, we surmise that
675
during production in both Recursion and Repetition conditions, a global representation of the
676
full sequence is retained in working memory and used to optimise motor control, hence a
677
correct motor sequence. It is very likely indeed that the sequence of 9 finger movements was
678
fully present during execution, being generated during planning in the Recursive condition (see
679
33
paragraph above) and carried over from step II in Repetition (see activity in right orbitofrontal
680
cortex during planning). Conversely, during planning, we did not observe increased activity in
681
Iteration in comparison with Recursion or Repetition. This may indicate less strain on the motor
682
buffer, either because a sequence of only 6 finger movements had to be carried over from step
683
II to be linearly completed in step III, or because participants used a generally different
684
execution strategy that was less hinging on the motor buffer (although it fell short off
685
significance at the cluster level, there was higher activity in left Somatosensory Cortex in
686
Iteration during step III than in recursion). Overall, the results suggest that execution of a
687
sequence formed by an incremental Iterative rule poses less demands on the motor control
688
system compared to buffering and releasing the complete motor sequence in the Recursion and
689
Repetition conditions.
690
In sum, we found that while generating new hierarchical levels in the Recursive rule
691
demands more planning resources, serial completion of motor sequences in the Iterative rule
692
might be achieved using sensorimotor areas during execution. Interestingly, these additional
693
planning resources in Recursion were instantiated by the motor imagery network, and they did
694
not require IFG.
695
Prior hypotheses: The Role of lateral PFC/IFG
696
Based on current views that IFG is involved in the processing of hierarchies across many
697
domains (e.g. language, music and action, as reviewed by Fadiga et al., 2009; Fitch & Martins,
698
2014; Jeon, 2014) and in line with models of a posterior-to-anterior gradient of lateral PFC for
699
hierarchical organization of actions (Badre, 2008; Koechlin & Summerfield, 2007), we
700
hypothesized lateral PFC, and particularly IFG, to support motor generation of new
701
hierarchical levels in our Recursive rule condition. However, we did not find evidence for
702
involvement of this area in the generation of new hierarchical levels. How can our results be
703
reconciled with the previous literature?
704
34
On the one hand, the absence of evidence for lateral PFC activation in our task might
705
indicate that this region is sensitive to hierarchies of action goals (or other non-motor
706
contextual dependencies; Badre, 2008), rather than to transparent rules describing cross-level
707
relations in motor hierarchies (i.e. inducible without prior instruction) as tested in our task.
708
Alternatively, the resources necessary to discriminate hierarchical sequences may not
709
completely overlap with those used for the generation of new hierarchical levels, in that
710
discrimination recruits numerous additional cognitive mechanisms that are not relevant during
711
generation but may well account for IFG effects. For example, representing hierarchies from
712
sequential input during discrimination also poses demands on resources required more
713
generally for sequence encoding, buffering and template matching (Bornkessel-Schlesewsky,
714
Schlesewsky, Small, & Rauschecker, 2015; Fitch & Martins, 2014), that may not be taxed to
715
the same degree during the generation of hierarchical structures in the motor domain.
716
Importantly, most discrimination studies found greater IFG involvement in material that drew
717
strongly on these general resources, e.g., by using sequences that were violations (Bianco et
718
al., 2016; Molnar-szakacs, Iacoboni, & Koski, 2005; Novick et al., 2005), had greater
719
ambiguity (Rodd, Vitello, Woollams, & Adank, 2015; Vitello & Rodd, 2015), longer
720
dependencies or posed higher demands on working memory than respective control sequences
721
(Baddeley, 2003; Braver et al., 1997). This makes it difficult to dissociate the contribution of
722
specific hierarchical generativity and general cognitive control/sequence encoding processes
723
to the observed IFG activations (see also Fedorenko et al., 2012). Our design not only balanced
724
the amount of required cognitive control across conditions (recall that final sequences were
725
always correct, unambiguous and identical across conditions, although based on different
726
rules); it also allowed us to study hierarchy processing stripped off general processes required
727
for parsing temporally evolving sequences by specifically targeting hierarchy generation (in
728
the planning phase). Consequently, the fact that we did not find evidence for lateral PFC
729
35
involvement does not support the notion of multi-domain hierarchical generativity in IFG
730
(Fadiga et al., 2009; Fitch & Martins, 2014) and rather argues for its more general function
731
during encoding of structured sequences. It is important to note that while we did not find
732
evidence for the role of IFG in the generation of hierarchies in the motor domain, this region
733
could play a pivotal role in other domains such as language. Since we focus on both generation
734
of motor hierarchies (in contrast with, for instance, processing of linguistic syntax), we cannot
735
draw strong conclusions about other domains. Although recent experiments in the music
736
domain seem to suggest a similar absence of specialized IFG activity in the generation of new
737
hierarchical levels vs. serial iteration (Martins, 2017) further work is needed.
738
Conclusion
739
In this study, we isolated the processes involved in generating motor hierarchies while
740
separating them from other motor externalization components. Our results suggest that the
741
generation of motor hierarchical structures via the application of recursive rules was supported
742
by a neural system used for motor imagery and motor planning. Conversely, we did not find
743
evidence that a putative multi-domain hierarchical processor in the lateral PFC is necessary for
744
the generation of hierarchical levels in motor sequence production. While lateral PFC might be
745
important to parse hierarchical sequences in a multi-domain fashion, due to encoding and
746
externalization processes, it might not be necessary for the generation of new hierarchical
747
levels.
748
Acknowledgments
749
The authors are grateful to Sven Gutekunst and Jöran Lepsien for technical support.
750
Author contributions
751
Mauricio J.D. Martins and Roberta Bianco contributed project conception, experimental design
752
and setup, data acquisitionand analysis, data interpretation, writing the manuscript; Daniela
753
36
Sammler and Arno Villringer contributed supervision of the project, project conception, data
754
interpretation, writing the manuscript.
755
756
Data availability
757
The data that support the findings of this study are available from the corresponding author
758
upon request. Authors can confirm that all relevant data are included in the paper.
759
760
References
761
Aron, A. R., Robbins, T. W., & Poldrack, R. (2014). Inhibition and the right inferior frontal
762
cortex: one decade on. Trends in Cognitive Sciences, 18(4), 177185.
763
https://doi.org/10.1016/j.tics.2013.12.003
764
Baddeley, A. (2003). Working memory: Looking back and looking forward. Nature Reviews
765
Neuroscience, 4(10), 829839. https://doi.org/10.1038/nrn1201
766
Badre, D. (2008). Cognitive control, hierarchy, and the rostro-caudal organization of the
767
frontal lobes. Trends in Cognitive Sciences, 12(5), 193200.
768
https://doi.org/10.1016/j.tics.2008.02.004
769
Bahlmann, J., Schubotz, R. I., & Friederici, A. D. (2008). Hierarchical artificial grammar
770
processing engages Broca’s area. NeuroImage, 42(2), 525534.
771
https://doi.org/10.1016/j.neuroimage.2008.04.249
772
Bahlmann, J., Schubotz, R. I., Mueller, J. L., Koester, D., & Friederici, A. D. (2009). Neural
773
circuits of hierarchical visuo-spatial sequence processing. Brain Research, 1298, 161
774
170. https://doi.org/10.1016/j.brainres.2009.08.017
775
Bianco, R., Novembre, G., Keller, P. E. E., Kim, S.-G. G., Scharf, F., Friederici, A. D., …
776
37
Sammler, D. (2016). Neural networks for harmonic structure in music perception and
777
action. NeuroImage, 142, 454464. https://doi.org/10.1016/j.neuroimage.2016.08.025
778
Boecker, H., Jankowski, J., Ditter, P., & Scheef, L. (2008). A role of the basal ganglia and
779
midbrain nuclei for initiation of motor sequences. NeuroImage, 39(3), 13561369.
780
https://doi.org/10.1016/j.neuroimage.2007.09.069
781
Bornkessel-Schlesewsky, I., Schlesewsky, M., Small, S. L., & Rauschecker, J. P. (2015).
782
Neurobiological roots of language in primate audition: common computational
783
properties. Trends in Cognitive Sciences, 19(3), 142150.
784
https://doi.org/10.1016/j.tics.2014.12.008
785
Braver, T. S., Cohen, J. D., Nystrom, L. E., Jonides, J., Smith, E. E., & Noll, D. C. (1997). A
786
parametric study of prefrontal cortex involvement in human working memory.
787
NeuroImage, 5(1), 4962. https://doi.org/10.1006/nimg.1996.0247
788
Chomsky, N. (1957). The structure of language. The Hague: Mouton. Retrieved from
789
citeulike-article-id:931338
790
Conway, C. M., & Christiansen, M. H. (2001). Sequential learning in non-human primates.
791
Trends in Cognitive Sciences, 5(12), 539546. https://doi.org/10.1016/S1364-
792
6613(00)01800-3
793
Dehaene, S., Meyniel, F., Wacongne, C., Wang, L., & Pallier, C. (2015). The Neural
794
Representation of Sequences: From Transition Probabilities to Algebraic Patterns and
795
Linguistic Trees. Neuron, 88(1), 219. https://doi.org/10.1016/j.neuron.2015.09.019
796
Doyon, J., Bellec, P., Amsel, R., Penhune, V., Monchi, O., Carrier, J., … Benali, H. (2009).
797
Contributions of the basal ganglia and functionally related brain structures to motor
798
learning. Behavioural Brain Research, 199(1), 6175.
799
https://doi.org/10.1016/j.bbr.2008.11.012
800
38
Elsinger, C. L., Harrington, D. L., & Rao, S. M. (2006). From preparation to online control:
801
Reappraisal of neural circuitry mediating internally generated and externally guided
802
actions. NeuroImage, 31(3), 11771187.
803
https://doi.org/10.1016/j.neuroimage.2006.01.041
804
Everaert, M. B. H., Huybregts, M. A. C., Chomsky, N., Berwick, R. C., & Bolhuis, J. J.
805
(2015). Structures, Not Strings: Linguistics as Part of the Cognitive Sciences. Trends in
806
Cognitive Sciences, 19(12), 729743. https://doi.org/10.1016/j.tics.2015.09.008
807
Fadiga, L., Craighero, L., & D’Ausilio, A. (2009). Broca’s area in language, action, and
808
music. Annals of the New York Academy of Sciences, 1169(1), 448458.
809
https://doi.org/10.1111/j.1749-6632.2009.04582.x
810
Fazio, P., Cantagallo, A., Craighero, L., D’ausilio, A., Roy, A. C., Pozzo, T., … Fadiga, L.
811
(2009). Encoding of human action in Broca’s area. Brain, 132(7), 19801988.
812
https://doi.org/10.1093/brain/awp118
813
Fedorenko, E., Duncan, J., & Kanwisher, N. (2012). Language-Selective and Domain-
814
General Regions Lie Side by Side within Broca ’ s Area. Current Biology, 22(21),
815
20592062. https://doi.org/10.1016/j.cub.2012.09.011
816
Fischmeister, F. P., Martins, M. D., Beisteiner, R., & Fitch, W. T. (2017). Self-similarity and
817
Recursion as Default Modes in Human Cognition. Cortex, 97, 183201.
818
Fitch, W. T., Friederici, A. D., & Hagoort, P. (2012). Pattern perception and computational
819
complexity: introduction to the special issue. Philosophical Transactions of the Royal
820
Society B: Biological Sciences, 367(1598), 19251932.
821
https://doi.org/10.1098/rstb.2012.0099
822
Fitch, W. T., & Martins, M. D. (2014). Hierarchical processing in music, language, and
823
action: Lashley revisited. Annals of the New York Academy of Sciences, 1316(1), 87
824
39
104. https://doi.org/10.1111/nyas.12406
825
Friederici, A. D., Bahlmann, J., Friedrich, R., & Makuuchi, M. (2011). The Neural Basis of
826
Recursion and Complex Syntactic Hierarchy. Biolinguistics, 5(12), 87104.
827
Geambasu, A., Ravignani, A., & Levelt, C. C. (2016). Preliminary experiments on human
828
sensitivity to rhythmic structure in a grammar with recursive self-similarity. Frontiers in
829
Neuroscience, 10(June), 17. https://doi.org/10.3389/fnins.2016.00281
830
Hardwick, R. M., Rottschy, C., Miall, R. C., & Eickhoff, S. B. (2013). A quantitative meta-
831
analysis and review of motor learning in the human brain. NeuroImage, 67, 283297.
832
https://doi.org/10.1016/j.neuroimage.2012.11.020
833
Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of language: what is it, who
834
has it, and how did it evolve? Science, 298(5598), 15691579.
835
https://doi.org/10.1126/science.298.5598.1569
836
Hétu, S., Grégoire, M., Saimpont, A., Coll, M. P., Eugène, F., Michon, P. E., & Jackson, P.
837
L. (2013). The neural network of motor imagery: An ALE meta-analysis. Neuroscience
838
and Biobehavioral Reviews, 37(5), 930949.
839
https://doi.org/10.1016/j.neubiorev.2013.03.017
840
Humphries, M. D., & Gurney, K. N. (2002). The role of intra-thalamic and thalamocortical
841
circuits in action selection. Network: Computation in Neural Systems, 13(1), 131156.
842
https://doi.org/10.1080/net.13.1.131.156
843
Jackendoff, R., & Lerdahl, F. (2006). The capacity for music: What is it, and What’s specical
844
about it? Cognition, 100(1), 3372. https://doi.org/10.1016/j.cognition.2005.11.005
845
Jackson, P. L., Lafleur, M. F., Malouin, F., Richards, C. L., & Doyon, J. (2003). Functional
846
cerebral reorganization following motor sequence learning through mental practice with
847
motor imagery. NeuroImage, 20(2), 11711180. https://doi.org/10.1016/S1053-
848
40
8119(03)00369-0
849
Jeon, H.-A. (2014). Hierarchical processing in the prefrontal cortex in a variety of cognitive
850
domains. Frontiers in Systems Neuroscience, 8(November), 223.
851
https://doi.org/10.3389/fnsys.2014.00223
852
Koechlin, E., & Jubault, T. (2006). Broca’s Area and the Hierarchical Organization of
853
Human Behavior. Neuron, 50(6), 963974. https://doi.org/10.1016/j.neuron.2006.05.017
854
Koechlin, E., & Summerfield, C. (2007). An information theoretical approach to prefrontal
855
executive function. Trends in Cognitive Sciences, 11(6), 229235.
856
https://doi.org/10.1016/j.tics.2007.04.005
857
Lashley, K. S. (1951). The problem of serial order in behavior. In L. A. Jeffress (Ed.),
858
Cerebral mechanisms in behavior (pp. 112131). New York: Wiley.
859
https://doi.org/10.1093/rfs/hhq153
860
Lerdahl, F., & Jackendoff, R. (1977). Toward a Formal Theory of Tonal Music. Journal of
861
Music Theory, 21(1), 111171.
862
Lindenmayer, A. (1968). Mathematical models for cellular interactions in development I.
863
Filaments with one-sided inputs. Journal of Theoretical Biology, 18(3), 280299.
864
https://doi.org/10.1016/0022-5193(68)90079-9
865
Lungu, O., Monchi, O., Albouy, G., Jubault, T., Ballarin, E., Burnod, Y., & Doyon, J. (2014).
866
Striatal and Hippocampal Involvement in Motor Sequence Chunking Depends on the
867
Learning Strategy, 9(8), 2527. https://doi.org/10.1371/journal.pone.0103885
868
Maess, B., Koelsch, S., Gunter, T. C., & Friederici, A. D. (2001). Musical syntax is processed
869
in Broca’s area: an MEG study. Nature Neuroscience, 4(5), 540545.
870
https://doi.org/10.1038/87502
871
41
Martins, M. D. (2012). Distinctive signatures of recursion. Philosophical Transactions of the
872
Royal Society B: Biological Sciences, 367(1598), 20552064.
873
https://doi.org/10.1098/rstb.2012.0097
874
Martins, M. D. (2017). The Cognitive Architecture of Recursion: Behavioral and fMRI
875
Evidence from the Visual, Musical and Motor Domains. In Proceedings of the 39th
876
Annual Conference of the Cognitive Science Society. London.
877
Martins, M. D., Fischmeister, F. P., Puig-Waldmüller, E., Oh, J., Geißler, A., Robinson, S.,
878
Beisteiner, R. (2014). Fractal image perception provides novel insights into
879
hierarchical cognition. NeuroImage, 96, 300308.
880
https://doi.org/10.1016/j.neuroimage.2014.03.064
881
Martins, M. D., & Fitch, W. T. (2014). Investigating Recursion Within a Domain-Specific
882
Framework. In F. L. & L. Lefebre (Ed.), Language and Recursion (pp. 1526). New
883
York: Springer. Retrieved from http://link.springer.com.ezp-
884
prod1.hul.harvard.edu/chapter/10.1007/978-1-4614-9414-0%7B_%7D2
885
Martins, M. D., Gingras, B., Puig-Waldmueller, E., & Fitch, W. T. (2017). Cognitive
886
representation of “musical fractals”: Processing hierarchy and recursion in the auditory
887
domain. Cognition, 161. https://doi.org/10.1016/j.cognition.2017.01.001
888
Martins, M. D., Martins, I. P., & Fitch, W. T. (2015). A novel approach to investigate
889
recursion and iteration in visual hierarchical processing. Behavior Research Methods, 1
890
22. https://doi.org/10.3758/s13428-015-0657-1
891
Martins, M. D., Martins, I. P., & Fitch, W. T. (2016). A novel approach to investigate
892
recursion and iteration in visual hierarchical processing. Behavior Research Methods,
893
48(4). https://doi.org/10.3758/s13428-015-0657-1
894
Matchin, W. G., Hammerly, C., & Lau, E. (2017). The role of the IFG and pSTS in syntactic
895
42
prediction: Evidence from a parametric study of hierarchical structure in fMRI. Cortex,
896
88, 106123. https://doi.org/10.1016/j.cortex.2016.12.010
897
Miyachi, S., Hikosaka, O., Miyashita, K., Kárádi, Z., & Rand, M. K. (1997). Differential
898
roles of monkey striatum in learning of sequential hand movement. Experimental Brain
899
Research, 115(1), 15. https://doi.org/10.1007/PL00005669
900
Molnar-szakacs, I., Iacoboni, M., & Koski, L. (2005). Functional Segregation within Pars
901
Opercularis of the Inferior Frontal Gyrus : Evidence from fMRI Studies of Imitation and
902
Action Observation, (July), 986994. https://doi.org/10.1093/cercor/bhh199
903
Müllensiefen, D., Gingras, B., Musil, J., & Stewart, L. (2014). The musicality of non-
904
musicians: An index for assessing musical sophistication in the general population.
905
PLoS ONE. https://doi.org/10.1371/journal.pone.0089642
906
Neubert, F. X., Mars, R. B., Thomas, A. G., Sallet, J., & Rushworth, M. F. S. (2014).
907
Comparison of Human Ventral Frontal Cortex Areas for Cognitive Control and
908
Language with Areas in Monkey Frontal Cortex. Neuron, 81(3).
909
https://doi.org/10.1016/j.neuron.2013.11.012
910
Novick, J. M., Trueswell, J. C., & Thompson-Schill, S. L. (2005). Cognitive control and
911
parsing: reexamining the role of Broca’s area in sentence comprehension. Cognitive,
912
Affective & Behavioral Neuroscience, 5(3), 263281.
913
https://doi.org/10.3758/CABN.5.3.263
914
Oechslin, M. S., Gschwind, M., & James, C. E. (2017). Tracking Training-Related Plasticity
915
by Combining fMRI and DTI : The Right Hemisphere Ventral Stream Mediates Musical
916
Syntax Processing, 110. https://doi.org/10.1093/cercor/bhx033
917
Opitz, B., & Friederici, A. D. (2003). Interactions of the hippocampal system and the
918
prefrontal cortex in learning language-like rules. NeuroImage, 19(4), 17301737.
919
43
https://doi.org/10.1016/S1053-8119(03)00170-8
920
Pallier, C., Devauchelle, A.-D., & Dehaene, S. (2011). Cortical representation of the
921
constituent structure of sentences. Proceedings of the National Academy of Sciences of
922
the United States of America, 108(6), 25222527.
923
https://doi.org/10.1073/pnas.1018711108
924
Patel, A. D. (2003). Language, music, syntax and the brain. Nature Neuroscience, 6(7), 674
925
681. https://doi.org/10.1038/nn1082
926
Rilling, J. K., Glasser, M. F., Preuss, T. M., Ma, X., Zhao, T., Hu, X., & Behrens, T. E. J.
927
(2008). The evolution of the arcuate fasciculus revealed with comparative DTI. Nature
928
Neuroscience, 11(4), 426428. https://doi.org/10.1038/nn2072
929
Rodd, J. M., Vitello, S., Woollams, A. M., & Adank, P. (2015). Localising semantic and
930
syntactic processing in spoken and written language comprehension: An Activation
931
Likelihood Estimation meta-analysis. Brain and Language, 141, 89102.
932
https://doi.org/10.1016/j.bandl.2014.11.012
933
Sammler, D., Koelsch, S., Ball, T., Brandt, A., Grigutsch, M., Huppertz, H., … Schulze-
934
bonhage, A. (2013). NeuroImage Co-localizing linguistic and musical syntax with
935
intracranial EEG. NeuroImage, 64, 134146.
936
https://doi.org/10.1016/j.neuroimage.2012.09.035
937
Sammler, D., Koelsch, S., & Friederici, A. D. (2011). Are left fronto-temporal brain areas a
938
prerequisite for normal music-syntactic processing? Cortex, 47(6), 659673.
939
https://doi.org/10.1016/j.cortex.2010.04.007
940
Schenker, N. M., Hopkins, W. D., Spocter, M. A., Garrison, A. R., Stimpson, C. D., Erwin, J.
941
M., … Sherwood, C. C. (2010). Broca’s area homologue in chimpanzees (Pan
942
troglodytes): Probabilistic mapping, asymmetry, and comparison to humans. Cerebral
943
44
Cortex, 20(3), 730742. https://doi.org/10.1093/cercor/bhp138
944
Schroll, H., Vitay, J., & Hamker, F. H. (2012). Working memory and response selection: A
945
computational account of interactions among cortico-basalganglio-thalamic loops.
946
Neural Networks, 26, 5974. https://doi.org/10.1016/j.neunet.2011.10.008
947
Taubert, M., Draganski, B., Anwander, A., Muller, K., Horstmann, A., Villringer, A., &
948
Ragert, P. (2010). Dynamic Properties of Human Brain Structure: Learning-Related
949
Changes in Cortical Areas and Associated Fiber Connections. Journal of Neuroscience,
950
30(35), 1167011677. https://doi.org/10.1523/JNEUROSCI.2567-10.2010
951
Taubert, M., Lohmann, G., Margulies, D. S., Villringer, A., & Ragert, P. (2011). Long-term
952
effects of motor training on resting-state networks and underlying brain structure.
953
NeuroImage, 57(4), 14921498. https://doi.org/10.1016/j.neuroimage.2011.05.078
954
Udden, J., Martins, M. D., Fitch, W. T., & Zuidema, W. (n.d.). Hierarchical structure in
955
sequence processing: how do we measure it and what’s the neural implementation?
956
Topics in Cognitive Science.
957
van der v.d. Hulst, H. (2010). Re Recursion. In H. van der Hulst (Ed.), Recursion and Human
958
Language (pp. xv--liii). Berlin, Boston: de Gruyter Mouton.
959
https://doi.org/10.1515/9783110219258
960
Verwey, W. B. (2001). Concatenating familiar movement sequences: The versatile cognitive
961
processor. Acta Psychologica, 106(12), 6995. https://doi.org/10.1016/S0001-
962
6918(00)00027-5
963
Verwey, W. B., Shea, C. H., & Wright, D. L. (2014). A cognitive framework for explaining
964
serial processing and sequence execution strategies . Psychological Bulletin, 159.
965
https://doi.org/10.3758/s13423-014-0773-4
966
Vitay, J. (2010). A computational model of basal ganglia and its role in memory retrieval in
967
45
rewarded visual memory tasks. Frontiers in Computational Neuroscience, 4(May), 1
968
18. https://doi.org/10.3389/fncom.2010.00013
969
Vitello, S., & Rodd, J. M. (2015). Resolving Semantic Ambiguities in Sentences: Cognitive
970
Processes and Brain Mechanisms. Language and Linguistics Compass, 9(10), 391405.
971
https://doi.org/10.1111/lnc3.12160
972
Wiestler, T., & Diedrichsen, J. (2013). Skill learning strengthens cortical representations of
973
motor sequences, 120. https://doi.org/10.7554/eLife.00801
974
975
976
977
978
979
... However, by handling hierarchical sequential elements, syntactic processes allow us to understand such complex structures. Studies suggest that action and language share syntactic processes (17)(18)(19)(20)(21). Actions indeed involve hierarchies of interdependent subcomponents within an entire motor sequence (22)(23)(24)(25). Dexterous tool use in particular implies incorporating an external object (1). ...
... These findings bolster the hypothesis of a supramodal syntactic function serving both action and language (20,25), which is consistent with the documented role of the dorsal striatum in processing complex hierarchical structures in both the motor (24) and linguistic (33) domains. The dorsal striatum supports a wide range of procedural learning processes across several species (44)(45)(46) and tasks (24,47). This part of the procedural system is involved ...
... The writer that admires the poet writes the paper The writer that the poet admires writes the paper in syntactic training (47) and in the implementation of grammatical rules (21,33). Furthermore, it acts as a parser of actions to chunk motor sequences (24,48). Accurate and efficient tool use requires embedding an external object into the motor sequence and thus relies more on the striatum than on manual actions to parse the motor primitives (4). ...
Article
Does tool use share syntactic processes with language? Acting with a tool is thought to add a hierarchical level into the motor plan. In the linguistic domain, syntax is the cognitive function handling interdependent elements. Using functional magnetic resonance imaging, we detected common neurofunctional substrates in the basal ganglia subserving both tool use and syntax in language. The two abilities elicited similar patterns of neural activity, indicating the existence of shared functional resources. Manual actions and verbal working memory did not contribute to this common network. Consistent with the existence of shared neural resources, we observed bidirectional behavioral enhancement of tool use and syntactic skills in language so that training one function improves performance in the other. This reveals supramodal syntactic processes for tool use and language.
... An explicit learning session was adopted to help participants grasp the rules for solving the grammaticality judgment task in the scanner for each grammar. Therefore, the current study focused on how participants applied these rules, once they successfully passed the learning session, to process the structures in the scanning session (see also Bahlmann et al., 2008Bahlmann et al., , 2009Jeon & Friederici, 2013;Martins, Bianco, Sammler, & Villringer, 2019;Ohta et al., 2013). ...
... The higher drop-out rate for the HG is due to this grammar's inherent structural complexity. Crucially, the same outcome was also reported in Martins et al. (2019), who had a higher drop-out rate (50%) for learning the most complex grammar with a criterion that was much looser (accuracy of the last learning session [containing 20 trials] >80%) than that of our current study. In line with their study, we set the drop-out learners aside and only included those HG learners who could successfully apply the HG rules for the corresponding task after a relatively short learning time (1 hr), similar to the successful NG learners. ...
... (c), error bar shows the SEM. A, anterior; AG, angular gyrus; aTL, anterior temporal lobe; BA, Brodmann area; gr, grammatical condition; HG, Hierarchical syntactic structure-building Grammar; I, inferior; IFG, inferior frontal gyrus; IFGorb, inferior frontal pars orbitalis; NG, Nested associating Grammar; P, posterior; pSTG, posterior superior temporal gyrus; pTL, posterior temporal lobe; S, superior; ungr, ungrammatical condition adopted two separate anatomical masks for BA 44 and BA 45 (Amunts et al., 1999) to investigate IFG at a finer-grained resolution (see also Martins et al., 2019;Matchin et al., 2017). ...
Article
Full-text available
Grammar is central to any natural language. In the past decades, the artificial grammar of the AⁿBⁿ type in which a pair of associated elements can be nested in the other pair was considered as a desirable model to mimic human language syntax without semantic interference. However, such a grammar relies on mere associating mechanisms, thus insufficient to reflect the hierarchical nature of human syntax. Here, we test how the brain imposes syntactic hierarchies according to the category relations on linearized sequences by designing a novel artificial “Hierarchical syntactic structure‐building Grammar” (HG), and compare this to the AⁿBⁿ grammar as a “Nested associating Grammar” (NG) based on multilevel associations. Thirty‐six healthy German native speakers were randomly assigned to one of the two grammars. Both groups performed a grammaticality judgment task on auditorily presented word sequences generated by the corresponding grammar in the scanner after a successful explicit behavioral learning session. Compared to the NG group, we found that the HG group showed a (a) significantly higher involvement of Brodmann area (BA) 44 in Broca's area and the posterior superior temporal gyrus (pSTG); and (b) qualitatively distinct connectivity between the two regions. Thus, the present study demonstrates that the build‐up process of syntactic hierarchies on the basis of category relations critically relies on a distinctive left‐hemispheric syntactic network involving BA 44 and pSTG. This indicates that our novel artificial grammar can constitute a suitable experimental tool to investigate syntax‐specific processes in the human brain.
... inferior/middle frontal gyrus (IFG / MFG), as well as further posterior into inferior/superior parietal lobule (IPL / SPL) (Koechlin and Jubault 2006;Bianco, Novembre, Keller, Kim, et al. 2016;Yokoi and Diedrichsen 2019). However, the highest representational levels under investigation typically lack the flexible rule-based arrangement of single acts and movements outlined above, but remain limited to fixed combinations of motor chunks that precisely map onto well-learned sequences of finger movements (Hikosaka et al. 2002;Koechlin and Jubault 2006;Doyon 2008;Martins et al. 2019;Yokoi and Diedrichsen 2019). Exactly how the brain generates novel sequences based on abstract combinatorial rules and which neural networks underlie this ability remains largely unexplored (Ahlheim et al. 2016;Martins et al. 2019), despite the importance of flexible, non-habitual action sequences for people's everyday life and communication (De Renzi and Lucchelli 1988;Clerget et al. 2009;Foundas and Duncan 2019). ...
... However, the highest representational levels under investigation typically lack the flexible rule-based arrangement of single acts and movements outlined above, but remain limited to fixed combinations of motor chunks that precisely map onto well-learned sequences of finger movements (Hikosaka et al. 2002;Koechlin and Jubault 2006;Doyon 2008;Martins et al. 2019;Yokoi and Diedrichsen 2019). Exactly how the brain generates novel sequences based on abstract combinatorial rules and which neural networks underlie this ability remains largely unexplored (Ahlheim et al. 2016;Martins et al. 2019), despite the importance of flexible, non-habitual action sequences for people's everyday life and communication (De Renzi and Lucchelli 1988;Clerget et al. 2009;Foundas and Duncan 2019). ...
Preprint
Full-text available
Complex sequential behaviours, such as speaking or playing music, often entail the flexible, rule-based chaining of single acts. However, it remains unclear how the brain translates abstract structural rules into concrete series of movements. Here we demonstrate a multi-level contribution of anatomically distinct cognitive and motor networks to the execution of novel musical sequences. We combined functional and diffusion-weighted neuroimaging to dissociate high-level structural and low-level motor planning of musical chord sequences executed on a piano. Fronto-temporal and fronto-parietal neural networks were involved when sequences violated pianists’ structural or motor plans, respectively. Prefrontal cortex is identified as a hub where both networks converge within an anterior-to-posterior gradient of action control linking abstract structural rules to concrete movement sequences.
... For example, it is controversial whether structured sequencing is supported by a single, domain-general mechanism [49][50][51] or by parallel computations in multiple, modality-or task-specific systems 9,52 . Similarly, it remains unclear to what extent mechanisms for sequence perception overlap with those involved in sequence production 53 . Such unresolved questions hinder attempts to determine if observed species differences in sequence learning reflect general cognitive constraints or the particular sensory, motor, and motivational features of different experimental paradigms 40,54 . ...
Article
Full-text available
Human behaviors from toolmaking to language are thought to rely on a uniquely evolved capacity for hierarchical action sequencing. Testing this idea will require objective, generalizable methods for measuring the structural complexity of real-world behavior. Here we present a data-driven approach for extracting action grammars from basic ethograms, exemplified with respect to the evolutionarily relevant behavior of stone toolmaking. We analyzed sequences from the experimental replication of ~ 2.5 Mya Oldowan vs. ~ 0.5 Mya Acheulean tools, finding that, while using the same “alphabet” of elementary actions, Acheulean sequences are quantifiably more complex and Oldowan grammars are a subset of Acheulean grammars. We illustrate the utility of our complexity measures by re-analyzing data from an fMRI study of stone toolmaking to identify brain responses to structural complexity. Beyond specific implications regarding the co-evolution of language and technology, this exercise illustrates the general applicability of our method to investigate naturalistic human behavior and cognition.
... For example, it is controversial whether structured sequencing is supported by a single, domain-general mechanism [49][50][51] or by parallel computations in multiple, modality-or task-specific systems 9,52 . Similarly, it remains unclear to what extent mechanisms for sequence perception overlap with those involved in sequence production 53 . Such unresolved questions hinder attempts to determine if observed species differences in sequence learning reflect general cognitive constraints or the particular sensory, motor, and motivational features of different experimental paradigms 40,54 . ...
Preprint
Full-text available
Human behaviors from tool-making to language are thought to rely on a uniquely evolved capacity for hierarchical action sequencing. Testing this idea will require objective, generalizable methods for measuring the structural complexity of real-world behavior. Here we present a data-driven approach for extracting action grammars from basic ethograms, exemplified with respect to the evolutionarily-relevant behavior of stone tool-making. We analyzed sequences from the experimental replication of ~2.5 Mya Oldowan vs. ~0.5 Mya Acheulean tools, finding that, while using the same “alphabet” of elementary actions, Acheulean sequences are quantifiably more complex and Oldowan grammars are a subset of Acheulean grammars. We illustrate the utility of our complexity measures by re-analyzing data from an fMRI study of stone tool-making to identify brain responses to structural complexity. Beyond specific implications regarding the co-evolution of language and technology, this exercise illustrates the general applicability of our method to investigate naturalistic human behavior and cognition.
... Different linguistic functions can be allocated to different structures and cortical networks (Friederici, 2011). Of central relevance here is Broca's area, given that its involvement in different cognitive domains has brought to the question of whether language is a unique and unprecedented human faculty or whether it shares evolutionary and/or structural properties with other human faculties (Binkofski & Buccino, 2004;Grafton & Hamilton, 2007;Leslie, Johnson-Frey, & Grafton, 2004;Martins, Bianco, Sammler, & Villringer, 2019;Nishitani, Schürmann, Amunts, & Hari, 2005;Wakita, 2014). At the cytoarchitectonic level, Broca's area can be differentiated in two neighboring regions: Brodmann Areas (BA) 44 and 45 (Amunts & Zilles, 2012;Brodmann, 1909). ...
Article
Full-text available
Actions have been proposed to follow hierarchical principles similar to those hypothesized for language syntax. These structural similarities are claimed to be reflected in the common involvement of certain neural populations of Broca’s area, in the Inferior Frontal Gyrus (IFG). In this position paper, we follow an influential hypothesis in linguistic theory to introduce the syntactic operation Merge and the corresponding motor/conceptual interfaces. We argue that actions hierarchies do not follow the same principles ruling language syntax. We propose that hierarchy in the action domain lies in predictive processing mechanisms mapping sensory inputs and statistical regularities of action-goal relationships. At the cortical level, distinct Broca’s subregions appear to support different types of computations across the two domains. We argue that anterior BA44 is a major hub for the implementation of the syntactic operation Merge. On the other hand, posterior BA44 is recruited in selecting premotor mental representations based on the information provided by contextual signals. This functional distinction is corroborated by a recent meta-analysis (Papitto, Friederici, & Zaccarella, 2020). We conclude by suggesting that action and language can meet only where the interfaces transfer abstract computations either to the external world or to the internal mental world.
... Historically, the greatest barrier to domain-nonspecific models of language use has been recursion [6,204]. Recursive sequence learning and production is not, however, limited to language; visual processing [205] and motor planning [206,207] are now known to be both recursive and independent of language. These earlier-evolving systems may have been co-opted for language processing, a co-option supported by comparative phenotypic analysis of FoxP mutations [208]. ...
Article
Full-text available
Meaning has traditionally been regarded as a problem for philosophers and psychologists. Advances in cognitive science since the early 1960s, however, broadened discussions of meaning, or more technically, the semantics of perceptions, representations, and/or actions, into biology and computer science. Here, we review the notion of “meaning” as it applies to living systems, and argue that the question of how living systems create meaning unifies the biological and cognitive sciences across both organizational and temporal scales.
Preprint
Full-text available
In this article, we explore the extraction of recursive nested structure in the processing of binary sequences. Our aim was to determine whether the brain learns the higher order regularities of a highly simplified input where only sequential order information marks the hierarchical structure. To this end, we implemented sequence generated by the Fibonacci grammar in a serial reaction time task. This deterministic grammar generates aperiodic but self-similar sequences. The combination of these two properties allowed us to evaluate hierarchical learning while controlling for the use of low-level strategies like detecting recurring patterns. The deterministic aspect of the grammar allowed us to predict precisely which points in the sequence should be subject to anticipation. Results showed that participants' pattern of anticipation could not be accounted for by "flat" statistical learning processes and was consistent with them anticipating upcoming points based on hierarchical assumptions. We also found that participants were sensitive to the structure constituency, suggesting that they organized the signal into embedded constituents. We hypothesized that the participants built this structure by merging recursively deterministic transitions.
Article
Several previous authors have proposed a kind of specious or subjective present moment that covers a few seconds of recent information. This article proposes a new hypothesis about the subjective present, renamed the extended present, defined not in terms of time covered but as a thematically connected information structure held in working memory and in transiently accessible form in long-term memory. The three key features of the extended present are that information in it is thematically connected, both internally and to current attended perceptual input, it is organised in a hierarchical structure, and all information in it is marked with temporal information, specifically ordinal and duration information. Temporal boundaries to the information structure are determined by hierarchical structure processing and by limits on processing and storage capacity. Supporting evidence for the importance of hierarchical structure analysis is found in the domains of music perception, speech and language processing, perception and production of goal-directed action, and exact arithmetical calculation. Temporal information marking is also discussed and a possible mechanism for representing ordinal and duration information on the time scale of the extended present is proposed. It is hypothesised that the extended present functions primarily as an informational context for making sense of current perceptual input, and as an enabler for perception and generation of complex structures and operations in language, action, music, exact calculation, and other domains.
Article
Full-text available
Neuroscience has studied deductive reasoning over the last 20 years under the assumption that deductive inferences are not only de jure but also de facto distinct from other forms of inference. The objective of this research is to verify if logically valid deductions leave any cerebral electrical trait that is distinct from the trait left by non-valid deductions. 23 subjects with an average age of 20.35 years were registered with MEG and placed into a two conditions paradigm (100 trials for each condition) which each presented the exact same relational complexity (same variables and content) but had distinct logical complexity. Both conditions show the same electromagnetic components (P3, N4) in the early temporal window (250–525 ms) and P6 in the late temporal window (500–775 ms). The significant activity in both valid and invalid conditions is found in sensors from medial prefrontal regions, probably corresponding to the ACC or to the medial prefrontal cortex. The amplitude and intensity of valid deductions is significantly lower in both temporal windows (p = 0.0003). The reaction time was 54.37% slower in the valid condition. Validity leaves a minimal but measurable hypoactive electrical trait in brain processing. The minor electrical demand is attributable to the recursive and automatable character of valid deductions, suggesting a physical indicator of computational deductive properties. It is hypothesized that all valid deductions are recursive and hypoactive.
Article
Full-text available
In many domains of human cognition, hierarchically structured representations are thought to play a key role. In this paper, we start with some foundational definitions of key phenomena like “sequence” and “hierarchy," and then outline potential signatures of hierarchical structure that can be observed in behavioral and neuroimaging data. Appropriate behavioral methods include classic ones from psycholinguistics along with some from the more recent artificial grammar learning and sentence processing literature. We then turn to neuroimaging evidence for hierarchical structure with a focus on the functional MRI literature. We conclude that, although a broad consensus exists about a role for a neural circuit incorporating the inferior frontal gyrus, the superior temporal sulcus, and the arcuate fasciculus, considerable uncertainty remains about the precise computational function(s) of this circuitry. An explicit theoretical framework, combined with an empirical approach focusing on distinguishing between plausible alternative hypotheses, will be necessary for further progress.
Article
Full-text available
The human ability to process hierarchical structures has been a longstanding research topic. However, the nature of the cognitive machinery underlying this faculty remains controversial. Recursion, the ability to embed structures within structures of the same kind, has been proposed as a key component of our ability to parse and generate complex hierarchies. Here, we investigated the cognitive representation of both recursive and iterative processes in the auditory domain. The experiment used a two-alternative forced-choice paradigm: participants were exposed to three-step processes in which pure-tone sequences were built either through recursive or iterative processes, and had to choose the correct completion. Foils were constructed according to generative processes that did not match the previous steps. Both musicians and non-musicians were able to represent recursion in the auditory domain, although musicians performed better. We also observed that general ‘musical’ aptitudes played a role in both recursion and iteration, although the influence of musical training was somehow independent from melodic memory. Moreover, unlike iteration, recursion in audition was well correlated with its non-auditory (recursive) analogues in the visual and action sequencing domains. These results suggest that the cognitive machinery involved in establishing recursive representations is domain-general, even though this machinery requires access to information resulting from domain-specific processes.
Article
Full-text available
We present the first rhythm detection experiment using a Lindenmayer grammar, a self-similar recursive grammar shown previously to be learnable by adults using speech stimuli. Results show that learners were unable to correctly accept or reject grammatical and ungrammatical strings at the group level, although five (of 40) participants were able to do so with detailed instructions before the exposure phase.
Article
Full-text available
There are many questions one can ask about human language: its distinctive properties, neural representation, characteristic uses including use in communicative contexts, variation, growth in the individual, and origin. Every such inquiry is guided by some concept of what 'language' is. Sharpening the core question - what is language? - and paying close attention to the basic property of the language faculty and its biological foundations makes it clear how linguistics is firmly positioned within the cognitive sciences. Here we will show how recent developments in generative grammar, taking language as a computational cognitive mechanism seriously, allow us to address issues left unexplained in the increasingly popular surface-oriented approaches to language.
Article
Full-text available
Linguistic analyses suggest that sentences are not mere strings of words but possess a hierarchical structure with constituents nested inside each other. We used functional magnetic resonance imaging (fMRI) to search for the cerebral mechanisms of this theoretical construct. We hypothesized that the neural assembly that encodes a constituent grows with its size, which can be approximately indexed by the number of words it encompasses. We therefore searched for brain regions where activation increased parametri-cally with the size of linguistic constituents, in response to a visual stream always comprising 12 written words or pseudowords. The results isolated a network of left-hemispheric regions that could be dissociated into two major subsets. Inferior frontal and posterior temporal regions showed constituent size effects regardless of whether actual content words were present or were replaced by pseudowords (jabberwocky stimuli). This observation suggests that these areas operate autonomously of other language areas and can extract abstract syntactic frames based on function words and morphological information alone. On the other hand, regions in the temporal pole, anterior superior temporal sulcus and temporo-parietal junction showed constituent size effect only in the presence of lexico-semantic information, suggesting that they may encode semantic constituents. In several inferior frontal and superior temporal regions, activation was delayed in response to the largest constituent structures, suggesting that nested linguistic structures take increasingly longer time to be computed and that these delays can be measured with fMRI.
Article
As a functional homolog for left-hemispheric syntax processing in language, neuroimaging studies evidenced involvement of right prefrontal regions in musical syntax processing, of which underlying white matter connectivity remains unexplored so far. In the current experiment, we investigated the underlying pathway architecture in subjects with 3 levels of musical expertise. Employing diffusion tensor imaging tractography, departing from seeds from our previous functional magnetic resonance imaging study on music syntax processing in the same participants, we identified a pathway in the right ventral stream that connects the middle temporal lobe with the inferior frontal cortex via the extreme capsule, and corresponds to the left hemisphere ventral stream, classically attributed to syntax processing in language comprehension. Additional morphometric consistency analyses allowed dissociating tract core from more dispersed fiber portions. Musical expertise related to higher tract consistency of the right ventral stream pathway. Specifically, tract consistency in this pathway predicted the sensitivity for musical syntax violations. We conclude that enduring musical practice sculpts ventral stream architecture. Our results suggest that training-related pathway plasticity facilitates the right hemisphere ventral stream information transfer, supporting an improved sound-to-meaning mapping in music.
Article
The ability to predict upcoming structured events based on long-term knowledge and contextual priors is a fundamental principle of human cognition. Tonal music triggers predictive processes based on structural properties of harmony, i.e., regularities defining the arrangement of chords into well-formed musical sequences. While the neural architecture of structure-based predictions during music perception is well described, little is known about the neural networks for analogous predictions in musical actions and how they relate to auditory perception. To fill this gap, expert pianists were presented with harmonically congruent or incongruent chord progressions, either as musical actions (photos of a hand playing chords) that they were required to watch and imitate without sound, or in an auditory format that they listened to without playing. By combining task-based functional magnetic resonance imaging (fMRI) with functional connectivity at rest, we identified distinct sub-regions in right inferior frontal gyrus (rIFG) interconnected with parietal and temporal areas for processing action and audio sequences, respectively. We argue that the differential contribution of parietal and temporal areas is tied to motoric and auditory long-term representations of harmonic regularities that dynamically interact with computations in rIFG. Parsing of the structural dependencies in rIFG is co-determined by both stimulus- or task-demands. In line with contemporary models of prefrontal cortex organization and dual stream models of visual-spatial and auditory processing, we show that the processing of musical harmony is a network capacity with dissociated dorsal and ventral motor and auditory circuits, which both provide the infrastructure for predictive mechanisms optimising action and perception performance.
Article
In our TICS Review in 2004, we proposed that a sector of the right inferior frontal cortex (rIFC) in humans is critical for inhibiting response tendencies. Here we survey new evidence, discuss ongoing controversies, and provide an updated theory. We propose that the rIFC (along with one or more fronto-basal-ganglia networks) is best characterized as a brake. This brake can be turned on in different modes (totally, to outright suppress a response; or partially, to pause), and in different contexts (externally, by stop or salient signals; or internally, by goals). We affirm inhibition as a central component of executive control that relies upon the rIFC and associated networks, and explain why rIFC disruption could generally underpin response control disorders.