Content uploaded by Karl J Friston
Author content
All content in this area was uploaded by Karl J Friston
Content may be subject to copyright.
Noppeney et al.
1
Two distinct neural mechanisms
for category-selective responses
Uta Noppeney, Cathy J. Price, Will D. Penny, Karl J. Friston
Running title: Neural mechanisms of category-selectivity
Correspondence should be addressed to:
U. Noppeney
Wellcome Department of Imaging Neuroscience
University College London
12 Queen Square
WC1 N3BG London, UK
Tel.: +44-(20)-7833 7483
Fax.: +44-(20)-7813 1420
e-mail: u.noppeney@fil.ion.ucl.ac.uk
Noppeney et al.
2
Abstract
The cognitive and neural mechanisms mediating category-selective responses in the
human brain remain controversial. Using fMRI and effective connectivity analyses
(Dynamic Causal Modelling), we investigated animal and tool-selective responses by
manipulating stimulus modality (pictures vs. words) and task (implicit vs. explicit
semantic). We dissociated two distinct mechanisms that engender category-
selectivity: In the ventral occipito-temporal cortex, tool-selective responses were
observed irrespective of task, greater for pictures and mediated by bottom-up effects.
In a left temporo-parietal action system, tool-selective responses were observed
irrespective of modality, greater for explicit semantic tasks and mediated by top-down
modulation from the left prefrontal cortex. These distinct activation and connectivity
patterns suggest that the two systems support different cognitive operations, with the
ventral occipito-temporal regions engaged in structural processing and the dorsal
visuo-motor system in strategic semantic processing. Consistent with current
semantic theories, explicit semantic processing of tools might thus rely on re-
activating their associated action representations via top-down modulation. In terms
of neuronal mechanisms, the category-selectivity may be mediated by distinct top-
down (task-dependent) and bottom-up (stimulus-dependent) mechanisms.
Keywords: semantic memory, dynamic causal modelling, category-selectivity,
effective connectivity, functional imaging
Noppeney et al.
3
Introduction
A central question in cognitive neuroscience is how object concepts are represented
and processed in the human brain. Category-selective impairments in patients with
focal cortical lesions suggest specialized neuronal systems that are engaged by
different semantic categories such as animals and tools (Warrington and Shallice,
1984;Gainotti et al., 1995;Capitani et al., 2003). These category-selective deficits
have been found at multiple processing levels, ranging from structural to semantic
(Humphreys and Forde, 2001). Similarly, functional brain imaging studies have
reported category-selective activations in multiple cortical regions. Activations within
the fusiform gyrus have been found medially for tools and laterally for animals (Chao
et al., 1999). In addition, tools have been associated with activations in a visuo-motor
action system encompassing a left posterior middle temporal area (lpMT; Martin et
al., 1996;Devlin et al., 2002;Damasio et al., 1996;Kellenbach et al., 2002), the
anterior intraparietal sulcus (AIP; Chao and Martin, 2000) and the ventral premotor
cortex (Rizzolatti et al., 1996;Grabowski et al., 1998;Grafton et al., 1996). Despite
this extensive evidence for category-selective regions, the associated cognitive
processes and their neural implementation remain unclear.
The present fMRI study addressed two key questions: First, we asked whether
category-selective fMRI responses were differentially modulated by stimulus
modality (i.e. pictures vs. words) and/or task-context. Second, using effective
connectivity analyses (Dynamic Causal Modelling; DCM; Friston et al., 2003), we
investigated the neural mechanisms that mediate context-sensitive, category-selective
responses entailed by our first question.
Noppeney et al.
4
The aim of DCM is to make inferences about the coupling among brain areas within a
simple but reasonably realistic neuronal model. DCM is a generalization of the linear
convolution model
used in conventional analyses of regionally specific effects.
However, in a conventional analysis, the experimental effects are expressed through a
direct or extrinsic influence of experimental effects on each region. In contrast, DCM
tries to explain regional responses in terms of interactions among brain regions and,
critically, an effect of experimental manipulations on connections between brain
regions.
Subjects were engaged in a one-back-task on animals and tools that were presented as
pictures, written words or spoken words. The one-back-task used either implicit (i.e.
stimulus identity) or explicit semantic (i.e. typical action or real-life-size of the
stimulus) attributes. This design allowed us to segregate category-selective regions
into two classes: In one class, category-selectivity was stimulus modality-dependent
and observed primarily for pictures. In the other class, it was task-dependent and
observed when subjects were engaged in explicit semantic tasks.
Using DCM, we then investigated the neural mechanisms underlying category-
selectivity in two representative brain regions, one exhibiting modality-dependent, the
other one task-dependent tool-selective responses. The model included bottom-up
input from early visual areas and top-down influences from left prefrontal areas. This
allowed us to address the following three questions: (1) Are stimulus modality-
dependent tool-selective responses mediated by forward connections from early
visual areas that are enabled when tools are presented as pictures? (2) Are task-
dependent tool-selective activations mediated via increased backward influences from
left prefrontal regions during explicit semantic tasks? (3) Can the distinct category-
Noppeney et al.
5
selective activation patterns in the two modality- and task-dependent regions be
explained by differential modulation of forward or backward connections?
Noppeney et al.
6
Materials and Methods
Subjects
22 healthy right-handed English native speakers (14 males; mean age: 25; range: 19-
35) gave informed consent to participate in the study. All subjects had normal or
corrected to normal vision and no diagnosis of dyslexia. All of them reported good
reading abilities. 21 were either currently university students or reported having been
educated to degree level. The study was approved of by the joint ethics committee of
the Institute of Neurology and University College London Hospital, London, UK.
Experimental Design
The activation conditions conformed to a 2 x 3 x 3 factorial design manipulating
(i) Semantic category: animals or tools,
(ii) Stimulus modality: pictures, written words or spoken words. This manipulation
allowed us to investigate whether category-effects are elicited by verbal (written or
spoken words) and non-verbal (pictures) material. We hypothesized that verbal
material would elicit category-effects primarily at the semantic level, while pictures
would also induce category-effects at the structural level.
(iii) Task: Subjects were engaged in a one back-task and decided whether subsequent
stimuli within a block were identical (= implicit semantic task; e.g. sparrow, sparrow),
performed a similar action (=explicit action semantic task; e.g. stork, butterfly) or
were of similar size in real life (=explicit visual semantic task; e.g. pigeon, rabbit).
Irrespective of task, subjects were instructed to view the pictures, read the written
words silently and listen to the spoken words. While the identity task elicits only
stimulus-driven implicit semantic activations that are not required for task
Noppeney et al.
7
performance (see (Price et al., 1996)), we expected the action and size tasks to evoke
additional task-induced, strategic semantic activations necessary for explicit semantic
categorization.
Altogether, there were 90 animals and 90 tools that were matched for word frequency
and number of letters. Each stimulus was presented once in each stimulus modality
and during each task (i.e. three times during the entire experiment) yielding 270
animal and 270 tool events. ~30% of the stimuli were targets. As the identity task
inevitably required successive repetitions of the targets, 16 additional target stimuli
were used for the implicit condition to avoid repetition priming confounds. Yes/No
responses to all conditions were indicated (as quickly and as accurately as possible)
by a two-choice key press. The stimuli (SOA = 3.3 s; stimulus duration = 1.2 s) were
presented in blocks of 5 stimuli interleaved with 5.5 s fixation. The category and
modality factors were manipulated across the activation blocks, the task factor in long
periods covering one third of each session. The order of semantic conditions was
counterbalanced within and across subjects.
fMRI scanning
A 1.5 T Siemens Sonata system was used to acquire both T1 anatomical volume
images and T2*-weighted axial echoplanar images with blood oxygenation level-
dependent (BOLD) contrast (gradient echo, Cartesian k-space sampling, TE=50ms,
TR 2.97 s, 33 slices acquired sequentially in descending direction, matrix 64X64,
spatial resolution 3X3X3.4 mm
3
voxels, interslice gap 1.4 mm, slice thickness 2.0
mm, tilted from transverse to coronal orientation by –30 degree to reduce
susceptibility artefacts). To avoid Nyquist ghost artefacts a generalized reconstruction
algorithm was used for data processing (Josephs et al., 2000). There were three
Noppeney et al.
8
sessions with a total of 340 volume images per session. The first six volumes were
discarded to allow for T1 equilibration effects.
Conventional SPM analysis
The data were analysed with statistical parametric mapping (using SPM2 software
from the Wellcome Department of Imaging Neuroscience, London;
http//www.fil.ion.ucl.ac.uk/spm;Friston et al., 1995). Scans from each subject were
realigned using the first as a reference, spatially normalised into standard space
(Talairach and Tournoux, 1988), resampled to 3X3X3mm
3
voxels and spatially
smoothed with a Gaussian kernel of 8mm FWHM. The time series in each voxel were
highpass filtered to 1/128 Hz and globally normalized with proportional scaling. The
fMRI experiment was modelled in an event related fashion with regressors (i.e.
explanatory variables) made by convolving each event-related stimulus function with
a canonical hemodynamic response function and its first temporal derivative.
Stimulus functions were a series of delta or “stick” functions encoding the occurrence
of each trial type. In addition to modelling the 18 conditions in our 2 x 3 x 3 factorial
design, the statistical model included instructions, targets during the implicit
condition and non-responses. Covariates of no interest included the realignment
parameters (to account for motion artefacts). The analysis was performed twice: (i)
including all trials, (ii) including only the trials that were equated for reaction times
with respect to the main effect of category and the interaction between category and
modality/task. This involved excluding (1) tool trials with reaction times that were
1.25 std above the mean and (2) animal trials with reaction times that were 1.25 std
below the mean during the explicit semantic conditions (the excluded trials that
accounted for less than 10% of all trials in any subject, were modelled as an extra
Noppeney et al.
9
confounding effect). Condition-specific effects for each subject were estimated
according to the general linear model and passed to a second-level or subject-level
analysis as contrasts. Here each contrast was the estimated response for each
condition. This involved creating 18 contrast images (i.e each of the 18 conditions
averaged across the three sessions) for each subject and entering them into a second
level ANOVA. This ANOVA modelled the 18 effects in our 2 x 3 x 3 factorial design.
Inferences were made at the second level to allow a random effects analysis and
inferences at the population level (Friston et al., 1999).
The random effects analysis tested for the main effects of tools relative to animals and
animals relative to tools. Pooling over written and spoken words, we tested for the
interactions between category and stimulus modality i.e. tool or animal selective
responses that were increased or decreased for pictures relative to words. Pooling
over action and visual explicit semantic tasks, we tested for the interactions between
category and task i.e. tool or animal selective responses that were increased or
decreased for explicit relative to implicit tasks.
All effects were inclusively masked with all stimuli > baseline (at p<0.001 uncorr.).
The interactions were further characterized by inclusively masking each contrast with
(i) tools > animals or (ii) animals > tools (at p<0.001 uncorr.). Unless otherwise
stated, we only report activations that are significant (p<0.05) corrected for the entire
brain volume.
Noppeney et al.
10
DCM analysis
DCM treats the brain as a dynamic input-state-output system. The inputs correspond
to conventional stimulus functions encoding experimental manipulations. The state
variables are neuronal activities and the outputs are the regional hemodynamic
responses measured with fMRI. The idea is to model changes in the states, which
cannot be observed directly, using the known inputs and outputs. Critically, changes
in the states of one region depend on the states (i.e. activity) of others. This
dependency is parameterized by effective connectivity. There are three types of
parameters in a DCM (i) input parameters which describe how much brain regions
respond to experimental stimuli, (ii) intrinsic parameters that characterise effective
connectivity among regions and (iii) modulatory parameters that characterise changes
in effective connectivity caused by experimental manipulation. This third set of
parameters, the modulatory effects, allows us to explain context-sensitive category-
selective activations by changes in coupling among brain areas. Importantly, this
coupling (effective connectivity) is expressed at the level of neuronal states. DCM
employs a forward model, relating neuronal activity to fMRI data, that can be
inverted during the model fitting process. Put simply, the forward model is used to
predict outputs using the inputs. The parameters are adjusted [using gradient descent]
so that the predicted and observed outputs match. This adjustment corresponds to the
model-fitting.
22 subject-specific DCMs were constructed. The regions (see Table 3) were selected
using the maxima from the random effects analysis. The left posterior medial fusiform
and AIP were selected as representative regions for modality- and task-dependent
category-selectivity respectively. Region-specific time-series (concatenated over three
sessions and adjusted for confounds) comprised the first eigenvariate of all voxels
Noppeney et al.
11
within a 4 mm radius centred on each location. The DCM (Figure 4) included five
regions, (i) a left superior temporal area that was activated by spoken words relative
to fixation (STG), (ii) a left occipital region that was activated for both written words
and pictures and did not show any category-selectivity (OCC), (iii) a task-sensitive
left prefrontal region (PF), (iv) tool-selective AIP and (v) the tool-selective left
posterior medial fusiform area. Based on our scientific question (see introduction), the
tool-selective regions were selected that showed an interaction-effect i.e. we selected
a modality-dependent fusiform and a task-dependent AIP tool-selective region. In
other words, we used DCM to understand how regionally-specific interactions are
mediated by changes in coupling. Visual input (words
written
and pictures) was
connected to OCC, the auditory input (words
spoken
) to STG. The main effect of task
entered directly in the left prefrontal area. Tool pictures, tool words
written
and pictures
modulated or enabled the forward connections from OCC to the category-selective
regions. These three effects were chosen to cover the main effect of category,
stimulus modality and their interaction. Category-effects (tools in all modalities) were
entered to modulate the backward connection from PF to the category-selective
regions.
The subject-specific modulatory effects were entered into t-tests at the group level
(see Table 4). This allowed us to summarize the consistent findings from the subject-
specific DCMs using classical statistics. First, we tested whether tool pictures relative
to tool words increased the strength of forward connections (i.e. we tested for a
modulatory effect of the category x modality interaction on forward connections).
Second, we tested whether tools increased the backward connections from the left
prefrontal to the category-selective regions. As the left prefrontal response is caused
Noppeney et al.
12
primarily by the main effect of task, this effectively tests for a category x task
interaction mediated by backward connections. Finally, we tested for differences in
modulatory effects between connections to the fusiform and AIP regions using a
paired t-test. This allowed us to characterize category-selective effects in terms of a
differential enabling of dorsal and ventral pathways.
Noppeney et al.
13
Results
Behavioural results
For performance accuracy, a three-way ANOVA with category (tools, animals),
stimulus modality (pictures, spoken words, written words) and task (identity, action,
real life size) identified a significant main effect of stimulus modality
(F(1.7,36.5)=9.6; p < 0.01) and of task (F(1.8,39)=247; p < 0.001) after Greenhouse-
Geisser correction. Importantly, there was no significant effect of category or the
interactions between category and task/modality. For reaction times, the three-way
ANOVA identified (i) main effects of category (F(1,21)=66; p < 0.001), stimulus
modality (F(1.5,31)=597; p < 0.001) and task (F(1.7,35)=203; p < 0.001) and (ii)
significant interactions of category by stimulus modality (F(1.9,41)=7; p < 0.01) and
category by task (F(1.8,38)=13; p < 0.001) following Greenhouse-Geisser correction.
After equating the reaction times with respect to the main effect of category and the
interaction between category and modality/task, the three-way ANOVA only
identified (i) main effects of stimulus modality (F(1.2,26)=597; p < 0.001) and task
(F(1.7,34)=203; p < 0.001) following Greenhouse-Geisser correction.
Table 1, Figure 1 about here
Conventional SPM analysis
The SPM analysis was performed in two steps: First, we identified regions that
responded selectively to tools or animals. Second, we tested for category-selective
responses that were significantly modulated by stimulus modality or task-context.
Analyses including all trials or only the trials that were equated for reaction times (see
Noppeney et al.
14
methods) gave nearly identical results. We report the results of the latter conventional
analysis.
Tools evoked increased responses relative to animals, in the left posterior medial and
anterior fusiform regions. At a lower significance threshold, we observed increased
responses in the right medial fusiform (co-ordinates: [27 –42 –21]; z=4.4; p<0.001
uncorr.). In addition, tools evoked selective responses in a visuo-motor system
encompassing a left posterior middle/inferior temporal area (lpMT), the anterior
intraparietal sulcus (AIP) and several left prefrontal regions. Left prefrontal activation
was found in the ventral pre-motor area and along the left inferior frontal sulcus
extending into the triangular part of the left inferior frontal gyrus. Critically, tool-
selective responses in the occipito-temporal areas showed a significant interaction
with stimulus modality and were greater for pictures. For pictures relative to written
words only (i.e. excluding spoken words) a left medial fusiform, middle and superior
occipital region showed increased tool-selective activations, the right middle and
superior occipital regions showed increased animal-selective activation. This
demonstrates that the interaction effect reflects the difference between verbal and
non-verbal stimuli. In contrast, responses in lpMT and AIP exhibited a significant
interaction with task and were greater for explicit semantic tasks that required
retrieval of an associated action or the real life size of the stimulus. The apparent
increased activation for animals relative to tools in AIP (see Figure 2) during the
implicit condition was not significant at p<0.05 uncorrected.
We did not detect any tool-selective activation that was enhanced for (i) words
relative to pictures or (ii) implicit relative to explicit semantic tasks.
Figure 2, Table 2 about here
Noppeney et al.
15
Animals increased responses relative to tools in the right middle occipital and the
lateral fusiform gyri. Animal-selective responses in both regions and additional left
and right lateral occipito-temporal areas interacted with stimulus modality and were
greater when the stimuli were presented as pictures. No animal-selective responses
were detected that (i) were enhanced for words relative to pictures or (ii) interacted
with task context.
In summary, a ventral object recognition system, comprising occipito-temporal
regions showed modality-dependent category-selective effects, while a dorsal visuo-
motor system showed task-dependent category-effects. Our DCM analysis addressed
how this dissociation was mediated in terms of functional integration:
Figure 3, Table 3 about here
Effective Connectivity analysis
First, we found that tool pictures enabled the forward connections from OCC to the
tool-selective left posterior medial fusiform and AIP areas. Furthermore, this effect
was significantly greater for tool pictures than tool words i.e. there was a significant
modulatory effect of the stimulus modality by category interaction on the forward
connections (Fusiform: p<0.001; AIP: p<0.05). These results imply that modality-
dependent category-selective responses can be explained by modulation or selective
enabling of forward connections, in the context of tool pictures (see Table 1).
Second, left prefrontal areas that showed greater responses during explicit semantic
tasks exerted more top-down influence on the fusiform and AIP when subjects were
actively engaged in semantic tasks on tools than on animals. These results
demonstrate that task-dependent category-selective responses can be explained in
Noppeney et al.
16
terms of category-sensitive modulation of backward connections, during explicit
semantic tasks on tools.
Finally, the effect of the stimulus modality x category interaction was greater for the
forward connections from early visual areas to the fusiform than to AIP (p<0.01).
Conversely, the task-dependent category-effect of tools was greater for backward
connections from the left prefrontal area to the AIP than the fusiform area (p<0.05).
These results demonstrate that the distinct patterns of category-selectivity over
regions can be explained by differences in top-down and bottom-up influences that
show a dorso-ventral dissociation (i.e. differences in modulatory effects between
connections to the fusiform and AIP).
Table 4, Table 5, Figure 4
In a subsequent analysis, we have investigated the neural mechanisms that mediate
modality-dependent animal-selective responses in the ventral occipito-temporal
cortex using a DCM model including the STG, OCC and the animal-selective right
lateral posterior fusiform area. Similar to our DCM analysis for tools, we found
increased forward connections from OCC to the animal-selective right lateral
posterior fusiform area for animal pictures relative to animal words i.e. there was a
significant modulatory effect of the stimulus modality by category interaction on the
forward connections. In short, we reached equivalent conclusions for the modality-
dependent animal-selective activations in the ventral occipito-temporal cortex
(detailed results not reported).
Noppeney et al.
17
In summary, we found that ventral category-effects could be explained by a stimulus
modality-dependent increase in bottom-up category-specific influences, whereas
dorsal regions were subject primarily to category-selective top-down influences of
task-related prefrontal activity.
Effect of gender on modulatory effects
At the random effects level, we compared modulatory effects between female and
male subjects. This analysis did not reveal any gender effects (p>0.05).
Effect of performance on modulatory effects
To further characterize the connectivity results, we investigated the effect of subject’s
performance on the modulatory effects (see Buchel et al., 1999;Goncalves and Hall,
2003;Glabus et al., 2003) using subject-specific reaction times as predictors for the
connection strengths. First, we used the reaction time difference for tools – animals
during semantic decision tasks to predict the tool-effect on backward connections
from the prefrontal cortex. Second, we used the reaction time difference for tools –
animals during picture conditions to predict the tool picture-effect on the forward
connections from the occipital cortex. Finally, we used the reaction time difference
for tools – animals during written word conditions to predict the modulatory tool
written word-effect on forward connections from the occipital cortex. None of these
regression analyses revealed a significant relation between reaction time and
connection strength.
Noppeney et al.
18
Discussion
Our results demonstrate robust category-selective responses in multiple cortical
regions: Within the fusiform gyrus, category-selective activations were found
medially for tools and laterally for animals. In addition, tools elicited increased
responses in a left-lateralized visuo-motor action system encompassing ventral
premotor, anterior intraparietal and posterior middle temporal regions. Critically, in
the ventral occipito-temporal cortex, tool-selective activations were observed
irrespective of task but depended on stimulus modality (picture vs. words). In
contrast, tool-selective responses in the dorsal visuo-motor action system emerged
irrespective of modality but were modulated by task. Therefore, category-selectivity
rests on the interaction of semantic content with either (i) stimulus-bound factors such
as modality or (ii) task. From a cognitive perspective, category-selective responses
may be better understood in terms of the cognitive operations induced by a
semantically invested stimulus in a particular context, rather than its semantic content
alone. In particular, our results demonstrate a dorso-ventral dissociation with ventral
occipito-temporal regions engaged by stimulus-bound structural processing and dorsal
visuo-motor action regions by task-induced semantic operations. In terms of neural
mechanisms, our results suggest that the stimulus modality- and task-dependent tool-
selective responses are not properties intrinsic to a region but are mediated by
changes in the influence of, or the responsiveness to, other regions (McIntosh,
2000;Mesulam, 1990;Friston and Price, 2001;Buchel and Friston, 2000;Horwitz,
2003). These two distinct classes of tool-selectivity can be explained by differential
top-down and bottom up influences for task and modality-dependent effects
respectively.
Noppeney et al.
19
The tool and animal-selective responses within the ventral occipito-temporal cortex
are consistent with numerous studies of object recognition demonstrating focal
regions with preferential responses to various semantic categories including faces,
houses and chairs (Haxby et al., 2001;Spiridon and Kanwisher, 2002;Ishai et al.,
1999). A recent study reported ventral category-selective activations that were
mediated by bottom-up effects during perception and top-down effects during
imagery (Mechelli et al., 2004). In our study, category-selective responses in the
occipito-temporal cortex were modulated by stimulus modality and were evident only
for pictures, irrespective of the task context. According to our DCM, this functional
specialization is not an intrinsic property of the ventral occipito-temporal regions, but
is mediated via bottom up mechanisms that render them especially responsive to
certain patterns of input from early visual areas during object perception.
Collectively, these results suggest that the ventral occipito-temporal regions are
specialized for processing structural features that are sufficiently abstract to be shared
by different exemplars of the same category. These structural features permit object
categorization during perception and possibly imagery.
In contrast to the modality-dependent category-selective responses in the ventral
occipito-temporal cortex, the tool-selective responses in the dorsal visuo-motor action
system showed a distinct activation pattern: The tool-selective responses in the left
inferior/middle temporal area (lpMT), anterior inferior parietal sulcus (AIP) and
ventral pre-motor cortex (i.e. the putative homologue of area F5) were observed
irrespective of the stimulus modality. These three regions correspond to those with
the highest lesion overlap in patients with impaired action retrieval (Tranel et al.,
2003) and have been implicated in tool and action observation and retrieval by
Noppeney et al.
20
previous functional imaging studies (Grezes and Decety, 2001;Hauk et al., 2004). In
the macaque, neurons in areas F5 and AIP have been identified that respond
selectively to action execution, observation and presentation of graspable objects
(Rizzolatti and Luppino, 2001;Rizzolatti and Arbib, 1998). Our study demonstrates
that lpMT and AIP in humans respond to both tool pictures and names suggesting a
role in semantic processing. However, their responses were not obligatory but
strongly context-sensitive with tool-selective responses being enhanced when subjects
process stimuli at a deeper semantic level. Consistent with studies in primates and
neuropsychology that have implicated the left prefrontal cortex as a key player in top-
down control processes (Fuster, 1989;Miller, 2000), our DCM analysis demonstrated
that task-dependent tool-selectivity is mediated via increased backward influences
from the left prefrontal cortex to AIP during semantic decisions on tools. Thus, tool-
related action responses, for instance in AIP, might be enabled during explicit
semantic tasks by top-down modulation from the prefrontal cortex.
The DCMs discussed so far have established bottom-up and top-down modulations as
sufficient explanations for modality- and task-dependent category-selectivity.
Obviously, most brain regions will -to a certain degree- be exposed to both bottom-up
and top-down influences. Our study also demonstrated significant bottom up and top-
down influences for AIP and the fusiform region. Directly comparing the modulatory
components of connections to the left posterior fusiform and AIP demonstrated that
(i) modality-dependent bottom-up category effects were greater for the fusiform than
AIP and (ii) task-dependent top-down category effects were greater for AIP than the
fusiform. Thus, distinct classes of category-selectivity in AIP and left posterior
Noppeney et al.
21
medial fusiform can result from differential enabling of ventral and dorsal
connections.
Further evidence for distinct functional roles of the dorsal visuo-motor system and the
ventral occipito-temporal cortex in tool processing is provided by two recent studies:
The first (Chao et al., 2002) manipulated semantic category (animals vs. tools) and
visual experience (primed vs. unprimed) and demonstrated a priming- induced
response reduction selectively for tools in LPMT, but non-selectively in the medial
fusiform. The second (Beauchamp et al., 2003) manipulated (i) semantic type (i.e.
human motion vs. tool motion) and (ii) stimulus display (real objects vs point light
display) in a factorial design and showed a tool selective response in LPMT
irrespective of stimulus display but in the medial fusiform primarily for real tool
motion. These results suggest that the tool-selective responses in occipito-temporal
regions are strongly influenced by stimulus-bound factors such as modality (pictures
vs. words), display (real objects vs point lights) or perceptual priming.
In conclusion, our results demonstrate two classes of category-selectivity: In the
ventral occipito-temporal cortex, category-selective responses were observed
primarily for pictures and mediated by bottom-up effects. In lpMT and AIP, they were
observed during semantic decision tasks and mediated by increased top-down
modulation from left prefrontal cortex. These distinct activation and connectivity
patterns suggest that the two classes of category-selective systems may support
different cognitive operations with ventral occipito-temporal regions engaged in
structural processing and dorsal visuo-motor regions activated during strategic
semantic processing. Consistent with current semantic theories that emphasize the
link between tools and action features, we thus provide evidence that explicit
Noppeney et al.
22
semantic processing of tools relies on re-activating their associated action
representations via top-down modulation (Damasio, 1989;Martin and Chao,
2001;Barsalou et al., 2003).
Acknowledgements
UN, KF, WP and CP were supported by the Wellcome Trust. We thank the
radiographers at the Functional Imaging Laboratory for their assistance in collecting
the data.
Noppeney et al.
23
Figure Legends
Figure 1
Top
: Main effects of category: Tool- and Animal-selective activations are rendered
on an averaged normalised brain. Height threshold: p<0.05 corrected. Extent
threshold: > 1 voxel, including only voxels that were activated for stimulus > fixation
at p < 0.001 (uncorrected).
Bottom:
Main effects of category and category x task/modality interactions on
coronal and sagittal slices of a structural image created by averaging the subjects’
normalized images. Red: Tools > Animals; Green: Animals > Tools; Blue: Tools >
Animals for Semantic Decision > Implicit task; Yellow: (i) Tools > Animals for
Pictures > Words or (ii) Animals > Tools for Pictures > Words. Height threshold:
p<0.001 uncorrected for illustration purposes. Extent threshold: > 19 voxels restricted
to voxels that showed a significant effect of category-selectivity and stimulus >
fixation.
Figure 2
Left:
Interactions: Stimulus modality- and task-dependent tool-selective activations
on transverse slices of a structural image created by averaging the subjects’
normalized images. Modality-dependent: Tools > Animals for Pictures > Words.
Task-dependent: Tools > Animals for Semantic Decision > Implicit task. Height
threshold: p<0.001 uncorrected for illustration purposes. Extent threshold: > 19
voxels, restricted to voxels that showed a significant effect of tool-selectivity and
stimulus > fixation.
Noppeney et al.
24
Right: Parameter estimates for Tools (T, grey) and Animals (A, black) relative to
fixation during Implicit (I) and Explicit Semantic (S) tasks. The bar graphs represent
the size of the effect in adimensional units (corresponding to % whole brain mean)
These effects are activations pooled (i.e. summed) over appropriate conditions.
Figure 3
Left:
Interactions: Stimulus modality-dependent animal-selective activations on
transverse slices of a structural image created by averaging the subjects’ normalized
images. Animals > Tools for Pictures > Words. Height threshold: p<0.001
uncorrected for illustration purposes. Extent threshold: > 19 voxels, restricted to
voxels that showed a significant effect of animal-selectivity and stimulus > fixation.
Right:
Parameter estimates for Tools (T, grey) and Animals (A, black) relative to
fixation during Implicit (I) and Explicit Semantic (S) task.. The bar graphs represent
the size of the effect in adimensional units (corresponding to % whole brain mean)
These effects are activations pooled (i.e. summed) over appropriate conditions.
Figure 4
DCM for left anterior intraparietal (AIP) and left posterior medial fusiform gyrus
(FG) responses. Black: Intrinsic connections; Purple: Extrinsic input; Green:
Modulatory effects
Values are the mean (SD) of changes in connection strength (over subjects; at
p<0.001 in bold). These parameters quantify how experimental manipulations change
the values of intrinsic connections. In dynamic systems the strength of a coupling can
be thought of as a rate constant or the reciprocal of the time constant. Typically
Noppeney et al.
25
regional activity has a time constant in the order of 1-2 seconds (rate of 1-0.5 s
-1
).
Therefore, a modulatory effect of 0.05 s
-1
corresponds to a 5%-10% increase in
coupling.
Noppeney et al.
26
Table 1.a Response Accuracy
Task Implicit Action Visual
Words written
Tools 0.99(0.03) 0.91(0.06) 0.85(0.07)
Animals 0.99(0.02) 0.90(0.07) 0.87(0.07)
Words spoken
Tools 0.99(0.04) 0.89(0.07) 0.86(0.06)
Animals 0.98(0.09) 0.89(0.07) 0.84(0.09)
Pictures
Tools 0.99(0.03) 0.87(0.05) 0.81(0.08)
Animals 0.98(0.06) 0.87(0.06) 0.81(0.06)
Table 1.b Reaction times
Task Implicit Action Visual
Words written
Tools 624(99) 944(118) 951(96)
Animals 617(86) 920(104) 897(98)
Words spoken
Tools 1057(138) 1432(159) 1419(125)
Animals 1017(136) 1353(174) 1317(97)
Pictures
Tools 635(77) 1014(132) 953(92)
Animals 638(73) 905(102) 870(80)
Table 1.c Reaction times (equated)
Task Implicit Action Visual
Words written
Tools 624(99) 902(104) 919(92)
Animals 617(86) 939(108) 911(94)
Words spoken
Tools 1057(139) 1401(173) 1368(124)
Animals 1022(137) 1399(227) 1353(95)
Pictures
Tools 636(78) 966(123) 910(76)
Animals 640(72) 931(104) 893(84)
Values are across-volunteer means (SD)
Noppeney et al.
27
Table 2.Tool-selective activation
Region Co-ordinates Z-score p-value (corr.) voxels
Tools > Animals
L. post. middle temporal g. -51 -66 -6 >8.0 0.0 195
L. medial fusiform -24 -57 -15 5.9 0.0 14
L. supramarginal g. -57 -30 39 6.9 0.0 32
L. prefrontal triangular -48 36 6 6.2 0.0 64
opercular -54 18 15 5.5
L. ant fusiform -33 -33 -24 5.8 0.01 5
Interaction: Tool-selective activation for Pictures >
Words
L. medial fusiform g. -27 -63 -12 5.5 0.0 4
L. middle occipital g. -45 -66 -9 4.9 0.0 2
Interaction: Tool-selective activation for Semantic Decision > Implicit task
L. supramarginal g. -60 -30 42 5.7 0.0 12
(at p<0.001
uncorr.)
L. post. middle temporal g. -54 -57 -12 3.8 0.8 27
L. prefrontal, opercular -54 12 24 3.4 1.0 14
activation at p<0.05 (corr.); extent threshold > 1 voxel
Noppeney et al.
28
Table 3. Animal-selective activation
Region Co-ordinates Z-score p-value (corr.) voxels
Animals > Tools
R. middle occipital g./ 51 -78 0 6.2 0.0 17
Lat. occipital sulcus
R. fusiform g. 39 -60 -21 5.2
5
Interaction: Tool-selective activation for Pictures >
Words
R. middle occipital g./ 51 -78 0 >8.0 0.0 26
Lateral occipital sulcus
R. fusiform g. 45 -48 -27 5.8
22
42 -57 -21 4.9
R. sup. occipital sulcus 15 -102 9 5.7
5
R. inf. occipital sulcus 36 -84 -12 5.4
3
L. middle occipital g. -45 -84 3 5.5
2
activation at p<0.05 (corr.); extent threshold > 1 voxel
Noppeney et al.
29
Table 4. DCM Regions
Region Co-ordinates
L. inf. frontal sulcus -45 9 27
L. middle occ. g. -30 -93 6
L. sup. temp. g. -60 -15 3
L. medial fusiform -27 -63 -12
L. anterior intraparietal sulcus -60 -30 42
Noppeney et al.
30
Table 5. Modulatory Effect on Forward and Backward connections
Bilinear Effects Connections
T-value
df=21
p-value
Forward and Backward Bilinear
Effects
Forward Tool Pictures - Tool Words Occ => AIP 2.78 0.01
Tool Pictures - Tool Words Occ => FG 8.06 0.00
Backward Tools PF => AIP 7.31 0.00
Tools PF => FG 4.18 0.00
Dorso-ventral Dissociation of Bilinear Effects
Forward Tool Pictures - Tool Words
(Occ => FG) - (Occ =>
AIP) 3.3 0.00
Backward Tools (PF => AIP) - (PF => FG) 2.5 0.02
Noppeney et al.
31
Reference List
Barsalou LW, Kyle SW, Barbey AK, Wilson CD (2003) Grounding conceptual knowledge in
modality-specific systems. Trends Cogn Sci 7: 84-91.
Beauchamp MS, Lee KE, Haxby JV, Martin A (2003) FMRI responses to video and point-light
displays of moving humans and manipulable objects. J Cogn Neurosci 15: 991-1001.
Buchel C, Coull JT, Friston KJ (1999) The predictive value of changes in effective connectivity for
human learning. Science 283: 1538-1541.
Buchel C, Friston K (2000) Assessing interactions among neuronal systems using functional
neuroimaging. Neural Netw 13: 871-882.
Capitani E, Laiacona M, Mahon B, Caramazza A (2003) What are the facts semantic category-specific
deficits? A critical review of the clinical evidence. Cognitive Neuropsychology 20: 213-262.
Chao LL, Haxby JV, Martin A (1999) Attribute-based neural substrates in temporal cortex for
perceiving and knowing about objects. Nat Neurosci 2: 913-9.
Chao LL, Martin A (2000) Representation of manipulable man-made objects in the dorsal stream [In
Process Citation]. Neuroimage 12: 478-84.
Chao LL, Weisberg J, Martin A (2002) Experience-dependent modulation of category-related cortical
activity. Cereb Cortex 12: 545-551.
Damasio AR (1989) Time-locked multiregional retroactivation: a systems-level proposal for the neural
substrates of recall and recognition. Cognition 33: 25-62.
Damasio H, Grabowski TJ, Tranel D, Hichwa RD, Damasio AR (1996) A neural basis for lexical
retrieval [see comments] [published erratum appears in Nature 1996 Jun 27;381(6595):810].
Nature 380: 499-505.
Devlin JT, Moore CJ, Mummery CJ, Gorno-Tempini ML, Phillips JA, Noppeney U, Frackowiak RS,
Friston KJ, Price CJ (2002) Anatomic constraints on cognitive theories of category specificity.
Neuroimage 15: 675-85.
Friston KJ, Harrison L, Penny W (2003) Dynamic causal modelling. Neuroimage 19: 1273-1302.
Friston KJ, Holmes A, Worsley KJ, Poline JB, Frith CD, Frackowiak R (1995) Statistical parametric
mapping: a general linear approach. Hum Brain Mapping 2: 189-210.
Friston KJ, Holmes AP, Price CJ, Buchel C, Worsley KJ (1999) Multisubject fMRI studies and
conjunction analyses. Neuroimage 10: 385-96.
Friston KJ, Price CJ (2001) Generative models, brain function and neuroimaging. Scand J Psychol 42:
167-177.
Fuster JM (1989) The prefrontal cortex. New York: Raven Press.
Gainotti G, Giustolisi L, Daniele A, Silveri MC (1995) Neuroanatomical correlates of category-
specific semantic disorders: A critical survet. Memory 3: 247-264.
Glabus MF, Horwitz B, Holt JL, Kohn PD, Gerton BK, Callicott JH, Meyer-Lindenberg A, Berman
KF (2003) Interindividual differences in functional interactions among prefrontal, parietal and
parahippocampal regions during working memory. Cereb Cortex 13: 1352-1361.
Noppeney et al.
32
Goncalves MS, Hall DA (2003) Connectivity analysis with structural equation modelling: an example
of the effects of voxel selection. Neuroimage 20: 1455-1467.
Grabowski TJ, Damasio H, Damasio AR (1998) Premotor and prefrontal correlates of category-related
lexical retrieval. Neuroimage 7: 232-43.
Grafton ST, Arbib MA, Fadiga L, Rizzolatti G (1996) Localization of grasp representations in humans
by positron emission tomography. 2. Observation compared with imagination. Exp Brain Res
112: 103-11.
Grezes J, Decety J (2001) Functional anatomy of execution, mental simulation, observation, and verb
generation of actions: a meta-analysis. Hum Brain Mapp 12: 1-19.
Hauk O, Johnsrude I, Pulvermuller F (2004) Somatotopic representation of action words in human
motor and premotor cortex. Neuron 41: 301-307.
Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P (2001) Distributed and
overlapping representations of faces and objects in ventral temporal cortex. Science 293: 2425-
30.
Horwitz B (2003) The elusive concept of brain connectivity. Neuroimage 19: 466-470.
Humphreys GW, Forde EM (2001) Hierarchies, similarity, and interactivity in object recognition:
"category-specific" neuropsychological deficits. Behav Brain Sci 24: 453-76.
Ishai A, Ungerleider LG, Martin A, Schouten JL, Haxby JV (1999) Distributed representation of
objects in the human ventral visual pathway. Proc Natl Acad Sci U S A 96: 9379-84.
Josephs, O., Deichmann, R., and Turner, R. Trajectory measurement and generalized reconstruction in
rectilinear EPI. ISMRM meeting , 1517. 2000.
Kellenbach M, Brett M, Patterson K (2002) Action speak louder than functions. The importance of
manipulability and action in tool representation. Journal of Cognitive Neuroscience in press.
Martin A, Chao LL (2001) Semantic memory and the brain: structure and processes. Curr Opin
Neurobiol 11: 194-201.
Martin A, Wiggs CL, Ungerleider LG, Haxby JV (1996) Neural correlates of category-specific
knowledge. Nature 379: 649-52.
McIntosh AR (2000) Towards a network theory of cognition. Neural Netw 13: 861-870.
Mechelli A, Price CJ, Friston KJ, Ishai A (2004) Where bottom-up meets top-down: neuronal
interactions during perception and imagery. Cereb Cortex 14: 1256-1265.
Mesulam MM (1990) Large-scale neurocognitive networks and distributed processing for attention,
language, and memory. Ann Neurol 28: 597-613.
Miller EK (2000) The prefrontal cortex and cognitive control. Nat Rev Neurosci 1: 59-65.
Price CJ, Wise RJ, Frackowiak RS (1996) Demonstrating the implicit processing of visually presented
words and pseudowords. Cereb Cortex 6: 62-70.
Rizzolatti G, Arbib MA (1998) Language within our grasp. Trends Neurosci 21: 188-94.
Rizzolatti G, Fadiga L, Matelli M, Bettinardi V, Paulesu E, Perani D, Fazio F (1996) Localization of
grasp representations in humans by PET: 1. Observation versus execution. Exp Brain Res 111:
246-52.
Noppeney et al.
33
Rizzolatti G, Luppino G (2001) The cortical motor system. Neuron 31: 889-901.
Spiridon M, Kanwisher N (2002) How distributed is visual category information in human occipito-
temporal cortex? An fMRI study. Neuron 35: 1157-1165.
Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human brain. Stuttgart: Thieme.
Tranel D, Kemmerer D, Adolphs R, Damasio H, Damasio AR (2003) Neural Correlates of conceptual
knowledge. Cognitive Neuropsychology 20: 409-432.
Warrington EK, Shallice T (1984) Category specific semantic impairments. Brain 107: 829-54.
