Content uploaded by Srikanth Damera
Author content
All content in this area was uploaded by Srikanth Damera on Jul 16, 2020
Content may be subject to copyright.
Available via license: CC BY-NC-ND
Content may be subject to copyright.
From shape to meaning: Evidence for multiple fast feedforward hierarchies
of concept processing in the human brain
Srikanth R. Damera , Jacob G. Martin , Clara Scholl , Judy S. Kim , Laurie Glezer ,
Patrick S. Malone , Maximilian Riesenhuber
*
Department of Neuroscience, Georgetown University Medical Center, Washington, DC, USA
ABSTRACT
A number of fMRI studies have provided support for the existence of multiple concept representations in areas of the brain such as the anterior temporal lobe (ATL)
and inferior parietal lobule (IPL). However, the interaction among different conceptual representations remains unclear. To better understand the dynamics of how the
brain extracts meaning from sensory stimuli, we conducted a human high-density electroencephalography (EEG) study in which we first trained participants to
associate pseudowords with various animal and tool concepts. After training, multivariate pattern classification of EEG signals in sensor and source space revealed the
representation of both animal and tool concepts in the left ATL and tool concepts within the left IPL within 250 ms. Finally, we used Granger Causality analyses to
show that orthography-selective sensors directly modulated activity in the parietal-tool selective cluster. Together, our results provide evidence for distinct but parallel
“perceptual-to-conceptual”feedforward hierarchies in the brain.
1. Introduction
Humans can rapidly and efficiently assign meaning to visual objects.
For visual stimuli, this process is traditionally thought to be mediated by
the visual ventral stream, along which information is processed by a
simple-to-complex hierarchy, up to neurons in ventral temporal cortex
that are selective for complex objects such as faces, objects and words
(Kravitz et al., 2013). According to computational models (Ashby and
Spiering, 2004;Freedman et al., 2003;Nosofsky, 1986;Riesenhuber and
Poggio, 2000;Thomas et al., 2006) as well as human functional magnetic
resonance imaging (fMRI) and electroencephalography (EEG) studies
(Jiang et al., 2007;Scholl et al., 2013), these object-selective neurons in
high-level visual cortex can then provide input to task modules located in
downstream cortical areas, such as prefrontal cortex (PFC) and the
anterior temporal lobe (ATL), to mediate the identification, discrimina-
tion, or categorization of stimuli. It is at this level where these theories of
object categorization in the brain connect with influential theories of
semantic cognition that have proposed that the ATL may act as a “se-
mantic hub”(Ralph et al., 2016), based on neuropsychological findings
(Hodges et al., 2000;Jefferies, 2013;Mion et al., 2010) and studies that
have used fMRI (Coutanche and Thompson-Schill, 2015;Malone et al.,
2016;Vandenberghe et al., 1996) or intracranial EEG (iEEG) (Chan et al.,
2011) to decode category representations in the anteroventral temporal
lobe.
Yet, the processing of some object classes has been shown to involve
additional pathways. A prominent example is the object class of tools,
which, in addition to the ventral stream, engages the dorsal, “vision for
action”(Goodale, 2011;Goodale and Milner, 1992;Kravitz et al., 2011)
pathway, with areas in parietal cortex being selectively activated by tool
stimuli, and lesions in parietal cortex affecting tool praxis (Buxbaum
et al., 2014;Vingerhoets et al., 2009). A key, but unresolved, question in
models of semantic processing is how the brain accesses these different
types of concept knowledge. In particular, are there separate pathways
that link sensory representations to distinct domain-specific concept
representations, compatible with distributed models of semantic pro-
cessing (Chen and Rogers, 2015), or does knowledge from one domain
access that from another indirectly through representations in a
domain-general hub (Almeida et al., 2013;Hodges et al., 2000)?
Resolving the question of how different types of concept knowledge
are processed and interact requires the ability to not only delineate the
underlying network of brain areas, but also, and crucially, the informa-
tion flow between them. Recent fMRI research has uncovered a complex
network of brain areas underlying semantic processing (Chen et al.,
2017;Pulvermüller, 2013). Yet, due to its limited temporal resolution,
fMRI is unable to directly probe the information flow within these net-
works, considering that numerous prior studies have shown that the
brain is able to extract meaning from sensory stimuli within about 200 ms
(Chan et al., 2011;Scholl et al., 2013;Thorpe et al., 1996), and, equally
* Corresponding author. Department of Neuroscience, Georgetown University Medical Center Research, Building Room WP-12, 3970 Reservoir Rd. NW Washington,
DC, 20007, USA.
E-mail address: mr287@georgetown.edu (M. Riesenhuber).
Contents lists available at ScienceDirect
NeuroImage
journal homepage: www.elsevier.com/locate/neuroimage
https://doi.org/10.1016/j.neuroimage.2020.117148
Received 30 March 2020; Received in revised form 10 June 2020; Accepted 6 July 2020
Available online 11 July 2020
1053-8119/©2020 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
NeuroImage 221 (2020) 117148
important, that neural processing dynamics do not just include a “feed-
forward”flow of information, but also, starting within 150 ms and
continuing for several hundred milliseconds, several re-entrant waves of
activation, e.g., associated with stimulus awareness (Cul et al., 2007;
Fahrenfort and Lamme, 2012), whereas typical fMRI scan paradigms
have temporal resolutions that are ten times slower. In contrast, EEG with
its millisecond resolution is better matched to the speed of visual pro-
cessing in the brain. We used a combination of multivariate pattern
analysis (MVPA) and EEG to address these challenges and gain a better
understanding of the dynamics of the neural networks underlying
concept processing. We trained participants to associate a vocabulary of
pseudowords (PWs) with various animal and tool categories (Fig. 1A).
Next, we collected high-density EEG data while subjects performed a
delayed-match-to-sample task. We applied searchlight MVPA in sensor as
well as source space to temporally and spatially localize concept infor-
mation in the brain. In addition, Granger Causality analyses were used to
elucidate the dynamics of signal flow within the semantic network.
2. Materials and Methods
2.1. Participants
A total of 11 right-handed healthy adults who were native English
speakers were enrolled in the experiment (ages 20–29, 5 females).
Georgetown University’s Institutional Review Board approved all
experimental procedures, and written informed consent was obtained
from all subjects before the experiment.
2.2. Behavioral training
PWs (all four letters in length) matched for bigram and trigram fre-
quency, and orthographic neighborhood were generated using MCWord
(Medler and Binder, 2005). Subjects were trained to learn a vocabulary of
60 PWs, with each PW assigned to one of six categories: monkey, donkey,
elephant, hammer, wrench, and screwdriver (i.e., 10 PWs were defined
as “monkey,”10PW as “donkey,”etc.). To learn the PW definitions, each
subject performed 0.5–1 h of training per session for a total of 8 sessions
in which a PW was presented on a screen followed by 6 pictures: one
picture for each of the animal and tool categories (randomly selected
from a large database of images for each category to prevent subjects
from associating particular words with particular images). Subjects
indicated their category choice using a numeric keypad. When subjects
answered incorrectly, an auditory beep and the correct answer were
presented. No feedback was given for correct answers. A single training
session consisted of 5 blocks, and a unique set of 2 PWs (within the 10 per
category) was used for each block to facilitate learning.
2.3. EEG paradigm
Following training, we probed the neural bases of the learned concept
memberships using an EEG paradigm. Each trial of the EEG experiment
consisted of a fixation cross for 500 ms, a blank screen for 200 ms with
jitter, followed by the first PW for 300 ms, blank screen for 400 m, and
second PW for 300 ms. Following the second PW, subjects were
instructed to indicate with a button click if both words referred to the
same or different superordinate categories (Fig. 1B). Trials were broken
into four different conditions depending on the relationship between the
first and second PWs: same word/same basic-level category (SWSC),
different word/same basic-level category (DWSC, e.g., first: monkey PW,
second: a different monkey PW), different word/different basic-level
category (DWDC, e.g., first: monkey PW, second: a donkey PW), and
different word/different superordinate category (DWDSC, e.g., first: an-
imal PW, second: a tool PW). Each subject participated in a single session
with two runs each. Each run had three blocks, each of which were 136
trials long, for a total of 408 trials in a run. Counts for the four trial types
were matched in each session and different conditions were presented in
a random manner. Trials with incorrect responses were removed from the
analyses. In this study, only data from the first PW presentation were
analyzed for the decoding analyses.
2.4. EEG data acquisition and preprocessing
Scalp voltages were measured using an Electrical Geodesics (EGI,
Eugene, OR) 128-channel Hydrocel Geodesic Sensor Net and Net Amps
300 amplifier. Incoming data were digitally low-pass filtered at 200 Hz
and sampled at 500 Hz using common mode rejection with vertex
reference. Impedances were set below 40 kΩbefore recording began and
maintained below this threshold throughout the recording session with
an impedance check during each break between blocks.
Pre-Processing: Data processing and statistical analyses were per-
formed using EEGLAB (Delorme and Makeig, 2004). Data were first
high-pass filtered at 0.2 Hz and then low-pass filtered at 30 Hz. A
minimum-phase causal filter ensured that information did not get
smeared backwards and was used for decoding analyses to accurately
determine the onset latency of classification (Widmann and Schr€
oger,
2012). Bad channel identification was automated using the clean_raw-
data EEGLab plugin that cleaned continuous data using artifact subspace
reconstruction. Bad channels identified with this method were then
reconstructed using spherical interpolation. Continuous data were
epoched on the interval of [-200 300] ms relative to the onset of the first
word. The mean of the baseline period was then subtracted from the
signal. Bad trials were marked if the voltage exceeded 50
μ
V on any
EEG channel. Finally, the data were re-referenced to average reference
since this has been shown to improve the quality of the source recon-
struction (Dien, 1998).
2.5. EEG source estimation
Source reconstruction was conducted using common forward and
inverse models implemented in Brainstorm (Tadel et al., 2011). In
Brainstorm (version 11-13- 2019), the EEG forward problem was solved
in the ICBM152 template anatomy. Realistically-shaped volume meshes
of the brain, skull and scalp were extracted from the provided template
MR image using the default number of 1922 vertices per layer for a total
of 15765 sources. The forward model (lead field) from these source lo-
cations to the 128 channels was calculated using the symmetric boundary
Element method as implemented in the Open-MEEG package. Inverse
estimation of sources was carried out using the LCMV Beamformer. The
noise covariance matrix for this method was built using the 200 ms of
baseline data, and the data covariance matrix was constructed using the
300 ms of word presentation.
2.6. MVPA decoding
MVPA analyses were performed using The Decoding Toolbox (Hebart
et al., 2014) and custom MATLAB code (R2017b, The Mathworks, MA).
All classifications implemented a linear support vector machine (SVM)
classifier with a fixed cost parameter (c ¼1). For each subject, all trials of
a particular PW were averaged over the entire session so that a single
pseudo-trial was created for each PW. This procedure has been shown to
increase the signal-to-noise-ratio (Grootswagers et al., 2016). 100-fold
Monte Carlo cross-validation was performed with an approximate
50/50 training to testing split. Training and testing sets were carefully
partitioned so that no PW was included in both the training and testing
set within a fold. Accuracies were averaged across folds to compute an
unbiased measure of classifier performance.
Sensor Space Spatiotemporal Searchlight Decoding: Time-
resolved spatial decoding analyses were then performed in sensor
space. For these decoding analyses, a 3 cm searchlight was constructed
around each sensor (searchlight size ranged from 1 to 7 sensors, but
similar results were obtained with a 4 cm searchlight which had a range
of 3–8 sensors in a searchlight). Temporal data for each sensor in a given
S.R. Damera et al. NeuroImage 221 (2020) 117148
2
searchlight were averaged in order to create an N-channel-by-1 feature
vector per searchlight. This was done in 20 ms windows and advanced 2
ms over the duration of the word (Fig. 1C).
Source Space Spatial Decoding: To spatially localize sensor level
classification, voltage data were projected to source space (see above)
and a searchlight MVPA was performed using a 10 mm searchlight. The
value at a given source was taken as the average over a time window of
interest defined by sensor level decoding results (Fig. 1D).
2.7. MVPA analysis 1 (superordinate classification)
For each subject, an animal versus tool classification was first per-
formed in sensor space to identify the onset latency and time window of
superordinate decoding. Within each cross-validation fold, the classifier
was trained on 40 PWs (20 animal and 20 tool PW) and tested on the
remaining 20 PWs. Once a significant time-window of decoding was
identified at the sensor level, this time window was then submitted to the
classification procedure in source space in order to estimate a neuroan-
atomical source of the classification.
2.8. MVPA analysis 2 (basic-level classification)
Next, a within-category classification was performed in sensor space
to identify the onset latency and time window of basic-level decoding.
There was a total of 6 (3 within-animal and 3 within-tool) unique clas-
sifications. One subject was not shown any elephant or wrench PWs and
was excluded from the basic-level classification analyses. Within each
cross-validation fold, the classifier was trained on 10 PWs (e.g., 5
hammer and 5 wrench PWs) and tested on the remaining 10 PWs. The
resulting spatiotemporal classification maps for all within-animal com-
parisons were averaged to produce a single accuracy map for basic-level
animal concepts. This was repeated for all within-tool comparisons as
well. Once a significant time-window of decoding was identified at the
sensor level this was then submitted to the classification procedure in
source space in order to estimate a neuroanatomical source of the
classification.
2.9. Statistical analysis of classification results
Each MVPA analysis produced an accuracy value at each sensor or
source. At the group-level, multiple comparisons across sensors and time-
points or sources was done using cluster-mass correction (Maris and
Oostenveld, 2007). For this method, an empirical null distribution was
constructed for each subject at every sensor and time-point by permuting
labels across the entire data set, setting up cross-validation folds, and
then performing the classification procedure as before. This procedure
was repeated 1000 times, and then the observed and null distributions
were averaged across subjects at every sensor and time point. The p-value
associated with the accuracy observed at a single electrode time-point
combination was calculated as the proportion of values in the null dis-
tribution greater than the observed value. Next, an initial cluster level
threshold,
α
, was used to generate either spatiotemporal or spatial clus-
ters corresponding to sensor space and source space analyses, respec-
tively.
α
was set to .002, for both sensor and source space analyses.
Previous fMRI studies have noted that a liberal
α
(i.e., >0.001) increases
the rate of false positives (cf. Fig. 1 in Eklund et al., 2016). However, this
is specific to parametric approaches using random field theory and not
non-parametric permutation-based approaches (Eklund et al., 2016). It
Fig. 1. Task and analysis paradigms. (A) Subjects were trained on a set of 60 pseudowords (PW) belonging to one of three animal or tool categories (two PWs are
shown as examples with each category), as in Malone et al. (2016). (B) After training, subjects participated in an EEG experiment, in which they performed a
delayed-match-to-superordinate category task in which two PWs were sequentially presented on each trial and subjects had to indicate after the presentation of the
second word if both words belonged to the same or different superordinate categories (i.e., animals or tools). (C) At each sensor and for different time bins (the blue
shaded region illustrates one example time bin), a set of time-lagged support vector machine (SVM) classifiers was constructed on half the trained pseudowords (PWs)
from animal and tool categories and then tested on the other half, resulting in dynamic classification accuracy maps for each subject in sensor space. (D) In separate
analyses, EEG sensor data were projected into source space (see Materials and Methods), followed by averaging over a time period of interest. Searchlight MVPA was
then performed, resulting in a spatial accuracy map per subject.
S.R. Damera et al. NeuroImage 221 (2020) 117148
3
has been shown that permutation-based methods control the false posi-
tive rate at the desired level independent of the
α
chosen (Maris and
Oostenveld, 2007;Sassenhagen and Draschkow, 2019;Woo et al., 2014).
The mass of clusters was computed by summing the accuracies of all the
sensors or sources in the cluster. The mass of each cluster in the actual
data was compared to an empirical distribution of maximal clusters of the
permuted data and was marked as significant if it was greater than 95%
of the values.
2.10. Identification of clusters selective for orthography
Orthographically-selective space-time clusters were identified by
contrasting responses to the second words in each trial in the SWSC and
DWSC conditions, and significance was assessed using cluster-based
permutation testing. Due to strong a priori hypotheses about the la-
tency of the N170 response to words (Brem et al., 2010;Dehaene et al.,
2015;Maurer et al., 2005) this analysis was constrained to left lateralized
sensors between 100 and 200 ms after the onset of the second word.
Using fMRI and EEG it has been shown that neuronal populations that are
selective for a feature of interest (e.g., orthography) have a reduced
(adapted) response for rapidly presented pairs of stimuli that are
consistent versus inconsistent for that feature (Glezer et al., 2009;Jiang
et al., 2007;Scholl et al., 2013). In the current study, we hypothesized
that sensors recording from populations of neurons sensitive to orthog-
raphy would exhibit an adapted N170 response to sequential presenta-
tion of the same PW versus presentation of two different PWs. Crucially,
to control for differences due to semantics, we used SWSC and DWSC
trials in which both the first and the second word referred to the same
basic-level concept. As for the classification results, spatiotemporal
clusters were identified by setting the cluster-defining
α
to 0.002 and
then their significance was controlled at the two-tailed p <.05 level.
2.11. Granger Causality
Granger Causality (Granger, 1969) (GC) has been used to investigate
causal relationships between prefrontal and early visual neural recording
sites in monkey and human electrophysiology recordings (Cui et al., 2008;
Gregoriouet al., 2009;Seth et al., 2015).In this work, we computedGranger
Causality within each subject at the single trial level between all pairs of
channels in the left anterior temporal superordinate-classification, parietal
basic-level tool, and left posterior orthographic sensors (see Fig. 5A). We
used the BSMART toolbox (Cui et al., 2008) with a model order of 15 and a
sliding temporal window of 60 ms. We only used artifact-free trials for
which correct responses were given. Before computing the Granger Cau-
sality, we performed the standard preprocessing step of subtracting the
temporal mean of each trial (in each channel) from all data points in that
trial. Next, we calculated the Granger causalities for each time point and
subject separately for each channel pair. Finally, we averaged the
frequency-specific Granger Causality values for all channel pairs into the
theta (4–7 Hz), alpha (8–12 Hz), and beta (13–30 Hz) frequency bands for
each condition and subject to obtain a time-based version of Granger Cau-
sality within that subject, and frequency band.
2.12. Statistical analysis of Granger Causality results
A mixed effects modelling approach was taken to test the significance
of Granger Causality effects. Models were built to test when the GC of
each of the proposed pathways (e.g., orthography to basic-level tool se-
lective sensors) deviated from baseline. The model included a fixed-effect
of frequency band, and a random-effect of subjects. Since orthographic
processing indexed by the N170 ERP begins at ~150 ms, this model was
constructed at each time point between 150 and 300 ms, and two-tailed
p-values for the fixed-effects of interest were FDR-corrected at the .05
level to control for multiple comparisons across time and frequency
bands.
3. Results
3.1. Behavior
Each subject (n ¼11) performed 8 training sessions over an average
of 13.8 days (SD 3.2). Subjects performed only a single training session
per day. Both accuracy and RT for identification of PW category
improved across training sessions (Fig. 2A). On average, subjects reached
an accuracy of 98.7% by their eighth training session. During the
delayed-match-to-basic-level category task, subjects reached 93.7% ac-
curacy on average (Fig. 2B). A two-way ANOVA with the factors condi-
tion (same word/same basic category, SWSC; different word/same basic
category, DWSC; different word/different basic category, DWDC;
different word/different superordinate category, DWDSC) and superor-
dinate category (animals and tools) of the first word in each trial was
conducted in order to test for any effect of condition or superordinate
category on task performance. This revealed a significant main effect of
condition on accuracy (F
(3,79)
¼6.88, two-tailed p ¼0.004). Impor-
tantly, there was no significant main effect of superordinate category on
task accuracy (F
(1,79)
¼1.78, two-tailed p ¼0.18). In order to investigate
the source of the significant effect of condition, post-hoc t-tests
comparing accuracy between all pairs of conditions showed that there
was a significant difference in accuracy for SWSC vs. DWSC (two-tailed p
¼0.002) contrast as well as the DWDSC vs. DWSC (two-tailed p ¼0.008)
contrast. There was no significant interaction between superordinate
category and condition on accuracy (F
(3,79)
¼1.03, two-tailed p ¼0.39).
3.2. EEG single-trial analysis reveals fast superordinate concept selectivity
over temporal sensors, localized to the ATL
We conducted a series of single-trial classification analyses of the EEG
signal in response to the first word in each trial to analyze how the brain
maps sensory stimuli to concepts. We first explored the onset of
superordinate-level (animals vs. tools) information at the sensor level
within the 100–300 ms interval. This revealed one spatiotemporal cluster
with significantly above-chance classification performance (Fig. 3A and
B). The left-anterior temporal cluster was significantly above chance
from 204 to 268 ms post-word presentation with peak accuracy at 236 ms
(Fig. 3B; cluster-defining
α
¼0.002, one-tailed p ¼.0240, see Material
and Methods). We then projected sensor data to source space using an
LCMV (Linearly Constrained Minimum Variance) beamformer (Grech
et al., 2008) to localize the cortical source(s) of this classification.
Furthermore, to better characterize the timing of concept processing in
our broad temporal cluster (204–268 ms) (Sassenhagen and Draschkow,
2019), we separately averaged source maps in the early (204–236 ms)
and late (236–268 ms) halves of the cluster (see Materials and Methods).
We hypothesized that concept information in the ATL should localize to
the early rather than late time window, given our overarching hypothesis
that activation of concept information in ATL would be the next step in
the ventral stream processing hierarchy following the orthographic rep-
resentation (which has been associated with the N170 in the literature
(Brem et al., 2010;Dehaene et al., 2015;Maurer et al., 2005), with a
temporal extent from about 170 to 200 ms), taking into account estimates
in the literature of each cortical processing stage taking on the order of
30 ms (Thorpe and Fabre-Thorpe, 2001). Searchlight source space
analysis indeed localized superordinate category information in the early
time window to the LATL (Fig. 3C; cluster-defining
α
¼0.002, one-tailed
p¼.025).
3.3. EEG single-trial analysis reveals fast basic-level decoding of tools in
left parietal sensors
We next examined the latency and location of basic-level (within-
tool and within-animal) category decoding. We identified sensors that
distinguished among the different tools: Sensor-level searchlight
analysis between 100 and 300 ms after stimulus onset revealed a
S.R. Damera et al. NeuroImage 221 (2020) 117148
4
cluster of left posterior sensors that show significantly above-chance
classification from 208 to 228 ms post-stimulus onset with peak ac-
curacy at 220 ms (Fig. 4A and B; cluster-defining
α
¼0.002, one-tailed
p¼.031). Searchlight source space analysis localized the classification
from this time period to the left inferior parietal lobe (Fig. 4C; cluster-
defining
α
¼0.002, one-tailed p ¼.025. We also conducted basic-level
animal classification, but no significant spatiotemporal clusters were
found.
3.4. Granger Causality analysis supports feedforward “simple-to-concept”
hierarchy
The latency of concept information in the preceding analyses pro-
vides evidence for fast, feedforward concept processing areas in the
brain. The regions identified are compatible with the ventral and dorsal
concept processing streams discussed in the Introduction. We next used
Granger Causality analyses (Granger, 1969) to test if a common
perceptual representation could be used to access these different
Fig. 2. Pseudoword training performance and delayed-match-to-superordinate category task accuracy. (A) Mean accuracies for identification of pseudoword category
across training sessions during a 6 alternative-forced choice task (n ¼10). (B) Accuracy in the delayed-match-to-superordinate category task during which EEG data
was collected for each condition: SWSC, first and second word were identical; DWSC, second word was different word, same basic-level category as the first word;
DWDC, second word was different word, belonging to different basic-level category than the first word; DWDSC, second word was different word, different super-
ordinate category than the first word. Brackets indicate significant differences across conditions: *p <0.01; n ¼11. Errors bars indicate SEM across subjects.
Fig. 3. Timing and location of superordinate classification of PW concept membership in sensor and source spaces and comparison to previous, fMRI results. (A) The
spatial topography of the sensors that showed significant decoding of animal and tool PWs. Green sensors are referred to as “left anterior”cluster. (B) Time course of
classification accuracy averaged across all sensors in the left anterior cluster. Red asterisks show time-window of significant classification, which extends from 204 to
268 ms post-word presentation (n ¼11;
α
<0.002; one-tailed p ¼.024). Shading shows SEM across subjects. (C) Source estimation of Animals vs. Tools classification.
The locus of above-chance classification was investigated in an early (204–236 ms) and late (236–268 ms) time window. Activity in the early, but not the late time-
window was estimated in source space to the LATL (n ¼11;
α
<0.002; one-tailed p ¼.025).
S.R. Damera et al. NeuroImage 221 (2020) 117148
5
conceptual representations –a key prediction of two-stage models of
category learning. Specifically, we investigated whether activity at
orthographically-selective sensors directly modulated activity in dorsal
tool-selective sensors. To do so, we first identified a high-level
perceptual (orthography) selective cluster (Fig. 5A; see methods), pu-
tatively identifying the “visual word form area”,VWFA(Brem et al.,
Fig. 4. Timing and location of tool classification of PW concept membership in sensor and source space. (A) The spatial topography of the sensors (shown in purple)
that show significant decoding of tool PWs. (B) Time course of classification accuracy averaged across all sensors in the cluster. Red asterisks show time-window of
significant classification, which extends from 208 to 228 ms post-word presentation (n ¼10;
α
<0.002; one-tailed p ¼.031), shading shows SEM across subjects. (C)
Source estimation of within-tools classification shown in panels A and B. The locus of above-chance classification during the time window of classification was
estimated in source space to the left parietal lobe (n ¼10;
α
<0.002; one-tailed p ¼.025).
Fig. 5. Neural dynamics in the concept processing network. (A) The three seeds are marked orthographic (red), superordinate (green, see Fig. 3), and basic-level tool-
selective (purple, see Fig. 4) sensor groups. (B) Average ERP of all sensors that show an orthographic response (shown in red in (A)) shows significant adaptation of the
N170 response between 148 and 178 ms. Red asterisks mark the time window of the significant cluster (n ¼11;
α
<0.002; one-tailed p ¼.034). Average ERP of all
sensors that show an orthographic response (shown in (A)) shows an N170 response during the first word (inset). (C) Baseline-normalized Granger Causality (GC) from
orthographic (red) and anterior temporal (green) to parietal basic-level tool selective sensors in the theta frequency band. Red asterisks mark time points exhibiting
significant increase in GC relative to baseline between orthographic and basic-level parietal tool selective sensors from 204 to 214 ms and from 246 to 284 ms (n ¼11;
two-tailed p <.05, FDR-corrected).
S.R. Damera et al. NeuroImage 221 (2020) 117148
6
2010;Dehaene-Lambertz et al., 2018;Maurer et al., 2005)–the highest
orthographically selective stage in the ventral visual pathway. This
cluster showed a significant adaptation effect between 148 and 178 ms
with a peak difference at 162 ms (Fig. 5B; cluster-defining
α
¼0.002,
two-tailed p ¼.034). This orthographic N170 cluster overlapped in both
space and time with those reported in previous EEG studies (Maurer
et al., 2005;Scholl et al., 2013). The ERP for these sensors showed an
N170 response during the first word with the negative deflection
starting at 150 ms post-stimulus onset (Fig. 5B inset). The sensors in this
cluster were used as our orthographic seed.
We next calculated the change in Granger Causality (GC) relative to
a 200 ms pre-stimulus baseline among the orthographically-selective
sensors and the sensors representing concept information in the
ventral and dorsal pathways that were identified in the previous sec-
tions. To examine if orthographic information directly accessed parietal
tool-selective representations, we tested the change in GC between
orthography and parietal-tool selective sensors from baseline in the
theta (4–7 Hz) frequency band, which has been implicated in feedfor-
ward information flow (Bastos et al., 2014), as well as the alpha (8–12
Hz) and beta (13–30 Hz) frequency bands. After FDR correction across
the three frequency bands and time points (see Materials and Methods),
we found evidence that theta-frequency (but not alpha or beta) activity
in orthography-selective sensors significantly modulated activity in
parietal tool-selective sensors from 204 to 214 ms and 244–284 ms
post-stimulus onset –just prior to the onset of basic-level tool repre-
sentations (Fig. 5C; two-tailed p <.05 FDR corrected).
4. Discussion
The current study applied multivariate searchlight decoding of EEG
data from a word learning paradigm to cleanly dissociate perceptual and
conceptual information in the neural signals at high temporal resolution.
This approach allowed us to address two key unanswered questions:
when do different conceptual representations come online, and how are
they accessed by sensory input? We provide evidence for superordinate
category selectivity in left anterior sensors within 250 ms after word
presentation. Using source estimation techniques, this selectivity was
localized to the left anterior temporal lobe (LATL). Basic-level tool
category membership was also decoded in a set of left parietal sensors
between 208 and 228 ms after word presentation. The cortical generator
of this signal was source estimated to the left parietal lobe. Importantly,
these source-estimated ROI matched well with our previous fMRI study
that used the same training paradigm (Malone et al., 2016). Finally, we
used Granger Causality to investigate how high-level perceptual infor-
mation feeds into different conceptual representations in the brain. This
analysis revealed that orthography-selective sensors Granger-caused ac-
tivity in tool-selective sensors in the dorsal stream between 204 and 214
ms.
Our results show that concept-selective circuits receive rapid, feed-
forward input from perceptual (orthographic) representations. This
finding is consistent with the aforementioned two-stage models of cate-
gory learning (Ashby and Spiering, 2004;Riesenhuber and Poggio, 2002,
2000), which predict that learned perceptual representations can be
flexibly recruited by distinct higher-order circuits in order to accomplish
different tasks. These two-stage models make clear predictions regarding
the latency of concept-selective circuits, predicting that, due to their
putative location in the cortical hierarchy following object-selective
representations which have been associated with the N170 EEG signal
component (Gauthier et al., 2003;Pegado et al., 2014;Rossion et al.,
2002;Tanaka and Curran, 2001), they should be activated around 200
ms (Thorpe and Fabre-Thorpe, 2001). Yet, the temporal onset of
concept-level processing is still debated (Carreiras et al., 2014), with
many studies claiming that concept representations (frequently localized
to the ATL (Coutanche and Thompson-Schill, 2015;Jackson et al., 2015;
Lau et al., 2008;Ralph et al., 2016) are activated around 400 ms
post-stimulus onset (Kutas and Federmeier, 2011;Jackson et al., 2015).
Our results add to a growing body of literature (Bankson et al., 2018;
Chan et al., 2011;Clarke et al., 2011) showing that concept information
comes online prior to 250 ms post-stimulus onset, indicating that concept
processing can be understood as a straightforward extension of Hubel
and Wiesel’s“simple-to-complex”feedforward model of visual process-
ing from visual to concept processing. The qualitatively sustained rise in
classification accuracy above chance starting at ~160 ms post-stimulus
onset (Fig. 3B) suggests that the true onset of concept information may
be even earlier than identified in our study. Our results are in line with a
recent study that used a combination of behavioral modeling, deep
neural networks, and MEG to estimate the lower-bound of semantic
processing in the visual ventral stream at ~150 ms and a peak processing
at ~230 ms (Bankson et al., 2018).
Of note, we find the rapid onset of concept representations not only
in left anterior temporal sensors, but also in left parietal sensors. Pre-
vious work (Garcea et al., 2018;Ishibashi et al., 2011;Pobric et al.,
2010) showed evidence that left parietal tool representations encode
tool manipulation knowledge, and LATL representations store tool
function knowledge. The interplay between these representations, and
to what degree access to one relies on access to the other is unclear. In
certain theories of semantic cognition, the ATL serves as a “semantic
hub”that integrates over different attributes (shape, manipulability,
sound, etc.) of a concept and helps coordinate interactions between
them (Patterson et al., 2007;Ralph et al., 2016). Thus, some have
suggested that degeneration of the hub in semantic dementia precludes
access to manipulation knowledge in the visual dorsal stream via shape
representations in the visual ventral stream (Almeida et al., 2013;
Hodges et al., 2000). However, other lesion and TMS studies show that
tool manipulation knowledge can be accessed even when tool function
knowledge is impaired and vice versa (Buxbaum et al., 2000;Buxbaum
and Saffran, 2002;Garcea and Mahon, 2012;Ishibashi et al., 2011). Our
study supports the latter view by showing a direct feedforward func-
tional pathway between orthographic representations, likely in poste-
rior fusiform cortex, and left parietal tool representations that is
independent of the LATL. This is supported by DTI studies (Binder and
Desai, 2011;Wakana et al., 2004) that show direct anatomical con-
nections between the VWFA and LIPL, which could underlie the direct
Granger-causal connectivity seen in our data. An important limitation of
the current work is that we were unable to identify a parallel pathway
between orthography-selective sensors and anterior concept-selective
sensors in our data. However, the existence of feedforward anatom-
ical connections from posterior fusiform cortex to anterior temporal
areas has been well established (Bouhali et al., 2014;Kravitz et al.,
2013;Papinutto et al., 2016). Furthermore, in the context of concept
processing, Clarke et al. (2011) in a MEG study identified a cluster of
significant phase-locking between posterior shape selective and ante-
rior concept selective sensors in the gamma frequency between 120 and
220 ms. Thus, the literature provides strong existing evidence for a
ventral pathway from orthography to concepts, and the novelty of our
study lies in showing the existence of a parallel ventral-to-dorsal stream
pathway from orthography-selective ventral stream areas to parietal
cortex.
A notable aspect of our results is that the direct pathway between
orthography and parietal tool concepts exists after training despite the
fact that tool PW orthography, unlike tool images, does not contain any
visual features that could be leveraged to automatically access dorsal
stream manipulation information. Furthermore, our training paradigm
did not explicitly train participants on the association between orthog-
raphy and manipulation. This raises the intriguing question of how this
dorsal word-to-manipulation knowledge network is learned. Findings
from a recent computational investigation (Chen and Rogers, 2015)
suggest that feedback from other conceptual representations can help set
up this shape-to-manipulation pathway. In that study, the authors
implemented a connectionist model in which units encoding shape rep-
resentations in the visual ventral stream were linked to concept units
representing a semantic hub and to a separate set of concept units
S.R. Damera et al. NeuroImage 221 (2020) 117148
7
representing manipulation knowledge. In this model, as validated by our
results, information from shape units directly engaged the correct in-
formation in the manipulation units after training. In addition, they
found that hub units exerted a feedback influence on shape units, which
in turn influenced correct access to manipulation information. This
feedback from the semantic hub units could suggest a mechanism by
which the proper connections between mid-fusiform and parietal regions
can be set up. Alternatively, given its extensive connections with parietal
cortex, regions in prefrontal cortex (Friederici, 2009;Ruschel et al., 2014;
de Schotten et al., 2011) (e.g., in the inferior frontal gyrus, known to be
involved in semantic processing) could be involved in coordinating the
learning of these multiple perceptual-to-conceptual hierarchies in the
brain.
In summary, our results indicate that multiple perceptual-to-
conceptual hierarchies, once learned, exist independently of each other
in the brain, allowing fast, feedforward computation of function and
manipulation knowledge, respectively. Further work needs to be done to
understand how different types of representation interact with each other
to set up these hierarchies in the brain.
Declaration of competing interest
The authors declare no competing financial interests.
CRediT authorship contribution statement
Srikanth R. Damera: Formal analysis, Investigation, Writing - orig-
inal draft, Writing - review &editing. Jacob G. Martin: Formal analysis,
Writing - review &editing. Clara Scholl: Conceptualization, Methodol-
ogy, Investigation. Judy S. Kim: Conceptualization, Methodology,
Investigation. Laurie Glezer: Conceptualization, Methodology, Investi-
gation. Patrick S. Malone: Methodology. Maximilian Riesenhuber:
Supervision, Conceptualization, Methodology, Writing - original draft,
Writing - review &editing, Funding acquisition.
Acknowledgements
This work was supported by National Science Foundation Grant
1026934 to M.R., National Science Foundation Grant PIRE OISE-
0730255, and NIH Intellectual and Development Disorders Research
Center Grant 5P30HD040677.
References
Almeida, J., Fintzi, A.R., Mahon, B.Z., 2013. Tool manipulation knowledge is retrieved by
way of the ventral visual object processing pathway. Cortex 49, 2334–2344. https://
doi.org/10.1016/j.cortex.2013.05.004.
Ashby, G.F., Spiering, B.J., 2004. The neurobiology of category learning. Behav. Cognit.
Neurosci. Rev. 3, 101–113. https://doi.org/10.1177/1534582304270782.
Bankson, B.B., Hebart, M.N., Groen, I.I.A., Baker, C.I., 2018. The temporal evolution of
conceptual object representations revealed through models of behavior, semantics
and deep neural networks. Neuroimage 178, 172–182. https://doi.org/10.1016/
j.neuroimage.2018.05.037.
Bastos, A.M., Vezoli, J., Bosman, C.A., Schoffelen, J.-M., Oostenveld, R., Dowdall, J.R.,
Weerd, P.D., Kennedy, H., Fries, P., 2014. Visual areas exert feedforward and
feedback influences through distinct frequency channels. Neuron 85, 390–401.
https://doi.org/10.1016/j.neuron.2014.12.018.
Binder, J.R., Desai, R.H., 2011. The neurobiology of semantic memory. Trends Cognit.
Sci. 15, 527–536. https://doi.org/10.1016/j.tics.2011.10.001.
Bouhali, F., de Schotten, M., Pinel, P., Poupon, C., Mangin, J.-F., Dehaene, S., Cohen, L.,
2014. Anatomical connections of the visual word form area. J. Neurosci. 34,
15402–15414. https://doi.org/10.1523/JNEUROSCI.4918-13.2014.
Brem, S., Bach, S., Kucian, K., Kujala, J.V., Guttorm, T.K., Martin, E., Lyytinen, H.,
Brandeis, D., Richardson, U., 2010. Brain sensitivity to print emerges when children
learn letter–speech sound correspondences. Proc. Natl. Acad. Sci. Unit. States Am.
107, 7939–7944. https://doi.org/10.1073/pnas.0904402107.
Buxbaum, L.J., Saffran, E., 2002. Knowledge of object manipulation and object function:
dissociations in apraxic and nonapraxic subjects. Brain Lang. 82, 179–199. https://
doi.org/10.1016/S0093-934X(02)00014-7.
Buxbaum, L.J., Shapiro, A.D., Coslett, B.H., 2014. Critical brain regions for tool-related
and imitative actions: a componential analysis. Brain 137, 1971–1985. https://
doi.org/10.1093/brain/awu111.
Buxbaum, L.J., Veramontil, T., Schwartz, M.F., 2000. Function and manipulation tool
knowledge in apraxia: knowing ‘what for’but not ‘how. Neurocase 6, 83–97. https://
doi.org/10.1080/13554790008402763.
Carreiras, M., Armstrong, B.C., Perea, M., Frost, R., 2014. The what, when, where, and
how of visual word recognition. Trends Cognit. Sci. 18, 90–98. https://doi.org/
10.1016/j.tics.2013.11.005.
Chan, A.M., Baker, J.M., Eskandar, E., Schomer, D., Ulbert, I., Marinkovic, K., Cash, S.S.,
Halgren, E., 2011. First-pass selectivity for semantic categories in human
anteroventral temporal lobe. J. Neurosci. 31, 18119–18129. https://doi.org/
10.1523/JNEUROSCI.3122-11.2011.
Chen, L., Ralph, M.A., Rogers, T.T., 2017. A unified model of human semantic knowledge
and its disorders. Nat. Human Behav. 1 https://doi.org/10.1038/s41562-016-0039,
0039.
Chen, L., Rogers, T.T., 2015. A model of emergent category-specific activation in the
posterior fusiform gyrus of sighted and congenitally blind populations. J. Cognit.
Neurosci. 27, 1981–1999. https://doi.org/10.1162/jocn_a_00834.
Clarke, A., Taylor, K.I., Tyler, L.K., 2011. The evolution of meaning: spatio-temporal
dynamics of visual object recognition. J. Cognit. Neurosci. 23, 1887–1899. https://
doi.org/10.1162/jocn.2010.21544.
Coutanche, M.N., Thompson-Schill, S.L., 2015. Creating concepts from converging
features in human cortex. Cerebr. Cortex 25, 2584–2593. https://doi.org/10.1093/
cercor/bhu057.
Cui, J., Xu, L., Bressler, S.L., Ding, M., Liang, H., 2008. BSMART: a Matlab/C toolbox for
analysis of multichannel neural time series. Neural Network. 21, 1094–1104. https://
doi.org/10.1016/j.neunet.2008.05.007.
Cul, A., Baillet, S., Dehaene, S., 2007. Brain dynamics underlying the nonlinear threshold
for access to consciousness. PLoS Biol. 5, e260 https://doi.org/10.1371/
journal.pbio.0050260.
de Schotten, M., Dell’Acqua, F., Forkel, S.J., Simmons, A., Vergani, F., Murphy, D.G.,
Catani, M., 2011. A lateralized brain network for visuospatial attention. Nat.
Neurosci. 14, 1245. https://doi.org/10.1038/nn.2905.
Dehaene, S., Cohen, L., Morais, J., Kolinsky, R., 2015. Illiterate to literate: behavioural
and cerebral changes induced by reading acquisition. Nat. Rev. Neurosci. 16,
234–244. https://doi.org/10.1038/nrn3924.
Dehaene-Lambertz, G., Monzalvo, K., Dehaene, S., 2018. The emergence of the visual
word form: longitudinal evolution of category-specific ventral visual areas during
reading acquisition. PLoS Biol. 16, e2004103 https://doi.org/10.1371/
journal.pbio.2004103.
Delorme, A., Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of single-trial
EEG dynamics including independent component analysis. J. Neurosci. Methods 134,
9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009.
Dien, J., 1998. Issues in the application of the average reference: review, critiques, and
recommendations. Behav. Res. Methods Instrum. Comput. 30, 34–43. https://
doi.org/10.3758/BF03209414.
Eklund, A., Nichols, T.E., Knutsson, H., 2016. Cluster failure: why fMRI inferences for
spatial extent have inflated false-positive rates. Proc. Natl. Acad. Sci. 113,
7900–7905. https://doi.org/10.1073/pnas.1602413113.
Fahrenfort, J.J., Lamme, V., 2012. A true science of consciousness explains
phenomenology: comment on Cohen and Dennett. Trends Cognit. Sci. 16, 138–139.
https://doi.org/10.1016/j.tics.2012.01.004.
Freedman, D.J., Riesenhuber, M., Poggio, T., Miller, E.K., 2003. A comparison of primate
prefrontal and inferior temporal cortices during visual categorization. J. Neurosci. 23,
5235–5246. https://doi.org/10.1523/JNEUROSCI.23-12-05235.2003.
Friederici, A.D., 2009. Pathways to language: fiber tracts in the human brain. Trends
Cognit. Sci. 13, 175–181. https://doi.org/10.1016/j.tics.2009.01.001.
Garcea, F.E., Chen, Q., Vargas, R., Narayan, D.A., Mahon, B.Z., 2018. Task- and domain-
specific modulation of functional connectivity in the ventral and dorsal object-
processing pathways. Brain Struct. Funct. 223, 2589–2607. https://doi.org/10.1007/
s00429-018-1641-1.
Garcea, F.E., Mahon, B.Z., 2012. What is in a tool concept? Dissociating manipulation
knowledge from function knowledge. Mem. Cognit. 40, 1303–1313. https://doi.org/
10.3758/s13421-012-0236-y.
Gauthier, I., Curran, T., Curby, K.M., Collins, D., 2003. Perceptual interference supports a
non-modular account of face processing. Nat. Neurosci. 6, nn1029. https://doi.org/
10.1038/nn1029.
Glezer, L.S., Jiang, X., Riesenhuber, M., 2009. Evidence for highly selective neuronal
tuning to whole words in the “visual word form area.”. Neuron 62, 199–204. https://
doi.org/10.1016/j.neuron.2009.03.017.
Goodale, M.A., 2011. Transforming vision into action. Vis. Res. 51, 1567–1587. https://
doi.org/10.1016/j.visres.2010.07.027.
Goodale, M.A., Milner, A.D., 1992. Separate visual pathways for perception and action.
Trends Neurosci. 15, 20–25. https://doi.org/10.1016/0166-2236(92)90344-8.
Granger, C., 1969. Investigating causal relations by econometric models and cross-
spectral methods. Econometrica 37, 424. https://doi.org/10.2307/1912791.
Grech, R., Cassar, T., Muscat, J., Camilleri, K.P., Fabri, S.G., Zervakis, M.,
Xanthopoulos, P., Sakkalis, V., Vanrumste, B., 2008. Review on solving the inverse
problem in EEG source analysis. J. NeuroEng. Rehabil. 5, 25. https://doi.org/
10.1186/1743-0003-5-25.
Gregoriou, G.G., Gotts, S.J., Zhou, H., Desimone, R., 2009. High-frequency, long-range
coupling between prefrontal and visual cortex during attention. Science 324,
1207–1210. https://doi.org/10.1126/science.1171402.
Grootswagers, T., Wardle, S.G., Carlson, T.A., 2016. Decoding dynamic brain patterns
from evoked responses: a tutorial on multivariate pattern analysis applied to time
series neuroimaging data. J. Cognit. Neurosci. 1–21. https://doi.org/10.1162/jocn_a_
01068.
S.R. Damera et al. NeuroImage 221 (2020) 117148
8
Hebart, M.N., G€
orgen, K., Haynes, J.-D.D., 2014. The Decoding Toolbox (TDT): a versatile
software package for multivariate analyses of functional imaging data. Front.
Neuroinf. 8, 88. https://doi.org/10.3389/fninf.2014.00088.
Hodges, J.R., Bozeat, S., Ralph, M.A., Patterson, K., Spatt, J., 2000. The role of conceptual
knowledge in object use Evidence from semantic dementia. Brain 123, 1913–1925.
https://doi.org/10.1093/brain/123.9.1913.
Ishibashi, R., Ralph, M.A., Saito, S., Pobric, G., 2011. Different roles of lateral anterior
temporal lobe and inferior parietal lobule in coding function and manipulation tool
knowledge: evidence from an rTMS study. Neuropsychologia 49, 1128–1135.
https://doi.org/10.1016/j.neuropsychologia.2011.01.004.
Jackson, R.L., Ralph, M.A., Pobric, G., 2015. The timing of anterior temporal lobe
involvement in semantic processing. J. Cognit. Neurosci. 27, 1388–1396. https://
doi.org/10.1162/jocn_a_00788.
Jefferies, E., 2013. The neural basis of semantic cognition: converging evidence from
neuropsychology, neuroimaging and TMS. Cortex 49, 611–625. https://doi.org/
10.1016/j.cortex.2012.10.008.
Jiang, X., Bradley, E., Rini, R.A., Zeffiro, T., VanMeter, J., Riesenhuber, M., 2007.
Categorization training results in shape- and category-selective human neural
plasticity. Neuron 53, 891–903. https://doi.org/10.1016/j.neuron.2007.02.015.
Kravitz, D.J., Saleem, K.S., Baker, C.I., Mishkin, M., 2011. A new neural framework for
visuospatial processing. Nat. Rev. Neurosci. 12, 217. https://doi.org/10.1038/
nrn3008.
Kravitz, D.J., Saleem, K.S., Baker, C.I., Ungerleider, L.G., Mishkin, M., 2013. The ventral
visual pathway: an expanded neural framework for the processing of object quality.
Trends Cognit. Sci. 17, 26–49. https://doi.org/10.1016/j.tics.2012.10.011.
Kutas, M., Federmeier, K.D., 2011. Thirty years and counting: finding meaning in the
N400 component of the event-related brain potential (ERP). Ann. Rev. Psychol. 62,
621–647. https://doi.org/10.1146/annurev.psych.093008.131123.
Lau, E.F., Phillips, C., Poeppel, D., 2008. A cortical network for semantics: (de)
constructing the N400. Nat. Rev. Neurosci. 9, 920–933. https://doi.org/10.1038/
nrn2532.
Malone, P.S., Glezer, L.S., Kim, J., Jiang, X., Riesenhuber, M., 2016. Multivariate pattern
analysis reveals category-related organization of semantic representations in anterior
temporal cortex. J. Neurosci. 36, 10089–10096. https://doi.org/10.1523/
JNEUROSCI.1599-16.2016.
Maris, E., Oostenveld, R., 2007. Nonparametric statistical testing of EEG- and MEG-data.
J. Neurosci. Methods 164, 177–190. https://doi.org/10.1016/
j.jneumeth.2007.03.024.
Maurer, U., Brandeis, D., McCandliss, B.D., 2005. Fast, visual specialization for reading in
English revealed by the topography of the N170 ERP response. Behav. Brain Funct. 1,
13. https://doi.org/10.1186/1744-9081-1-13.
Medler, D.A., Binder, J.R., 2005. An on-line orthographic database of the English
language. Available from. http://www.neuro.mcw.edu/mcword/. Accessed Sept. 10,
2010.
Mion, M., Patterson, K., Acosta-Cabronero, J., Pengas, G., Izquierdo-Garcia, D.,
Hong, Y.T., Fryer, T.D., Williams, G.B., Hodges, J.R., Nestor, P.J., 2010. What the left
and right anterior fusiform gyri tell us about semantic memory. Brain 133,
3256–3268. https://doi.org/10.1093/brain/awq272.
Nosofsky, R.M., 1986. Attention, similarity, and the identification–categorization
relationship. J. Exp. Psychol. Gen. 115, 39. https://doi.org/10.1037/0096-
3445.115.1.39.
Papinutto, N., Galantucci, S., Mandelli, M., Gesierich, B., Jovicich, J., Caverzasi, E.,
Henry, R.G., Seeley, W.W., Miller, B.L., Shapiro, K.A., Gorno -Tempini, M., 2016.
Structural connectivity of the human anterior temporal lobe: a diffusion magnetic
resonance imaging study. Hum. Brain Mapp. 37, 2210–2222. https://doi.org/
10.1002/hbm.23167.
Patterson, K., Nestor, P.J., Rogers, T.T., 2007. Where do you know what you know? The
representation of semantic knowledge in the human brain. Nat. Rev. Neurosci. 8,
976–987. https://doi.org/10.1038/nrn2277.
Pegado, F., Comerlato, E., Ventura, F., Jobert, A., Nakamura, K., Buiatti, M., Ventura, P.,
Dehaene-Lambertz, G., Kolinsky, R., Morais, J., Braga, L.W., Cohen, L., Dehaene, S.,
2014. Timing the impact of literacy on visual processing. Proc. Natl. Acad. Sci. U.S.A.
111, E5233–E5242. https://doi.org/10.1073/pnas.1417347111.
Pobric, G., Jefferies, E., Ralph, M.A., 2010. Category-specific versus category-general
semantic impairment induced by transcranial magnetic stimulation. Curr. Biol. 20,
964–968. https://doi.org/10.1016/j.cub.2010.03.070.
Pulvermüller, F., 2013. How neurons make meaning: brain mechanisms for embodied and
abstract-symbolic semantics. Trends Cognit. Sci. 17, 458–470. https://doi.org/
10.1016/j.tics.2013.06.004.
Ralph, M.A., Jefferies, E., Patterson, K., Rogers, T.T., 2016. The neural and computational
bases of semantic cognition. Nat. Rev. Neurosci. 18, 42–55. https://doi.org/10.1038/
nrn.2016.150.
Riesenhuber, M., Poggio, T., 2002. Neural mechanisms of object recognition. Curr. Opin.
Neurobiol. 12, 162–168. https://doi.org/10.1016/S0959-4388(02)00304-5.
Riesenhuber, M., Poggio, T., 2000. Models of object recognition. Nat. Neurosci. 3,
1199–1204. https://doi.org/10.1038/81479.
Rossion, B., Gauthier, I., Goffaux, V., Tarr, M., Crommelinck, M., 2002. Expertise training
with novel objects leads to left-lateralized facelike electrophysiological responses.
Psychol. Sci. 13, 250–257. https://doi.org/10.1111/1467-9280.00446.
Ruschel, M., Kn€
osche, T.R., Friederici, A.D., Turner, R., Geyer, S., Anwander, A., 2014.
Connectivity architecture and subdivision of the human inferior parietal cortex
revealed by diffusion MRI. Cerebr. Cortex 24, 2436–2448. https://doi.org/10.1093/
cercor/bht098.
Sassenhagen, J., Draschkow, D., 2019. Cluster -based permutation tests of MEG/EEG data
do not establish significance of effect latency or location. Psychophysiology e13335.
https://doi.org/10.1111/psyp.13335.
Scholl, C.A., Jiang, X., Martin, J.G., Riesenhuber, M., 2013. Time course of shape and
category selectivity revealed by EEG rapid adaptation. J. Cognit. Neurosci. 26,
408–421. https://doi.org/10.1162/jocn_a_00477.
Seth, A.K., Barrett, A.B., Barnett, L., 2015. Granger causality analysis in neuroscience and
neuroimaging. J. Neurosci. 35, 3293–3297. https://doi.org/10.1523/
JNEUROSCI.4399-14.2015.
Tadel, F., Baillet, S., Mosher, J.C., Pantazis, D., Leahy, R.M., 2011. Brainstorm: a user-
friendly application for MEG/EEG analysis. Comput. Intell. Neurosci. 2011, 879716.
https://doi.org/10.1155/2011/879716.
Tanaka, J., Curran, T., 2001. A neural basis for expert object recognition. Psychol. Sci. 12,
43–47. https://doi.org/10.1111/1467-9280.00308.
Thomas, E., Hulle, M.M., Vogel, R., 2006. Encoding of categories by noncategory-specific
neurons in the inferior temporal cortex. J. Cognit. Neurosci. 13, 190–200. https://
doi.org/10.1162/089892901564252.
Thorpe, S.J., Fabre-Thorpe, M., 2001. NEUROSCIENCE: seeking categories in the brain.
Science 291, 260–263. https://doi.org/10.1126/science.1058249.
Thorpe, S., Fize, D., Marlot, C., 1996. Speed of processing in the human visual system. Am.
J. Ophthalmol. 122, 608–609. https://doi.org/10.1016/S0002-9394(14)72148-8.
Vandenberghe, R., Price, C., Wise, R., Josephs, O., Frackowiak, R., 1996. Functional
anatomy of a common semantic system for words and pictures. Nature 383, 254–256.
https://doi.org/10.1038/383254a0.
Vingerhoets, G., Acke, F., Vandemaele, P., Achten, E., 2009. Tool responsive regions in
the posterior parietal cortex: effect of differences in motor goal and target object
during imagined transitive movements. Neuroimage 47, 1832–1843. https://doi.org/
10.1016/j.neuroimage.2009.05.100.
Wakana, S., Jiang, H., Nagae-Poetscher, L.M., van Zijl, P.C., Mori, S., 2004. Fiber
tract–based atlas of human white matter anatomy 1. Radiology 230, 77–87. https://
doi.org/10.1148/radiol.2301021640.
Widmann, A., Schr€
oger, E., 2012. Filter effects and filter artifacts in the analysis of
electrophysiological data. Front. Psychol. 3, 233. https://doi.org/10.3389/
fpsyg.2012.00233.
Woo, C.-W., Krishnan, A., Wager, T.D., 2014. Cluster-extent based thresholding in fMRI
analyses: pitfalls and recommendations. Neuroimage 91, 412–419. https://doi.org/
10.1016/j.neuroimage.2013.12.058.
S.R. Damera et al. NeuroImage 221 (2020) 117148
9
