Differences in early sensory-perceptual processing in synesthesia: a visual evoked potential study.

Abstract

Synesthesia is a condition where stimulation of a single sensory modality or processing stream elicits an idiosyncratic, yet reliable perception in one or more other modalities or streams. Various models have been proposed to explain synesthesia, which have in common aberrant cross-activation of one cortical area by another. This has been observed directly in cases of linguistic-color synesthesia as cross-activation of the 'color area', V4, by stimulation of the grapheme area. The underlying neural substrates that mediate cross-activations in synesthesia are not well understood, however. In addition, the overall integrity of the visual system has never been assessed and it is not known whether wider differences in sensory-perceptual processing are associated with the condition. To assess whether fundamental differences in perceptual processing exist in synesthesia, we utilised high-density 128-channel electroencephalography (EEG) to measure sensory-perceptual processing using stimuli that differentially bias activation of the magnocellular and parvocellular pathways of the visual system. High and low spatial frequency gratings and luminance-contrast squares were presented to 15 synesthetes and 15 controls. We report, for the first time, early sensory-perceptual differences in synesthetes relative to non-synesthete controls in response to simple stimuli that do not elicit synesthetic color experiences. The differences are manifested in the early sensory components of the visual evoked potential (VEP) to stimuli that bias both magnocellular and parvocellular responses, but are opposite in direction, suggesting a differential effect on these two pathways. We discuss our results with reference to widespread connectivity differences as a broader phenotype of synesthesia.

Full text

Page 1

Differences in early sensory-perceptual processing in synesthesia: A visual evoked
potential study
Kylie J. Barnetta,d, John J. Foxec, Sophie Molholmc, Simon P. Kellyc, Shani Shalgia,
Kevin J. Mitchella,d,⁎, Fiona N. Newella,b,⁎
aInstitute of Neuroscience (TCIN), Lloyd Building, Trinity College Dublin, Dublin 2, Ireland
bSchool of Psychology, Trinity College Dublin, Dublin 2, Ireland
cCognitive Neurophysiology Lab, Nathan Kline Institute for Psychiatric Research, USA
dSmurfit Institute of Genetics, Trinity College Dublin, Dublin 2, Ireland
a b s t r a c ta r t i c l ei n f o
Article history:
Received 20 March 2008
Revised 2 July 2008
Accepted 12 July 2008
Available online 25 July 2008
Keywords:
Synesthesia
EEG
VEP
Magnocellular
Parvocellular
Synesthesia is a condition where stimulation of a single sensory modality or processing stream elicits an
idiosyncratic, yet reliable perception in one or more other modalities or streams. Various models have been
proposed to explain synesthesia, which have in common aberrant cross-activation of one cortical area by
another. This has been observed directly in cases of linguistic-color synesthesia as cross-activation of the
‘color area’, V4, by stimulation of the grapheme area. The underlying neural substrates that mediate cross-
activations in synesthesia are not well understood, however. In addition, the overall integrity of the visual
system has never been assessed and it is not known whether wider differences in sensory-perceptual
processing are associated with the condition. To assess whether fundamental differences in perceptual
processing exist in synesthesia, we utilised high-density 128-channel electroencephalography (EEG) to
measure sensory-perceptual processing using stimuli that differentially bias activation of the magnocellular
and parvocellular pathways of the visual system. High and low spatial frequency gratings and luminance-
contrast squares were presented to 15 synesthetes and 15 controls. We report, for the first time, early
sensory-perceptual differences in synesthetes relative to non-synesthete controls in response to simple
stimuli that do not elicit synesthetic color experiences. The differences are manifested in the early sensory
components of the visual evoked potential (VEP) to stimuli that bias both magnocellular and parvocellular
responses, but are opposite in direction, suggesting a differential effect on these two pathways. We discuss
our results with reference to widespread connectivity differences as a broader phenotype of synesthesia.
© 2008 Elsevier Inc. All rights reserved.
Introduction
It is now well established that synesthesia is a genuine perceptual
phenomenon (reviewed in Hubbard et al., 2005; Hubbard and
Ramachandran, 2005; Ward and Mattingley, 2006) yet its neural
substrates are not well understood. For the synesthete, stimulation of
a single sensory modality elicits an idiosyncratic, yet reliable
perception in one or more other modalities. Various models have
beenproposed toexplain synesthesia, which have in commontheidea
of aberrant cross-activation of one cortical area by another (Hubbard
and Ramachandran, 2005; Grossenbacher and Lovelace, 2001;
Ramachandran and Hubard, 2001; Smilek et al., 2001). Such cross-
activation has been observed directly in neuroimaging studies of one
of the most common forms of synesthesia, which we term linguistic-
color synesthesia (Barnett et al., 2008). In this form, letters of the
alphabet, numbers, days of the week, months of the year and words
each induce a consistent color percept or association. A number of
neuroimaging studies have found that color-inducing stimuli for
linguistic-color synesthetes (e.g., spoken words, visually presented
letters of the alphabet) aberrantly activate area V4, known to be
involved in color processing (Hubbard et al., 2005; Nunn et al., 2002;
Sperling et al., 2005), as well as a number of other visual and parietal
areas (Aleman et al., 2001; Paulesu et al.,1995; Rich et al., 2006; Weiss
et al., 2005). However, the poor temporal resolution of fMRI leaves
open the question of whether this cross-activation is direct, from one
area (such as the grapheme area) to another (the adjacent area V4 in
this case), or whether it arises later in time, consistent with post-
perceptual feedback from processing in higher cortical areas. In
NeuroImage 43 (2008) 605–613
Abbreviations: EEG, electroencephalogram; VEP, visual evoked potential; HSF, high
spatial frequency; LSF, low spatial frequency.
⁎ Corresponding authors. K.J. Mitchell is to be contacted at Smurfit Institute of
Genetics, Trinity College Dublin, Dublin 2, Ireland. Fax: +1 612 9211 2710. F.N. Newell,
School of Psychology, Trinity College Dublin, Dublin 2, Ireland.
E-mail addresses: Kevin.Mitchell@tcd.ie (K.J. Mitchell), Fiona.Newell@tcd.ie
(F.N. Newell).
1053-8119/$ – see front matter © 2008 Elsevier Inc. All rights reserved.
doi:10.1016/j.neuroimage.2008.07.028
Contents lists available at ScienceDirect
NeuroImage
journal homepage: www.elsevier.com/locate/ynimg

Page 2

addition, the overall integrity of the visual system has never been
assessed and it is not known whether wider differences in sensory-
perceptual processing are associated with the condition.
Electroencephalography (EEG) offers a method to resolve the
temporal characteristics of the synesthetic experience and also to
assess early sensory processing. To the best of our knowledge only
three studies have used evoked potentials to compare groups of
synesthetes with matched controls. Schiltz et al. (1999) recorded EEG
while synesthetes viewed numbers and letters that induced color
and compared their responses to controls. Synesthetes had more
positive responses over frontal and prefrontal regions that were
evident at relatively late time points (200–600 ms). The frontal
differences were interpreted as reflecting either prefrontal cortical
inhibition or multisensory integration. A more recent study recorded
auditory evoked potentials in response to words, letters and
pseudowords that induced color percepts in synesthetes. In this
study, differences between synesthetes and controls over inferior
posterior temporal sites were present as early as 122 ms after the
onset of an auditory stimulus (Beeli et al., 2008). These data were
interpreted as evidence for a rapid, automatic cross-activation of the
color area at early stages of processing, and this as the physiological
basis for the synesthetic experience. More recently, Brang et al.
(2008) reported that the contextual congruity of graphemes
presented in sentences (i.e., whether the synesthetic color of the
grapheme fit the context of the color-word it replaced in the
sentence) affected the evoked responses to achromatic graphemes in
synesthetes, who showed a more negative N1, more positive P2 and
less negative N400 component in response to contextually appro-
priate graphemes. Thus, the interaction between the expectation
induced by the preceding sentence and the synesthetic color effect
was apparent in the evoked response to graphemes at a very early
stage of processing.
In a preliminary experiment, designed to measure the time-course
of synesthetic cross-activation, we measured visual evoked potentials
(VEPs) in a set of linguistic-color synesthetes, using a 128-electrode
array, in response to visual presentation of stimuli that induced
synesthesia (e.g., letters) and control stimuli that did not induce
synesthesia (e.g., Mondrian color patterns and scrambled letters).
During this we noted a very surprising trend: synesthetes showed
altered VEPs at very early stages of processing in response to control
stimuli that did not induce synesthetic percepts (data not shown). To
investigate this intriguing finding further, we conducted a more
detailed experiment that we report here, designed specifically to
assess early visual processing and to dissect the contributions of
different visual processing streams.
The visual system is thought to be both structurally and
functionally divided into two main parallel but interacting processing
streams (Ungerleider and Mishkin,1982). The ‘dorsal’ stream is largely
driven by magnocellular lateral geniculate nucleus (LGN) inputs and
involves projections from V1 to parietal cortex. Functional processing
in the dorsal stream, sometimes referred to as the ‘where’ pathway,
involves spatial perception, action-related behavior and attention. The
‘ventral’ stream, on the other hand, is largely driven by parvocellular
inputs, and in the cortex, involves projections from V1 to the temporal
cortex. The ventral visual stream is considered the ‘what’ pathway,
being involved in processes such as object, face and word recognition
(see Ungerleider and Mishkin,1982). The functioning of each pathway
at a basic sensory level can be assessed using simple stimuli that bias
processing to one stream or the other, based on the preferred
receptive field characteristics of the retinal ganglion cells that
stimulate each pathway (Ungerleider and Mishkin, 1982). For
example, magnocellular LGN neurons are sensitive to low lumi-
nance-contrast stimuli (1% to 10% contrast) and saturate at contrast
levels of 16 to 32% (Kaplan, 1991). Additionally, the magnocellular
system is sensitive to low spatial frequency (LSF) stimuli (Derrington
and Lennie, 1984; Kaplan, 1991), but not chromatic stimuli. On the
other hand, parvocellular neurons do not begin responding vigorously
until stimuli are at a contrast level of around 8–10% and show a non-
saturating increase in amplitude across a range of luminance contrasts
(Kaplan, 1991, Tootell et al., 1988). The parvocellular system prefers
high spatial frequency (HSF) stimuli (Derrington and Lennie, 1984;
Kaplan, 1991) and its neurons are highly responsive to color (Kaplan,
1991, Merrigan and Maunsell, 1993). The differential sensitivities of
the two pathways mean that relatively simple stimuli such as Gabor
patches and isolated luminance squares (i.e., check patterns) can be
manipulated to bias each pathway and their integrity can then be
assessed using VEPs.
The VEP from scalp recordings of electrical brain activity in
response to visual stimulation have been well characterized in
humans, providing precise information on the time-course of visual
information processing in the brain and allowing for good estima-
tions of the underlying neuronal generators. The C1, the first
discernable VEP, has a central occipital scalp topography, peaks at
about 90 ms, and reflects activity in both primary and secondary
visual cortex (see Foxe and Simpson, 2002, for a detailed description).
The subsequent P1 has a more dorsal bilateral occipital scalp
topography and peaks at about 100 ms. The P1 has generators
largely in the dorsal visual stream, but also reflects activity from the
ventral visual stream (Di Russo et al., 2002). The N1, with a lateral
occipito-temporal scalp distribution, peaks at about 160 ms, with
major generators located in the ventral visual stream, in lateral
occipito-temporal areas. Importantly, componentry arising from the
dorsal or ventral visual stream can modulate independently (Doniger
et al., 2000), indicating that the integrity of each can be indepen-
dently assessed (see Butler et al., 2007).
The current study used high-density, 128-channel EEG to assess
electrophysiological markers of sensory-perceptual processing in 15
linguistic-color synesthetes and 15 age- and sex-matched non-
synesthete controls. Stimuli were designed to selectively bias the
dorsal or the ventral visual stream. To date, early sensory-perceptual
differences in synesthesia have not been assessed. Since linguistic-
color synesthesia involves stimulus features that are largely processed
in the ventral stream (i.e., letters and words), we reasoned that any
early processing differences in synesthesia might be mainly found in
the parvocellular, rather than the magnocellular pathway.
Methods
Participants
15 female linguistic-color synesthetes (mean age=34.9, SE=3.3
years) and 15 female, non-synesthete controls (mean age=37.1,
SE=3.9 years) were tested. Recruitment criteria and consistency
testing of synesthetes are described in detail in Barnett et al. (2008).
Non-synesthetes were screened with a detailed questionnaire to
ensure that they did not experience synesthesia or any associated
traits (e.g., personalities for numbers). None of our participants
reported a history of neurological disorders and all reported normal
or corrected-to-normal vision without color blindness which we
confirmed using the Ishihara test (Ishihara, 1992). The synesthesia
group contained two left-handers and the control group one left-
hander according to the Edinburgh Handedness Inventory (Oldfield,
1971). There were no between-group differences for age [t(28)=.428,
P=.627] or laterality quotient [t(28)=.445, P=.660]. The average
consistency score for synesthetes was 95.4% (SE=1.2). The majority
of synesthetes in this sample (12 of the 15) reported experiencing
concurrents (color to linguistic stimuli) in the ‘mind's eye’. Three
reported that concurrents are experienced both in the ‘mind's eye’
and projected externally. The experimental protocol was approved by
the School of Psychology Ethics Committee, Trinity College Dublin
and all participants gave written informed consent to participate
prior to the study.
606
K.J. Barnett et al. / NeuroImage 43 (2008) 605–613

Page 3

Experimental stimuli
Stimuli were presented using Presentation®software (Neurobe-
havioural Systems, http://www.nbs.neuro-bs.com/). The EEG assess-
ment was run in a quiet, dimly lit room and stimuli were presented on
an Iyamaha CRT monitor with a frame rate of 100 Hz. See Fig. 1 for
examples of stimuli.
Gabor stimuli
Gabor patches were presented against a light grey background that
was isoluminant with the mean luminance of the stimuli. The spatial
frequency of each stimulus was either 1.0 cycle/degree (‘Low Spatial
Frequency’ orLSF) or5 cycles/degree (‘High SpatialFrequency’ orHSF).
Stimuli subtended 5.9 by 3.9° of visual angle with a viewing distance
of 114 cm. Gabor stimuli were presented over 3 separate blocks in
random order for 60 ms, with random ITIs of 750, 800, 850, 900, 950,
1000 and 1050 ms. To maintain attention, line drawings of 6 different
animals were incorporated into the random sequence. Two different
types of animal images were presented within each block, one of
which was assigned as the target stimulus. Each block contained 100
presentations of each Gabor patch (for a total of 300 LSF and 300 HSF
stimuli) and 40 animals, 20 of which were targets. Participants were
required to respond to target animals and withhold responses to non-
target animals. Performance was high for both groups and there was
no difference in accuracy betweencontrols (mean=96.0%,SE=1.5) and
synesthetes (mean=98.7%, SE=3.8) [t(28)=−1.798, P=.083].
Check stimuli
Each check stimulus comprised an 8 by 8 array of isolated squares
presented against a white background. Check patterns subtended 7.2
by 7.2 of visual angle at a viewing distance of 114 cm and each square
within the array measured 0.49 by 0.49° of visual angle. Stimuli were
presented at 5 luminance contrasts (4%, 8%, 16%, 32%, 64%). A
chromatic check pattern was presented in a separate block. This
stimulus was red and appeared against a luminance-matched green
background. Perceptual isoluminance of the chromatic check pattern
was established for each participant using flicker photometry
(Greenstein et al., 1998; Zemon et al., 1991). The achromatic check
stimuli were presented over 6 blocks in random order for 60 ms, with
random ITIs of 750, 800, 850, 900, 950,1000 and 1050 ms. Each block
contained 50 presentations of each check stimulus at each contrast
level (fora total of300 presentationsofeach stimulus) and20 animals,
10 of which were assigned as targets. The last, chromatic block
consisted of 300 presentations of the colored check with the same ITIs
and animals randomly distributed. Again, participants were required
to respond to target animals and withhold responses to non-targets.
Performance was high for both groups and there was no difference in
accuracy between controls (mean=98.6%, SE=4.7) and synesthetes
(mean=99.4%, SE=1.8) [t(28)=−1.379, P=.229].
On completion of the EEG study, synesthetes completed a short
questionnaire in which they were asked whether they experienced
color associated with any of the stimuli involved in the experiment
(e.g., Gabor patches, achromatic and chromatic checks or animals).
None of the synesthetes reported color experiences elicited in
response to any of these stimuli.
EEG acquisition
EEG was recorded continuously using an Active 2 system
(Biosemi™, The Netherlands) from 128 scalp electrodes and two
additional electrodes on the mastoids. The EEG was continuously
sampled at 512 Hz and stored for offline analysis. The impedances at
each electrodewere kept below 25 kΩ. Eye movements were recorded
using two electrodes at the outer canthi of the right and left eyes and
two above and below the centre of the right eye. Data were analysed
using Brain Electric Source Analysis (BESA) Version 5.18 software
(http://www.besa.de/). To display and analyse the data, data were
filtered with a 0.5 Hz high-pass filter and a 35 Hz low-pass filter and
referenced tothe electrode FPz. Stimulus-locked datawere segmented
into epochs of −100 to 300 ms.
Artifact removal
EEGsignals werecorrected forhorizontaland vertical EOG artifacts
using the movement correction procedure outlined by Berg and
Scherg (1994). Data from noisy or flat electrodes was replaced using
spherical spline interpolation (Perrin et al., 1989) implemented by
BESA. All electrode channels were subjected to an artifact criterion
Fig. 1. Electrode location and stimuli. (A) Location of occipital electrodes subjected to ANOVA for (A) C1 (midline) and (B) P1/N1/P2 (left and right lateral occipital) components.
Magnocellular and parvocellular stimuli. (B) Examples of spatial frequency Gabor patches (1 and 5 cycles/degree) used in experimental condition 1 and (C) check stimuli used in
experimental condition 2. Check stimuli were presented at 5 different contrasts (4%, 8%, 16%, 32%, 64%) and in color.
607
K.J. Barnett et al. / NeuroImage 43 (2008) 605–613

Page 4

of ±100 μV. Trials that passed this criterion were averaged separately
for each condition and group. For the first main experimental
condition (i.e. Gabor patches), 14.2% of all trials were rejected on
average. There was no difference between groups for the number of
trials that survived artifact correction (synesthetes: mean=519.3±9.3;
controls: mean=506.3±22.0) [t(28)=.544, P=.591]. For the second
experimental condition (i.e. check patterns), 18.2% of trials were
rejected on average. Again, the groups did not differ in the
number of trials that survived artifact correction (synesthetes:
mean=1528.3±51.6; controls: mean=1415.8±80.6), [t(28)=−1.503,
P=.250].
Data analysis
Neither group elicited a C1 in response to the LSF Gabor patches.
Independent samples t-tests were used to assess ERP components
elicited by the HSF Gabor stimuli. Checks at 4% contrast did not elicit
waveforms with an obvious C1, P1, N1 morphology. To assess group
differences in this condition, grand-average waveforms were
Table 1
Latenciesat which datawere exportedforeach experimental condition (Gabors, checks)
for each component (C1, P1, N1, P2)
Component Experiment 1: Gabors Experiment 2: checks
StimulusLatency (ms) StimulusLatency (ms)
C1 LSF
HSF
n/a
65–85
8%
16%
32%
64%
Chromatic
8%
16%
32%
64%
Chromatic
8%
16%
32%
64%
Chromatic
8%
16%
32%
64%
Chromatic
66–86
56–76
56–76
50–70
45–65
90–120
95–105
85–105
75–95
85–105
150–170
130–150
115–135
115–135
125–135
205–225
210–230
210–230
210–230
180–200
P1LSF
HSF
105–125
135–155
N1 LSF
HSF
144–184
158–178
P2LSF
HSF
210–230
214–234
Fig. 2. Group averaged waveforms for controls (n=15) and synesthetes (n=15) in
response to (A) LSF (1 cycle/degree) and (B) HSF (5 cycle/degree) Gabor patches.
Waveforms show the mean amplitude (μV) of the C1 response at six midline sites.
Fig. 3. Check stimuli presented at 4% contrast. Grand-average waveforms are shown for
controls (n=15) and synesthetes (n=15). Error bars represent 1 SE of the mean and are
plotted for waveforms obtained from (A) C1 midline electrodes and P1 electrodes in the
left (B) and right (C) hemispheres respectively.
608
K.J. Barnett et al. / NeuroImage 43 (2008) 605–613

Page 5

calculated for each group and plotted with 1 SE of the mean shown
at each time window (Fig. 4). For all other stimuli, peaks were
detected for each condition in the grand-average waveforms across
groups. The latency at which the peak occurred was used to obtain
individual subject amplitudes (μV)±1 ms around the peak latency
(Table 1).
Amplitude (μV) data were analysed using SPSS software. Separate
mixed repeated measures analyses of variance (ANOVAs) were run for
eachcomponent(C1,P1,N1,P2)withgroup(controlsorsynesthetes)as
the between-subjects factor and hemisphere (left, right) and either
spatialfrequency(HSF,LSF)orcontrast(4%,8%,32%,64%)asthewithin-
subjects factors. For the N1 and P2 there was an additional within-
subjectfactorofhemisphere(left, right).Alltestswere2-tailedwithan
αlevelofPb0.05.TheERPcomponentselicitedbythechromaticcheck
patterns were analysed separately using independent t-tests.
Grand-average waveforms for each group in each condition were
used to select electrode sites where C1, P1, N1 and P2 components
were the most pronounced (Figs. 2A and B); these distributions were
entirelyconsistent with the extant literature (e.g., Murrayet al., 2001).
For the C1 component, 6 midline occipital electrodes were chosen. For
the P1, N1 and P2 components, a cluster of 7 electrodes in mirror
image locations in each hemisphere was chosen. In the left hemi-
sphere P1, N1 and P2 locations were close to or included O1, P3, PO7
and PO3. In the right hemisphere, locations were close to or included
O2, P4, PO8 and PO4 (Fig. 1).
Results
Responses to Gabor patches
While LSF Gabor stimuli did not elicit a C1 response in either
group, HSF Gabor stimuli elicited a robust C1 response. Synesthetes
showed a significantly enhanced C1 component (negative peak at
90 ms) in response to HSF Gabor patches relative to controls
[t(28)=2.412, P=.023] (Fig. 3). It should be noted that differences
appeared to precede the onset of the C1 response, which is considered
the earliest discernable VEP cortical visual response (Molholm et al.,
2002), raising the possibility of baseline differences.
In contrast, therewere no between-group differences in amplitude
for LSF or HSF Gabor stimuli for either the P1 [F(1, 28)=.001, P=.972] or
N1 [F(1, 28)=.171, P=.682] components. There was no between-group
difference for the P2 component for Gabor stimuli [t(28)=1.295,
P=.265]. Therewasa trend towardsa group byhemisphere interaction
[t(28)=3.964, P=.056]. Synesthetes had higher amplitudes to Gabors in
the left hemisphere, while controls had higher amplitudes to Gabors
in the right hemisphere.
Responses to achromatic check patterns at 4% to 64% contrast
Stimuli presented at 4% contrast are known to be processed almost
exclusively by the magnocellular system (Kaplan, 1991). We found
that the responses from synesthetes were characterized by trend
towards a decrease relative to controls at both the midline and lateral
occipital electrode sites across all time points from 70 ms on (Fig. 4
and 5). However, checks at 4% contrast did not elicit waveforms with
an obvious C1, P1, N1, P2 morphology in either group making it
impossible to statistically compare amplitudes of specific compo-
nents. Nevertheless, at both midline and lateral sites there are
multiple time points where the amplitude of the average waveforms
do not overlap at one SE of the mean and the overall, qualitative
difference is consistent across all time points and across electrode
sites (Fig. 3). We performed topographical analyses to compare the
overall differences in amplitude in response to 4% contrast checks
between synesthetes and controls in 4 ms time windows (Fig. 4). The
maps show a clear decrease in response to stimuli presented at 4%
contrast in the synesthesia group.
There were no differences in responses between the groups in the
C1 component when stimuli were presented at 8% to 64% levels of
contrast [t(28)=.324, P=.574] (responses to the 4% level of contrast
were not included because the peaks could not be identified). How-
ever, we found marked differences in P1 response amplitudes when
checks were presented at contrast levels from 8% to 64% [F(1, 28)=4.717,
P=.038]: P1 amplitudes were almost doubled in the synesthesia group
(Figs. 5 and 6). This group difference was consistent across the 8–64%
contrast range as demonstrated by the lack of a significant interaction
between group and contrast [F(3, 84)=.144, n.s.]. There were no bet-
ween-group differences found for the N1 component [F(1, 28)b1, n.s.].
A greater effect of high luminance contrast is synesthetes is consistent
with enhancement of the parvocellular pathway in this group. There
wasnosignificantP2between-groupdifferenceforcheckstimuli[t(28)=
0.658, P=.424], although there was a trend towards a group by
hemisphere interaction [t(28)=3.579, P=.068]. Synesthetes had higher
amplitudes to checks in the left hemisphere while controls had higher
amplitudes to checks in the right hemisphere. Topographical analyses
confirm the increase in the P1 amplitude of the P1 in the synesthesia
group (Fig. 6).
Responses to chromatic check patterns
Since the red checks and green background used in the chromatic
stimuli were matched for luminance, this stimulus should not activate
the magnocellular system but was instead expected to saturate the
parvocellular system (as this stimulus uses maximal color contrast).
Fig. 4. Check stimuli presented at 4% contrast. Scalp topography map showing amplitude distribution in 40 ms time windows from 60 to 300 ms for controls (n=15) and synesthetes
(n=15).
609
K.J. Barnett et al. / NeuroImage 43 (2008) 605–613

Page 6

There were no between-group differences in the amplitude of the C1
[t(28)=1.266, P=.216] or P1 [F(1, 28)=.636, P=.432] responses to this
stimulus. There was no difference in the N1 component for chromatic
stimuli in either hemisphere [both F(1, 28)b1, n.s.]. Visual inspection of
the waveform (Fig. 7) however, shows that synesthetes exhibited an
average increase in amplitude in response to chromatic checks at
Fig. 5. Grand-average waveforms are shown for controls (n=15) and synesthetes (n=15) in response to check stimuli presented in each contrast condition (8%, 16%, 32%, 64%).
Waveforms represent mean amplitude (μV) of the P1 response at seven electrodes sites in the left and right hemispheres respectively.
610
K.J. Barnett et al. / NeuroImage 43 (2008) 605–613

Page 7

around 190 ms relative to controls. Post hoc analysis of data at 180
to 200 ms showed that this difference was significant in both the left
[F(1,
P=.044] (Fig. 7).
28)=−2.712, P=.011] and right hemisphere [F(1,28)=−2.105,
Discussion
We report for the first time differences in early sensory-perceptual
components of the VEP in linguistic-color synesthetes compared to
non-synesthete controls. While synesthete and control groups had
identical VEP morphologies, responses from synesthetes were
characterized by marked differences in amplitude in early sensory-
perceptual components of the VEP in response to simple visual
stimuli, that affect both magnocellular and parvocellular pathway
responses. Importantly, there were no reports of color experiences
elicited in response to any of these simple stimuli, indicating that the
effects observed reflect fundamental differences in early visual
sensory processing in this fascinating population.
Synesthetes showed an enhanced C1 in response to HSF Gabors
that preferentially bias activation of the parvocellular system. This
difference was manifested between 65 to 85 ms, suggesting possible
hyperactivation of very early sensory processing, with the major
generators in the primary and secondary visual cortices (V1 and V2)
(see Kelly et al., in press, Martinez et al., 1999). In contrast, LSF Gabor
stimuli did not elicit a C1 response in either group, consistent with the
notion that the C1 is largely generated by parvocellular inputs (see
Schechteret al., 2005). Synesthetes showed a trend towards decreased
cortical responsiveness in response to magnocellular-biased check
patterns presented at 4% contrast. However, the lack of marked,
identifiable peaks in response to the 4% stimuli in either group makes
interpretation of this condition difficult. At 8% contrast the parvo-
cellular sub-systemis recruited (Tootellet al.,1988) andfrom this level
to 64% contrast synesthetes showed a consistent increase in the P1
response to parvocellular-biased check patterns. A simple explanation
that is consistent with all these data is that synesthetes have a
decrease in magnocellular system responsiveness and a concomitant
increase in parvocellular system responsiveness.
Synesthetes showed only a slight but non-significant increase in
the P1 in response to a chromatic check of red squares presented
against a green background. This stimulus saturates the parvocellular
system and may thus occlude any advantage in parvocellular
responsiveness in synesthetes in this condition. We also noted a
significant increase in the amplitude of the VEP at around 190 ms in a
post hoc analysis. The interpretation of this relatively late difference in
processing simple color stimuli is not obvious.
Fig. 6. Check stimuli presented at 8%,16%, 32% and 64% contrast. Scalp topography map
showing amplitude distribution for the P100 for controls (n=15) and synesthetes
(n=15).
Fig. 7. Grand-average waveforms for controls (n=15) and synesthetes (n=15) in response to the color stimuli. Waveforms represent mean amplitude (μV) of the P1 response at seven
electrodes sites in the left and right hemispheres respectively.
611
K.J. Barnett et al. / NeuroImage 43 (2008) 605–613

Page 8

This is the first study to report cortical responses to stimuli that do
not induce synesthesia in a group of linguistic-color synesthetes.
Indeed, the electrophysiological differences we report are manifested
earlier than the stimuli that commonly induce synesthesia are
recognised: letter strings, for example, are not usually differentiated
until approximately 140 to 220 ms (Allison et al., 1999, Nobre et al.,
1998, Schendan et al., 1998). The C1 and P1 components of the VEP
reflect earlier aspects of visual processing in V1 and extrastriate
regionsrespectively.BoththeC1andP1areessentiallyautomatic,non-
task-specific components that index sensory-perceptual processing in
dorsal and ventral regions of extrastriate visual cortex (Foxe and
Simpson, 2002). They have both been shown to be largely cognitively
impenetrable (see Clark and Hillyard, 1996), especially for central
stimulus presentations (Handy and Khoe, 2005). Butler et al. (2007)
argue that bottom-up processing deficits in the magnocellular system
might be responsible for visuo-perceptual differences in schizophre-
nia. For future investigations, it would be interesting to determine the
behavioral correlates of the effectswe observed here.Forexample, it is
likely that synesthetes will show heightened sensitivity in tasks that
recruit parvocellular systems such as color discrimination or differ-
ences in the resolution of high spatial frequency information. There is
alreadysomecomplementarybehavioralevidencetosuggestlowlevel
sensory differences in synesthesia. Banissy et al. (2008) report that
synestheteswhoexperiencetouchwhenwatchingsomeoneelsebeing
touched, have heightened spatial tactile discrimination of gratings.
Additionally, linguistic-color synesthetes have been found to perform
better on a test color perception (Yaro and Ward, 2007).
There are a number of possible scenarios to explain the relation-
ship of the differences we observe in early visual processing to the
overt manifestation of the synesthesia phenotype in these subjects,
i.e., the induction of specific color percepts in response to specific
linguistic stimuli. First, these phenomena may be caused indepen-
dently by differences in cortical wiring at several levels and unrelated
functionally. Second, differences in early visual processing may
indirectly cause a tendency to develop the paired associations of
inducing stimuli with color percepts. Perhaps the increased responses
to stimuli that are biased towards the parvocellular processing stream
reflect an over-elaboration of this pathway that, through the period
when graphemes are being learned, alters the normal consolidation of
responses to graphemes and manifests as a tendency to co-activate
color responses. It is difficult to see why hyperactivation alone would
have this result but it cannot be excluded as a possibility, especially as
the perceptual consequences of the differences in VEP amplitudes we
observe here are unknown. It is also not clear, however, whether this
model can be extended to explain other types of synesthesia (Barnett
et al., 2008, Bargary and Mitchell, 2008). Third, the experience of
synesthesia may, by altering perceptual experience, feed back some-
how and result in alterations in early visual circuitry. This seems
highly unlikely, especially as early visual processing areas mature
much earlier than higher-order areas associated with letter proces-
sing, for example (reviewed in Dehaene and Cohen, 2007, Guillery,
2005). The least convoluted interpretation is that the early differences
reflect altered circuitry in early visual areas, while the cross-activation
that results in the overt perceptual phenotype of synesthesia is the
result of altered circuitry between higher perceptual areas (e.g. the
grapheme area and V4) (Ramachandran and Hubbard, 2001). The
concurrent experience of color predominates in synesthesia. If
adjacency (e.g., between grapheme and color regions in the fusiform)
influences synesthetic experience we might expect face-color
synesthesia to be as common as linguistic-color synesthesia (in fact
it is extremely rare). The current data suggest that the balance of
spatial frequency and contrast inputs differ in the visual system of
synesthetes. This may be a factor in explaining why graphemes might
more commonly induce synesthesia, given their high spatial
frequency and contrast, whereas face perception relies on low spatial
frequency information.
It will be interesting to determine whether similar differences in
early visual processing are apparent in other forms of synesthesia,
especially those without obvious visual involvement such as tasting
words (Ward and Simner, 2003), for example, and also to ask
whether such early processing differences extend to other sensory
domains. A recent study reported differences in the auditory evoked
potential of synesthetes that occurred at around 122 ms (Beeli et al.,
2008). This was interpreted as the signature of very early cross-
activation of the color area, leading to the synesthetic experience.
However, synesthetes in this study reported the induction of color
even to the control pseudoword stimuli used, preventing a
comparison with stimuli that did not induce synesthesia. Given our
findings, an alternative possibility is that the early differences in the
auditory evoked potential were not in fact specific to synesthesia-
inducing stimuli nor directly linked to synesthetic cross-activation,
but rather representative of more general early processing differences
in the auditory domain similar to those we observe in the visual
domain.
We and others have recently described the co-occurrence in single
families of diverse forms of synesthesia (Barnett et al., 2008, Ward and
Simner, 2005). We proposed that connectivity differences might be
initially widespread in synesthetes but resolved differently through
experience-dependent mechanisms or subject to stochastic develop-
mental variation, resulting in an apparently discrete expression of
synesthesia in each individual (Barnett et al., 2008, Bargary and
Mitchell, 2008). The current data are consistent with this model, as
early differences in visual processing may be an independent marker
of widespread connectivitydifferences. Theyarealso consistent witha
recent diffusion tensor imaging study which found greater anisotropic
diffusion indicating increased structural connectivity in a group of
synesthetes (Rouw and Scholte, 2007). Interestingly, differences were
not confined to regions of fusiform and inferior temporal cortex,
where the grapheme area and V4 are located, but were also present in
parietal and frontal regions. Differences in cortical circuitry in
synesthetes may be thus far more extensive than the apparently
discrete, overt phenomenology of synesthesia would suggest.
Acknowledgments
We thank all participants for their time. Our thanks also go to
Doreen Hoerold and Redmond O'Connell for their help with data
collection. This research was funded by grant HRB RP/2004/191 from
the Health Research Board of Ireland to KJM and FNN (principal
investigators) and by Trinity College Institute of Neuroscience (TCIN),
Trinity College Dublin, Ireland. Drs. Molholm, Foxe and Kelly received
additional support from a US National Institute of Mental Health grant
(RO1 – MH65350 to JJF).
References
Aleman, A., Rutten, G.J.M., Sitskoorn, M.M., Dautzenberg, G., Ramsey, N.F., 2001.
Activation of striate cortex in the absence of visual stimulation: an fMRI study of
synesthesia. Neuroreport 12, 2827–2830.
Allison, T., Puce, A., Spencer, D.D., McCarthy, G., 1999. Electrophysiological studies of
human face perception I: potentials generated in occipitotemporal cortex by face
and non-face stimuli. Cereb. Cortex 9, 415–430.
Banissy, M., Walsh, V., Ward, J., 2008. Mirror-touch synaesthesia is associated with
enhanced tactile discrimination. Abstract presented at the Cognitive Neuroscience
Society meeting.
Bargary, G., Mitchell, K.J., 2008. Synaesthesia and cortical connectivity. Trends Neurosci.
31 (7), 335–342.
Barnett, K.J., Finucane, C., Asher, J., Bargary, G., Corvin, A.P., Newell, F.N., Mitchell, K.J.,
2008. Familial patterns and origins of individual differences in synaesthesia.
Cognition 106, 871–893.
Beeli, G., Esslen, M., Jäncke, L., 2008. Time course of neural activity correlated with
colored-hearing synesthesia. Cereb. Cortex 35, 379–385.
Berg, P., Scherg, M., 1994. A multiple source approach to the correction of eye artifacts.
Electroencephalogr. Clin. Neurophysiol. 90, 229–241.
Brang, D., Edwards, L., Ramachandran, V.S., Coulson, S., 2008. Is the Sky 2? Contextual
priming in grapheme-color synaesthesia. Psychol. Sci. 19 (5), 421–428.
612
K.J. Barnett et al. / NeuroImage 43 (2008) 605–613

Page 9

Butler, P.D., Martinez, A., Foxe, J.J., Kim, D., Zemon, V., Silipo, G., Mahooney, J., Shpaner,
M., Jalbrzikowski, M., Javitt, D.C., 2007. Subcortical visual dysfunction in schizo-
phrenia drives secondary impairments. Brain 130, 417–430.
Clark, V.P., Hillyard, S.A., 1996. Spatial selective attention affects early extrastriate but
notstriate components of the visual evoked potential. J. Cogn. Neurosci. 8, 387–402.
Dehaene, S., Cohen, L., 2007. Cultural recycling of cortical maps. Neuron 56, 384–398.
Derrington, A.M., Lennie, P., 1984. Spatial and temporal contrast sensitivities of
neurones in lateral geniculate nucleus of macaque. J. Physiol. 357, 219–240.
Di Russo, F., Martinez, A., Sereno, M.I., Pitzalis, S., Hillyard, S.A., 2002. Cortical sources of
the early components of the visual evoked potential. Hum. Brain Mapp.15, 95–111.
Doniger, G.M., Foxe, J.J., Murray, M.M., Higgins, B.A., Snodgrass, J.G., Schroeder, C.E.,
Javitt, D.C., 2000. Activation time course of ventral visual stream object-recognition
areas: High density electrical mapping of perceptual closure processes. J. Cogn.
Neurosci. 12, 615–621.
Foxe, J.J., Simpson, G.V., 2002. Flow of activation from V1 to frontal cortex in humans: a
framework for defining “early” visual processing. Exp. Brain Res. 142, 139–150.
Greenstein,V.C.,Seliger,S.,Zemon,V.,Ritch,R.,1998.Visualevokedpotentialassessment
of the effects of glaucoma on visual sub-systems. Vis. Res. 38,1901–1911.
Grossenbacher, P.G., Lovelace, C.T., 2001. Mechanisms of synaesthesia: cognitive and
physiological constraints. Trends Cogn. Sci. 5, 36–41.
Guillery, R.W., 2005. Is postnatal neocortical maturation hierarchical? Trends Neurosci.
28, 512–517.
Handy, T.C., Khoe, W., 2005. Attention and sensory gain control: a peripheral visual
process? J. Cogn. Neurosci. 17, 1936–1949.
Hubbard, E.M., Ramachandran, V.S., 2005. Neurocognitive mechanisms of synesthesia.
Neuron 48, 509–520.
Hubbard, E.M., Arman, A.C., Ramachandran, V.S., Boynton, G.M., 2005. Individual
differences among grapheme-color synesthetes: Brain-behavior correlations.
Neuron 45, 975–984.
Ishihara, S., 1992. Ishihara's Tests for Colour-Blindness. Kanehara & Co, Tokyo.
Kaplan, E., 1991. The receptive field of retinal ganglion cells in cat and monkey. In:
Leventhal, A.G. (Ed.), Vision and Visual Dysfunction. CRC Press, Boston, pp.10–40.
Kelly, S.P., Gomez-Ramirez, M., Foxe, J.J., (in press). Spatial attention modulates initial
afferent activity in human primary visual cortex. Cereb. Cortex
Martínez, A., Anllo-Vento, L., Sereno, M.I., Frank, L.R., Buxton, R.B., Dubowitz, D.J., Wong,
E.C., Hinrichs, H., Heinze, H.J., Hillyard, S.A., 1999. Involvement of striate and
extrastriate visual cortical areas in spatial attention. Nat. Neurosci. 2, 364–369.
Merigan, W.H., Maunsell, J.H.R., 1993. How parallel are the primate visual pathways?
Annu. Rev. Neurosci. 16, 369–402.
Molholm, S., Ritter, W., Murray, M.M., Javitt, D.C., Schroeder, C.E., Foxe, J.J., 2002.
Multisensory auditory–visual interactions during early sensory processing in
humans: a high-density electrical mapping study. Cogn. Brain Res. 14, 115–128.
Murray, M.M., Foxe, J.J., Higgins, B.A., Javitt, D.C., Schroeder, C.E., 2001. Visuo-spatial
neural response interactions in early cortical processing during a simple reaction
time task: a high-density electrical mapping study. Neuropsychologia 39, 828–844.
Nobre, A.C., Allison, T., McCarthy, G., 1998. Modulation of human extrastriate visual
processing by selective attention to colours and words. Brain 121, 1357–1368.
Nunn, J.A., Gregory, L.J., Brammer, M., Williams, S.C.R., Parslow, D.M., Morgan, M.J.,
Morris, R.G., Bullmore, E.T., Baron-Cohen, S., Gray, J.A., 2002. Functional magnetic
imaging of synaesthesia: activation of V4/V8 by spoken words. Nat. Neurosci. 5,
371–375.
Oldfield, R.C., 1971. The assessment and analysis of handedness: The Edinburgh
Handedness Inventory. Neuropsychologia 9, 97–113.
Paulesu, E., Harrison, J., Baron-Cohen, S., Watson, J.D.G., Goldstein, L.H., Heather, J.,
Frackowiak, R.S.J., Frith, C.D., 1995. The physiology of coloured hearing: A PET
activation study of colour-word synaesthesia. Brain 118, 661–676.
Perrin, F., Pernier, J., Bertrand, O., Echallier, J.F.,1989. Spherical splines for scalp potential
and current density mapping. Electroencephalogr. Clin. Neurophysiol. 72, 184–187.
Ramachandran, V.S., Hubbard, E.M., 2001. Psychophysical investigations into the neural
basis of synaesthesia. Proc. R. Soc. Lond. 268, 979–983.
Rich, A.N., Williams, M.A., Puce, A., Syngeniotis, A., Howard, M.A., McGlone, F.,
Mattingley, J.B., 2006. Neural correlates of imagined and synaesthetic colours.
Neuropsychologia 44 (14), 2918–2925.
Rouw, R., Scholte, S., 2007. Increased structural connectivity in grapheme-colour
synaesthesia. Nat. Neurosci. 10, 792–797.
Schechter, I., Butler, P.D., Zemon, V.M., Revheim, N., Saperstein, A.M., Jalbrzikowski, M.,
Pasternak, R., Silipo, G., Javitt, D.C., 2005. Impairments in generation of early-stage
transient visual evoked potentials to magno- and parvocellular-selective stimuli in
schizophrenia. Clin. Neurophysiol. 116, 2204–2215.
Schendan, H.E., Ganis, G., Kutas, M., 1998. Neurophysiological evidence for visual
perceptual categorization of words and faces within 150 ms. Psychophysiology 35,
240–251.
Schiltz, K., Trocha, K., Wieringa, B.M., Emdrich, H.M., Johannes, S., Münte, T.F., 1999.
Neurophysiological aspects of synaesthetic experience. J. Neuropsychiat. Clin.
Neurosci. 11, 58–65.
Smilek, D., Dixon, M.J., Cudahy, C., Merikle, P.M., 2001. Synaesthetic photisms influence
visual perception. J. Cogn. Neurosci. 13, 930.
Sperling, J.M., Prvulovic, D., Linden, D.E.J., Singer, W., Stirn, A., 2005. Neuronal correlates
of colour-graphemic synaesthesia: A fMRI study. Cortex 42, 203–295.
Tootell, R.B., Hamilton, S.L., Switkes, E., 1988. Functional anatomy of macaque striate
cortex. IV. Contrast and magno-parvo streams. J. Neurosci. 8, 1504–1609.
Ungerleider, L.G., Mishkin, M., 1982. Two cortical visual systems. In: Ingle, D.J.,
Mansfield, R.J.W., Goodale, M.S. (Eds.), The Analysis of Visual Behavior. MIT Press
Cambridge, MA, pp. 549–586.
Ward, J., Simner, J., 2003. Lexical-gustatory synaesthesia: Linguistic and conceptual
factors. Cognition 89 (3), 237–261.
Ward, J., Simner, J., 2005. Is synaesthesia an X-linked dominant trait with lethality in
males? Perception 34, 611–623.
Ward, J., Mattingley, J.B., 2006. Synaesthesia: an overviewof contemporary findings and
controversies. Cortex 42 (2), 129–136.
Weiss, P.H., Zilles, K., Fink, G.R., 2005. When visual perception causes feeling: enhanced
crossmodal processing in grapheme-color synaesthesia. Neuroimage 28, 859–868.
Yaro, C., Ward, J., 2007. Searching for Shereshevskii: what is superior about the memory
of synaesthetes? Q. J. Exp. Psychol. 60 (5), 681–695.
613
K.J. Barnett et al. / NeuroImage 43 (2008) 605–613