Common or redundant neural circuits for duration processing across audition and touch.

Abstract

Certain features of objects or events can be represented by more than a single sensory system, such as roughness of a surface (sight, sound, and touch), the location of a speaker (audition and sight), and the rhythm or duration of an event (by all three major sensory systems). Thus, these properties can be said to be sensory-independent or amodal. A key question is whether common multisensory cortical regions process these amodal features, or does each sensory system contain its own specialized region(s) for processing common features? We tackled this issue by investigating simple duration-detection mechanisms across audition and touch; these systems were chosen because fine duration discriminations are possible in both. The mismatch negativity (MMN) component of the human event-related potential provides a sensitive metric of duration processing and has been elicited independently during both auditory and somatosensory investigations. Employing high-density electroencephalographic recordings in conjunction with intracranial subdural recordings, we asked whether fine duration discriminations, represented by the MMN, were generated in the same cortical regions regardless of the sensory modality being probed. Scalp recordings pointed to statistically distinct MMN topographies across senses, implying differential underlying cortical generator configurations. Intracranial recordings confirmed these noninvasive findings, showing generators of the auditory MMN along the superior temporal gyrus with no evidence of a somatosensory MMN in this region, whereas a robust somatosensory MMN was recorded from postcentral gyrus in the absence of an auditory MMN. The current data clearly argue against a common circuitry account for amodal duration processing.

Full text

Page 1

Behavioral/Systems/Cognitive
CommonorRedundantNeuralCircuitsforDuration
ProcessingacrossAuditionandTouch
JohnS.Butler,1,2SophieMolholm,1,2IanC.Fiebelkorn,1,2ManuelR.Mercier,1,2,3TheodoreH.Schwartz,3
andJohnJ.Foxe1,2
1TheCognitiveNeurophysiologyLaboratory,Children’sEvaluationandRehabilitationCenter,DepartmentsofPediatricsandNeuroscience,AlbertEinstein
CollegeofMedicine,NewYork,NewYork10461,2TheCognitiveNeurophysiologyLaboratory,PrograminCognitiveNeuroscience,Departmentsof
PsychologyandBiology,TheCityCollegeofNewYork,NewYork,NewYork10031,and3DepartmentofNeurologicalSurgery,WeillCornellMedical
College,NewYorkPresbyterianHospital,NewYork,NewYork10021
Certainfeaturesofobjectsoreventscanberepresentedbymorethanasinglesensorysystem,suchasroughnessofasurface(sight,sound,
andtouch),thelocationofaspeaker(auditionandsight),andtherhythmordurationofanevent(byallthreemajorsensorysystems).
Thus,thesepropertiescanbesaidtobesensory-independentoramodal.Akeyquestioniswhethercommonmultisensorycorticalregions
process these amodal features, or does each sensory system contain its own specialized region(s) for processing common features? We
tackledthisissuebyinvestigatingsimpleduration-detectionmechanismsacrossauditionandtouch;thesesystemswerechosenbecause
fine duration discriminations are possible in both. The mismatch negativity (MMN) component of the human event-related potential
providesasensitivemetricofdurationprocessingandhasbeenelicitedindependentlyduringbothauditoryandsomatosensoryinves-
tigations. Employing high-density electroencephalographic recordings in conjunction with intracranial subdural recordings, we asked
whether fine duration discriminations, represented by the MMN, were generated in the same cortical regions regardless of the sensory
modalitybeingprobed.ScalprecordingspointedtostatisticallydistinctMMNtopographiesacrosssenses,implyingdifferentialunder-
lying cortical generator configurations. Intracranial recordings confirmed these noninvasive findings, showing generators of the audi-
toryMMNalongthesuperiortemporalgyruswithnoevidenceofasomatosensoryMMNinthisregion,whereasarobustsomatosensory
MMNwasrecordedfrompostcentralgyrusintheabsenceofanauditoryMMN.Thecurrentdataclearlyargueagainstacommoncircuitry
accountforamodaldurationprocessing.
Introduction
The separate sensory epithelia are specialized for very different
energy sources, such that light can have no impact on the hair
cellsalongthebasilarmembrane,anditisnonsensical,outsideof
poetry, to talk of tickling the photoreceptors of the retina. There
are, however, also properties of objects that can be explored and
representedbymorethanasinglesensorysystem.Wecanreadily
perceive the frequency of a vibration through both hearing and
touch, the speed of a moving bus through vision and audition,
and the duration of an event through all three of these primary
senses.Inthisway,theselatterpropertiescanbesaidtobeamodal
or sensory-independent qualia of a given object or event. This
notion of amodal properties gives rise to an obvious question
when one considers the neural processes responsible for their
analysis. If these features are sensory-independent, are they then
analyzed by a network of multisensory regions such that all sen-
sory systems feed into and rely upon multisensory amodal sys-
tems for the analysis of common properties? This is an attractive
thesis in that it furnishes what is potentially an economical solu-
tion. The alternative, whereby each sensory system has its own
specific system for analyzing such properties, seems to require
unnecessary duplication of function. However, it is also possible
that duration is an elemental feature that is best extracted at the
level of sensory registration. Of course, a complication with the
simpledichotomyoutlinedisthatnotallsensesarecreatedequal
when it comes to analyzing a given amodal feature. Audition is
extremely sensitive to temporal properties, with a far higher res-
olution than vision has. Vision is sensitive to the spatial location
of objects whereas audition is a poor substitute. Touch is more
appropriate for texture discrimination than vision. Thus, if a
givensensorysystemhasconsiderablysuperiortuningforagiven
property, one could imagine a scenario whereby more local cor-
tical real estate would be sequestered for a largely unisensory
analysisofthatproperty.Inthisscenario,thesensorysystemwith
lower sensitivity might be expected to rely on its own unisensory
circuitry, perhaps with ancillary input from the regions respon-
sible for analysis within the superior sensory system.
The current study was designed to assess this notion of com-
mon circuitry, with an eye to adjudicating between the possibil-
ReceivedJune25,2010;revisedDec.14,2010;acceptedDec.18,2010.
This work was supported by a grant from the U.S. National Institute of Mental Health J.J.F. and S.M.
(MH85322). M.R.M. received additional support from a postdoctoral fellowship award from the Swiss Na-
tional Science Foundation (PBELP3-123067). The Cartool software (http://brainmapping.unige.ch/Cartool.
htm) was programmed by Denis Brunet of the Functional Brain Mapping Laboratory, Geneva, Switzerland,
andissupportedbytheCenterforBiomedicalImagingofGenevaandLausanne.WethankourcolleagueDr.
PierfilippoDeSanctisforhelpfuldiscussions.
CorrespondenceshouldbeaddressedtoProf.JohnJ.Foxe,DepartmentsofPediatricsandNeuroscience,Albert
EinsteinCollegeofMedicine,1225MorrisParkAvenue,Bronx,NY10461.E-mail:john.foxe@einstein.yu.edu.
DOI:10.1523/JNEUROSCI.3296-10.2011
Copyright©2011theauthors 0270-6474/11/313400-07$15.00/0
3400 • TheJournalofNeuroscience,March2,2011 • 31(9):3400–3406

Page 2

ities outlined above. We chose to begin our investigations with a
studyofdurationprocessingacrosstheauditoryandsomatosen-
sory systems for two main reasons. First, our previous work in
multisensoryintegrationhasshownintimatelinksbetweenthese
two sensory systems, which share common neural circuitry dur-
ing very early sensory processing (Foxe et al., 2000, 2002, 2005;
SchroederandFoxe,2002,2005;Kayseretal.,2005;Murrayetal.,
2005). Second, duration is an amodal property for which both
sensory systems have a good degree of sensitivity and thus, nei-
ther system is so much more sensitive than the other as to render
the comparison blatantly imbalanced. Our dependent measure
was the mismatch negativity (MMN), an event related potential
(ERP) component of the auditory and somatosensory systems
(Na ¨a ¨ta ¨nen, 1992; Kekoni et al., 1997) that has been shown to
index automatic change detection for duration (Na ¨a ¨ta ¨nen, 1992;
Akatsuka et al., 2005). Here, using both high-density electrical
mapping of scalp-recorded potentials and intracranial electro-
corticographic recordings, we ask whether the MMN to auditory
and somatosensory duration change is generated in the same
cortical regions or whether each sensory system contains its own
specialized duration-processing mechanism.
MaterialsandMethods
Participants. Eight participants (five male) ranging in age from 22 to 33
years (mean 26.3) with normal hearing completed the experiment for a
modest fee of $12 per hour. All participants were right handed and re-
ported normal hearing and no known neurological deficits. All partici-
pants gave written informed consent, and all procedures were approved
by the ethical review board of The City College of New York. Ethical
guidelines were in accordance with the Declaration of Helsinki.
Apparatus. Tones were presented to the right ear via headphones
(Sennheiser HD600). Somatosensory stimulation was presented via a
low-cost linear amplifier (Piezo Systems) to the index finger of the right
hand. All experiments were performed in a darkened, acoustically and
electrically shielded room.
Mismatch task. The MMN component is a robust metric for studying
preattentive processing of stimulus properties for both the auditory and
somatosensory systems. While recording the electroencephalogram
(EEG)fromparticipants,themismatchevokedpotentialiselicitedauto-
matically by deviations in an otherwise repetitive stream of stimuli
(Na ¨a ¨ta ¨nen, 1992; Na ¨a ¨ta ¨nen et al., 2007). The auditory mismatch
(aMMN) has been extensively studied for several auditory stimulus
properties, such as duration (Na ¨a ¨ta ¨nen, 1992; De Sanctis et al., 2009),
frequency (Sams et al., 1985), and complex stimuli (Saint-Amour et al.,
2007).TheamplitudeoftheaMMNcomponentreflectstheparticipant’s
ability to discriminate between standard and deviant stimuli (Amenedo
and Escera, 2000; De Sanctis et al., 2009). The somatosensory mismatch
(sMMN),incomparison,hasnotbeennearlyaswellcharacterized.Ithas,
however, been similarly elicited for many different stimulus properties,
such as a duration (Akatsuka et al., 2005; Spackman et al., 2007), fre-
quency (Kekoni et al., 1997), and location (Shinozaki et al., 1998). As
with the aMMN, the sMMN component correlates with the ability to
discriminate between standard and deviant stimuli (Spackman et al.,
2007). The MMN is elicited by the infrequent occurrence of a deviant
stimulus within the context of a frequently occurring standard stimulus.
Itisusuallymeasuredbysubtractingtheresponsetothedeviantfromthe
responsetothestandard.FortheauditoryMMN,thisresultsinarelative
negativity ?100 ms following the onset of deviance (which is delayed
withrespecttostimulusonsetforadurationdeviant)thatisfocusedover
frontocentral scalp areas (Na ¨a ¨ta ¨nen, 1992). For the somatosensory
MMN, the timing and scalp distribution have been shown to be quite
similar, but the initial negative deflection is followed by a relative posi-
tivity (Spackman et al., 2007). Here, participants viewed a silent movie
with subtitles during stimulus delivery and were told to ignore the audi-
tory and somatosensory stimuli. In the aMMN condition, the standard
stimuli ( p ? 0.75) were 200 Hz tones of 50 ms duration and the deviant
stimuli ( p ? 0.25) were 200 Hz tones of 100 ms duration. In the sMMN
condition, the standard stimuli were vibrotactile stimuli of 200 Hz of 50
msdurationandthedeviantstimuliwerevibrotactilestimuliof200Hzof
100 ms duration. All stimuli were sinusoids convolved with a trapezoid
such that there was a 5 ms rise at the onset and a 5 ms ramp at the offset.
A separate behavioral testing session in seven additional subjects, who
didnotserveinthemainMMNexperiment,indicatedthattherewereno
differences in the ability to detect the auditory and the somatosensory
durationdeviants(t(6)??0.667,p?0.52),withmeandetectionratesof
93.5% in both conditions.
Thereweretwoblockedconditions:aMMNandsMMN.Therewerea
total of 1600 stimuli with 400 deviants per block. The order of presenta-
tion was counterbalanced across participants. The standard and deviant
stimuli were delivered in a pseudorandom order to ensure that deviants
wereneverpresentedsequentially.Theinterstimulusintervalwasfixedat
1000 ms.
EEG recording and analysis. Brain activity was recorded using a 168-
channel EEG system (BioSemi). The data were recorded at 512 Hz and
bandpassfilteredfrom2to45Hz(24dB/octave).Thedatawereanalyzed
offlineusingtheMatlabprogramminglanguage(Mathworks).Epochsof
600 ms with 100 ms prestimulus baseline were extracted from the con-
tinuousdata.Weappliedanautomaticartifactrejectioncriterionof?75
?V across all electrodes in the array. Trials with more than six artifact
channels were rejected. In trials with less than six such channels, we
interpolated any remaining bad channels using the nearest neighbor
spline (Perrin et al., 1987, 1989). The data were re-referenced to the
averageofallchannels(Micheletal.,2004;NunezandSrinivasan,2006).
The average accepted trials per condition were ?300 deviants and ?900
standards. The mismatch waveforms were classically obtained by sub-
tractingthestandardevokedpotentialfromthedeviantevokedpotential.
Mismatch responses. To test for the presence of the MMN, mean am-
plitude measurements were obtained in a 40 ms window, centered at the
group-mean peak latency for the largest negative minimum component
between 100 and 200 ms for both the aMMN and the sMMN, as well as
the following largest negative minimum component of the aMMN and
largestpositivemaximumcomponentofthesMMNfrom200to300ms.
Fromtheliterature,theelectrodeclustersurroundingthemidlinefrontal
site(Fz)wasidentifiedasaregionofinterestforboththesomatosensory
andauditoryMMN.Sincewerecordedwithahigh-densitymontage,the
Fz site hereafter indicates the average of the five nearest electrodes sites,
which gives a better signal-to-noise ratio and is a more representative
waveformofthefrontocentralregionthanthatgivenbyasingleelectrode
site.
Topographical analysis. In line with our primary question, the key
analysis strategy involved a determination of whether common circuitry
underlies both the aMMN and sMMN. For this, we used the topograph-
ical ANOVA (TANOVA), as implemented in the Cartool software, to
statistically test for possible topographical differences between the
sMMN and aMMN using global dissimilarity and nonparametric ran-
domized testing (Lehmann and Skrandies, 1980). Global dissimilarity is
an index of configuration differences between two scalp distributions,
independent of their strength as the data were normalized using the
global field power. For each subject and time point, the global dissimi-
larity indexes a single value, which varies between 0 and 2 (0, homoge-
neity; 2, inversion of topography). To create an empiric probability
distribution against which the global dissimilarity can be tested for sta-
tistical significance, the Monte Carlo MANOVA was applied, as de-
scribedpreviously(Manly,1991).TocontrolfortypeIerrors,aperiodof
statisticalsignificancewasonlyplottedandconsideredassignificantifan
alpha criterion of 0.05 or less was obtained for at least 11 consecutive
sample points (?21 ms) (Guthrie and Buchwald, 1991; Foxe and Simp-
son, 2002).
Dipolesourceanalysis.Toestimatethelocationoftheintracranialgen-
erators of the scalp-recorded grand mean MMNs, we performed source
modeling using brain electric source analysis (BESA 5.1.8; MEGIS Soft-
ware) (Scherg and Von Cramon, 1985). Results of this procedure are
reportedinthesupplementalmaterials(availableatwww.jneurosci.org).
Intracranial data. An additional dataset was collected from a patient
(female,age48years)whowasundergoinginvasiveintracranialmapping
for intractable epilepsy following an earlier right anterior temporal lobe
Butleretal.•SomatosensoryandAuditoryDurationProcessing J.Neurosci.,March2,2011 • 31(9):3400–3406 • 3401

Page 3

resection. Intracranial EEG recordings were obtained using a multiarray
grid of 48 linear contacts (6 rows ? 8 columns, 10 mm intercontact
spacing), which was placed over the auditory cortex and surrounding
regions of the right hemisphere, and from a single linear strip of six
contacts (1 row ? 6 columns, 10 mm intercontact spacing), which was
placed over somatosensory cortex of the right hemisphere.
The precise anatomical location of each electrode contact was deter-
mined by coregistration of the postoperative CT scan on preoperative
and postoperative anatomical magnetic resonance imaging (MRI) and
then normalized into the Montreal Neurological Institute space (using
SPM8 developed by Wellcome Department of Imaging Neuroscience).
The preoperative MRI was used for its accurate anatomical information,
the postoperative CT scan provided undistorted placement of the elec-
trode contacts, and the postoperative MRI allowed assessment of the
quality of the entire coregistration process since it includes both elec-
trodes and anatomical information. In supplemental Figure 3 (available
atwww.jneurosci.orgassupplementalmaterial),forthepurposeofillus-
trating activity and relative electrode locations, the grid is depicted on a
template brain.
Subdural electrodes are highly sensitive to local field potentials gener-
ated within an ?4.0 mm2area and are much less sensitive to distant
activity (Allison et al., 1999; Lachaux et al., 2005; Molholm et al., 2006;
Fiebelkorn et al., 2010), which allows for improved localization of un-
derlyingcurrentsourcesrelativetoscalp-recordedEEG.Astheelectrodes
werelocatedovertherightsideofthebrain,thetonesandsomatosensory
stimulation were presented to the left middle finger and left ear. Brain
activity was recorded using a Brain Amp system (Brain Products). Re-
cordings were digitized online at 1000 Hz and bandpass filtered offline
from2to45Hz(24db/octave).Epochsof600mswith100msprestimu-
lus baseline were extracted from the continuous data. The data were
analyzed offline using a ?350 ?V artifact rejection criterion. Statistical
analyses of intracranial data were identical to those for scalp data, except
that they were based on the variance measured across single trials of the
current source density rather than the variance of the evoked potential
measured across participants.
Current source density. Current source density (CSD) profiles were
calculated using either a five-point formula (Equation 1, below), if the
electrode was on a grid, or a three-point formula (Equation 2, below), if
the electrode was on a strip, to estimate the second spatial derivative of
thevoltageaxis.CSDprofilesprovideanindexofthelocation,direction,
and density of transmembrane current flow, the first-order neuronal
response to synaptic input (Nicholson and Freeman, 1975):
CSDi, j? 4 ? Vi, j? Vi ? 1, j? Vi ? 1, j? Vi, j ? 1? Vi, j ? 1
(1)
CSDi, j? 2 ? Vi, j? Vi ? 1, j? Vi ? 1, j,(2)
where Vi,jdenotes the recorded field potential at the ith row and jth
column in the electrode grid. CSD profiles for electrode contacts located
on the border of the grid or the ends of the strip were not calculated
becauseofthelackofnearneighbors,andnosuchelectrodeswereusedin
our analysis.
Results
Scalp recordings
InspectionofthegrandmeanERP-differencewaveforms,deviant
minus standard, showed clear mismatch responses for both the
auditory and somatosensory conditions over the Fz (Fig. 1). For
both sensory systems, there were two separable phases of the
mismatchresponse.Theauditorymismatchhadapairofsucces-
sive negative peaks, one centered at ?155 ms and the other at
?235 ms. The somatosensory mismatch had an early negative
component at ?147 ms, followed by a positive component that
peaked at ?235 ms.
To statistically verify the presence of the components at the two
phases,eachparticipant’sstandardanddeviantwaveformswereav-
eragedover?40mscenteredateachgroup-meancomponentpeak
and submitted to a t test. The auditory standard and deviant were
foundtobestatisticallydifferentforbothcomponents(121–161ms:
tdf ? 7? ?3.161, p ? 0.05; 217–256 ms: tdf ? 7? ?4.0515, p ?
0.005), and the somatosensory standard and deviant were statisti-
cally different at both components (128–168 ms: tdf ? 7? ?2.522,
p?0.05;214–254ms:tdf ? 7?8.945,p?0.005).
The scalp topographic maps representing the potential distri-
butions of the grand mean somatosensory and auditory mis-
matchesforbothphasesareshowninFigure2.Thefirstauditory
mismatch component has a frontal left-lateralized negative dis-
tribution, whereas the somatosensory has a frontocentral nega-
tivedistribution.Thesecondcomponentsforthesomatosensory
and auditory mismatch have very different topographic distri-
butions. The auditory topographic map has a slightly left-
lateralized frontocentral negative distribution whereas the
somatosensory mismatch topographic distribution has a fron-
tal positive distribution.
Figure1.
atthefrontocentralsite,Fz,fordeviant(50ms;redline),standard(100ms;greenline),andthe
subtractionwaveform(mismatch,blackline).Thetimesindicatethemidpointlatenciesofthe
40mswindowsoverwhichtheaverageamplitudeoftheMMNsweremeasured.
GrandmeanERPsforthesomatosensory(Soma)andauditory(Audio)conditions
Figure2.
and each phase (column). The MMN is plotted at peak latency for each modality at the two
differentphases.TheTANOVAtestedfortopographicaldifferencesateachtimepointbetween
thesMMNandtheaMMN.
Topographicalplotsofthegrandaveragemismatchwavesforeachmodality(row)
3402 • J.Neurosci.,March2,2011 • 31(9):3400–3406Butleretal.•SomatosensoryandAuditoryDurationProcessing

Page 4

Toverifythedifferencesintopographiesofthesomatosensory
and auditory mismatches, a TANOVA analysis was conducted
(Fig. 2). There were three clear periods of statistical difference
between the topographies of the auditory and somatosensory
mismatchresponses.Thefirsttwoperiodsalignedwiththelaten-
cies of the two components revealed during the analysis over the
frontal scalp (Fz) for both sensory modalities, at ?105–135 ms
followed by ?200–250 ms. The third period was ?275–325 ms,
which coincided with a third positive peak of the aMMN, cen-
teredat?300ms.Anotherpossibilityisthatanamodalprocessor
mightnotnecessarilybeengagedatpreciselythesametimebythe
auditoryandsomatosensorystimuli.Wethereforeperformedan
additional test between the topographies of the late aMMN and
the early sMMN. This, too, revealed statistically significant dif-
ferences between the auditory and somatosensory MMN topog-
raphies (not shown).
Intracranial recordings
All contacts were examined for the presence of an MMN (see
supplemental Fig. S3, available at www.jneurosci.org as supple-
mental material, for more extensive representation of the re-
sponses across 22 of the contact sites). For the sMMN, the
response was clearly focused along a small part of the strip over
the postcentral gyrus (PcG) and was not seen in any of the grid
contacts (see supplemental material, available at www.jneurosci.
org). PcG1(Fig. 3 and supplemental Fig. S3, available at www.
jneurosci.org as supplemental material) had the largest negative
and positive sMMNs and was used in our analyses; these re-
sponses appeared to invert polarity in channel PcG2(Fig. 3 and
supplemental Fig. S3), which was also used in our analyses. An
aMMN was not seen in any of the strip contacts. For the aMMN,
the response was somewhat more extensive due to the greater
coverage over relevant sites, and was seen in contacts over the
superiortemporalgyrus(STG).Theearlynegativitywasofgreat-
estamplitudeinSTG1,andthelaternegativityinSTG2(Fig.3and
supplemental Fig. S3)—the two contacts used from the grid for
our analyses. Figure 3 shows current source density waveforms
from the four selected subdural electrodes for both the somato-
sensory and auditory mismatch. Two adjacent electrodes (STG1
andSTG2)werepositionedalongtheSylvianfissureoverinferior
frontal gyrus and the STG. Two adjacent electrodes (PcG1and
PcG2) were positioned over the postcentral gyrus.
Inspection of the mean auditory CSD difference waveforms
averaged over trials at electrodes STG1and STG2showed a clear
aMMN response. The mean somatosensory CSD difference
waveforms at the same electrodes did not show evidence of an
Figure3.
representtheelectrodelocationsontheMRI-CTs.TheyellowandmagentaarrowsindicatetheSylvianfissureandthepostcentralgyrus,respectively.
GrandmeanCSD,atintracranialelectrodessitesdeviant(redline),standard(greenline),andthesubtractionwaveform,MMN(blackandgraylines).Theyellowandmagentadots
Butleretal.•SomatosensoryandAuditoryDurationProcessingJ.Neurosci.,March2,2011 • 31(9):3400–3406 • 3403

Page 5

sMMN response. The reverse was the case for PcG1and PcG2,
which showed clear sMMN responses but did not show aMMN
responses.Intheauditorycondition,themismatchwaveformsat
STG1showedaclearpolarityinversionatSTG2,indicatingalocal
generator. The aMMN waveform at STG1and STG2showed
polarity-inverted components centered at ?176 ms, followed by
mirrored components at ?295 ms. For the somatosensory stim-
ulation, there were no discernible MMN components and no
obvious inversions at STG1and STG2. The sMMN response at
PcG1, however, showed a clear polarity inversion at PcG2, indi-
cating a local generator of the sMMN. The sMMN at PcG1and
PcG2had inverted components centered at ?110 and 226 ms.
The aMMN at PcG1and PcG2had no discernible peaks and ex-
hibited no obvious polarity inversions.
To statistically verify the presence of the MMN responses,
individual trials were averaged over a 40 ms interval that was
centered on the peak amplitude of the average MMN waveform.
The standard and deviant trials were sorted and submitted to an
unpairedttest.Theauditorystandardanddeviantwerefoundto
be statistically different for both components at STG1(156–196
ms: tdf ? 827? ?8.753, p ? 0.0001; 276–316 ms: tdf ? 827? 5.96,
p?0.0001)andSTG2(156–196ms:tdf ? 827?6.25,p?0.0001;
242–282ms:tdf ? 827?6.70,p?0.0001),andthesomatosensory
standard and deviant were not statistically different at STG1
(156–196ms:p?0.18;276–316ms:p?0.256)andSTG2(156–
196 ms: p ? 0.7839; 242–282 ms: p ? 0.9). The somatosensory
standard and deviant were found to be statistically different for
both components at PcG1(101–141 ms: tdf ? 864? ?6.67, p ?
0.0001; 207–247 ms: tdf ? 864? 6.18, p ? 0.0001) and PcG2(95–
135 ms: tdf ? 864? ?4.27, p ? 0.0001; 206–246 ms: tdf ? 864?
?3.03, p ? 0.0005), and the auditory standard and deviant were
not statistically different at both components at PcG1(101–141
ms: p ? 0.87; 207–247 ms: p ? 0.588), and only the first compo-
nent was statistically different at PcG2(95–135 ms: tdf ? 827?
2.59, p ? 0.005; 206–246 ms: p ? 0.22). Although this latter
finding appears to suggest that there is cross-sensory MMN ac-
tivity in PcG2, observation of the waveforms themselves suggests
otherwise, since there is no evidence of an auditory evoked po-
tential at this site to the auditory standard and the deviant wave-
form shows no obvious evoked component either. Further,
follow-up running t tests indicated that the onset of differences
between the standard and deviant responses actually preceded
the onset of physical deviance, making it clear that this apparent
effect could not be related to the duration difference.
Discussion
In launching this investigation, we theorized that there was a
strong case for suspecting that the auditory and somatosensory
systems might share processing resources for amodal features,
since research on auditory–somatosensory integration processes
hadalreadyidentifiedconsiderablecommoncircuitry.Theaudi-
tory and somatosensory systems have been shown to have strong
neurophysiologicallinks(Foxeetal.,2000,2002,2005;Schroeder
et al., 2001, 2004; Lu ¨tkenho ¨ner et al., 2002; Schroeder and Foxe,
2002, 2005; Fu et al., 2003; Kayser et al., 2005; Murray et al.,
2005), making them ideal candidates in which to investigate the
processing of amodal stimulus properties. For example, multi-
contact linear array electrode recordings in macaques showed
that both the somatosensory and auditory systems feed directly
intolayerIV,theafferentinputlayer,ofthecaudomedialbeltarea
with identical timing (Schroeder and Foxe, 2002), and this mul-
tisensory region is just a single synapse from primary auditory
cortex. Using functional MRI (fMRI), Foxe et al. (2002) found
that tactile stimulation elicited activation in a very similar region
in human auditory cortex, a finding since replicated and ex-
tendedusingfMRIinmacaques(Kayseretal.,2005).Akeyques-
tionbecomeswhatthefunctionalroleofthesehierarchicallyearly
multisensory regions might be. We reasoned that one possibility
might be basic feature detection for properties common to both
sensory systems, in this case, simple duration processing. The
MMNresponsewasthereforeusedtotestwhetherthegenerators
of duration-change detection for somatosensory and auditory
cues might rely on common multisensory circuitry. Previous
work had pointed to the gross similarities between the sMMN
and the aMMN (Spackman et al., 2010), but a common circuitry
account had not been directly tested. Here, using high-density
mapping, we found that both the morphology and the topo-
graphicaldistributionsofthedurationMMNwerehighlydistinct
across sensory systems. The sMMN topography at 235 ms
showed a clear frontal positive distribution, whereas the aMMN
topographyatthesametimingshowedadistinctlydifferentfron-
tocentral negative distribution. Further supporting separate un-
derlying neural generators, source modeling solutions indicated
that the dominant neural generators resided in or close to audi-
tory and somatosensory cortical regions for the aMMN and
sMMN, respectively (supplemental data, available at www.
jneurosci.orgassupplementalmaterial).Assuch,theseresults,in
conjunction with the intracranial recordings, make it clear that
the somatosensory and auditory MMNs to duration deviants are
generated in separate cortical regions, and we find no evidence
for shared common amodal processing regions for duration
detection.
GeneraloverviewofaMMNand sMMN
The aMMN responses and topographies presented in this paper
are similar to other duration aMMNs reported in the literature
(Na ¨a ¨ta ¨nen, 1992; Amenedo and Escera, 2000; De Sanctis et al.,
2009). The aMMN response over frontal midline scalp had two
negativepeaksfollowedbyapositivepeak.Thesecondandlarger
amplitude negative peak at ?235 ms is most commonly defined
asthedurationaMMN,andherehasatopographythatisentirely
typical of an auditory duration MMN (Doeller et al., 2003; De
Sanctis et al., 2009). The intracranial data localized the generator
of the aMMN to the STG and along the Sylvian fissure, which is
consistent with findings in other studies (Dittmann-Balc ¸ar et al.,
2001; Schall et al., 2003; Edwards et al., 2005, Molholm et al.,
2005; Rinne et al., 2005).
The sMMN response over the frontal midline scalp is also
representative of what has been shown in previous literature
(Kekonietal.,1997;Shinozakietal.,1998;Spackmanetal.,2007,
2010). The sMMN waveform has two phases: an earlier negative
peak at ?145 ms, followed by a positive peak at ?235 ms. Since
this is the first study, that we are aware of, to use a high-density
montagetoassessthesMMN,adetailedassessmentandcompar-
ison of its topographical distribution to that of the aMMN was
possible.TheearlyphaseofthesMMNhadanegativefrontocen-
traldistributionthatevolvedintoalaterpositivefrontaldistribu-
tion. From the intracranial recordings, the sMMN was localized
overthepostcentralgyrus,withverysimilartimingtothatseenin
the scalp recordings, consistent with other intracranial record-
ings of the sMMN (Spackman et al., 2010).
Comparisonoftheauditoryandsomatosensory mismatch
To the best of our knowledge, only one paper has previously
looked at the sMMN and aMMN using a within-subjects design
(Restuccia et al., 2007). These authors recorded scalp EEG data
3404 • J.Neurosci.,March2,2011 • 31(9):3400–3406Butleretal.•SomatosensoryandAuditoryDurationProcessing

Page 6

frompatientswithunilateralcerebellardamagewhileconducting
a location mismatch paradigm, where the standard and deviant
were vibrotactile stimulation of the first and fifth fingers respec-
tively. When the ipsilateral hand to the damaged cerebellum was
stimulated, an abnormal sMMN was elicited, whereas stimula-
tion on the contralateral hand elicited a normal sMMN. In a
control condition, two of the patients were also tested using an
auditory MMN paradigm where the deviant feature was a fre-
quencychange;thepatients’aMMNswereunaffectedbythecer-
ebellar lesion, regardless of side of presentation. Thus, the data
point to some dissociation of the generators of the MMN across
thesetwomodalities.However,sinceRestucciaetal.(2007)were
not specifically investigating the processing of amodal stimulus
features, they did not equate the tasks across modalities, using
different feature dimensions as deviants in both (location vs fre-
quency). An amodal system would be expected to operate on a
single feature and so it is difficult to draw any direct conclusions
regarding this issue based on their data. Here, we made a direct
comparisonoftheaMMNandsMMNforthecommonfeatureof
duration.
Althoughourresultsclearlydemonstratethatsensory-specific
cortical areas code for duration processing, the possibility re-
mainsthattheseregionsmightinteractundermultisensorystim-
ulation conditions, or indeed that additional amodal regions
mightbeactivatedformultisensoryinputs.Somesupportforthis
thesis comes from a recent transcranial magnetic stimulation
(TMS) study (Bolognini et al., 2010). In this study, a single TMS
pulse was delivered over auditory processing regions along the
STG at varying intervals (60, 120, and 180 ms) after the presen-
tationoftactilestimuliuponwhichparticipantswererequiredto
performtemporaldiscriminations.ItwasshownthatTMSat180
msoverSTGresultedinimpairmentoftemporaldiscrimination,
suggesting that auditory regions on the STG were involved in
cross-sensory tactile temporal processing. These results make a
compelling case for interactions between unisensory auditory
and tactile duration-processing regions. We suggest that our
finding of anatomically distinct automatic duration processing
for auditory and tactile information makes the case for duration
being processed and represented in sensory-specific areas under
conditions of unisensory stimulation, but that these areas may
well be expected to interact when concurrently engaged by mul-
tisensory inputs. Under this account, one could posit that the
TMS manipulation of Bolognini and colleagues (2010) resulted
incoactivationofauditoryandtactileregions,andthatitwasthis
coactivation that lead to interference in tactile duration discrim-
inations. Certainly there is known interconnectivity between the
auditoryandsomatosensorysystems(SmileyandFalchier,2009),
and there are numerous examples of the bidirectional influence
of these sensory systems on perception and performance
(Jousma ¨ki and Hari, 1998; Guest et al., 2002; Guest and Spence,
2003; Lappe et al., 2008; Foxe, 2009; Sperdin et al., 2009; Yau et
al., 2009). The highly similar timing (?235 ms) of the primary
aMMN and sMMN peak responses seen here also lends further
credence to the possibility of auditory and tactile duration infor-
mation interacting under multisensory stimulation conditions,
but this remains to be directly tested.
Despite a clear dissociation of the areas involved in auditory
andsomatosensorydurationprocessingasindexedbytheMMN,
these data do not rule out the possibility that amodal duration
representations come online during the active performance of a
durationtask,orthattheremightbeamodalprocessingthatfeeds
into sensory-specific areas. For example, Moberget et al. (2008)
found that patients with cerebellar degeneration showed a selec-
tive auditory deficit in the duration MMN (relative to frequency
and location MMNs), and it is possible that the role of the cere-
bellum in the processing of temporal information is amodal. It is
also possible that there are other metrics of unisensory duration
processing that would reveal an amodal representation. How-
ever,theMMNprovidesasolidmetricofdurationprocessingfor
which there are no alternatives that we are aware of, and the loci
ofMMNgenerationiscommonlyconsideredtoreflectwherethe
deviant feature is processed (Molholm et al., 2005). It is also the
casethattheauditoryMMNisinfluencedbymultisensoryinputs
(Saint-Amour et al., 2007), and thus it seems an excellent candi-
date for measuring a potentially amodal process.
Conclusion
Wetestedwhetherprocessingoftheamodalpropertyofduration
mightrelyoncommoncircuitryacrossauditionandtouch,given
that there is a large emerging body of anatomical and physiolog-
ical work pointing to hierarchically early common multisensory
regions that process inputs from both of these sensory systems.
Instead, data from both scalp and subdural recordings point to
clearly separate generators for simple duration detections across
thesetwosensorysystems,stronglyimplicatingasensory-specific
account.
References
Akatsuka K, Wasaka T, Nakata H, Inui K, Hoshiyama M, Kakigi R (2005)
Mismatchresponsesrelatedtotemporaldiscriminationofsomatosensory
stimulation. Clin Neurophysiol 116:1930–1937.
Allison T, Puce A, Spencer DD, McCarthy G (1999) Electrophysiological
studies of human face perception. I. Potentials generated in occipitotem-
poral cortex by face and non-face stimuli. Cereb Cortex 9:415–430.
Amenedo E, Escera C (2000) The accuracy of sound duration representa-
tion in the human brain determines the accuracy of behavioural percep-
tion. Eur J Neurosci 12:2570–2574.
Bolognini N, Papagno C, Moroni D, Maravita A (2010) Tactile temporal
processing in the auditory cortex. J Cogn Neurosci 22:1201–1211.
De Sanctis P, Molholm S, Shpaner M, Ritter W, Foxe JJ (2009) Right hemi-
spheric contributions to fine auditory temporal discriminations: high-
density electrical mapping of the duration mismatch negativity (MMN).
Front Integr Neurosci 3:5.
Dittmann-Balc ¸ar A, Ju ¨ptner M, Jentzen W, Schall U (2001) Dorsolateral
prefrontal cortex activation during automatic auditory duration-
mismatch processing in humans: a positron emission tomography study.
Neurosci Lett 308:119–122.
Doeller CF, Opitz B, Mecklinger A, Krick C, Reith W, Schro ¨ger E (2003)
Prefrontal cortex involvement in preattentive auditory deviance detec-
tion: neuroimaging and electrophysiological evidence. Neuroimage
20:1270–1282.
Edwards E, Soltani M, Deouell LY, Berger MS, Knight RT (2005) High
gamma activity in response to deviant auditory stimuli recorded directly
from human cortex. J Neurophysiol 94:4269–4280.
Fiebelkorn IC, Foxe JJ, Schwartz TH, Molholm S (2010) Staying within the
lines: the formation of visuospatial boundaries influences multisensory
feature integration. Eur J Neurosci 31:1737–1743.
Foxe JJ (2009) Multisensory integration: frequency tuning of audio-tactile
integration. Curr Biol 19:R373–R375.
Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory con-
vergence during early cortical processing. Neuroreport 16:419–423.
Foxe JJ, Simpson GV (2002) Flow of activation from V1 to frontal cortex in
humans. A framework for defining “early” visual processing. Exp Brain
Res 142:139–150.
Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE
(2000) Multisensoryauditory-somatosensoryinteractionsinearlycorti-
cal processing revealed by high-density electrical mapping. Brain Res
Cogn Brain Res 10:77–83.
Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter
W,MurrayMM (2002) Auditory-somatosensorymultisensoryprocess-
ing in auditory association cortex: an fMRI study. J Neurophysiol
88:540–543.
Butleretal.•SomatosensoryandAuditoryDurationProcessingJ.Neurosci.,March2,2011 • 31(9):3400–3406 • 3405

Page 7

FuKM,JohnstonTA,ShahAS,ArnoldL,SmileyJ,HackettTA,GarraghtyPE,
Schroeder CE (2003) Auditory cortical neurons respond to somatosen-
sory stimulation. J Neurosci 23:7510–7515.
Guest S, Spence C (2003) Tactile dominance in speeded discrimination of
textures. Exp Brain Res 150:201–207.
Guest S, Catmur C, Lloyd D, Spence C (2002) Audiotactile interactions in
roughness perception. Exp Brain Res 146:161–171.
Guthrie D, Buchwald JS (1991) Significance testing of difference potentials.
Psychophysiology 28:240–244.
Jousma ¨ki V, Hari R (1998) Parchment-skin illusion: sound-biased touch.
Curr Biol 8:R190.
Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch
and sound in auditory cortex. Neuron 48:373–384.
KekoniJ,Ha ¨ma ¨la ¨inenH,SaarinenM,Gro ¨hnJ,ReinikainenK,LehtokoskiA,
Na ¨a ¨ta ¨nen R (1997) Rate effect and mismatch responses in the somato-
sensory system: ERP-recordings in humans. Biol Psychol 46:125–142.
Lachaux JP, George N, Tallon-Baudry C, Martinerie J, Hugueville L, Minotti
L, Kahane P, Renault B (2005) The many faces of the gamma band re-
sponse to complex visual stimuli. Neuroimage 25:491–501.
Lappe C, Herholz SC, Trainor LJ, Pantev C (2008) Cortical plasticity in-
duced by short-term unimodal and multimodal musical training. J Neu-
rosci 28:9632–9639.
Lehmann D, Skrandies W (1980) Reference-free identification of compo-
nents of checkerboard-evoked multichannel potential fields. Electroen-
cephalogr Clin Neurophysiol 48:609–621.
Lu ¨tkenho ¨nerB,LammertmannC,Simo ˜esC,HariR (2002) Magnetoencepha-
lographiccorrelatesofaudiotactileinteraction.Neuroimage15:509–522.
Manly BF (1991) Randomization and Monte Carlo methods in biology, 1st
Edition. London: Chapman and Hall.
Michel CM, Murray MM, Lantz G, Gonzalez S, Spinelli L, Grave de Peralta R
(2004) EEG source imaging. Clin Neurophysiol 115:2195–2222.
MobergetT,KarnsCM,DeouellLY,LindgrenM,KnightRT,IvryRB (2008)
Detecting violations of sensory expectancies following cerebellar degen-
eration: a mismatch negativity study. Neuropsychologia 46:2569–2579.
Molholm S, Martinez A, Ritter W, Javitt DC, Foxe JJ (2005) The neural
circuitry of preattentive auditory change-detection: an fMRI study of
pitch and duration mismatch negativity generators. Cereb Cortex
15:545–551.
Molholm S, Sehatpour P, Mehta AD, Shpaner M, Gomez-Ramirez M, Or-
tigue S, Dyke JP, Schwartz TH, Foxe JJ (2006) Audio-visual multisen-
sory integration in superior parietal lobule revealed by human
intracranial recordings. J Neurophysiol 96:721–729.
Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC,
Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-
somatosensorymultisensoryinteractionsinlow-levelsensorycorticesare
not constrained by stimulus alignment. Cereb Cortex 15:963–974.
Na ¨a ¨ta ¨nen R (1992) Attention and brain function. Hillsdale, New Jersey:
Erlbaum.
Na ¨a ¨ta ¨nen R, Paavilainen P, Rinne T, Alho K (2007) The mismatch negativ-
ity(MMN)inbasicresearchofcentralauditoryprocessing:areview.Clin
Neurophysiol 118:2544–2590.
Nicholson C, Freeman JA (1975) Theory of current source-density analysis
and determination of conductivity tensor for anuran cerebellum. J Neu-
rophysiol 38:356–368.
Nunez PL, Srinivasan R (2006) Electric fields of the brain: the neurophysics
of EEG, 2nd Edition. Oxford: Oxford UP.
Perrin F, Pernier J, Bertrand O, Giard MH, Echallier JF (1987) Mapping of
scalp potentials by surface spline interpolation. Electroencephalogr Clin
Neurophysiol 66:75–81.
Perrin F, Pernier J, Bertrand O, Echallier JF (1989) Spherical splines for
scalp potential and current-density mapping. Electroencephalogr Clin
Neurophysiol 72:184–187.
Restuccia D, Della Marca G, Valeriani M, Leggio MG, Molinari M (2007)
Cerebellar damage impairs detection of somatosensory input changes: a
somatosensory mismatch-negativity study. Brain 130:276–287.
RinneT,DegermanA,AlhoK (2005) Superiortemporalandinferiorfrontal
cortices are activated by infrequent sound duration decrements: an fMRI
study. Neuroimage 26:66–72.
Saint-Amour D, De Sanctis P, Molholm S, Ritter W, Foxe JJ (2007) Seeing
voices: high-density electrical mapping and source-analysis of the multi-
sensory mismatch negativity evoked during the McGurk illusion. Neuro-
psychologia 45:587–597.
Sams M, Paavilainen P, Alho K, Na ¨a ¨ta ¨nen R (1985) Auditory frequency
discrimination and event-related potentials. Electroencephalogr Clin
Neurophysiol 62:437–448.
Schall U, Johnston P, Todd J, Ward PB, Michie PT (2003) Functional neu-
roanatomyofauditorymismatchprocessing:anevent-relatedfMRIstudy
of duration-deviant oddballs. Neuroimage 20:729–736.
Scherg M, Von Cramon D (1985) Two bilateral sources of the late AEP as
identified by a spatio-temporal dipole model. Electroencephalogr Clin
Neurophysiol 62:32–44.
Schroeder CE, Foxe JJ (2002) The timing and laminar profile of converging
inputs to multisensory areas of the macaque neocortex. Brain Res Cogn
Brain Res 14:187–198.
Schroeder CE, Foxe J (2005) Multisensory contributions to low-level, ‘uni-
sensory’ processing. Curr Opin Neurobiol 15:454–458.
Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC
(2001) Somatosensory input to auditory association cortex in the ma-
caque monkey. J Neurophysiol 85:1322–1327.
Schroeder CE, Molholm S, Lakatos P, Ritter W, Foxe JJ (2004) Human-
simian correspondence in the early cortical processing of multisensory
cues. Cogn Process 5:140–151.
Shinozaki N, Yabe H, Sutoh T, Hiruma T, Kaneko S (1998) Somatosensory
automatic responses to deviant stimuli. Brain Res Cogn Brain Res
7:165–171.
Smiley JF, Falchier A (2009) Multisensory connections of monkey auditory
cerebral cortex. Hear Res 258:37–46.
Spackman LA, Boyd SG, Towell A (2007) Effects of stimulus frequency and
duration on somatosensory discrimination responses. Exp Brain Res
177:21–30.
SpackmanLA,TowellA,BoydSG (2010) Somatosensorydiscrimination:an
intracranial event-related potential study of children with refractory epi-
lepsy. Brain Res 1310:68–76.
Sperdin HF, Cappe C, Foxe JJ, Murray MM (2009) Early, low-level
auditory-somatosensory multisensory interactions impact reaction time
speed. Front Integr Neurosci 3:2.
Yau JM, Olenczak JB, Dammann JF, Bensmaia SJ (2009) Temporal fre-
quency channels are linked across audition and touch. Curr Biol 19:561–
566.
3406 • J.Neurosci.,March2,2011 • 31(9):3400–3406Butleretal.•SomatosensoryandAuditoryDurationProcessing