ArticlePDF Available

The eardrums move when the eyes move: A multisensory effect on the mechanics of hearing


Abstract and Figures

Interactions between sensory pathways such as the visual and auditory systems are known to occur in the brain, but where they first occur is uncertain. Here, we show a multimodal interaction evident at the eardrum. Ear canal microphone measurements in humans (n = 19 ears in 16 subjects) and monkeys (n = 5 ears in three subjects) performing a saccadic eye movement task to visual targets indicated that the eardrum moves in conjunction with the eye movement. The eardrum motion was oscillatory and began as early as 10 ms before saccade onset in humans or with saccade onset in monkeys. These eardrum movements, which we dub eye movement-related eardrum oscillations (EMREOs), occurred in the absence of a sound stimulus. The amplitude and phase of the EMREOs depended on the direction and horizontal amplitude of the saccade. They lasted throughout the saccade and well into subsequent periods of steady fixation. We discuss the possibility that the mechanisms underlying EMREOs create eye movement-related binaural cues that may aid the brain in evaluating the relationship between visual and auditory stimulus locations as the eyes move.
Experimental design and results. (A) Recordings for all subjects were made via a microphone (Mic) in the ear canal set into a custom-fit ear bud. On each trial, a subject fixated on a central LED and then made a saccade to a target LED (−24° to +24° horizontally in 6° increments and 6° above the fixation point) without moving his/her head. The ±24° locations were included on only 4.5% of trials and were excluded from analysis (Methods); other target locations were equally likely (∼13% frequency). (B) Humans (black text) received randomly interleaved silent and click trials (50% each). Clicks were played via a sound transducer coupled with the microphone at four times during these trials: during the initial fixation and saccade and at 100 ms and 200 ms after target fixation. Monkeys' trials had minor timing differences (red text), and all trials had one click at 200-270 ms after target fixation (red click trace). (C) Average eye trajectories for one human subject and session for each of the included targets are shown; colors indicate saccade target locations from ipsilateral (blue) to contralateral (red). deg, degrees; Horiz., horizontal; Vert., vertical. Mean eye position is shown as a function of time, aligned on saccade onset (D) and offset (E). H, horizontal; V, vertical. Mean microphone recordings of air pressure in the ear canal, aligned on saccade onset (F) and offset (G), indicate that the eardrum oscillates in conjunction with eye movements. The phase and amplitude of the oscillation varied with saccade direction and amplitude, respectively. The oscillations were, on average, larger when aligned on saccade onset than when aligned on saccade offset.
Content may be subject to copyright.
The eardrums move when the eyes move: A
multisensory effect on the mechanics of hearing
Kurtis G. Gruters
, David L. K. Murphy
, Cole D. Jenson
, David W. Smith
, Christopher A. Shera
and Jennifer M. Groh
Department of Psychology and Neuroscience, Duke University, Durham, NC 27708;
Department of Neurobiology, Duke University, Durham, NC 27708;
Duke Institute for Brain Sciences, Duke University, Durham, NC 27708;
Program in Behavioral and Cognitive Neuroscience, Department of Psychology,
University of Florida, Gainesville, FL 32611;
Caruso Department of Otolaryngology, University of Southern California, Los Angeles, CA 90033;
Department of Physics and Astronomy, University of Southern California, Los Angeles, CA 90033
Edited by Peter L. Strick, University of Pittsburgh, Pittsburgh, PA, and approved December 8, 2017 (received for review October 19, 2017)
Interactions between sensory pathways such as the visual and
auditory systems are known to occur in the brain, but where they
first occur is uncertain. Here, we show a multimodal interaction
evident at the eardrum. Ear canal microphone measurements in
humans (n=19 ears in 16 subjects) and monkeys (n=5earsin
three subjects) performing a saccadic eye movement task to visual
targets indicated that the eardrum moves in conjunction with the
eye movement. The eardrum motion was oscillatory and began as
early as 10 ms before saccade onset in humans or with saccade
onset in monkeys. These eardrum movements, which we dub eye
movement-related eardrum oscillations (EMREOs), occurred in the
absence of a sound stimulus. The amplitude and phase of the
EMREOs depended on the direction and horizontal amplitude of
the saccade. They lasted throughout the saccade and well into sub-
sequent periods of steady fixation. We discuss the possibility that
the mechanisms underlying EMREOs create eye movement-related
binaural cues that may aid the brain in evaluating the relationship
between visual and auditory stimulus locations as the eyes move.
reference frame
otoacoustic emissions
middle ear muscles
Visual information can aid hearing, such as when lip reading
cues facilitate speech comprehension. To derive such benefits,
the brain must first link visual and auditory signals that arise from
common locations in space. In species with mobile eyes (e.g., hu-
mans, monkeys), visual and auditory spatial cues bear no fixed re-
lationship to one another but change dramatically and frequently as
the eyes move, about three times per second over an 80° range of
space. Accordingly, considerable effort has been devoted to de-
termining where and how the brain incorporates information about
eye movements into the visual and auditory processing streams (1).
In the primate brain, all of the regions previously evaluated have
shown some evidence that eye movements modulate auditory
processing [inferior colliculus (26), auditory cortex (79), parietal
cortex (1012), and superior colliculus (1317)]. Such findings raise
the question of where in the auditory pathway eye movements first
impact auditory processing. In this study, we tested whether eye
movements affect processing in the auditory periphery.
The auditory periphery possesses at least two means of tailoring
its processing in response to descending neural control (Fig. 1).
First, the middle ear muscles (MEMs), the stapedius and tensor
tympani, attach to the ossicles that connect the eardrum to the
oval window of the cochlea. Contraction of these muscles tugs on
the ossicular chain, modulating middle ear sound transmission and
moving the eardrum. Second, within the cochlea, the outer hair
cells (OHCs) are mechanically active and modify the motion of
both the basilar membrane and, through mechanical coupling via
the ossicles, the eardrum [i.e., otoacoustic emissions (OAEs)]. In
short, the actions of the MEMs and OHCs affect not only the
response to incoming sound but also transmit vibrations backward
to the eardrum. Both the MEMs and OHCs are subject to
descending control by signals from the central nervous system
(reviewed in refs. 1820), allowing the brain to adjust the cochlear
encoding of sound in response to previous or ongoing sounds in
either ear and based on global factors, such as attention (2127).
The collective action of these systems can be measured in real
time with a microphone placed in the ear canal (28). We used this
technique to study whether the brain sends signals to the auditory
periphery concerning eye movements, the critical information
needed to reconcile the auditory spatial and visual spatial worlds.
The Eardrums Move with Saccades. Sixteen humans executed sac-
cades to visual targets varying in horizontal position (Fig. 2A).
Half of the trials were silent, whereas the other half incorporated
a series of task-irrelevant clicks presented before, during, and at
two time points after the saccades (Fig. 2B). This allowed us to
compare the effects of eye movement-related neural signals on
the auditory periphery in silence as well as in the presence of
sounds often used to elicit OAEs.
We found that the eardrum moved when the eyes moved, even
in the absence of any externally delivered sounds. Fig. 2 Dand E
shows the average eye position as a function of time for each
target location for the human subjects on trials with no sound
stimulus, aligned with respect to (w.r.t.) saccade onset (Fig. 2D)
and offset (Fig. 2E). The corresponding average microphone
The peripheral hearing system contains several motor mecha-
nisms that allow the brain to modify the auditory transduction
process. Movements or tensioning of either the middle ear
muscles or the outer hair cells modifies eardrum motion, pro-
ducing sounds that can be detected by a microphone placed in
the ear canal (e.g., as otoacoustic emissions). Here, we report a
form of eardrum motion produced by the brain via these sys-
tems: oscillations synchronized with and covarying with the di-
rection and amplitude of saccades. These observations suggest
that a vision-related process modulates the first stage of hearing.
In particular, these eye movement-related eardrum oscillations
may help the brain connect sights and sounds despite changes in
the spatial relationship between the eyes and the ears.
Author contributions: K.G.G., D.L.K.M., and J.M.G. designed research; K.G.G., D.L.K.M.,
and C.D.J. performed research; K.G.G., D.L.K.M., and J.M.G. analyzed data; and K.G.G.,
D.L.K.M., D.W.S., C.A.S., and J.M.G. wrote the paper.
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
This open access article is distributed under Creative Commons Attribution-NonCommercial-
NoDeriv atives L icense 4.0 ( CC BY-NC- ND).
Data deposition: The data and code reported in this paper have been deposited in fig-
share (
K.G.G. and D.L.K.M. contributed equally to this work.
To whom correspondence should be addressed. Email:
This article contains supporting information online at
1073/pnas.1717948115/-/DCSupplemental. PNAS
Published online January 23, 2018
Downloaded by guest on October 26, 2021
voltages are similarly aligned and color-coded for saccade direction
and amplitude (Fig. 2 Fand G). The microphone readings oscil-
lated, time-locked to both saccade onset and offset with a phase
that depended on saccade direction. When the eyes moved toward
a visual target contralateral to the ear being recorded, the micro-
phone voltage deflected positively, indicating a change in ear canal
pressure, beginning about 10 ms before eye movement onset. This
was followed by a more substantial negative deflection at about
5 ms after the onset of the eye movement, after which additional
oscillatory cycles occurred. The period of the oscillation is typically
about 30 ms (33 Hz). For saccades in the opposite (ipsilateral)
direction, the microphone signal followed a similar pattern but in
the opposite direction: The initial deflection of the oscillation was
negative. The amplitude of the oscillations appears to vary with the
amplitude of the saccades, with larger saccades associated with
larger peaks than those occurring for smaller saccades.
Comparison of the traces aligned on saccade onset vs. saccade
offset reveals that the oscillations continue into at least the initial
portion of the period of steady fixation that followed saccade
offset. The eye movement-related eardrum oscillation (EMREO)
observed following saccade offset was similar in form to that ob-
served during the saccade itself when the microphone traces were
aligned on saccade onset, maintaining their phase and magnitude
dependence on saccade direction and length. The fact that this
postsaccadic continuation of the EMREO is not seen when the
traces are aligned on saccade onset suggests a systematic re-
lationship between the offset of the movement and the phase of
the EMREO, such that variation in saccade duration obscures the
ongoing nature of the EMREO.
To obtain a portrait of the statistical significance of the re-
lationship between the direction and amplitude of the eye
movements and the observed microphone measurements of ear
canal pressure across time, we calculated a regression of mi-
crophone voltage vs. saccade target location for each 0.04-ms
sample from 25 ms before to 100 ms after saccade onset. The
regression was conducted separately for each individual subject.
Since this involved many repeated statistical tests, we compared the
real results with a Monte Carlo simulation in which we ran the
same analysis but scrambled the relationship between each trial
and its true saccade target location (details are provided in Meth-
ods). As shown in Fig. 3A, the slope of the regression involving the
real data (red trace, averaged across subjects) frequently deviates
from zero, indicating a relationship between saccade amplitude
and direction and microphone voltage. The value of the slope os-
cillates during the saccade period, beginning 9 ms before and
continuing until about 60 ms after saccade onset, matching the
oscillations evident in the data in Fig. 2F. In contrast, the scram-
bled data trace (Fig. 3A, gray) is flat during this period. Similarly,
values) of the real data deviates from the
scrambled baseline in a similar but slightly longer time frame,
dropping back to the scrambled data baseline at 75100 ms after
saccade onset (Fig. 3C). Fig. 3Eshows the percentage of subjects
showing a statistically significant (P<0.05) effect of saccade target
location at each time point. This curve reaches 80100% during the
peaks of the oscillations observed at the population level. A similar,
although weaker, dependence of the EMREO on target location
wasobservedwhentherecordings were synchronized to saccade
offset (Fig. 3 B,D,andF). We repeated this test in an additional
dataset (human dataset II; Methods) involving finer grained sam-
pling within a hemifield (i.e., analyzing target locations within the
contralateral or ipsilateral hemifield). This analysis confirmed the
relationship between the microphone voltage and saccade ampli-
tude, as well as saccade direction (Fig. S1).
EMREO Phase Is Related to Relative, Not Absolute, Saccade Direction.
Comparison of the EMREOs in subjects who had both ears
recorded confirmed that the direction of the eye movement
relative to the recorded ear determines the phase pattern of the
EMREO. Fig. 4 shows the results for one such subject. When the
left ear was recorded, leftward eye movements were associated
Cochlea, unfurled
Basilar membrane
Basilar membrane Outer hair cells: expansion and contraction
create vibrations of basilar membrane,
transmitted to eardrum via ossicles
Middle ear muscles modulate motion of ossicles; resulting
vibrations are transmitted back to eardrum as well as forward
to cochlea
Ear canal
Microphone &
earbud (speaker)
Fig. 1. Motile cochlear OHCs expand and contract in a way that depends both on the incoming sound and on the descending input received from the
superior olivary complex in the brain. OHC motion moves the basilar membrane, and subsequently the eardrum, via fluid/mechanical coupling of these
membranes through the ossicular chain. The MEMs also pull on the ossicles, directly moving the eardrum. These muscles are innervated by motor neurons
near the facial and trigeminal nerve nuclei, which receive input from the superior olive bilaterally. In either case, eardrum motion can be measured with a
microphone in the ear canal.
| Gruters et al.
Downloaded by guest on October 26, 2021
with a positive voltage peak at saccade onset (Fig. 4A,bluetraces),
whereas when the right ear was recorded, that same pattern oc-
curred for rightward eye movements (Fig. 4B,bluetraces).
Translating this into eardrum motion, the implication is that for a
given moment during a given saccade, the eardrums bulge inward
in one ear while bulging outward in the other ear. Which ear does
what when is determined by the direction of the eye movement
relative to the recorded ear (contralaterality/ipsilaterality), not
whether it is directed to the left vs. right in space. The other
subjects tested with both ears showed similar patterns, and those
tested with one ear were also generally consistent (although there
were some individual idiosyncrasies in timing and amplitude; the
remaining individual subject data are shown in Fig. S2).
Relating the EMREO Signal to Eardrum Displacement. At the low
frequencies characteristic of EMREO oscillations (30 Hz),
mean eardrum displacement is directly proportional to the
pressure produced in the small volume enclosed between the
microphone and the eardrum (1 cc for humans). We converted
the EMREO voltage signal to pressure using the microphone
calibration curve. At frequencies below about 200 Hz, the nominal
microphone sensitivity quoted by the manufacturer does not ap-
ply, and so we used the complex frequency response measured by
Christensen et al. (29) (details are provided in Methods). We
found that the group average EMREO had peak-to-peak pressure
changes near 42 mPa for 18° contralateral saccades, as well as an
initial phase opposite to the microphone voltage (Fig. 5C,maxi-
mum excursion of dark red traces). Using ear canal dimensions
typical of the adult ear (30), this pressure, the equivalent of about
57-dB peak-equivalent sound pressure level (SPL), corresponds to
a mean peak-to-peak eardrum displacement of 4 nm. Thus, the
inferred initial deflection of the eardrum was always opposite to
the direction of the saccade: When the eyes moved left, the ear-
drums moved right, and vice versa.
Fig. 2. Experimental design and results. (A) Recordings for all subjects were made via a microphone (Mic) in the ear canal set into a custom-fit ear bud. On
each trial, a subject fixated on a central LED and then made a saccade to a target LED (24° to +24° horizontally in 6° increments and 6° above the fixation
point) without moving his/her head. The ±24° locations were included on only 4.5% of trials and were excluded from analysis (Methods); other target lo-
cations were equally likely (13% frequency). (B) Humans (black text) received randomly interleaved silent and click trials (50% each). Clicks were played via a
sound transducer coupled with the microphone at four times during these trials: during the initial fixation and saccade and at 100 ms and 200 ms after target
fixation. Monkeystrials had minor timing differences (red text), and all trials had one click at 200270 ms after target fixation (red click trace). (C) Average
eye trajectories for one human subject and session for each of the included targets are shown; colors indicate saccade target locations from ipsilateral (blue)
to contralateral (red). deg, degrees; Horiz., horizontal; Vert., vertical. Mean eye position is shown as a function of time, aligned on saccade onset (D)and
offset (E). H, horizontal; V, vertical. Mean microphone recordings of air pressure in the ear canal, aligned on saccade onset (F) and offset (G), indicate that the
eardrum oscillates in conjunction with eye movements. The phase and amplitude of the oscillation varied with saccade direction and amplitude, respectively.
The oscillations were, on average, larger when aligned on saccade onset than when aligned on saccade offset.
Gruters et al. PNAS
Published online January 23, 2018
Downloaded by guest on October 26, 2021
EMREOs in Monkeys. As noted earlier, eye movements are known
to affect neural processing in several areas of the auditory
pathway (29). Because this previous work was conducted using
nonhuman primate subjects, we sought to determine whether
EMREOs also occur in monkeys. We tested the ears of rhesus
monkeys (Macaca mulatta;n=5 ears in three monkeys) per-
forming a paradigm involving a trial type that was a hybrid of
those used for the humans. The hybrid trial type was silent until
after the eye movement, after which a single click was delivered
with a 200- to 270-ms variable delay. EMREOs were observed
with timing and waveforms roughly similar to those observed in
the human subjects, both at the population level (Fig. 6 BD)
and at the individual subject level (Fig. 6E, individual traces are
shown in Fig. S3). The time-wise regression analysis suggests that
the monkey EMREO begins at about the time of the saccade and
reaches a robust level about 8 ms later (Fig. 4 BE).
Controls for Electrical Artifact. Control experiments ruled out
electrical artifacts as possible sources of these observations. In
particular, we were concerned that the microphones circuitry
could have acted as an antenna and could have been influenced
by eye movement-related electrical signals in either the eye
movement measurement system in monkeys (scleral eye coils)
and/or some other component of the experimental environment:
electrooculographic signals resulting from the electrical dipole of
the eye or myogenic artifacts, such as electromyographic signals,
originating from the extraocular, facial, or auricular muscles. If
such artifacts contributed to our measurements, they should have
continued to do so when the microphone was acoustically plugged
Fig. 3. Regression results for data aligned to saccade onset (A,C, and E) and offset (B,D,andF). (Aand B) Mean ±SEM slope of regression of microphone
voltage vs. saccade target location (conducted separately for each subject and then averaged across the group) at each time point for real (red) vs. scrambled
(gray) data. In the scrambled Monte Carlo analysis, the true saccade target locations were shuffled and arbitrarily assigned to the microphone tracesfor
individual trials. (Cand D) Proportion of variance (Var.) accounted for by regression fit (R
). (Eand F) Percentage of subject ears showing P<0.05 for the
corresponding time point. Additional details are provided in Methods.
Fig. 4. Recordings in left (A) and right (B) ears in an individual human subject. The eye movement-related signals were similar in the two ears when saccade
direction is defined with respect to the recorded ear. The remaining individual subjectsdata can be found in Fig. S2. Contra, contralateral; Ipsi, ipsilateral;
Mic, microphone.
| Gruters et al.
Downloaded by guest on October 26, 2021
without additional electrical shielding. Accordingly, we selected
four subjects with prominent effects of eye movements on the
microphone signal (Fig. 7A) and repeated the test while the
acoustic port of the microphone was plugged (Fig. 7B). The eye
movement-related effects were no longer evident when the mi-
crophone was acoustically occluded (Fig. 7B). This shows that eye
movement-related changes in the microphone signal stem from its
capacity to measure acoustic signals rather than electrical artifacts.
Additionally, EMREOs were not observed when the microphone
was placed in a 1-mL syringe (the approximate volume of the ear
Fig. 6. Eye position (A), microphone signal of ear canal pressure (B), and results of point-by-point regression (CE) for monkey subjects (n=5 ears). All
analyses were calculated in the same way as for the human data (Figs. 2 and 3). Contra, contralateral; deg, degrees; Ipsi, ipsilateral.
Fig. 5. Estimated EMREO pressure for human subjects at saccade onset (Aand C) and saccade offset (Band D). Pressures were obtained from the measured
microphone voltage using the microphones complex frequency response measured at low frequencies as described by Christensen et al. (29). Contra, con-
tralateral; deg, degrees; Horiz., horizontal; Ipsi, ipsilateral.
Gruters et al. PNAS
Published online January 23, 2018
Downloaded by guest on October 26, 2021
canal) and positioned directly behind the pinna of one human
subject (Fig. 7C). In this configuration, any electrical contamina-
tion should be similar to that in the regular experiment. We saw
none, supporting the interpretation that the regular ear canal
microphone measurements are detecting eardrum motion and not
electrical artifact.
EMREOs and Click Stimuli. EMREOs occurred not only in silence
but also when sounds were presented. During half of our trials
for human subjects, acoustic clicks (65-dB peak-equivalent SPL)
were delivered during the initial fixation, during the saccade to
the target, and at both 100 ms and 200 ms after obtaining target
fixation. Clicks presented during the saccade superimposed on
the ongoing EMREO (Fig. 8B1). Subtraction of the clicks
measured during the fixation period (Fig. 8A1) indicated that the
EMREO was not appreciably changed by the presence of the
click (Fig. 8B2). Unlike saccade onset or offset, the click did not
appear to reset the phase of the EMREO.
Clicks delivered after the eye movement was complete (Fig. 8
C1,C2,D1, and D2) revealed no new sound-triggered effects
attributable to the different static eye fixation positions achieved
by the subjects by that point of the trial. Results in monkeys, for
which only one postsaccadic click was delivered (at 75-dB peak-
equivalent SPL), were similar (Fig. 8E).
Finally, we tested the peak-to-peak amplitude of the micro-
phone signal corresponding to the click itself. Any changes in the
recorded amplitude of a click despite the same voltage having
been applied to the earbud speaker would indicate changes to
Fig. 7. EMREOs recorded normally (A) are not observed when the microphone input port is plugged to eliminate acoustic but not electrical (B) contributions
to the microphone signal. The plugged microphone sessions were run as normal sessions except that after calibration, the microphone was placed in a closed
earbud before continuing with the session (n=4 subjects). (C) Similarly, EMREOs were not evident when the microphone was placed in a test tube behind a
human subjects pinna while he/she performed the experiment as normal. Contra, contralateral; Ipsi, ipsilateral.
B1 C1 D1 E1
B3 C3 D3 E3
Fig. 8. Clicks do not appear to alter EMREOs. (Upper Row) Mean microphone signals for human subjects (AD) and monkeys (E) for click presentations at
various time points are shown: during the initial fixation (A), during the saccade (B,20 ms after saccade onset), and 100 ms (C) and 200 ms (D) after target
fixation was obtained in humans, and 200270 ms after target fixation in monkeys (E). (Insets) Zoomed-in views of the periclick timing for a more detailed
view (gray backgrounds). (Middle,BD) Residual microphone signal after the first click in each trial was subtracted (human subjects) is shown. There were no
obvious distortions in the EMREOs at the time of the (removed) click, suggesting that the effects of EMREOs interact linearly with incoming sounds. (Lower,A
E) Mean peak-to-peak amplitude of the clicks (mean ±SE) is shown. There were no clear differences in the peak click amplitudes for any epochs, indicating
that the acoustic impedance did not change as a function of saccade target location. Contra, contralateral; Ipsi, ipsilateral.
| Gruters et al.
Downloaded by guest on October 26, 2021
the acoustic impedance of the ear as a function of saccade di-
rection. We calculated the peak-to-peak amplitude of the click as
a function of saccade target location, but found no clear evidence
of any saccade target location-related differences during any of
the temporal epochs (Fig. 8 A3E3). If the mechanisms causing
the eardrum to move produce concomitant changes in the dy-
namic stiffness of the eardrum, they are too small to be detected
with this technique.
Here, we demonstrated a regular and predictable pattern of
oscillatory movements of the eardrum associated with move-
ments of the eyes. When the eyes move left, both eardrums
initially move right and then oscillate for three to four cycles,
with another one to two cycles occurring after the eye movement
is complete. Eye movements in the opposite direction produce
an oscillatory pattern in the opposite direction. This EMREO is
present in both human and nonhuman primates and appears
similar across both ears within individual subjects.
The impact of the eye movement-related process revealed
here on subsequent auditory processing remains to be determined.
In the case of OAEs, the emissions themselves are easily over-
looked, near-threshold sounds in the ear canal. However, the
cochlear mechanisms that give rise to them have a profound im-
pact on the sensitivity and dynamic range of hearing (31, 32).
OAEs thus serve as a biomarker for the health of this important
mechanism. EMREOs may constitute a similar biomarker in-
dicating that information about the direction, amplitude, and time
of occurrence of saccadic eye movements has reached the pe-
riphery and that such information is affecting underlying me-
chanical processes that may not themselves be readily observed.
EMREO assessment may therefore have value in understanding
auditory and visual-auditory deficits that contribute to disorders
ranging from language impairments to autism and schizophrenia,
as well as multisensory deficits associated with traumatic
brain injuries.
One important unresolved question is whether the oscillatory
nature of the effect observed here reflects the most important as-
pects of the phenomenon. It may be that the oscillations reflect
changes to internal structures within the ear that persist statically
until the eye moves again. If so, then the impact on auditory pro-
cessing would not be limited to the period of time when the ear-
drum is actually oscillating. Instead, processing could be affected at
all times, updating to a new state with each eye movement.
If the oscillations themselves are the most critical element of
the phenomenon, and auditory processing is only affected during
these oscillations, then it becomes important to determine for
what proportion of the time they occur. In our paradigm, we
were able to demonstrate oscillations occurring over a total pe-
riod of 110 ms, based on the aggregate results from aligning on
saccade onset and offset. They may continue longer than that,
but with a phase that changes in relation to some unmeasured or
uncontrolled variable. As time passes after the eye movement,
the phase on individual trials might drift such that the oscillation
disappears from the across-trial average. However, with the eyes
moving about three times per second under normal conditions,
an EMREO duration of about 110 ms for each saccade corre-
sponds to about one-third of the time in total.
Regardless, evidence arguing for a possible influence on audi-
tory processing can be seen in the size of EMREOs. A head-to-
head comparison of EMREOs with OAEs is difficult since auditory
detection thresholds are higher in the frequency range of
EMREOs than in the range of OAEs, but EMREOs appear to be
at least comparable and possibly larger than OAEs. For 18° hor-
izontal saccades, EMREOs produce a maximum peak-equivalent
pressure of about 57-dB SPL, whereas click-evoked OAEs range
from 0- to 20-dB SPL in healthy ears. The similar or greater
scale of EMREOs supports the interpretation that they too likely
reflect underlying mechanisms of sufficient strength to make a
meaningful contribution to auditory processing.
In whatever fashion EMREOs, or their underlying mecha-
nism, contribute to auditory processing, we surmise that the ef-
fect concerns the localization of sounds with respect to the visual
scene or using eye movements. Determining whether a sight and
a sound arise from a common spatial position requires knowing
the relationship between the visual eye-centered and auditory
head-centered reference frames. Eye position acts as a conver-
sion factor from eye- to head-centered coordinates (1), and eye
movement-related signals have been identified in multiple
auditory-responsive brain regions (217). The direction/phase
and amplitude of EMREOs contain information about the di-
rection and amplitude of the accompanying eye movements,
which situates them well for playing a causal role in this process.
Note that we view the EMREO as most likely contributing to
normal and accurate sound localization behavior. Sounds can be
accurately localized using eye movements regardless of initial eye
position (33), and this is true even for brief sounds presented
when the eyes are in flight (34). Although some studies have
found subtle influences of eye position on sound perception tasks
(3542), such effects are only consistent when prolonged fixation
is involved (4345). Holding the eyes steady for a seconds to
minutes represents a substantial departure from the time frame
in which eye movements normally occur. It is possible that de-
ficiencies in the EMREO system under such circumstances
contribute to these inaccuracies.
The source of the signals that cause EMREOs is not presently
known. Because the EMREOs precede, or occur simultaneously,
with actual movement onset, it appears likely that they derive
from a copy of the motor command to generate the eye move-
ment rather than a proprioceptive signal from the orbits, which
would necessarily lag the actual eye movement. This centrally
generated signal must then affect the activity of either MEMs or
OHCs, or a combination of both.
Based on similarities between our recordings and known
physiological activity of the MEMs, we presume that the mech-
anism behind this phenomenon is most likely the MEMs. Spe-
cifically, the frequency (around 2040 Hz) is similar to
oscillations observed in previous recordings of both the tensor
tympani and stapedius muscles (4651). Furthermore, it seems
unlikely, given the measured fall-off in reverse middle ear
transmission at low frequencies (52), that OHC activity could
produce ear canal sounds of the magnitude observed (42-mPa
peak-to-peak or 57-dB peak-equivalent SPL). Although MEMs
are often associated with bilaterally attenuating loud environ-
mental and self-generated sounds (which their activity may
precede), they are also known to be active in the absence of
explicit auditory stimuli, particularly during rapid eye movement
sleep (5356) and nonacoustic startle reflexes (51, 5761), and
have been found to exhibit activity associated with movements of
the head and neck in awake cats (57, 62). (This latter observation
is worthy of further investigation; perhaps there is also a head
movement-related eardrum oscillation to facilitate conversion of
auditory information into a body-centered reference frame).
More generally, we have demonstrated that multisensory in-
teractions occur at the most peripheral possible point in the
auditory system and that this interaction is both systematic and
substantial. This observation builds on studies showing that at-
tention, either auditory-guided or visually guided, can also
modulate the auditory periphery (2127). Our findings also raise
the intriguing possibility that efferent pathways in other sensory
systems [e.g., those leading to the retina (6373)] also carry
multisensory information to help refine peripheral processing.
This study contributes to an emerging body of evidence sug-
gesting that the brain is best viewed as a dynamic system in which
top-down signals modulate feed-forward signals; that is, the brain
integrates top down information early in sensory processing to
Gruters et al. PNAS
Published online January 23, 2018
Downloaded by guest on October 26, 2021
make the best-informed decision about the world with which it has
to interact.
The data in this study are available at
Human Subjects and Experimental Paradigm.
Human dataset I. Human subjects (n=16, eight females, aged 1845 y; par-
ticipants included university students as well as young adults from the local
community) were involved in this study. All procedures involving human
subjects were approved by the Duke University Institutional Review Board.
Subjects had apparently normal hearing and normal or corrected vision.
Informed consent was obtained from all participants before testing, and all
subjects received monetary compensation for participation. Stimulus (visual
and auditory) presentation, data collection, and offline analysis were run on
custom software utilizing multiple interfaces [behavioral interface and visual
stimulus presentation: Beethoven software (eye position sampling rate: 500 Hz),
Ryklin, Inc.; auditory stimulus presentation and data acquisition: Tucker Davis
Technologies (microphone sampling rate: 24.441 kHz); and data storage and
analysis: MATLAB; MathWorks].
Subjects were seated in a dark, sound-attenuating room. Head movements
were minimized using a chin rest, and eye movements were tracked with an
infrared camera (EyeLink 1000 Plus). Subjects performed a simple saccade task
(Fig. 2 Aand B). The subject initiated each trial by obtaining fixation on an
LED located at 0° in azimuth and elevation and about 2 m away. After 200 ms
of fixation, the central LED was extinguished and a target LED located 6°
above the horizontal meridian and ranging from 24° to 24°, in intervals of
6°, was illuminated. The most eccentric targets (±24°) were included despite
being slightly beyond the range at which head movements normally accom-
pany eye movements, which is typically 20° (74, 75), because we found in
preliminary testing that including these locations improved performance for
the next most eccentric locations (i.e., the ±18° targets). However, because
these (±24°) locations were difficult for subjects to perform well, we pre-
sented them on only 4.5% of the trials (in comparison to 13% for the other
locations) and excluded them from analysis. After subjects made saccades to
the target LED, they maintained fixation on it (9° window diameter) for
250 ms until the end of the trial. If fixation was dropped (i.e., if the eyes
traveled outside of the 9° window at any point throughout the initial or
target fixation period), the trial was terminated and the next trial began.
On half of the trials, task-irrelevant sounds were presented via the ear-
phones of an earphone/microphone assembly (Etymotic 10B+microphone
with ER 1 headphone driver) placed in the ear canal and held in position
through a custom-molded silicone earplug (Radians, Inc.). These sounds were
acoustic clicks at 65-dB peak-equivalent SPL, produced by brief electrical
pulses (40-μs positive monophasic pulse) and were presented at four time
points within each trial: during the initial fixation period (100 ms after
obtaining fixation), during the saccade (20 ms after initiating an eye
movement), 100 ms after obtaining fixation on the target, and 200 ms after
obtaining fixation on the target.
Acoustic signals from the ear canal were recorded via the in-ear micro-
phone throughout all trials, and were recorded from one ear in 13 subjects
(left/right counterbalanced) and from both ears in separate sessions in the
other three subjects, for a total of n=19 ears tested. Testing for each subject
ear was conducted in two sessions over two consecutive days or within the
same day but separated by a break of at least 1 h between sessions. Each
session involved about 600 trials and lasted a total of about 30 min. The
sound delivery and microphone system was calibrated at the beginning of
every session using a custom script (MATLAB) and again after every block of
200 trials. The calibration routine played a nominal 80-dB SPL sound (a click,
a broadband burst, and a series of tones at frequencies ranging from 112 kHz
in 22 steps) in the ear canal and recorded the resultant sound pressure. It then
calculated the difference between the requested and measured sound pres-
sures and calculated a gain adjustment profile for all sounds tested. Auditory
recording levels were set with a custom software calibration routine (MATLAB)
at the beginning of each data collection block (200 trials). All conditions were
randomly interleaved.
Human dataset II. Additional data were collected from eight ears of four new
subjects (aged 2027 y, three females and one male, one session per subject)
to verify that the relationship between microphone voltage and saccade
amplitude/target location held up when sampled more finely within a
hemifield. All procedures were the same as in the first dataset with three
exceptions: (i) The horizontal target positions ranged from 20° to +20° in
4° increments (the vertical component was still 6°), (ii ) the stimuli were
presented from a color LCD monitor (70 cm ×49 cm) at a distance of 85 cm,
and (iii) no clicks were played during trials. The microphone signal was also
calibrated differently: Clicks played between trials revealed the stability of
its impulse response and overall gain during and across experimental ses-
sions. This helped identify decreasing battery power to the amplifiers, oc-
clusion of the microphone barrel, or a change in microphone position. A
Focusrite Scarlett 2i2 audio interface was used for auditory stimulus pre-
sentation and data acquisition (microphone sampling rate of 48 kHz).
Stimulus presentation and data acquisition were controlled through custom
MATLAB scripts using The Psychophysics Toolbox extension (7678) and the
Eyelink MATLAB toolbox (79).
These data are presented in Fig. S1, and the remaining human figures are
derived from dataset I.
Monkey Subjects and Experimental Paradigm. All procedures conformed to the
guidelines of the National Institutes of Health (NIH publication no. 86-23,
revised 1985) and were approved by the Institutional Animal Care and Use
Committee of Duke University. Monkey subjects (n=3, all female) un-
derwent aseptic surgical procedures under general anesthesia to implant a
head post holder to restrain the head and a scleral search coil (Riverbend Eye
Tracking System) to track eye movements (80, 81). After recovery with
suitable analgesics and veterinary care, monkeys were trained in the saccade
task described above for the human subjects. Two monkeys were tested with
both ears in separate sessions, whereas the third monkey was tested with
one ear, for a total of n=5 ears tested.
The trial structure was similar to that used in human dataset I but with the
following differences (Fig. 2B, red traces): (i) Eye tracking was done with a
scleral eye coil; (ii) task-irrelevant sounds were presented on all trials, but
only one click was presented at 200270 ms after the saccade to the visual
target (the exact time jittered in this range, and was therefore at the same
time or slightly later than the timing of the fourth click for human subjects);
(iii)the±24° targets were presented in an equal proportion to the other
target locations but were similarly excluded from analysis as above, and
there were additional targets at ±9° that were also discarded as they were
not used with the human subjects; (iv) initial fixation was 110160 ms
(jittered time range), while target fixation duration was 310430 ms;
(v) monkeys received a fluid reward for correct trial performance; (vi ) dis-
posable plastic ear buds containing the earphone/microphone assembly as
above were placed in the ear canal for each session; (vii ) auditory recording
levels were set at the beginning of each data collection session (the ear bud
was not removed during a session, and therefore no recalibration occurred
within a session); (viii) sessions were not divided into blocks (the monkeys
typically performed consistently throughout an entire session and dividing it
into blocks was unnecessary); and (ix) the number of trials per session was
different for monkeys vs. humans (which were always presented exactly
1,200 trials over the course of the entire study), and the actual number of
trials performed varied based on monkeys performance tolerance and ca-
pabilities for the day. Monkeys MNN012 and MYY002 were both run for
four sessions per ear (no more than one session per day) over the course of
2 wk; MNN012 correctly performed an average of 889 of 957 trials at in-
cluded target locations per session for both ears, while MYY002 correctly
performed 582 of 918 trials per session on average. Monkey MHH003 was
only recorded for 1 d and yielded 132 correct out of 463 trials. Despite this
much lower performance, visual inspection of the data suggested they were
consistent with other subject data and were therefore included for analysis.
It is worth highlighting that the effect reported in this study can be seen, in
this case, with comparatively few trials.
Control Sessions. To verify that the apparent effects of eye movements on ear-
generated sounds were genuinely acoustic in nature and did not reflect
electrical contamination from sources such as myogenic potentials of the
extraocular muscles or the changing orientation of the electrical dipole of the
eyeball, we ran a series of additional control studies. Some subjects (n=4for
plugged microphone control, n=1 for syringe control) were invited back as
participants in these control studies.
In the first control experiment, the microphone was placed in the ear canal
and subjects performed the task but the microphones input port was
physically plugged, preventing it from detecting acoustic signals (Fig. 7). This
was accomplished by placing the microphone in a custom ear mold in which
the canal-side opening for the microphone was blocked. Thus, the micro-
phone was in the same physical position during these sessions, and should
therefore have continued to be affected by any electrical artifacts that
might be present, but its acoustic input was greatly attenuated by the plug.
Four subject ears were retested in this paradigm in a separate pair of data
collection sessions from their initial normalsessions. The control sessions
were handled exactly as the normal sessions except that the plugged ear
mold was used to replace the regular open ear mold after the microphone
| Gruters et al.
Downloaded by guest on October 26, 2021
was calibrated in the open ear. Calibration was executed exactly as in the
normal sessions before plugging the ear mold.
In the second control, trials were run exactly as a normal session except
that the microphone was set into a 1-mL syringe [approximately the average
volume of the human ear canal (82)] using a plastic ear bud and the syringe
was placed on top of the subjects ear behind the pinna. Acoustic recordings
were taken from within the syringe while a human subject executed the
behavioral paradigm exactly as normal. Sound levels were calibrated to the
syringe at the start of each block.
Data Analysis.
Initial processing. Unless specifically stated otherwise, all data are reported
using the raw voltage recorded from the Etymotic microphone system. Re-
sults for individual human and monkey subjects were based on all of a
subjects correct and included trials (with ±24° and 9° target locations ex-
cluded, the average number of correct trials per human subject ear for each
session was 150, for a total of 900 across the six sessions).
Trial exclusion criteria. Trial exclusion criteria were based on saccade perfor-
mance and microphone recording quality. For saccade performance, trials
were excluded if: (i) the reaction time to initiate the saccade was >250 ms or
(ii) the eyes deviated from the path between the fixation point and the
target by more than 9°. These criteria resulted in the exclusion of 18.5 ±
11.1% per ear recorded.
Microphone recording quality was examined for click and no-click trials
separately, but followed the same guidelines. The mean and SD of micro-
phone recordings over a whole block were calculated from all successful trials
(after exclusions based on saccade performance). If the SD of the voltage
values of a given trial was more than threefold the SD of the voltage values
across the whole block, it was excluded. Sample-wise z-scores were calculated
relative to the mean and SD of successful trials within a block. Trials with
50 samples or more with z-scores in excess of 10 were also excluded. These
requirements removed trials containing noise contamination from bodily
movements (e.g., swallowing, heavy breaths) or from external noise sources
(acoustic and electric). These two criteria excluded an average of 1.7 ±1.5%
of trials that had passed the saccade exclusion criteria. Overall, 20% of trials
were excluded because of saccade performance or recording quality.
Saccade-microphone synchronization. Eye position data were resampled to the
microphone sampling rate (from 500 Hz to 24.5 kHz) and smoothed to
minimize the compounding error of nonmonotonic recording artifacts after
each successive differentiation of eye position to calculate eye velocity, ac-
celeration, and jerk (the time derivative of acceleration).
Microphone data were synchronized to two time points in the saccades:
initiation and termination. Saccade onsets were determined by locating the
time of the first peak in the jerk of the saccades. These onset times for each
trial were then used to synchronize the microphone recordings at the be-
ginning of saccades. Because saccades of different lengths took different
amounts of time to complete and because performance of saccades to
identical targets varied between trials, saccade completions occurred over a
range of tens of milliseconds. As a result, recordings of eardrum behavior
related to the later stages of saccades would be poorly aligned if synchro-
nized by saccade onset alone. Therefore, saccade termination, or offset, was
determined by locating the second peak in the saccade jerk and used to
synchronize microphone recordings with saccade completion.
Statistical analyses. We evaluated the statistical significance of the effects of
eye movements on signals recorded with the ear canal microphone with a
regression analysis at each time point (microphone signal vs. saccade target
location) for each subject. This produced a time series of statistical results
(slope, R
, and Pvalue; Fig. 3 AF). This analysis was performed twice: first,
with all trials synchronized to saccade onset and, second, with all trials
synchronized to saccade offset.
Given that the dependence of one sample relative to the next sample was
unknown, a post hoc correction (e.g., Bonferroni correction) was not prac-
tical. Instead, we used a Monte Carlo technique to estimate the chance-
related effect size and false-positive rates of this test. We first scrambled
the relationship between each trial and its saccade target location assign-
ment and then ran the same analysis as before. This provided an estimate of
how our results should look if there was no relationship between eye
movements and the acoustic recordings.
Peak click amplitudes were calculated for each trial involving sounds by
isolating the maximum and minimum peaks during the click stimuli. This
allowed us to look for possible changes in acoustic impedance in the
ear canal.
Estimation of eardrum motion. The sensitivity of the microphone used in the
experiments (Etymotic Research ER10B+) rolls off at frequencies below about
200 Hz, where it also introduces a frequency-dependent phase shift (29).
Consequently, the microphone voltage waveform is not simply proportional
to the ear canal pressure (or eardrum displacement) at frequencies charac-
teristic of the EMREO waveform. To obtain the ear canal pressure, and es-
timate the eardrum motion that produced it, we converted the microphone
voltage to pressure using published measurements of the microphones
complex-valued frequency response (29).
Because the measured frequency response (H
, with units of volts per
pascal) was sampled at 48 kHz, we first transformed it into the time-domain
representation of the microphones impulse response, resampled the result
at our sampling rate of 24.4 kHz, and then retransformed it back into the
frequency domain as H
. For each trial, the fast Fourier transform of mi-
crophone voltage (V
) was divided by H
and the inverse fast Fourier
transform was then calculated to produce the estimate of ear-canal, P
Pec =F1FðVmicÞ
Eardrum displacement, x
, was computed from the measured pressure us-
ing the equation x
, where Vis the volume of the residual
ear canal space (2cc),Ais the cross-sectional area of the eardrum (60 mm
is the density of air, and cis the speed of sound.
ACKNOWLEDGMENTS. We thank Tom Heil, Jessi Cruger, Karen Waterstradt,
Christie Holmes, and Stephanie Schlebusch for technical assistance. We thank
Marty Woldorff, Jeff Beck, Tobias Overath, Barbara Shinn-Cunningham,
Valeria Caruso, Daniel Pages, Shawn Willett, and Jeff Mohl for numerous
helpful discussions and comments during the course of this project.
1. Groh JM, Sparks DL (1992) Two models for transforming auditory signals from head-
centered to eye-centered coordinates. Biol Cybern 67:291302.
2. Groh JM, Trause AS, Underhill AM, Clark KR, Inati S (2001) Eye position influences
auditory responses in primate inferior colliculus. Neuron 29:509518.
3. Zwiers MP, Versnel H, Van Opstal AJ (2004) Involvement of monkey inferior colliculus
in spatial hearing. J Neurosci 24:41454156.
4. Porter KK, Metzger RR, Groh JM (2006) Representation of eye position in primate
inferior colliculus. J Neurophysiol 95:18261842.
5. Bulkin DA, Groh JM (2012) Distribution of eye position information in the monkey
inferior colliculus. J Neurophysiol 107:785795.
6. Bulkin DA, Groh JM (2012) Distribution of visual and saccade related information in
the monkey inferior colliculus. Front Neural Circuits 6:61.
7. Werner-Reiss U, Kelly KA, Trause AS, Underhill AM, Groh JM (2003) Eye position af-
fects activity in primary auditory cortex of primates. Curr Biol 13:554562.
8. Fu KM, et al. (2004) Timing and laminar profile of eye-position effects on auditory
responses in primate auditory cortex. J Neurophysiol 92:35223531.
9. Maier JX, Groh JM (2010) Comparison of gain-like properties of eye position signals in
inferior colliculus versus auditory cortex of primates. Front Integr Neurosci 4:121132.
10. Stricanne B, Andersen RA, Mazzoni P (1996) Eye-centered, head-centered, and in-
termediate coding of remembered sound locations in area LIP. J Neurophysiol 76:
11. Mullette-Gillman OA, Cohen YE, Groh JM (2009) Motor-related signals in the intra-
parietal cortex encode locations in a hybrid, rather than eye-centered reference
frame. Cereb Cortex 19:17611775.
12. Mullette-Gillman OA, Cohen YE, Groh JM (2005) Eye-centered, head-centered, and
complex coding of visual and auditory targets in the intraparietal sul cus.
J Neurophysiol 94:23312352.
13. Jay MF, Sparks DL (1987) Sensorimotor integration in the primate superior colliculus. I.
Motor convergence. J Neurophysiol 57:2234.
14. Jay MF, Sparks DL (1987) Sensorimotor integration in the primate superior colliculus.
II. Coordinates of auditory signals. J Neurophysiol 57:3555.
15. Jay MF, Sparks DL (1984) Auditory receptive fields in primate superior colliculus shift
with changes in eye position. Nature 309:345347.
16. Populin LC, Tollin DJ, Yin TC (2004) Effect of eye position on saccades and neuronal
responses to acoustic stimuli in the superior colliculus of the behaving cat.
J Neurophysiol 92:21512167.
17. Lee J, Groh JM (2012) Auditory signals evolve from hybrid- to eye-centered coordi-
nates in the primate superior colliculus. J Neurophysiol 108:227242.
18. Guinan JJ, Jr (2014) Cochlear mechanics, otoacoustic emissions, and medial olivoco-
chlear efferents: Twenty years of advances and controversies along with areas ripe for
new work. Perspectives on Auditory Research (Springer, New York), pp 229246.
19. Guinan JJ, Jr (2006) Olivocochlear efferents: Anatomy, physiology, function, and the
measurement of efferent effects in humans. Ear Hear 27:589607.
20. Mukerji S, Windsor AM, Lee DJ (2010) Auditory brainstem circuits that mediate the
middle ear muscle reflex. Trends Amplif 14:170191.
21. Delano PH, Elgueda D, Hamame CM, Robles L (2007) Selective attention to visual
stimuli reduces cochlear sensitivity in chinchillas. J Neurosci 27:41464153.
22. Harkrider AW, Bowers CD (2009) Evidence for a cortically mediated release from in-
hibition in the human cochlea. J Am Acad Audiol 20:208215.
Gruters et al. PNAS
Published online January 23, 2018
Downloaded by guest on October 26, 2021
23. Srinivasan S, Keil A, Stratis K, Woodruff Carr KL, Smith DW (2012) Effects of cross-
modal selective attention on the sensory periphery: Cochlear sensitivity is altered by
selective attention. Neuroscience 223:325332.
24. Srinivasan S, et al. (2014) Interaural attention modulates outer hair cell function. Eur J
Neurosci 40:37853792.
25. Walsh KP, Pasanen EG, McFadden D (2014) Selective attention reduces physiological
noise in the external ear canals of humans. I: Auditory attention. Hear Res 312:
26. Walsh KP, Pasanen EG, McFadden D (2015) Changes in otoacoustic emissions during
selective auditory and visual attention. J Acoust Soc Am 137:27372757.
27. Wittekindt A, Kaiser J, Abel C (2014) Attentional modulation of the inner ear: A
combined otoacoustic emission and EEG study. J Neurosci 34:999510002.
28. Goodman SS, Keefe DH (2006) Simultaneous measurement of noise-activated middle-
ear muscle reflexand stimulus frequencyotoacoustic emissions. J Assoc Res Otolaryngol
29. Christensen AT, Ordonez R, Hammershoi D (2015) Design of an acoustic probe to
measure otoacoustic emissions below 0.5 kHz. Proceedings of the Fifty-Eighth Audio
Engineering Society International Conference: Music Induced Hearing Disorders
(Audio Engineering Society, New York), paper no. 2-4.
30. Abdala C, Keefe DH (2012) Morphological and functional ear development. Human
Auditory Development, eds Werner L, Fay RR, Popper AN (Springer, New York), pp
31. Robinette M, Glattke T, eds (2007) Otoacoustic EmissionsClinical Applications
(Thieme Medical Publishers, New York).
32. Ramos JA, Kristensen SG, Beck DL (2013) An overview of OAEs and normative data for
DPOAEs. Hear Rev 20:3033.
33. Metzger RR, Mullette-Gillman OA, Underhill AM, Cohen YE, Groh JM (2004) Auditory
saccades from different eye positions in the monkey: Implications for coordinate
transformations. J Neurophysiol 92:26222627.
34. Boucher L, Groh JM, Hughes HC (2001) Afferent delays and the mislocalization of
perisaccadic stimuli. Vision Res 41:26312644.
35. Getzmann S (2002) The effect of eye position and background noise on vertical sound
localization. Hear Res 169:130139.
36. Bohlander RW (1984) Eye position and visual attention influence perceived auditory
direction. Percept Mot Skills 59:483510.
37. Weerts TC, Thurlow WR (1971) The effects of eye position and expectation on sound
localization. Percept Psychophys 9:3539.
38. Lewald J, Ehrenstein WH (2001) Effect of gaze direction on sound localization in rear
space. Neurosci Res 39:253257.
39. Lewald J, Getzmann S (2006) Horizontal and vertical effects of eye-position on sound
localization. Hear Res 213:99106.
40. Lewald J (1998) The effect of gaze eccentricity on perceived sound direction and its
relation to visual localization. Hear Res 115:206216.
41. Lewald J (1997) Eye-position effects in directional hearing. Behav Brain Res 87:3548.
42. Lewald J, Ehrenstein WH (1996) The effect of eye position on auditory lateralization.
Exp Brain Res 108:473485.
43. Cui QN, ONeill WE, Paige GD (2010) Advancing age alters the influence of eye po-
sition on sound localization. Exp Brain Res 206:371379.
44. Cui QN, Razavi B, ONeill WE, Paige GD (2010) Perception of auditory, visual, and
egocentric spatial alignment adapts differently to changes in eye position.
J Neurophysiol 103:10201035.
45. Razavi B, ONeill WE, Paige GD (2007) Auditory spatial perception dynamically re-
aligns with changing eye position. J Neurosci 27:1024910258.
46. Eliasson S, Gisselsson L (1955) Electromyographic studies of the middle ear muscles of
the cat. Electroencephalogr Clin Neurophysiol 7:399406.
47. Borg E, Moller AR (1968) The acoustic middle ear reflex in unanesthetized rabbits.
Acta Otolaryngol 65:575585.
48. Borg E (1972) Excitability of the acoustic m. stapedius and m. tensor tympani reflexes
in the nonanesthetized rabbit. Acta Physiol Scand 85:374389.
49. Borg E (1972) Regulation of middle ear sound transmission in the nonanesthetized
rabbit. Acta Physiol Scand 86:175190.
50. Teig E (1972) Tension and contraction time of motor units of the middle ear muscles
in the cat. Acta Physiol Scand 84:1121.
51. Greisen O, Neergaard EB (1975) Middle ear reflex activity in the startle reaction. Arch
Otolaryngol 101:348353.
52. Puria S (2003) Measurements of human middle ear forward and reverse acoustics:
Implications for otoacoustic emissions. J Acoust Soc Am 113:27732789.
53. Dewson JH, 3rd, Dement WC, Simmons FB (1965) Middle ear muscle activity in cats
during sleep. Exp Neurol 12:18.
54. Pessah MA, Roffwarg HP (1972) Spontaneous middle ear muscle activity in man: A
rapid eye movement sleep phenomenon. Science 178:773776.
55. De Gennaro L, Ferrara M (2000) Sleep deprivation and phasic activity of REM sleep:
Independence of middle-ear muscle activity from rapid eye movements. Sleep 23:
56. De Gennaro L, Ferrara M, Urbani L, Bertini M (2000) A complementary relationship
between wake and REM sleep in the auditory system: A pre-sleep increase of middle-
ear muscle activity (MEMA) causes a decrease of MEMA during sleep. Exp Brain Res
57. Carmel PW, Starr A (1963) Acoustic and nonacoustic factors modifying middle-ear
muscle activity in waking cats. J Neurophysiol 26:598616.
58. Holst HE, Ingelstedt S, Ortegren U (1963) Ear drum movements following stimulation
of the middle ear muscles. Acta Otolaryngol Suppl 182:7389.
59. Salomon G, Starr A (1963) Electromyography of middle ear muscles in man during
motor activities. Acta Neurol Scand 39:161168.
60. Yonovitz A, Harris JD (1976) Eardrum displacement following stapedius muscle con-
traction. Acta Otolaryngol 81:115.
61. Avan P, Loth D, Menguy C, Teyssou M (1992) Hypothetical roles of middle ear muscles
in the guinea-pig. Hear Res 59:5969.
62. Carmel PW, Starr A (1964) Non-acoustic factors influencing activity of middle ear
muscles in waking cats. Nature 202:195196.
63. Gastinger MJ, Tian N, Horvath T, Marshak DW (2006) Retinopetal axons in mammals:
Emphasis on histamine and serotonin. Curr Eye Res 31:655667.
64. Gastinger MJ, Yusupov RG, Glickman RD, Marshak DW (2004) The effects of histamine
on rat and monkey retinal ganglion cells. Vis Neurosci 21:935943.
65. Gastinger MJ (2004) Function of histaminergic retinopetal axons in rat and primate
retinas. PhD dissertation (Texas Medical Center, Houston, TX).
66. Repérant J, et al. (2007) The evolution of the centrifugal visual system of vertebrates.
A cladistic analysis and new hypotheses. Brain Res Brain Res Rev 53:161197.
67. Lörincz ML, Oláh M, Juhász G (2008) Functional consequences of retinopetal fibers
originating in the dorsal raphe nucleus. Int J Neurosci 118:13741383.
68. Gastinger MJ, Bordt AS, Bernal MP, Marshak DW (2005) Serotonergic retinopetal
axons in the monkey retina. Curr Eye Res 30:10891095.
69. Gastinger MJ, OBrien JJ, Larsen NB, Marshak DW (1999) Histamine immunoreactive
axons in the macaque retina. Invest Ophthalmol Vis Sci 40:487495.
70. Abudureheman A, Nakagawa S (2010) Retinopetal neurons located in the di-
encephalon of the Japanese monkey (Macaca fuscata). Okajimas Folia Anat Jpn 87:
71. Honrubia FM, Elliott JH (1968) Efferent innervation of the retina. I. Morphologic study
of the human retina. Arch Ophthalmol 80:98103.
72. Honrubia FM, Elliott JH (1970) Efferent innervation of the retina. II. Morphologic
study of the monkey retina. Invest Ophthalmol 9:971976.
73. Itaya SK, Itaya PW (1985) Centrifugal fibers to the rat retina from the medial pretectal
area and the periaqueductal grey matter. Brain Res 326:362365.
74. Freedman EG, Stanford TR, Sparks DL (1996) Combined eye-head gaze shifts produced
by electrical stimulation of the superior colliculus in rhesus monkeys. J Neurophysiol 76:
75. Stahl JS (2001) Eye-head coordination and the variation of eye-movement accuracy
with orbital eccentricity. Exp Brain Res 136:200210.
76. Kleiner M, et al. (2007) Whats new in psychtoolbox-3? Perception 36:116.
77. Pelli DG (1997) The VideoToolbox software for visual psychophysics: Transforming
numbers into movies. Spat Vis 10:437442.
78. Brainard DH (1997) The Psychophysics Toolbox. Spat Vis 10:433436.
79. Cornelissen FW, Peters EM, Palmer J (2002) The Eyelink Toolbox: Eye tracking with
MATLAB and the Psychophysics Toolbox. Behavior Research Methods, Instruments, &
Computers 34:613.
80. Robinson DA (1963) A method of measuring eye movement using a scleral search coil
in a magnetic field. IEEE Trans Biomed Eng 10:137145.
81. Judge SJ, Richmond BJ, Chu FC (1980) Implantation of magnetic search coils for
measurement of eye position: An improved method. Vision Res 20:535538.
82. Gerhardt KJ, Rodriguez GP, Hepler EL, Moul ML (1987) Ear canal volume and vari-
ability in the patterns of temporary threshold shifts. Ear Hear 8:316321.
| Gruters et al.
Downloaded by guest on October 26, 2021
... Groh and Sparks,30 1992; Boucher et al., 2001;Metzger et al., 2004). Most previous work about how eye 31 movement information is incorporated into auditory processing has focused on cortical 32 and subcortical brain structures Sparks, 1984, 1987a, b;Russo and Bruce, 33 1994; Hartline et al., 1995;Stricanne et al., 1996;Cohen and Andersen, 2000;Groh et 34 al., 2001;Zella et al., 2001;Werner-Reiss et al., 2003;Fu et al., 2004;Populin et al., 35 3 2012; Caruso et al., 2021), but the recent discovery of eye-movement related eardrum 38 oscillations (EMREOs) (Gruters et al., 2018) suggests that the process might be 39 manifest much earlier in the auditory periphery. EMREOs can be thought of as a 40 biomarker of underlying efferent information impacting the internal structures of the ear 41 in association with eye movements. ...
... 404 However, it is also not clear that the underlying mechanism is, in fact, oscillatory. 1 We note that EMREOs are unlikely to be due to the actual sound of the eyes moving in the orbits. Our original study, Gruters et al (2018) showed that when microphone recordings are aligned on saccade offset (as opposed to onset, as we did here), EMREOs continue for at least several 10's of ms after the eyes have stopped moving. We also have unpublished observations in patients with various hearing abnormalities; EMREOs are altered in such patients despite normal eye movements. ...
... Boucher et al (Boucher et al., 2001) reported that 483 perisaccadic sound localization is quite accurate, which suggests that EMREOs (or their 484 underlying mechanism) do not impair perception. This is an important insight because 485 given the rate at which eye movements occur -about 3/sec -and with each associated 486 EMREO signal lasting 100 ms or longer (due to extending past the end of saccades, as 487 explored by Gruters, Murphy et al. 2018), it would be highly problematic if sounds could 488 not be accurately detected or localized when they occur in conjunction with saccades. ...
Eye movements alter the relationship between the visual and auditory spatial scenes. Signals related to eye movements affect neural pathways from the ear through auditory cortex and beyond, but how these signals contribute to computing the locations of sounds with respect to the visual scene is poorly understood. Here, we evaluated the information contained in eye movement-related eardrum oscillations (EMREOs), pressure changes recorded in the ear canal that occur in conjunction with simultaneous eye movements. We show that EMREOs contain parametric information about horizontal and vertical eye displacement as well as initial/final eye position with respect to the head. The parametric information in the horizontal and vertical directions combines linearly, allowing accurate prediction of the EMREOs associated with oblique eye movements from their respective horizontal and vertical components. Target location can also be inferred from the EMREO signals recorded during eye movements to those targets. We hypothesize that the thus-far unknown mechanism underlying EMREOs could impose a two-dimensional eye-movement related transfer function on any incoming sound, permitting subsequent processing stages to compute the positions of sounds in relation to the visual scene.
... Cohen et al. 2005), recent literature suggests that action-related sensory input mediates multisensory effects. For example, eye movements during auditory attention inform individual group differences within the dorsal attention network (Braga et al. 2016), and eye-movement-related eardrum oscillations link sounds and images in space (Gruters et al. 2018;Murphy et al. 2020). Thus, alternatively, an affirmative case for the presence of saccades in register of auditory cue location might offer some explanation. ...
... The involvement of eye-movements towards the attended loudspeaker location (Experiment 1) and cued ear (Experiment 2) provides evidence for the existence of a reciprocal relationship as the recently discovered saccades induced eardrum oscillations (Gruters et al. 2018;Murphy et al. 2020) and consistent modulation of neural excitability within auditory areas by saccadic eye movements (Leszczynski et al. 2022). That is, an auditory cue presentation at a particular location in space elicits oculomotor responses consistent with the sound origin. ...
Full-text available
Spatially selective modulation of alpha power (8–14 Hz) is a robust finding in electrophysiological studies of visual attention, and has been recently generalized to auditory spatial attention. This modulation pattern is interpreted as reflecting a top-down mechanism for suppressing distracting input from unattended directions of sound origin. The present study on auditory spatial attention extends this interpretation by demonstrating that alpha power modulation is closely linked to oculomotor action. We designed an auditory paradigm in which participants were required to attend to upcoming sounds from one of 24 loudspeakers arranged in a circular array around the head. Maintaining the location of an auditory cue was associated with a topographically modulated distribution of posterior alpha power resembling the findings known from visual attention. Multivariate analyses allowed the prediction of the sound location in the horizontal plane. Importantly, this prediction was also possible, when derived from signals capturing saccadic activity. A control experiment on auditory spatial attention confirmed that, in absence of any visual/auditory input, lateralization of alpha power is linked to the lateralized direction of gaze. Attending to an auditory target engages oculomotor and visual cortical areas in a topographic manner akin to the retinotopic organization associated with visual attention.
... fixation onset). Recently, saccade-related eardrum motion has been observed in the absence of auditory stimuli and this motion appeared to have oscillatory characteristics (Gruters et al., 2018). Additionally, saccade-related entrainment and modulation of excitability has been shown to occur in A1 of NHPs (O'Connell et al., 2020). ...
... ; /2022 that these kinds of motor signals can impact sensory processing at the very earliest stages, saccadic eye movements have been shown to lead to oscillatory-type movement of the eardrum even when no auditory stimuli are presented (Gruters et al., 2018). These motorsensory influences are not unidirectional in nature. ...
The auditory and visual sensory systems are both used by the brain to obtain and organize information from our external environment, yet there are fundamental differences between these two systems. Visual information is acquired using systematic patterns of fixations and saccades, which are controlled by internal motor commands. Sensory input occurs in volleys that are tied to the timing of saccades. In contrast, the auditory system does not use such an overt motor sampling routine so the relationship between sensory input timing and motor activity is less clear. Previous studies of primary visual cortex (V1) in nonhuman primates (NHP) have shown that there is a cyclical modulation of excitability tied to the eye movement cycle and suggests that this excitability modulation stems from the phase reset of neuronal oscillations. We hypothesized that if saccades provide a supramodal temporal context for environmental information then we should also see saccade-related modulation of oscillatory activity in primary auditory cortex (A1) as NHPs shift their gaze around their surroundings. We used linear array multielectrodes to record cortical laminar neuroelectric activity profiles while subjects sat in a dark or dimly lit and silent chamber. Analysis of oscillatory activity in A1 suggests that saccades lead to a phase reset of neuronal oscillations in A1. Saccade-related phase reset of delta oscillations were observed across all layers while theta effects occurred primarily in extragranular layers. Although less frequent, alpha oscillations also showed saccade-related phase reset within the extragranular layers. Our results confirm that saccades provide a supramodal temporal context for the influx of sensory information into A1 and highlight the importance of considering the effects of eye position on auditory processing. Significance Statement Using laminar multielectrodes, the current study examined saccade-related neuronal activity during resting state while NHPs sat in a dark or dimly lit room. Our results confirm that saccade-related modulation of delta band oscillatory activity occurs across all layers of A1. Interestingly, our data also show a saccade-related phase reset of theta and alpha bands that preferentially occurs in extragranular layers. These results confirm that saccades provide a supramodal temporal context for the influx of environmental information into A1 and emphasizes the importance of considering eye position when examining auditory processing.
... Their sixth observation was a gaze-associated tinnitus, which they attributed to a then invisible oculostapedial response. Subsequent studies using more objective methods partially reiterated the findings of Carmichael and Critchley (14,15) on the oculoauricular (16)(17)(18)(19)(20), oculomandibular (21,22), oculofrontalis movements (21), and verified eye movement-related eardrum oscillations associated with horizontal saccades (23). Among these, the most widely studied has been the oculoauricular phenomenon, presumably due to its high prevalence (up to 98%) (14, 17-19, 24, 25) and its potential application in clinical brainstem topodiagnostics (17). ...
... Although this application has not been transferred into routine clinical diagnostics, coordinated activation of eye and ear muscles is suggested today as an easily accessible means of monitoring visual and auditory attention. The results of such recordings have the potential to shed further light on deficits in visualauditory processing of orientation and directed attention, such as in schizophrenia, autism, and traumatic brain injury (23). Furthermore, the ability of the external ear muscles to be activated during a shift of visual and auditory attention might be useful to design "intelligent" neuroprosthetics (43), like attentionally directed hearing aids that particularly amplify the sound coming from the direction the listener pays attention to (62). ...
Integrated motor behaviors involving ocular motion-associated movements of the head, neck, pinna, and parts of the face are commonly seen in animals orienting to a visual target. A number of coordinated movements have also been observed in humans making rapid gaze shifts to horizontal extremes, which may be vestiges of these. Since such integrated mechanisms point to a non-pathological co-activation of several anatomically separate cranial circuits in humans, it is important to see how the different pairs of integrative motor behaviors with a common trigger (i.e., ocular motion) manifest in relation to one another. Here, we systematically examined the pattern of eye movement-induced recruitment of multiple cranial muscles in humans. Simultaneous video-oculography and bilateral surface electromyograms of transverse auricular, temporalis, frontalis, and masseter muscles were recorded in 15 healthy subjects (8 females; 29.3±5.2 years) while they made head-fixed, horizontal saccadic, pursuit and optokinetic eye movements. Potential chin laterotrusion linked to contractions of masticator muscles was captured with a yaw-fixed accelerometer. Our findings objectively show an orchestrated aural-facial-masticatory muscle response to a range of horizontal eye movements (prevalence of 21-93%). These responses were most prominent during eccentric saccades. We further reveal distinctions between the various observed activation patterns in terms of their profile (transient or sustained), laterality (with respect to direction of gaze) and timing (with respect to saccade onset). Possible underlying neural substrates, their atavistic behavioral significance, and potential clinical applications for monitoring sensory attention and designing attention-directed hearing aids in the future are discussed.
... While phase opposition between both ears could be an adaptation to (cochlear) physiological processes, phase opposition in one ear between both attention conditions allows one to draw the conclusion that attention modulates the phase of OOA. Gruters et al. (2018) demonstrated that the eardrums oscillate in relation to horizontal eye movements. These so-called eye movement-related eardrum oscillations (EMREOs) occur predominantly between 20 -40 Hz of the acoustic spectrum measured from the ear canals. ...
... We failed to show significant phase locking to eye movements in the left and right ear for both attention conditions. Thus, the ocular process that underlies the previously reported EMREOs (Gruters et al., 2018) is only a very weak candidate explanation for our attentional OOA effects. ...
Full-text available
It is widely established that sensory perception is a rhythmic process as opposed to a continuous one. In the context of auditory perception this effect is only established on a cortical and behavioral level. Yet, the unique architecture of the auditory sensory system allows its primary sensory cortex to modulate the processes of its sensory receptors at the cochlear level. Previously, we could demonstrate the existence of a genuine cochlear theta (~6 Hz) rhythm that is modulated in amplitude by intermodal selective attention. As the study's paradigm was not suited to assess attentional effects on the oscillatory phase of cochlear activity the question whether attention can also affect the temporal organization of the cochlea's ongoing activity remained open. The present study utilizes an interaural attention paradigm to investigate ongoing otoacoustic activity during a stimulus-free cue-target interval and an omission period of the auditory target. We were able to replicate the existence of the cochlear theta rhythm. Importantly, we found significant phase opposition between the two ears and attention conditions of anticipatory as well as cochlear oscillatory activity during target presentation. Yet, the amplitude was unaffected by interaural attention. These results are the first to demonstrate that intermodal and interaural attention deploy different aspects of excitation and inhibition at the first level of auditory processing. While intermodal attention modulates the level of cochlear activity, interaural attention modulates the timing.
... It has been postulated that these stimuli are processed in modality-specific unisensory systems before integration in the multi-modal process for coherent perception, known as multisensory integration (Meredith and Stein, 1983;Choi et al., 2018). However, increasing evidence has recently suggested this integration to occur even earlier for auditory, visual, and somatosensory input at various subcortical layers and the sensory periphery (Driver and Noesselt, 2008;Bizley et al., 2016;Wang et al., 2017;Gruters et al., 2018). Yet, the extent at which multisensory integration occurs in each area remains highly debated (Rohe and Noppeney, 2016). ...
Full-text available
We experience various sensory stimuli every day. How does this integration occur? What are the inherent mechanisms in this integration? The “unity assumption” proposes a perceiver’s belief of unity in individual unisensory information to modulate the degree of multisensory integration. However, this has yet to be verified or quantified in the context of semantic emotion integration. In the present study, we investigate the ability of subjects to judge the intensities and degrees of similarity in faces and voices of two emotions (angry and happy). We found more similar stimulus intensities to be associated with stronger likelihoods of the face and voice being integrated. More interestingly, multisensory integration in emotion perception was observed to follow a Gaussian distribution as a function of the emotion intensity difference between the face and voice—the optimal cut-off at about 2.50 points difference on a 7-point Likert scale. This provides a quantitative estimation of the multisensory integration function in audio-visual semantic emotion perception with regards to stimulus intensity. Moreover, to investigate the variation of multisensory integration across the population, we examined the effects of personality and autistic traits of participants. Here, we found no correlation of autistic traits with unisensory processing in a nonclinical population. Our findings shed light on the current understanding of multisensory integration mechanisms.
The making and remaking of the living can be described from a variety of perspectives. The genetic and epigenetic aspects of life dynamics are focused on the reproduction of organisms. Reproduction of life is never a repeat, but rather always an original. The anticipatory nature of life is ontological in nature. There is no life in the absence of anticipatory processes. Understanding interaction is the premise for a coherent foundation for the study of the relation between epigenetics and anticipation.KeywordsInteractionCreativityNon-determinismMulticausalityMeaning
Full-text available
Gaze behavior in dyadic conversations can indicate active listening and attention. However, gaze behavior that is different from the engagement expected during neurotypical social interaction cues may be interpreted as uninterested or inattentive, which can be problematic in both personal and professional situations. Neurodivergent individuals, such as those with autism spectrum conditions, often exhibit social communication differences broadly including via gaze behavior. This project aims to support situational social gaze practice through a virtual reality (VR) mock job interview practice using the HTC Vive Pro Eye VR headset. We show how gaze behavior varies in the mock job interview between neurodivergent and neurotypical participants. We also investigate the social modulation of gaze behavior based on conversational role (speaking and listening). Our three main contributions are: (i) a system for fully-automatic analysis of social modulation of gaze behavior using a portable VR headset with a novel realistic mock job interview, (ii) a signal processing pipeline, which employs Kalman filtering and spatialtemporal density-based clustering techniques, that can improve the accuracy of the headset's built-in eye-tracker, and (iii) being the first to investigate social modulation of gaze behavior among neurotypical/divergent individuals in the realm of immersive VR.
Middle ear muscle contractions (MEMCs) are most commonly considered a response to high-level acoustic stimuli. However, MEMCs have also been observed in the absence of sound, either as a response to somatosensory stimulation or in concert with other motor activity. The relationship between MEMCs and non-acoustic sources is unclear. This study examined associations between measures of voluntary unilateral eye closure and impedance-based measures indicative of middle ear muscle activity while controlling for demographic and clinical factors in a large group of participants (N=190) with present clinical acoustic reflexes and no evidence of auditory dysfunction. Participants were instructed to voluntarily close the eye ipsilateral to the ear canal containing a detection probe at three levels of effort. Orbicularis oculi muscle activity was measured using surface electromyography. Middle ear muscle activity was inferred from changes in total energy reflected in the ear canal using a filtered (0.2 to 8 kHz) click train. Results revealed that middle ear muscle activity was positively associated with eye muscle activity. MEMC occurrence rates for eye closure observed in this study were generally higher than previously published rates for high-level brief acoustic stimuli in the same participant pool suggesting that motor activity may be a more reliable elicitor of MEMCs than acoustic stimuli. These results suggest motor activity can serve as a confounding factor for auditory exposure studies as well as complicate the interpretation of any impulsive noise damage risk criteria that assume MEMCs serve as a consistent, uniform protective factor. The mechanism linking eye and middle ear muscle activity is not understood and is an avenue for future research.
The vestibular end organs differ in terms of anatomical and physiological characteristics. Sensory modalities’ stimuli including visual stimuli and vestibular sensation can influence these organs differently. This paper explores differences between vestibular responses to axial tilts in physical and virtual environments. Four passive whole-body movements (linear: up-down, and angular: yaw, pitch, and roll) were applied to twenty-seven healthy participants once using a hydraulic chair (physical) and once visually using a head-mounted display (virtual). Electrovestibulography (EVestG) was used as the outcome measure to investigate the magnitude of vestibular-response-change in both ears for physical and virtual stimuli. Three features including average action potential (AP) area, AP amplitude, and mean detected firing rate change were used as indices of response. The results show that for both physical and virtual stimuli (1) generally the pitch and roll tilts produce the largest EVestG changes compared to other tilts (2) roll and pitch tilt responses are not significantly different from each other and (3) right side and left side roll tilts’ responses are not significantly different. The findings indicate although visually- and physically-induced vestibular responses are different in terms of afferent activity, visual stimuli can still result in distinct responses when exposed to different axial tilts.
Full-text available
The inferior colliculus (IC) is an essential stop early in the ascending auditory pathway. Though normally thought of as a predominantly auditory structure, recent work has uncovered a variety of non-auditory influences on firing rate in the IC. Here, we map the location within the IC of neurons that respond to the onset of a fixation-guiding visual stimulus. Visual/visuomotor associated activity was found throughout the IC (overall, 84 of 199 sites tested or 42%), but with a far reduced prevalence and strength along recording penetrations passing through the tonotopically organized region of the IC, putatively the central nucleus (11 of 42 sites tested, or 26%). These results suggest that visual information has only a weak effect on early auditory processing in core regions, but more strongly targets the modulatory shell regions of the IC.
Reviewed are progress, exciting new developments, and areas that need work. Topics considered are: Cochlear amplification is from energy injected into the traveling wave by outer-hair-cell somatic motility. Calcium-activated stereocilia motility does not work at high frequencies because of the slowness of calcium binding and unbinding. Cochlear mechanics and micromechanics in the apical half of the cochlea are different from in the base. Cochlear micromechanics and the multiple fluid drives to inner hair cell (IHC) stereocilia. Interaction of the multiple IHC fluid drives can explain phase reversals in auditory nerve fiber responses without phase reversals in basilar membrane responses. The mechanisms by which medial olivocochlear (MOC) efferents change cochlear mechanics and micromechanics. The generation mechanisms for otoacoustic emissions (OAEs). Distortion product OAEs (DPOAEs) travel backward by slow traveling waves. Stimulus frequency OAEs (SFOAEs) arise mainly from near the peak of the traveling wave. Using OAEs to reveal cochlear properties. Cochlear tuning is sharper in humans than in cats, guinea pigs, and chinchillas. Measuring MOC effects using changes in OAEs and the need for high OAE signal-to-noise ratios. MOC effects in humans. The role of MOC efferents in hearing. MOC activity makes it easier to hear signals in noise. MOC activity and selective attention. MOC activity reduces acoustic trauma.
The present study examines whether the direction of gaze can influence sound lateralization. For this purpose, dichotic stimuli with variable interaural level difference (ILD) were presented under different conditions of visual fixation. In experiment 1, subjects with their head fixed directed their gaze to a given target, simultaneously adjusting the ILD of continuous pure tone or noise stimuli so that their location was perceived in the median plane of the head. The auditory adjustments were significantly correlated with gaze direction. During eccentric fixation, the psychophysical adjustments to the median plane shifted slightly toward the direction of gaze. The magnitude of the shift was about 1-3 dB, over a range of fixation angles of 45 degrees to either side. The eye position effect, measured as a function of pure-tone frequency, was most pronounced at 2 kHz and showed a tendency to decrease at lower and higher frequencies. The effect still occurred, although weaker, even when the eyes were directed to eccentric positions in darkness and without a fixation target. In experiment 2, the adjustment method was replaced by a two-alternative forced-choice method. Subjects judged whether sound bursts, presented with variable ILDs, were perceived on the left or right of the median plane during fixation of targets in various directions. Corresponding to experiment 1, the psychometric functions shifted significantly with gaze direction. However, the shift was only about half as large as that found in experiment 1. The shift of the subjective auditory median plane in the direction of eccentric gaze, observed in both experiments, indicates that dichotic sound is localized slightly to the opposite side, i.e., to the left when the gaze is directed to the right and vice versa. The effect may be related to auditory neurons which exhibit spatially selective receptive fields that shift with eye position. Erratum in Experimental Brain Research 110(2):322 (1996).
The development of peripheral auditory function in humans has been observed and documented using a variety of investigative tools. Because these tools must all be noninvasive in nature, they are indirect and, therefore, somewhat imprecise probes of function. Measured function at one level of the peripheral auditory system is undoubtedly influenced by the functional status of other parts of the system. Thus, the most effective way to define and document the physiology and developmental course of the human auditory system is to consider and integrate findings, with an acute awareness of the limitations of each assay and the relationship among results. To make this treatise a reasonable endeavor, only human auditory developmental data are presented and discussed, although when it elucidates a pattern common to humans, mammalian development in general may be considered.
Previous studies have demonstrated that the otoacoustic emissions (OAEs) measured during behavioral tasks can have different magnitudes when subjects are attending selectively or not attending. The implication is that the cognitive and perceptual demands of a task can affect the first neural stage of auditory processing—the sensory receptors themselves. However, the directions of the reported attentional effects have been inconsistent, the magnitudes of the observed differences typically have been small, and comparisons across studies have been made difficult by significant procedural differences. In this study, a nonlinear version of the stimulus-frequency OAE (SFOAE), called the nSFOAE, was used to measure cochlear responses from human subjects while they simultaneously performed behavioral tasks requiring selective auditory attention (dichotic or diotic listening), selective visual attention, or relative inattention. Within subjects, the differences in nSFOAE magnitude between inattention and attention conditions were about 2–3 dB for both auditory and visual modalities, and the effect sizes for the differences typically were large for both nSFOAE magnitude and phase. These results reveal that the cochlear efferent reflex is differentially active during selective attention and inattention, for both auditory and visual tasks, although they do not reveal how attention is improved when efferent activity is greater.
Mounting evidence suggests that auditory attention tasks may modulate the sensitivity of the cochlea by way of the corticofugal and the medial olivocochlear (MOC) efferent pathways. Here, we studied the extent to which a separate efferent tract, the ‘uncrossed’ MOC, which functionally connects the two ears, mediates inter-aural selective attention. We compared distortion product otoacoustic emissions (DPOAEs) in one ear with binaurally presented primaries, using an intermodal target detection task in which participants were instructed to report the occurrence of brief target events (visual changes, tones). Three tasks were compared under identical physical stimulation: (i) report brief tones in the ear in which DPOAE responses were recorded; (ii) report brief tones presented to the contralateral, non-recorded ear; and (iii) report brief phase shifts of a visual grating at fixation. Effects of attention were observed as parallel shifts in overall DPOAE contour level, with DPOAEs relatively higher in overall level when subjects ignored the auditory stimuli and attended to the visual stimulus, compared with both of the auditory-attending conditions. Importantly, DPOAE levels were statistically lowest when attention was directed to the ipsilateral ear in which the DPOAE recordings were made. These data corroborate notions that top-down mechanisms, via the corticofugal and medial efferent pathways, mediate cochlear responses during intermodal attention. New findings show attending to one ear can significantly alter the physiological response of the contralateral, unattended ear, probably through the uncrossed-medial olivocochlear efferent fibers connecting the two ears.
Attending to a single stimulus in a complex multisensory environment requires the ability to select relevant information while ignoring distracting input. The underlying mechanism and involved neuronal levels of this attentional gain control are still a matter of debate. Here, we investigated the influence of intermodal attention on different levels of auditory processing in humans. It is known that the activity of the cochlear amplifier can be modulated by efferent neurons of the medial olivocochlear complex. We used distortion product otoacoustic emission (DPOAE) measurements to monitor cochlear activity during an intermodal cueing paradigm. Simultaneously, central auditory processing was assessed by electroencephalography (EEG) with a steady-state paradigm targeting early cortical responses and analysis of alpha oscillations reflecting higher cognitive control of attentional modulation. We found effects of selective attention at all measured levels of the auditory processing: DPOAE levels differed significantly between periods of visual and auditory attention, showing a reduction during visual attention, but no change during auditory attention. Primary auditory cortex activity, as measured by the auditory steady-state response (ASSR), differed between conditions, with higher ASSRs during auditory than visual attention. Furthermore, the analysis of cortical oscillatory activity revealed increased alpha power over occipitoparietal and frontal regions during auditory compared with visual attention, putatively reflecting suppression of visual processing. In conclusion, this study showed both enhanced processing of attended acoustic stimuli in early sensory cortex and reduced processing of distracting input, both at higher cortical levels and at the most peripheral level of the hearing system, the cochlea.
In this study, a nonlinear version of the stimulus-frequency OAE (SFOAE), called the nSFOAE, was used to measure cochlear responses from human subjects while they simultaneously performed behavioral tasks requiring, or not requiring, selective auditory attention. Appended to each stimulus presentation, and included in the calculation of each nSFOAE response, was a 30-ms silent period that was used to estimate the level of the inherent physiological noise in the ear canals of our subjects during each behavioral condition. Physiological-noise magnitudes were higher (noisier) for all subjects in the inattention task, and lower (quieter) in the selective auditory-attention tasks. These noise measures initially were made at the frequency of our nSFOAE probe tone (4.0 kHz), but the same attention effects also were observed across a wide range of frequencies. We attribute the observed differences in physiological-noise magnitudes between the inattention and attention conditions to different levels of efferent activation associated with the differing attentional demands of the behavioral tasks. One hypothesis is that when the attentional demand is relatively great, efferent activation is relatively high, and a decrease in the gain of the cochlear amplifier leads to lower-amplitude cochlear activity, and thus a smaller measure of noise from the ear.