Oscillatory Beta Activity Predicts Response
Speed during a Multisensory Audiovisual
Reaction Time Task: A High-Density
Electrical Mapping Study
Daniel Senkowski1, Sophie Molholm1,
Manuel Gomez-Ramirez1,2and John J. Foxe1,2
1The Cognitive Neurophysiology Laboratory, Program in
Cognitive Neuroscience, Nathan S. Kline Institute for
Psychiatric Research, Orangeburg, NY 10962, USA and
2Program in Cognitive Neuroscience, Department of
Psychology, The City College of the City University of
New York, New York, NY 10031, USA
Bisensory redundant targets are processed faster than the respec-
tive unisensory target stimuli alone as evidenced by substantially
faster reaction times (RTs). This multisensory RT facilitation has
been interpreted as an expression of integrative processing be-
anisms underlying the RT facilitation effect are not well understood.
Oscillatory responses in the beta frequency range (13--30 Hz) have
been related to sensory-motor processing. Here, we investigated
whether modulation of beta responses might also underlie the
faster RTs seen for multisensory stimuli. Using high-density elec-
trical mapping, we explored the association between early (50--
170 ms) multisensory processing in the evoked beta response and
RTs recorded during a simple RT task. Subjects were instructed to
indicate the appearance of any stimulus in a stream of auditory-
alone (A), visual-alone (V), and multisensory (AV) stimuli by a button
press. Beta responses were analyzed using Morlet wavelet trans-
formations. Multisensory interactions were found over frontal,
occipital, central, and sensory-motor regions. Critically, beta ac-
tivity correlated with mean RTs over all stimulus types. Significant
negative correlations were found for frontal, occipital, and sensory-
motor scalp regions. We conclude that the association between
oscillatory beta activity and integrative multisensory processing is
directly linked to multisensory RT facilitation effects.
Keywords: cross-modal, EEG, ERP, motor system, sensory integration
The integration of inputs from multiple sensory modalities can
facilitate motor behavior. For example, a tennis player can esti-
mate the speed, direction, and spin of a ball by integrating visual
and auditory information when his or her opponent hits the ball.
The visual inputs may allow the player to estimate the direction
of the ball, whereas the sound can give him or her further infor-
mation about the speed and spin of the ball. One illustration of
the advantage incurred by multiple sensory inputs comes from
studies showing that multisensory stimuli can be processed
faster than the corresponding unisensory targets (Schro ¨ ger and
Widmann 1998; Molholm and others 2002, 2004; Teder-Sa ¨ leja ¨ rvi
and others 2002; Murray and others 2005; Senkowski and others
2005; Talsma and Woldorff 2005). Although the reaction time
(RT) facilitation effect is a robust finding and can be attributed,
at least in some cases, to the occurrence of multisensory inter-
actions (e.g., Molholm and others 2002; Patching and Quinlan
2004), the underlying neuronal mechanisms remain to be
One mechanism that might be associated with the multisen-
sory RT facilitation effect is neuronal processing in the beta
frequency range (about 13--30 Hz). Studies using intracranial
(Crone and others 1998) and scalp-surface recordings in
humans (Classen and others 1998; Pfurtscheller, Graimann,
and others 2003), as well as studies investigating local field
potentials in monkeys (Liang and others 2002), have indicated
that oscillatory beta activity is closely related to sensory-motor
processing. Classen and others (1998) reported an increase of
interregional coherence in the beta frequency over different
scalp areas, including regions that have been associated with
activity from the visual and motor cortices, when subjects
performed a tracking task that included visual feedback. The
authors suggested a functional relationship between beta
activity and large-scale motor network processing.
Although a number of studies have examined the association
between beta activity and motor behavior for unisensory
stimuli, to our knowledge this has not been done for multisen-
sory stimuli. One study did investigate multisensory integration
effects for beta activity (among other frequency bands), but this
was recorded while subjects passively experienced unisensory
auditory, unisensory visual, and multisensory audiovisual stimuli
(Sakowitz and others 2005). Because ofthe lack of a task and the
fact that only 11 scalp channels were recorded, it is not possible
to draw conclusions regarding the functional significance of the
beta activity or the intracranial sources of the underlying
generators, except in the most general terms. Nevertheless, in
this study, audiovisual integration effects in the beta frequency
range from data recorded over central and parietooccipital
scalp regions were found, starting as early as 50 ms after
stimulus onset. These showed increased activity for the
multisensory compared with summed unisensory responses.
Thus, the Sakowitz and others (2005) study revealed that
multisensory interactions for auditory and visual stimuli do
occur in the beta frequency range and that these interactions
onset early in the cortical information-processing hierarchy.
The link between oscillatory beta activity and sensory-motor
processing and evidence of multisensory effects on beta activity
suggested that such activity may play an important role in the
RT facilitation effect for multisensory stimuli. We tested this
hypothesis by examining the correlations between mean RTs
and evoked beta activity in high-density electrical recordings
from a data set for which the event-related potentials (ERPs)
have been previously reported (Molholm and others 2002). In
this study, a response-time advantage was found for auditory
and visual stimuli when presented together compared to
when presented alone during a simple RT task in which the 3
stimulus types were presented in a randomly interleaved order.
Significantly, examination of the ERPs elicited by these stimuli
showed a number of multisensory integration effects, revealing
a widespread network of multisensory activity, which began at
the early latency of 46 ms over the right parieto-occipital scalp
(Molholm and others 2002).
Cerebral Cortex November 2006;16:1556--1565
Advance Access publication December 15, 2005
? The Author 2005. Published by Oxford University Press. All rights reserved.
For permissions, please e-mail: email@example.com
For the present analysis, we focused on activity that occurred
prior to 200 ms, under the assumption that the multisensory
effects on beta activity would have to occur relatively early
poststimulus onset to affect motor behavior. We expected that
putative beta effects would be generated, at least in part, in
cortical regions that were involved in the sensory processing of
the auditory and visual inputs. There is ample evidence from
both human (Giard and Peronnet 1999; Foxe and others 2000;
Calvert 2001; Foxe and Schroeder 2005) and nonhuman
(Schroeder and others 2001, 2003) primate investigations of
cortical multisensory interactions that occur relatively early in
time and in cortical regions previously thought to be ‘‘uni-
sensory’’ (for a recent review, see Schroeder and Foxe 2005);
and what’s more, a number of studies have shown evoked
oscillatory beta activity over scalp regions that are traditionally
associated with projections from sensory specific cortex
(Tzelepi and others 2000; Haenschel and others 2000; Sakowitz
and others 2005). In addition, because recent studies have
indicated an association between frontal beta activity and
sensory-motor processing (Liang and others 2002; Alegre and
others 2004), we also inspected beta activity over frontal scalp
sites and sensory-motor areas.
To summarize, the primary objective of the present study was
to examine the relationship between multisensory interactions
inthe beta frequency range and motorprocessing. We therefore
tested for correlations between RTs and evoked beta activity.
We expected that higher amplitude beta activity would corre-
late with faster RTs primarily for those areas for which
multisensory interactions were observed. Significantly, this
addressed a mechanism by which the RT facilitation for
bisensory redundant targets may be achieved. The positive
results that we found support the notion that multisensory
effects on beta activity may contribute to the RT facilitation
effects that are commonly observed for multisensory stimuli.
Twelve neurologically normal, paid volunteers participated in the study
(5 female and 11 right handed, mean age 23.8 ± 2.69). All subjects
reported normal hearing and had normal or corrected-to-normal vision.
Data from 2 additional subjects were excluded due to excessive eye
movements and failure to maintain central fixation. The Institutional
Review Board of the Nathan Kline Institute for Psychiatric Research
approved the experimental procedures, and each subject provided
written informed consent.
Stimuli and Procedure
The auditory stimulus consisted of a 1000-Hz tone (60-ms duration,
75-dB sound pressure level, 5-ms rise/fall times) presented from a single
speaker located atop the monitor on which the visual stimuli were
presented. The visual stimulus consisted of a red circular disk sub-
tending 1.2 degrees in diameter presented on a black background for 60
ms. The location of the circle was individually adapted after completion
of a pretest session in which the circle was presented from 16 different
spatial locations (1.6 or 2.6 degrees laterally from fixation and 1.6 or 2.0
degrees vertically from fixation). One focus of the ERP study of Molholm
and others (2002) was on the analyses of early integration effects on the
visual C1 component in ERPs. This component is usually observed after
about 50 ms poststimulus (e.g., Clark and Hillyard 1996; Foxe and
Simpson 2002). For this reason, the spatial location for which the
highest C1 amplitudes were observed was chosen for visual stimulation.
In 11 of the 12 subjects, the highest C1-P1 amplitudes were found in the
upper visual field at 1.6 degrees lateral left (n = 8) or lateral right (n = 3).
Only 1 subject showed the highest C1-P1 amplitudes in the lower left
visual field (2.6 degrees left, 2 degrees below fixation). Subjects were
instructed to maintaincentral fixation at all times and to make a speeded
button response with their right index finger when a stimulus in either
sensory modality was detected (unisensory or multisensory stimuli).
Sensory stimuli could appear in 3 different conditions: unisensory
auditory (A), unisensory visual (V), or multisensory audiovisual (AV).
The 3 stimulus conditions were presented with equal probability in
a random order. Interstimulus interval varied randomly between 750
and 3000 ms. Each block consisted of 150 trials, and each subject
completed a minimum of 18 blocks.
Analysis of RTs
For individual subjects, the response-time distributions were recorded,
and mean RTs were calculated for each of the stimulus conditions.
Response-time facilitation for the bisensory stimulus was followed up
with a test of the so-called ‘‘Race model’’ (e.g., Miller 1982, 1991). The
Race model tests whether simple statistical facilitation can account for
the observed effects. This analysis has been reported in detail in
Molholm and others (2002) and will be briefly recapitulated here.
Calculation of the Race Model
The Race model places an upper limit on the cumulative probability
(CP) of RT at a given latency for stimulus pairs. For any latency, t, the
Race model holds when this CP value is less than or equal to the sum of
the CP from each of the single stimuli minus an expression of their joint
probability (CP(t)simultaneous< ((CP(t)alone 1+ CP(t)alone 2) – (CP(t)alone 13
CP(t)alone 2))). For each subject, the RT range within the valid RTs (100--
800 ms) was calculated over the 3 stimulus conditions and divided into
quantiles from the first to the hundredth percentile in 5% increments
(1%, 5%, ..., 95%, 100%). T-tests comparing the actual (CP(t)simultaneous)
and predicted ((CP(t)alone 1+ CP(t)alone 2) – (CP(t)alone 13 CP(t)alone 2))
facilitation were performed to assess the reliability of violations of the
Race model. Any violation of the Race model would indicate that the RT
facilitation was at least partially due to interactions between the
auditory and visual inputs.
The main assumption of the Race model is that each stimulus of a
given stimulus pair (e.g., the auditory and visual stimulus of a multisen-
sory audiovisual stimulus) independently competes for response initi-
ation and that the faster of the two will trigger the response on a given
trial. Due to the mechanism of 2 independently competing unisensory
processing streams, mean RTs to multisensory stimuli will, by definition,
be faster than mean RTs to unisensory stimuli alone, even without
multisensory interactions. To draw a parallel, a bisensory stimulus is akin
to having 2 rolls of a dice to hit a ‘‘six’’ rather than one, simply due to
probability summation. The Race model allows one to test whether RTs
to multisensory stimuli are faster again than the predictions of simple
The electroencephalogram (EEG) was recorded from 2 electrooculo-
gram (EOG) and 126 scalp electrodes (impedances <5 kX), referenced
to the nose, and band-pass filtered from 0.05 to 100 Hz at a sample rate
of500Hz.In addition,datawere filteredoff-linewith a 60-Hznotchfilter
(57.74--62.25 (–3 dB)). The purpose of this filter was to suppress the 60-
Hz high-frequency noise. Averaging epochs for EEG beta activity lasted
from 400 ms before to 800 ms after stimulus onset. Prior to averaging,
baselines were computed for each trial from –300- to
prestimulus time interval. For artifact suppression, trials were automat-
ically excluded from averaging if the standard deviation within a moving
200-ms time interval exceeded 30 lV in any one of the EEG channels
and 40 lV at the EOG channels in a time interval between –300 and 500
ms. The reader should note that throughout this paper we will use the
familiar nomenclature of the modified 10--20 electrode system to refer
to the positioning of electrode sites (American Electroencephalo-
graphic Society 1991). Because our montage contains considerably
more scalp sites than this nomenclature allows for, in some cases, we
will be referring to the nearest neighboring site within the 10--20
Morlet Wavelet Transformation
For the analysis of oscillatory beta activity, a wavelet transform based on
Morlet wavelet was employed (Herrmann and others 1999; Senkowski
Cerebral Cortex November 2006, V 16 N 11 1557
and Herrmann 2002). An established method to examine multisensory
interactions inERP is thecomparisonofaudiovisual (AV) responseswith
summed auditory plus visual (A + V) responses (Giard and Perronet
1999). However, as illustrated in Figure 1, the event-related oscillatory
responses to unisensory auditory and unisensory visual stimuli (left
panel) cannot simply be summed up after the wavelet transformation
(right panel). As such, a comparison of the summed induced unisensory
responses (i.e., activity that is stimulus related but not necessarily phase
locked to the onset of events over trials) is not valid except in the very
unlikely scenario that these responses are in perfect phase across all
epochs. Thus, the lack of additivity for the (AV – (A + V)) comparison
prevents us from considering induced beta responses for the analysis of
The additivity problem, however, can be solved for the evoked
responses as follows: One first computes the sum of the event-related
responses to unisensory auditory and unisensory visual stimuli and then
calculates a wavelet transformation for these summed event-related
responses (Fig. 1, lower panel left). The result of this transformation is
the evoked activity (i.e., activity that is strictly phase and time locked to
the onset of events) of the summed unisensory auditory plus unisensory
visual (A + V) stimuli. The evoked beta activity of summed A + V
responses was compared with the evoked beta activity of multisensory
The frequency for the wavelet transformation was individually
adapted for each subject using the frequency with the highest ampli-
tudes in the early evoked oscillatory responses in the beta frequency
range (13--30 Hz) to multisensory stimuli. This estimate was made over
right frontal scalp (in the region of the F69 electrode site) because the
maximum beta amplitudes to multisensory stimuli were observed in this
channel. The average of individually adapted evoked beta activity was
17.8 Hz(range15--23Hz). Theduration(2rt) ofwavelettransformations
with central frequencies of 18 Hz (f0= 18) was 111 ms, resulting in
a spectral bandwidth (2rf) of 5.74 Hz.
Regions of Interests
To avoid the loss of statistical power inherent when repeated-measures
analysis of variance (ANOVA) is used to quantify multichannel EEG data,
our tests were limited to activity within 6 regions of interest (ROIs)
(Fig. 2). We delineated a pair of bilateral occipital ROIs to reflect activity
from occipital areas and 1 central ROI to reflect processing in auditory
regions. Furthermore, because previous studies have shown a close
association between beta responses and frontal scalp regions (Liang and
others 2002; Alegre and others 2004), we also computed beta activity
for a pair of bilateral frontal ROIs. Finally, we analyzed 1 scalp region
consisting of channels located over the left sensory-motor cortex. This
ROI was chosen to reflect processing of the sensory-motor system
because subjects used their right index finger to respond to all stimulus
Time--Frequency Planes of Evoked Beta Activity
Figure 3 shows multisensory (AV) and summed auditory plus visual (A +
V) ERP responses of a single subject. An event-related oscillatory
pattern, which starts at about 70 ms after stimulus onset, with a fre-
quency of about 19 Hz, can be observed over multiple scalp regions and
particularly strongly in the right frontal scalp site (F69). It should be
noted that the oscillatory beta pattern is riding on lower frequency
activity (in particular at occipital electrode sites) and therefore is not as
clearly visible in the time-locked ERPs as it might be. Morlet wavelet
transformations were calculated separately for each frequency in a
range between 13 and 30 Hz for a frontal right (F69) and a frontal
left (F59) electrode and plotted into time--frequency representations.
Figure 4 displays the grand average time--frequency representations
for the 12 subjects showing anterior effects that begin at about 70 ms
Analysis of Multisensory Interactions in the Evoked
For each of the 6 scalp ROIs, the mean values (over electrodes) of max-
imum evoked beta amplitudes in a time interval between 50 and 170 ms
were computed and analyzed. First, we performed a single global
ANOVA using the within-subject factors ‘‘ROI’’ (6 ROIs) and ‘‘stimulus
type’’ (multisensory AV beta activity and auditory plus visual (A + V) beta
activity) to establish whether a multisensory beta effect was evident. In
the second phase of our analysis, we performed separate repeated-
measures ANOVAs for the frontal and occipital ROIs, using the within-
subject factors stimulus type (multisensory AV beta activity and auditory
plus visual (A + V) beta activity) and ‘‘hemisphere’’ (left and right). Lastly,
for the central ROI and for the left sensory-motor ROI, ANOVAs with the
Figure 1. The figure illustrates the differences between the linear summation of
event-related oscillatory beta responses to unisensory auditory and unisensory visual
stimuli (left panel) and the linear summation of the same responses after wavelet
transformation(rightpanel)forsimulateddata. Hereweshowa casewherethere is no
multisensory interaction (i.e., A + V = AV). Due to phase variability, the summation of
unisensory oscillatory responses in the time domain, in nearly all cases, results in
amplitude that is less than the sum of the amplitudes of these responses (left panel).
For this reason, the wavelet-transformed responses to unisensory stimuli (right panel)
cannot simply be summated and compared with wavelet-transformed responses to
Figure 2. Definition of ROIs for the statistical analyses. Two frontal ROIs, 2 occipital
ROIs, and 1 central ROI were analyzed. FL, frontal left; FR, frontal right; OL, occipital
left; OR, occipital right; SML, sensory-motor left.
Beta Activity Predicts Response Speed
Senkowski and others
factor stimulus type (multisensory AV beta activity and auditory plus
visual (A + V) beta activity) were performed.
Analysis of Beta Activity and RTs
The relationships between beta activity and RTs over the 3 stimulus
types (unisensory A, unisensory V, and multisensory AV) were in-
vestigated by calculating their correlations (Pearson correlation) at the
6 ROIs. In addition, we calculated the RTs for single trials (separately for
each subject and each modality) and compared the beta responses of
the fastest 50% trials with the slowest 50%. This was done using an
ANOVA with factors of ‘‘RTs’’ (fastest 50%, slowest 50%), ‘‘stimuli’’
(unisensory A, unisensory V, multisensory AV), and ROI (6 ROIs). The
lack of additivity for the induced responses in the AV – (A + V)
comparison does not preclude studying the relationships between
induced beta responses and RTs. The comparison of beta response
and RTs was therefore performed for both evoked and induced
responses. In case of nonsphericity, all calculated ANOVAs in this
work were adjusted with the Greenhouse--Geisser epsilon correction.
Fastest responses were observed for multisensory AV stimuli
(255 ± 35 ms) followed by auditory (297 ± 48 ms) and visual
stimuli (305 ± 31 ms). Multisensory stimuli were processed
significantly faster than both unisensory auditory (t11= 7.97, P <
0.001) and unisensory visual (t11= 11.06, P < 0.001) stimuli. In
addition, robust violations of the Race model were also observed
such that fully 25% of all RTs to AV stimuli were faster than the
predictions ofsimple probability summation (described in detail
Figure 3. ERPs for 1 subject for multisensory AV stimuli and summed A + V ERPs. The oscillatory pattern in the ERPs is best visible at channel F69. This pattern reflects an event-
related oscillatory activity in the beta frequency range (here at about 19 Hz). ERPs were 30-Hz low-pass filtered before displaying.
Figure 4. Time--frequency plots of evoked beta activity for auditory (A) stimuli (left panel), visual (V) stimuli (middle panel), and multisensory audiovisual (AV) stimuli (right panel)
for 1 frontal right electrode (channel F69, upper panel) and 1 frontal left electrode (channel F59, lower panel). The figure indicates enhanced evoked beta activity for multisensory
stimuli as compared with unisensory stimuli.
Cerebral Cortex November 2006, V 16 N 11 1559
in Molholm and others 2002, p. 119). This violation of the Race
model verifies that nonlinear multisensory integration pro-
cesses must have taken place.
Beta Activity and Multisensory Integration
Evoked beta responses to visual stimuli peaked over occipital
brain regions and beta responses to auditory stimuli over central
regions (Figs. 5 and 6). Furthermore, for multisensory stimuli,
we observed the strongest beta responses over frontal brain
regions. Interestingly, the maximum differences between mul-
tisensory and summed auditory plus visual grand mean beta
activity peaked between 90 and 130 ms across the ROIs. The
pattern of beta activity showed general enhanced responses to
multisensory as compared with summed auditory plus visual
stimuli. We tested the differences between multisensory and
summed auditory plus visual beta activity at the 6 ROIs in
repeated-measures ANOVAs using the factor stimulus type
(maximum of multisensory AV beta responses and summed
(A + V) beta responses in a time interval between 50 and
170 ms). In addition, the factor hemisphere (left, right) was
included in the ANOVAs for frontal and occipital ROIs.
For the global ANOVA, where all 6 ROIs were included, a
significant main effect of the within-subject factor stimulus type
was observed (F1,11= 33.87, P <0.001). This effect showed that
evoked beta responses were greater for multisensory AV stim-
uli (0.89 lV) as compared with summed A + V stimuli (0.65 lV).
Figure5. Wavelet-transformedevoked betaactivityin responsetomultisensoryAVstimuliandsummedA +V beta responses. Thefigure illustratesasubadditivity(A+ V <AV) of
evoked beta activity at multiple electrode sites in a time range between 70 and 150 ms.
Figure 6. Topographic maps of evoked auditory (A), visual (V), and audiovisual (AV) beta activity for a time range between 50 and 170 ms after stimulus onset (upper panel). Beta
activity in response to auditory stimuli was strongest over central scalp regions, whereas beta responses to visual stimuli peaked over occipital brain areas. For multisensory stimuli,
activity was mainly observed over frontal, central, and occipital scalp regions. The lower panel shows the subtraction of audiovisual (AV) minus summed auditory and visual (A + V)
Beta Activity Predicts Response Speed
Senkowski and others
No further effects were observed in this ANOVA. Then, planned
follow-up analyses of individual ROIs were performed.
For the frontal ROIs, we found a significant main effect of the
within-subject factors stimulus type (F1,11= 7.36, P <0.03) and
hemisphere (F1,11= 6.50, P < 0.03). Higher beta amplitudes
were observed for multisensory AV stimuli (0.88 lV) as com-
pared with summed A + V stimuli (0.66 lV) (Fig. 7). Further-
more, beta amplitudes were higher over the right (0.84 lV) than
over the left frontal ROI (0.69 lV). No significant interaction
between the factors stimulus type and hemisphere was found.
Separate analyses of multisensory AV versus summed beta
responses for the left and right hemisphere revealed significant
effects of the factor stimulus type for the left (F1,11= 8.35, P <
0.02) and the right (F1,11= 5.11, P < 0.05) ROIs due to higher
evoked beta amplitudes in response to multisensory AV stimuli.
The ANOVA for the occipital ROIs only approached signifi-
cance for the within-subject factor stimulus type (F1,11= 4.66,
P < 0.06), suggesting a trend toward higher amplitudes for
multisensory (0.98 lV) as compared with summed A + V stimuli
to further investigate potential occipital effects. These separate
effect of the factor stimulus type for the right ROI (F1,11= 8.83,
P < 0.02) but not for the left ROI (F1,11= 1.80, P = 0.21).
Next, multisensory interactions were investigated for the
central ROI. The ANOVA for this ROI revealed a significant
effect of the factor stimulus type (F1,11= 11.21, P <0.01) due to
higher amplitudes for multisensory (0.85 lV) as compared with
summed A + V beta responses (0.56 lV).
Finally, multisensory interactions over the left sensory-motor
region were analyzed. The ANOVA revealed a significant main
effect of the within-subject factor stimulus type (F1,11= 21.32,
P < 0.001), indicating higher amplitudes for multisensory
(0.75 lV) as compared with summed A + V beta responses
Beta Activity and RTs
Figure 8 shows the correlations between RTs over the
stimulus 3 types with evoked beta activity at the 6 ROIs.
Significant negative correlations between the RTs and beta
activity were observed for the left frontal, right frontal, right
occipital, and left sensory-motor ROIs (left frontal: r = –0.44, P <
0.008; right frontal: r = –0.38, P < 0.03; right occipital: r = –0.44,
P <0.008; left sensory-motor: r = –0.346, P <0.04; N = 36). These
correlations show a clear relationship between higher ampli-
tude beta activity and faster RTs. The correlations between the
left occipital and central beta activity with RTs, however, did
not approach significance (left occipital: r = –0.25, P = 0.15;
central: r = –0.25, P = 0.15; N = 36).
Figure 7. Evoked beta activity for auditory (A), visual (V), audiovisual (AV), and summed A + V stimuli at the 5 ROIs. Significant enhanced beta responses were observed at left
frontal, right frontal, central, and right occipital scalp regions. Notice that auditory and visual beta responses do not sum up in a linear way because of the different phases of
oscillatory beta responses to auditory and visual stimuli (see Methods for details).
Cerebral Cortex November 2006, V 16 N 11 1561
The ANOVA for the comparison of the evoked responses of
the 50% fastest RTs with the evoked responses of the 50%
slowest RTs revealed a significant main effect of the within-
subject factor RTs (F1,11= 5.04, P < 0.05) due to higher evoked
RTs.This findingfurthersupports theassumption ofadirectlink
between RTs and evoked oscillatory beta activity. No other ef-
fects or interactions were observed in relation to the factor RTs.
The same analyses were performed for induced beta re-
sponses. For these responses, no correlations with RTs were
found. In addition, there were no differences in induced beta
responses between trials with the 50% fastest and trials with the
induced responses differ between trials with the 25% fastest and
the 25% slowest RTs. Again, this analysis did not reveal any
differences between induced responses to fast and slow trials.
Here we show robust audiovisual multisensory interactions in
the evoked oscillatory beta band, which occur early during
sensory processing. These interactions were mainly seen over
scalp regions also activated by the unisensory stimuli and
peaked substantially before the overt motor response. Consis-
tent with the proposed link between multisensory interactions
for beta activity and response-time facilitation, correlation
analysis demonstrated that the evoked beta responses are
predictive of RT speed over the different stimulus types. We
discuss these findings in detail below.
Multisensory Effects on Beta Activity
Multisensory effects were found to be widespread, with en-
hanced beta activity for multisensory compared with summed
unisensory evoked responses over a number of ROIs. These
were specifically over the bilateral frontal ROIs, the central ROI,
the right occipital ROI, and the left sensory-motor regions. The
multisensory interactions occurred in the same general time
frame across the different ROIs, with differences between the
multisensory and summed responses peaking around 120 ms, as
illustrated in the wavelet-transformed evoked beta activity
depicted in Figure 5. Thus, the data reveal a network of mul-
tisensory interactions occurring within a relatively restricted
time range in the stimulus-evoked oscillatory beta activity.
Figure 8. Correlations between evoked beta activity to auditory (A), visual (V), and audiovisual (AV) stimuli and mean RTs of the 3 stimulus types. Significant negative correlations
wereobserved for leftand right frontal and right occipitalscalp regions. This indicatesthat faster RTs werepredicted by higherbeta responses. Linearregression lines betweenbeta
activity and RTs are plotted in the graphs.
Beta Activity Predicts Response Speed
Senkowski and others
The multisensory effects appeared to be largely due to the
amplification of unisensory activity. More specifically, the
responses to auditory stimuli when presented alone showed
peak beta activity largely over central scalp regions (as in
Haenschel and others 2000; Makinen and others 2004), and
there was an increase in this activity when the same auditory
stimulus was paired with the visual stimulus. The auditory
stimuli also resulted in clear activity focused over right frontal
scalp and relatively lower amplitude activity focused over left
frontal scalp. Again, across these regions, this activity was
greater when the auditory stimulus was paired with the visual
stimulus. Similarly, the beta responses to visual stimuli pre-
sented alone resulted in peak beta activity largely over right
occipital scalp, and there was an increase in this activity when
the same visual stimulus was paired with the auditory stimulus.
In addition to these effects over sensory regions, evoked beta
responses were also enhanced over the sensory-motor scalp
region, further supporting the assumption that the multisensory
interactions reported here are likely to be linked to sensory-
The multisensory interactions in the evoked beta response
showed some similarities to the ERP results reported by
Molholm and others (2002). As in the present study, Molholm
and others reported a multisensory interaction over right
frontal scalp, and this was found within the same time frame
(starting at about 100 ms). However, the frontal interaction in
the ERP appeared to be subadditive (i.e., the audiovisual (AV)
ERP was lower in amplitude than the summed (A + V) ERP),
whereas the interaction seen in the frontal beta activity was
superadditive (i.e., audiovisual (AV) beta activity was higher
than the summed (A + V) beta activity). Although ‘‘subadditivity’’
in the ERP does not necessarily indicate a diminution of activity
within a generator configuration (a new generator with
opposite scalp polarity could result in such subadditivity), this
does raise the possibility that the frontal effects on evoked beta
activity and ERPs actually reflect distinct neuronal processes
that may be involved in different aspects of cortical multisen-
Finally, it is of note that the multisensory interactions and
correlations to behavior were observed over the right but not
over the left occipital ROI. This observation is in line with
previous ERP studies that have reported stronger early-latency
integration effects over right occipital than over left occipital
cortical regions (Giard and others 1999; Fort and others 2002;
Molholm and others 2002) and may suggest some degree of
hemispheric specialization for early audiovisual multisensory
Oscillatory Beta Activity and RT Performance
Correlation analysis revealed that higher amplitude--evoked
beta activity was significantly related to faster RTs for bilateral
frontal, right occipital, and sensory-motor ROIs. This suggests
that early evoked beta activity is indeed related to motor
behavior and that multisensory interactions in the beta fre-
quency range can result in faster RTs. Further supporting a link
between evoked beta activity and RTs, higher evoked beta
activity was associated with trials with short RTs compared to
trials with long RTs.
It is also important to consider that even in the absence of
a motor task, such beta activity is present and modulated by
multisensory inputs (Sakowitz and others 2005) and that
further, for the present study, not all beta activity that was
analyzed was shown to correlate with RTs (i.e., the central ROI
did not). Assuch, we arguethat these multisensory effects inthe
evoked beta responses are likely associated with subsequent
cortical mechanisms directly involved in sensory-motor pro-
cessing but that not all the beta activity examined here directly
reflects the motor processes themselves. However, overall, the
present correlation analyses revealed a robust pattern of results
that suggests a close association between early evoked beta
responses and RTs at the same ROIs for which we observed
It could be argued that the present results can be explained in
terms of overall stimulus energy rather than a multisensory
integration effect per se. For example, one might posit that the
same effect could be found if 2 visual stimuli were used
simultaneously or 2 auditory stimuli rather than stimuli across
2 sensory systems. However, the extant literature argues
otherwise. That is, the violations of the Race model seen for
multisensory stimuli are not observed for multiple copies of
unisensory stimuli. For example, Murray and others (2001)
assessed RT effects as a function of 1 versus 2 copies of a visual
stimulus (both within and across hemifields) and although RTs
sped up (the so-called redundant target effect), they did so in
keeping with a simple probability summation account—that is,
there was no nonlinear shortening of RT for unisensory pairs of
stimuli. This has been a consistent finding in the literature (e.g.,
Meijers and Eijkman 1977; Reuter-Lorenz and others 1995;
Corballis 1998). In addition, we observed higher beta activity for
trials with short RTs as compared with trials with long RTs in an
overall analysis including the 3 stimulus conditions. This further
demonstrates the direct association between evoked beta
activity and RTs, which cannot be explained by differences in
stimulus energy across conditions.
Comparison with Beta Activity Reported in Other Studies
There are some major differences between the beta responses
observed here and the beta responses that have been reported
in a number of previous studies (e.g., Pfurtscheller and Lopes da
Silva 1999; Feige and others 2000; Pfurtscheller, Woertz, and
others 2003). In part, this is due to different approaches to data
analysis. For instance, in some of these studies, beta activity has
been investigated during the execution (Feige and others 2000;
Pfurtscheller, Graimann, and others 2003) or observation
(Babiloni and others 2002) of a motor response, whereas here
we investigated early activity that was observed clearly before
the motor response. In these studies looking at induced beta
activity, a common finding is a decrease in induced responses
during and after the execution of a movement, and this is typ-
ically seen over sensory-motor areas (Neuper and Pfurtscheller
2001; Babiloni and others 2002). The absence of a relationship
between early-induced beta responses and RTs in our study,
however, suggests that these early responses are not sub-
stantially linked to the response-time facilitation effect ob-
served for multisensory stimuli. Similarly, it has been previously
shown that early latency effects of different experimental
manipulations such as attention and memory are primarily
reflected in the evoked and not in the induced responses
(Senkowski and Herrmann 2002; Herrmann and others 2004;
Busch and others 2005; Senkowski and others 2005).
Evoked beta responses have been examined with respect
to RTs, with several studies reporting an increase in evoked
beta activity after the offset of a movement (Pfurtscheller
1992; Pfurtscheller, Woertz, and others 2003). The observation
Cerebral Cortex November 2006, V 16 N 11 1563
of postmovement-evoked beta responses led Neuper and
Pfurtscheller (2001) to hypothesize that these responses may
reflect a decreased excitability of cortical networks in the
motor system. However, the beta responses in the present study
were observed prior to the behavioral motor responses, and it
was higher beta amplitude that was related to faster RTs. As
such, our study appears to be at odds with their prediction.
Rather, our results suggest that evoked beta activity prior to
a motor response is more likely to reflect increased activation.
Finally, intercortical coherence in beta activity has been
associated with multisensory processing. One study showed
that intercortical beta coherence was increased between
posterior regions of parietal and temporal scalp for stimuli
representing objects, whether presented as images, spoken
words, or written words (von Stein and others 1999). This
activity was interpreted as reflecting supramodal aspects of
the stimuli, further supporting the assumption that beta
activity plays an important role in multisensory audiovisual
This work focused specifically on the analysis of beta activity
in the context of the RT facilitation effect on multisensory
stimuli. The motivation for choosing beta responses for the
present analysis was that this activity has been most frequently
related to sensory-motor processing and RTs (e.g., Crone and
others 1998; Liang and others 2002; Pfurtscheller, Woertz, and
others 2003) and was also very recently related to audiovisual
multisensory processing (Sakowitz and others 2005). We should
point out though that it is also possible that activity within other
frequency bands, not investigated here, may also be involved in
these processes. For example, multisensory interactions have
also been observed in the gamma and theta bands (Sakowitz and
others 2000, 2001; Senkowski and others 2005) and another
frequency band that has been consistently related to behavioral
performance is the alpha band (e.g., Foxe and others 1998;
Klimesch 1999; Fu and others 2001; Klimesch and others 2003).
As such, future studies should address how oscillatory activity in
these frequency bands might also relate to multisensory
The Use of Wavelet Analysis to Investigate Multisensory
Previous studies have shown that ERPs can be displayed as
a superposition of electrical responses in different frequency
ranges (Karakas and others 2000; Basar and others 2001), and as
such, multisensory processing within a specific frequency may
be obscured in the time-locked ERP. One reason for this is that
evoked beta responses will partially cancel out in the grand
average ERPs when they differ in phase over subjects. Another
reason is that event-related oscillatory responses (in particular
those in higher frequencies) are often superimposed by activity
in lower frequency ranges. The Morlet wavelet transformation
that was employed here for the analysis of multisensory inter-
actions, however, is insensitive to phase differences of beta
activity over subjects and allowed us to specifically extract beta
responses in the time and the frequency domains. Thus, the
application of the wavelet transformation might allow a more
accurate examination of multisensory interactions in the beta
frequency range than the analyses of ERPs alone does. Here we
show that examination of evoked oscillatory activity can be
highly useful in understanding therelationship between cortical
multisensory processing and behavior.
Summary and Conclusions
To our knowledge this is the first study to describe a close
association between early evoked beta activity and RTs during
multisensory processing. We observed early multisensory inter-
actions in the beta frequency range over multiple scalp areas,
suggesting a widespread network of cortical regions that begins
to oscillate at a similar latency after the onset of an audiovisual
stimulus. Consistent with a link between multisensory inter-
actions for beta activity and response-time facilitation, beta
activity at most of the regions that showed multisensory
interactions also predicted the average RTs across the different
stimulus types. We propose that evoked beta activity plays an
important role in early multisensory integrative processing and
that such processing is linked to the sensory-motor system.
This work was supported by grants from the National Institute of
Mental Health (MH65350) and the National Institute on Aging
(AG22696) to JJF. We would also like to thank two anonymous
reviewers for their helpful comments on this article.
Address correspondence to Dr John J. Foxe, PhD, Nathan Kline
Institute for Psychiatric Research, Program in Cognitive Neuroscience
and Schizophrenia, 140 Old Orangeburg Road, Orangeburg, NY 10962,
USA. Email: firstname.lastname@example.org.
Alegre M, Gurtubay IG, Labarga A, Iriarte J, Valencia M, Artieda J. 2004.
Frontal and central oscillatory changes related to different aspects of
the motor process: a study in go/no-go paradigms. Exp Brain Res
American Electroencephalographic Society. 1991. American Electro-
encephalographic Society guidelines for standard electrode position
nomenclature. Clin Neurophysiol 8:200--202.
Babiloni C, Babiloni F, Carducci F, Cincotti F, Cocozza G, Del Percio C.
2002. Human cortical electroencephalography (EEG) rhythms
during the observation of simple aimless movements: a high-
resolution EEG study. Neuroimage 17:559--572.
Basar E, Schurmann M, Demiralp T, Basar-Eroglu C, Ademoglu A. 2001.
Event-related oscillations are ‘real brain responses’—wavelet analysis
and new strategies. Int J Psychophysiol 39:91--127.
Busch NA, Schadow I, Frund I, Herrmann CS. October 20, 2005. Time-
frequency analysis of target detection reveals an early interface
between bottom-up and top-down processes in the gamma-band.
Neuroimage [Epub ahead of print].
Calvert GA. 2001. Crossmodal processing in the human brain:
insights from functional neuroimaging studies. Cereb Cortex 11:
Clark VP, Hillyard SA. 1996. Spatial selective attention affects early
extrastriate but not striate components of the visual evoked
potential. J Cogn Neurosci 8:387--402.
Classen J, Gerloff C, Honda M, Hallett M. 1998. Integrative visuomotor
behavior is associated with interregionally coherent oscillations in
the human brain. J Neurophysiol 79:1567--1573.
Corballis MC. 1998. Interhemispheric neural summation in the absence
of the corpus callosum. Brain 121:1795--1807.
Crone NE, Miglioretti DL, Gordon B, Sieracki JM, Wilson MT, Uematsu S,
Lesse RP. 1998. Functional mapping of human sensorimotor cortex
with electrocorticographic spectral analysis. I. Alpha and beta event-
related desynchronization. Brain 121(Pt 12):2271--2299.
Feige B, Aertsen A, Kristeva-Feige R. 2000. Dynamic synchronization
between multiple cortical motor areas and muscle activity in phasic
voluntary movements. J Neurophysiol 84:2622--2629.
Fort A, Delpuech C, Pernier J, Giard MH. 2002. Dynamics of cortico-
subcortical cross-modal operations involved in audio-visual object
detection in humans. Cereb Cortex 12:1031--1039.
Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE.
2000. Multisensory auditory-somatosensory interactions in early
Beta Activity Predicts Response Speed
Senkowski and others
cortical processing revealed by high-density electrical mapping. Download full-text
Brain Res Cogn Brain Res 10:77--83.
Foxe JJ, Schroeder CE. 2005. The case for feedforward multisensory
convergenceduring early cortical
Foxe JJ, Simpson GV. 2002. Flow of activation from V1 to frontal cortex
in humans. A framework for defining ‘‘early’’ visual processing. Exp
Brain Res 142:139--150.
Foxe JJ, Simpson GV, Ahlfors SP. 1998. Cued shifts of intermodal
attention: parieto-occipital ~10 Hz activity reflects anticipatory state
of visual attention mechanisms. Neuroreport 9:3929--3933.
Fu KG, Foxe JJ, Murray MM, Higgins BA, Javitt DC, Schroeder CE. 2001.
Attention-dependent suppression of distracter visual input can be
cross-modally cued as indexed by anticipatory parieto-occipital
alpha-band oscillations. Brain Res Cogn Brain Res 12:145--152.
Giard MH, Peronnet F. 1999. Auditory-visual integration during multi-
modal object recognition in humans: a behavioral and electrophys-
iological study. J Cogn Neurosci 11:473--490.
Haenschel C, Baldeweg T, Croft RJ, Whittington M, Gruzelier J. 2000.
Gamma and beta frequency oscillations in response to novel auditory
stimuli: a comparison of human electroencephalogram (EEG) data
with in vitro models. Proc Natl Acad Sci USA 97:7645--7650.
Herrmann CS, Mecklinger A, Pfeifer E. 1999. Gamma responses and ERPs
in a visual classification task. Clin Neurophysiol 110:636--642.
Herrmann CS, Munk MH, Engel AK. 2004. Cognitive functions of gamma-
band activity: memory match and utilization. Trends Cogn Sci 8:
Karakas S, Erzengin OU, Basar E. 2000. The genesis of human event-
related responses explained through the theory of oscillatory neural
assemblies. Neurosci Lett 285:45--48.
Klimesch W. 1999. EEG alpha and theta oscillations reflect cognitive and
memory performance: a review and analysis. Brain Res Brain Res Rev
Klimesch W, Sauseng P, Gerloff C. 2003. Enhancing cognitive perfor-
mance with repetitive transcranial magnetic stimulation at human
individual alpha frequency. Eur J Neurosci 17:1129--1133.
Liang H, Bressler SL, Ding M, Truccolo WA, Nakamura R. 2002.
Synchronized activity in prefrontal cortex during anticipation of
visuomotor processing. Neuroreport 13:2011--2015.
Makinen VT, May PJ, Tiitinen H. 2004. Human auditory event-
related processes in the time-frequency plane. Neuroreport 15:
Mejers LM, Eijkman EG. 1997. Distributions of simple RT with single and
double stimuli. Percept Psychophys 22:41--48.
Miller J. 1982. Divided attention: evidence for coactivation with
redundant signals. Cognit Psychol 14:247--279.
Miller J. 1991. Channel interaction and the redundant-targets effect in
bimodal divided attention. J Exp Psychol Hum Percept Perform
Molholm S, Ritter W, Javitt DC, Foxe JJ. 2004. Multisensory visual-
auditory object recognition in humans: a high-density electrical
mapping study. Cereb Cortex 14:452--465.
Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ. 2002.
Multisensory auditory-visual interactions during early sensory pro-
cessing in humans: a high-density electrical mapping study.Brain Res
Cogn Brain Res 14:115--128.
Murray MM, Foxe JJ, Higgins BA, Javitt DC, Schroeder CE. 2001. Visuo-
spatial neural response interactions in early cortical processing
during a simple reaction time task: a high-density electrical mapping
study. Neuropsychologia 39:828--844.
Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC,
Schroeder CE, Foxe JJ. 2005. Grabbing your ear: rapid auditory-
somatosensory multisensory interactions in low-level sensory
cortices are not constrained by stimulus alignment. Cereb Cortex
Neuper C, Pfurtscheller G. 2001. Event-related dynamics of cortical
rhythms: frequency-specific features and functional correlates. Int J
Patching GR, Quinlan PT. 2004. Cross-modal integration of simple
auditory and visual events. Percept Psychophys 66:131--140.
Pfurtscheller G. 1992. Event-related synchronization (ERS): an electro-
physiological correlate of cortical areas at rest. Electroencephalogr
Clin Neurophysiol 83:62--69.
Pfurtscheller G, Graimann B, Huggins JE, Levine SP, Schuh LA. 2003.
Spatiotemporal patterns of beta desynchronization and gamma
synchronization in corticographic data during self-paced movement.
Clin Neurophysiol 114:1226--1236.
Pfurtscheller G, Lopes da Silva FH. 1999. Event-related EEG/MEG
synchronization and desynchronization: basic principles. Clin Neu-
Pfurtscheller G, Woertz M, Supp G, Lopes da Silva FH. 2003. Early onset
of post-movement beta electroencephalogram synchronization in
the supplementary motor area during self-paced finger movement in
man. Neurosci Lett 339:111--114.
Reuter-Lorenz PA, Nozawa G, Gazzaniga MS, Hughes HC. 1995. Fate of
neglected targets: a chronometric analysis of redundant target
effects in the bisected brain. J Exp Pyschol Hum Percept Perform
Sakowitz O, Quian Quiroga R, Schurmann M, Basar E. 2005. Spatio-
temporal frequency characteristics of intersensory components in
audiovisual evoked potentials. Brain Res Cogn Brain Res 23:316--326.
Sakowitz OW, Quian Quiroga R, Schurmann M, Basar E. 2001. Bisensory
stimulation increases gamma-responses over multiple cortical re-
gions. Brain Res Cogn Brain Res 11:267--279.
Sakowitz OW, Schurmann M, Basar E. 2000. Oscillatory frontal theta
responses are increased upon bisensory stimulation. Clin Neuro-
Schroeder CE, Foxe JJ. 2005. Multisensory contributions to low-level,
‘unisensory’ processing. Curr Opin Neurobiol 15:454--458.
Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC.
2001. Somatosensory input to auditory association cortex in the
macaque monkey. J Neurophysiol 85:1322--1327.
Schroeder CE, Smiley J, Fu KG, McGinnis T, O’Connell MN, Hackett TA.
2003. Anatomical mechanisms and functional implications of multi-
sensory convergence in early cortical processing. Int J Psychophysiol
Schro ¨ ger E, Widmann A. 1998. Speeded responses to audiovisual signal
Senkowski D, Herrmann CS. 2002. Effects of task difficulty on evoked
gamma activity and ERPs in a visual discrimination task. Clin
Senkowski D, Talsma D, Herrmann CS, Woldorff MG. 2005. Multisensory
processing and oscillatory gamma responses: effects of spatial
selective attention. Exp Brain Res 166:411--426.
Talsma D, Woldorff MG. 2005. Selective attention and multisensory
integration: multiple phases of effects on the evoked brain activity.
J Cogn Neurosci 17:1098--1111.
Teder-Sa ¨ leja ¨ rviWA,McDonaldJJ,DiRussoF,HillyardSA.2002.Ananalysis
of audio-visual crossmodal integration by means of event-related
potential (ERP) recordings. Brain Res Cogn Brain Res 14:106--114.
Tzelepi A, Bezerianos T, Bodis-Wollner I. 2000. Functional properties of
sub-bands of oscillatory brain waves to pattern visual stimulation in
man. Clin Neurophysiol 111:259--269.
von Stein A, Rappelsberger P, Sarnthein J, Petsche H. 1999. Synchroni-
zation between temporal and parietal cortex during multimodal
object processing in man. Cereb Cortex 9:137--150.
Cerebral Cortex November 2006, V 16 N 11 1565