Looking for faces: Attention modulates early occipitotemporal object processing.

Looking for faces: Attention modulates early
occipitotemporal object processing
ANDREAS LUESCHOW,a TILMANN SANDER,a,b STEPHAN G. BOEHM,a GUIDO NOLTE,a,b
LUTZ TRAHMS,b and GABRIEL CURIOa
aNeurophysics Group, Department of Neurology, Campus Benjamin Franklin, Charite´-University Medicine, Berlin, Germany
bPhysikalisch-Technische Bundesanstalt, Berlin, Germany
Abstract
Looking for somebody’s face in a crowd is one of themost important examples of visual search. For this goal, attention
has to be directed to awell-defined perceptual category.When this categorically selective process starts is, however, still
unknown. To this end, we used magnetoencephalography (MEG) recorded over right human occipitotemporal cortex
to investigate the time course of attentional modulation of perceptual processes elicited by faces and by houses. The
first face-distinctive MEG response was observed at 160–170ms (M170). Nevertheless, attention did not start to
modulate face processing before 190ms. The first house-distinctive MEG activity was also found around 160–170ms.
However, house processing was not modulated by attention before 280ms (90ms later than face processing). Further
analysis revealed that the attentional modulation of face processing is not due to later, for example, back-propagated
activation of theM170 generator. Rather, subsequent stages of occipitotemporal object processing were modulated in
a category-specific manner and with preferential access to face processing.
Descriptors: Visual attention, Face processing, MEG, M170, Signal-space projection
To select an object in the environment for closer scrutiny, an
observer must engage visual attention, either by selecting a
position of interest in space, or by the selection of relevant object
features. This dichotomous organization (Treisman & Gelade,
1980) of visual attention has a distinct temporal organization, as
revealed in humans by event-related potentials (ERP; Heinze et
al., 1994; Hillyard & Anllo-Vento, 1998; Luck & Ford, 1998;
Mangun, 1995). Spatial attention modulates visual cortical
processing as early as 70–100ms after stimulus onset, whereas for
attention directed to elementary features, such as form, color, or
motion, ERP modulations start at about 150ms (Anllo-Vento,
Luck, & Hillyard, 1998; Harter & Guido, 1980; Torriente,
Valdes-Sosa, Ramirez, & Bobes, 1999), suggesting that feature
selection could occur contingent upon the selection of spatial
position (Anllo-Vento & Hillyard, 1996). These macroscopic
ERP findings have their counterpart in single cell recordings.
Desimone and colleagues showed for the ventral processing
stream in nonhuman primates how attention exerts effects on
different levels of extrastriate visual cortex (Chelazzi, Miller,
Duncan, & Desimone, 1993; Luck, Chelazzi, Hillyard, &
Desimone, 1997; Moran & Desimone, 1985; Reynolds, Paster-
nak, & Desimone, 2000). Analogous effects were shown for
single cells in the dorsal processing stream (Recanzone & Wurtz,
2000; Seidemann & Newsome, 1999; Treue & Maunsell, 1996).
Compared to these early attentional modulations of low-level
visual processing, the process of selection and identification of
complex visual objects is less understood. Face perception can
serve as an efficient tool to investigate this operation in more
detail. Faces are visually complex objects of primary social
importance that require an exquisitely refined identification.
They are processed in a specialized area of the human
occipitotemporal cortex, the fusiform face area (FFA; Kan-
wisher, McDermott, & Chun, 1997; Puce, Allison, Gore, &
McCarthy, 1995; Sergent, Ohta, & MacDonald, 1992), as
shown, for example, by functional magnetic resonance imaging
(fMRI), which maps neuronal activations with high spatial
resolution through the concomitant neurovascular responses.
Specifically, when attention is directed selectively to visual stimuli
from one of two different object categories (e.g., faces or houses),
the fMRI signal increases in that area of the human occipito-
temporal cortex that is specialized for processing of the respective
category, for example, the FFA if attention is directed to faces
(Clark et al., 1996; Haxby et al., 1994; O’Craven, Downing, &
Kanwisher, 1999; Wojciulik, Kanwisher, & Driver, 1998).
The present study extends these findings by exploiting the
high time resolution of magnetoencephalography (MEG) to
determine the earliest latency at which attention influences visual
category-specific responses in human occipitotemporal cortex.
This study has been supported by contracts Ma 1782/1-3,1-4 and
GRK 423/1 from the German Research Association (DFG). We
acknowledge Walter Endl for kind provision of stimulus materials.
Address reprint requests to: Dr. Andreas Lueschow, Neurophysics
Group, Department ofNeurology, Campus Benjamin Franklin, Charite´-
University Medicine, 12200 Berlin, Germany. E-mail: lueschow@zedat.
fu-berlin.de
Psychophysiology, 41 (2004), 350–360. Blackwell Publishing Inc. Printed in the USA.
Copyrightr 2004 Society for Psychophysiological Research
DOI: 10.1111/j.1469-8986.2004.00159.x
350

Specifically, it tested the null hypothesis that attention
modulates the earliest stages in the visual system that are
selectively involved in object processing. In the case of face
processing, likely candidates for such attentional modulation are
the N170 or its magnetic analogon, the M170.
EEG studies have described the N170 (a negativity occurring
bilaterally over occipitotemporal cortex between 150 and 200ms)
as the first face-specific response and it has been suggested that
this evoked component is the electric counterpart of the process
of structural encoding (Bentin, Allison, Puce, Perez, &
McCarthy, 1996; Bentin & Deouell, 2000; Eimer, 1998, 2000b;
Eimer & McCarthy, 1999).
MEG studies using helmet-shaped sensor systems covering
the whole head (Halgren, Raij, Marinkovic, Jousmaki, & Hari,
2000; Ioannides, Liu, Kwapien, Drozdz, & Streit, 2000; Liu,
Higuchi, Marantz, & Kanwisher, 2000; Sams, Hietanen, Hari,
Ilmoniemi, & Lounasmaa, 1997; Swithenby et al., 1998;
Watanabe, Kakigi, Koyama, & Kirino, 1999) have convergingly
identified a deep neural generator at the fusiform gyrus which (1)
is preferentially activated by faces; (2) has a peak response
latency around 170ms, prompting its generic tagging as ‘‘M170’’
generator; (3) can be modeled adequately in most cases as an
equivalent current dipole that generates a characteristic magnetic
field signature, that is, a bipolar field distribution over the
occipitotemporal cortex at the lateral head; and (4) shows a
right4left hemisphere preponderance in several studies.
A recent MEG study (Downing, Liu, & Kanwisher, 2001)
showed a modulation of the M170. Subjects were first cued with
a stimulus (either a face or a house) and then had to decide
whether the cue appeared in a compound stimulus showing a face
and a house superimposed. A critical aspect of such compound
stimuli is that spatial attention could be used to disambiguate the
face and the house at different imaginary depth levels. Thus the
modulation of theM170 could be an effect of spatial attention. A
partial contribution of spatial attention can also not be ruled out
for earlier fMRI studies that used either displays comprising two
peripheral faces and two peripheral houses (all presented
simultaneously; Wojciulik et al., 1998), or stimuli consisting of
a face transparently superimposed on a house (O’Craven et al.,
1999).
To exclusively investigate the effect of object-specific atten-
tion, the present study used a continuous target detection task
where throughout the experimental session only one stimulus at a
time was presented to the subjects, always in the center of the
visual field, thereby approximating natural viewing conditions
during saccadic exploration of the visual environment.
Subjects viewed a random sequence of face and house gray-
scale photographs. In a block design, either a particular face or a
house was the designated target for the following block, and
subjects pressed a button upon appearance of the target (Figure
1). By the explicit instruction to attend to the target stimuli, we
induced our subjects to attend indirectly also to the nontarget
stimuli in the target category as they were naı¨ve with respect to
the main focus of the study, which was the analysis of nontarget
stimuli.
The contrast of nontarget stimuli (attended vs. unattended)
has two advantages: first, it minimizes any possible contribution
ofmotor-related activity to the measured attentional modulation
because no response was to be given to the nontarget stimuli.
Second, it excludes possible contributions of working memory to
the measured attentional modulation because the nontarget
stimuli are not maintained in working memory.
Simultaneously with MEG, EEG was recorded from nine
electrodes referenced against the tip of the nose. This was done
for two reasons: (1) MEG is insensitive for radially oriented
dipoles. In principle, it is possible that a generator of an
attentional modulation has an exclusively radial orientation. We
believe that this is practically of little relevance because already a
slight deviation from a radial orientation should lead to signal,
measurable for MEG. (2) The study should be comparable to
EEG work. For example, a recent EEG study with a similar
design (Eimer, 2000a) shows an attentional modulation for faces
already at 135–180ms poststimulus.
Study design and pilot results have been published before in
abstract form (Lueschow et al., 2000).
Methods
Participants
Ten participants took part (5 women, 5 men; all right-handed
with normal vision; age range: 20–32 years). The first 2
participants who only received MEG recordings had to be
excluded from the MEG data analysis due to excessive artefacts
(muscle and head movements). The remaining 8 participants
received simultaneous MEG/EEG recordings. All participants
were naı¨ve with respect to the experimental purpose. Partici-
pants lay supine on a bed, the head turned to the left side and
fixed using an evacuated cushion. Each participant gave written
informed consent and was compensated for participation. The
MEG protocol had been approved before by the local ethics
committee (Ethics Committee of the Medical Faculty, Free
University Berlin).
Stimuli and Task
The stimulus set consisted of gray tone images of six faces and six
houses that had been digitally scanned. The stimuli were
projected through an aperture into the magetically shielded
room (Vakuumschmelze Ak 3b, Germany) on a white back-
ground. Viewing distance was 0.65m and the stimuli subtended a
visual angle of 241 by 241. Stimulus presentation was controlled
byERTS (Experimental RunTime System, Frankfurt,Germany).
Attention modulates early occipitotemporal processing 351
Figure 1. The two blocks of the attention task. Presentation times are
indicated.

One picture out of each stimulus class served as the target
(Figure 1). The stimuli were shown for 500ms, followed by a
luminance-matched checkerboard mask that stayed on for 1.5 s.
During presentation of the mask, a red fixation square was
present in the center. The stimuli appeared in random order.
Using a block design either a face or a house was the attended
target for the following block. Upon appearance of the target
participants pressed an optical switch.
The experiment consisted of 24 blocks with 60 stimuli each
(25 nontarget face stimuli1 5 target face stimuli1 25 nontarget
house stimuli 1 5 target house stimuli). Each block had a
duration of approximately 2.5min. The blocks were interdigi-
tated such that participants alternately had to respond to the face
target or to the house target. This succession was counter-
balanced across participants. The total number of presentations
of nontargets is 300 for each of the four conditions (face with or
without attention; house with or without attention). At the
beginning of the experiment, participants were familiarized with
the paradigm and the stimulus set by running four blocks of the
experimental session.
Recording
MEG was recorded in a conventional magnetically shielded
room (VAC AK 3b) using a home-made helium cooled SQUID
system consisting of 49 axial first-order gradiometers (70mm
baseline, 2.7 fT/
p
Hzwhite noise level), which were arranged in a
hexagonal lattice of 30mm spacing over a planar area of 210mm
diameter (Drung, 1995). The dewar was positioned tangentially
over the right occipitotemporal cortex, centered over T6
(international 10–20 system). Simultaneously, EEG was re-
corded from T3, T5, O1, Fz, Cz, Pz, T4, T6, and O2 in 8 of 10
participants (reference electrode at the tip of the nose;
impedances below 5kO). MEG/EEG data were recorded with
a bandpass 0.16–200Hz (sampling rate: 500Hz). The data were
low-pass filtered off-line at 20Hz corner frequency. EOG was
measured with two diagonally placed electrodes. Artifact
rejection (blinks, eye movements) was performed automatically
and removed about 15% of the recorded trials.
Data Analysis
Themain analysis was carried out on theMEG responses evoked
by nontarget objects. Accordingly, four stimulus conditions were
compared: responses to nontarget faces when participants
directed attention to the target face (FF), nontarget faces during
attention to the target house (FH), and the corresponding
conditions for nontarget houses (HH,HF). To identify attention-
related effects, evoked responses in the unattended condition
were subtracted from responses in the attended condition (FF �
FH; HH � HF).
Normalized Projection Derived from Signal-Space Projection
In the signal-space projection (SSP) method (Uusitalo &
Ilmoniemi, 1997) all channels are assumed to be elements of a
vector, and the similarity of vectors is measured using the scalar
product. We define a normalized projection p(t), which is derived
from the SSP: p(t)5F(t) � G/|G|2, where F and G are vectors
and � is the scalar product and |G| the standard vector norm. A
measured spatiotemporal pattern is taken as the time-dependent
vector F(t). A time-independent field pattern vector is chosen as
G, for example, themeasured pattern at 170ms in conditionHH.
The calculated function p(t) monitors then the similarity between
the time evolution of the visually evoked fields F(t) (t means it
varies over time) and the M170 pattern; the normalization
p(t)5 1 for F(t5 170ms) enables comparability between differ-
ent participants. Two different, physiologically motivated
choices of G were employed: (1) at the latency when the M170
upstroke was halfway between onset and peak (showing the
M170 pattern fully developed, but with still little contribution
from the upcoming attentional signal to the overall signal; range
in n5 7 participants: 150–200ms), (2) at the M230 peak (range:
190–270ms). Note that the selection of a single/few channels, as
often done in the literature, is a projection, too, operating with a
binary (0/1) channel weighting function.
Results
MEG Recordings
Neuromagnetic fields evoked by face or house stimuli were
recorded with a planar 49-channel MEG system (Drung, 1995).
The planar bottom of theMEG systemwas centered tangentially
at position T6 of the 10–20 EEG system to cover the face
processing areas in the right human occipitotemporal cortex.
This sensor placement allowed us to chart the typical M170
bipolar field signature (cf. Figure 2a).
For both stimulus categories, two early response components
were detected over the right occipitotemporal cortex (Figure 2),
peaking for faces on average at 164ms (M170) and at 228ms
(M230), for houses at 156ms (M160) and at 252ms (M250). The
attention-related difference curve for faces (Figure 2a) peaks
much earlier (208ms) compared to the difference curve for
houses (308ms, Figure 2b). The significance of these attention-
related difference traces was assessed using a running two-sided
t test. For the ensemble of eight individual difference curves,
mean amplitude values were calculated in a sliding window
(width 60ms, overlap 30ms) and compared against the mean
value in a baseline window (� 60 to 0ms; p values indicated by
black squares in Figure 2). The attentional modulation shows a
between-category difference in onset latency of 90ms (faces:
190ms, p5 .029; houses: 280ms, p5 .016; onset latency defined
as center of the first window with po.05). When we included a
Bonferroni correction to the data because of multiple t tests (20
tests in the time range from � 0.1 to 0.5 s), the threshold of
significance fell from .05 to .0025; onset latency is then at 220ms
for face processing and at 310ms for house processing, the
between-category difference being still 90ms.
Differences in difficulty do not account for the latency
difference of the attentional modulation: Mean reaction times do
not differ between categories: 588ms for house targets, and
590ms for face targets (p5 .88; two-tailed paired t test); nor does
the error rate although house targets seem to be detected with
slightly fewer errors: house targets 4.4%; face targets 6.4%;
p5 .2; two-tailed paired t test.
The onset of the attentional modulation of face processing
occurs well after the onset of the M170, identified in earlier
studies as face-specific activity, probably generated in the
fusiform gyrus (Halgren et al., 2000; Ioannides et al., 2000; Liu
et al., 2000; Sams et al., 1997; Swithenby et al., 1998;Watanabe et
al., 1999). Despite this difference in onset latency, attention
might operate on the very same M170 generator, with top-down
activity entering this module simply at a later point in time. This
hypothesis was not confirmed when applying SSP (Uusitalo &
Ilmoniemi, 1997); themore technically oriented reader is referred
to the Methods section.
352 A. Lueschow et al.

The logic of this approach is qualitatively illustrated as
follows: Suppose the attentionalmodulation that onsets at 190ms
is due to later activation of the M170 generator, for example, by
back-projections. In this case, there should be significant if not
total overlap between the neuronal ensemble that constitutes the
M170 generator and the neuronal ensemble that constitutes the
generator of the attentional modulation. The problem with the
difference curves that describe the evolution of the attentional
modulation is that at any point in time, multiple generators
contribute to the measured signal. As a consequence, inspection
of the curve does not give the answer to the question of which
generator contributes significantly and which one does not.
To solve this problem of multiple generators that are
simultaneously active we applied SSP. SSP was used to maximize
selectively the contribution of two obviously separate generators
that underlie the first two evoked components in our data, namely
the M170 and the M230 and to test which one contributes to the
attentional modulation and which one does not.
For the resulting curves, the same running t testwas applied as
for the original difference curves (attended minus unattended
stimuli). The difference curve with maximized contribution from
the M170 (dashed line of Figure 3) shows no statistically
significant modulation whereas the difference curve with
maximized contribution from the M230 (continuous line of
Figure 3) shows a significant modulation around 200 ms. A
straightforward interpretation is that the M230 module does
significantly contribute to the attentional modulation whereas
theM170 does not. In otherwords, attention does not operate on
the generator of the M170 but on subsequent stages of
processing.
The general result for the group is evident by visual inspection
of the field pattern of subject M.H. in Figure 2a, the bipolar field
pattern at the peak of the evoked M170 component has an
oblique orientation obviously different from the horizontal
pattern orientation at the peak of the M230 component.
Analysis of Target Stimuli
The main purpose of this experiment was to study MEG
responses to nontarget objects that are expected to be less
contaminated by motor-related activity and working memory.
The paradigm was thus designed so as to yield a high signal-to-
noise ratio for the nontargets, that is, 300 trials in each of the
conditions (cells) of the 2 � 2 factorial design (FF, FH, HH,
HF). The target stimuli were shown with a frequency of one out
of six (cf. Methods section) which gives 60 trials per condition.
The relevant conditions are called: FFT (face targets attended),
Attention modulates early occipitotemporal processing 353
Figure 2. Attention effects for visual MEG responses recorded over the right occipitotemporal cortex: attention directed to faces
(large panel in a) or houses (large panel in b); grand average from 8 subjects. Conditions with attention on category of the presented
stimulus (green traces: FF and HH) are compared to conditions with attention off the stimulus category (red traces: FH and HF).
Difference curves extract attention-related effects (yellow traces: FF � FH andHH � HF). The black traces (small panel) indicate
the significance level for the yellow difference traces. The pink line represents the level of significance, corrected for multiple
comparisons. Below each panel field maps of a representative subject (M.H.) are shown. The first and the third field maps
correspond to the peaks of the two early evoked components for faces (M170,M230) and houses (M160,M250). The second and the
fourth field maps (labeled att200 and att300) correspond to the peaks of the difference curve for attention to faces or to houses
(yellow traces). Before forming the grand average, for each subject a spatialmean had been calculated for the channel with the largest
M170/M160 and their six neighbors (the number is due to the hexagonal channel arrangement, cf. Methods section). Then, these
originalMEGwaveformes (showingmagnetic field strength in fT) had been normalized for each subject with respect to theM230 of
condition FF.

FHT (face targets unattended), and HHTand HFT for the house
targets, respectively. As expected, due to the lower number of
conditions, conventional averaging gave very poor results with a
low signal-to-noise ratio and no clear indication of an early
attentional effect for face stimuli.
To remove stimulus-unrelated background activity (alpha-
waves and cardiac artifacts), we applied independent component
analysis (ICA). The method can reduce substantially the number
of averages that are needed to obtain a reasonable signal-to-noise
ratio in cognitive MEG data (Sander, Wu¨bbeler, Lueschow,
Curio, & Trahms, 2002).
Figure 4 shows a grand average for 7 subjects. As for
nontargets, two early responses are obtained for both categories,
at 165ms and 236ms for faces (M170, M230) and at 188 and
266ms for houses (M160, M250), which do not systematically
differ from the nontarget responses given the overall lower signal-
to-noise ratio that is evident in Figure 4.
The time course of the attentional modulation for target
stimuli is almost identical to the nontarget stimuli: an early
modulation of face processing that peaks at 196ms (nontarget
stimuli 208ms). As for nontarget stimuli, there is no early
attentional modulation of house processing for target stimuli.
The attentional modulation of house processing shows a broad
peak between 340 and 350ms. The significance of the attention-
related difference traces was evaluated using the same running
t test as for the nontarget data of Figure 2a,b (cf. above). Onset
latency, defined as center of the first window with po.05 is
220ms for face processing, p5 .019, and 310ms for house
processing, p5 .045. For target stimuli of both categories, onset
latency is 30ms longer compared to nontarget stimuli, which is
presumably due to the lower signal-to-noise ratio.
EEG Recordings
Simultaneously to MEG, EEG was recorded from nine
electrodes (cf. Figures 5 and 6 and Methods section) referenced
against the tip of the nose. For faces, two deflections can be seen
at occipital and temporal sites, the first being an initial negativity
with a peak between 140 and 150ms, presumably corresponding
to the well-described N170 (e.g., Bentin et al., 1996). The second
deflection is a broader positivity peaking around 230ms (here
P230). At midline electrodes, three deflections are present: first, a
positivity around 170ms, maximal at Cz, probably the P150
(Botzel &Grusser, 1989; Jeffreys, 1989); second, a negativity with
a peak at 230ms; and third, a broader positivity with a peak at
300ms. No lateralization was evident. For houses, at midline
electrodes similar deflections with similar peak latencies are
obtained as for faces, which indicates that they are caused by
overlapping or identical neural sources. At lateral sites, the picture
for houses is quite different from that for faces. As for faces, an
initial negative deflection with a peak between 140 and 150ms is
present. But in contrast to face stimuli, the positive deflectionwith
a peak at 230ms is lacking; instead a broad negativity with a dip
around 300ms is present. In addition, EEG, like MEG
recordings, are lacking early responses around 100ms (P1).
In comparison to Figures 5 and 6 in which the pink bars
indicate the area where the attention-related difference curves
differ from the baseline at po.0025 (significance threshold
corrected for multiple comparisons) Table 1 shows the onsets of
the attentional modulations for face and house processing
defined as the first window in the running t test with po.05.
354 A. Lueschow et al.
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
0 0.1 0.2 0.3 0.4
M230
M170
P=0.003 ( Bonferroni)
time [s]
p-values
The M170 and the M230 contribute differentially
to the attentional modulation of face processing
Figure 3. Significance of attention-related neuronal activations as a
function of latency poststimulus. Depicted are levels of significance from
running t tests that compare the attention-related difference curves (faces
attended minus faces unattended) against prestimulus baseline. The
continuous trace represents the attention related difference curve with
‘‘maximized’’ contribution of the M230; the dashed line represents the
attention-related difference curve with ‘‘maximized’’ contribution of the
M170. Higher significance indicates that a component is more similar to
the attention-related field pattern, that is, ‘‘it carries more of the
attentional effect’’ observed in the evoked magnetic response with
contribution of all possible components. The horizontal black line
represents the level of significance, corrected for multiple comparisons.
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
1.2
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Responses to face targets (MEG)
p < 0.05
face targets attended (FF )T
face targets unattended (FHT)
difference(FFT-FHT)
n
o
rm
al
iz
ed
v
al
ue
s
n
o
rm
al
iz
ed
v
al
ue
s
time [sec]
time [sec]
-0.1 0.0 0.1 0.2 0.3 0.4 0.5
-0.1 0.0 0.1 0.2 0.3 0.4 0.5
Responses to house targets (MEG)
p < 0.05
house targets attended (HHT)
house targets unattended (HFT)
difference (HHT-HFT)
Figure 4. MEG responses to target stimuli (upper panel: face targets;
lower panel: house targets). Grand average from 7 subjects. For each
subject the same spatial mean was calculated as in Figure 2a,b. The gray
bars mark the intervals when the attention-related difference curves
(black lines) differ from the base line at po.05 (t test).