Investigating local network interactions underlying first- and second-order processing.

Abstract

We compared the spatial lateral interactions for first-order cues to those for second-order cues, and investigated spatial interactions between these two types of cues. We measured the apparent modulation depth of a target Gabor at fixation, in the presence and the absence of horizontally flanking Gabors. The Gabors' gratings were either added to (first-order) or multiplied with (second-order) binary 2-D noise. Apparent "contrast" or modulation depth (i.e., the perceived difference between the high and low luminance regions for the first-order stimulus, or between the high and low contrast regions for the second-order stimulus) was measured with a modulation depth-matching paradigm. For each observer, the first- and second-order Gabors were equated for apparent modulation depth without the flankers. Our results indicate that at the smallest inter-element spacing, the perceived reduction in modulation depth is significantly smaller for the second-order than for the first-order stimuli. Further, lateral interactions operate over shorter distances and the spatial frequency and orientation tuning of the suppression effect are broader for second- than first-order stimuli. Finally, first- and second-order information interact in an asymmetrical fashion; second-order flankers do not reduce the apparent modulation depth of the first-order target, whilst first-order flankers reduce the apparent modulation depth of the second-order target.

Full text

Page 1

Investigating local network interactions underlying
first- and second-order processing
Dave Ellemberga,*, Harriet A. Allenb, Robert F. Hessa
aDepartment of Ophthalmology, McGill Vision Research Unit, McGill University, 687 Pine Ave. West H4-14, Montreal, Que., Canada H3A 1A1
bSchool of Psychology, University of Birmingham, UK
Received 4 September 2003; received in revised form 25 February 2004
Abstract
We compared the spatial lateral interactions for first-order cues to those for second-order cues, and investigated spatial inter-
actions between these two types of cues. We measured the apparent modulation depth of a target Gabor at fixation, in the presence
and the absence of horizontally flanking Gabors. The Gabors’ gratings were either added to (first-order) or multiplied with (second-
order) binary 2-D noise. Apparent ‘‘contrast’’ or modulation depth (i.e., the perceived difference between the high and low lumi-
nance regions for the first-order stimulus, or between the high and low contrast regions for the second-order stimulus) was measured
with a modulation depth-matching paradigm. For each observer, the first- and second-order Gabors were equated for apparent
modulation depth without the flankers. Our results indicate that at the smallest inter-element spacing, the perceived reduction in
modulation depth is significantly smaller for the second-order than for the first-order stimuli. Further, lateral interactions operate
over shorter distances and the spatial frequency and orientation tuning of the suppression effect are broader for second- than first-
order stimuli. Finally, first- and second-order information interact in an asymmetrical fashion; second-order flankers do not reduce
the apparent modulation depth of the first-order target, whilst first-order flankers reduce the apparent modulation depth of the
second-order target.
? 2004 Elsevier Ltd. All rights reserved.
Keywords: Spatial lateral interactions; First-order; Second-order; Apparent contrast; Apparent modulation depth
1. Introduction
Contextual interactions are prevalent in visual per-
ception. The literature from research in psychophysics
provides several examples in which the perception of
aspects of an image is influenced to some extent by the
features of other neighboring local elements or by
characteristics of the ensemble itself. For example, the
local properties of micropatterns are perceptually linked
into salient contours according to a specific set of rules
(Field, Hayes, & Hess, 1993). Texture boundaries are
segregated, in part, by the coding of local element fea-
tures, and by detecting the difference between adjacent
texture regions (Julesz, 1971; Malik & Perona, 1990;
Nothdurft, 1985). The detectability of local elements is
affected by the local characteristics and by the global
configuration of the neighboring stimuli (Polat & Sagi,
1993). The appearance and discriminability of texture
components can be altered by the presence of neigh-
boring elements and their characteristics (Ellemberg,
Wilkinson, Wilson, & Arsenault, 1998; Wilkinson,
Wilson, & Ellemberg, 1997). Although the exact nature
of these lateral interactions remains unfathomed, evi-
dence suggests that there is a general gain control
mechanism that underlies the management of contextual
interactions (Albrecht & Geisler, 1991; Cannon & Ful-
lenkamp, 1996; Ellemberg et al., 1998; Heeger, 1992).
Several psychophysical paradigms were developed to
investigate the influence of lateral interactions on human
visual performance. The most common paradigms
measure perceived contrast and spatial frequency
(Cannon & Fullenkamp, 1991; Ellemberg et al., 1998),
discrimination of contrast and orientation (Wilkinson
et al., 1997), and detection (Polat & Sagi, 1993) of a
central target Gabor as a function of the distance be-
tween the target and flanking Gabors and as a function
of their local characteristics (e.g., spatial frequency,
orientation, contrast). Using a matching task, Ellemberg
et al. (1998) measured a reduction of about 20% in the
*Corresponding author. Tel.: +1-5148421231; fax: +1-5148431691.
E-mail address: dave.ellemberg@staff.mcgill.ca (D. Ellemberg).
0042-6989/$ - see front matter ? 2004 Elsevier Ltd. All rights reserved.
doi:10.1016/j.visres.2004.02.012
Vision Research 44 (2004) 1787–1797
www.elsevier.com/locate/visres

Page 2

perceived contrast and an increase of about 20% in the
perceived spatial frequency of a target Gabor when it is
flanked by a group of Gabor elements that have the
same spatial frequency and orientation. This induction
effect extends up to inter-element spacings of about four
carrier cycles, providing a clear example that the per-
ception of contrast in a localized region of the visual
field can be influenced by the contrast of features located
in the adjacent regions.
The investigation of spatial interactions has mainly
used stimuli that are defined by spatial and temporal
parameters that vary in the luminance domain (first-
order cues). However, it is well documented that the
human visual system is able to detect objects defined by
image attributes other than luminance, such as texture,
in which there is no change in mean luminance (second-
order cues) and that this is the case for both for spatial
vision (Hess, Ledgeway, & Dakin, 2000; McGraw, Levi,
& Whitaker, 1999; Prins & Kingdom, 2003), and for the
perception of motion (Badcock & Derrington, 1985;
Baker & Hess, 1998; Cavanagh & Mather, 1989; Chubb
& Sperling, 1988).
Several lines of evidence suggest that first- and sec-
ond-order stimuli are analysed by different signal pro-
cessing mechanisms. Neurons in the cat’s striate cortex
have different spatial and temporal frequency tuning for
first-order stimuli than they do for second-order stimuli
(Mareschal & Baker, 1998, 1999; Zhou & Baker, 1993).
In humans, visual evoked potential latencies are longer
and psychophysical reaction times are slower for sec-
ond-order than for first-order motion-onset (Ellemberg
et al., 2003a). In young children, sensitivity to second-
order motion develops more slowly than sensitivity to
first-order motion (Ellemberg et al., 2003b), and is more
profoundly degraded by strabismus (Simmers, Ledge-
way, Hess, & McGraw, 2003). Further, neuropsycho-
logical studies report a ‘double dissociation’ in which
lesions in some central areas cause deficits in the per-
ception of second-order motion while relatively sparing
first-order motion (Plant & Nakayama, 1993; Vaina &
Cowey, 1996), and lesions in other central areas cause
deficits in the perception of first-order motion with little
effect on second-order motion (Vaina, Makris, Ken-
nedy, & Cowey, 1998; Vaina, Soloviev, Bienfang, &
Cowey, 2000). Finally, a functional magnetic resonance
imaging study indicates that, although some visual areas
(including V3A, and HMT+) respond equally to both
first- and second-order motion, area V1 responds
more strongly to first-order, while the lateral occipital
area responds more strongly to second-order motion
(Dumoulin, Baker, Hess, & Evans, 2003; also see Smith,
Greenlee, Singh, Kraemer, & Hennig, 1998).
Computational modeling suggests that the detection
of second-order images requires not only a first stage of
linear filtering but also additional processing steps
(Chubb & Sperling, 1988, 1989; Wilson, Ferrara, & Yo,
1992). Baker and colleagues found evidence from single-
cell recording studies in cats that is consistent with this
additional processing (Mareschal & Baker, 1998, 1999;
Zhou & Baker, 1993; for a review see Baker, 1999). They
recorded responses from neurons to luminance gratings
(first-order) and to contrast envelope gratings (second-
order) which were created by multiplying a static high
spatial frequency sinusoidal grating (carrier) with a
drifting low spatial frequency sinusoidal grating (enve-
lope). They found that neurons are tuned to a narrow
range of spatial frequencies that is much higher for the
second-order carrier than for the first-order luminance
grating. Further, in these same neurons, the preferred
range of spatial frequency is lower for the second-order
contrast envelope than for the first-order luminance
grating. These data support a ‘filter–rectify–filter’ model,
in which an early linear filtering occurs when neurons
that are sensitive to high spatial frequencies respond to
the carrier grating (but see Baloch, Grossberg, Mingolla,
& Nogueira, 1999; Johnston & Clifford, 1995). This
is followed by a non-linear processing stage (e.g.,
full-wave, half-wave rectification, or squaring) that
introduces first-order characteristics into the neural
representation of the second-order image, and a second
stage filtering by neurons that are sensitive to lower
spatial frequencies. This processing scheme by itself
cannot account for responses to luminance gratings be-
cause the spatial frequency tuning of the early and late
filters do not overlap. Recently, Prins and Kingdom
(2003) provided evidence that in the human observer the
perception of texture discontinuities is mediated by such
a ‘filter–rectify–filter’ mechanism. Sensitivity to the sec-
ond-order component of a texture (i.e., its orientation
and frequency modulation) composed of densely packed
Gabor elements is decreased by the previous adaptation
to afirst-order gratingthat matched the characteristics of
the first-order signal in the stimulus. This is consistent
with other reports suggesting that distinct first- and
second order mechanisms underlie spatial vision in
humans (Graham & Sutter, 1996, 1998; Lin & Wilson,
1996; Schofield & Georgeson, 1999).
Very little is known about the nature of the cortical
interactions mediating the perception of second-order
images. However, there is some evidence that the neural
substrate underlying first- versus second-order percep-
tual interactions operates differently. Contour integra-
tion and motion trajectory detection, two aspects of
visual perception believed to implicate long range intra-
cortical connections, are easily demonstrated for first-
order stimuli but are absent for second-order stimuli
(Hess et al., 2000). The goal of the present series of
experiments was to study the nature of second-order
lateral interactions. We ask whether localized second-
order images are subjected to the same kind of lateral
interactions as those previously found for first-order
images. If so, is the origin of these interactions at the
1788
D. Ellemberg et al. / Vision Research 44 (2004) 1787–1797

Page 3

level of early first-order filtering (indicated by C in Fig.
1), after rectification of the output of the early filters
(indicated by B in Fig. 1), or at the level of the second-
stage of filtering in the filter–rectify–filter scheme (indi-
cated by A in Fig. 1). To answer these questions we
compared the apparent ‘‘contrast’’ of first- versus sec-
ond-order Gabor stimuli as a function of the spatial
extent of flanking elements and as a function of the local
properties of the surrounding elements.
2. Methods
2.1. Observers
Two of the authors and two experienced observers,
who were unaware of the issues examined, participated
in this study. Each participated to all conditions, except
for HAA who was not available to contribute to the
spatial frequency and orientation tuning data. Two had
normal acuity and the others had corrected to normal
acuity.
2.2. Stimuli and apparatus
The stimuli consisted of a horizontal array of spa-
tially localized 2-D Gabors. A Gabor is a sinusoidal
modulation of luminance multiplied by a Gaussian
envelope in the horizontal and orthogonal dimensions.
The first-order stimuli were created by adding the
sinusoidal component of a Gabor to 2-D noise that was
binary. This stimulus is represented by the following
equation:
Gðx;yÞ ¼ Lmeanþ LmeanðGcosð2pfxÞ þ RCÞ
? expð?x2=r2
where Lmeanis the mean luminance of the pattern, f is
the spatial frequency of the sinusoidal modulation, G is
the contrast of the grating, R is the random carrier
(having contrast C), and ðrxÞ and ðryÞ are vertical and
horizontal space constants, respectively.
Unless mentioned otherwise, the orientation of the
Gabor’s grating component was vertical and its peak
spatial frequency was 3 cpd (therefore, k ¼ r ¼ 20 arc-
min). The displayed horizontal and vertical spread of
each Gabor was 0.58?. The noise carrier had a contrast
of 50% and each noise element was 1.9·1.9 arcmin.
The second-order stimuli were created by multiplying
the sinusoidal component of a Gabor by 2-D noise that
was binary. This produced Gabors with an internal
sinusoidal structure that varied in contrast and had a
mean luminance that was constant across different re-
gions of the pattern. The geometry of the second-order
stimulus is represented in the following equation:
xÞ ? expð?y2=r2
yÞð1Þ
Gðx;yÞ ¼ Lmeanð1 þ ðRðM cosð2pfxÞ þ 1Þ
? expð?x2=r2
where M is the modulation depth of the sinusoidal
component and all other parameters are the same as
indicated above. We created our second-order stimuli so
that any change in modulation depth varied both the
high and low contrast parts of the pattern. Fig. 2 pro-
vides an example of the first- and second-order stimulus
arrays.
The stimuli and presentation routine were pro-
grammed with the MatlabTMPsychophysics Toolbox
routine (Brainard, 1997; Pelli, 1997). The experiments
were run on a Mactintosh G4 computer and the images
were displayed on a monitor that had a frame rate of 75
Hz and a resolution of 1152·870 pixels. The display
had a mean luminance of 34 cdm?2. The relationship
between voltage and screen luminance was measured
with a photometer. The Gabors were produced with a
subset of achromatic luminance values that were or-
dered linearly, thus correcting for the monitor’s gamma
non-linearity.
xÞ ? expð?y2=r2
yÞC=2ÞÞð2Þ
2.3. Procedure
Using both eyes, observers viewed the display from a
distance of 57 cm. At the beginning of each trial,
observers were instructed to fixate a cross at the cen-
tre of a uniformly illuminated screen. The apparent
A
B
C
Early
Filtering
Rectification
Late
Filtering
Fig. 1. Schematic representation of the origin of the lateral inhibitory
signal in the filter–rectify–filter scheme hypothesized to process second-
order information: (A) lateral inhibitory signal that operates after the
envelope filtering stage; (B) lateral inhibitory signal that operates after
rectification of the carrier signal and (C) lateral inhibitory signal that
originates after the initial carrier filtering stage.
D. Ellemberg et al. / Vision Research 44 (2004) 1787–1797
1789

Page 4

‘‘contrast’’ or modulation depth (i.e., the perceived dif-
ference between the high and low luminance regions of
the first-order stimulus, or between the high and low
contrast regions of the second-order stimulus) of a
foveated first- or second-order Gabor was measured
using a temporal two-alternative forced-choice (2-AFC)
procedure and the method of constant stimuli. The po-
sition of the target Gabor embedded in the array was
indicated by a thin, low contrast marker positioned
0.30? above it that was presented only between trials and
not when the stimulus was displayed. On each trial, the
modulation depth of the central Gabor in the array was
compared to that of a single Gabor appearing in the
same spatial location but in the other temporal interval.
For each experimental run, five stimulus values (mod-
ulation depth of the single ‘‘comparison’’ Gabor) were
pre-selected to span the observer’s psychometric func-
tion. Twenty-five trials were run for each test value and
each observer completed three runs of 125 trials for each
condition. Each stimulus was presented for 200 ms,
separated by a 500 ms interval during which the screen
returned to mean luminance. Each interval was indi-
cated by a tone. For each trial, observers were asked to
indicate which interval contained the central Gabor with
the highest ‘‘contrast’’. For comparison, and to obtain a
baseline we also measured the apparent ‘‘contrast’’ of an
isolated Gabor pattern. For each condition, the point of
subjective equality was determined from the 50% prob-
ability level estimated from data that were fitted with a
cumulative normal function.
2.3.1. Equating the visibility of the first- and second-order
Gabors
For each observer, the visibility of first- and second-
order Gabor targets, without flankers, was equated
using the matching paradigm described above. One
interval, chosen at random, contained a second-order
Gabor with a modulation depth of 60% and the other
interval contained a first-order Gabor at one of five pre-
selected modulation depths, to span the observer’s psy-
chometric function. Each observer was instructed to
identify which interval contained the stimulus with the
highest ‘‘contrast’’, or in other words, which had the
most visible spatial structure. On the remaining experi-
mental conditions each observer was tested at the
modulation depth of 60% for the second-order Gabors
and at the modulation depth that corresponded to the
match in perceived visibility for first-order Gabors.
Therefore, for first-order Gabors, observers DE, HAA,
HD, and SG were tested at modulation depths of 32%,
29%, 38%, and 28%, respectively.
2.3.2. Experimental conditions
The apparent ‘‘contrast’’ of the central first-order and
second-order targets was measured on separate runs as a
function of the following experimental parameters.
(a) Apparent ‘‘contrast’’ of a target as a function of
inter-element distance was measured in separate runs for
a first-order Gabor surround by first-order flanks and
for a second-order Gabor surrounded by second-order
flanks. The stimulus array consisted of a foveated Gabor
that was flanked laterally by a single Gabor on each
side. Inter-element spacing ranged from 1.5 to 6 cycles
from the centre of the target to the centre of either of the
flankers. In this case a cycle is calculated from the peak
spatial frequency of the Gabor’s grating (i.e., 20 arcmin
per cycle for a peak spatial frequency of 3 cpd). There-
fore, at the smallest inter-element spacing of 1.5 cycles
the flankers abutted the target. Because the Gaussian
spread of each Gabor was truncated at 2r, there was
no overlap between the Gabors at the smallest inter-
element spacing.
(b) The apparent ‘‘contrast’’ of a central Gabor was
measured as a function of the orientation and the spatial
frequency of the flankers. For both conditions this was
tested at the smallest inter-element spacing (1.5 cycles).
(c) The apparent ‘‘contrast’’ of a first-order target was
measured when flanked by second-order Gabors, and
that apparent ‘‘contrast’’ of a second-order target was
measured when flanked by first-order Gabors. This was
also tested at the smallest inter-element spacing (1.5
cycles).
Fig. 2. Schematic representation of the stimuli. The top array presents
the first-order Gabors, the middle presents the second-order Gabors
and the bottom array provides an example of a second-order Gabor
flanked by first-order Gabors.
1790
D. Ellemberg et al. / Vision Research 44 (2004) 1787–1797

Page 5

3. Results
Fig. 3 presents the effect of lateral flanking Gabors on
the apparent ‘‘contrast’’ of a central Gabor for the first-
(filled circles) and second-order (circles) conditions as a
function of inter-element spacing, for each observer. To
compare the findings for the first- and second-order
conditions the data were normalized and are presented
in this and the following figures as the percentage
reduction from baseline (i.e., apparent ‘‘contrast’’ of the
isolated target (baseline) minus the apparent ‘‘contrast’’
of the target in the array, divided by the baseline). The
first point indicated by the data is that the reduction in
apparent ‘‘contrast’’ extends over a further distance for
the first-order condition than for the second-order
condition. For the second-order condition, the induc-
tion effect breaks down at a spacing of approximately
2.5 cycles for three of the four observers, whilst for the
first-order condition it breaks down between twice (DE
and HAA) or three times (SG) that distance. For ob-
server HD the induction effect breaks down at a distance
of 3.5 cycles for second-order, less than half of the dis-
tance found for first-order (5.5 cycles). The second point
indicated by the data is that at the smallest inter-element
distance, for three of the four observers (DE, SG, and
HAA) the reduction in ‘‘contrast’’ is greater for the first-
order stimuli than for the second-order stimuli.
Fig. 4A plots the apparent ‘‘contrast’’ of the target
Gabor as a function of the spatial frequency of the
flankers for the first-order (filled circles) and second-
order conditions (circles). The spatial frequency of the
target was always 3 cpd. The spatial frequency of the
surround varied in half octave steps from one octave
above to one octave below the target’s spatial frequency.
The pattern of results is similar across the three
observers. For both the first- and second-order condi-
tions, apparent ‘‘contrast’’ is most reduced when the
surround and target have the same spatial frequency,
and least reduced when the spatial frequency of the
surround is one octave away from that of the target.
When the surround spatial frequency is one octave away
from that of the target, the induction effect is stronger
for the second-order condition than for the first-order
condition. Fig. 4B presents the shift in apparent ‘‘con-
trast’’ for the first- and second-order target Gabors on
the same scale by plotting the ratio of the percent
Fig. 3. The percent reduction in apparent ‘‘contrast’’ of the central target for the first-order stimuli (filled circles) and second-order stimuli (circles) as
a function of the distance between the target and the flankers. Each graph presents the data of a different participant. Error bars indicate ±1 S.E.
D. Ellemberg et al. / Vision Research 44 (2004) 1787–1797
1791