Linear systems analysis of the fMRI signal.

Abstract

In 1995 when we began our investigations of the human visual system using fMRI, little was known about the temporal properties of the fMRI signal. Before we felt comfortable making quantitative estimates of neuronal responses with this new technique, we decided to first conduct a basic study of how the time-course of the fMRI response varied with stimulus timing and strength. The results ended up showing strong evidence that to a first approximation the hemodynamic transformation was linear in time. This was both important and remarkable: important because nearly all fMRI data analysis techniques assume or require linearity, and remarkable because the physiological basis of the hemodynamic transformation is so complex that we still have a far from complete understanding of it. In this paper, we provide highlights of the results of our original paper supporting the linear transform hypothesis. A reanalysis of the original data provides some interesting new insights into the published results. We also provide a detailed appendix describing of the properties and predictions of a linear system in time in the context of the transformation between neuronal responses and the BOLD signal.

Full text

Page 1

Review
Linear systems analysis of the fMRI signal
Geoffrey M. Boyntona,⁎, Stephen A. Engelb,1, David J. Heegerc,2
aDepartment of Psychology, University of Washington, P.O. Box 351525, Seattle, WA 98195-1525, USA
bDepartment of Psychology, University of Minnesota, N218 Elliot Hall, 75 East River Road, Minneapolis, MN, USA
cDepartment of Psychology and Center for Neural Science, New York University, 6 Washington Place, New York, NY, 10003, USA
a b s t r a c ta r t i c l ei n f o
Article history:
Received 15 December 2011
Revised 12 January 2012
Accepted 16 January 2012
Available online xxxx
Keywords:
fMRI
Linearity
V1
Hemodynamics
In 1995 when we began our investigations of the human visual system using fMRI, little was known about the
temporal properties of the fMRI signal. Before we felt comfortable making quantitative estimates of neuronal
responses with this new technique, we decided to first conduct a basic study of how the time-course of the
fMRI response varied with stimulus timing and strength. The results ended up showing strong evidence
that to a first approximation the hemodynamic transformation was linear in time. This was both important
and remarkable: important because nearly all fMRI data analysis techniques assume or require linearity,
and remarkable because the physiological basis of the hemodynamic transformation is so complex that we
still have a far from complete understanding of it. In this paper, we provide highlights of the results of our
original paper supporting the linear transform hypothesis. A reanalysis of the original data provides some in-
teresting new insights into the published results. We also provide a detailed appendix describing of the prop-
erties and predictions of a linear system in time in the context of the transformation between neuronal
responses and the BOLD signal.
© 2012 Elsevier Inc. All rights reserved.
Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Why primary visual cortex?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Time domain study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Scaling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Additivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Parameterized fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Frequency domain study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Parameterized fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Impact. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Additivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Shift invariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Frequency response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Introduction
Our paper “Linear Systems Analysis of Functional Magnetic Reso-
nance Imaging in Human V1” published in the Journal of Neurosci-
ence in 1996 (Boynton et al., 1996) began as a pilot study. Our
primary interest was to use the new technique of fMRI to study the
neuronal response properties in the human primary visual cortex
NeuroImage xxx (2012) xxx–xxx
⁎ Corresponding author. Fax: +1 206 685 3157.
E-mail addresses: gboynton@uw.edu (G.M. Boynton), engel@umn.edu (S.A. Engel),
david.heeger@nyu.edu (D.J. Heeger).
1Fax: +1 612 626 2079.
2Fax: +1 212 995 4981.
YNIMG-09147; No. of pages: 10; 4C:
1053-8119/$ – see front matter © 2012 Elsevier Inc. All rights reserved.
doi:10.1016/j.neuroimage.2012.01.082
Contents lists available at SciVerse ScienceDirect
NeuroImage
journal homepage: www.elsevier.com/locate/ynimg
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 2

across a range of stimulus conditions. We were inspired by the work
of animal physiologists who make quantitative measurements of neu-
ronal responses to visual stimuli that vary along dimensions such as
contrast, spatial frequency and orientation. However, at that time
there were very few fMRI studies that used parametric stimulus ma-
nipulations to obtain quantitative measurements. Instead, fMRI stud-
ies typically reported binary decisions about whether statistically
significant responses were found in various brain regions or voxels
(e.g. statistical parameter maps). It occurred to us that before we
could get started on a quantitative investigation of the human visual
system, we first had to conduct an investigation into the nature of
the fMRI signal itself.
What we were hoping for, and were generally able to support, was
a linear transform model for the time-course of the hemodynamic re-
sponse (Fig. 1). The linear transform model wasn't a new idea. In fact,
it had already been assumed in previous papers proposing methods
for analyzing fMRI data (Bandettini et al., 1993; Friston et al., 1995).
Surprisingly, however, no test of linearity had been conducted to
date.
The linear transform model posits that the time-course of the fMRI
response can be predicted by convolving the time-course of the neu-
ronal response with a hemodynamic impulse response, followed by
additive Gaussian noise. For those less familiar with the definition
and properties of a linear system, we have provided a brief descrip-
tion in Appendix A.
Why primary visual cortex?
We focused on the primary visual cortex (V1), for a few reasons.
First, since feasible applications of retinotopic mapping procedures
were still being worked out (in our lab and others), we needed a re-
gion of interest that we could find based on structural information.
Fortunately for us, V1 lies consistently within the calcarine sulcus.
Knowing where to look was particularly important because at that
time we only had a single slice (!) of functional data.
Second, we wanted to study an area where we had some under-
standing of what the neuronal responses might be to a given stimu-
lus. A true test of the linear model should compare the relation
between the neuronal responses and the fMRI responses, but of
course we were not directly manipulating the neuronal response. Of
all areas in the mammalian brain, V1 is perhaps the most studied
and well understood. Most important for our purposes, it is well
known that nearly all V1 neurons increase their firing rate monoton-
ically with stimulus contrast (Albrecht and Hamilton, 1982). We
could therefore be reasonably confident that the mean firing rate of
neurons in V1 increases monotonically with stimulus contrast. Note
that we did not assume that the visual system itself was linear: we in-
cluded a nonlinear contrast–response function in the linear transform
model. We did, however, assume that when a stimulus was presented
continuously with a constant contrast that the neuronal responses
would likewise be constant over time. That is known to be incorrect
because of nonlinearities in neuronal activity including adaptation,
something which we dealt with post-hoc in the original paper, and
which we discuss further below.
Third, we wanted robust reliable data, and we knew even then
that fMRI responses in V1 are stronger and more reliable than most
other brain areas — probably due to the relative homogeneity of the
neuronal population and to the experimenter's control over the
time-course of the input to that area.
We designed our experiments to test the linear transform model
in both the time domain and in the frequency domain (See
Appendix A for an explanation of time and frequency domains). The
general procedure was to manipulate the intensity and duration of
the visual stimulus and measure the time-courses and amplitudes of
the fMRI responses in V1. We used two kinds of analyses to test the
linear model hypothesis. The first was a non-parametric investigation
of the data to see if the responses were consistent with some of the
properties of linearity. The second was a brute-force parameterized
model fit to the data. Both of these tests were applied to two sets of
experiments — one emphasizing the time domain and the other em-
phasizing the frequency domain.
Time domain study
We first tested linearity in the time domain using what is now
called a slow event-related design. We simply measured the time-
course of the fMRI response to flickering checkerboards of different
contrasts and durations. We waited 30 s or so between repeated trials
to let the hemodynamics recover to baseline. After acquiring several
runs across several days, we averaged the time-course of each re-
sponse from voxels within a hand-drawn region of interest, time-
locked to the onset of each stimulus.
Scaling
If the hemodynamic transfer function is linear, then the time-
courses of the fMRI response to stimuli for a given duration should
vary only by a scale factor with contrast. This follows from the scal-
ability property of a linear system. Note that we are assuming that
the time-course of the neuronal response follows the time-course of
Fig. 1. The linear transform model.
2
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 3

the stimulus (at least as far as the time-scale of fMRI is concerned).
This means that even though the magnitude neuronal response
does not scale in proportion to stimulus contrast, the shape of the he-
modynamic responses should remain the same across contrasts.
The results were surprisingly clear. The response magnitudes
scaled nicely across the three contrasts (25, 50 and 100%). Fig. 2
shows an example response from one of the subjects to a 12 s stimu-
lus. Note how the shape of the time-course did not vary — it only
scaled with contrast. (Note how these results compare to the predic-
tions of the linear model shown in Appendix A, Fig. A1).
We called this less-than-perfect scaling time–contrast separabili-
ty. From a statistical sense it can be thought of as time–contrast inde-
pendence, in which there is no interaction between the effects of the
independent variables of time and contrast. Figs. 10 and 11 in the
original paper showed how scalability held up for the four durations
tested (3, 6, and 24 s) for the two subjects.
It is important to point out that we did not find (or expect to find)
that the BOLD signal would scale directly with stimulus contrast.
Most biological systems show compression for large inputs, so that
doubling the strength of the input produces much less than doubling
of the output. This is true for the sensory systems and for contrast in
particular. It is clearly visible in Fig. 2 (and Fig. A1): Doubling the con-
trast from 50% to 100% produced a scaling of the output, but the scal-
ing was much less than a factor of two. We know that this is true of
the underlying neuronal responses so it is not surprising that it is
inherited by the hemodynamic responses. But this was not a violation
of the linear transformation model, because it is still possible that the
BOLD signal is scaling with the underlying (compressed) neuronal re-
sponse. Time–contrast separability was consistent with a linear sys-
tem relating the neuronal and hemodynamic responses that was
preceded by a nonlinear relationship between stimulus strength and
neuronal response. This nonlinear component is termed the con-
trast–response function, and was easily inferred from our data by
plotting the stimulus contrast against the magnitude of the corre-
sponding fMRI response.
Time–contrast separability, or more generally time–stimulus
separability, is critical for allowing meaningful data analysis. Sup-
pose, for example, that the vascular system reacted more quickly
to a stronger neuronal input, leading to a differently shaped hemo-
dynamic response for high contrast stimuli. This would be a disas-
ter for data analysis since we wouldn't be able to generate a single
template to correlate with our data. We'd be stuck in a circular di-
lemma in which we'd need to know the strength of the neuronal
response to generate the appropriate template time-course which
would then be used to detect and estimate the underlying
neuronal response.
Additivity
We next tested the property of additivity with the same time-
domain data by, for example, treating a 12 second stimulus as
two successive 6 second stimuli. If additivity and shift invariance
hold, then we should be able to predict the response to the 12 sec-
ond stimulus by adding two responses to the 6 second stimulus,
with the second response shifted by 6 s. Fig. 3(A) shows an exam-
ple of this from one of the subjects. Shown in red is the average
time course to a 12-second stimulus. The blue curve was generated
by adding the response to the 6-econd stimulus to a shifted copy
of itself. The red and blue boxes on the time-line represent the
time-courses of the stimuli.
The curvesoverlap. This means thatthe sum of theresponses to two
successive6-secondstimuliwasequaltotheresponsetothesumoftwo
successive 6-second stimuli. That is, the results were consistent with
the principal of additivity (compare these results with the predictions
from the linear transform model shown in Appendix A, Fig. A2).
Since we used 3, 6, 12, and 24-second stimuli we were able to
make a total of 6 predictions (e.g. a 24 second response was predicted
by shifting and adding six 3-second responses, etc.). Most of the pre-
dictions matched nicely, like the example in Fig. 3(A).
Fig. 3(B) shows what happened when a 3-second responses was
used to predict the same 12-second response. The fit wasn't as
good. The shifted and summed 3-second prediction was too large
compared to actual response to the 12-second stimulus. This violation
of the additivity property was also evident in our parametric fit to the
data, which will be described below.
Like scaling, it is not hard to imagine how additivity could have
failed for a complex phenomenon like the hemodynamic response,
especially considering that the vascular response provides an over-
supply of oxygenated blood. Suppose, for example, that once a re-
sponse to the first stimulus is underway, the vascular response to
the second stimulus is not needed, so little additional hemodynamic
response is added. The disproportionately large response to the 3-
second stimulus might mean that this may actually be the case to
some extent. Or it might simply reflect a nonlinearity (such as adap-
tation) in the neuronal response.
Parameterized fit
So far we have shown support for scaling and additivity using
parameter-free analyses of the data. Given this support, we went
ahead and fit the entire time-domain data sets with a parameterized
version of the linear transform model. The model had two separate
parts. The first was a parameterized contrast–response function,
and the second was a parameterization of the impulse response
function.
The form for the contrast response function was borrowed from
the single-unit electrophysiology (Albrecht and Hamilton, 1982)
and psychophysics (Legge and Foley, 1980) literature. The function
predicts the relation between stimulus contrast c, and neuronal re-
sponse r as:
r c ð Þ ¼
acp
cpþ σ
ð1Þ
This function, called a hyperbolic ratio function or sometimes the
Naka-Rushton equation, predicts a decelerating or compressive re-
sponse as a function of contrast, as is typically seen in single-unit
data from the macaque primary visual cortex (Albrecht and Geisler,
1991).
Fig. 2. Example of time–contrast separability. The shape of the response to a 12 second
stimulus does not vary across the three stimulus contrasts.
3
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 4

For the impulse response function (later called the hemodynamic
response function (Friston et al., 1998)) we chose the Gamma
distribution function (not to be confused with the regular Gamma
function, Γ, which is an extension of the factorial function for non-
integers).
h t ð Þ ¼
1=τ
ðÞn−1
τ n−1
ð
ðÞe1=τ
Þ!
ðÞ
ð2Þ
The Gamma distribution function has two parameters, n (an in-
teger) and a time-constant, τ. Gamma distribution functions often
shows up in the context of linear systems. Perhaps the simplest
linear system is one with an exponential impulse response func-
tion, which describes any self-equilibrating system that changes
over time at a rate proportional to the difference between the cur-
rent state and the asymptotic state. An RC circuit is an example of
a linear system with an exponential impulse response function.
The Gamma distribution function is the impulse response function
for a cascade of n such exponential systems — that is a series of
exponential systems that feed in to the next. Gamma distribution
impulse response functions therefore show up in all sorts of
models, such as those describing fluid dynamics, heat conduction
and membrane potentials.
The first author became aware of the Gamma distribution func-
tions in a linear stage of a model for the temporal properties of the
human visual system (Watson, 1986). The interested reader should
visit this chapter in the Handbook of Perception and Human Perfor-
mance to see how closely our paper follows Watson's, 1986 working
model of the temporal sensitivity of the human visual system. We
essentially borrowed Watson's analysis and applied it at the scale of
seconds rather than milliseconds.
The full model predicts the fMRI response as the time course of the
stimulus convolved with the Gamma distribution function, scaled by
the contrast response function. Time–contrast separability is a simple
consequence of the two separate parameterizations for time and con-
trast. The full parameterized model has a total of five free parameters:
two for contrast, two for the Gamma distribution function, and one
more describing a delay in the impulse response function. This five
parameter model was fit to the fMRI time courses for the three con-
trasts and four durations using the nonlinear optimization algorithm
provided by MATLAB.
The model fit the data from the time-domain study well for both
subjects, except that it systematically underestimated the magnitude
of the fMRI response for the stimuli having shorter durations. This can
be seen in the original paper (Figs. 10 and 11) where the data points
exceed the predicted model curves for the 3-second stimulus. This ap-
parent nonlinearity for brief stimuli was noted above, and has since
been replicated in both the visual system (Vazquez and Noll, 1998;
Birn et al., 2001) and the auditory system (Robson et al., 1998). The
nonlinearity can be quite large; Birn et al. (2001) later found that
the fMRI response to a 250 ms visual stimuli can be 3–5 times larger
than what is expected from a longer stimulus. This has important im-
plications for the interpretation and analysis of rapid event-related
fMRI studies that typically use brief stimulus events.
In the original paper we speculated about the possible causes of
the relatively large fMRI responses we found for short stimuli. One
possibility is a nonlinearity in the function relating neuronal activity
to hemodynamic response. But it could also be a purely neuronal ef-
fect. For example, transient bursts of neuronal firing occur at the
Fig. 4. Reanalysis of the time-domain results. The amplitude parameters were allowed to vary for each of the four durations, showing how the fMRI responses to shorter stimuli are
relatively stronger than the responses to longer stimuli.
Fig. 3. Example of additivity. (A) Response to a 12 second stimulus predicted by the responses to two 6-second stimuli. (B) Response to the same 12 second stimulus predicted by
the responses to four 3-second stimuli.
4
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 5

onset and offset of neuronal responses (Albrecht et al., 1984; Maddess
et al., 1988; Bonds, 1991). Linear convolution with a typical hemody-
namic impulse response function predicts that these transient bursts
would have a relatively large influence on brief stimuli because they
contribute relatively more to the average response over time.
In a subsequent paper, the first author teamed up with the Ban-
dettini lab at NIH and tested the transient hypothesis by replicating
the original fMRI result using stimuli of durations 1, 3 and 6 s, but
also adding a gradual contrast ramp that is was intended to reduce
the strength of the onset and offset neuronal transients. A parallel
MEG study with the same subjects and stimuli showed that the
ramps did effectively reduce the transient neuronal activity. Howev-
er, the nonlinearity in the BOLD signal remained (Tuan et al., 2008).
A related neuronal explanation is long-term adaptation. It is well
known that V1 neurons reduce their responses over time to long-
duration stimuli (e.g. Bonds, 1991). Our flickering checkerboards,
lasting up to 24 s, are exactly the sort of stimuli used in adaptation ex-
periments. In the original paper, we showed that a better fit was
obtained by incorporating a model of neuronal adaptation.
A reanalysis of our original data provides further support for this
idea. We have re-fit the original data with the linear transform
model, but allowed four different amplitudes (the parameter a in
Eq. (1) above) to vary for the four different stimulus durations
while the other parameters of the modified model were held fixed
to the original best-fitting parameters. Fig. 4 shows a plot of the
best-fitting amplitude parameters for each of the four stimulus dura-
tions for the two subjects. Error bars represent bootstrapped esti-
mates of parameter variability based on resampling using the
standard errors of the original data.
The reanalysis shows something not apparent in the original fig-
ures; the best-fitting amplitude parameters decrease across the entire
range of stimulus durations. The effect seems to decay with duration
in an exponential fashion and a time constant of seconds, which is re-
markably consistent with the time-course of adaptation seen in elec-
trophysiology (Bonds,1991)and
(Greenlee et al., 1991). For a similar analysis of the nonlinearity
using event-related designs, see (Heckman et al., 2007).
psychophysical experiments
Frequency domain study
As described in Appendix A, another way to investigate linearity is
in the frequency domain using sinusoidally modulating stimuli. Ideal-
ly we would have liked to invoke a sinusoidal neuronal response, but
this was not possible without knowing the exact relation between
stimulus contrast and neuronal response. Instead we chose to use a
blocked-design in which the stimuli turned on and off as a square
wave over time. The actual stimulus was a square-wave in space (ver-
tically oriented bars of flickering checkerboards) that slowly drifted
horizontally at different rates. The time-course of stimulation at any
given point in space therefore turned on and off at a rate related to
the speed of the drift. We chose this design over a full-field on/off
stimulus because we were trying to minimize the effects of blood ves-
sels that pool across large regions of cortex. The overall stimulation
across the visual field in our design remained roughly constant, unlike
the case for a full-field stimulus. It is interesting to note that the ef-
fects of draining veins on fMRI signals was a major concern back
then, but receives relatively little attention today — yet the problem
remains. Indeed, a recent study supports the idea that discrepancies
in the literature on retinotopic maps in visual area V4 may be due
an artifact caused by the transverse sinus that passes near V4 in
some subjects (Winawer et al., 2010).
In our study, we measured fMRI responses to drifting bars with
temporal periods of 10, 15, 30 and 45 s using five contrasts ranging
from about 3% to 100%. Our dependent measure was the amplitude
of the fMRI response at the fundamental frequency of modulation.
With a linear system, this measure should be proportional to the
on/offamplitude of theunderlying
Appendix A for an explanation). Not surprisingly, the amplitudes of
the responses decreased with decreasing temporal period (or increas-
ing temporal frequency) due to the low-pass nature of the hemody-
namic transformation.
More importantly, the results again showed strong support of
time-contrast separability, meaning that changing the temporal peri-
od had only a scaling influence on the shape of the contrast–response
(see the original Fig. 6). Replotting the original results on a log-
response axis emphasizes this point (Fig. 5). Curves that are multiple
scales of each other (as predicted by time–contrast separability)
should be parallel on a log–amplitude axis. The curves in Fig. 5 are
roughly parallel with no systematic deviations in shape across stimu-
lus contrast.
neuronalresponse(See
Parameterized fit
We fit the same linear transform model to the data from the fre-
quency domain experiment and found similar parameter values as
for the time-domain data. This was important because though the ex-
perimental designs and dependent measures were quite different, the
same model should fit both data sets with roughly the same
parameters.
Noise
A final important result of our study related to the noise found in
fMRI data. In the linear transform model, noise is added after the con-
volution stage (see Fig. 1). This makes specific predictions about the
fMRI response that can best be thought of in the frequency domain.
Fig. 5. Frequency-domain results plotted on a log–response axis to illustrate time–contrast separability.
5
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 6

Truly additive and independent Gaussian noise (where an indepen-
dent random number drawn from a normal distribution is added to
the response at every time point) contains sinusoids at all frequencies
with roughly equal amplitudes. If Gaussian noise is added after the
linear transform, then the Fourier transform of the output should
look like a signal (amplitudes at frequencies related to the input)
plus a flat spectrum of amplitudes across all frequencies. If, however,
noise was added before the linear transform, then the higher frequen-
cy components of the noise should be attenuated by the low-pass
process of the hemodynamics.
We found that the noise spectrum of our frequency-domain study
was relatively flat (our original Fig. 9). This has important implica-
tions for those using deconvolution to estimate the hemodynamic re-
sponse functionfrom event-related
deconvolution is essentially linear regression which assumes additive
Gaussian noise. It also means filtering fMRI data with a low-pass tem-
poral filter is a good idea, since the variability in the fMRI time-course
at high temporal frequencies is likely due to stimulus-independent
noise factors.
A side note-subject gmb showed a splattering of noise around the
3–6 second temporal period, which is around the period of respira-
tion (See Fig. 9B in the original publication). In retrospect, this respi-
ratory artifact is probably not a hemodynamic issue but was probably
related to the physical stimulus. We projected our images from the
foot of the scanner table over the subjects' chest on to a screen just
at the subject's neck. The rising and falling of subject gmb's chest
may have periodically blocked the bottom of the image, leading to a
temporal modulation of the visual signal. Subject sae did not have
this problem.
experiments,because
Impact
Our results supported the linear transform model well enough
for us to confidently move on to quantitative studies of the
human visual system. As stated above, the results satisfied our
main concern that there might be an interaction between the
strength of the neuronal response and the shape of the hemody-
namic response.
Our plans at the time were to use our knowledge of linearity to
analyze results from blocked-design experiments. This is why
much of our paper focused on the frequency-domain experiment
and results. With linearity, we could be confident that the amplitude
of the fMRI signal in a blocked design experiment was proportional
to the amplitude of the underlying neuronal response. We then
went on and applied blocked-designs in a variety of studies, using
the amplitude of the Fourier transform at the stimulus frequency as
our measure of neuronal response (e.g. Demb et al., 1997, 1998;
Boynton et al., 1999; Gandhi et al., 1999; Heeger et al., 1999;
Wandell et al., 1999).
What we did not anticipate was the impact that our paper would
have on the emerging method of rapid event-related designs (see
Huettel in this issue) in which brief stimuli are presented close
enough in time for their responses to overlap. In a typical event-
related fMRI experiment, different types of events or conditions are
presented in a pseudo random order, resulting in a time-varying
fMRI signal that contains the sum of responses to individual events.
The hemodynamic response function for each event type is then
found that best-predicts the fMRI results when convolved with the
time-course of the stimulus (Dale, 1999). Clearly this deconvolution
process relies heavily on the assumption of linearity. Alternatively, a
canonical hemodynamic response function is assumed which, when
convolved with the stimulus time-course, produces a predicted fMRI
time-course that can be correlated with individual voxel time-
courses. These correlations provide a statistical parameter map for lo-
calizing event-related activity.
An anecdotal note: we actually tried to test the linear transform
model by measuring the fMRI response to a stimulus that followed
a complicated time-course (ramping up and down in contrast, turn-
ing on and off, etc.) In theory, we should have been able to use our
model and best-fitting parameters to the original data to predict the
resulting fMRI time course in V1. Unfortunately we didn't have
enough signal-to-noise in our data set to provide a convincing figure
for the paper. Perhaps this is why we went on to using the blocked
design in future studies instead of pursuing similar overlapping de-
signs, including event-related ones.
Note that brief stimulus presentations like those used in event-
related fMRI appears to show the greatest violation of the linearity as-
sumption. However, the reanalysis above (see Fig. 4) shows that an-
other way to look at our results is that linearity fails for longer
stimulus durations (due perhaps to neuronal adaptation). The over-
prediction of our original model in fits for short stimulus presenta-
tions was probably simply because the model placed more weight
on the longer-stimulus duration signals since they contained more
data points.
Overthepast15 yearsmuchefforthasbeenputintostudyingthebi-
ologicalbasisofthehemodynamic couplingprocess.Asecondimpactof
our paper has been to provide a framework for some of these studies,
particularly for correlative studies that compare more direct measures
of neuronal activity to more direct measures of blood flow and oxygen-
ation. Perhaps the most well-cited example of this is the study by
(Logothetis et al., 2001) that simultaneously measured the BOLD signal
and electrophysiological responses inthe monkey visualcortex.This ef-
fortledtothesuggestionthattheBOLDsignalismorelinearlyrelatedto
localfield potentialsthanspikes,thoughbothare good predictorsofthe
BOLDsignal. A veryrecentexampleof a correlative study is (Kahn et al.,
2011)thatusedacombinationofoptogeneticstostimulatethesomato-
sensory cortex in mice while measuring the BOLD signal and electro-
physiological recordings.
Another recent example is a study by Sirotin and Das. They mea-
sured simultaneously spiking activity and intrinsic optical signals.
They found that there was a component of the hemodynamic re-
sponses that was time-locked to the stimulus presentations but not
differentially responsive to the different stimulus conditions. They
found likewise that there was such a baseline component in the spik-
ing activity. The two baseline components (spiking and hemodynam-
ic) were not related to one another (Sirotin and Das, Nature, 2009).
But after removing each of the two baseline response components,
the residual hemodynamic responses are very tightly linked with
spiking activity, such that the time courses of the hemodynamic re-
sponses evoked by different stimulus contrasts are very well fit as a
shift-invariant linear transform of the firing rates (A. Das, personal
communication). Temporal summation experiments, much like our
additivity ones, showed that once again, the hemodynamic coupling
process is remarkably linear.
Another unanticipated impact of our paper is that the best-fitting
parameters from our study are often used to model the hemody-
namic response in a variety of studies. This is probably because cer-
tain software packages, such as BrainVoyager have a model of the
hemodynamic response with parameters set to those in our paper.
Though the parameters are adjustable, many users just use the de-
fault parameters. The two subjects (gmb and sae) were both au-
thors, so it is amusing to note how the authors' brains contributed
to this paper in more than one way. But using these exact parame-
ters is dangerous because it is well known that the hemodynamic
response function varies significantly across subjects and brain
areas e.g. (Aguirre et al., 1998). With our extremely small sample
size of two subjects, it doesn't seem wise to use our parameters
for estimates of the population. Indeed, given that the hemodynamic
response varies with age, (Kannurpatti et al., 2010) the parameters
from this paper are probably not even appropriate for our own,
now middle-aged brains!
6
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 7

Appendix A
In this appendix we define the properties of a linear system in the
context of the transformation between a neuronal response and the
BOLD signal.
A linear system satisfies the two properties of scaling and additiv-
ity. For our model, we also assume a third property, shift invariance.
Scaling
For a system operating in the time domain, scaling means that the
time-course of the output directly scales with the strength of the
input. For the case of a hemodynamic transform, L, we'll define the
input as a neuronal response over time, n(t), and the hemodynamic
response H(t) as the output of the system:
H t ð Þ ¼ L n t ð Þ½?ðA1Þ
For scaling to hold, if we multiply our input n(t) by a factor k,
then:
L kn t ð Þ½? ¼ kL n t ð Þ½? ¼ kH t ð ÞðA2Þ
The new output is equal to the original output multiplied by the
scale factor k.
In the case of the hemodynamic response, the slow, delayed time-
course of the output is very different than the shape of the input, and
we are interested in whether a linear system relates the two. If this
system shows scaling, then when the input is scaled the shape of
the output will always remain the same except that it is scaled by
the same factor. Fig. A1 illustrates the scaling property for an example
hemodynamic response to three neuronal inputs identical in time,
but with different strengths. Shown are three neuronal responses
and the predicted hemodynamic responses over time, each scaled in
proportion to their corresponding neuronal response.
Additivity
The property of additivity states that given two inputs, the response
to the sum of the inputs is equal to the sum of the responses to each of
the inputs alone. For our system L, given two inputs n1(t) and n2(t):
L n1t ð Þ þ n2t ð Þ½? ¼ L n1t ð Þ½ ? þ L n2t ð Þ½?ðA3Þ
For the hemodynamic response, the additivity property is best il-
lustrated for two neuronal responses that follow closely in time. If
they fall within a few seconds of each other, the output hemodynamic
response to the first input will still be ongoing when the
hemodynamic response to the second input begins. The additivity
principal says that this second hemodynamic response should simply
add to the first response, without interacting in any way.
Fig. A2 shows an example of how, for a linear shift-invariant sys-
tem, the hemodynamic responses to two successive 6-second inputs
is identical to the sum of the responses to a 6-second input added
to the same 6-second input delayed by 6 s. That is, you can either
add the inputs and then measure the response, or you can measure
the responses to the individual stimuli first and then add them.
Shift invariance
The additional property of shift invariance is also assumed in the lin-
ear transform model. Shift invariance means that the hemodynamic re-
sponse to a later input neuronal response should look just like the
response to an earlier, identical input, but shifted in time by the delay
between the two inputs.
Convolution
The two properties of scaling and additivity combine in a very
powerful way. Consider an arbitrary neuronal time-series n(t). We
can think of this time-series a sequence of shifted brief impulses
δ(t), each scaled accordingly:
n t ð Þ ¼ ∑
i
kiδ t−i
ðÞðA4Þ
The response to an impulse input is called (naturally) the impulse
response function,
H t ð Þ ¼ L δ t ð Þ½?ðA5Þ
Then by the properties of additivity and scaling, the response to
the arbitrary input is:
L ∑
i
kiδ t−i
ðÞ
??
¼ ∑
i
kiL δ t−i
ðÞ½? ¼ ∑
i
kiH t−i
ðÞðA6Þ
That is, the response is the sum of shifted and scaled responses to
the impulse.
The above argument shows that the impulse response function
completely characterizes a linear system, because it can be used to pre-
dict the response to any input. Of course, characterizing a time-series
as a sum of discrete scaled impulses is only an approximation, but the
above argument holds for any size impulse, no matter how small. In
the limit, the impulse response function is defined to be the response
to an infinitely short input.
Fig. A3 shows a hemodynamic response to an arbitrary neuronal
time-series defined as a sequence of scaled impulses. Each of these
brief impulses produces its own scaled response, and the response
to the entire time series is simply the sum of these individual scaled
responses.
This process of shifting and scaling a series of impulse responses to
predict a linear system's response to an arbitrary input is called con-
volution. It is possible to go the other way and estimate the impulse
response given a linear system's input and measured output. The re-
verse process is called deconvolution and is used in the analysis of
event-related fMRI experiments.
Frequency response
A second way to characterize a linear system is to work in the fre-
quency domain, which is to think about how the system transforms
inputs that vary sinusoidally over time. Given a system that obeys
scaling and additivity, the response to a sinusoidal input is also a sinu-
soid of the same frequency, though possibly shifted in phase.
Fig. A1. Example of the scaling property. Scaling the size of the input to a linear system
scales the size of the output without changing the shape of the response over time.
7
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 8

Remarkably, this is only true for sinusoids (technically, it's true for all
complex exponentials which includes sinusoids), and is true for any
shaped impulse response function. Fig. A4 shows how a linear hemo-
dynamic system should respond to sinusoidal inputs of varying fre-
quency (temporal periods of 30, 15 and 10 s). Note how the
frequency of the output matches the frequency of the input.
According to the Fourier theorem, any arbitrary time-series can be
described as a sum of sinusoids, each having a specific amplitude fre-
quency and phase. Using an argument like the one above for convolu-
tion, if we know how a linear system shifts and scales each of these
sinusoids then we can predict how the linear system will transform
any arbitrary input, since the output will simply be the sum of shifted
and scaled sinusoids that composed the input.
The way a linear system transforms a series of sinusoids is called
the frequency response function. Looking at Fig. A4, it can be seen
that the amplitude of the response is decreasing as the frequency of
the input increases (or as the period decreases). There is also a
specific phase delay for each frequency, but here we'll focus only on
amplitudes. This illustrates the typical low-pass property of the hemo-
dynamic response; the higher the frequency of the input, the lower
the relative amplitude of the output of the system.
It is not hard to show that there is a direct relationship between
the impulse response function for a linear system and the corre-
sponding frequency response function. This is the basis of the convo-
lution theorem which states that convolution in the time domain is the
same is multiplying in the frequency domain. That is, the Fourier
transform of the output of a linear system is equal to the product of
the Fourier transforms of the input and the impulse response
function.
Fig. A5 illustrates the Fourier theorem for a blocked-design exper-
iment. The input to the system is an on/off blocked design with a
30 second period (period=1/frequency), plus some additive neuro-
nal noise. It is assumed in this simulation that the input neuronal re-
sponses are turned on and off following a square wave. The top row
shows the linear system operating in the time domain by convolving
the input neuronal response with the impulse response function. The
output is a smoothly varying, roughly sinusoidal modulation. Now
consider the Fourier transform of the neuronal input (in the bottom
left of Fig. A5). It shows that to build a square-wave signal, we need
to add discrete sinusoidal components of increasing frequency and
decreasing amplitude. If our temporal period is P seconds, then the
periods we need are P/n for n=1, 3, 5, 7,… and the corresponding
amplitudes are 4π/n. In our example of a square wave with a 30 sec-
ond period, we need to add sinusoids of decreasing amplitudes start-
ing with a sinusoid with a period of 30 s (called the fundamental
period), followed by sinusoids having periods of 30/3=10, 30/
5=6, 30/7=4.29, 30/9=3.33 s and so on.
We can think of a linear system as working on each of these
sinusoidal components independently. To see how the hemody-
namics affect the input, we can look at the Fourier transform of
the impulse response function, shown in the middle of the bottom
row in Fig. A5. This graph shows how sinusoids are scaled by each
input frequency, and specifically, how the amplitude of the output
decreases with increasing frequency (decreasing temporal period).
To predict the output of the system to our square-wave input, we
simply multiply each of the amplitudes in the stimulus by their
corresponding value in the frequency response (multiplication is
actually done with complex numbers which takes into account
both amplitude and phase).
The Fourier transform of the resulting output is shown in the bot-
tom right of Fig. A5. The already decreasing amplitudes in the input
decrease more rapidly in the output due to the low-pass nature of
the impulse response function. The higher frequencies in the input
serve to “square-off” the fundamental 30-second period sinusoid. At-
tenuating these higher frequencies by the hemodynamic linear sys-
tem causes the output to look less square and more sinusoidal. For
this example, the fMRI response to a 30 second period input has
only a small 10-second period component and negligible higher fre-
quency components. Nearly all of the response is at the fundamental
Fig. A2. Example of the additivity property. The response to the sum of two inputs is
equal to the sum of the responses to the inputs alone.
Fig. A3. Combining scaling and additivity. The response to any arbitrary stimulus can be predicted by the summed and shifted responses to an impulse input.
8
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 9

frequency. This means that the analysis of results from a blocked-
design experiment using a 30-second period can focus on just the am-
plitude of the 30-second period Fourier component without throwing
out much information in the signal.
Note also that the property of scaling applies in the frequency
domain as well. Doubling the amplitude of a sinusoidal input dou-
bles the amplitude of the sinusoidal output. In the early days of
fMRI, the majority of studies used a standard blocked-design
paradigm in which two stimulus conditions alternated back and
forth throughout a scan at an even period. Using an on/off period
of 30 s or so, the resulting fMRI signal should modulate in a
roughly sinusoidal manner. With scaling, the amplitude of the si-
nusoidal fMRI signal can serve as a measure of neuronal activity
(e.g. Boynton et al., 1999; Gandhi et al., 1999; Heeger et al.,
1999). That is, given a linear hemodynamic transform, the amplitude
of the sinusoidal output should be proportional to the amplitude of
Fig. A4. The response of a linear system to a sinusoidal input is equal to a sinusoid of the same frequency, only scaled in amplitude and delayed in time.
Fig. A5. The convolution theorem: For a linear system, convolution in the time domain is equivalent to multiplication in the frequency domain.
9
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082

Page 10

the underlying neuronal response. A common application of the
blocked design is for phase-encoded retinotopic mapping experi-
ments (See Engel, this issue). The analysis of these experiments
only relies on the response at the fundamental frequency of the
stimulus. In analyzing these experiments, a sinusoidal model of the
BOLD signal was sufficient due to the low-pass nature of the hemo-
dynamic impulse response function (Engel et al., 1994).
References
Aguirre, G.K., Zarahn, E., et al., 1998. The variability of human, BOLD hemodynamic re-
sponses. NeuroImage 8 (4), 360–369.
Albrecht, D.G., Farrar, S.B., et al., 1984. Spatial contrast adaptation characteristics of
neurones recorded in the cat's visual cortex. J. Physiol. (Lond) 347, 713–739.
Albrecht, D.G., Geisler, W.S., 1991. Motion selectivity and the contrast-response func-
tion of simple cells in the visual cortex. Vis. Neurosci. 7 (6), 531–546.
Albrecht, D.G., Hamilton, D.B., 1982. Striate cortex of monkey and cat: contrast re-
sponse function. J. Neurophysiol. 48 (1), 217–237.
Bandettini, P.A., Jesmanowicz, A., et al., 1993. Processing strategies for time-course data
sets in functional MRI of the human brain. Magn. Reson. Med. 30 (2), 161–173.
Birn, R.M., Saad, Z.S., et al., 2001. Spatial heterogeneity of the nonlinear dynamics in the
FMRI BOLD response. NeuroImage 14 (4), 817–826.
Bonds, A.B., 1991. Temporal dynamics of contrast gain in single cells of the cat striate
cortex. Vis. Neurosci. 6 (3), 239–255.
Boynton, G.M., Demb, J.B., et al., 1999. Neuronal basis of contrast discrimination. Vis.
Res. 39 (2), 257–269.
Boynton, G.M., Engel, S.A., et al., 1996. Linear systems analysis of functional magnetic
resonance imaging in human V1. J. Neurosci. 16 (13), 4207–4221.
Dale, A.M., 1999. Optimal experimental design for event-related fMRI. Hum. Brain
Mapp. 8 (2–3), 109–114.
Demb, J.B., Boynton, G.M., et al., 1997. Brain activity in visual cortex predicts individual
differences in reading performance. Proc. Natl. Acad. Sci. U. S. A. 94 (24),
13363–13366.
Demb, J.B., Boynton, G.M., et al., 1998. Functional magnetic resonance imaging of early
visual pathways in dyslexia. J. Neurosci. 18 (17), 6939–6951.
Engel, S.A., Rumelhart, D.E., et al., 1994. fMRI of human visual cortex. Nature 369
(6481), 525.
Friston, K.J., Frith, C.D., et al., 1995. Characterizing evoked hemodynamics with fMRI.
NeuroImage 2 (2), 157–165.
Friston, K.J., Josephs, O., et al., 1998. Nonlinear event-related responses in fMRI. Magn.
Reson. Med. 39 (1), 41–52.
Gandhi, S.P., Heeger, D.J., et al., 1999. Spatial attention affects brain activity in human
primary visual cortex. Proc. Natl. Acad. Sci. U. S. A. 96 (6), 3314–3319.
Greenlee, M.W., Georgeson, M.A., et al., 1991. The time course of adaptation to spatial
contrast. Vision Res 31 (2), 223–236.
Heckman, G.M., Bouvier, S.E., et al., 2007. Nonlinearities in rapid event-related fMRI
explained by stimulus scaling. NeuroImage 34 (2), 651–660.
Heeger, D.J., Boynton, G.M., et al., 1999. Motion opponency in visual cortex. J. Neurosci.
19 (16), 7162–7174.
Kahn, I., Desai, M., et al., 2011. Characterization of the functional MRI response tempo-
ral linearity via optical control of neocortical pyramidal neurons. J. Neurosci. 31
(42), 15086–15091.
Kannurpatti, S.S., Motes, M.A., et al., 2010. Neural and vascular variability and the fMRI-
BOLD response in normal aging. Magn. Reson. Imaging 28 (4), 466–476.
Legge, G.E., Foley, J.M., 1980. Contrast masking in human vision. J. Opt. Soc. Am. 70
(12), 1458–1471.
Logothetis, N.K., Pauls, J., et al., 2001. Neurophysiological investigation of the basis of
the fMRI signal. Nature 412 (6843), 150–157.
Maddess, T., McCourt, M.E., et al., 1988. Factors governing the adaptation of cells in
area-17 of the cat visual cortex. Biol. Cybern. 59 (4–5), 229–236.
Robson, M.D., Dorosz, J.L., et al., 1998. Measurements of the temporal fMRI response of
the human auditory cortex to trains of tones. NeuroImage 7 (3), 185–198.
Sirotin, Y.B., Das, A., 2009. Anticipatory haemodynamic signals in sensory cortex not
predicted by local neuronal activity. Nature 457 (7228), 475–479.
Tuan, A.S., Birn, R.M., et al., 2008. Differential transient MEG and fMRI responses to vi-
sual stimulation onset rate. Int. J. Imaging Syst. Technol. 18 (1), 17–28.
Vazquez, A.L., Noll, D.C., 1998. Nonlinear aspects of the BOLD response in functional
MRI. NeuroImage 7 (2), 108–118.
Wandell, B.A., Poirson, A.B., et al., 1999. Color signals in human motion-selective cortex.
Neuron 24 (4), 901–909.
Watson, A.B., 1986. Temporal sensitivity. Handbook of perception and human perfor-
mance. Wiley, New York, p. 1. Boff et al.
Winawer, J., Horiguchi, H., et al., 2010. Mapping hV4 and ventral occipital cortex: the
venous eclipse. J. Vis. 10 (5), 1.
10
G.M. Boynton et al. / NeuroImage xxx (2012) xxx–xxx
Please cite this article as: Boynton, G.M., et al., Linear systems analysis of the fMRI signal, NeuroImage (2012), doi:10.1016/
j.neuroimage.2012.01.082