On studentising and blocklength selection for the bootstrap on time series.

Abstract

For independent data, non-parametric bootstrap is realised by resampling the data with replacement. This approach fails for dependent data such as time series. If the data generating process is at least stationary and mixing, the blockwise bootstrap by drawing subsamples or blocks of the data saves the concept. For the blockwise bootstrap a blocklength has to be selected. We propose a method for selecting the optimal blocklength. To improve the finite size properties of the blockwise bootstrap, studentised statistics is considered. If the statistic can be represented as a smooth function model this studentisation can be approximated efficiently. The studentised blockwise bootstrap method is applied for testing hypotheses on medical time series.

Full text

Page 1

On Studentising and Blocklength Selection for the Bootstrap
on Time Series
M. Peifer*;1, B. Schelter1;2, B. Guschlbauer2, B. Hellwig2, C. H. L?cking2,
and J. Timmer1
1Freiburg Centre for Data Analysis and Modelling, Eckerstr. 1, 79104 Freiburg, Germany
2Neurologische Universit?tsklinik Freiburg, Breisacher Str. 64, 79106 Freiburg, Germany
Received 18 December 2003, revised 24 May 2004, accepted 29 June 2004
Summary
For independent data, non-parametric bootstrap is realised by resampling the data with replacement.
This approach fails for dependent data such as time series. If the data generating process is at least
stationary and mixing, the blockwise bootstrap by drawing subsamples or blocks of the data saves the
concept. For the blockwise bootstrap a blocklength has to be selected. We propose a method for select-
ing the optimal blocklength. To improve the finite size properties of the blockwise bootstrap, studen-
tised statistics is considered. If the statistic can be represented as a smooth function model this studenti-
sation can be approximated efficiently. The studentised blockwise bootstrap method is applied for
testing hypotheses on medical time series.
Key words: Blockwise bootstrap; Studentising; Smooth function model; Tremor.
1Introduction
Since the introduction of the bootstrap, Efron (1979), methods based on resampling have been applied
on numerous statistical problems. The success of bootstrap may be explained by its easy implementa-
tion whenever the data are identical distributed and statistically independent. If time series of stochas-
tic processes are taken into account, the observations are in the most cases not independent. Then,
statistical inference using bootstrap methods becomes much more difficult and is often based on pro-
found model assumptions, see e.g. model based bootstrap, Davison and Hinkley (1997) or the sieve
bootstrap for ARMA-models, B?hlmann (1997).
If the model class or the model equations are unknown, a possible method of estimating the variance of
a statistic was proposed by Hall (1985) and Carlstein (1986). This non-parametric method is based on
building subsamples of the sequence of observations. Adapting this idea to the bootstrap methodology
leads to blockwise bootstrap, K?nsch (1989); Liu and Singh (1992). Here, the subsamples or data blocks
are resampled, instead of the data points itself. For establishing a consistent approximation of the distribu-
tion or variance of the statistic, structural assumptions on the underlying process has to be posed. These
assumptions are usually a-mixing and strong stationary. It is assumed that these basic conditions are valid
throughout this paper. The free parameter – blocklength l – has to be adjusted to each specific problem.
By minimising the mean squared error, Hall et al. (1995) showed that the optimal blocklength for a series
of length n is proportional to n
distribution functions and n
proportionality depends on the process and on the statistic. In the following a method to select this con-
stant is described. We show that this method gives appropriate estimates for the blocklength.
The second issue deals with the studentisation of the blockwise bootstrap method. If the statistic Tn
can be described by a smooth function of means, the variance of Tncan be approximated by its first
1
3for bias or variance approximation, n
5 for the approximation of two-sided distribution functions. The constant of
1
4for the approximation of one-sided
1
*Corresponding author: e-mail: peifer@fdm.uni-freiburg.de
Biometrical Journal 47 (2005) 3, 346–357 DOI: 10.1002/bimj.200310112
#2005 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim

Page 2

order Taylor expansion. A more general approach is to estimate this variance by linearising the statis-
tic using the empirical influence function Hampel et al. (1986); Huber (1980). These two methods
differ only by the way the statistic is linearised and therefore it is sufficient to regard only the smooth
function model.
By using a formal Edgeworth expansion, it can be shown that the rate of convergence of e.g. confi-
dence intervals increases if the statistic is studentised, Davison and Hall (1993); G?tze and K?nsch
(1996); Hall (1988); Hall and Wilson (1991); Hall (1992), Fisher and Hall (1990). Note that for the
validity of an Edgeworth expansion (see e.g. Bhattacharya and Ghosh (1978) for independent observa-
tions) in case of time dependent observations, the mixing coefficient has to decay exponentially, moment
conditions have to be satisfied and a conditional Cram?r condition has to be fulfilled, G?tze and Hipp
(1983). We conjecture that the convergence of bootstrap is increased even if there is no valid Edgeworth
expansion, which might result from the scale correcting nature of the studentisation.
Recent contributions on blockwise bootstrap are mostly found in the econometric sector, such as
Paparoditis and Politis (2003, 2001); Fitzenberger (1996). In case of biometrics, the contributions are
rather sparse. For demonstrating the practical significance of the blockwise bootstrap in the area of
medical statistics, a suitable application is discussed in Section 7.
2 Blockwise Bootstrap
Let X1; ...; Xn be an observed sample from a strongly stationary, a-mixing, p-variate sequence
ðXtÞt2Z. The real-valued statistics Tn¼ TnðX1; ...; XnÞ is assumed to be invariant under permutations
of the observations.
For the blockwise bootstrap, b subsamples or blocks of length l are formed from the observations.
We further assume, without loss of generality, that n=l 2 N (otherwise, the data sample is truncated
until n=l 2 N holds). In the framework of blockwise bootstrap, two kinds of building subsamples are
predominating, the overlapping blocks and the non-overlapping blocks. The overlapping blocks are
defined by
Yi¼ ðXi; ...; XiþlÞ
and non-overlapping are defined as follows:
i ¼ 1; ...; b ¼ n ? l ? 1;
ð1Þ
Yi¼ ðXði?1Þlþ1; ...; XilÞ
It turns out that the blockwise bootstrap gives quite comparable results if either overlapping or non-
overlapping blocks are used, K?nsch (1989); Hall et al. (1995). Unless otherwise mentioned, the re-
sults of this paper remain the same whether overlapping or non-overlapping blocks are used.
Blockwise bootstrap is realised by resampling the blocks Yi and gluing them together to form a
kind of surrogate time series of length n. Finally, the statistic is applied on each bootstrapped series to
estimate quantities like distribution functions, bias or variance. The algorithmic representation of this
procedure is:
1. Drawing blocks with replacement from fY1; ...; Ybg and forming Y*1; ...; Y*b by gluing the
drawn blocks together.
2. Repeat 1. B times to generate B bootstrap samples X*1; ...; X*B.
3. Calculate T*n;k¼ TnðX*1;k; ...; X*n;kÞ; k ¼ 1; ...; B:
4. Finally, determine the bootstrap approximation of e.g.:
– the distribution function F*ðxÞ ¼ B?1P
k¼1
– the variance s2*¼ B?1P
where qð?Þ is the step function.
i ¼ 1; ...; b ¼n
l:
ð2Þ
B
k¼1
q x ? T*n;k
??
– the bias Bias*¼ B?1P
B
T*n;k? TnðX1; ...; XnÞ
B
T*n;k? B?1P
k¼1
B
j¼1
T*n; j
!2
,
Biometrical Journal 47 (2005) 3 347
#2005 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim

Page 3

Some remarks on the consistency of the described blockwise bootstrap: Only demanding strong
stationarity and that the mixing coefficient aðtÞ vanishes with respect to the time lag t is too weak for
a valid bootstrap approximation. The consistency can for example be achieved, as Naik-Nimbalkar
and Rajarshi (1994) have shown, if
P
? Xihas a continuous distribution function on R
? l ¼ lðnÞ ¼ Oðn1=2?EÞ; with 0 < E <1
? the statistic Tn has to be Hadamard or compactly differentiable within a sufficient space of dis-
tributions (see also Gill (1989)).
?
1
i¼0
ði þ 1Þ7aðiÞ1=2?t< 1; for t 2 ð0;1
2Þ
2
For the choice of the blocklength l, we assume that the mixing coefficient decays exponentially with
respect to the time lag. This also assures under mild moment conditions and under some kind of Cram?r
condition the validity of the Edgeworth expansion, G?tze and Hipp (1983). Here again, the compact differ-
entiability cannot be dropped. The described smooth function model is always compact differentiable.
3Studentising the Blockwise Bootstrap
3.1Smooth function model
Suppose
? W W ¼ n?1P
an
n
i.i.d.sample of
p-variaterandom vectors,
W1; ...; Wn.Let
m ¼ E½Wi?
and
i¼1
A : Rp! D ? R;
Wi. A function A is called a smooth function model if
A 2 C1Rp; D
ðÞ
and
AðmÞ ¼ 0:
ð3Þ
This concept can be applied on statistical problems in which the statistic can be expressed by a
smooth function g such as Tn¼ gð? W WÞ. The associated smooth function model then yields:
AðwÞ ¼ gðwÞ ? gðmÞ. The main advantage of the smooth function model is that the variance of the
statistic Tn can be well approximated. Additionally, by using the chain rule, the existence of the
Fr?chet derivative with respect to the distribution is proven. In the following the i-th component of a
vector, say Y, is denoted by YðiÞ. To approximate the variance of Tnlet
ai¼@AðwÞ
@wðiÞ
w¼m
@2AðwÞ
@wðiÞ@wðjÞ
w¼m
Because of the independence of the Wi’s, E½ZðiÞ? ¼ 0 and E½ZðiÞZðjÞ? ¼ Cij. Now, Taylor-expanding the
function Sn¼ n
Sn¼P
And therefore the expectations E Sn
½
E Sn
½
2
i; j¼1
?
Using Var Tn
ð
VarðTnÞ ¼ n?1P
#2005 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
Z ¼ n
1
2 ? W W ? m
ðÞ;
????
bij¼
????
and
Cij¼ E½ðW1? mÞðiÞðW1? mÞðjÞ?:
1
2Að? W WÞ at w ¼ m, leads to:
p
aiZðiÞþ n?1
i¼1
21
2
P
p
i; j¼1
bijZðiÞZðjÞþ Opðn?1Þ:
?
bijCijþ Oðn?1Þ;
? and E S2
n
?are given by:
? ¼ n?1
21
P
aiajCijþ Oðn?1Þ:
p
E S2
n
?¼
Þ ¼ n?1Var Sn
P
p
i; j¼1
ð
p
Þ ¼ n?1ðE½S2
aiajCijþ Oðn?2Þ:
n? ? ðE½Sn?Þ2Þ þ Oðn?2Þ givesthe approximativevariance ofTn:
i; j¼1
ð4Þ
348 M. Peifer et al.: Studentising and Blocklength Selection for Blockwise Bootstrap

Page 4

Finally, the estimators ^ a ai¼@AðwÞ
Eq. (9) to estimate the variance:
@wðiÞ
????
w¼? W W
and^C Cij¼ n?1P
n
k¼1
Wk?? W W
ðÞðiÞWk?? W W
ðÞðjÞare plugged into
^ s s2
n¼d
VarVarðTnÞ ¼ n?1P
p
i; j¼1
^ a ai^ a aj^ C Cij:
ð5Þ
Extending these results to blockwise defined statistics leads to variance estimators which are sufficient
for studentising the described bootstrap method.
3.2The procedure of studentising blockwise bootstrap
Since Eq. (5) does not give consistent results if the observation Wiare not i.i.d., a modification of the
smooth function A is needed before applying these results on time series. To approximate the variance
of Tnunder conditions outlined in Section 2, the observations Xiare transformed into a new series Wi
in which Tn¼ gð? W WÞ, as in Section 3.1. Again, the series Wi is not i.i.d.. To incorporate the depen-
dence structure into ^ s s2
n, the blocking scheme is implemented into the statistic. For this purpose define
~ W Wi¼ ðWð1Þ
ði?1Þlþ1; ...; Wð1Þ
il; ...; WðpÞ
ði?1Þlþ1; ...; WðpÞ
ilÞ;
i ¼ 1; ...;n
l
for non-overlapping blocks and for overlapping blocks
~ W Wi¼ ðWð1Þ
i ; ...; Wð1Þ
iþl; ...; WðpÞ
i ; ...; WðpÞ
iþlÞ;
i ¼ 1; ...; n ? l ? 1:
The statistic is then rewritten into
?
?
the blocklength is adequately chosen and therefore the blocks are approximately independent, such
that the approximation in Eq. (5) can be used.
To giveanexample of thedescribed
Tn¼ n?1P
^ s s2
^ E E½X1?
i¼1
l^ g g3ð0Þ þ 2P
where
Tn¼ ~ g g b?1P
b
i¼1
~ W Wi
?
;
where ~ g gðwÞ ¼ g l?1P
l
i¼1ðwðiÞ; ...; wððp?1ÞlþiÞÞ
?
. And hence again, AðwÞ ¼ ~ g gðwÞ ? ~ g gðmÞ. Suppose that
procedure,let
Tn
bethesamplevariance
n
i¼1
X2
i?
n?1P
?
n
i¼1
?
?
Xi
??2
. Then
n¼ n?1l?24
?2
l^ g g1ð0Þ þ 2P
l?1
ðl ? iÞ ^ g g3ðiÞ þ l^ g g4ð0Þ þ 2P
l?1
ðl ? iÞ ^ g g1ðiÞ
??
þ l^ g g2ð0Þ þ 2P
l?1
ðl ? iÞ ^ g g4ðiÞ
l?1
i¼1
ðl ? iÞ ^ g g2ðiÞ
??
þ 2^ E E½X1?
i¼1
i¼1
;
^ E E½X1? ¼? X X ¼ n?1P
^ g g1ðtÞ ¼
n ? t
^ g g3ðtÞ ¼
n ? t
and
n
i¼1
Xi?? X X
Xi;
1
P
P
n?t
i¼1
n?t
ðÞ Xiþt?? X X
?
ðÞ;
^ g g2ðtÞ ¼
1
n ? t
P
n?t
n?t
i¼1
X2
i?~ X X2
??
X2
iþt?~ X X2
?
?
?;
Þ;
1
i¼1
Xi?? X X
ðÞ X2
iþt?~ X X2
?;
^ g g4ðtÞ ¼
1
n ? t
P
i¼1
X2
i?~ X X2
?
Xiþt?? X X
ð
~ X X2¼ n?1P
n
i¼1
X2
i:
Biometrical Journal 47 (2005) 3 349
#2005 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim

Page 5

Studentised bootstrap is realised by studentising each bootstrap sample T*stud;n¼T*n? Tn
strap approximation of the studentised statistic is now determined similar to Section 2 and can be used
e.g. to approximate confidence intervals of level a
^ s s*n
. The boot-
^I Ia¼ ½t?? ^ s snþ Tn; tþ? ^ s snþ Tn?;
where
t?¼ sup
x2R
a
2
? ?
? F*studðxÞ
no
and
tþ¼ inf
x2R
1 ?a
2
??
? F*studðxÞ
no
:
The distribution function F*studðxÞ has to be calculated as in Section 2. The only difference is that T*n
is replaced by its studentised bootstrap samples T*stud;n.
4 Blocklength Selection
The success of the method depends on the choice of the blocklength l. Taking the mean squared error
as objective measure to be minimised, the blocklength selection is similar to the choice of the smooth-
ing parameter in non-parametric function estimation, Peifer et al. (2003); Hall et al. (1995). In both
cases a tradeoff between bias and variance is present. For the sample mean, the mean squared error
can be calculated and is in case of the bootstrap variance approximation, Hall et al. (1995),
1
n2l2C1þl
where
8
>
Here, gðkÞ is the auto-covariance function of the process. The optimal blocklength is therefore given
by lopt¼ ð2C1=C2Þ1=3n1=3. To generalise this concept, the statistic Tnis linearised before, using either
the smooth function model or the empirical influence function. The constants C1 and C2 are then
determined by the covariance structure of the linearised statistic. A first idea to estimate the block-
length is to plug-in the empirical auto-covariance function into C1and C2. Since the correlated errors
MSE ðlÞ ¼ E½ðVar*ð? X XÞ ? Var ð? X XÞÞ2? ?
n3C2;
ð6Þ
C1¼
P
1
k¼?1
jkj gðkÞ
??2
;
C2¼
2
P
P
1
k¼?1
1
gðkÞ
?
?
?2
?2
for non-overlapping blocks
4
3
k¼?1
gðkÞ
for overlapping blocks.
>
>
>
>
>
:
<
of the estimated auto-covariance function are amplified by the factor jkj in C1¼
this method fails. To avoid this problem, the global behaviour of the auto-covariance function is para-
meterised and fitted to the empirical auto-covariance function. For this, it is assumed that the mixing
coefficient decays exponentially and hence the auto-covariance function decays in maximum exponen-
tially, too. In making use of this extra assumption, some points of the correlation function are esti-
mated: ^ g gðkÞ;k ¼ 0; ...; m < n, and the function fðkÞ ¼ fk;0 ? f < 1 is fitted to the envelope of
^ g gðkÞ. Then the estimated parameter f contains the characteristic time scale of the process and the
blocklength is finally estimated by replacing gðkÞ in C1;C2 with fk. The procedure of selecting the
blocklength is therefore the following:
P
1
k¼?1
jkj gðkÞ
??2
? Linearise the statistic Tnby transforming the data points to Visuch that
Tn¼ gð? W WÞ ? n?1P
function model of Section 3.1. Or alternatively linearise the statistic using the influence function
approach, Hampel et al. (1986); Huber (1980).
? Estimate the auto-covariance function of the transformed series Vi.
n
i¼1
P
p
j¼1
@gðwÞ
@wðjÞ
????
w¼ ? W
WðjÞ
i
¼ n?1P
n
i¼1
Vi) Vi¼P
p
j¼1
@gðwÞ
@wðjÞ
????
w¼ ? W
WðjÞ
i
for the smooth
350 M. Peifer et al.: Studentising and Blocklength Selection for Blockwise Bootstrap
#2005 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim