Root n Consistent and Optimal Density Estimators for Moving Average Processes

Abstract

The marginal density of a rst order moving average process can be written as convolution of two innovation densities. Saavedra and Cao (2000) propose to estimate the marginal density by plugging in kernel density estimators for the innovation densities, based on estimated innovations. They obtain that for an appropriate choice of bandwidth the variance of their estimator decreases at the rate 1=n. Their estimator can be interpreted as a speci c U-statistic. We suggest a slightly simpli ed U-statistic as estimator of the marginal density, prove that it is asymptotically normal at the same rate, and describe the asymptotic variance explicitly. We show that the estimator is asymptotically ecient if no structural assumptions are made on the innovation density. For innovation densities known to have mean zero or to be symmetric, we describe improvements of our estimator which are again asymptotically ecient.

Full text

Page 1

Root n consistent and optimal density estimators
for moving average processes
Anton Schick
Binghamton University
Wolfgang Wefelmeyer
Universit¨ at Siegen
Abstract
The marginal density of a first order moving average process can be written as convolu-
tion of two innovation densities. Saavedra & Cao (2000) propose to estimate the marginal
density by plugging in kernel density estimators for the innovation densities, based on esti-
mated innovations. They obtain that for an appropriate choice of bandwidth the variance
of their estimator decreases at the rate 1/n.
Their estimator can be interpreted as a specific U-statistic.
simplified U-statistic as estimator of the marginal density, prove that it is asymptotically
normal at the same rate, and describe the asymptotic variance explicitly. We show that the
estimator is asymptotically efficient if no structural assumptions are made on the innovation
density. For innovation densities known to have mean zero or to be symmetric, we describe
improvements of our estimator which are again asymptotically efficient.
We suggest a slightly
Key words: Efficient estimator, least dispersed estimator, plug-in estimator, semiparamet-
ric model, time series.
Running head: Root n consistent density estimators.
1Introduction
Suppose X1,...,Xnare observations from an MA(1) process Xt= εt− ϑεt−1, where εtare
i.i.d. innovations with density f and finite second moment, and |ϑ| < 1. We want to estimate
the density of Xt, say g, at a fixed point x. A widely used estimator for g(x) is the kernel
estimator
g(x) =1
n
j=1
n
?
Kb(x − Xj),
where Kb(u) = K(u/b)/b with kernel function K and bandwidth b > 0. Note that
?
g(x) =
f(x + ϑy)f(y)dy.
1

Page 2

Motivated by this representation, Saavedra & Cao (1999a, 2000) propose the plug-in estimator
?
whereˆϑ is n1/2-consistent andˆf is a kernel estimator based on estimated innovations ˆ εj. They
obtain that their estimator is n1/2-consistent for the bandwidth b = n−2/5. For this they as-
sume that the innovations have mean zero. They also use rather strong regularity conditions.
Saavedra & Cao (1999b) compare the mean integrated squared error of their estimator and
the usual kernel estimator. We note that for continuous-time processes, corresponding consis-
tency rates for kernel-type estimators are more common. Castellana & Leadbetter (1986) give
conditions for such results to hold in general stationary processes; see also Bosq (1993, 1995),
Blanke & Bosq (1997), Bosq et al. (1999) and Veretennikov (1999). Kutoyants (1997, 1998,
1999) obtains efficiency of estimators for the density of diffusion processes.
The parametric convergence rate n−1/2for a density estimator may seem striking but
becomes plausible once one notes that g(x) is represented as an integral functional of the
innovation density f. For models with i.i.d. observations it is well known that certain integral
functionals of densities and their derivatives are estimated at the rate n−1/2if appropriate
kernel estimators for the density are plugged into the functional. Similar results hold for
integral functionals of regression functions and quantile regression functions. Some recent
references are Birg´ e & Massart (1995), Tsybakov & van der Meulen (1996), Chaudhuri et
al. (1997) and Efromovich & Samarov (2000). Almost sure representations are obtained by
Eggermont & LaRiccia (1999,2001) and Mason (2003). Note that to estimate g(x), we must
also plug in an estimator for ϑ.
The estimator ˆ gSC(x) can be written
ˆ gSC(x) =
ˆf(x +ˆϑy)ˆf(y)dy,
ˆ gSC(x) =
1
n2b
n
?
i,j=1
Lˆϑ
?x − ˆ εi+ˆϑˆ εj
b
?
with Lϑ(u) =?K(u + ϑv)K(v)dv. A closely related estimator is the U-statistic
ˆ g(x) =
n(n − 1)
1
n
?
i?=j
i,j=1
Kb(x − ˆ εi+ˆϑˆ εj).
This is essentially ˆ gSC(x) with the random kernel Lˆϑreplaced by K. In this paper we prove
that n1/2(ˆ g(x) − g(x)) is asymptotically normal, and calculate the asymptotic variance. Our
result is valid for bandwidths b = n−awith a strictly between 1/6 and 1/8. We prove it under
much weaker regularity conditions. We also do not assume that the innovations have mean
zero.
Of course, these results are valid only if ϑ ?= 0 because for ϑ = 0 the model reduces to i.i.d.
observations Xj= εjfor which it is well-known that no n1/2-consistent estimators exist.
2

Page 3

A slightly more difficult argument would show that ˆ gSC(x), with random kernel Lˆϑ, is
asymptotically equivalent to our estimator (under weaker regularity conditions, for our choices
of bandwidth, and without the assumption of mean zero innovations).
Since kernel estimators are not n1/2-consistent, there is no attainable variance bound for
them. Here we have n1/2-consistency, and hence a theoretically attainable variance bound. We
prove that our estimator attains the variance bound if we choose an efficient estimator for ϑ.
More precisely, we then have semiparametric efficiency as described in Bickel et al. (1998) for
i.i.d. observations.
If we have additional restrictions on the innovation distributions, such as mean zero or
symmetry, we can improve our estimator. For symmetric innovations, we symmetrize the
estimator as
1
4n(n − 1)
i?=j
For mean zero innovations as in Saavedra & Cao (1999a, 2000), improvements are possible by
subtracting from the estimator for g(x) a term of the form ˆ a1
random coefficient ˆ a.
Estimators of other functionals of MA processes have been considered before. Efficient
estimators of ϑ under various assumptions are in Kreiss (1987), Jeganathan (1995), Drost et
al. (1997) and Schick & Wefelmeyer (2000b). Kreiss (1990) also estimates the coefficients of
infinite-order MA processes. Efficient estimators of linear functionals of the stationary law
are in Schick & Wefelmeyer (2002c). The innovation distribution function of infinite-order MA
processes is estimated in Kreiss (1991). Schick & Wefelmeyer (2002d) prove a functional central
limit theorem and efficiency of a version of ˆ gSCfor higher order moving average processes.
In Section 2 we state our results. In Section 3 we present the results of a small simulation
study. Appendix 1 contains auxiliary results on U-statistics and related statistics, Appendices
2 and 3 contain proofs of Theorems 1 and 2.
?
?Kb(x − ˆ εi+ˆϑˆ εj) + Kb(x + ˆ εi+ˆϑˆ εj) + Kb(x − ˆ εi−ˆϑˆ εj) + Kb(x + ˆ εi−ˆϑˆ εj)?.
n
?n
j=1ˆ εj with properly chosen
2Results
We consider an MA(1) process Xt= εt−ϑεt−1, where 0 < |ϑ| < 1 and εtare i.i.d. innovations
with distribution function F, density f, mean µ and finite variance σ2. The innovations εt
have representation
∞
?
see e.g. Brockwell & Davis (2002, Section 3.1). For later use, we also introduce the derivative
of εtwith respect to ϑ,
∞
?
εt=
s=0
ϑsXt−s; (1)
˙ εt=
s=1
sϑs−1Xt−s=
∞
?
s=1
ϑs−1εt−s.
(2)
3

Page 4

Its expectation is
E[˙ ε] =
∞
?
s=1
sϑs−1(1 − ϑ)µ =
µ
1 − ϑ,
and its variance is
Var[˙ ε] =
σ2
(1 − ϑ)(1 + ϑ).
Let g denote the density of Xt. We want to estimate g at a fixed point x, based on obser-
vations X1,...,Xn. Letˆϑ be a n1/2-consistent estimator of ϑ. Representation (1) motivates
estimating εjby
r
?
for some integer r. We modify the estimator given in the Introduction correspondingly and
estimate g(x) by the U-statistic
ˆ εj=
s=0
ˆϑsXj−s,j = r + 1,...,n,
ˆ g(x) =
1
(n − r)(n − r − 1)
n
?
i?=j
i,j=r+1
Kb(x − ˆ εi+ˆϑˆ εj),
where Kb(u) = K(u/b)/b for some kernel function K and some bandwidth b > 0.
Condition 1
The kernel K has compact support, is twice continuously differentiable, and satisfies
?
K(u)du = 1and
?
uiK(u)du = 0 for i = 1,2,3.
Condition 2
The innovation density f has an absolutely continuous derivative, and the almost everywhere
derivative f??of f?is integrable and bounded.
The density of ϑε is fϑ(u) = f(u/ϑ)/|ϑ|. We have the representations
?
The derivatives of g with respect to ϑ and x are
g(x) =
f(x + ϑy)f(y)dy =
?
fϑ(y − x)f(y)dy.
˙ g(x)=
?
?
yf?(x + ϑy)f(y)dy,
g?(x)=
f?(x + ϑy)f(y)dy.
We set
ψ(y) = f(x + ϑy) + fϑ(y − x) − 2g(x),y ∈ R.
4

Page 5

Theorem 1
Assume Conditions 1 and 2. Letˆϑ be a n1/2-consistent estimator of ϑ. Let the bandwidth b
fulfill nb6→ ∞ and nb8→ 0. Let r/logn → ∞ and r/n → 0. Then
n
?
If f also has a finite third moment and K has a bounded third derivative, then we can relax
nb6→ ∞ to nb4→ ∞.
Typical estimatorsˆϑ of ϑ are such that n1/2(ˆϑ − ϑ) is asymptotically normal with mean
zero and variance ζ2, say, and asymptotically independent of n−1/2?n
variance
τ2=
n1/2(ˆ g(x) − g(x)) = n−1/2
j=1
ψ(εj) +?˙ g(x) − µg?(x)?n1/2(ˆϑ − ϑ) + op(1).
j=1ψ(εj). In this case,
it follows from Theorem 1 that n1/2(ˆ g(x)−g(x)) is asymptotically normal with mean zero and
?
We assume two derivatives for f. The corresponding kernel estimator has optimal band-
width proportional to n−1/5. For our plug-in estimator, a larger bandwidth, with rate between
n−1/6and n−1/8, say n−1/7, is needed, unless we require the additional assumptions at the
end of Theorem 1. Because we have two derivatives for f, we have four derivatives for g. The
optimal bandwidth for a kernel estimator of g would therefore be proportional to n−1/9.
We show now that ˆ g(x) is efficient for g(x) ifˆϑ is efficient for ϑ. This is an instance of a
parametric “plug-in principle”: If ˆ κ(ϑ) is an efficient estimator for some functional κ of (ϑ,f)
when ϑ is known, andˆϑ is efficient for ϑ, then the plug-in estimator ˆ κ(ˆϑ) will be efficient when
ϑ is not known. We refer to Klaassen & Putter (2002) for sufficient conditions in the i.i.d.
case. We make the following assumption.
ψ2dF + ζ2?˙ g(x) − µg?(x)?2.
Condition 3
The innovation density f has finite Fisher information for location. This means that f is
absolutely continuous and
?
J =
?2(y)f(y)dy < ∞,
where ? = f?/f.
Then we can write
g?(x) =
?
?(y)fϑ(y − x)f(y)dy = −1
ϑ
?
?(y)f(x + ϑy)f(y)dy.
(3)
The first identity follows from a substitution, while the second follows from integration by parts.
It is well-known that the MA(1) model is locally asymptotically normal under Condition 3.
For various versions see Kreiss (1987), Jeganathan (1995), Drost et al. (1997), Koul & Schick
5