Cramér-Chernoff Theorem for L1-norm in Kernel Density Estimator for Two Independent Samples

Abstract

In this paper a Chernoff type theorem for the L1 distance between kernel estimators from two independent and identically distributed random samples is developed. The harmonic mean is used to correct the distance for inequal sample sizes case. Moreover, the proved result is used to compute the Bahadur slope of a test based on L1 distance and to compare it with the classical nonparametric Mann-Whitney test by using the Bahadur relative efficiency.

Full text

Page 1

Revista Colombiana de Estadística
Diciembre 2009, volumen 32, no. 2, pp. 289 a 299
Cramér-Chernoff Theorem for L1-norm in Kernel
Density Estimator for Two Independent Samples
Teorema de Cramér-Chernoff para la norma L1del estimador núcleo
para dos muestras independientes
Pablo Martínez-Camblor1,a, Norberto Corral2,b, Teresa López2,c
1CIBER de Epidemiología y Salud Pública (CIBERESP), Subdirección Salud
Pública de Gipuzkoa, Donostia, Spain
2Estadística e Investigación Operativa y Didáctica de la Matemática, Universidad
de Oviedo, Asturias, Spain
Resumen
In this paper a Chernoff type theorem for the L1 distance between kernel
estimators from two independent and identically distributed random sam-
ples is developed. The harmonic mean is used to correct the distance for
inequal sample sizes case. Moreover, the proved result is used to compute
the Bahadur slope of a test based on L1 distance and to compare it with the
classical nonparametric Mann-Whitney test by using the Bahadur relative
efficiency.
Palabras clave: Kernel estimator, Large deviation, Bahadur slope.
Abstract
En este trabajo se desarrolla un teorema de tipo Chernoff para la distan-
cia L1 entre estimadores núcleo procedentes de muestras aleatorias indepen-
dientes e idénticamente distribuidas. Se usa la media armónica para corregir
esta distancia en el caso de muestras de distintos tamaños. Además, se usa
el resultado demostrado para el cálculo de la pendiente de Bahadur de un
test para la comparación de densidades basado en la distancia L1 y se com-
para con el clásico test de Mann-Whitney a partir de la eficiencia relativa de
Bahadur.
Key words: estimador núcleo, grandes muestras, pendiente de Bahadur.
aInvestigador postdosctoral. E-mail: pmcamblor@hotmail.com
bCatedrático. E-mail: norbert@uniovi.es
cProfesora titular. E-mail: teresa@uniovi.es
289

Page 2

290
Pablo Martínez-Camblor, Norberto Corral & Teresa López
1. Introduction
Let X = {x1,...,xn} be an independent random sample from a random va-
riable X which is absolutely continuous with probability density function f. The
kernel density estimator (KDE) of f introduced by Rosenblatt (1956) and Parzen
(1962) is defined for each t ∈ R as
fn(X,t) =
1
nhn
n
?
i=1
K
?xi− t
hn
?
where K is a kernel function which is often chosen to be a continuous symmetric
density with finite variance, and {hn}n∈Nis a sequence of positive real numbers.
The kernel estimator and its properties have been widely studied by several
authors. Silverman (1978) proved its uniform consistence and Konakov (1978)
derived the asymptotic distribution for the L∞-norm. Devroye & Wagner (1979)
proved the L1 convergence between the kernel density estimator and its target,
and Devroye & Gyorfi (1985) carry out a widely study about the kernel density
estimators from the L1 approach. Berlinet et al. (1995) proved the asymptotic
normality for the L1-norm for the histogram density estimator, Hórvath (1991)
demonstrated the asymptotic normality of the Lpnorm between the kernel density
estimator and the underlying density function, Martínez-Camblor & Corral (2008)
proved the same result under weaker assumptions. Large deviation approaches
were also considered. Louani (1998, 2000, 2005) studied Chernoff type theorems
for the L1distance using for goodness of fit test. Cao & Lugosi (2005) studied the
propierties of several goodness of fit tests based on the kernel density estimate.
Osmoukhina (2001) applied these techniques in a symmetry test and Beirlant et al.
(2001) studied the large deviations of divergence measures.
Let X = {x1,...,xn1} and Y = {y1,...,yn2} be two independent random
samples from a random variable with density function f. The L1distance between
two estimators fn1and fn2of f is defined by
D(fn1,fn2) = ?fn1− fn2?L1=
?
|fn1(X,t) − fn2(Y,t)|dt
The main objective of this paper is to obtain a Chernoff type theorem for the
L1 distance, D(fn1,fn2), between two kernel density estimators when both are
computed from samples which are taken from the same population. That is, our
purpose is to investigate the expression
P
??
|fn1(X,t) − fn2(Y,t)|dt > λ
?
(1)
where λ is close to zero.
We are interested in this result because it can be applied in two sample pro-
blems and let us to compare different tests by using the Bahadur relative slope
(BRS). In Section 2 and, in a similar way than Louani (2000), we prove the large
deviation theorem. In Section 3 we consider several alternative hypothesis and,
Revista Colombiana de Estadística 32 (2009) 289–299

Page 3

Cramér-Chernoff Theorem for Two Independent Samples
291
from the previous results, the kernel density estimator based test with the clas-
sical nonparametric one of Mann-Whitney are compared in the Bahadur relative
efficiency (BRE) sense.
2. Results and Proofs
In order to prove the main result, we will consider the following assumptions:
(C1) X = {x1,...,xn1} and Y = {y1,...,yn2} are independent random samples
from a continuous random variable.
(C2) l´ ımnn1/n2= 1, where n → ∞ means that n1→ ∞ and n2→ ∞.
(C3) The used kernel function, K, is a continuous, symmetric density function.
(C4) l´ ımn1hn1= 0, l´ ımn2hn2= 0, l´ ımn1n1hn1= ∞ and l´ ımn2n2hn2= ∞
Nota 1. In practice, the used bandwith, hn, is chosen to minimize certain error
criterion (for example, the mean integrated squared error; MISE). Since the kernel
function is a symmetric and differentiable density function having finite variance,
these conditions are satisfied for most commonly used kernel functions like Gaus-
sian, Epanechnikov, Triangular, among others. As consequence, the assumptions
(C3) and (C4) are mild conditions.
Lema 1. Let X = {x1,...,xn} and Y = {y1,...,yn} be two independent random
samples from a random variable with density function f. For each interval B ∈ B
(Borel σ-field) we define
ZB,n,i=
?
B
?
1
hn1
K
?xi− t
hn1
?
−
1
hn2
K
?yi− t
hn2
??
dt with 1 ≤ i ≤ n
Then, under conditions (C1) (C2) and (C4), we have that
(1)
l´ ım
nZB,n,i=















1
if (xi,yi) ∈ (˚ B × B
if (xi,yi) ∈ (∂B × B
if (xi,yi) ∈ (B
if (xi,yi) ∈ (B
otherwise
c)
1/2
c)?(˚ B × ∂B)
c×˚ B)
−1/2
− 1
c× ∂B)?(∂B ×˚ B)
0
where˚ B and ∂B denote the interior and the boundary of B, respectively,
and B = B ∪ ∂B.
(2) For all λ > 0,
where a =?
limn1
nlogP?1
n
?n
i=1ZB,n,i> λ?= inf{t>0}{−λt + qa(t)}
Bf(t)dt and qa(t) = log(a2+ (1 − a)2+ 2a(1 − a)cosh(t))
Revista Colombiana de Estadística 32 (2009) 289–299

Page 4

292
Pablo Martínez-Camblor, Norberto Corral & Teresa López
Demostración. For 1 ≤ i ≤ n, we have that
l´ ım
n
?
B
?
1
hn1
K
?xi− t
hn1
?
−
1
hn2
?
K
?yi− t
?xi− t
hn2
??
dt =
l´ ım
n1
B
1
hn1
K
hn1
?
dt − l´ ım
n2
?
B
1
hn2
K
?yi− t
hn2
?
dt
Taking xi= t + hn1u and yi= t + hn2v we have that
l´ ım
n
?
B
?
1
hn1
K
?xi− t
l´ ım
n1
hn1
?
?
−
1
hn2
K
?yi− t
hn2
??
?
dt =
R
IBn1(u)K(u)du − l´ ım
n2
R
IBn2(v)K(v)dv =
?
R
[l´ ım
n1IBn1(u)]K(u)du −
?
R
[l´ ım
n2IBn2(v)]K(v)dv
where Bn1= {u/xi− uhn1∈ B} and Bn2= {v/yi− vhn1∈ B}.
By taking into account that
l´ ım
n1Bn1=









R
if xi ∈˚
if xi= sup(B)
if xi=´ ınf(B)
if xi ∈ Bc
B
R+
R−
φ
the result (1) is easily concluded.
Therefore, from the first part of this Lemma, for 1 ≤ i ≤ n there exists
ZB,i= l´ ım
nZB,n,i
random sample from the same random variable, ZB, whose moment generating
function is given by
gZB(t) =
??
?
etZBf(x)f(y)dxdy
=
B
?
Bcetf(x)f(y)dxdy
?
?
Bc
?
B
e−tf(x)f(y)dxdy
+
B
?
B
f(x)f(y)dxdy +
?
Bc
?
Bcf(x)f(y)dxdy
=a2+ (1 − a)2+ 2a(1 − a)cosh(t)
with a =?
Applying the Cramér-Chernoff Theorem (Van der Vaart 1998) the proof is
completed.
Bf(t)dt.
Observación 1. This result can be immediately extended to the case in which B
is a countable union of intervals.
Revista Colombiana de Estadística 32 (2009) 289–299

Page 5

Cramér-Chernoff Theorem for Two Independent Samples
293
From the above result we can derive a Chernoff type theorem. This result lets
us to compare the test based on the L1-norm of the kernel density estimator with
different tests by using their respective Bahadur relative slope (BRS).
Teorema 1. If conditions (C1), (C2), (C3) and (C4) are fulfilled and λ is a
nonnegative constant close to zero, then
l´ ım
n
n1+ n2
2n1n2
logP
??
|fn1(X,t) − fn2(Y,t)|dt > λ
?
= −λ2
4(1 + o(1)) a.s.
Demostración. To prove this theorem we will assume that n1= n2= n. Now,
we define the function,
Qa(λ) = −λ
2
arccosh
?
(1 − 2a + 2a2)λ2+ 2?λ2(1 − 2a)2+ 16a2(a − 1)2
(1 − a)2+ a2+(1 − 2a + 2a2)λ2+ 2?λ2(1 − 2a)2+ 16a2(a − 1)2
2a(a − 1)(λ2− 4)
?
?
+ log
?
(4 − λ2)
It is easy to check that Qa(λ) =´ ınft>0
expansion with λ in a neighborhood of zero, we have that
?−λ
2t + qa(t)?, and by using its Taylor
sup
a∈(0,1)Qa(λ) = Q1/2(λ) = −λ2
4(1 + o(1))
On the other hand, by using the Scheffé Theorem
?
|fn1(X,t) − fn2(Y,t)|dt = 2 sup
B∈B
????
?
B
(fn1(X,t) − fn2(Y,t))dt
????
Moreover, for every B ∈ B
P
?
2
?
B
(fn1(X,t) − fn2(Y,t))dt > λ
?
?xi− t
=
P
?
1
n
n
?
i=1
?
B
?
1
hn1
K
hn1
?
−
1
hn2
K
?yi− t
hn2
??
dt >λ
2
?
from the properties of Qa(λ), the previous Lemma 1 and by taking a =?
1
nlogP
B
Bf(t)dt,
we conclude that
l´ ım
n
?
2
?
(fn1(X,t) − fn2(Y,t))dt > λ
?
= Qa(λ)
Now, taken an arbitrary B0∈ B such that a = 1/2,
Q1/2(λ) = −λ
2arccosh
?4 + λ2
4 − λ2
?
+ log
?1
2
?
1 +4 + λ2
4 − λ2
??
Revista Colombiana de Estadística 32 (2009) 289–299