Adaptive Hausdorff estimation of density level sets

Abstract

Consider the problem of estimating the $\gamma$-level set $G^*_{\gamma}=\{x:f(x)\geq\gamma\}$ of an unknown $d$-dimensional density function $f$ based on $n$ independent observations $X_1,...,X_n$ from the density. This problem has been addressed under global error criteria related to the symmetric set difference. However, in certain applications a spatially uniform mode of convergence is desirable to ensure that the estimated set is close to the target set everywhere. The Hausdorff error criterion provides this degree of uniformity and, hence, is more appropriate in such situations. It is known that the minimax optimal rate of error convergence for the Hausdorff metric is $(n/\log n)^{-1/(d+2\alpha)}$ for level sets with boundaries that have a Lipschitz functional form, where the parameter $\alpha$ characterizes the regularity of the density around the level of interest. However, the estimators proposed in previous work are nonadaptive to the density regularity and require knowledge of the parameter $\alpha$. Furthermore, previously developed estimators achieve the minimax optimal rate for rather restricted classes of sets (e.g., the boundary fragment and star-shaped sets) that effectively reduce the set estimation problem to a function estimation problem. This characterization precludes level sets with multiple connected components, which are fundamental to many applications. This paper presents a fully data-driven procedure that is adaptive to unknown regularity conditions and achieves near minimax optimal Hausdorff error control for a class of density level sets with very general shapes and multiple connected components. Comment: Published in at http://dx.doi.org/10.1214/08-AOS661 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Full text

Page 1

arXiv:0908.3593v1 [math.ST] 25 Aug 2009
The Annals of Statistics
2009, Vol. 37, No. 5B, 2760–2782
DOI: 10.1214/08-AOS661
c ? Institute of Mathematical Statistics, 2009
ADAPTIVE HAUSDORFF ESTIMATION OF DENSITY LEVEL
SETS
By Aarti Singh,1Clayton Scott and Robert Nowak1
University of Wisconsin–Madison, University of Michigan–Ann Arbor and
University of Wisconsin–Madison
Consider the problem of estimating the γ-level set G∗
γ} of an unknown d-dimensional density function f based on n inde-
pendent observations X1,...,Xn from the density. This problem has
been addressed under global error criteria related to the symmetric
set difference. However, in certain applications a spatially uniform
mode of convergence is desirable to ensure that the estimated set is
close to the target set everywhere. The Hausdorff error criterion pro-
vides this degree of uniformity and, hence, is more appropriate in such
situations. It is known that the minimax optimal rate of error con-
vergence for the Hausdorff metric is (n/logn)−1/(d+2α)for level sets
with boundaries that have a Lipschitz functional form, where the pa-
rameter α characterizes the regularity of the density around the level
of interest. However, the estimators proposed in previous work are
nonadaptive to the density regularity and require knowledge of the
parameter α. Furthermore, previously developed estimators achieve
the minimax optimal rate for rather restricted classes of sets (e.g.,
the boundary fragment and star-shaped sets) that effectively reduce
the set estimation problem to a function estimation problem. This
characterization precludes level sets with multiple connected com-
ponents, which are fundamental to many applications. This paper
presents a fully data-driven procedure that is adaptive to unknown
regularity conditions and achieves near minimax optimal Hausdorff
error control for a class of density level sets with very general shapes
and multiple connected components.
γ= {x:f(x) ≥
1. Introduction.
many applications including clustering [6, 8, 21], anomaly detection [16, 20,
24], functional neuroimaging [12, 25], bioinformatics [27], digital elevation
Level sets provide useful summaries of a function for
Received November 2007; revised August 2008.
1Supported in part by NSF Grants ECS-05-29381, CCF-03-53079 and CCR-03-50213.
AMS 2000 subject classifications. 62G05, 62G20.
Key words and phrases. Density level set, Hausdorff error, rates of convergence, adap-
tivity.
This is an electronic reprint of the original article published by the
Institute of Mathematical Statistics in The Annals of Statistics,
2009, Vol. 37, No. 5B, 2760–2782. This reprint differs from the original in
pagination and typographic detail.
1

Page 2

2
A. SINGH, C. SCOTT AND R. NOWAK
mapping [19, 26] and environmental monitoring [22]. In practice, however,
the function itself is unknown a priori, and only a finite number of observa-
tions related to f are available. In this paper, we focus on the density level
set problem; extensions to general regression level set estimation should be
possible using a similar approach, but they are beyond the scope of this
paper. Let X1,...,Xnbe independent, identically distributed observations
drawn from an unknown probability measure P, having density f with re-
spect to the Lebesgue measure and defined on the domain X ⊆ Rd. Given a
desired density level γ, consider the γ-level set of the density f
G∗
γ:= {x ∈ X :f(x) ≥ γ}.
The goal of the density level set estimation problem is to generate an esti-
mate?G of the level set based on the n observations {Xi}n
performance measure which gauges the closeness of the two sets, is small.
Most literature available on level set estimation methods [9, 13, 14, 15,
16, 20, 23, 26] considers error measures related to the symmetric set dif-
ference, G1∆G2= (G1\ G2) ∪ (G2\ G1). However, level set methods based
on a measure of the symmetric difference error may produce estimates that
veer greatly from the desired level set at certain places, since the symmetric
difference is a global measure of average closeness between two sets. Some
applications may need a more local or spatially uniform error measure as
provided by the Hausdorff metric, for example, to preserve topological prop-
erties of the level set as in clustering [6, 8, 21] or ensure robustness to outliers
in level set-based anomaly detection [16, 20, 24] and data ranking [11]. The
Hausdorff error metric is defined as follows between two nonempty sets:
?
where ρ(x,G) = infy∈G?x − y?, the smallest Euclidean distance of a point
in G to the point x. If G1or G2is empty, then let d∞(G1,G2) be defined
as the largest distance between any two points in the domain. Control of
this error measure provides a uniform mode of convergence, as it implies
control of the deviation of a single point from the desired set. A symmetric
set difference-based estimator may not provide such a uniform control as it is
easy to see that a set estimate can have a very small measure of symmetric
difference error but large Hausdorff error. Conversely, as long as the set
boundary is not space filling and the domain is bounded, small Hausdorff
error implies small symmetric-difference measure.
Existing results pertaining to nonparametric level set estimation using
the Hausdorff metric [2, 9, 23] focus on rather restrictive classes of level sets
(e.g., the boundary fragment and star-shaped set classes). These restrictions,
which effectively reduce the set estimation problem to a boundary function
i=1, such that the
γ, as assessed by someerror between the estimator?G and the target set G∗
d∞(G1,G2) = maxsup
x∈G2
ρ(x,G1), sup
x∈G1
ρ(x,G2)
?
,

Page 3

ADAPTIVE HAUSDORFF ESTIMATION OF DENSITY LEVEL SETS
3
estimation problem (in rectangular or polar coordinates, resp.), are typi-
cally not met in practical applications. In particular, the characterization
of level set estimation as a boundary function estimation problem requires
prior knowledge of a reference coordinate or interior point (in rectangular
or polar coordinates, resp.) and precludes level sets with multiple connected
components. Moreover, the estimation techniques proposed in [2, 9, 23] re-
quire precise knowledge of the regularity of the density (quantified by the
parameter α, to be defined below) in the vicinity of the desired level in order
to achieve minimax optimal rates of convergence. Such prior knowledge is
unavailable in most practical applications. Recently, a plug-in method based
on sup-norm density estimation was put forth in [3] that can handle more
general classes than boundary fragments or star-shaped sets. However, sup-
norm density estimation requires the density to satisfy global smoothness
assumptions. Also, the method only deals with a special case of the density
regularity condition considered in this paper (α = 1) and is therefore not
adaptive to unknown density regularity.
In this paper, we propose a plug-in procedure based on a regular histogram
partition that can adaptively achieve minimax optimal rates of Hausdorff er-
ror convergence over a broad class of level sets with very general shapes and
multiple connected components, without assuming a priori knowledge of the
density regularity parameter α. Adaptivity is achieved by a new data-driven
procedure for selecting the histogram resolution. The procedure bears some
similarity to Lepski-type methods [10], as further discussed in Section 3.2.
However, our procedure is specifically designed for the level set estimation
problem and only requires local regularity of the density in the vicinity of
the desired level. A shorter version of this paper appeared in [17]; however,
it relies on more stringent assumptions on the class of level sets under con-
sideration. In this paper, we generalize the class of level sets to allow for
spatial variations in the density regularity along the level set boundary, and
we also discuss extensions to support set estimation and discontinuity in the
density at all points around the level of interest.
The paper is organized as follows. Section 2 states our basic assumptions
which allow Hausdorff accurate level set estimation and presents a minimax
lower bound on the Hausdorff performance of any level set estimator for
the class of densities under consideration. In Section 3, we present the pro-
posed histogram-based approach to Hausdorff accurate level set estimation.
In Section 3.1, we show that the proposed estimator can achieve the mini-
max optimal rate of convergence given knowledge of the density regularity
parameter α, and Section 3.2 extends the estimator to achieve adaptivity
to unknown density regularity. We also comment on extensions that address
discontinuity in the density at the level of interest and support set estima-
tion. Concluding remarks are given in Section 4 and the Appendices contain
proofs of the main results.

Page 4

4
A. SINGH, C. SCOTT AND R. NOWAK
2. Density assumptions. We assume that the domain of the density f is
the unit hypercube in d dimensions, that is, X = [0,1]d. Extensions to other
compact domains are straightforward. Furthermore, the density is assumed
to be bounded with range [0,fmax], though we do not assume knowledge of
fmax. Controlling the Hausdorff accuracy of level set estimates requires some
smoothness assumptions on the density and the level set boundary, which
are stated below. Before that, we introduce the following definitions:
• ε-ball: An ε-ball centered at a point x ∈ X is defined as
B(x,ε) = {y ∈ X :?x−y? ≤ ε}.
Here ?·? denotes the Euclidean distance.
• Inner ε-cover: An inner ε-cover of a set G ⊆ X is defined as the union of
all ε-balls contained in G. Formally,
Iε(G) =
?
x: B(x,ε)⊆G
B(x,ε).
We are now ready to state the assumptions. The first one characterizes
the relationship between distances and changes in density, and the second
one is a topological assumption on the level set boundary that essentially
generalizes the notion of Lipschitz functions to closed hypersurfaces.
[A] Local density regularity. The density is α-regular around the γ-level set,
0 < α < ∞ and 0 < γ < fmax, if:
[A1] there exist constants C1,δ1> 0 such that for all x ∈ X with |f(x)−
γ| ≤ δ1,
|f(x)− γ| ≥ C1ρ(x,∂G∗
where ∂G∗
[A2] there exist constants C2,δ2> 0 and x0∈ ∂G∗
x ∈ B(x0,δ2),
|f(x)− γ| ≤ C2ρ(x,∂G∗
This condition characterizes the behavior of the density around the level
γ. Assumption [A1] states that the density cannot be arbitrarily “flat”
around the level, and changes as at least the αth power of the distance
from the level set boundary. Assumption [A2] states that there exists
a fixed neighborhood around some point on the boundary where the
density changes no faster than the αth power of the distance from the
level set boundary. The latter condition is only required for adaptivity, as
we discuss later. The regularity parameter α determines the rate of error
convergence for level set estimation. Accurate estimation is more difficult
at levels where the density is relatively flat (large α), as intuition would
γ)α,
γdenotes the boundary of the true level set G∗
γ.
γsuch that for all
γ)α.

Page 5

ADAPTIVE HAUSDORFF ESTIMATION OF DENSITY LEVEL SETS
5
suggest. It is important to point out that in this paper we do not assume
knowledge of α, unlike previous investigations into Hausdorff accurate
level set estimation [2, 3, 9, 23]. Therefore, here the assumption simply
states that there is a relationship between distance and density level,
but the precise nature of the relationship is unknown. In Section 3, we
briefly discuss extensions to address the case α = 0 which corresponds to
discontinuity in the density at all points around the level set boundary
and the case γ = 0 which corresponds to support set estimation.
[B] Level set regularity. There exist constants εo> 0 and C3> 0 such that
for all ε ≤ εo, Iε(G∗
assumption implies that the level set is not arbitrarily narrow anywhere.
It precludes space-filling boundaries and features like cusps, arbitrarily
thin ribbons and isolated connected components of arbitrarily small size.
This condition is necessary since arbitrarily small features cannot be
detected and resolved from a finite sample.
γ) ?= ∅, and for all x ∈ ∂G∗
γ, ρ(x,Iε(G∗
γ)) ≤ C3ε. This
For a fixed set of positive numbers C1, C2, C3, ε0, δ1, δ2, fmax, γ < fmax, d
and α, we consider the following classes of densities.
Definition 1.
tions [A1] and [B].
F∗
1(α) denotes the class of densities satisfying assump-
Definition 2.
tions [A1], [A2] and [B].
F∗
2(α) denotes the class of densities satisfying assump-
The dependence on other parameters is omitted as these do not influence
the minimax optimal rate of convergence (except for the dimension d). In
the paper, we present a method that provides minimax optimal rates of
convergence for the class F∗
parameter α. We also extend the method to achieve adaptivity to α for the
class F∗
Assumption [A] is similar to the one employed in [2, 23], except that the
upper bound assumption on the density deviation in [2, 23] holds provided
that the set {x:|f(x)−γ| ≤ δ1} is nonempty. This implies that the densities
either jump across the level γ at any point on the level set boundary (i.e.,
the deviation is greater than δ1) or change exactly as the αth power of the
distance from the boundary. Our formulation allows for densities with reg-
ularities that vary spatially along the level set boundary—it requires that
the density changes no slower than the αth power of the distance from the
boundary, except in a fixed neighborhood of one point where the density
changes exactly as the αth power of the distance from the boundary. While
the formulation in [2, 23] requires the upper bound on the density deviation
to hold for at least one point on the boundary, our assumption [A2] requires
1(α), given knowledge of the density regularity
2(α), while preserving the minimax optimal performance.