-
[show abstract]
[hide abstract]
ABSTRACT: Inter-sensor communication often comprises a significant portion of energy expenditures in a sensor network as compared to sensing and computation. We discuss an integrated approach to dynamically routing measurements and models in a sensor network. Specifically, we examine the problem of tracking objects within a region wherein the responsibility for combining measurements and updating a posterior state distribution is assigned to a single sensor at any given time step. The so called leader node may change over time. Sensor nodes communicate for two reasons: firstly, measurements of target state are transmitted from sensors to the current leader node for incorporation into the state estimate model; secondly, the state model is transmitted between sensors when the leader node changes. The trade-off between these two types of communication is of primary importance to dynamic selection of the leader node. We propose an algorithm based on a dynamic programming roll-out formulation of the minimum cost problem. We obtain a cost function which can be efficiently minimized by simplifying the problem to that of an open loop feedback controller which is an upper bound to the optimal cost. We present empirical results which compare methods previously proposed in the literature to the algorithm presented here.
Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP '05). IEEE International Conference on; 04/2005 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Graphical models provide a powerful general framework for encoding the structure of large-scale estimation problems. However, the graphs describing typical real-world phenomena contain many cycles, making direct estimation procedures prohibitively costly. In this paper, we develop an iterative inference algorithm for general Gaussian graphical models. It operates by exactly solving a series of modified estimation problems on spanning trees embedded within the original cyclic graph. When these subproblems are suitably chosen, the algorithm converges to the correct conditional means. Moreover, and in contrast to many other iterative methods, the tree-based procedures we propose can also be used to calculate exact error variances. Although the conditional mean iteration is effective for quite densely connected graphical models, the error variance computation is most efficient for sparser graphs. In this context, we present a modeling example suggesting that very sparsely connected graphs with cycles may provide significant advantages relative to their tree-structured counterparts, thanks both to the expressive power of these models and to the efficient inference algorithms developed herein. The convergence properties of the proposed tree-based iterations are characterized both analytically and experimentally. In addition, by using the basic tree-based iteration to precondition the conjugate gradient method, we develop an alternative, accelerated iteration that is finitely convergent. Simulation results are presented that demonstrate this algorithm's effectiveness on several inference problems, including a prototype distributed sensing application.
IEEE Transactions on Signal Processing 12/2004; · 2.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Determining the structure of dependencies among a set of variables is a common task in many signal and image processing applications, including multitarget tracking and computer vision. In this paper, we present an information-theoretic, machine learning approach to problems of this type. We cast this problem as a hypothesis test between factorizations of variables into mutually independent subsets. We show that the likelihood ratio can be written as sums of two sets of Kullback-Leibler (KL) divergence terms. The first set captures the structure of the statistical dependencies within each hypothesis, whereas the second set measures the details of model differences between hypotheses. We then consider the case when the signal prior models are unknown, so that the distributions of interest must be estimated directly from data, showing that the second set of terms is (asymptotically) negligible and quantifying the loss in hypothesis separability when the models are completely unknown. We demonstrate the utility of nonparametric estimation methods for such problems, providing a general framework for determining and distinguishing between dependency structures in highly uncertain environments. Additionally, we develop a machine learning approach for estimating lower bounds on KL divergence and mutual information from samples of high-dimensional random variables for which direct density estimation is infeasible. We present empirical results in the context of three prototypical applications: association of signals generated by sources possessing harmonic behavior, scene correspondence using video imagery, and detection of coherent behavior among sets of moving objects.
IEEE Transactions on Signal Processing 09/2004; · 2.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We consider the problem of enforcing a sparsity prior in underdetermined linear problems, which is also known as sparse signal representation in overcomplete bases. The problem is combinatorial in nature, and a direct approach is computationally intractable, even for moderate data sizes. A number of approximations have been considered in the literature, including stepwise regression, matching pursuit and its variants, and, recently, basis pursuit (ℓ<sub>1</sub>) and also ℓ<sub>p</sub>-norm relaxations with p<1. Although the exact notion of sparsity (expressed by an ℓ<sub>0</sub>-norm) is replaced by ℓ<sub>1</sub> and ℓ<sub>p</sub> norms in the latter two, it can be shown that under some conditions these relaxations solve the original problem exactly. The seminal paper of D.L. Donoho and X. Huo (see Stanford Univ. Tech. report: http://www-sccm.stanford.edu/pub/sccm/sccm02-17.pdf) establishes this fact for ℓ<sub>1</sub> (basis pursuit) for a special case where the linear operator is composed of an orthogonal pair. We extend their results to a general underdetermined linear operator. Furthermore, we derive conditions for the equivalence of ℓ<sub>0</sub> and ℓ<sub>p</sub> problems, and extend the results to the problem of enforcing sparsity with respect to a transformation (which includes total variation priors as a special case). Finally, we describe an interesting result relating the sign patterns of solutions to the question of ℓ<sub>1</sub>-ℓ<sub>0</sub> equivalence.
Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP '04). IEEE International Conference on; 06/2004 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Automatic self-calibration of ad-hoc sensor networks is a critical need for their use in military or civilian applications. In general, self-calibration involves the combination of absolute location information (e.g. GPS) with relative calibration information (e.g. time delay or received signal strength between sensors) over regions of the network. Furthermore, it is generally desirable to distribute the computational burden across the network and minimize the amount of inter-sensor communication. We demonstrate that the information used for sensor calibration is fundamentally local with regard to the network topology and use this observation to reformulate the problem within a graphical model framework. We then demonstrate the utility of nonparametric belief propagation (NBP), a recent generalization of particle filtering, for both estimating sensor locations and representing location uncertainties. NBP has the advantage that it is easily implemented in a distributed fashion, admits a wide variety of statistical models, and can represent multi-modal uncertainty. We illustrate the performance of NBP on several example networks while comparing to a previously published nonlinear least squares method.
Information Processing in Sensor Networks, 2004. IPSN 2004. Third International Symposium on; 05/2004
-
[show abstract]
[hide abstract]
ABSTRACT: We present a source localization method based upon a sparse representation of sensor measurements with an overcomplete basis composed of samples from the array manifold. We enforce sparsity by imposing an ℓ<sub>1</sub>-norm penalty; this can also be viewed as an estimation problem with a Laplacian prior. Explicitly enforcing the sparsity of the representation is motivated by a desire to obtain a sharp estimate of the spatial spectrum which exhibits superresolution. To summarize multiple time samples we use the singular value decomposition (SVD) of the data matrix. Our formulation leads to an optimization problem, which we solve efficiently in a second-order cone (SOC) programming framework by an interior point implementation. We demonstrate the effectiveness of the method on simulated data by plots of spatial spectra and by comparing the estimator variance to the Cramer-Rao bound (CRB). We observe that our approach has advantages over other source localization techniques including increased resolution; improved robustness to noise, limitations in data quantity, and correlation of the sources; as well as not requiring an accurate initialization.
Statistical Signal Processing, 2003 IEEE Workshop on; 11/2003
-
[show abstract]
[hide abstract]
ABSTRACT: An information-theoretic method for multiphase image segmentation, in an active contour-based framework is proposed. Our approach is based on nonparametric density estimates, and is able to solve problems involving arbitrary probability densities for the region intensities. This is achieved by maximizing the mutual information between the region labels and the image pixel intensities, in order to segment up to 2<sup>m</sup> regions using m curves. The method does not require any prior training regarding the regions of interest, but rather learns the probability densities during the evolution process. We present some illustrative experimental results, demonstrating the power of the proposed segmentation approach.
Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on; 10/2003
-
[show abstract]
[hide abstract]
ABSTRACT: In many applications of graphical models arising in computer vision, the hidden variables of interest are most naturally specified by continuous, non-Gaussian distributions. There exist inference algorithms for discrete approximations to these continuous distributions, but for the high-dimensional variables typically of interest, discrete inference becomes infeasible. Stochastic methods such as particle filters provide an appealing alternative. However, existing techniques fail to exploit the rich structure of the graphical models describing many vision problems. Drawing on ideas from regularized particle filters and belief propagation (BP), this paper develops a nonparametric belief propagation (NBP) algorithm applicable to general graphs. Each NBP iteration uses an efficient sampling procedure to update kernel-based approximations to the true, continuous likelihoods. The algorithm can accommodate an extremely broad class of potential functions, including nonparametric representations. Thus, NBP extends particle filtering methods to the more general vision problems that graphical models can describe. We apply the NBP algorithm to infer component interrelationships in a parts-based face model, allowing location and reconstruction of occluded features.
Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on; 07/2003
-
[show abstract]
[hide abstract]
ABSTRACT: We present a tree-based reparameterization (TRP) framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation (BP) or sum-product algorithm as well as variations and extensions of BP. Algorithms in this class can be formulated as a sequence of reparameterization updates, each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph (i.e., a tree, or more generally, a hypertree). The ultimate goal is to obtain an alternative but equivalent factorization using functions that represent (exact or approximate) marginal distributions on cliques of the graph. Our framework highlights an important property of the sum-product algorithm and the larger class of reparameterization algorithms: the original distribution on the graph with cycles is not changed. The perspective of tree-based updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency. We develop interpretations of these results in terms of information geometry. The invariance of the distribution, in conjunction with the fixed-point characterization, enables us to derive an exact expression for the difference between the true marginals on an arbitrary graph with cycles, and the approximations provided by belief propagation. More broadly, our analysis applies to any algorithm that minimizes the Bethe free energy. We also develop bounds on the approximation error, which illuminate the conditions that govern their accuracy. Finally, we show how the reparameterization perspective extends naturally to generalizations of BP (e.g., Kikuchi (1951) approximations and variants) via the notion of hypertree reparameterization.
IEEE Transactions on Information Theory 06/2003; · 3.01 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Efficient computation of extensions of banded, partially known covariance matrices is provided by the classical Levinson algorithm. One contribution of this paper is the introduction of a generalization of this algorithm that is applicable to a substantially broader class of extension problems. This generalized algorithm can compute unknown covariance elements in any order that satisfies certain graph-theoretic properties, which we describe. This flexibility, which is not provided by the classical Levinson algorithm, is then harnessed in a second contribution of this paper, the identification of a multiscale autoregressive (MAR) model for the maximum-entropy (ME) extension of a banded, partially known covariance matrix. The computational complexity of MAR model identification is an order of magnitude below that of explicitly computing a full covariance extension and is comparable to that required to build a standard autoregressive (AR) model using the classical Levinson algorithm.
IEEE Transactions on Information Theory 03/2003; · 3.01 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We address the task of source localization using a novel non-parametric data-adaptive approach based on regularized linear inverse problems with sparsity constraints. The class of penalty functions that we use for regularization favors sparsity of the reconstructions, thus producing superb resolution of the sources. We present a computationally efficient technique to carry out the numerical optimization of the resulting cost function. In comparison to conventional source localization methods, the proposed approach provides numerous improvements, including increased resolution, reduced sidelobes, and better robustness properties to noise, limited snapshots, and coherence of the sources. The method is developed for the general source localization scenario, encompassing nearfield and farfield, narrowband and broadband, and non-linear array geometry cases. Simulation results manifest the capabilities of the approach.
Sensor Array and Multichannel Signal Processing Workshop Proceedings, 2002; 09/2002
-
[show abstract]
[hide abstract]
ABSTRACT: We present a novel information theoretic approach to image segmentation. We cast the segmentation problem as the maximization of the mutual information between the region labels and the image pixel intensities, subject to a constraint on the total length of the region boundaries. We assume that the probability densities associated with the image pixel intensities within each region are completely unknown a priori, and we formulate the problem based on nonparametric density estimates. Due to the nonparametric structure, our method does not require the image regions to have a particular type of probability distribution, and does not require the extraction and use of a particular statistic. We solve the information-theoretic optimization problem by deriving the associated gradient flows and applying curve evolution techniques. We use fast level set methods to implement the resulting evolution The evolution equations are based on nonparametric statistics, and have an intuitive appeal. The experimental results based on both synthetic and real images demonstrate that the proposed technique can solve a variety of challenging image segmentation problems.
Image Processing. 2002. Proceedings. 2002 International Conference on; 07/2002
-
[show abstract]
[hide abstract]
ABSTRACT: We propose a method for edge-preserving regularized reconstruction in coherent imaging systems. In our framework, image formation from measured data is achieved through the minimization of a cost function, which includes nonquadratic regularizing constraints for suppressing noise artifacts, while preserving the object boundaries in the reconstruction. The cost function we use effectively deals with the complex-valued and random-phase nature of the scattered field, which is inherent in many coherent systems. We solve the challenging optimization problems posed in our framework by a novel extension of half-quadratic regularization methods. We present experimental results from three coherent imaging applications: digital holography, synthetic aperture radar, and medical ultrasound. The proposed technique produces images where coherent speckle artifacts are effectively suppressed, and boundaries between different regions in the scene are preserved.
Image Processing. 2002. Proceedings. 2002 International Conference on; 02/2002
-
[show abstract]
[hide abstract]
ABSTRACT: In this paper, we introduce a novel approach for simultaneous restoration and segmentation of blurred noisy images by approaching a variant of the Mumford-Shah functional from a curve evolution perspective. In particular, by viewing the active contour as the set of discontinuities in the image, we derive a gradient flow to minimize an extended Mumford-Shah functional where the known blurring function is incorporated as part of the data fidelity term. Each gradient step involves solving a discrete approximation of the corresponding partial differential equation to obtain a smooth and deblurred estimate of the observed image without blurring across the curve. The experimental results based on both synthetic and real images demonstrate that the proposed method segments and restores the blurred images effectively. We conclude that our work is an edge-preserving image restoration technique that couples segmentation, denoising, and deblurring within a single framework. In addition, this framework provides an intellectual connection between regularization theory (used to solve the deblurring inverse problem) and the theory of curve evolution.
Image Processing. 2002. Proceedings. 2002 International Conference on; 02/2002
-
[show abstract]
[hide abstract]
ABSTRACT: We develop a realization theory for a class of multiscale
stochastic processes having white-noise driven, scale-recursive dynamics
that are indexed by the nodes of a tree. Given the correlation structure
of a 1-D or 2-D random process, our methods provide a systematic way to
realize the given correlation as the finest scale of a multiscale
process. Motivated by Akaike's use of canonical correlation analysis to
develop both exact and reduced-order models for time-series, we too
harness this tool to develop multiscale models. We apply our realization
scheme to build reduced-order multiscale models for two applications,
namely linear least-squares estimation and generation of random-field
sample paths. For the numerical examples considered, least-squares
estimates are obtained having nearly optimal mean-square errors, even
with multiscale models of low order. Although both field estimates and
field sample paths exhibit a visually distracting blockiness, this
blockiness is not an important issue in many applications. For such
applications, our approach to multiscale stochastic realization holds
promise as a valuable, general tool
IEEE Transactions on Automatic Control 11/2001; · 2.11 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We first address the problem of simultaneous image segmentation
and smoothing by approaching the Mumford-Shah (1989) paradigm from a
curve evolution perspective. In particular, we let a set of deformable
contours define the boundaries between regions in an image where we
model the data via piecewise smooth functions and employ a gradient flow
to evolve these contours. Each gradient step involves solving an optimal
estimation problem for the data within each region, connecting curve
evolution and the Mumford-Shah functional with the theory of
boundary-value stochastic processes. The resulting active contour model
offers a tractable implementation of the original Mumford-Shah model
(i.e., without resorting to elliptic approximations which have
traditionally been favored for greater ease in implementation) to
simultaneously segment and smoothly reconstruct the data within a given
image in a coupled manner. Various implementations of this algorithm are
introduced to increase its speed of convergence. We also outline a
hierarchical implementation of this algorithm to handle important image
features such as triple points and other multiple junctions. Next, by
generalizing the data fidelity term of the original Mumford-Shah
functional to incorporate a spatially varying penalty, we extend our
method to problems in which data quality varies across the image and to
images in which sets of pixel measurements are missing. This more
general model leads us to a novel PDE-based approach for simultaneous
image magnification, segmentation, and smoothing, thereby extending the
traditional applications of the Mumford-Shah functional which only
considers simultaneous segmentation and smoothing
IEEE Transactions on Image Processing 09/2001; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We present extensions to our previous work in modelling dynamical
processes. The approach uses an information theoretic criterion for
searching over subspaces of the past observations, combined with a
nonparametric density characterizing its relation to one-step-ahead
prediction and uncertainty. We use this methodology to model handwriting
stroke data, specifically signatures, as a dynamical system and show
that it is possible to learn a model capturing their dynamics for use
either in synthesizing realistic signatures and in discriminating
between signatures and forgeries even though no forgeries have been used
in constructing the model. This novel approach yields promising results
even for small training sets
Acoustics, Speech, and Signal Processing, 2001. Proceedings. (ICASSP '01). 2001 IEEE International Conference on; 02/2001 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In this work, we first address the problem of simultaneous image segmentation and smoothing by approaching the Mumford-Shah paradigm from a curve evolution perspective. In particular, we let a set of deformable contours define the boundaries between regions in an image where we model the data via piecewise smooth functions and employ a gradient flow to evolve these contours. Each gradient step involves solving an optimal estimation problem for the data within each region, connecting curve evolution and the Mumford-Shah functional with the theory of boundary-value stochastic processes. The resulting active contour model offers a tractable implementation of the original Mumford-Shah model (i.e., without resorting to elliptic approximations which have traditionally been favored for greater ease in implementation) to simultaneously segment and smoothly reconstruct the data within a given image in a coupled manner. Various implementations of this algorithm are introduced to increase its speed of convergence. We also outline a hierarchical implementation of this algorithm to handle important image features such as triple points and other multiple junctions. Next, by generalizing the data fidelity term of the original Mumford-Shah functional to incorporate a spatially varying penalty, we extend our method to problems in which data quality varies across the image and to images in which sets of pixel measurements are missing. This more general model leads us to a novel PDE-based approach for simultaneous image magnification, segmentation, and smoothing, thereby extending the traditional applications of the Mumford-Shah functional which only considers simultaneous segmentation and smoothing.
IEEE Transactions on Image Processing 02/2001; 10(8):1169-86. · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: This paper addresses the problem of both segmenting and
reconstructing a noisy signal or image. The work is motivated by large
problems arising in certain scientific applications, such as medical
imaging. Two objectives for a segmentation and denoising algorithm are
laid out: it should be computationally efficient and capable of
generating statistics for the errors in the reconstruction and estimates
of the boundary locations. The starting point for the development of a
suitable algorithm is a variational approach to segmentation (Shah
1992). This paper then develops a precise statistical interpretation of
a one dimensional (1-D) version of this variational approach to
segmentation. The 1-D algorithm that arises as a result of this analysis
is computationally efficient and capable of generating error statistics.
A straightforward extension of this algorithm to two dimensions would
incorporate recursive procedures for computing estimates of
inhomogeneous Gaussian Markov random fields. Such procedures require an
unacceptably large number of operations. To meet the objective of
developing a computationally efficient algorithm, the use of previously
developed multiscale statistical methods is investigated. This results
in the development of an algorithm for segmenting and denoising which is
not only computationally efficient but also capable of generating error
statistics, as desired
IEEE Transactions on Image Processing 04/2000; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We introduce a family of first-order multidimensional ordinary
differential equations (ODEs) with discontinuous right-hand sides and
demonstrate their applicability in image processing. An equation
belonging to this family is an inverse diffusion everywhere except at
local extrema, where some stabilization is introduced. For this reason,
we call these equations “stabilized inverse diffusion
equations” (SIDEs). Existence and uniqueness of solutions, as well
as stability, are proven for SIDEs. A SIDE in one spatial dimension may
be interpreted as a limiting case of a semi-discretized Perona-Malik
equation (1990, 19994). In an experiment, SIDE's are shown to suppress
noise while sharpening edges present in the input signal. Their
application to image segmentation is also demonstrated
IEEE Transactions on Image Processing 03/2000; · 3.04 Impact Factor
