No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Nonparametric density estimation with region-censored data. : 1098-1102
Abstract
The paper proposes a new Maximum Entropy estimator for non-parametric density estimation from region censored observations in the context of population studies, where standard Maximum Likelihood is affected by over-fitting and non-uniqueness problems.
The link between Maximum Entropy and Maximum Likelihood estimation for the exponential family has often been invoked in the literature. When, as it is the case for censored observations, the constraints on the Maximum Entropy estimator are derived from independent observations of a set of non-linear functions, this link is lost increasing the difference between the two criteria. By combining the two criteria we propose a novel density estimator that is able to overcome the singularities of the Maximum Likelihood estimator while maintaining a good fit to the observed data, and illustrate its behavior in real data (hyperbaric diving).
... Presented in [22]) Figure 17shows the densities obtained for the real dataset by the three different estimators discussed in the previous sections: the least informative NPMLEˆπNPMLEˆ NPMLEˆπ L θ , the minimally-regularized , leads to a much smoother solution, resembling π θ,Q and covering nearly all of the domain, which seems to provide a more natural model of a biological population than the solution found by the two other estimators. For the dataset sizes of our study with Q = 665, we observed very fast convergence of (11) for the complete Q (35 iterations for δ = 10 −4 ), confirming the applicability of the proposed algorithm. ...
The paper proposes a new non-parametric density estimator from region-censored observations with application in the context of population studies, where standard maximum likelihood is affected by over-fitting and non-uniqueness problems. It is a maximum entropy estimator that satisfies a set of constraints imposing a close fit to the empirical distributions associated with the set of censoring regions. The degree of relaxation of the data-fit constraints is chosen, such that the likelihood of the inferred model is maximal. In this manner, the estimator is able to overcome the singularity of the non-parametric maximum likelihood estimator and, at the same time, maintains a good fit to the observations. The behavior of the estimator is studied in a simulation, demonstrating its superior performance with respect to the non-parametric maximum likelihood and the importance of carefully choosing the degree of relaxation of the data-fit constraints. In particular, the predictive performance of the resulting estimator is better, which is important when the population analysis is done in the context of risk assessment. We also apply the estimator to real data in the context of the prevention of hyperbaric decompression sickness, where the available observations are formally equivalent to region-censored versions of the variables of interest, confirming that it is a superior alternative to non-parametric maximum likelihood in realistic situations.
An approach to the Shannon and Rényi entropy maximization problems with constraints on the mean and law-invariant deviation measure for a random variable has been developed. The approach is based on the representation of law-invariant deviation measures through corresponding convex compact sets of nonnegative concave functions. A solution to the problem has been shown to have an alpha-concave distribution (log-concave for Shannon entropy), for which in the case of comonotone deviation measures, an explicit formula has been obtained. As an illustration, the problem has been solved for several deviation measures, including mean absolute deviation (MAD), conditional value-at-risk (CVaR) deviation, and mixed CVaR-deviation. Also, it has been shown that the maximum entropy principle establishes a one-to-one correspondence between the class of alpha-concave distributions and the class of comonotone deviation measures. This fact has been used to solve the inverse problem of finding a corresponding comonotone deviation measure for a given alpha-concave distribution.
Two variations of a simple monotunic algorithm for computing optimal designs on a finite design space are presented. Various properties are listed. Comparisons witn other algorithms are made.
We consider the problem of estimating an unknown probability dis- tribution from samples using the principle of maximum entropy (maxent). To alleviate overfitting with a very large number of features, w e propose applying the maxent principle with relaxed constraints on the expectations of the features. By convex duality, this turns out to be equivalent to finding t he Gibbs distribu- tion minimizing a regularized version of the empirical log loss. We prove non- asymptotic bounds showing that, with respect to the true underlying distribu- tion, this relaxed version of maxent produces density estimates that are almost as good as the best possible. These bounds are in terms of the deviation of the feature empirical averages relative to their true expectat ions, a number that can be bounded using standard uniform-convergence techniques. In particular, this leads to bounds that drop quickly with the number of samples, and that depend very moderately on the number or complexity of the features. We also derive and prove convergence for both sequential-update and parallel-update algorithms. Fi- nally, we briefly describe experiments on data relevant to th e modeling of species geographical distributions.
We improve the inequality used in Pronzato [2003. Removing non-optimal support points in D-optimum design algorithms. Statist. Probab. Lett. 63, 223-228] to remove points from the design space during the search for a D-optimum design. Let [xi] be any design on a compact space with a nonsingular information matrix, and let m+[epsilon] be the maximum of the variance function d([xi],x) over all . We prove that any support point x* of a D-optimum design on must satisfy the inequality . We show that this new lower bound on d([xi],x*) is, in a sense, the best possible, and how it can be used to accelerate algorithms for D-optimum design.
In this paper methods for finding the non--parametric maximum likelihood estimate (NPMLE) of the distribution function of time to event data will be presented. The basic approach is to use graph theory (in particular intersection graphs) to simplify the problem. Censored data can be represented in terms of their intersection graph. Existing combinatorial algorithms can be used to find the important structures, namely the maximal cliques. When viewed in this framework there is no fundamental difference between right censoring, interval censoring, double censoring or current status data and hence the algorithms apply to all types of data. These algorithms can be extended to deal with bivariate data and indeed there are no fundamental problems extending the methods to higher dimensional data. Finally we will show how to obtain the NPMLE using convex optimization methods and methods for mixing distributions. The implementation of these methods is greatly simplified through the ...