Article

Adaptive rejection Metropolis sampling using Lagrange interpolation polynomials of degree 2

Department of Statistics, University of Auckland, Private Bag 92019, Auckland, New Zealand
Computational Statistics & Data Analysis (Impact Factor: 1.4). 03/2008; 52(7):3408-3423. DOI: 10.1016/j.csda.2008.01.005
Source: DBLP

ABSTRACT

A crucial problem in Bayesian posterior computation is efficient sampling from a univariate distribution, e.g. a full conditional distribution in applications of the Gibbs sampler. This full conditional distribution is usually non-conjugate, algebraically complex and computationally expensive to evaluate. We propose an alternative algorithm, called ARMS2, to the widely used adaptive rejection sampling technique ARS [Gilks, W.R., Wild, P., 1992. Adaptive rejection sampling for Gibbs sampling. Applied Statistics 41 (2), 337–348; Gilks, W.R., 1992. Derivative-free adaptive rejection sampling for Gibbs sampling. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (Eds.), Bayesian Statistics, Vol. 4. Clarendon, Oxford, pp. 641–649] for generating a sample from univariate log-concave densities. Whereas ARS is based on sampling from piecewise exponentials, the new algorithm uses truncated normal distributions and makes use of a clever auxiliary variable technique [Damien, P., Walker, S.G., 2001. Sampling truncated normal, beta, and gamma densities. Journal of Computational and Graphical Statistics 10 (2) 206–215]. Furthermore, we extend this algorithm to deal with non-log-concave densities to provide an enhanced alternative to adaptive rejection Metropolis sampling, ARMS [Gilks, W.R., Best, N.G., Tan, K.K.C., 1995. Adaptive rejection Metropolis sampling within Gibbs sampling. Applied Statistics 44, 455–472]. The performance of ARMS and ARMS2 is compared in simulations of standard univariate distributions as well as in Gibbs sampling of a Bayesian hierarchical state-space model used for fisheries stock assessment.

Full-text preview

Available from: sciencedirect.com
  • Source
    • "Iterations (while n < N ): 2. Build a proposal, πt(x), given the set St = {s 1 , . . . , sm t }, using a convenient procedure (e.g. the ones described in [28], [30] or the simpler ones proposed in Section III-C). 3. Draw x ∼ e πt(x) ∝ πt(x) and u ∼ U ([0] [1]). "
    Full-text · Dataset · Jul 2015
  • Source
    • "2. Build a proposal, πt(x), given the set St = {s1, . . . , sm t }, using a convenient procedure (e.g. the ones described in [28], [30] or the simpler ones proposed in Section III-C). "
    [Show abstract] [Hide abstract] ABSTRACT: Bayesian methods have become very popular in signal processing lately, even though performing exact Bayesian inference is often unfeasible due to the lack of analytical expressions for optimal Bayesian estimators. In order to overcome this problem, Monte Carlo (MC) techniques are frequently used. Several classes of MC schemes have been developed, including Markov Chain Monte Carlo (MCMC) methods, particle filters and population Monte Carlo approaches. In this paper, we concentrate on the Gibbs-type approach, where automatic and fast samplers are needed to draw from univariate (full-conditional) densities. The Adaptive Rejection Metropolis Sampling (ARMS) technique is widely used within Gibbs sampling, but suffers from an important drawback: an incomplete adaptation of the proposal in some cases. In this work, we propose an alternative adaptive MCMC algorithm (IA 2 RMS) that overcomes this limitation, speeding up the convergence of the chain to the target, allowing us to simplify the construction of the sequence of proposals, and thus reducing the computational cost of the entire algorithm. Note that, although IA2RMS has been developed as an extremely efficient MCMC-within-Gibbs sampler, it also provides an excellent performance as a stand-alone algorithm when sampling from univariate distributions. In this case, the convergence of the proposal to the target is proved and a bound on the complexity of the proposal is provided. Numerical results, both for univariate (stand-alone IA2RMS) and multivariate (IA 2 RMS-within-Gibbs) distributions, show that IA2RMS outperforms ARMS and other classical techniques, providing a correlation among samples close to zero.
    Full-text · Article · Apr 2015 · IEEE Transactions on Signal Processing
  • Source
    • "(a) illustrates an example of this construction. More sophisticated approaches to build W t (x) (e.g., using quadratic segments when possible [12] "
    [Show abstract] [Hide abstract] ABSTRACT: Adaptive Rejection Metropolis Sampling (ARMS) is a well-known MCMC scheme for generating samples from one-dimensional target distributions. ARMS is widely used within Gibbs sampling, where automatic and fast samplers are of-ten needed to draw from univariate full-conditional densities. In this work, we propose an alternative adaptive algorithm (IA 2 RMS) that overcomes the main drawback of ARMS (an uncomplete adaptation of the proposal in some cases), speed-ing up the convergence of the chain to the target. Numerical results show that IA 2 RMS outperforms the standard ARMS, providing a correlation among samples close to zero. Index Terms— Monte Carlo methods, Gibbs sampler, adaptive rejection Metropolis sampling (ARMS).
    Full-text · Conference Paper · May 2014
Show more