No file available
To read the file of this research,
you can request a copy directly from the authors.
Fluctuations of Rare Event Simulation with Monte Carlo Splitting in the Small Noise Asymptotics
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
Diffusion processes with small noise conditioned to reach a target set are considered. The AMS algorithm is a Monte Carlo method that is used to sample such rare events by iteratively simulating clones of the process and selecting trajectories that have reached the highest value of a so-called importance function. In this paper, the large sample size relative variance of the AMS small probability estimator is considered. The main result is a large deviations logarithmic equivalent of the latter in the small noise asymptotics, which is rigorously derived. It is given as a maximisation problem explicit in terms of the quasi-potential cost function associated with the underlying small noise large deviations. Necessary and sufficient geometric conditions ensuring the vanishing of the obtained quantity ('weak' asymptotic efficiency) are provided. Interpretations and practical consequences are discussed.
ResearchGate has not been able to resolve any citations for this publication.
This paper focuses on estimating reach probability of a closed unsafe set by a stochastic process. A well-developed approach is to make use of multi-level MC simulation, which consists of encapsulating the unsafe set by a sequence of increasing closed sets and conducting a sequence of MC simulations to estimate the reach probability of each inner set from the previous set. An essential step is to copy (split) particles that have reached the next level (inner set) prior to conducting a MC simulation to the next level. The aim of this paper is to prove that the variance of the multi-level MC estimated reach probability under fixed assignment splitting is smaller or equal than under random assignment splitting methods. The approaches are illustrated for a geometric Brownian motion example.
We present a complete framework for determining the asymptotic (or logarithmic) efficiency of estimators of large deviation probabilities and rate functions based on importance sampling. The framework relies on the idea that importance sampling in that context is fully characterized by the joint large deviations of two random variables: the observable defining the large deviation probability of interest and the likelihood factor (or Radon–Nikodym derivative) connecting the original process and the modified process used in importance sampling. We recover with this framework known results about the asymptotic efficiency of the exponential tilting and obtain new necessary and sufficient conditions for a general change of process to be asymptotically efficient. This allows us to construct new examples of efficient estimators for sample means of random variables that do not have the exponential tilting form. Other examples involving Markov chains and diffusions are presented to illustrate our results.
The average time between two occurrences of the same event, referred to as its return time (or return period), is a useful statistical concept for practical applications. For instance insurances or public agency may be interested by the return time of a 10m flood of the Seine river in Paris. However, due to their scarcity, reliably estimating return times for rare events is very difficult using either observational data or direct numerical simulations. For rare events, an estimator for return times can be built from the extrema of the observable on trajectory blocks. Here, we show that this estimator can be improved to remain accurate for return times of the order of the block size. More importantly, we show that this approach can be generalised to estimate return times from numerical algorithms specifically designed to sample rare events. So far those algorithms often compute probabilities, rather than return times. The approach we propose provides a computationally extremely efficient way to estimate numerically the return times of rare events for a dynamical system, gaining several orders of magnitude of computational costs. We illustrate the method on two kinds of observables, instantaneous and time-averaged, using two different rare event algorithms, for a simple stochastic process, the Ornstein-Uhlenbeck process. As an example of realistic applications to complex systems, we finally discuss extreme values of the drag on an object in a turbulent flow.
We apply the adaptive multilevel splitting (AMS) method to the Ceq → Cax transition of alanine dipeptide in vacuum. Some properties of the algorithm are numerically illustrated, such as the unbiasedness of the probability estimator and the robustness of the method with respect to the reaction coordinate. We also calculate the transition time obtained via the probability estimator, using an appropriate ensemble of initial conditions. Finally, we show how the AMS method can be used to compute an approximation of the committor function. © 2019 Wiley Periodicals, Inc.
In the Monte Carlo simulation of particle transport, and especially for shielding applications, variance reduction techniques are widely used to help simulate realisations of rare events and reduce the relative errors on the estimated scores for a given computation time. Adaptive Multilevel Splitting is one of these variance reduction techniques that has recently appeared in the literature. In the present paper, we propose an alternative version of the AMS algortihm, adapted for the first time to the field of particle tranport. Within this context, it can be used to build an unbiased estimator of any quantity associated with particle tracks, such as flux, reaction rates or even non-Boltzmann tallies. Furthermore, the efficiency of the AMS algorithm is shown not to be very sensitive to variations of its input parameters, which makes it capable of significant variance reduction without requiring extended user effort.
We introduce a generalization of the Adaptive Multilevel Splitting algorithm
in the discrete time dynamic setting, namely when it is applied to sample rare
events associated with paths of Markov chains. By interpreting the algorithm as
a sequential sampler in path space, we are able to build an estimator of the
rare event probability (and of any non-normalized quantity associated with this
event) which is unbiased, whatever the choice of the importance function and
the number of replicas. This has practical consequences on the use of this
algorithm, which are illustrated through various numerical experiments.
The Adaptive Multilevel Splitting algorithm is a very powerful and versatile
method to estimate rare events probabilities. It is an iterative procedure on
an interacting particle system, where at each step, the k less well-adapted
particles among n are killed while k new better adapted particles are
resampled according to a conditional law. We analyze the algorithm in the
idealized setting of an exact resampling and prove that the estimator of the
rare event probability is unbiased whatever k. We also obtain a precise
asymptotic expansion for the variance of the estimator and the cost of the
algorithm in the large n limit, for a fixed k.
We investigate a class of simple models for Langevin dynamics of turbulent
flows, including the one-layer quasi-geostrophic equation and the
two-dimensional Euler equations. Starting from a path integral representation
of the transition probability, we compute the most probable fluctuation paths
from one attractor to any state within its basin of attraction. We prove that
such fluctuation paths are the time reversed trajectories of the relaxation
paths for a corresponding dual dynamics, which are also within the framework of
quasi-geostrophic Langevin dynamics. Cases with or without detailed balance are
studied. We discuss a specific example for which the stationary measure
displays either a second order (continuous) or a first order (discontinuous)
phase transition and a tricritical point. In situations where a first order
phase transition is observed, the dynamics are bistable. Then, the transition
paths between two coexisting attractors are instantons (fluctuation paths from
an attractor to a saddle), which are related to the relaxation paths of the
corresponding dual dynamics. For this example, we show how one can analytically
determine the instantons and compute the transition probabilities for rare
transitions between two attractors.
We consider a general class of branching methods with killing for the estimation of rare events. The class includes a number
of existing schemes, including RESTART and DPR (Direct Probability Redistribution). A method for the design and analysis is
developed when the quantity of interest can be embedded in a sequence whose limit is determined by a large deviation principle.
A notion of subsolution for the related calculus of variations problem is introduced, and two main results are proved. One
is that the number of particles and the total work scales subexponentially in the large deviation parameter when the branching
process is constructed according to a subsolution. The second is that the asymptotic performance of the schemes as measured
by the variance of the estimate can be characterized in terms of the subsolution. Some examples are given to demonstrate the
performance of the method.
We present in this article a genetic type interacting particle systems algorithm and a genealogical model for estimating a class of rare events arising in physics and network analysis. We represent the distribution of a Markov process hitting a rare target in terms of a Feynman–Kac model in path space. We show how these branching particle models described in previous works can be used to estimate the probability of the corresponding rare events as well as the distribution of the process in this regime.
Rare events play a crucial role in many physics, chemistry, and biology phenomena, when they change the structure of the system, for instance in the case of multistability, or when they have a huge impact. Rare event algorithms have been devised to simulate them efficiently, avoiding the computation of long periods of typical fluctuations. We consider here the family of splitting or cloning algorithms, which are versatile and specifically suited for far-from-equilibrium dynamics. To be efficient, these algorithms need to use a smart score function during the selection stage. Committor functions are the optimal score functions. In this work we propose a new approach, based on the analogue Markov chain, for a data-based learning of approximate committor functions. We demonstrate that such learned committor functions are extremely efficient score functions when used with the adaptive multilevel splitting algorithm. We illustrate our approach for a gradient dynamics in a three-well potential, and for the Charney–DeVore model, which is a paradigmatic toy model of multistability for atmospheric dynamics. For these two dynamics, we show that having observed a few transitions is enough to have a very efficient data-based score function for the rare event algorithm. This new approach is promising for use for complex dynamics: the rare events can be simulated with a minimal prior knowledge and the results are much more precise than those obtained with a user-designed score function.
In molecular simulations, the identification of suitable reaction coordinates is central to both the analysis and sampling of transitions between metastable states in complex systems. If sufficient simulation data are available, a number of methods have been developed to reduce the vast amount of high-dimensional data to a small number of essential degrees of freedom representing the reaction coordinate. Likewise, if the reaction coordinate is known, a variety of approaches have been proposed to enhance the sampling along the important degrees of freedom. Often, however, neither one nor the other is available. One of the key questions is therefore, how to construct reaction coordinates and evaluate their validity. Another challenges arises from the physical interpretation of reaction coordinates, which is often addressed by correlating physically meaningful parameters with conceptually well-defined but abstract reaction coordinates. Furthermore, machine learning based methods are becoming more and more applicable also to the reaction coordinate problem. This perspective highlights central aspects in the identification and evaluation of reaction coordinates and discusses recent ideas regarding automated computational frameworks to combine the optimization of reaction coordinates and enhanced sampling.
This article first presents a short historical perpective of the importance splitting approach to simulate and estimate rare events, with a detailed description of several variants. We then give an account of recent theoretical results on these algorithms, including a central limit theorem for Adaptive Multilevel Splitting (AMS). Considering the asymptotic variance in the latter, the choice of the importance function, called the reaction coordinate in molecular dynamics, is also discussed. Finally, we briefly mention some worthwhile applications of AMS in various domains.
Adaptive Multilevel Splitting (AMS for short) is a generic Monte Carlo method for Markov processes that simulates rare events and estimates associated probabilities. Despite its practical efficiency, there are almost no theoretical results on the convergence of this algorithm. The purpose of this paper is to prove both consistency and asymptotic normality results in a general setting. This is done by associating to the original Markov process a level-indexed process, also called a stochastic wave, and by showing that AMS can then be seen as a Fleming-Viot type particle system. This being done, we can finally apply general results on Fleming-Viot particle systems that we have recently obtained.
The first edition of this book was published in 1979 in Russian. Most of the material presented was related to large-deviation theory for stochastic pro cesses. This theory was developed more or less at the same time by different authors in different countries. This book was the first monograph in which large-deviation theory for stochastic processes was presented. Since then a number of books specially dedicated to large-deviation theory have been pub lished, including S. R. S. Varadhan [4], A. D. Wentzell [9], J. -D. Deuschel and D. W. Stroock [1], A. Dembo and O. Zeitouni [1]. Just a few changes were made for this edition in the part where large deviations are treated. The most essential is the addition of two new sections in the last chapter. Large deviations for infinite-dimensional systems are briefly conside:red in one new section, and the applications of large-deviation theory to wave front prop agation for reaction-diffusion equations are considered in another one. Large-deviation theory is not the only class of limit theorems arising in the context of random perturbations of dynamical systems. We therefore included in the second edition a number of new results related to the aver aging principle. Random perturbations of classical dynamical systems under certain conditions lead to diffusion processes on graphs. Such problems are considered in the new Chapter 8.
Significance
We propose an algorithm to sample rare events in climate models with a computational cost from 100 to 1,000 times less than direct sampling of the model. Applied to the study of extreme heat waves, we estimate the probability of events that cannot be studied otherwise because they are too rare, and we get a huge ensemble of realizations of an extreme event. Using these results, we describe the teleconnection pattern for the extreme European heat waves. This method should change the paradigm for the study of extreme events in climate models: It will allow us to study extremes with higher-complexity models, to make intermodel comparison easier, and to study the dynamics of extreme events with unprecedented statistics.
The distribution of a Markov process with killing, conditioned to be still alive at a given time, can be approximated by a Fleming-Viot type particle system. In such a system, each particle is simulated independently according to the law of the underlying Markov process, and branches onto another particle at each killing time. The consistency of this method in the large population limit was the subject of several recent articles. In the present paper, we go one step forward and prove a central limit theorem for the law of the Fleming-Viot particle system at a given time under two conditions: a "soft killing" assumption and a boundedness condition involving the "carr\'e du champ" operator of the underlying Markov process.
Adaptive Multilevel Summation (AMS) is a rare event sampling method that requires minimal parameter tuning and that allows unbiased sampling of transition pathways of a given rare event. Previous simulation studies have verified the efficiency and accuracy of AMS in the calculation of transition times for simple systems in both Monte Carlo and molecular dynamics (MD) simulations. Now, AMS is applied for the first time to a MD simulation of protein-ligand dissociation, representing a leap in complexity from the previous test cases. Of interest is the dissociation rate, which is typically too low to be accessible to conventional MD. The present study joins other recent efforts to develop advanced sampling techniques in MD to calculate dissociation rates, which are gaining importance in the pharmaceutical field as indicators of drug efficacy. The system investigated here, benzamidine bound to trypsin, is an example common to many of these efforts. The AMS estimate of the dissociation rate was found to be (2.6 ± 2.4) × 10(2) s(-1) , which compares well with the experimental value.
The goal of importance sampling is to estimate the expected value of a given
function with respect to a probability measure using a random sample of
size n drawn from a different probability measure . If the two measures
and are nearly singular with respect to each other, which is often
the case in practice, the sample size required for accurate estimation is
large. In this article it is shown that in a fairly general setting, a sample
of size approximately is necessary and sufficient for
accurate estimation by importance sampling, where is the
Kullback--Leibler divergence of from . In particular, the required
sample size exhibits a kind of cut-off in the logarithmic scale. The theory is
applied to obtain a fairly general formula for the sample size required in
importance sampling for exponential families (Gibbs measures). We also show
that the standard variance-based diagnostic for convergence of importance
sampling is fundamentally problematic. An alternative diagnostic that provably
works in certain situations is suggested.
Adaptive multilevel splitting algorithms have been introduced rather recently
for estimating tail distributions in a fast and efficient way. In particular,
they can be used for computing the so-called reactive trajectories
corresponding to direct transitions from one metastable state to another. The
algorithm is based on successive selection-mutation steps performed on the
system in a controlled way. It has two intrinsic parameters, the number of
particles/trajectories and the reaction coordinate used for discriminating good
or bad trajectories. We investigate first the convergence in law of the
algorithm as a function of the timestep for several simple stochastic models.
Second, we consider the average duration of reactive trajectories for which no
theoretical predictions exist. The most important aspect of this work concerns
some systems with two degrees of freedom. They are studied in details as a
function of the reaction coordinate in the asymptotic regime where the number
of trajectories goes to infinity. We show that during phase transitions, the
statistics of the algorithm deviate significatively from known theoretical
results when using non-optimal reaction coordinates. In this case, the variance
of the algorithm is peaking at the transition and the convergence of the
algorithm can be much slower than the usual expected central limit behavior.
The duration of trajectories is affected as well. Moreover, reactive
trajectories do not correspond to the most probable ones. Such behavior
disappears when using the optimal reaction coordinate called committor as
predicted by the theory. We finally investigate a three-state Markov chain
which reproduces this phenomenon and show logarithmic convergence of the
trajectory durations.
By making use of martingale representations, we derive the asymptotic
normality of particle filters in hidden Markov models and a relatively simple
formula for their asymptotic variances. Although repeated resamplings result in
complicated dependence among the sample paths, the asymptotic variance formula
and martingale representations lead to consistent estimates of the standard
errors of the particle filter estimates of the hidden states.
We present a large deviations analysis for the performance of an interacting particle method for rare event estimation. The analysis is restricted to a one-dimensional setting, though even in this restricted setting a number of new techniques must be developed. In contrast to the large deviations analyses of related algorithms, for interacting particle schemes it is an occupation measure analysis that is relevant, and within this framework many standard assumptions (stationarity, Feller property) can no longer be assumed. The methods developed are not limited to the question of performance analysis, and in fact give the full large deviations principle for such systems.
A heuristic that has emerged in the area of importance sampling is that the changes of measure used to prove large deviation lower bounds give good performance when used for importance sampling. Recent work, however, has suggested that the heuristic is incorrect in many situations. The perspective put forth in the present paper is that large deviation theory suggests many changes of measure, and that not all are suitable for importance sampling. In the setting of Cramer's Theorem, the traditional interpretation of the heuristic suggests a fixed change of distribution on the underlying independent and identically distributed summands. In contrast, we consider importance sampling schemes where the exponential change of measure is adaptive, in the sense that it depends on the historical empirical mean. The existence of asymptotically optimal schemes within this class is demonstrated. The result indicates that an adaptive change of measure, rather than a static change of measure, is what the large deviations analysis truly suggests. The proofs utilize a control-theoretic approach to large deviations, which naturally leads to the construction of asymptotically optimal adaptive schemes in terms of a limit Bellman equation. Numerical examples contrasting the adaptive and standard schemes are presented, as well as an interpretation of their different performances in terms of differential games.
Particle splitting methods are considered for the estimation of rare events. The probability of interest is that a Markov process first enters a set B before another set A, and it is assumed that this probability satisfies a large deviation scaling. A notion of subsolution is defined for the related calculus of variations problem, and two main results are proved under mild conditions. The first is that the number of particles generated by the algorithm grows subexponentially if and only if a certain scalar multiple of the importance function is a subsolution. The second is that, under the same condition, the variance of the algorithm is characterized (asymptotically) in terms of the subsolution. The design of asymptotically optimal schemes is discussed, and numerical examples are presented.
It was established in (6, 7) that importance sampling algorithms for estimating rare-event probabilities are intimately connected with two-person zero-sum dierential games and the associated Isaacs equa- tion. This game interpretation shows that dynamic or state-dependent schemes are needed in order to attain asymptotic optimality in a gen- eral setting. The purpose of the present paper is to show that classical subsolutions of the Isaacs equation can be used as a basic and flexible tool for the construction and analysis of ecient dynamic importance sampling schemes. There are two main contributions. The first is a basic theoretical result characterizing the asymptotic performance of importance sampling estimators based on subsolutions. The second is an explicit method for constructing classical subsolutions as a mollifi- cation of piecewise ane functions. Numerical examples are included for illustration and to demonstrate that simple, nearly asymptotically optimal importance sampling schemes can be obtained for a variety of problems via the subsolution approach.
Rare event simulation in the context of queueing networks has been an active area of research for more than two decades. A commonly used technique to increase the eciency of Monte Carlo simulation is impor- tance sampling. However, there are few rigorous results on the design of ecient or asymptotically optimal importance sampling schemes for queueing networks. Using a recently developed game/subsolution ap- proach, we construct simple and ecient state-dependent importance sampling schemes for simulating buer overflows in stable open Jackson networks. The sampling distributions do not depend on the particular event of interest, and hence overflow probabilities for dierent events can be estimated simultaneously. A by-product of the analysis is the identification of the minimizing trajectory for the calculus of variation problem that is associated with the sample-path large deviation rate function.
We propose a methodology to sample sequentially from a sequence of probability distributions that are defined on a common space, each distribution being known up to a normalizing constant. These probability distributions are approximated by a cloud of weighted random samples which are propagated over time by using sequential Monte Carlo methods. This methodology allows us to derive simple algorithms to make parallel Markov chain Monte Carlo algorithms interact to perform global optimization and sequential Bayesian estimation and to compute ratios of normalizing constants. We illustrate these algorithms for various integration tasks arising in the context of Bayesian inference. Copyright 2006 Royal Statistical Society.
Analysis and approximation of rare events
- A Budhiraja
- P Dupuis
A. Budhiraja and P. Dupuis. Analysis and approximation of rare events,
volume 94 of Probability Theory and Stochastic Modelling. Springer, New
York, 2019. Representations and weak convergence methods.
Branching and interacting particle systems approximations of Feynman-Kac formulae with applications to non-linear filtering
- Del Moral
- L Miclo
P. Del Moral and L. Miclo. Branching and interacting particle systems
approximations of Feynman-Kac formulae with applications to non-linear
filtering. In Jacques Azéma, Michel Ledoux, MichelÉmery, and Marc Yor,
editors, Séminaire de Probabilités XXXIV, volume 1729 of Lecture Notes
in Mathematics, pages 1-145. Springer Berlin Heidelberg, 2000.
Random Perturbations of Dynamical Systems. Grundlehren der mathematischen Wissenschaften
- M I Freidlin
- J Szücs
- A D Wentzell
M.I. Freidlin, J. Szücs, and A.D. Wentzell. Random Perturbations of
Dynamical Systems. Grundlehren der mathematischen Wissenschaften.
Springer Berlin Heidelberg, 2012.
Asymptotic robustness of estimators in rare-event simulation
- P Ecuyer
- J Blanchet
- B Tuffin
- P W Glynn
P. L'Ecuyer, J. Blanchet, B. Tuffin, and P.W. Glynn. Asymptotic robustness of estimators in rare-event simulation. ACM Transactions on Modeling
and Computer Simulation, 18(3):1269-1283, 2008.